Fig 10 - uploaded by Tiffany Li
Content may be subject to copyright.
Distribution of QV responses per video element in QV in Boxen plots. The maximum possible number of votes on an element was 10 votes given 100 voice credits. Most distributions in all three QV set-ups follow a normal distribution.
Source publication
Surveys are a common instrument to gauge self-reported opinions from the crowd for scholars in the CSCW community, the social sciences, and many other research areas. Researchers often use surveys to prioritize a
subset of given options when there are resource constraints. Over the past century, researchers have developed a wide range of surveying...
Context in source publication
Context 1
... shown in Figure 9 and Figure 10, on an aggregated level, participants expressed similar relative preferences across the five video elements in both Likert and QV. Audio quality ranked the highest in both cases, while motion smoothness and audio-video synchronization had the lowest ranks. ...
Citations
... Human-computer interaction research on recommender systems has shown that user preferences can be inferred from ranking and ordering interactions, although inconsistencies in user rankings and ratings can challenge accurate preference understanding [1,20]. Quadratic voting has been introduced as a method to more accurately capture individual preferences in surveys, showing better alignment with genuine preferences than traditional Likert scales [10]. ...
In the evolution of software systems, especially in domains like autonomous vehicles, dynamic user preferences are critical yet challenging to accommodate. Existing methods often misrepresent these preferences, either by overlooking their dynamism or overburdening users as humans often find it challenging to express their objectives mathematically. Addressing this, we introduce a novel framework interpreting dynamic preferences as inherent uncertainty, anchored on a “human-on-the-loop” mechanism enabling users to give feedback when dissatisfied with system behaviors. Leveraging a designed fitness function, our system employs a genetic algorithm to adapt preference values, aligning preferences with user expectations through feedback-driven adaptation. We validated its effectiveness with an autonomous driving prototype and a user study involving 20 participants. The findings highlight our framework’s capability to effectively merge algorithm-driven adjustments with user complaints, leading to improved participants’ subjective satisfaction in autonomous systems.
... Recent Human-Computer Interaction (HCI) studies highlight how certain survey response formats can increase errors [43,66] and influence survey effectiveness [93]. In this paper, our goal is to introduce an effective interface for a Quadratic Survey (QS), a survey tool designed to elicit preferences more accurately than traditional methods [7]. Despite the promise of QSs, there has been no research on designing interfaces to support their unique quadratic mechanisms [31], where participants must rank and rate items -a task that poses significant cognitive challenges. ...
... We envision an effective interface that navigates participants through the complex mechanism and preference construction process, tailored to a QS. A QS improves accuracy in individual preference elicitation compared to traditional methods like Likert scales by requiring participants to make trade-offs using a fixed budget of credits, where purchasing votes for an option in QS costs 2 credits [7,68]. This quadratic cost structure forces respondents to carefully evaluate their preferences, balancing the strength of their support or opposition against the limited budget. ...
... Unlike traditional surveys that elicit either rankings or ratings, QS allows for both, enabling participants to cast multiple votes for or against options, incurring a quadratic cost. Cheng et al. [7] showed that this mechanism aligns individual preferences with behaviors more accurately than Likert Scale surveys, particularly in resource-constrained scenarios like prioritizing user feedback on user experiences. ...
Quadratic Surveys (QSs) elicit more accurate preferences than traditional methods like Likert-scale surveys. However, the cognitive load associated with QSs has hindered their adoption in digital surveys for collective decision-making. We introduce a two-phase "organize-then-vote'' QS to reduce cognitive load. As interface design significantly impacts survey results and accuracy, our design scaffolds survey takers' decision-making while managing the cognitive load imposed by QS. In a 2x2 between-subject in-lab study on public resource allotment, we compared our interface with a traditional text interface across a QS with 6 (short) and 24 (long) options. Two-phase interface participants spent more time per option and exhibited shorter voting edit distances. We qualitatively observed shifts in cognitive effort from mechanical operations to constructing more comprehensive preferences. We conclude that this interface promoted deeper engagement, potentially reducing satisficing behaviors caused by cognitive overload in longer QSs. This research clarifies how human-centered design improves preference elicitation tools for collective decision-making.
... under the circumstance that all other confounding variables (based on the DAG) in the model are controlled. We choose Bayesian analysis as the main tool for our quantitative study due to the following reasons [2,18]: ...
Online teaching has expanded access to education, offering flexibility compared to traditional face-to-face instruction. While early research has explored online teaching, it is important to understand the perspective of instructors who conducted their first online classes during the Covid-19 pandemic. This study focuses on instructors teaching online for the first time, regardless of whether they volunteered. Surveys were conducted when universities transitioned from in-person to online instruction in April 2020, with a follow-up survey after their first online teaching semester. The study investigated instructors' expectations of class success before their first online teaching experience. Using Bayesian modeling, we analyzed how these expectations varied based on instructors' characteristics (self-efficacy in online teaching, technological proficiency, and acceptance of technology) and course attributes (subject area, class size, and instructional design). Results showed that instructors' self-efficacy significantly impacted their expectations of success, while smaller class sizes were associated with lower expectations. Interestingly, factors like prior use of technology platforms and classroom design did not contribute significantly to expectations. The study offers practical recommendations to support online teaching. To improve self-efficacy, instructors should collaborate with colleagues and familiarize themselves with online platforms. Universities should provide workshops or training to enhance teaching skills. In small interactive classes, nonverbal communication should be emphasized, and institutions should establish support teams and feedback mechanisms to ensure quality and effectiveness in online education.
... Indeed, this may discourage users with small number of votes from participating in governance, perhaps inducing them to drop out of the platform. For this reason, a voting method, which in recent years has gained some attention for social decisions in general, hence also as a possible solution to the above problem in blockchain platforms, is quadratic voting (QV) [1][2][3][4][5][6][7][8][9] Its interest is additionally testified by an analogous quadratic criterion, which has recently been proposed for project co-funding [10]. In its most common application, represented by a voting session with a list of binary items to vote of the type A/B, QV allows participants to express both the direction as well as the intensity of one's preferences as it takes place, for example, with oral acclamation [1][2][3][4][5]. ...
... For this reason, a voting method, which in recent years has gained some attention for social decisions in general, hence also as a possible solution to the above problem in blockchain platforms, is quadratic voting (QV) [1][2][3][4][5][6][7][8][9] Its interest is additionally testified by an analogous quadratic criterion, which has recently been proposed for project co-funding [10]. In its most common application, represented by a voting session with a list of binary items to vote of the type A/B, QV allows participants to express both the direction as well as the intensity of one's preferences as it takes place, for example, with oral acclamation [1][2][3][4][5]. For this reason, unlike the standard 51% majority voting, QV sets up a framework where minority voters, that is those subjects with a limited number of votes, could still have chances to obtain the desirable outcomes for those issues which they care particularly about. ...
Governance in blockchain platforms is an increasingly important topic. A particular concern related to voting procedures is the formation of dominant positions, which may discourage participation of minorities. A main feature of standard majority voting is that individuals can indicate their preferences but cannot express the intensity of their preferences. This could sometimes be a drawback for minorities who may not have the opportunity to obtain their most desirable outcomes, even when such outcomes are particularly important for them. For this reason a voting method, which in recent years gained visibility, is quadratic voting (QV), which allows voters to manifest both their preferences and the associated intensity. In voting rounds, where in each round users express their preference over binary alternatives, what characterizes QV is that the sum of the squares of the votes allocated by individuals to each round has to be equal to the total number, budget, of available votes. That is, the cost associated with a number of votes is given by the square of that number, hence it increases quadratically. In the paper, we discuss QV in proof-of-stake-based blockchain platforms, where a user’s monetary stake also represents the budget of votes available in a voting session. Considering the stake as given, the work focuses mostly on a game theoretic approach to determine the optimal allocation of votes across the rounds. We also investigate the possibility of the so-called Sybil attacks and discuss how simultaneous versus sequential staking can affect the voting outcomes with QV.
Conversational Agents (CAs) can facilitate information elicitation in various scenarios, such as semi-structured interviews. Current CAs can ask predetermined questions but lack skills for asking follow-up questions. Thus, we designed three approaches for CAs to automatically ask follow-up questions, i.e., follow-ups on concepts, follow-ups on related concepts, and general follow-ups. To investigate their effects, we conducted a user study (N=26) in which a CA interviewer asked follow-up questions generated by algorithms and crafted by human wizards. Our results showed that the CA's follow-up questions were readable and effective in information elicitation. The follow-ups on concepts and related concepts achieved a lower drop rate and better relevance, while the general follow-ups elicited more informative responses. Further qualitative analysis of the human-CA interview data revealed algorithm drawbacks and identified follow-up question techniques used by the human wizards. We provided design implications for improving information elicitation of future CAs based on the results.
