Table 1 - uploaded by Alessio Bellino
Content may be subject to copyright.
Source publication
We present SEQUENCE, a novel interaction technique for selecting objects from a distance. Objects display different rhythmic patterns by means of animated dots, and users can select one of them by matching the pattern through a sequence of taps on a smartphone. The technique works by exploiting the temporal coincidences between patterns displayed b...
Contexts in source publication
Context 1
... general, computing the hamming distance between rhythm combinations is a smart way to determine their robustness against false activations. For example, in rhythms composed of four active events and four pauses, we can have 14 combinations that have a hamming distance of 4 from each other ( Table 1). It means that users should make at least four rhythmic errors for erroneously activating one of the combinations in Table 1. ...
Context 2
... example, in rhythms composed of four active events and four pauses, we can have 14 combinations that have a hamming distance of 4 from each other ( Table 1). It means that users should make at least four rhythmic errors for erroneously activating one of the combinations in Table 1. Making some trials with two users (contextualized later in section 3.3.2), ...
Context 3
... some trials with two users (contextualized later in section 3.3.2), we observed several difficulties to activate some of the combinations in Table 1. Many combinations were easy to activate (e.g., combinations 1, 2, 3, 4, 6, 11, 12, 13, and 14 in Table 1), but synchronizing with others could be tricky (e.g., combinations 5, 7, 8, 9, and 10 in Table 1) because of their irregularity. ...
Context 4
... observed several difficulties to activate some of the combinations in Table 1. Many combinations were easy to activate (e.g., combinations 1, 2, 3, 4, 6, 11, 12, 13, and 14 in Table 1), but synchronizing with others could be tricky (e.g., combinations 5, 7, 8, 9, and 10 in Table 1) because of their irregularity. In effect, a study conducted by Grahn and Brett [22] confirms the difficulties of users to synchronize with rhythms that have complex metrics, i.e., rhythms in which accents occur at irregular intervals. ...
Context 5
... observed several difficulties to activate some of the combinations in Table 1. Many combinations were easy to activate (e.g., combinations 1, 2, 3, 4, 6, 11, 12, 13, and 14 in Table 1), but synchronizing with others could be tricky (e.g., combinations 5, 7, 8, 9, and 10 in Table 1) because of their irregularity. In effect, a study conducted by Grahn and Brett [22] confirms the difficulties of users to synchronize with rhythms that have complex metrics, i.e., rhythms in which accents occur at irregular intervals. ...
Context 6
... some extent, this seemed to facilitate the matching of elements since users are free to start synchronizing when they prefer, i.e., when they "enter the rhythmic loop", perceiving in mind the regularity of the sequence with which they are trying to synchronize. After some trials with two users 4 , in fact, we noticed that in the condition in which translated rhythms were tested, users made generally fewer errors than in the condition in which different simple rhythms were tested, i.e., the one displayed in Table 1 combinations 1, 2, 3, 4, 6, 11, 12, 13, and 14. The differences were minimal but constant considering different repeated trials over time with the same users. ...
Context 7
... elements to be displayed are more than eight, designers could use the combinations in Table 1, but errors would increase since some of them are difficult to reproduce. Therefore, we suggest changing the design approach by creating an interface where an element open other sub elements that can be triggered mostly like submenu. ...
Citations
... This is the author's version of the article that has been published in the proceedings of 23rd IEEE International Symposium on Mixed and Augmented Reality (ISMAR). The final version of this record is available at: 10.1109/IS-MAR62088.2024.00046 of display-guided interactions, facilitated by tapping [10] or touch gestures on the screen [19]. When it comes to hand-based motion matching interaction, PathSync [29] and TraceMatch [13,37] have shown potential using a computer vision-based tracking system, while WaveTrace [40] demonstrated applicability using a smartwatch. ...
... Synchrowatch [33] permits users to control smartwatches by matching rhythm patterns on the screen using a passive magnetic ring as a rhythm detection device. Finally, SEQUENCE [34] employs a novel design that displays rhythmic patterns through eight animated dots arranged circularly around the target, making target selection easier since the associated rhythm patterns are fully displayed to the user [35]. ...
... In this context, the underlying principle is that once users acquire proficiency in interacting with visual orbits, they can seamlessly apply the same mechanism to sound orbits. Finally, it is noteworthy that motion correlation techniques, including SoundOrbit, are often viewed as complementary rather than replacement methods in smart home scenarios [5,34]. These techniques aim to enhance existing interaction styles, like remote control, particularly for routine tasks, emphasizing immediate and convenient user interaction without disrupting established habits [42]. ...
SoundOrbit is a novel input technique that uses motion correlation to control smart devices. The technique associates controls with specific orbital sounds, made of cyclically increasing/decreasing musical scales, and the user can activate a control by mimicking the corresponding sound by body motion. Unlike previous movement-correlation techniques based on visual displays, SoundOrbit operates independent of visual perception, enabling the development of cost-effective smart devices that do not require visual displays. We investigated SoundOrbit by conducting two user studies. The first study evaluated the effectiveness of binaural sound spatialization to create a distinct orbiting sound. In comparison to a cyclic musical scale that is fixed in the apparent auditory space, we found that spatial effects did not improve users’ ability to follow the sound orbit. In the second study, we aimed at determining the optimal system parameters, and discovered that users synchronize better with slower speeds. The technique was found to be feasible and reliable for one and two orbits simultaneously, each orbit using a distinct sound timbre, but not for three orbits due to a high error rate.
... c) Eye Aspect Ratio (EAR) (Soukupová and Cech, 2016). d) Mouth Aspect Ratio (MAR) (Bellino, 2018). e) Examples of interactions between point 33 and other face points. ...
Aims:
Our study aimed to develop a machine learning ensemble to distinguish "at-risk mental states for psychosis" (ARMS) subjects from control individuals from the general population based on facial data extracted from video-recordings.
Methods:
58 non-help-seeking medication-naïve ARMS and 70 healthy subjects were screened from a general population sample. At-risk status was assessed with the Structured Interview for Prodromal Syndromes (SIPS), and "Subject's Overview" section was filmed (5-10 min). Several features were extracted, e.g., eye and mouth aspect ratio, Euler angles, coordinates from 51 facial landmarks. This elicited 649 facial features, which were further selected using Gradient Boosting Machines (AdaBoost combined with Random Forests). Data was split in 70/30 for training, and Monte Carlo cross validation was used.
Results:
Final model reached 83 % of mean F1-score, and balanced accuracy of 85 %. Mean area under the curve for the receiver operator curve classifier was 93 %. Convergent validity testing showed that two features included in the model were significantly correlated with Avolition (SIPS N2 item) and expression of emotion (SIPS N3 item).
Conclusion:
Our model capitalized on short video-recordings from individuals recruited from the general population, effectively distinguishing between ARMS and controls. Results are encouraging for large-screening purposes in low-resource settings.
... c) Eye Aspect Ratio (EAR)[55]. d) Mouth Aspect Ratio (MAR)[56]. e) Examples of interactions between point 33 and other face points. f) Matching pairs used to calculate Spearman's correlation coefficient.This preprint research paper has not been peer reviewed. ...
To prevent the development of schizophrenia, preclinical stages of the disorder, known as "at-risk mental states for psychosis" (ARMS), have been intensively researched for the past three decades. Despite the many advances in the field, identification of ARMS is still resource-consuming and presents important issues regarding accuracy. To address this, our study aimed to develop a machine learning ensemble to distinguish ARMS from control individuals based on facial expression extracted from brief video-recordings.
... In this context, different interaction techniques were studied. Some of these techniques employ rhythmic synchronization [1][2][3][4]: they work by displaying multiple animated controls that show different rhythms in visual form, and the user can select one of them by synchronizing with the corresponding rhythm. Controls can be physical (e.g., in Figure 1) or virtual (i.e., shown on a screen, such as those used in this study, see Figure 2). ...
... From the point of view of the input device, rhythmic synchronization techniques can be simpler compared to those based on movement correlation. While motion-correlation techniques require sensors that detect movement (e.g., cameras [6,7] or kinect [9]), rhythmic synchronization techniques require sensors as simple as a button, e.g., [1,4]. As matter of fact, previous studies showed that these techniques can support a wide variety of sensors [1,3]; in particular, any sensor capable of generating a binary input through which users can perform the required rhythm. ...
... While motion-correlation techniques require sensors that detect movement (e.g., cameras [6,7] or kinect [9]), rhythmic synchronization techniques require sensors as simple as a button, e.g., [1,4]. As matter of fact, previous studies showed that these techniques can support a wide variety of sensors [1,3]; in particular, any sensor capable of generating a binary input through which users can perform the required rhythm. As a result, these techniques are quite flexible. ...
Rhythmic-synchronization-based interaction is an emerging interaction technique where multiple controls with different rhythms are displayed in visual form, and the user can select one of them by matching the corresponding rhythm. These techniques can be used to control smart objects in environments where there may be interfering auditory stimuli that contrast with the visual rhythm (e.g., to control Smart TVs while playing music), and this could compromise users’ ability to synchronize. Moreover, these techniques require certain reflex skills to properly synchronize with the displayed rhythm, and these skills may vary depending on the age and gender of the users. To determine the impact of interfering auditory stimuli, age, and gender on users’ ability to synchronize, we conducted a user study with 103 participants. Our results show that there are no significant differences between the conditions of interfering and noninterfering auditory stimuli and that synchronization ability decreases with age, with males performing better than females—at least as far as younger users are concerned. As a result, two implications emerge: first, users are capable of focusing only on visual rhythm ignoring the auditory interfering rhythm, so listening to an interfering rhythm should not be a major concern for synchronization; second, as age and gender have an impact, these systems may be designed to allow for customization of rhythm speed so that different users can choose the speed that best suits their reflex skills.
... However, as one of the most basic interaction tasks, target selection can be challenging on these new interfaces. There are three reasons: 1) cross-device interaction for a large number of devices calls for association-free target selection techniques [6,10,23]. It is not practical to have a designated controller for each individual device or require users to associate with the devices each time before usage, especially when there is a large number of devices; 2) the interaction expressivity (e.g., audio, gesture) and form factor (e.g. ...
... BitID [49] only senses binary inputs), so traditional target selection techniques are not applicable on such interfaces. Aimed at these challenges, temporal synchronous target selection [6,27,34,50] has been proposed by researchers to enable association-free target selection on devices with different interaction interfaces. Instead of browsing and selecting the target device from a list on a screen, users can generate temporal synchronized signals with a temporal pattern (e.g., blinking) to select the corresponding target. ...
... This kind of technique has three advantages: 1) It does not require device association as long as the pattern for each target is unique, which can save the total interaction time; 2) Temporal signal can be generated on multi-modality interfaces, which enables subtle and accessible selection experience, so that users can choose the appropriate interface when in different scenarios. For example, users can tap fingers (touchscreen [6]), clap hands (audio [22]), tap foot (vibration [52]), contract muscles (EMG [3,35,45]), blink eye (EOG [5]), and even breath [16,17] to sync with the target pattern. As the interaction paradigm-generating binary changing signals in sync with the target pattern-remains the same, users would be able to transfer the interaction experience across different interfaces; 3) The selection technique's extremely low requirement of sensing resources makes it compatible with a wide variety of both new and existing sensors. ...
Temporal synchronous target selection is an association-free selection technique: users select a target by generating signals (e.g., finger taps and hand claps) in sync with its unique temporal pattern. However, classical pattern set design and input recognition algorithm of such techniques did not leverage users' behavioral information, which limits their robustness to imprecise inputs. In this paper, we improve these two key components by modeling users' interaction behavior. In the first user study, we asked users to tap a finger in sync with blinking patterns with various period and delay, and modeled their finger tapping ability using Gaussian distribution. Based on the results, we generated pattern sets for up to 22 targets that minimized the possibility of confusion due to imprecise inputs. In the second user study, we validated that the optimized pattern sets could reduce error rate from 23% to 7% for the classical Correlation recognizer. We also tested a novel Bayesian, which achieved higher selection accuracy than the Correlation recognizer when the input sequence is short. The informal evaluation results show that the selection technique can be effectively scaled to different modalities and sensing techniques.
We introduce a novel one-handed input technique for mobile devices that is not based on pointing, but on motion matching -where users select a target by mimicking its unique animation. Our work is motivated by the findings of a survey (N=201) on current mobile use, from which we identify lingering opportunities for one-handed input techniques. We follow by expanding on current motion matching implementations - previously developed in the context of gaze or mid-air input - so these take advantage of the affordances of touch-input devices. We validate the technique by characterizing user performance via a standard selection task (N=24) where we report success rates (>95%), selection times (~1.6 s), input footprint, grip stability, usability, and subjective workload - in both phone and tablet conditions. Finally, we present a design space that illustrates six ways in which motion matching can be embedded into mobile interfaces via a camera prototype application.
