Bing-Yu Chen

National Taiwan University | NTU

Direct part-level manipulation of man-made shapes in an image is desired given its simplicity. However, it is not intuitive given the existing manually created cuboid and cylinder controllers. To tackle this problem, we present StylePart, a framework that enables direct shape manipulation of an image by leveraging generative models of both images a...

Neural Radiance Field (NeRF) has emerged as a versatile scene representation. However, it is still unintuitive to edit a pretrained NeRF because the network parameters and the scene appearance are often not explicitly associated. In this paper, we introduce the first framework that enables users to retouch undesired regions in a pretrained NeRF sc...

Colorizing icon is a challenging task, even for skillful artists, as it involves balancing aesthetics and practical considerations. Prior works have primarily focused on colorizing pixel‐based icons, which do not seamlessly integrate into the current vector‐based icon design workflow. In this paper, we propose a palette‐based colorization algorithm...

Interface icons are prevalent in various digital applications. Due to limited time and budgets, many designers rely on informal evaluation, which often results in poor usability icons. In this paper, we propose a unique human‐in‐the‐loop framework that allows our target users, that is novice and professional user interface (UI) designers, to improv...

Interface icons are prevalent in various digital applications. Due to limited time and budgets, many designers rely on informal evaluation, which often results in poor usability icons. In this paper, we propose a unique human-in-the-loop framework that allows our target users, i.e., novice and professional UI designers, to improve the usability of...

In this paper, we propose NFCStack, which is a physical building block system that supports stacking and frictionless interaction and is based on near-field communication (NFC). This system consists of a portable station that can support and resolve the order of three types of passive identifiable stackable: bricks, boxes, and adapters. The bricks...

Current haptic devices are usually designed to provide one type of force feedback; however, most VR scenarios require versatile force feedback, which may require the integration of different devices to provide various types of forces. In addition, besides the main haptic effects caused by the forces, multiple types of oscillation may also commonly...

NFC (Near-Field Communication) has been widely applied for human-computer interaction (HCI). However, the short sensing distance of NFC requires the users to initiate the tasks with extra effort mostly using their hands, so it is inconvenient to use NFC in hands-busy scenarios. This paper presents an investigation of body-centric interactions betwe...

Though Neural Radiance Field (NeRF) demonstrates compelling novel view synthesis results, it is still unintuitive to edit a pre-trained NeRF because the neural network's parameters and the scene geometry/appearance are often not explicitly associated. In this paper, we introduce the first framework that enables users to remove unwanted objects or r...

Many haptic feedback methods have been proposed to enhance realism in virtual reality (VR). However, friction on the feet in VR, which renders feedback as if walking on different terrains or ground textures or stepping on objects is still less explored. Herein, we propose a wearable device, FrictShoes a pair of foot accessories, to provide multilev...

Due to a lack of image-based "part controllers", shape manipulation of man-made shape images, such as resizing the backrest of a chair or replacing a cup handle is not intuitive because of the lack of image-based part controllers. To tackle this problem, we present StylePart, a framework that enables direct shape manipulation of an image by leverag...

This paper presents a novel deep learning-based approach for automatically vectorizing and synthesizing the clipart of man-made objects. Given a raster clipart image and its corresponding object category (e.g., airplanes), the proposed method sequentially generates new layers, each of which is composed of a new closed path filled with a single colo...

This article presents a novel deep learning-based approach for automatically vectorizing and synthesizing the clipart of man-made objects. Given a raster clipart image and its corresponding object category (e.g., airplanes), the proposed method sequentially generates new layers, each of which is composed of a new closed path filled with a single co...

We present an assistive system for clipart design by providing visual scaffolds from the unseen viewpoints. Inspired by the artists' creation process, our system constructs the visual scaffold by first synthesizing the reference 3D shape of the input clipart and rendering it from the desired viewpoint. The critical challenge of constructing this vi...

The human ear is highly sensitive and accessible, making it especially suitable for being used as an interface for interacting with smart earpieces or augmented glasses. However, previous works on ear-based input mainly address gesture sensing technology and researcher-designed gestures. This paper aims to bring more understandings of gesture desig...

We present an assistive system for clipart design by providing visual scaffolds from the unseen viewpoints. Inspired by the artists' creation process, our system constructs the visual scaffold by first synthesizing the reference 3D shape of the input clipart and rendering it from the desired viewpoint. The critical challenge of constructing this vi...

This symposium showcases the latest HCI work from Asia and those focusing on incorporating Asian sociocultural factors in their design and implementation. In addition to circulating ideas and envisioning future research in human-computer interaction, this symposium aims to foster social networks among academics (researchers and students) and practi...

This paper presents RFTouchPads, a system of batteryless and wireless modular hardware designs of two-dimensional (2D) touch sensor pads based on the ultra-high frequency (UHF) radio-frequency identification (RFID) technology. In this system, multiple RFID IC chips are connected to an antenna in parallel. Each chip connects only one of its endpoint...

We present TilePoP, a new type of pneumatically-actuated interface deployed as floor tiles which dynamically pop up by inflating into large shapes constructing proxy objects for whole-body interactions in Virtual Reality. TilePoP consists of a 2D array of stacked cube-shaped airbags designed with specific folding structures, enabling each airbag to...

We propose integrating an array of skin stretch modules with an head-mounted display (HMD) to provide two-dimensional skin stretch feedback on the user's face. Skin stretch has been found effective to induce the perception of force (e.g. weight or inertia) and to enable directional haptic cues. However, its potential as an HMD output for virtual re...

Impact is a common effect in both daily life and virtual reality (VR) experiences, e.g., being punched, hit or bumped. Impact force is instantly produced, which is distinct from other force feedback, e.g., push and pull. We propose ElastImpact to provide 2.5D instant impact on a head-mounted display (HMD) for realistic and versatile VR experiences....

Rendering haptic feedback in virtual reality is a common approach to enhancing the immersion of virtual reality content. However, current editing tools allow developers to access the haptic feedback only through physical contact with the actuators, making it difficult to fast iterate haptic interaction designs. This paper introduces SeeingHaptics,...

This work explores and evaluates the designs of finger-worn radio-frequency identification (RFID) motion tracking for activity recognition on tagged objects. We propose an index-finger-worn device that consists of a short-range (~2cm) RFID reader and a pair of two inertial measurement units (IMUs), which are mounted at the locations where an artifi...

In recent years, personalized fabrication has received considerable attention because of the widespread use of consumer‐level three‐dimensional (3D) printers. However, such 3D printers have drawbacks, such as long production time and limited output size, which hinder large‐scale rapid‐prototyping. In this paper, for the time‐ and cost‐effective fab...

In recent years, personalized fabrication has received considerable attention because of the widespread use of consumer-level three-dimensional (3D) printers. However, such 3D printers have drawbacks, such as long production time and limited output size, which hinder large-scale rapid-prototyping. In this paper, for the time- and cost-effective fab...

This paper proposes a robust localization system that employs deep learning for better scene representation, and enhances the accuracy of 6-DOF camera pose estimation. Inspired by the fact that global scene structure can be revealed by wide field-of-view, we leverage the large overlap of a fisheye camera between adjacent frames, and the powerful hi...

We propose a new type of haptic output for foreground interactions on an interactive chair, where input is carried out explicitly in the foreground of the user's consciousness. This type of force output restricts a user's motion by modulating the resistive force when rotating a seat, tilting the backrest, or rolling the chair. These interactions ar...

Resistive force (e.g., due to object elasticity) and impact (e.g., due to recoil) are common effects in our daily life. However, resistive force continuously changes due to users' movements while impact instantly occurs when an event triggers it. These feedback are still not realistically provided by current VR haptic methods. In this paper, a wear...

We propose autocomplete for the design and development of virtual breadboard circuits using software prototyping tools. With our system, a user inserts a component into the virtual breadboard, and it automatically provides a user with a list of suggested components. These suggestions complete or ex- tend the electronic functionality of the inserted...

This work introduces RFIDesk, an interactive surface that enables both multi-touch and rich-ID stackable tangible interactions. By using ultra-high frequency (UHF) radio-frequency identification (RFID) technology, the RFIDesk can effectively identify the elements of a stack. Furthermore, this system integrates capacitive multi-touch sensing based o...

This paper presents a technique enabling distributed batteryless near-field identification (ID) between two passive radio frequency ID (RFID) tags. Each conventional ultra-high-frequency (UHF) RFID tag is modified by connecting its antenna and chip to a reed switch and then attaching a magnet to one of the reed switch's terminals, thus transforming...

We propose using a single slip tactile pixel on virtual reality controllers to produce sensations of finger sliding and textures. When a user moves the controller on a virtual surface, we add a slip opposite to the movement, creating an illusion of a finger that is sliding on the surface, while varying the slip feedback changes lateral forces on fi...

The sensation of being able to feel the shape of an object when grasping it in Virtual Reality (VR) enhances a sense of presence and the ease of object manipulation. Though most prior works focus on force feedback on fingers, the haptic emulation of grasping a 3D shape requires the sensation of touch using the entire hand. Hence, we present Pop-up...

Using (casual) images to texture 3D models is a common way to create realistic 3D models, which is a very important task in computer graphics. However, if the shape of the casual image does not look like the target model or the target mapping area, the textured model will become strange since the image will be distorted very much. In this paper, we...

In the past few years, deep reinforcement learning has been proven to solve problems which have complex states like video games or board games. The next step of intelligent agents would be able to generalize between tasks, and using prior experience to pick up new skills more quickly. However, most reinforcement learning algorithms for now are ofte...

In recent years, personalized fabrication has attracted many attentions due to the widespread of consumer-level 3D printers. However, consumer 3D printers still suffer from shortcomings such as long production time and limited output size, which are undesirable factors to large-scale rapid-prototyping. We propose a hybrid 3D fabrication method that...

We present RFIBricks, an interactive building block system based on ultrahigh frequency radio-frequency identification (RFID) sensing. The system enables geometry resolution based on a simple yet highly generalizable mechanism: an RFID contact switch, which is made by cutting each RFID tag into two parts, namely antenna and chip. A magnetic connect...

We propose a deep learning based method to directly estimate the human joint positions in 3D space from 2D fisheye images captured in an egocentric manner. The core of our method is a novel network architecture based on Inception-v3 [4], featuring the asymmtric convolutional filter size, the long short-term memory module, and the anthropomorphic we...

We present Dwell+, a method that boosts the effectiveness of typical dwell selection by augmenting the passive dwell duration with active haptic ticks which promptly drives rapid switches of modes forward through the user's skin sensations. In this way, Dwell+ enables multi-level dwell selection using rapid haptic ticks. To select a mode from a but...

360-degree video contains a full field of environmental content. However, browsing these videos, either on screens or through head-mounted displays (HMDs), users consume only a subset of the full field of view per a natural viewing experience. This causes a search problem when a region-of-interest (ROI) in a video is outside of the current field of...

In virtual reality (VR) applications, the contents are usually generated by creating a 360° Video panorama of a real-world scene. Although many capture devices are being released, getting high-resolution panoramas and displaying a virtual world in real-time remains challenging due to its computationally demanding nature. In this paper, we propose a...

Insufficient input modality on touchscreens causes icons, toolbars and mode switching steps required to perform different functions. Although various methods are proposed to increase touchscreen input modality, touch gestures (e.g., swipe), usually used in touch input, are not provided in previous methods (e.g., Force Touch on iPhone 6s). This stil...

Automatic photo cropping is an important tool for improving visual quality of digital photos without resorting to tedious manual selection. Traditionally, photo cropping is accomplished by determining the best proposal window through visual quality assessment or saliency detection. In essence, the performance of an image cropper highly depends on t...

Accurate stylus tracking has been integrated in portable displays, which have numerous users currently, to realize drawing and writing applications on these displays. Nonetheless, for learning how to use such applications, users heavily rely on on-screen visual guidance, which can be easily occluded by a user's hand resting on the screen while perf...

Vibrotactile feedback provides immersive media experience. To facilitate the design of vibrotactile effects, an authoring system is essential. Soundtrack streamlined interfaces [Ryu and Choi 2008; Lee et al. 2009] were initially initially introduced to create temporal vibrotactile effects for individual actuators. Unfortunately, temporal effects de...

This paper presents EdgeVib, a system of spatiotemporal vibration patterns for delivering alphanumeric characters on wrist-worn vibrotactile displays. We first investigated spatiotemporal pattern delivery through a watch-back tactile display by performing a series of user studies. The results reveal that employing a 2×2 vibrotactile array is more e...

In this paper, we introduce an interactive method suitable for retargeting both 3D objects and scenes. Initially, the input object or scene is decomposed into a collection of constituent components enclosed by corresponding control bounding volumes which capture the intra-structures of the object or semantic grouping of objects in the 3D scene. The...

This paper investigates the feasibility of using a nail-mounted array of tactors, NailTactors, as an eyes-free output device. By rim-attached eccentric-rotating-mass (ERM) vibrators to artificial nails, miniature high-resolution tactile displays were realized as an eyes-free output device. To understand how to deliver rich signals to users for vali...

In recent years, personalized fabrication has attracted much attention due to the greatly improved accessibility of consumer-level 3D printers. However, 3D printers still suffer from the relatively long production time and limited output size, which are undesirable for large-scale rapid-prototyping. Zometool, which is a popular building block syste...

This work develops a system of spherical magnetic tangibles, GaussMarbles, that exploits the unique affordances of spherical tangibles for interacting with portable physical constraints. The proposed design of each magnetic sphere includes a magnetic polyhedron in the center. The magnetic polyhedron provides bi-polar magnetic fields, which are expa...

Thumb-to-fingers interfaces augment touch widgets on fingers, which are manipulated by the thumb. Such interfaces are ideal for one-handed eyes-free input since touch widgets on the fingers enable easy access by the stylus thumb. This study presents DigitSpace, a thumb-to-fingers interface that addresses two ergonomic factors: hand anatomy and touc...

We present GaussRFID, a hybrid RFID and magnetic-field tag sensing system that supports interactivity when embedded in retrofitted or new physical objects. The system consists of two major components - GaussTag, a magnetic-RFID tag that is combined with a magnetic unit and an RFID tag, and GaussStage, which is a tag reader that is combined with an...

We present GaussRFID, a hybrid RFID and magnetic-field tag sensing system that supports interactivity when embedded in retrofitted or new physical objects. The system consists of two major components - GaussTag, a magnetic-RFID tag that is combined with a magnetic unit and an RFID tag, and GaussStage, which is a tag reader that is combined with an...

Online bookstores have highly thrived and changed consumer behaviors in these years. However, most customers go to online bookstores only when they have specific targets. One reason is that the current web interfaces are usually too complex and cluttered for users to browse. In addition, current visualization interfaces only display the results ass...

The analog Hall-sensor grid, GaussSense, is a thin-form magnetic-field camera technology for designing expressive occlusion-free, near-surface tangible interactions on conventional portable displays. The studio will provide hands-on experiences that combine physical designs and the GaussSense technology. Through a series of brainstorming and making...