(left)Seven of the cameras used to capture the perfor- mance. (right) The array of 216 video projectors used to display the subject. 

Context in source publication

Context 1

... present a system for capturing and rendering life-size 3D human subjects on an automultiscopic display. Automultiscopic 3D displays allow a large number of viewers to experience 3D content simultaneously without the hassle of special glasses or head gear. Such displays are ideal for human subjects as they allow for natural personal interactions with 3D cues such as eye-gaze and complex hand gestures. In this talk, we will focus on a case-study where our system was used to digitize television host Morgan Spurlock for his documentary show ”Inside Man” on CNN. Automultiscopic displays work by generating many simultaneous views with high- angular density over a wide-field of view. The angular spacing between between views must be small enough that each eye perceives a distinct and different view. As the user moves around the display, the eye smoothly transitions from one view to the next. We generate multiple views using a dense horizontal array of video projectors. As video projectors continue to shrink in size, power consumption, and cost, it is now possible to closely stack hundreds of projectors so that their lenses are almost continuous. However this display presents a new challenge for content acquisition. It would require hundreds of cameras to directly measure every projector ray. We achieve similar quality with a new view interpolation algorithm suitable for dense automultiscopic displays. Our interpolation algorithm builds on Einarsson et al. [2006] who used optical flow to resample a sparse light field. While Einarsson et al. was limited to cyclical motions using a rotating turntable, we use an array of 30 unsynchronized Panasonic X900MK 60p con- sumer cameras spaced over 180 degrees to capture unconstrained motion. We first synchronize our videos within 1/120 of a sec- ond by aligning their corresponding sound waveforms. We com- pute pair-wise spatial flow correspondences between cameras using GPU optical flow. As each camera pair is processed independently, the pipeline can be highly parallelized. As a result, we achieve much shorter processing times than traditional multi-camera stereo reconstructions. Our view interpolation algorithm maps images directly from the original video sequences to all the projectors in real- time, and could easily scale to handle additional cameras or projectors. For the ”Inside Man” documentary we recorded a 54 minute interview with Morgan Spurlock, and processed 7 minutes of 3D video for the final show. Our projector array consists of 216 video projectors mounted in a semi-circle with a 3.4m radius. We have a narrow 0 . 625 ◦ spacing between projectors which provides a large display depth of field with minimal aliasing. We use LED-powered Qumi v3 projectors in a portrait orientation (Fig. 2). At this distance the projected pixels fill a 2m tall anisotropic screen with a life-size human body (Fig. 1). The screen material consists of a vertically-anisotropic light shaping diffuser manufactured by Luiminit Co. The material scatters light vertically ( 60 ◦ ) so that each pixel can be seen at multiple viewing heights and while maintaining a narrow horizontal blur ( 1 ◦ ) to smoothly fill in the gaps between the projectors with adjacent pixels. More details on the screen material can be found in Jones et al. [2014]. We use six computers to render the projector images. Each computer contains two ATI Eyefinity 7800 graphics cards with 12 total video outputs. Each video signal is then divided three ways using a Matrox TripleHead-to-Go video HDMI splitter. In the future, we plan on capturing longer format interviews and other dynamic performances. We are working to incorporate natural language processing to allow for true interactive conversations with realistic 3D humans. E INARSSON , P., C HABERT , C.-F., J ONES , A., M A , W.-C., L A MOND , B., H AWKINS , T., B OLAS , M., S YLWAN , S., AND D E BEVEC , P. 2006. Relighting human locomotion with flowed reflectance fields. In Rendering Techniques 2006: 17th Euro- graphics Symposium on Rendering , 183–194. J ONES , A., N AGANO , K., L IU , J., B USCH , J., Y U , X., B OLAS , M., AND D EBEVEC , P. 2014. Interpolating vertical parallax for an autostereoscopic 3d projector ...