Natan Jacobson

University of California, San Diego, San Diego, CA, United States

Are you Natan Jacobson?

Claim your profile

Publications (8)23.49 Total impact

  • Source
    Natan Jacobson, Truong Q Nguyen
    [Show abstract] [Hide abstract]
    ABSTRACT: Our understanding of human visual perception has been paramount in the development of tools for digital video processing. For this reason, saliency detection, i.e., the determination of visual importance in a scene, has come to the forefront in recent literature. In the proposed work, a new method for scale-aware saliency detection is introduced. Scale determination is afforded through a scale-space model utilizing color and texture cues. Scale information is fed back to a discriminant saliency engine by automatically tuning center-surround parameters through a soft weighting. Excellent results are demonstrated for the proposed method through its performance against a database of measured human fixations. Further evidence of the proposed algorithm's performance is demonstrated through an application to frame rate upconversion. The ability of the algorithm to detect salient objects at multiple scales allows for class-leading performance both objectively, in terms of peak signal-to-noise ratio/structural similarity index, and subjectively. Finally, the need for operator tuning of saliency parameters is dramatically reduced by the inclusion of scale information. The proposed method is well suited for any application requiring automatic saliency determination for images or video.
    IEEE Transactions on Image Processing 12/2011; 21(4):2198-206. · 3.20 Impact Factor
  • Source
    Natan Jacobson, Yoav Freund, Truong Q Nguyen
    [Show abstract] [Hide abstract]
    ABSTRACT: We propose a novel online learning-based framework for occlusion boundary detection in video sequences. This approach does not require any prior training and instead "learns" occlusion boundaries by updating a set of weights for the online learning Hedge algorithm at each frame instance. Whereas previous training-based methods perform well only on data similar to the trained examples, the proposed method is well suited for any video sequence. We demonstrate the performance of the proposed detector both for the CMU data set, which includes hand-labeled occlusion boundaries, and for a novel video sequence. In addition to occlusion boundary detection, the proposed algorithm is capable of classifying occlusion boundaries by angle and by whether the occluding object is covering or uncovering the background.
    IEEE Transactions on Image Processing 07/2011; 21(1):252-61. · 3.20 Impact Factor
  • Source
    N. Jacobson, Y. Freund, T.Q. Nguyen
    [Show abstract] [Hide abstract]
    ABSTRACT: In this work, a novel occlusion detection algorithm using online learning is proposed for video applications. Each frame of a video is considered as a time-step for which pixels are classified as being either occluded or non-occluded. The Hedge algorithm is employed to determine weights for a set of experts, each of which is tuned to detect a specific type of occlusion boundary. In contrast to previous training-based methods, the proposed algorithm does not require any training, and has a runtime linear with respect to the number of experts considered. Detection performance is excellent on novel video sequences for which training data does not exist. In addition, the proposed algorithm is easily extended to provide classification results supplementary to detection. We demonstrate results on a series of challenging video sequences including a dataset of hand-labelled occlusion boundaries.
    Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on; 06/2011
  • Source
    N. Jacobson, T.Q. Nguyen
    [Show abstract] [Hide abstract]
    ABSTRACT: A new method for scale-aware saliency detection is introduced in this work. Scale determination is realized through a fast scale-space algorithm using color and texture. Scale information is fed back to a Discriminant Saliency engine by automatically tuning center-surround parameters. Excellent results are demonstrated for predicted fixations using a public database of measured human fixations. Further evidence of the proposed algorithm's performance is exhibited through an application to Frame Rate Up-Conversion (FRUC). The ability of the algorithm to detect salient objects at multiple scales allows for class-leading performance both objectively in terms of PSNR/SSIM as well as subjectively. Finally, the need for operator tuning of saliency parameters is dramatically reduced by the inclusion of scale information. The proposed method is well-suited for any application requiring automatic saliency determination for images or video.
    Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on; 06/2011
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: An adaptive spatiotemporal saliency algorithm for video attention detection using motion vector decision is proposed, motivated by the importance of motion information in video sequences for human visual system. This novel system can detect the saliency regions quickly by using only part of the classic saliency features in each iteration. Motion vectors calculated by block matching and optical flow are used to determine the decision condition. When significant motion contrast occurs (decision condition is satisfied), the saliency area is detected by motion and intensity features. Otherwise, when motion contrast is low, color and orientation features are added to form a more detailed saliency map. Experimental results show that the proposed algorithm can detect salient objects and actions in video sequences robustly and efficiently.
    Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, May 22-27, 2011, Prague Congress Center, Prague, Czech Republic; 01/2011
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Motion-compensated frame interpolation (MCFI) is a technique used extensively for increasing the temporal frequency of a video sequence. In order to obtain a high quality interpolation, the motion field between frames must be well-estimated. However, many current techniques for determining the motion are prone to errors in occlusion regions, as well as regions with repetitive structure. We propose an algorithm for improving both the objective and subjective quality of MCFI by refining the motion vector field. We first utilize a discriminant saliency classifier to determine which regions of the motion field are most important to a human observer. These regions are refined using a multistage motion vector refinement (MVR), which promotes motion vector candidates based upon their likelihood given a local neighborhood. For regions which fall below the saliency-threshold, a frame segmentation is used to locate regions of homogeneous color and texture via normalized cuts. Motion vectors are promoted such that each homogeneous region has a consistent motion. Experimental results demonstrate an improvement over previous frame rate up-conversion (FRUC) methods in both objective and subjective picture quality.
    IEEE Transactions on Image Processing 12/2010; · 3.20 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Motion-Compensated Frame Interpolation (MCFI) is a technique used extensively for increasing the temporal frequency of a video sequence. In order to obtain a high quality interpolation, the motion field between frames must be well-estimated. However, many current techniques for determining the motion are prone to errors in occlusion regions, as well as regions with repetitive structure. An algorithm is proposed for improving both the objective and subjective quality of MCFI by refining the motion vector field. A Discriminant Saliency classifier is employed to determine regions of the motion field which are most important to a human observer. These regions are refined using a multi-stage motion vector refinement which promotes candidates based on their likelihood given a local neighborhood. For regions which fall below the saliency threshold, frame segmentation is used to locate regions of homogeneous color and texture via Normalized Cuts. Motion vectors are promoted such that each homogeneous region has a consistent motion. Experimental results demonstrate an improvement over previous methods in both objective and subjective picture quality.
    Multimedia and Expo (ICME), 2010 IEEE International Conference on; 08/2010
  • Source
    Natan Jacobson, Truong, Q Nguyen