Hang Yin’s research while affiliated with Carnegie Mellon University and other places

... By combining machine learning techniques for spatial understanding as well as object segmentation and classification (e.g., Augmented Object Intelligence [11]), our approach for transforming virtual objects relative to their 3D centroid can be extended to physical objects as well. Such a system could create a virtual replica of relevant physical objects [31] and apply diminished reality techniques [8] to remove the physical objects from view. This would allow the virtual replica objects to be transformed for each user in the same way as in the Decoupled Hands approach. ...

Reference:

Decoupled Hands: An Approach for Aligning Perspectives in Collaborative Mixed Reality