Collaborating on physical objects when individuals are not in the same room can be a daunting task. However, a groundbreaking new remote conferencing system called SharedNeRF is changing the game by allowing the remote user to manipulate a 3D view of the scene to assist in complex tasks such as debugging intricate hardware. Developed by Mose Sakashita, a doctoral student in information science, SharedNeRF combines two distinct graphics rendering techniques to provide a more immersive and interactive experience for remote collaboration.
SharedNeRF leverages a combination of slow and photorealistic rendering with instantaneous but less precise techniques to enable remote users to experience the physical space of their collaborators in real-time. This innovative approach aims to revolutionize how individuals work on tasks that were previously challenging to convey through traditional video-based conferencing systems with a static viewpoint.
Sakashita’s work on SharedNeRF, conducted during an internship at Microsoft in 2023 in collaboration with Andrew Wilson, explores the potential of merging photorealistic and view-dependent rendering for remote collaboration. The system is set to be presented at the ACM CHI conference on Human Factors in Computing Systems and has already received an honorable mention for its groundbreaking approach to addressing the limitations of existing video conferencing systems.
SharedNeRF utilizes a neural radiance field (NeRF) rendering method that employs artificial intelligence to construct a detailed 3D representation of a scene using 2D images. This advanced technique allows for the creation of highly realistic depictions with reflections, transparent objects, and accurate textures that can be viewed from any angle. By integrating a NeRF deep learning model with point cloud rendering technology, SharedNeRF offers a unique blend of detailed visuals and real-time scene updates.
User-Centric Design and Testing
In a recent study involving seven volunteers who tested SharedNeRF by engaging in a collaborative flower-arranging project, the system received positive feedback. Participants preferred SharedNeRF over traditional video conferencing tools or point cloud rendering alone, highlighting its ability to enhance visibility of design details and provide greater control over the viewing experience. Users appreciated the ability to independently change viewpoints, zoom in and out on the scene, and seamlessly interact with the local collaborator.
Future Prospects and Expansion
While SharedNeRF is currently designed for one-on-one collaboration, the research team behind the system envisions its potential for scalability to accommodate multiple users in the future. Future work will focus on improving image quality and exploring immersive experiences through virtual reality and augmented reality technologies. SharedNeRF represents a significant step forward in the realm of remote collaboration, offering a glimpse into the future of interactive and engaging conferencing systems.
Leave a Reply