Prototype of a Hybrid Communication System: Bridging VR and Video Conferencing

Abstract

The Metaverse and immersive technologies related to it open up new possibilities for remote communication and collaboration, but often create a barrier between users in virtual reality (VR) and those using traditional video conferencing tools. This work presents the design and implementation of a prototype hybrid communication system that addresses this problem by creating a bidirectional visual bridge. The system, developed in the Unity engine, allows users in VR to see participants of a standard video call on virtual panels, and conversely, video call participants see the view from the VR scene. It utilizes a combination of the Mirror library for synchronizing the state of the multiplayer VR scene and the Agora RTC SDK for real-time audio and video transmission between VR and non-VR clients, including sending custom video and audio sources from Unity. The result is a prototype demonstrating the technical feasibility of the proposed approach towards more flexible hybrid communication.