| dc.description.abstract | Mixed and augmented reality (MR/AR) is a blend of physical and virtual worlds, where computer-generated objects are overlaid on the real-world environment. This technology creates an immersive experience that allows users to interact with both physical and virtual items and scenarios. It seamlessly integrates digital information with the user's environment in real-time, enhancing the perception of reality. Its applications are extensive, ranging from gaming and entertainment to education, healthcare, and industrial training, revolutionizing the way we interact with the world around us.
This dissertation presents research for developing innovative techniques that increase the perceived realism of interactions with both humans and objects in MR/AR. This research aims to: (1) encourage natural and meaningful interactions between humans in a virtual environment by modifying the pose of virtual avatars to preserve the user’s intended body language, (2) demonstrate the ability to instill the perception of weight for virtual objects in mixed reality using a single degree-of-freedom kinesthetic feedback device, and (3) Create an innovative thermal feedback system designed for augmented reality that realistically represents the thermal properties of virtual objects and provides thermal patterns within a targeted region.
Preserving body language while accurately representing gestures and correcting positional errors in synchronized AR workspaces is addressed by developing a pose optimization algorithm based on a novel pose similarity cost function integrated within a multi-objective optimization framework. At a time when technological advancements are progressively merging the virtual and physical realms, the developments proposed in this dissertation carry significant promise. The methodology suggested enables avatars to meticulously emulate a user's actions, thus acting as a user's extension in the digital environment. This technique goes beyond merely mimicking gestures, it strives to replicate the user's body posture with a high degree of accuracy. This level of detail enhances the reflection of the user in the virtual world, amplifying the overall user experience.
Realistic perception of a virtual object is greatly impacted by instilling an accurate sense of weight. A single degree-of-freedom system providing force feedback about the user's wrist is developed to provide the perception of object weight while maintaining hand dexterity. This hypothesis broadens our understanding and challenges traditional views on our engagement with objects within virtual spaces. It questions the prevailing idea that haptic feedback should perfectly match the contact points with virtual objects and that a more complex, high-fidelity feedback is always the best approach. The credibility of this innovative concept is established through the design, fabrication, and thorough evaluation of a mobile platform equipped to offer kinesthetic feedback.
Thermal feedback is an often-overlooked component of physical interaction, with limited work in providing realistic thermal sensations. A thermal device consisting of an array of thermoelectric modules is presented, with the ability to not only allow users to accurately discriminate thermal sensations but also detect spatial and spatial-temporal patterns. This method differentiates itself from prior research by leveraging an array of compact thermoelectric actuators, arranged to accurately depict variations in heat distribution across a large portion of the user's palm. Utilizing this cluster of actuators, the technique successfully emulates and relays intricate thermal sensations, thus enriching the user's immersive experience. While thermoelectric modules have conventionally served the purpose of providing thermal feedback, the distinctive contribution of this dissertation is its innovative use of such modules to transmit spatial and spatial-temporal cues to the user.
This dissertation provides an initial exploration into advancing the realism of interactions in mixed and augmented reality through fostering natural human interactions, realistic perception of virtual object weight, and innovative thermal feedback. The development of a pose optimization algorithm, a kinesthetic feedback device, and an array of thermoelectric modules respectively addresses these facets of interactivity in MR/AR. This research underscores the value of natural body language representation, challenges conventional views on object interaction, and highlights the potential of spatial-temporal thermal feedback in enhancing immersive experiences. In a world where the lines between the virtual and physical are increasingly blurred, these techniques promise a more realistic and richer engagement with the digital world, positively affecting user experience in MR/AR settings. | |
