Augmented Reality Body Image (ARBI) - Technical Document

This application gives users the opportunity to compare and adjust a virtual avatar with an actual image of themselves through AR technology. Using real patient data from the EMR, this application presents an avatar of the patient, assuming proportions in relation to the data that is fetched. The user/patient then adjusts the avatar to more accurately represent themselves, while comparing the avatar to a photograph or other real-life representation. The goal of the application is to help those who struggle with body dysmorphia have a more healthy and realistic vision of their body image. Features: Fetching patient data (height and weight) from MyChart; Rough avatar representation generated by application; Customization features for avatar; Comparison scene; User compares avatar to a photograph of themselves in real space; Hand-gesture controls/input: pointing, grabbing, dragging through HoloLens technology.

EM(b)R Hololens Technical Document ***We've set up a GApp email to use for development testing: [e-mail] Password: SeriousGames999 Project Goals The EM(b)R Hololens project's goal is to create a meaningful interaction between patients and their personal data. Patients will be able to visualize essential data in a uniquely interactive and compelling way, thereby increasing their understanding of how they perceive about themselves. In this regard, the project uses augmented reality as the medium to provide the interactive environment to avatar customization. Current Scope Platform: Hololens Constraints: The platform is completely new and has a lot of learning. Patient data: We are currently dealing with the height and weight of the patient. Functional Requirements The application in its current state satisfies the following functional requirements: - Spatial Mapping which is responsible for placing objects on flat surfaces and also responsible for scanning the whole surroundings Avatar customization for modifying the body of the avatar. Intractable photo capture wto compare the customized avatar with the user. A visualization to provide a simple illustration of avatar animation. Non-Functional Requirements - Accessibility: Currently accessible at Gapp Lab Performance - for example Response Time, Throughput, Utilization, Static Scalability Capacity Availability Reliability Recoverability Maintainability Serviceability Security Regulatory Manageability Environmental Data Integrity Usability System Architecture Backend Responsibilities of the back end are, - Provide data to the client facing front end Store data required by the client application The backend constitutes a WAMP server, forming the second tier in the following architecture: The server is currently being hosted on 155.101.206.140 (EAE-G-M-06). It communicates with the FHIR sandbox (data server) to perform the following tasks: - Fetch patient token during user sign up Validate user based on the username and token received Fetch and store (temporarily) patient observation data from the sandbox using the corresponding LOINC code. Database on the WAMP server: diabetes-vr Tables on database: Patient Data : Contains the height and weight information responsible for comparision between the patient and the avatar. Fetcha1cdata.php - uses the required parameters (height, weight, name) to fetch data. Sandbox information: URL for fetching patient data- URL for fetching patient observationhttps://openic.epic.com/FHIR/api/FHIR/DSTU2/Observation?patient=<token>&code=<loinc> Things to remember: - Observation data is presented in a particular hierarchy, so this may be different for different observations and hence requires tweaking based on the requested data. Front End The client facing application is currently being developed using Unity 5.4.0 f3 HTP. Commented [1]: Same logic but still need to get data from FHIR Prerequisites: Tools to Instal: https://developer.microsoft.com/en-us/windows/holographic/install_the_tools Getting Started: Set up Unity with Hololens: https://developer.microsoft.com/enus/windows/holographic/unity_development_overview Building & Exporting: https://developer.microsoft.com/enus/windows/holographic/holograms_100 Flow: The unity project constitutes 1 scenes with 4 states: ➔ 1. 2. 3. 4. MainScene Spatial Mapping: The play space is mapped and recorded in this state. Avatar Customization: Avatar customization takes place Avatar Animation: Avatar animations using Mixamo animations Comparing Avatar: Comparing customized avatar with user's image. Scripts: Refer to the Scripts Doc for the detailed view of all used scripts. Todo with respect to front end: ➔ Follow strict coding standards and naming conventions. ➔ Add front end validation wherever required before communication with the backend. For example: login/signup scene.