Testing and Benchmarking VR Platforms
Last updated
Last updated
Benchmarking led to a test phase based on criteria that fit our purposes; free of access, support for collaboration and teamwork, subscription type, headset support, minimum of ten users. AltspaceVR (discontinued), Horizon Workrooms, Glue and later Spatial.io were selected for further testing with students and staff. AltspaceVR was discontinued by Microsoft in March 2023.
Most software allows test use to a certain number of participants. For example the maximum number of users to access a free session can be up to ten persons.
The platform testing phase that started in the spring of 2022 was first done among the Metropolia Agile XR project group to gain more knowledge of how user-friendly the platforms were. Students, teachers and other staff participated in the testing.
Below are some example data fields from student ratings completed in Figma:
Further testing was then done with bigger groups of students and pilots were done with upper secondary school students. All testing sessions are listed in Table 1.
4-15 persons were involved in each session. Sessions started with the onboarding process and included different activities, like sharing content, text, pictures or video, meeting, teamwork or just playing, on selected platforms.
Pilots with upper secondary school students included an organised learning experience and included learning material such as videos, instructions and teamwork among students. During a Learning, Training, Teaching Activities LTTA event staff from all of the Agile XR project partners participated in testing of chosen platforms. This allowed us to gain collective experience of the virtual environments themselves as well as activities in them
During a Learning, Training, Teaching Activities LTTA event staff from all of the Agile XR project partners participated in testing of chosen platforms. This allowed us to gain collective experience of the virtual environments themselves as well as activities in them.
Table 1. Comparing and testing chosen platforms
Testing different platforms gave us more understanding of the usability. We rated platforms with different features like how easy the signing in process was, user interface, or what kind of functionalities they offer, avatar qualities or the graphics of the environments.
Most software can be installed in both Mac and Windows environments and allow different ways to register and create an account / profile by using different and multiple user devices, desktop, VR-devices or mobile.
Many of the platforms still have cartoon-like graphics. This is due to many reasons, one being the capacity to perform screen pictures. Cartoon like environments suit well in the game industry but for work purposes it might disturb the credibility.
Virtual spaces people are able to hear in three dimensions. Within the virtual space it is also possible to control audio.
