Skip to main content

Assessing and Evaluating with XR

Effectively Evaluating XR Teaching: Reflection and Analysis as the Keys to Further Development

XR technologies open up new possibilities for teaching and learning, but their potential can only be realised if teachers are willing to reflect on and further develop their teaching. The evaluation of XR-supported learning settings thus becomes a central component of professional teaching practice, whether at the didactic level or in terms of technical implementation. Whether through personal reflection, systematic feedback procedures or learning analytics: different approaches provide valuable insights into learning processes, behaviour and success. Compared to analogue formats, XR environments offer additional opportunities to use evaluation not only as an obligation, but as an effective design tool. As a result, essential approaches are conveyed on which teachers can build:

Formative Assessment

The ongoing (formative) assessment of XR learning environments uses various methods to continuously improve the learning process. With regular feedback or surveys of students, teachers can further develop and improve their XR teaching in a targeted manner.

Summative Assessment

Summative assessment is used for the final evaluation of learning achievements and learning outcomes. In the XR context, it presents teachers with new challenges, but also offers innovative opportunities to measure competences. Since XR learning scenarios are often dynamic, interactive and experience-oriented, it is important to adapt and further develop assessment methods accordingly. The following section presents three key approaches to implementing summative assessment in XR in a meaningful way.

Continuous Feedback and Qualitative Methods

By regularly gathering feedback throughout the learning process, teachers can specifically improve the XR learning experience. Qualitative methods such as observations, the thinking aloud method, reflections or follow-up interviews help to better understand how learners think and feel.

What teachers can do:

  • Seek feedback from students and colleagues in order to adjust lessons immediately if necessary.
  • Observe XR use specifically to identify typical difficulties or particularly successful learning moments.
  • Ask learners to think aloud or conduct interviews and group discussions to better understand the experience.

Example: Students are working on an experiment in a virtual chemistry lab. They are asked to measure liquids precisely using a micropipette. As they think aloud, it becomes clear that some of them are having problems controlling the pipette in the VR environment, for example, positioning it correctly or pressing the virtual flask. Teachers observe these difficulties and adjust the interaction in the system, for example, by simplifying the controls, displaying a tutorial or providing visual aids for hand positioning.

Integrate Assessment Directly into XR Applications

XR applications can automatically record how learners complete tasks and move around in the virtual environment. This allows progress to be measured without interrupting the learning flow. Methods such as eye tracking can be used to observe even more precisely what learners are paying attention to or what they are overlooking.

What teachers can do:

  • Integrate tasks directly into the XR environment so that performance is tracked automatically.
  • Use tools such as eye tracking to see what learners are looking at – e.g. in complex visualisations.
  • Use the collected data to tailor content, e.g. by highlighting or simplifying difficult sections.

Example: In an AR application for biology lessons, learners view a 3D model of the human heart. An eye-tracking system shows that many students hardly pay any attention to the heart valves. As a (didactic) response, these areas are visually highlighted in the app and supplemented with short explanations. This ensures that important content is not overlooked.

Adaptive Tests and Comparison of Teaching Methods

Adaptive tests in XR adapt to the abilities of learners: those who are doing well are given more difficult tasks, while those who need support are given easier ones. Teachers can also compare two versions of a learning programme (A/B testing) to see which works better. When learner feedback is combined with measured performance, an accurate picture of learning success emerges.

What teachers can do:

  • Use adaptive tasks in XR to support learners individually.
  • Test two versions of a scenario to find out which method works better.
  • Combine learner feedback with test results to better understand the impact.

Example: A VR training programme for nursing staff simulates emergency situations. Depending on how confidently someone acts in the situation, the next scenario becomes easier or more difficult. Two groups receive different instructions: one via text, the other via a talking assistant figure. After the training, performances are compared and interviews are conducted. The results show that verbal guidance provides better support in stressful situations, so it will be used as the preferred method in future.

Performance Assessments in Virtual Scenarios

Virtual scenarios enable real-life tests in which learners can directly demonstrate their knowledge and skills. Assessment is based on what they do in the simulation.

What teachers can do:

  • Offer tests in VR, for example, an emergency drill for medical personnel.
  • Use digital lists or observation sheets to assess performance.
  • Automatically record the actions taken by learners in order to evaluate them.

Example: In a VR application for biology lessons, students explore the human body. During a task on breathing, the teacher observes the students using an observation sheet.

Observation sheet (excerpt):

  • Attention and concentration: Is the student focused? (Yes / Partially / No)
  • Understanding of the task: Can the student explain the function of the lungs? (Yes / Partially / No)
  • Interaction with the VR environment: Does the student use the VR tools confidently and correctly? (Yes / Partially / No)
  • Teamwork (when working in groups): Does the student communicate well with other classmates? (Yes / Partially / No)

After the observation, the teacher uses the results to provide targeted support to the students and adapt the lesson.

Digital Tests and Learning Data

Traditional tests can also be incorporated into XR, such as quizzes or short questions at specific points. With additional data (e.g. where students are looking or how long they need for something), performance can be assessed even more accurately.

What teachers can do:

  • Integrate quiz questions directly into the XR environment (e.g. multiple choice).
  • Include the learners’ gaze direction and actions in the assessment.
  • Link results to the learning curve to see individual development.

Example: In a virtual museum tour, students answer quiz questions on historical topics at several stations. The application stores their answers, processing time and navigation behaviour. At the end, the teacher can see which content was well understood and where there is still room for improvement.

Sources

Bortz, J., & Döring, N. (2006). Forschungsmethoden und Evaluation für Human- und Sozialwissenschaften (4. Aufl.). Springer.

Froschauer, U., & Lueger, M. (2024). Das qualitative Interview (3. Aufl.). facultas.

Mayring, P. (2023). Einführung in die qualitative Sozialforschung: Eine Anleitung zu qualitativem Denken (7. überarb. Aufl.). Beltz.

Przborski, A., & Wohlrab-Sahr, M. (2014). Qualitative Sozialforschung: Ein Arbeitsbuch (4. Aufl.). Oldenbourg.

Peer Feedback

Peer feedback can help improve the use of XR in the classroom. It strengthens collaborative learning, encourages participation and supports the further development of XR learning scenarios. When students evaluate the performance and actions of others in the XR environment, they gain a better understanding of the subject matter and train their critical thinking and reflection skills.

Creating a Supportive Learning Environment

Peer feedback in XR promotes an active learning environment in which students learn from each other. Through targeted feedback in groups, they learn new ways of collaborating, problem solving and using the virtual environment.

What teachers can do:

  • Plan joint XR projects in which students work in teams and give each other feedback.
  • Incorporate regular reflection phases into the XR environment to strengthen exchange.
  • Introduce a mentoring system in which experienced students help others find their way around XR.

Example: In a virtual chemistry lab, students work in groups to conduct an experiment on the reaction of substances. After the experiment, they look at the work of the other groups, give feedback on the approach and discuss together which strategies worked best. In this way, they learn not only through their own actions, but also through observation and evaluation of others.

Using Technical Feedback from Learners

Feedback from students helps to identify usability issues and potential barriers to learning at an early stage – things that teachers or developers may overlook.

What teachers can do:

  • Conduct regular short surveys on the user-friendliness of the XR application.
  • Hold ‘bug hunting’ sessions in which students specifically look for problems or improvements.
  • Set up an open feedback forum to collect technical feedback and pass it on to developers.

Example: In a virtual architecture course, students create 3D models of buildings. Several learners report that a particular tool in the application is difficult to find or complicated to use. In a feedback round, they make suggestions on how the toolbar could be structured more clearly. The suggestions are collected and passed on to the developers: the interface is redesigned so that future users can use it more easily

Strengthening Communication and Teamwork Skills

Peer feedback in XR helps students communicate better and work well in teams. These are important skills for school, training and work:

What teachers can do:

  • Offer workshops in which learners practise communicating clearly and respectfully in virtual environments.
  • Conduct role-plays in XR to practise giving and receiving feedback.
  • Use an assessment system that takes into account not only the content but also the quality of the feedback.

Example: In a virtual language course, students give presentations to the class, e.g. about a country or a current topic. Afterwards, they use a clear feedback form to give each other feedback on language, content and presentation skills. In a short reflection session, they discuss together what worked well and what could be improved. In this way, they not only learn the language, but also how to interact constructively with each other.

Learning Analytics in XR: Evaluate Specifically, Use Consciously

XR learning applications collect a lot of data that can help to better understand and evaluate learning progress. Teachers should know what analyses are possible in the apps they use and how these can be used specifically to assess competences. It is important to note that data protection must always be observed. Learners and teachers should be informed about which data is collected, what it is used for and how it is protected.

The following section presents key applications and examples of learning analytics in XR.

Identifying Learning Progress and Evaluating Performance

XR apps collect data on how learners work with the application. This information helps teachers accurately assess students’ learning progress and better evaluate their performance. This allows strengths and weaknesses to be identified.

What teachers can do:

  • Include progress such as points earned or tasks completed in the assessment.
  • Evaluate mistakes in order to provide targeted feedback.
  • Compare results in order to better assess performance levels.

Example: In a VR app for language learning, students practise vocabulary in a simulated everyday situation, e.g. while shopping. The app measures how many terms were used correctly and how often mistakes occur. This allows the teacher to see exactly which words have already been mastered and where further practice is needed, and to structure the repetition accordingly.

Individual Support Through Adaptive XR Systems

XR applications can adapt to the behaviour of learners. If someone has difficulties, the system automatically responds with tips, simpler tasks or additional explanations, for example. This means that everyone receives targeted support.

What teachers can do:

  • Activate automatic help that pops up when problems arise.
  • Suggest individual learning paths based on previous performance.
  • Use feedback that responds in real time to learners’ behaviour.

Example: In an AR maths app, students solve geometry problems by placing shapes in a room. If a student gets a problem wrong several times, the app automatically displays a brief explanation and a simplified exercise. This allows understanding to be improved step by step without interrupting the learning process.

Using XR Data to Improve Teaching

Not only learners benefit from learning analytics, teachers can also use it to improve their teaching in a targeted manner. The data shows what worked well and where there were still difficulties.

What teachers can do:

  • Evaluate data for planning, conducting and reflecting on lessons.
  • Recognise patterns in learning behaviour and adapt lessons accordingly.
  • Use interactions in the XR environment as a basis for practical assessments.

Example: After an XR lesson on the topic of ‘energy sources,’ the teacher sees in the evaluation that many students had problems with renewable energies in an interactive quiz. For the next lesson, she plans targeted repetitions and additional explanations that are tailored to the needs of the class.

Sources

Li, S. (2024). Learning analytics enhanced online learning support [E-Book-Ausgabe]. Routledge.