Observations

An online observation can be carried out in three ways:

1. Watch an online activity live. This could be an activity that needs to be done online (e.g. using a computer programme) or an activity that takes place online for the purpose of assessment (e.g. conducting an interview). The observation can take place at any location.
2. Watch an activity live on-location. The observers watch an activity live at another location (e.g. a student using a machine while on an internship). The observation can take place at any location.
3. Watch a video recording. The student records the activity (e.g. a teacher training student giving a lesson is recorded with equipment intended for the purpose).

Advantages and disadvantages

The main advantage of online observation is that it can be done from any location, and in the case of a video recording, at any time. This is less time-consuming for the observer, for example because travel to the location is not required. Assessment by a group of observers is also easier to arrange. Research shows that the validity of observations often increases considerably with an increase in the number of observations or observers, and this is made easier using online observation. Another advantage of using a video recording is that the examiner can review the recording together with the student, for example to explain the reasoning for the final grade. The student can also use the video to reflect on the activity.

However, online observation may mean that the observer has less information available than if they were on-location. For example, when observing a lesson given by a teacher training student at a primary school, the observer cannot walk around the classroom, or the view may be limited if children are working in the corridor. It may also be more difficult to interpret non-verbal communication. If not enough information is available, this can affect the validity of the observation. The advantages and disadvantages must therefore be carefully considered in each situation.

Main advantages

• Less time-consuming for the observer
• Easier to organise multiple observers
• Possible to watch recording with student

Main disadvantage

• May limit information available to observer

Combination An observation can be combined with an assessment interview to allow the student to explain why they made certain choices.

Preparation

Assessment form An assessment form is usually used when observing students. Make sure to check that all the assessment criteria can be properly assessed online; do you have all the information you need for each criterion as the observer, or will the lack of some information on the screen (e.g. a client’s reaction) make it impossible to assess certain criteria?

Number of observations and observers The main advantage of online observation is that it can take place at any time and location. This makes it less time-consuming and easier to organise, so that more observations can be carried out using more observers, increasing the validity and generalisability of the observation. In the case of formative assessment, other students can also take on the role of observer.

Live observation

Anticipate technical problems When carrying out an observation live, it is important to inform the students in plenty of time of the preparations they need to take to ensure that the observation can take place without any problems. You should also set ground rules for working online, such as sharing the screen, turning the camera/microphone on and off, the procedure for asking questions and what to do in the case of any technical problems. Students like to know beforehand what to do if there is a problem. You can also encourage students to practice with each other prior to the observation. Some institutions allow students to loan equipment or provide a physical location for an online observation. In any case, make sure to communicate any instructions/ground rules clearly and in plenty of time. Recording Privacy regulations The processing, storage and deletion of recordings must take place in accordance with legal requirements. Find out what the rules are for this at your educational institution. Making a recording If a student has to make a recording, you should provide clear instructions about what is expected: what should and should not be seen on the screen, how long should the recording be, and what equipment may and may not be used? This will prevent discrepancies in the quality of the recorded material, which may affect, possibly subconsciously, the assessment of the activity. We recommend that you consider the best equipment for students to use beforehand. For example, special recording kits are available at some educational institutions, with a camera that films the student and a camera that films the subjects of the activity.

Assessment Discuss with other examiners the rules that apply when watching the recording, to ensure that observations are assessed as uniformly as possible; this may also apply to the assessment of students on-campus. For example, is it permitted to stop or rewind a recording (which is not possible during a live observation)?

During the observation

Assessment Make sure that you have the required documents to hand (rubric, assessment form, etc.), either printed out or on a second screen. If you are working with other examiners and would like to provide direct feedback, you could ask the student to leave the online session for a moment or put them in a breakout room. You can then invite the student to return to the session once you have discussed the observations with the other examiners.

After the observation

If a recording is made of the activity, this can – together with the feedback and assessment – provide the student with a lot of useful information. You could therefore tell students to watch the recordings together, so that they can learn from each other.

• Toetsen in het hoger onderwijs (Van Berkel, Bax, and Joosten-ten Brinke 2017) (Assessment in higher education) is a general book about assessment with a chapter on assessment of performance that deals in more detail with the assessment of observable activities. It contains very useful information on ensuring reliability and validity during this form of assessment.