At last years 8th International Conference on Multimodality (8ICOM) in Cape Town, video producer at CILT, Don Leitch presented on “What does an effective e-learning video look like”. As part of the MOOC team he shares his thoughts with us. 

As the world of MOOCs (Massive Open Online Courses) and e-learning in general continues to expand, educators and researchers have unprecedented access to data and are finding new ways to interpret it. When it comes to video in e-learning however, there is surprisingly little research available, and some fundamental questions of how to interpret what little data there is, remain unanswered.

Much of the difficulty, both in theory and in practice, comes from the confluence of filmmaking with education. The two fields of knowledge can be boiled down, in terms of their comparison with one another, to two fundamentally different concerns. Filmmaking and videography in general is all about audience engagement, while pedagogy is about learning.

It is often pointed out that engagement is a necessary but not sufficient aspect of learning, meaning that one cannot learn anything without being engaged (i.e. caring) but one can easily be engaged and entertained from watching a video without learning a thing.

This has practical implications on how educational videos are produced. I work as a video producer on MOOCs at the University of Cape Town. Like almost all video producers my practical and theoretical training are centred around film as an art form and as a medium of entertainment. Many of the techniques and theory focused on audience emotion, engagement, and immersion. This presents a challenge when attempting to balance my idea of the best way to make a video with that of the learning designer or instructor who is relying on an entirely different practical and theoretical background.

This divergence of engagement and learning can also be found in the research done on videos in e-learning. I have noted five broad categories of studies assessing e-learning videos; surveys, taxonomies, video interaction data, AB testing, and controlled experiments.

Surveys are useful for knowing audience preference and opinion, but often don’t correlate well with actual learning. There are some interesting data points however, such as the fact that people are apparently more forgiving of longer educational videos than entertainment videos.

Taxonomies are broad studies that attempt to categorise videos and measure how many of what type are used where. I believe such research will be most useful when used in combination with other data to create a more complete picture of the e-learning arena.

Studies making use of video interaction data interpret information generated by viewers pausing and moving to different parts of videos. These studies have the advantage of very large sample sizes (making use of MOOCs with thousands of participants) and have interesting implications about how e-learning videos should be edited. Due to the nature of MOOCs, however, it is very difficult to marry this information to measurements of student learning and the studies are limited to measuring engagement alone.

AB testing is widely used by MOOC platforms to assess the best ways to retain students and have great potential for setting up controlled experiments of enormous sample sizes. For example, one could test two different styles of video production to see which is more effective. Unfortunately, MOOC platforms tend not to publicly publish their results. Furthermore, the same problem of linking engagement data to learning data exists.

Researches are able to get around the problem of measuring both learning and engagement by setting up smaller scale controlled experiments. These do have smaller sample sizes but have yielded some interesting results about the link between engagement and learning. 

Ultimately as producers of e-learning videos we largely rely on our cross-disciplinary interaction, some basic findings from research, as well as intuition and opinion. There is certainly much more that can be learned from the large amounts of data being generated, but we will need to think a lot more about how to make use of the findings  for improving production.

Written by Don Leitch

(For the full slideshow please click here)

References:

• Chen, Z., Chudzicki, C., Palumbo, D., Alexandron, G., Choi, Y.J., Zhou, Q. and Pritchard, D.E., 2016. Researching for better instructional methods using AB experiments in MOOCs: results and challenges. Research and Practice in Technology Enhanced Learning, 11(1), pp.1-20.
• Guo, P.J., Kim, J. and Rubin, R., 2014, March. How video production affects student engagement: An empirical study of mooc videos. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 41-50). ACM.
• Hansch, A., Hillers, L., McConachie, K., Newman, C., Schildhauer, T. and Schmidt, P., 2015. Video and online learning: Critical reflections and findings from the field
• Kim, J., Guo, P.J., Seaton, D.T., Mitros, P., Gajos, K.Z. and Miller, R.C., 2014, March. Understanding in-video dropouts and interaction peaks in online lecture videos. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 31-40). ACM.
• Kizilcec, R.F., Papadopoulos, K. and Sritanyaratana, L., 2014, April. Showing face in video instruction: effects on information retention, visual attention, and affect. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 2095-2102). ACM.
• Pierce, M., 2015. Learning and Development: What Makes Videos Effective? Iconlogic. http://iconlogic.blogs.com/weblog/2015/02/learning-and-development-what-makes-videos-effective.html
• Reutemann, J., 2016. Differences and Commonalities–A comparative report of video styles and course descriptions on edX, Coursera, Futurelearn and Iversity. Proceedings of the European Stakeholder Summit on experiences and best practices in and around MOOCs (EMOOCS 2016), p.383.
• Simonite, T., 2013. As data floods in, massive open online courses evolve. Technology Review.
• Wulff, B., Rupp, L., Fecke, A. and Hamborg, K.C., 2013, December. The lecture sight system in production scenarios and its impact on learning from video recorded lectures. In Multimedia (ISM), 2013 IEEE International Symposium on (pp. 474-479). IEEE.