Applying a framework to document and analyze the learning design intent with learning analytics in an online course

Abstract

“ETEC 565A: Understanding Learning Analytics” was a new course offered in a Master of Educational Technology programme in UBC in January 2019. In order to support students exploring learning analytics in a more relevant way to them, the final project allows students to choose their own learning analytics adventure. This presentation will be a showcase and a reflection of learning from our final (group) project. Our group wanted to focus on learning design and learning analytics. There has been a lot of interests in learning analytics in higher education, it has been appearing in the EDUCAUSE’s Horizon report for many years as a technology to adopt. We also know that learning technologies should support the educational goals. Therefore, it is important for us to understand, how would one combine learning design with learning analytics. Our instructor guided to Lockyer, Heathcote and Dawson (2013)’s work. Their article presented “learning design as a form of documentation of pedagogical intent that can provide the context for making sense of diverse sets of analytic data” (p.1439). Lockyer et al (2013) explored using the checkpoint and process analytics as broad categories of learning analytics and how through this documentation of pedagogical intent and the related learning analytics that can be collected, could support pedagogical actions.

Our instructor has given us permission to use our course as an example to apply Lockyer et al (2013)’s framework and conduct our analysis on the course. Our group also had access to some learning analytics data ourselves, through the “Threadz” tool for analysing our discussion forum activities. For all other types of data that we did not have access to, we made comments on what the data could be used for, and if it would provide enough information to assess if the pedagogical intent was met. Comments were also made about potential data that could have been better for informing pedagogical actions but were either not possible to get or too difficult and unpractical.

In the presentation, I will share some backgrounds for the Lockyer et al (2013)’s framework for aligning learning analytics with learning design, how one could use the framework to document their own course design and identify potential learning analytics data sources or the lack of. As the framework provides teachers and designers a tool to think, plan and reflect with. I will discuss some of our group’s findings and reflections from the analysis of our own online course.  Then discuss about potentials of using such framework on a more traditional face-to-face course.

As institutions and courses collect more data about their students, it is useful to have a framework to help teachers think about how they might use the learning analytics data to support their students through examining and documenting their pedagogical intents. It is also important to note what the existing data can and cannot do to support pedagogical goals.