Too often, it seems, conversations about learning analytics focus too much on means, and not enough on ends. Learning analytics initiatives are always justified by the promise of using data as a way of identifying students at risk, in order to develop interventions that would increase their chances of success. In spite of the fact that the literature almost always holds such intervention out as a promise, a surprising lack of attention is paid to what these interventions might look like. A recent paper presented by Wise, Zhao, and Hausknecht at the 2013 Conference on Learning Analytics and Knowledge (LAK’13) goes a long way in putting learning analytics in perspective, taking some crucial first steps in the direction of a model of learning analytics as a pedagogical practice.
Analytics ABOUT Learning
Like so many, I often find myself being sucked into the trap of thinking of learning analytics as a set of tools for evaluating learning, as if learning and analytics inform one another as processes that are complementary, but nonetheless distinct. In other words, it is easy for me to think of learning analytics as analytics ABOUT learning. What this group of researchers from Simon Fraser University show, however, is that it is possible to think of learning analytics as a robust pedagogical practice in its own right. From analytics ABOUT learning, Wise, Zhao, and Hausknecht encourage us to think about analytics AS learning.
Analytics AS Learning
The paper is ostensibly interested in analytics for online discussions, and is insightful in its emphasis on dialogical factors, like the extent to which students not only actively contribute their own thoughts and ideas, but also engage in ‘listening’-type behaviors (i.e. thoughtful reading) that would engender engagement in community and a deeper level of discussion. More generally, however, two observations struck me as generally applicable to thinking of learning analytics as a pedagogical practice.
1. Embedded Analytics are also Interventions
Wise et al make a distinction between embedded analytics, which are “embedded in the discussion interface and can be used by learners in real-time to guide their participation,” and extracted analytics, which involve the collection of traces from learning activity in order to interpret them apart from the learning activity itself. Now, the fact that student-facing activity dashboards are actually also (if not primarily) intervention strategies is perhaps fairly obvious, but I have never thought about them in this way before. #mindblown
2. Analytics are Valued, through and through
By now we all know that, whatever its form, research of any kind always involves values, no matter how much we might seek to be value neutral. The valued nature of learning analytics, however, is particularly salient as we blur the line between analysis (which concerns itself with objects) and learning (which concerns itself with subjects). Regardless of the extent to which we realize how our use of analytics reinforces values and behaviors beyond those explicitly articulated in a curriculum, THAT we are using analytics and HOW we are using them DO have an impact. Thinking carefully about this latent curriculum and actively identifying the core values and behaviors that we would like our teaching practices to reinforce allows ensure consistency across our practices and with the larger pedagogical aims that we are interested in pursuing.
Wise, Zhao, and Hausknecht identify six principles (with which I am generally sympathetic) that guide their use of analytics as, and for the sake of, pedagogical intervention:
- Integration – in order for analytics to be effectively used by students, the instructor must present those analytics are meaningfully connected to larger purposes and expectations for the course or activity. It is incumbent upon the ‘data-driven instructor’ to ensure that data are not presented merely as a set of numbers, but rather as meaningful information of immediate relevance to the context of learning.
- Diversity (of metrics) – if students are presented with too few sources of data, it becomes very easy for them to fixate upon optimizing those few data points to the exclusion of others. Sensitive also to the opposite extreme, which would be to overload students with too much data, it is important to present data in such a say as to encourage am holistic approach to learning and learning aims.
- Agency – students should be encouraged to use the analytics to set personal goals, and to use analytics as a way of monitoring their progress relative to these. Analytics should be used to cultivate sutonomy and a strong sense of personal responsibility. The instructor must be careful to mitigate against a ‘big-brother’ approach to analytics that would measure all students against a common and rigid set of instructor-driven standards. The instructor must also act to mitigate against the impression that this is what is going on, which has the same effect.
- Reflection – encouraging agency involves cultivating habits of self-reflection. The instructor should, therefore, provide explicit time and space for reflection on analytics. The authors, for example, use an online reflective journal that is shared between students and instructor.
- Parity – activities should be designed to avoid a balance of power situation in which the instructor collects data on the students, and instead use data as a reflective and dialogic tool between the instructor and students. In other words, data should not be used for purposes of evaluation or ranking, but rather should be used as a critical tool for the purpose of identifying and correcting faults or reinforcing excellences.
- Dialogue – just as analytics are used as an occasion for students to cultivate agency through active reflection on their behavior, the instructor should “expose themselves to the same vulnerability as the students.” Not only should instructors attend to and reflect upon their own analytics, but do so in full view of the class and in such a way as to allow students to criticize him/her in the same way as s/he does them.