Using Data You Already Have

I-WANT-DATA-NOWThis is a rich (and at times quite dense), article by authors from the University of Central Florida that effectively demonstrates some of the potential for developing predictive models of student (non-)success, but also some of the dangers. It emphasizes the fact that the data do not speak for themselves, but require interpretation at every level. Interpretation not only guides researchers in the questions they ask and the ways that certain insights become actionable, but also their interventions.

Dziuban, Moskal, Cavanagh & Watts (June 2012) “Analytics that Inform the University: Using Data You Already Have”

An interesting example from the article is the observation that, when teaching modalities are compared (ex Blended, Online, Face-To-Face, Lecture Capture), the blended approach is found to produce greater success (defined as a grade of C or higher) and fewer withdrawals. Lecture Capture, on the other hand, sees the least success and the most withdrawals, comparatively. This is a striking observation (especially as institutions invest more and more in products like Echo360, and as MOOC companies like coursera begin to move into the business of providing lecture-capture technology). When modality is included in a logistic regression, however, that includes other variables (ex. Cumulative GPA, High School GPA, etc), it is found to have nearly no predictive power. The lesson here, is that our predictive models need to be carefully assessed, and interventions carefully crafted, so that we are actually identifying students at risk, and that our well-meaning, but some-what mechanically generated, interventions do not have unexpected and negative consequences (i.e. What is the likelihood that identifying a PARTICULAR student as ‘at risk’ may in fact have the effect of DECREASING their chances of success? )

Personal Activity Monitors and the Future of Learning Analytics

Jawbone Up and Learning AnalyticsIn Spring 2013, while discussing the details of his final project, a gifted student of mine revealed that he was prone to insomnia. In an effort to understand and take control of his sleeping habits, had began wearing a device called a ‘Jawbone UP.’ I recently started wearing the device myself, and have found it an exciting (and fun) technology for increasing behavioral awareness, identifying activity patterns (both positive and negative), and motivating self-improvement. Part of the movement toward a quantification of self, this wearable technology not only exemplifies best practice in mobile dashboard design, but it also opens up exciting possibilities for the future of learning analytics.

Essentially, the UP is a bracelet that houses a precision motion sensor capable of recording physical activity during waking hours, and tracking sleep habits during the night. The wearable device syncs to a stunning app that presents the user with a longitudinal display of their activity and makes use of an ‘insight engine’ that identifies patterns and makes suggestions for positive behavioral improvements. The UP is made even more powerful by encouraging the user to record their mood, the specifics of deliberate exercise, and diet. The motto of the UP is “Know Yourself, Live Better.” In the age of ‘big data,’ an age in which it has become possible to record and analyze actual behavioral patterns in their entirety rather than simply relying upon samples of anecdotal accounts, and in which our mobile devices are powerful enough to effortlessly identify patterns of which we, ourselves, would otherwise be quite ignorant, the UP (and its main competitor, the Fitbit Flex) are exemplary personal monitoring tools, and represent exciting possibilities for the future of learning analytics.

Personal activity monitors like the UP effectively combine three of the six “technologies to watch,” as identified in the 2013 Higher Education Edition of the NMC Horizon Report: Wearable Technology, Learning Analytics, and Games and Gamification.

Wearable Technology. As a bracelet, the UP is obviously a wearable technology. This kind of device, however, is strikingly absent from the list of technologies listed in the report, which tend to have a prosthetic quality, extending the user’s ability to access and process information from their surroundings. The most interesting of these, of course, is Google’s augmented-reality-enabled glasses, Project Glass. In contrast to wearable technologies that aim at augmenting reality, motivated by a post-human ambition to generate a cyborg culture, the UP has an interestingly humanistic quality. Rather than aiming at extending consciousness, it aims at facilitating self-consciousness and promoting physical and mental well-being by revealing lived patterns of experience that we might otherwise fail to recognize. The technology is still in its infancy and is currently only capably of motion sensing, but it is conceivable that, in the future, such devices might be able to automatically record various other bodily activities as well (like heart-rate and geo-location, for example).

Learning Analytics. Learning Analytics is variously defined, but it essentially refers to the reporting of insights from learner behavior data in order to generate interventions that increase the chances of student success. Learning analytics takes many forms, but one of the most exciting is the development of student dashboards that identify student behaviors (typically in relation to a learning management system like Blackboard) and make relevant recommendations to increase academic performance. Acknowledging the powerful effect of social facilitation (the social-psychological insight that people often perform better in the presence of others than they do alone), such dashboards often also present students with anonymized information about class performance as a baseline for comparison. To the extent that the UP and the Fitbit monitor activity for the purpose of generating actionable insights that facilitate the achievement of personal goals, they function in the same way as student dashboards that monitor student performance. Each of these systems are also designed as application platforms, and the manufacturers strongly encourage the development of third-party apps that would make us of and integrate with their respective devices. Unsurprisingly, most of the third-party apps that have been built to date are concerned with fitness, but there is no reason why an app could not be developed that integrated personal activity data with information about academic behaviors and outcomes as well.

Games and Gamification. The ability to see one’s performance at a glance, to have access to relevant recommendations for improvement according to personal goals, and to have an objective sense of one’s performance relative to a group of like individuals can be a powerful motivator, and it is exactly this kind of dashboarding that the UP does exceptionally well. Although not aimed at academic success, but on physical and mental well-being, the UP (bracelet and app) functions in the same way as learning analytics dashboards, but better. To my mind, the main difference between the UP and learning analytics dashboards–and the main area in which learning analytics can learn from consumer products such as this–is that it is fun. The interface is user-friendly, appealing, and engaging. It is intentionally whimsical, like a video game, and so encourages frequent interaction and a strong desire to keep the graphs within desired thresholds. The desire to check in frequently is further increased by the social networking function, which allows friends to compare progress and encourage each other to be successful. Lastly, the fact that the primary UP interface takes the form of a mobile app (available for both  is reflective of the increasing push toward mobile devices in higher education. Learning analytics and student dashboarding can only promote student success if the students use it. More attention must be placed, then, on developing applications and interfaces that students WANT to use.

Screen shot of the iOS dashboard from the Jawbone UP app

Personal activity monitors like the UP should be exciting to, and closely examined by, educators. As a wearable technology that entices users to self-improvement by making performance analytics into a game, the UP does exactly what we are trying to do in the form of student activity dashboards, but doing it better. In this, the UP app should serve as an exemplar as we look forward to the development of reporting tools that are user-focused, promoting ease, access, and fun.

Looking ahead, however, what is even more exciting (to me at least) is the prospect that wearable devices like the UP might provide students with the ability to extend the kinds of data that we typically (and most easily) correlate with student success. We have LMS information, and more elaborate analytics programs are making effective use of more dispositional factors. Using the UP as a platform, I would like to see someone develop and app that draws upon the motion, mood, and nutrition tracking power of the UP and that allows students to relate this information to academic performance and study habits. Not only would such an application give students (I would hesitate to give personal data like this to instructors and / or administrators) a more holistic vision of the factors contributing or detracting from academic success, but it would also help to cultivate healthy habits that would contribute to student success in a way that extends beyond the walls of the university and into long-term relationships at work, to family, and with friends as well.

The Costs of Privacy

In November 2012, in response to threats of expulsion from John Jay Science & Engineering Academy on account of her refusal to wear a mandatory RFID badge, Andrea Hernandez filed a law suit against San Antonio’s Northside Independent School District. If she continues to refuse even to wear an RFID-disabled badge–an accommodation sanctioned by a federal district judge who ruled against her–Hernandez will be placed in Taft High School beginning in September 2013, the public school to which she would normally be assigned.

In refusing to wear even an RFID-disabled badge, Hernandez’s case seems to have lost its ‘bite’ (it’s difficult to justify her appeal to religious freedom once tracking mechanisms are disabled). In spite of the fact that her concerns were ultimately voiced in terms of an interest in preserving religious freedom, however, the case nonetheless draws attention to the potential costs of privacy.

As elite institutions increasingly adopt comprehensive analytics programs that require students to give up their privacy in exchange for student success, are they also strongly contributing to a culture in which privacy is no longer valued? A robust analytics program requires every student to opt-in (i.e. students are not given the option of opting out). If analytics programs are seen as effective mechanisms to increase the chances of student success, and such programs are effective only to the extent that they gather data that is representative of their entire student body, and, as such, consenting to being tracked is made a condition of enrollment at the most elite universities (universities with the resources necessary to build and sustain such programs), then students must ask what it is that they value more: an education at a world-class institution (and all of the job prospects and other opportunity that such an education affords), or the ability to proverbially click ‘do not track.’ My suspicion is that, if explicitly given the choice, the vast majority of students are willing to give up the latter for the former, a symptom of our growing acceptance of, and complacence toward, issues of electronic privacy, but perhaps also an indication that a willingness to sacrifice privacy for success increasingly forms a key part of the ‘hidden curriculum.’

Erasing Privacy

[Image Creative Commons licensed / Flickr user Alan Cleaver]

(Interestingly, in addition to gathering data from Learning Management and operational systems, universities also regularly collect data from student id card swipes. This data can easily be mobilized as part of a kind of ‘card-swipe surveillance’ program, as in fact has been done by Matthew S. Pittinksy (co-founder of Blackboard) at Arizona State University. According to Pittinsky, tracking card-swipe behavior can allow an institution to effectively map a student’s friend group, determine their level of social integration, and predict their chances of attrition.)