This is the second in my series on common misunderstandings about predictive analytics that hinder their adoption in higher education. Last week I talked about the language of predictive analytics. This week, I want to comment on another common misconception: that predictive analytics (and educational data mining more generally) is a social science. Continue reading
In a recent interview with John Jantsch for the Duct Tape Marketing podcast, Danny Iny argued that the difference between information and education essentially comes down to responsibility. Information is simply about presentation. Here are some things you might want to know. Whether and the extent to which you come to know them is entirely up to you.
In contrast, education implies that the one presenting information also takes on a degree of responsibility for ensuring that it is learned. Education is a relationship in which teachers and learners agree to share in the responsibility for the success of the learning experience.
This distinction, argues Iny, accounts for why books are so cheep and university is so expensive. Books merely present information, while universities take on an non-trivial amount of responsibility for what is learned, and how well.
(It is a shame that many teachers don’t appreciate this distinction, and their role as educators. I will admit that, when I was teaching, I didn’t fully grasp the extent of my responsibility for the success of my students. I wish I could go back and reteach those courses as an educator instead of as a mere informer.)
If we accept Iny’s distinction between information and education, what are the implications for what we today call educational technologies, or ‘Ed Tech’? As we look to the future of technology designed to meet specific needs of teachers and learners, is educational technology something that we wish to aspire to, or avoid?
Accepting Iny’s definition, I would contend that what we call educational technologies today are not really educational technologies at all. The reason is that neither they nor the vendors that maintain them take specific responsibility for the success or failure of the individual students they touch. Although vendors are quick to take credit for increased rates of student success, taking credit is not the same as taking responsibility. In higher education, the contract is between the student and the institution. If the student does not succeed, responsibility is shared between the two. No technology or ed tech vendor wants to be held accountable for the success of an individual student. In the absence of such a willingness or desire to accept a significant degree of responsibility for the success of particular individuals, what we have are not educational technologies, but rather information technologies designed for use in educational contexts. Like books…but more expensive.
With the advent of AI, however, we are beginning to see an increasing shift as technologies appear to take more and more responsibility for the learning process itself. Adaptive tutoring. Automated nudging. These approaches are designed to do more than present information. Instead, they are designed to promote learning itself. Should we consider these educational technologies? I think so. And yet they are not treated as such, because vendors in these areas are still unwilling (accountability is tricky) or unable (because of resistance from government and institutions) to accept responsibility for individual student outcomes. There is no culpability. That’s what teachers are for. In the absence of a willingness to carry the burden of responsibility for a student’s success, even these sophisticated approaches are still treated as information technologies, when they should actually be considered far more seriously.
As we look to the future, it does seem possible that the information technology platforms deployed in the context of education will, indeed, increasingly become and be considered full educational technologies. But this can only happen if vendors are willing to accept the kind of responsibility that comes with such a designation, and teachers are willing to share responsibility with technologies capable of automating them out of a job. This possible future state of educational technology may or may not be inevitable. It also may or may not be desirable.
RESOURCES
In analytics circles, it is common to quote Peter Drucker: “What gets measured get managed.” By quantifying our activities, it becomes possible to measure the impact of decisions on important outcomes, and optimize processes with a view to continual improvement. With analytics, there comes a tremendous opportunity to make evidence-based decisions where before there was only anecdote.
But there is a flip side to all this. Where measurement and management go hand in hand, the measurable can easily limit the kinds of things we think of as important. Indeed, this is what we have seen in recent years around the term ‘student success.’ As institutions have gained more access to their own institutional data, they have gained tremendous insight into the factors contributing to things like graduation and retention rates. Graduation and retention rates are easy to measure, because they don’t require access to data outside of institutions, and so retention and graduation have become the de facto metrics for student success. Because colleges and universities can easily report on these things, they are also easy to incorporate into rankings of educational quality, accreditation standards, and government statistics.
But are institutional retention and graduation rates actually the best measures of student success? Or are they simply the most expedient given limitations on data collection standards? What if we had greater visibility into how students flowed into and out of institutions? What if we could reward institutions for effectively preparing their students for success at other institutions despite a failure to retain high numbers through to graduation? In many ways, limited data access between institutions has led to conceptions of student success and a system of incentives that foster competition rather than cooperation, and may in fact create obstacles to the success of non-traditional students. These are the kind of questions that have recently motivated a bipartisan group of senators to introduce a bill that would lift a ban on the federal collection of employment and graduation outcomes data.
More than 98% of US institutions provide data and have access to the National Student Clearinghouse. For years, the National Student Clearinghouse (NSC) has provided a rich source of information about the flow of students between institutions in the U.S., but colleges and universities often struggle with making this information available for easy analysis. Institutions see the greatest benefit from access to NSC data when they combine it with other institutional data sources, and especially demographic and performance information stored in their student information systems. This kind of integration is helpful, not only for understanding and mitigating barriers to enrollment and progression, but also as institutions work together to understand the kinds of data that are important to them. As argued in a recent article in Politico, external rating systems have a significant impact on setting institutional priorities and, in so doing, may have the effect of promoting systematic inequity on the basis of class and other factors. As we see at places like Georgia State University, the more data that an institution has at their disposal, and the more power it has to combine multiple data sources the more it can align its measurement practices with its own values, and do what’s best for its students.
In March 2017, Manuela Ekowo and Iris Palmer co-authored a report for New America that offered five guiding practices for the ethical use of predictive analytics in higher education. This kind of work is really important. It acknowledges that, to the extent that analytics in higher education is meant to have an impact on human behavior, it it is a fundamentally ethical enterprise.
Work like the recent New America report is not merely about educational data science. It is an important facet of educational data science itself.
But ethics is hard. Ethics is not about generating a list of commandments. It is not about cataloging common opinion. It is about carefully establishing a set of principles on the basis of which it becomes possible to create a coherent system of knowledge and make consistent judgements in specific situations.
Unfortunately, most work on the ethics of analytics in higher education lacks this kind of rigor. Instead, ethical frameworks are the result of a process of pooling opinions in such a way as to strike a balance between the needs of a large number of stakeholders including students, institutions, the economy, the law, and public opinion. To call this approach ethics is to confuse the good with the expedient.
An ethical system worthy of the name needs to begin with a strong conception of the Good. Whether stated or implied, the most common paradigm is essentially utilitarian, concerned with maximizing benefit for the greatest number of people. The problem with this approach, however, is that it can only ever concern perceived benefit. People are famously bad at knowing what is good for them.
A benefit of this utilitarian approach, of course, is that it allows us to avoid huge epistemological and metaphysical minefields. In the absence of true knowledge of the good, we can lean on the wisdom of crowds. By pooling information about perceived utility, so the theory goes, we can approximate something like the good, or at least achieve enough consensus to mitigate conflict as much as possible.
But what if we were more audacious? What if our starting point was not the pragmatic desire to reduce conflict, but rather an interest in fostering the fullest expression of our potential as humans? As it specifically pertains to the domain of educational data analytics, what if we abandoned that instrumental view of student success as degree completion? What if we began with the question of what it means to be human, and wrestled with the ways in which the role of ‘student’ is compatible and incompatible with that humanity?
Let’s consider one example of how taking human nature seriously affects how we think about analytics technologies. As the Italian humanist Pier Paolo Vergerio observed, all education is auto-didactic. When we think about teaching and learning, the teacher has zero ability to confer knowledge. It is always the learner’s task to acquire it. True, it is possible to train humans just as we can train all manner of other creatures (operant and classical forms of conditioning are incredibly powerful). but this is not education. Education is a uniquely human capability whereby we acquire knowledge (with the aim of living life in accord with the Good). Teachers do not educate. Teachers do not ‘teach.’ Rather, it is the goal of the teacher to establish the context in which the student might become actively engaged as learners.
What does this mean for Education? Viewed from this perspective, it is incumbent on us as educators to create contexts that bring students to an awareness of themselves as learners in the fullest sense of the word. It is crucial that we develop technologies that highlight the student’s role as autodidact. Our technologies need to help bring students to self-knowledge at the same time as they create robust contexts for knowledge acquisition (in addition to providing opportunities for exploration, discovery, experimentation, imagination and other humane attributes).
It is in large part this humanistic perspective that has informed my excitement about student-facing dashboards. As folks like John Fritz have talked about, one of the great things about putting data in the hands of students is that it furthers institutional goals like graduation and retention as a function of promoting personal responsibility and self-regulated learning. In other words, by using analytics first and foremost with an interest in helping students to understand and embrace themselves as learners in the fullest sense of the term, we cultivate virtues that translate into degree completion, but also career success and life satisfaction.
In my opinion, analytics (predictive or otherwise) are most powerful when employed with a view to maximizing self-knowledge and the fullest expression of human capability rather than as way to constrain human behavior to achieve institutional goals. I am confident that such a virtuous and humanistic approach to educational data analytics will also lead to institutional gains (as indeed we have seen at places like Georgia State University), but worry that where values and technologies are not aligned, both human nature and institutional outcomes are bound to suffer.
The Latin word communitas refers to a collection of individuals who, motivated by a common goal, come together and act as one. Community is powerful.
Common approaches to college and university rankings can sometimes have the unfortunate effect of pitting institutions against each other in a battle for students and prestige. As the U.S. turns its attention to meeting the needs of 21st century students and 21st century labor demands, the power of traditional university ranking schemes is starting to erode.
Student success is not a zero-sum game. Rather than fostering competition, a commitment to student success encourages cooperation.
READ FULL STORY HERE >> http://blog.blackboard.com/fostering-community-through-a-common-language/
Co-Authoered with Mike Sharkey
In direct contradiction to Betteridge’s Law, we believe the answer is yes. Analytics in higher education is in the trough of disillusionment.
The trough of disillusionment refers to a specific stage of Gartner’s Hype Cycle. It is that moment when, after a rapid build up leading to a peak of inflated expectations, a technology’s failure to achieve all that was hoped for results in disillusionment. Those who might benefit from a tool perceive a gap between the hype and actual results. Some have rightly pointed out that not all technologies follow the hype cycle, but we believe that analytics in higher education has followed this pattern fairly closely.
READ FULL STORY >> http://er.educause.edu/blogs/2016/11/has-analytics-fallen-into-the-trough-of-disillusionment
In 2014, I wrote a blog post in which I claimed (along with others) that analytics had reached a ‘peak of inflated expectations.’ Is the use of analytics in higher education now entering what Gartner would call the ‘trough of disillusionment’?
In 2011, Long and Siemens famously argued that big data and analytics represented “the most dramatic factor shaping the future of higher education.” Since that time, the annual NMC Horizon Report has looked forward to the year 2016 as the year when we would see widespread adoption of learning analytics in higher education. But as 2016 comes to a close, the widespread adoption of learning analytics still lies on the distant horizon. Colleges and universities are still very much in their infancy when it comes to the effective use of educational data. In fact, poor implementations and uncertain ROI have led to what Kenneth C. Green has termed ‘angst about analytics.’
As a methodology, the Gartner Hype Cycle is not without criticism. Audrey Watters, for example, takes issue with the fact that it is proprietary and so ‘hidden from scrutiny.’ Any proprietary methodology is in fact difficult to take seriously as a methodology. It should also be noted that the methodology is also improperly named, as any methodology that assumes a particular outcome (i.e. that assumes that all technology adoption trends follow the same patters) is unworthy of the term. But as a heuristic or helpful model, it is helpful way of visualizing analytics adoption in higher education to date, and it offers some helpful language for describing the state of the field. Continue reading
Recent Posts On Medium