Predictive analytics are not social science: A common misunderstanding with major consequences for higher education

This is the second in my series on common misunderstandings about predictive analytics that hinder their adoption in higher education. Last week I talked about the language of predictive analytics. This week, I want to comment on another common misconception: that predictive analytics (and educational data mining more generally) is a social science. Continue reading

Traversing the Trough of Disillusionment: Where do Analytics Go from Here?

Co-Authored with Mike Sharkey

Last month we argued that analytics in higher education has entered a trough of disillusionment.  We posited that this is actually a good thing for higher education, because it means bringing attention to the hype itself. It means that we are making progress towards true productivity and student success.  We need to learn how to spot the hype before we can move beyond it and realize the true potential of educational data and learning analytics.

It is our hope that the ‘analytics angst’ that has accompanied increased data literacy will put pressure on vendors to reduce hyperbole in their marketing materials and encourage institutions to reset their expectations. A more realistic view of educational data will result in greater adoption, more successful implementations, and results that move the needle by positively impacting student success at scale.


Three Ways Higher Ed can Avoid IT ‘Lock-In’

In a recent Future Trends Forum discussion with Bryan Alexander, George Siemens expressed concern about lock-in: a situation in which technology investments become so integrated with the business practices of an institution that disentanglement becomes all but impossible. Where hyper-rationalized approaches to data-driven decision-making come together with inflexible technological ecosystems characterized by a lack of interoperability, what we end up with is a dystopian future in which colleges and universities are unable to change their investments.


How to plan a conference that doesn’t suck

recent article, Kristen Eshleman and Josh Kim explore five reasons for why educational technology conferences suck (my word, not theirs) and, consequently, five ways to make conferences better.  In their view, organizers too easily forget that a conference is by/for/about people.  Conferences are often not planned with a view to enabling practice and overcoming silos.  They are frequently over-hyped and create too little value.

I agree.

The second annual Southeast Educational Data Symposium (SEEDS16) has been guided by what I am going to call the Golden Rule of Conference Panning: “Don’t plan a conference that you wouldn’t want to attend.”  To my mind, you should only plan a conference or event if it is actually going to provide significant value to participants.  A conference takes a lot of effort to plan and, all things being equal, it would be much easier for everyone involved if it didn’t exist.  The bar here is pretty high.  Another way or articulating the golden rule of conference planning, perhaps, is to say “Don’t plan a conference unless it is going to be better than having no conference at all.”  Not planning or going to a conference is pretty great, in my opinion.  Any conference I organize has to be better than that.  As I have worked with an incredible organizing committee to put together that I think will be a valuable and successful event (fingers crossed!), several guiding principles have emerged that are direct consequences of the golden rule:

  1. Start with Goals – too often, conferences are organized with the goal of organizing a conference.  The problem with deciding to organize a conference is that you are likely to end up with one.  If you set out to organize a conference, it will be modeled on what you think a conference looks like.  Since most conferences suck, yours will too.  Instead of setting out to create something that looks like a conference, begin with a specific set of goals.  Once you have those goals in mind, decide whether a conference is the right way to achieve them in the first place.  If so, then design the conference strategically in order to achieve those goals.  What you end up with may look pretty ‘unconferency.’  It might not.
  2. Support the network – Bars don’t sell alcohol.  They sell the promise of human connection.  Too often, conference organizers think that the value of their event comes from content.  They worry about whether a particular keynote is going to draw a crowd, and if particular events are going to entice enough people.  But there is very little in the way of conference content that I cannot access in other ways.  Conferences don’t sell knowledge.  They sell the promise of human connection.  What draws people to a conference is the promise of entering into a community of like-interested people, of forming relationships, of developing opportunities for collaboration, and of being excited to pursue new projects.
  3. Create Value – Regardless of group size, the strongest relationships are formed as a result of a shared vision, and of movement toward a common goal.  Picture two people with eyes fixed upon a common point on the horizon.  It doesn’t take much knowledge of geometry to realize that the closer two people get to that common point, the closer they will get to each other.  If the goal of a conference is to encourage human connection, the best way to do that is to create opportunities for people to work together to solve shared problems.  It’s one thing to have your brain tickled.  It’s another thing entirely to walk away from a conference with a tangible solution (ideally an artifact).  This emphasis on practice can be seen in the afternoon workshops at SEEDS16 (inspired by the format of the Learning Analytics Summer Institute organized each year by the Society of Learning Analytics Research).  Our workshop on practical learning analytics will give participants the opportunity to acquire new skills as they collaborate to answer really questions using real student data.  Our workshop on ethics will lead participants to develop codes of practice for learning analytics that will guide their own efforts, and hopefully make an impact at their home institutions.
  4. More isn’t better – One of the most common pieces of feedback from SEEDS15 was that participants were energized at the end of the day.  They still had energy, and they wanted more of a good thing.  Wanting to be responsive to this feedback, our original plans for SEEDS16 included extending the event over two days.  But looking at a draft schedule, I couldn’t help but think that the two-day conference was no longer something that I wanted to attend.  How many times have you been to an event and felt energized at the end of day one, only to feel ‘over it’ at the end of day two?  More of a good thing isn’t always better.  By retaining the one-day format for SEEDS16, our goal is to leave on a high note, and to leave participants with the energy they need to carry ideas and practices back to their home institutions.
  5. Relatedly, bigger isn’t better.  Group size matters. One of the primary goals of the Southeast Educational Data Symposium is to foster a strong sense of community around the effective use of educational data in the southeast region of the US.  With this aim in mind, SEEDS15 had a hard cap of 50 participants.  The result was magical.  The conference drew administrators, faculty, researchers, and graduate students from 25 institutions.  Name tags did not include information about title or rank, and so an environment was fostered in which all entered as equal partners in a shared conversation.  But the event had a waiting list, and our hard cap meant that many who wanted to attend the conference could not.  In order to meet demand this year we increased the cap to 100.  Hopefully an increase in scale won’t spell a decrease in ‘magic.’  Scale is tricky.
  6. Lastly, more complex isn’t better.  Another piece of feedback that we received out of last year’s event was that not all presentations were of interest or relevant to everyone, and that the conference would benefit from having multiple ‘tracks.’  Wanting to be responsive, early plans for the conference included plans to have a leadership track and a practitioner track.  But we soon changed our minds.  Regardless of what one or two past participants might have said, the organizing committee felt that part of the magic of last year came about as a result of the fact that it did not respect disciplinary or hierarchical silos.  That everyone participated in a common conversation about a shared set of material was powerful.  In order to achieve small-scale intimacy despite an increase in the number of participants, we have organized simultaneous sessions.  Life is about trade-offs.  With an increase in choice and intimacy comes a decrease in shared experience.

It would be far easier NOT to plan a conference.  Truth be told, conference planning is not my favorite thing in the world to do.  What drives me and other organizers, however, is a commitment to serving our community, and to meeting a clear need – the need to better understand how to put educational data into practice.  As Aristotle famously observed, the best leader is one who leads from a desire, not for power, but out of duty and a sense of commitment to the good of their community.  If the same guiding principle was applied to conferences, there would be far fewer, and they would suck a whole lot less.