Five Must-See Analytics Sessions at EDUCAUSE 2016

EDUCAUSE is big.  Really big. With so much to take in, conference-goers (myself included) are easily faced with the paradox of choice: a sense of paralysis in the face of too many options.  To help myself and others, I have scanned this year’s conference agenda and selected five presentations that I think will be individually strong, and that as a group offer a good overview of the themes, issues, and state of analytics in higher education today.

Learning at Scale with Analytics: Findings from the Field and Open Questions

Wednesday, October 26, 2016 | 3:40 PM – 4:30 PM | Meeting Room 204B

Moderated by Michael Feldstein (e-Literate), and featuring John Whitmer (Blackboard), Russ Little (PAR), and Jeff Gold (California State University), and Avi Yashchin (IBM), this session promises to provide an engaging and insightful overview of why analytics are important for higher education, the biggest challenges currently facing the field, and opportunities for the future.  Although most of the speakers are strongly affiliated with vendors in the analytics space, they are strong data scientists in their own right and have demonstrated time and time again that they do not shy from critical honesty.  Attend this session for a raw glimpse into what analytics mean for higher education today.

Deploying Open Learning Analytics at National Scale: Lessons from the Real World

Thursday, October 27 | 8:00am – 8:50am | Ballroom A, Level Three

Jisc is a non-profit company that aims to create and maintain a set of shared services in support of higher education in the UK.  The Effective Learning Analytics project that Michael Webb will discuss in this session has aimed to provide a centralized learning analytics solution in addition to a library of shared resources.  The outputs of this project to date have valuable resources  to the international educational analytics community in general, including Code of practice for learning analytics and Learning Analytics in Higher Education.  Jisc’s work is being watched carefully by governments and non-governmental organizations worldwide and represents an approach that we may wish to consider emulating in the US (current laws notwithstanding).  Attend this session to learn about the costs and opportunities involved in the development of a centralized approach to collecting and distributing educational data.

Founding a Data Democracy: How Ivy Tech is Leading a Revolution in Higher Education

Thursday, October 27 | 1:30pm – 2:20pm | Meeting Room 202A/B, Level Two

The higher education community is abuzz with talk of how data and analytics can improve student success.  But data and analytics are worthless unless they are put in the hands of the right people and in the right ways.  I am really interested to see how Ivy Tech has worked to successfully democratize access to information, and also about the ways that access to data has driven the kind of institutional and cultural change necessary to see the most significant results from data-driven initiatives.

Analytics and Student Success: Research and Benchmarking

Thursday, October 27 | 8:00am – 8:50am | Meeting Room 304A/B, Level Three

Everyone’s talking about analytics, and every institution seemingly has the will to invest.  Attention paid to analytics in media and by vendors can lead to the impression that everybody’s doing it, and that everyone who’s doing it is seeing great results.  But the truth is far from the case.

I’m not the greatest fan of benchmarking in general.  Too often, benchmarking is productized by vendors and sold to universities despite providing very little actionable value.  Worse yet, they can exacerbate feelings of institutional insecurity and drive imprudent investments.  But when it comes to analytics, benchmarking done right can provide important evidence to counteract misperceptions about the general state of analytics in the US, and provide institutions with valuable information to inform prudent investment, planning, and policy decisions.  In this presentation, I look forward to hearing Christopher Brooks and Jeffery Pomerantz from EDUCAUSE discuss their work on the analytics and student success benchmarking tools.

Building with LEGOs: Leveraging Open Standards for Learning Analytics Data

Friday, October 28 | 8:00am – 8:50am | Meeting Room 304C/D, Level Three

I am a huge advocate of open standards in learning analytics.  Open standards mean greater amounts of higher quality data.  They mean that vendors and data scientists can spend more time innovating and less time just trying to get plumbing to work.  In this interactive presentation, Malcolm Brown (EDUCAUSE), Jenn Stringer (University of California, Berkeley), Sean DeMonner (University of Michigan-Ann Arbor), and Virginia Lacefield (University of Kentucky) talk about how open learning standards like IMS Caliper and xAPI are creating the foundation for the emergence of next generation learning environments.

Vlogging my way through BbWorld16

EPISODE I: Going to Vegas

Headed to Las Vegas for DevCon and BbWorld 2016. Having attended twice before as a customer, I am very excited to have played a part in organizing this year’s event.

In this vlog episode, I check in with Scott Hurrey (Code Poet at Blackboard) and ask him about what excites him the most about DevCon. Dan Rinzel (Product Manager, Blackboard Analytics) and John Whitmer (Director of Analytics and Research at Blackboard) tackle some extreme food portions.

EPISODE II: Teamwork makes the Dream Work

A day of rehearsal for the BbWorld16 opening general session leads to an air of playful excitement in anticipation of the main event. ‘Dr John’ talks about why data science isn’t scary, and why everyone should be interested and involved.

EPISODE III: Making Magic Happen

Want to go behind the scenes and get a sense of all of the work that goes into the opening main stage keynote presentation each year? Michelle Williams takes us on a tour!

EPISODE IV: Yoga and Analytics

Meet the Predictive Analytics ‘booth babes,’ learn from Michael Berman that yoga and analytics DO mix. Executive Director of the University Innovation Alliance, Bridget Burns, explains why there is a need for more empathy between institutions of higher educations and educational technology companies, and in higher education in general.

EPISODE V: We Are Family

Rachel Seranno from Appalachian State University talks about power poses and memes. Eric Silva praises the power of Twitter. Casey Nugent and Shelley White from the University of Nebraska – Lincoln describe how they are working with Blackboard consultants to understand and optimize instruction.

Voting is Open for #SXSWedu 2016!

Experts from the University Innovation Alliance‘s eleven member universities are among the educational rock stars featured on panels proposed for next year’s SXSWedu. Of the panels proposed for the 2016 event, five are noteworthy as highlighting the innovative work that the UIA’s member institutions have been engaged in over the past year.

1. How Universities are Crowdsourcing Innovation

How do you get eleven major public research universities, known for fiercely competing against one another in sport and in national rankings, to work together? How do you create a space where these institutions are encouraged to share in successes and failures in the name of testing and deploying proven ideas at scale? Arizona State University President Michael Crow, University of Kansas Chancellor Bernadette Gray-Little, and University of Texas at Austin President Gregory Fenves are listed on a panel alongside Paul Fain from Inside Higher Ed to explain the benefits of cooperation and to share the results that they have seen since joining together in pursuit of a common vision for Higher Education in America.

2. Putting Big Data into Action

For years now, ‘big data’ has been touted as a game changer in higher education. For years, prognosticators have been talking about the wide-spread adoption of learning and academic analytics as something that was just over the horizon. The future is now.

Georgia State University’s GPS advising system generates over 800 different kind of alerts which, in support of an intensive approach to student advising, has quickly seen a dramatic increase in student success including the total elimination of all achievement gaps on the basis of race, ethnicity, and income-level. GSU’s Vice Provost and Vice President of Enrollment Management and Student Success is slated to participate on a panel with Candace Thille from Stanford University’s Open Learning Initiative, Mark Milliron from Civitas Learning, and Greg Ratcliff from the Bill and Melinda Gates Foundation. Each of these panelists is widely acknowledged as a thought leader in the effective use of educational data, and is in high demand as an engaging and motivational speaker in their own right. The opportunity to see all four of them on a single stage is not one to be passed by.

3. Show Me the Money: Driving Fiscal Sustainability

The near-closure of Sweet Briar College this year in the face of economic challenges has served as a wake up call. With nearly one in five business officers saying that their institutions are likely to shut down in the coming decades, the financial health of universities is something that none of us can ignore. Maria Anguiano, University of California, Riverside’s Vice Chancellor for Planning and Budget, is among the speakers identified on a diverse panel proposal that will address controversial issues like activity-based costing, and discuss new models for thinking through financial decision-making in higher education.

4. Time to Stop, Collaborate, and Listen

Collaboration is hard to do. Executive Director the the University Innovation Alliance, Bridget Burns, is the new kid on the block in a conversation about the challenges facing any new collaboration. What are some effective strategies for developing effective national and regional collaboratives? How can higher education and the educational technology sector work together to magnify impact? Going it alone has gotten universities as far as it can. Competition is passe. The time for collaboration is now.

5. From Analytics to Action

Educators now have unparalleled access to student data, which promises to radically improve their success in the classroom. But access to data is not access to information. What kinds of data are available? What tools and techniques are there for processing those data? What kinds of problems can we use data to address? What are the best strategies for intervention? without good answers to these questions, our educational data is useless, or even dangerous.

Phillip Long is the Associate Vice Provost for Learning Sciences and Deputy Director of the Center for Teaching and Learning at the University of Texas at Austin. A pioneer in the emerging field of learning analytics, Phil is on a panel alongside entrepreneurs and instructional designers to discuss some specific ways that instructors are leveraging educational data to gain a richer understanding of their students. By carefully reading students’ digital traces, teachers are being empowered to improve student outcomes while rethinking what student success looks like in the twenty-first century.

The aim of SXSWedu is to create a platform to promote creativity and social change. Please take a moment to create an account on the panel picker website (http://panelpicker.sxsw.com/) and cast your vote in support of these proposals, as well as others that you feel will improve the quality of experience for participants, and that are most likely to make a significant and enduring impact.

Georgia State University Vice Provost Testifies at Senate Hearing on Higher Education

Photo Credit: U.S. Senate Committee on Health, Education, Labor & Pensions
Photo Credit: U.S. Senate Committee on Health, Education, Labor & Pensions
On August 5, 2015, Dr. Timothy M. Renick, Georgia State University Vice Provost and Vice President of Enrollment Management and Student Success, was invited to give testimony before the U.S. Senate, as part of a series of hearings meant to inform decisions about the reauthorization of the Higher Education Act.

During his oral testimony, Dr. Renick described how, under the leadership of President Mark Becker, Georgia State University had made a commitment to developing a model that would see students from all backgrounds succeed at high rates. Through programs like Panther Retention Grants — an initiative that distributes one-time micro grants to students who are on track for graduation, but who have small amounts of unmet need that present obstacles to progression — and GPS Advising — a system that uses predictive analytics to identify students at risk and that provides advisors with the intelligence necessary to get those students back on track — Georgia State University has seen a 22 point increase in graduation rates over the last ten years, and a complete elimination of all achievement gaps on the basis on race, ethnicity, and economics.

Continue reading

Tools for Evidence-Based Action (TEA) Meeting – Day Two

The second day of the Tools for Evidence-Based Action (TEA) Meeting opened with a continuation discussion from yesterday, about features and feature requests for the two tools introduced yesterday: the Ribbon Flow Tool and the General Observation and Reflection Protocol (GORP) Tool.

Feature requests for the Ribbon Flow Tool included adding more stops and a csv to json convertor, but also the ability to customize color-codes and tool-tips. Several security concerns were raised in discussion yesterday, which developer Matt Steinwachs resolved by the morning. Nevertheless, general consensus was that comprehensive documentation is necessary in order to gain the institutional approvals necessary before hosting any kind of student data, aggregate or otherwise, on an external server.

bored

As far as the GORP tool is concerned, Matt Steinwachs presented a kind of development roadmap that would see the app implemented in a way that was fully customizable, allowing users to record observations according to any observation protocol or variation. In future, users will be able to fully customize the buttons presented (including text, images, colors, and position). Also customizable will be time frame (i.e. interval or real time logging). This is clearly going to be a powerful and flexible tool for recording classroom observations, but what really excites me is its potential as an active learning tool. I imagine, for example, customizing the app for use by students so that they can record affective and/or learning states during the course of a class period. The simplest example of this would be to strip down the app to include only a single ‘bored’ button and asking students to push it when they lose interest during a class period. Data collected from a course (especially from a large course) could produce a heatmap on a lecture recording, in order to associate classroom activity patterns with lags in student engagement.

In the afternoon, Mary Hueber and Pat Hutchings, Evaluators for the Bay View Alliance, talked about the organization’s latest Research Action Cluster (RAC): Using Academic Analytics to Support and Catalyze Transformation. Led by Marco Molinaro and Chris Pagliarulo (UC Davis), the project will produce three case studies about three institutions at different stages in their learning analytics program implementations. The emphasis of these case studies will be on data-driven intervention effectiveness. UC Davis has been selected as the first institution to be evaluated. The others (which are likely to include one other school from the Bay View Alliance, and one from outside) will be selected in the coming months.

The day concluded with discussions of how the TEA community might be sustained and grow, and how participants might bring insights back to their home institutions and promote tool adoption. The Helmsley Charitable Trust has committed to funding another TEA meeting in a year. I look forward to continuing to inform the development of the UC Davis learning analytics toolkit, and to continuing my active involvement in this initiative. Big thanks are due to Marco Molinaro, Chris Pagliarulo, Alan Gamage and Margie Barr for their hard work in organizing this event, and catalyzing this community of interest in approaches to evidence-based action.

Tools for Evidence-Based Action (TEA) Meeting – Day One

Today was the first day of the Tools for Evidence-Based Action (TEA) Meeting, hosted by the UC Davis iAMSTEM Hub. Organized by Marco Molinaro (Assistant Vice Provost for Undergraduate Education, UC Davis), with assistance from Alan Gamage and Margie Barr, the TEA meeting is part of an effort supported by the Helmsley Trust to disseminate new tools and methodologies for classroom observations and visualizing student pathways.

Day one introduced the group of invited participants (representing colleges and universities from all over the United States) to three primary tools: Trellis, a Ribbon Tool, and a GORP Classroom Observation Tool.

Trellis Social Networking and Collaboration Platform

Trellis is a social network in development by the American Association for the Advancement of Science (AAAS). Trellis is still in development, but is currently being beta-tested among a number of communities, including TEA. When it goes live, Trellis is meant to provide a multidisciplinary space for anyone in science to engage in the communities to which they belong.

Trellis

As a social network, Trellis has a lot of potential. The UX is clean and, supported by AAAS, is uncluttered by advertisements. It is also free for scientific communities that choose to use it. A colleague observed that Trellis might function as a ‘Facebook for nerds,’ but only if all of the organizations and societies to which we belong were to choose a common social network. Short of this, Trellis is simply another social network among social networks.

In addition to its basic social networking functionality, Trellis also promises to provide a rich set of collaboration tools, to simplify document preparation, annotation, and sharing. In all this, however, it continues to be unclear to me what is supposed to distinguish Trellis from other social networks and collaboration platforms that are already such a central part of most of our lives (especially LinkedIn and Google+). Most importantly, however, I hesitate to fully support a new network in the absence of a clear revenue model, and without a firm commitment to data privacy.

Ribbon Tool

The cloud-hosted (via Amazon Web Services) Ribbon FLOW Tool that is being developed at UC Davis by Matt Steinwachs filters and visualizes change in group membership over time. In the demonstration, the tool was used to visualize change in declared major upon admission compared to graduation. Accepting data in json format, the tool is flexible enough to visualize any kind of data flow.

UC Davis Ribbon FLOW Tool

The tool is still in early development, but in future releases and before being made available more widely, Matt has promised to add the following

  • The ability to upload/download data in csv format (a must for usability among non-technical users)
  • Multiple stops, so that changes in group state that take place between start and end can be represented
  • Comprehensive documentation (including security model and privacy policy)

Once completed, the goal is to make the tool open source, and I would love to see it used by the Learning Analytics Initiative or as part of some other similar open learning analytics toolkit.

General Observation and Reflection Tool (GORP)

The General Observation and Reflection Tool (GORP) classroom observation tool is being developed by Matt Steinwachs for Chris Pagliarulo (Director of Instruction & Assessment, iAMSTEM, UC Davis). The web-based tool facilitates data entry for ‘agents’ (student coders) observing classes according to the Classroom Observation Protocol for Undergraduate STEM (COPUS). The tool seeks to solve the problem of having to manually enter data collected by coders at two minute intervals, by automating the collection and storage process.

GORP Classroom Observation Tool

BYOD and BYOT

UCDavis_TEA
The day concluded with a “Bring Your Own Data and Tools” session, which provided TEA Meeting participants with an opportunity to share tools that they had developed and/or had been using with great success. A few highlights include:

  • Marilyne Stains (University of Nebraska-Lincoln) presented the results of work that she and her co-investigators have been doing with the Classroom Observation Protocol for Undergraduate STEM (COPUS). Using the GORP tool, she has been able to identify coding redundancies (ex. Students listening’ and ‘Instructor Lecturing’ represent the same classroom state). A cluster analysis has identified several basic class types.
  • Andrew Feig (Wayne State University) demonstrate the use of Circos to represent individual student course pathways from semester to semester.
  • Erin Soloman (Washington University in St. Louis) presented a flexible classroom observation protocol (OPAL) and iOS app that records and timestamps every coding event. The advantage of this approach over others (which aggregate all events at 2 minute intervals) is that it allows educational analysts to correlate spikes in student engagement with specific moments in a recorded lecture.

Rethinking Student Success: Analytics in Support of Teaching and Learning

Presented at the 2014 Blackboard Institutional Performance Conference (30 – 31 October 2014)

Passing grades and retention through to degree are essential to success in higher education, but these factors are too often mistaken for ends in themselves. A high-performing student environment has provided teachers and researchers at Emory University with a space to think critically about what success means, and about the extent to which data might inform the design of successful learning environments. This presentation will (1) discuss some of the unique challenges encountered by Emory University during its 2013-2014 Blackboard Analytics pilot, (2) describe several provisional insights gained from exploratory data mining, and (3) outline how Emory’s pilot experience has informed support of learning analytics on campus.