Tools for Evidence-Based Action (TEA) Meeting – Day Two

The second day of the Tools for Evidence-Based Action (TEA) Meeting opened with a continuation discussion from yesterday, about features and feature requests for the two tools introduced yesterday: the Ribbon Flow Tool and the General Observation and Reflection Protocol (GORP) Tool.

Feature requests for the Ribbon Flow Tool included adding more stops and a csv to json convertor, but also the ability to customize color-codes and tool-tips. Several security concerns were raised in discussion yesterday, which developer Matt Steinwachs resolved by the morning. Nevertheless, general consensus was that comprehensive documentation is necessary in order to gain the institutional approvals necessary before hosting any kind of student data, aggregate or otherwise, on an external server.


As far as the GORP tool is concerned, Matt Steinwachs presented a kind of development roadmap that would see the app implemented in a way that was fully customizable, allowing users to record observations according to any observation protocol or variation. In future, users will be able to fully customize the buttons presented (including text, images, colors, and position). Also customizable will be time frame (i.e. interval or real time logging). This is clearly going to be a powerful and flexible tool for recording classroom observations, but what really excites me is its potential as an active learning tool. I imagine, for example, customizing the app for use by students so that they can record affective and/or learning states during the course of a class period. The simplest example of this would be to strip down the app to include only a single ‘bored’ button and asking students to push it when they lose interest during a class period. Data collected from a course (especially from a large course) could produce a heatmap on a lecture recording, in order to associate classroom activity patterns with lags in student engagement.

In the afternoon, Mary Hueber and Pat Hutchings, Evaluators for the Bay View Alliance, talked about the organization’s latest Research Action Cluster (RAC): Using Academic Analytics to Support and Catalyze Transformation. Led by Marco Molinaro and Chris Pagliarulo (UC Davis), the project will produce three case studies about three institutions at different stages in their learning analytics program implementations. The emphasis of these case studies will be on data-driven intervention effectiveness. UC Davis has been selected as the first institution to be evaluated. The others (which are likely to include one other school from the Bay View Alliance, and one from outside) will be selected in the coming months.

The day concluded with discussions of how the TEA community might be sustained and grow, and how participants might bring insights back to their home institutions and promote tool adoption. The Helmsley Charitable Trust has committed to funding another TEA meeting in a year. I look forward to continuing to inform the development of the UC Davis learning analytics toolkit, and to continuing my active involvement in this initiative. Big thanks are due to Marco Molinaro, Chris Pagliarulo, Alan Gamage and Margie Barr for their hard work in organizing this event, and catalyzing this community of interest in approaches to evidence-based action.