Tools for Evidence-Based Action (TEA) Meeting – Day One

Today was the first day of the Tools for Evidence-Based Action (TEA) Meeting, hosted by the UC Davis iAMSTEM Hub. Organized by Marco Molinaro (Assistant Vice Provost for Undergraduate Education, UC Davis), with assistance from Alan Gamage and Margie Barr, the TEA meeting is part of an effort supported by the Helmsley Trust to disseminate new tools and methodologies for classroom observations and visualizing student pathways.

Day one introduced the group of invited participants (representing colleges and universities from all over the United States) to three primary tools: Trellis, a Ribbon Tool, and a GORP Classroom Observation Tool.

Trellis Social Networking and Collaboration Platform

Trellis is a social network in development by the American Association for the Advancement of Science (AAAS). Trellis is still in development, but is currently being beta-tested among a number of communities, including TEA. When it goes live, Trellis is meant to provide a multidisciplinary space for anyone in science to engage in the communities to which they belong.

Trellis

As a social network, Trellis has a lot of potential. The UX is clean and, supported by AAAS, is uncluttered by advertisements. It is also free for scientific communities that choose to use it. A colleague observed that Trellis might function as a ‘Facebook for nerds,’ but only if all of the organizations and societies to which we belong were to choose a common social network. Short of this, Trellis is simply another social network among social networks.

In addition to its basic social networking functionality, Trellis also promises to provide a rich set of collaboration tools, to simplify document preparation, annotation, and sharing. In all this, however, it continues to be unclear to me what is supposed to distinguish Trellis from other social networks and collaboration platforms that are already such a central part of most of our lives (especially LinkedIn and Google+). Most importantly, however, I hesitate to fully support a new network in the absence of a clear revenue model, and without a firm commitment to data privacy.

Ribbon Tool

The cloud-hosted (via Amazon Web Services) Ribbon FLOW Tool that is being developed at UC Davis by Matt Steinwachs filters and visualizes change in group membership over time. In the demonstration, the tool was used to visualize change in declared major upon admission compared to graduation. Accepting data in json format, the tool is flexible enough to visualize any kind of data flow.

UC Davis Ribbon FLOW Tool

The tool is still in early development, but in future releases and before being made available more widely, Matt has promised to add the following

  • The ability to upload/download data in csv format (a must for usability among non-technical users)
  • Multiple stops, so that changes in group state that take place between start and end can be represented
  • Comprehensive documentation (including security model and privacy policy)

Once completed, the goal is to make the tool open source, and I would love to see it used by the Learning Analytics Initiative or as part of some other similar open learning analytics toolkit.

General Observation and Reflection Tool (GORP)

The General Observation and Reflection Tool (GORP) classroom observation tool is being developed by Matt Steinwachs for Chris Pagliarulo (Director of Instruction & Assessment, iAMSTEM, UC Davis). The web-based tool facilitates data entry for ‘agents’ (student coders) observing classes according to the Classroom Observation Protocol for Undergraduate STEM (COPUS). The tool seeks to solve the problem of having to manually enter data collected by coders at two minute intervals, by automating the collection and storage process.

GORP Classroom Observation Tool

BYOD and BYOT

UCDavis_TEA
The day concluded with a “Bring Your Own Data and Tools” session, which provided TEA Meeting participants with an opportunity to share tools that they had developed and/or had been using with great success. A few highlights include:

  • Marilyne Stains (University of Nebraska-Lincoln) presented the results of work that she and her co-investigators have been doing with the Classroom Observation Protocol for Undergraduate STEM (COPUS). Using the GORP tool, she has been able to identify coding redundancies (ex. Students listening’ and ‘Instructor Lecturing’ represent the same classroom state). A cluster analysis has identified several basic class types.
  • Andrew Feig (Wayne State University) demonstrate the use of Circos to represent individual student course pathways from semester to semester.
  • Erin Soloman (Washington University in St. Louis) presented a flexible classroom observation protocol (OPAL) and iOS app that records and timestamps every coding event. The advantage of this approach over others (which aggregate all events at 2 minute intervals) is that it allows educational analysts to correlate spikes in student engagement with specific moments in a recorded lecture.