How Valuable is the Gallup-Purdue Index?

In May 2014, Gallup in partnership with Purdue University and the Lumina Foundation released the inaugural Gallup-Purdue Index Report, which seeks to capture the relationship between college experience and success later in life.

GreatJobs_GreatLives

As an index, the Gallup-Purdue Index merely adapts Gallup research on customer engagement to assess graduates’ perceptions of their colleges experience. The index consists of six elements of college experience that the researchers at Gallup and Purdue found to be the most significant predictors of workplace engagement and well-being:

  1. I had at least one professor at [College] who make me excited about learning.
  2. My professors at [College] cared about me as a person.
  3. I had a mentor who encouraged me to pursue my goals and dreams.
  4. I worked on a project that took a semester or more to complete.
  5. I had an internship or job that allowed me to apply what I was learning in the classroom.
  6. I was extremely active in extracurricular activities and organizations who attending [College].

What the GPI report does is identify correlations between items present in the Gallup-Purdue Index, and items in other indices that Gallup has developed: The Q12® employee engagement index, and the Gallup-Healthways Well-Being Index (they also look at support and alumni attachment, to gauge likelihood to donate back to the institution). In this, the report serves merely to provide evidence for the strength of the Gallup-Purdue Index itself. In other words, it comes as no surprise that we find the six elements of the Gallup-Purdue Index highly correlated with workplace engagement and well-being, since it was exactly this high level of correlation that served as the criterion on the basis of which these items were selected for the index in the first place.

What Gallup and Purdue have done, then, is to provide institutions with a very nice set of six questions, the responses to which can serve as proxies for ‘likely success later in life.’ The workplace engagement index that Gallup has employed is not new (development of the Q12® index began in the 1950’s and was completed in 1988). Nor is the Well-Being Index. And yet, both of these indices continue to be refreshingly timeless, as they serve to reflect the fact that success in life means something other than merely income or prestige. Perceived job satisfaction (as a predictor of workplace effectiveness) and well-being are complex. The excellence of these indices is a function of the fact that they view the individual as a multi-faceted whole. Gallup and Purdue are to be praised for using these as the foundation of an index that would capture the most important features of college experience.

A Note on the Arizona State University “Great Jobs, Great Lives” Report

Gallup is now actively marketing the GPI to colleges and universities in the US, as a service for helping institutions to understand the nature and relationship of student experience to workplace engagement and well-being among their particular students and alumni. In the summer of 2014, Gallup worked with Arizona State University to survey a sample of Arizona State undergraduate alumni and published the results in a “Great Jobs, Great Lives” report tailored to that institution. What value does the GPI have as a service to particular institutions of higher education in the US? Should we all do it?

An institution-specific GPI report might be valuable as a way of comparing a local GPI to that of the nation as a whole. It should be emphasized that the GPI is not meant to generate another university ranking scheme. In fact, chief among the findings of the national report is that “where graduates went to college — public or private, small or large, very selective or not selective — hardly matters at all to their current well-being and their work lives in comparison to their experiences in college.” In the Arizona Report, ASU is compared to National GPI, Public Universities, and Large Public Universities, something that is only possible with access to Gallup’s national survey results. Aggregate response percentages were not included as part of the national report, but thanks to ASU, those numbers are now public.

Arizona State University has done a great service by commissioning and making public the findings from their GPI report. The information about the National GPI, Public Universities, and Large Public Universities fills out the picture begun with the national report. With both the ASU and national reports in hand, however, it is my opinion that commissioning reports for other institutions would offer very little additional value.

An institution-specific GPI report might be valuable as a way of checking to see if the six features of the GPI are as correlated to workplace engagement and well-being locally as they are nationally. This is, of course, not a question that anyone should entertain seriously for long, since viewed from this perspective the reports are essentially tautological: the GPI was developed based on the fact that its features are highly correlated with features of workplace engagement and well-being. It should come as no surprise, then, to find high correlations between those elements in the ASU report. In fact, since the GPI was developed as a strong predictor of workplace engagement and well-being, it shouldn’t actually be necessary for an institution to ask questions about anything other than student experience. The ASU report is nice as a validation of the GPI but, to the extent that the GPI has been validated, it becomes less and less important for an institution to ask Q12® and Well-Being questions of their alumni. It actually becomes unnecessary to assess alumni by the GPI at all. Acknowledging the predictive power of the GPI, schools can with little effort begin regularly surveying their current students, and optimizing their learning environments according to student feedback along each of the six GPI measures.

Lastly, an institution-specific GPI report might be valuable in order to establish a baseline student experience score or, better yet, to identify trends in student experience over time. Unfortunately, this kind of longitudinal work is absent from the ASU report. Also absent is any information about the experience of current students. Taking a representative sample of one population, and comparing it with another is interesting, I suppose, but in this case not particularly helpful to administrators invested in making decisions that would impact the future success of students enrolled at their school. Has student experience at our institution changed over time? How does the experience of current students compare with that of alumni? Are we seeing declines in the quality of our students’ experience that would put their future workplace engagement and well-being at risk? The emphasis that the ASU report places on comparing ASU to other populations is, to my mind, not particularly informative or actionable. The same index applied longitudinally, and with an emphasis on current rather than past students, would be far more useful as a tool for identifying obstacles to student success, and informing strategies for mitigating risk as part of ongoing process improvement.

How Big Data Is Taking Teachers Out of the Lecturing Business

A Summary and Response

In his Scientific American article, How Big Data is Taking Teachers Out of the Lecturing Business” Seth Fletcher describes the power of data-driven adaptive learning for increasing the efficacy of education while also cutting the costs associated with hiring teachers. Looking specifically at the case of Arizona State University, where computer-assisted learning has been adopted as an efficient way to facilitate the completion of general education requirements (math in particular), Fletcher describes a situation in which outcomes for students scores increase, teacher satisfaction improves (as teachers shift from lecturing to mediating), and profit is to be made by teams of data-scientists for hire.

Plato_Aristotle_della_Robbia_OPA_FlorenceThere are, of course, concerns about computer-assisted adaptive learning, including those surrounding issues of privacy and the question of whether such a data-driven approach to education doesn’t tacitly favor STEM (training in which can be easily tested and performance quantified) over the humanities (which demands an artfulness not easily captured by even the most elaborate of algorithms). In spite of these concerns, however, Fletcher concludes with the claim that “sufficiently advanced testing is indistinguishable from instruction.” This may very well be the case, but his conception of ‘instruction’ needs to be clarified here. If by instruction Fletcher means to say teaching in general, then the implication of his statement is that teachers are becoming passé, and will at some point become entirely unnecessary. If, on the other hand, instruction refers only to a subset of activities that take place under the broader rubric of education, then there remains an unquantifiable space for teachers to practice pedagogy as an art, the space of criticism and imagination…the space of the humanities, perhaps?

As the title of Fletcher’s piece suggests, Big Data may very well be taking teachers out of the lecturing business, but it is not taking teachers out of the teaching business. In fact, one could argue that lecturing has NEVER been the business of teaching. In illustrating the aspects of traditional teaching that CAN be taken over by machines, big data initiatives are providing us with the impetus to return to questions about what teaching is, to clarify the space of teaching as distinct from instruction, and with respect to which instruction is of a lower-order even as it is necessary. Once a competence has been acquired and demonstrated, the next step is not only to put that competency to use in messy, real-world situations–situations in which it is WE who must swiftly adapt–but also to take a step back in order to criticize the assumptions of our training. Provisionally (ALWAYS provisionally), I would like to argue that it is here, where technê ends and phronesis begins, that the art of teaching begins as well.