August 15, 2013

Reviewing Service-learning at MCLA: The Story Unfolds

The practice of service-learning at MCLA, a small, public liberal arts college in the Berkshires, can be traced back to 1997 when the college received a Massachusetts Campus Compact grant to support faculty who were willing to explore the possibility of integrating this pedagogy into their course offerings. As interest in this pedagogy grew and relationships were forged with appropriate community partners, a more formal infrastructure, the Center for Service and Citizenship, was instituted in 1999 to promote service-learning, as well as to identify and foster stronger connections with relevant service sites.

In 2011-12, members of the Service-learning Advisory Board began to compose a formal document, articulating the mission, definition, goals, structures, and outreach strategies of the Service-learning Operations Committee (composed of two faculty members and the director of the Center for Service and Citizenship).When we started to ask how we know if our program is achieving its goals, it raised a larger question concerning whether faculty participants would even share the goals we had enumerated. We also looked at the AAC&U Civic Engagement Value Rubric, but we decided that it was too broad and that we needed to identify learning outcomes more specific to service-learning. The faculty service-learning co-coordinators expressed a concern that the learning that takes place in service-learning is multi-faceted and synergistic, resisting isolated and quantitative measures of assessment. These questions and concerns served as a catalyst for an exploratory journey that has led us to this particular juncture in our process. The journey has also been propelled by a 2012 publication of the AAC&U entitled A Crucible Moment: College Learning and Democracy’s Future, which argues for the importance of colleges and universities to strengthen and expand their civic education programs in order to facilitate student understanding of the value of first-hand experiences with organizations and agencies, thus enabling students to see themselves as capable of making a difference in their communities.

In the summer of 2012, one of the service-learning co-coordinators negotiated with the Dean of Academic Affairs to provide stipends to entice faculty with some experience in service-learning to form a small work group to develop open-ended questions that could be posed to faculty and students. The work group was comprised of faculty in anthropology, computer science, interdisciplinary studies, and psychology, as well as the Associate Dean for Assessment and Planning. We discussed ways in which to elicit responses and determined that it would be most efficacious to begin by conversing with faculty as that might influence ways in which we engaged students. We decided to hold two faculty focus groups in October 2012 to be facilitated by the faculty service-learning co-coordinator and the director of the Center for Service and Citizenship. We were sensitive to issues of power dynamics and differentials so we concluded that no administrators would be present at the focus groups, and we also developed an informed consent form for faculty who agreed to participate, letting them know that they could walk out of the focus group if, at any time, they felt uncomfortable, and that we would share the focus group summary transcripts and report with them to corroborate the accuracy of our records. Additionally, we composed an invitation to the focus groups, explicating their purpose and how the information they yielded would be used. At the end of the summer, we shared our work and our plans with the Dean of Academic Affairs and were given the go ahead to continue.

In Fall 2012, the primary emphasis of this work involved facilitating the focus groups and synthesizing the responses (Questions, Report). Five faculty members who were not able to attend the focus groups volunteered to be interviewed as well. Altogether responses were gleaned from 15 faculty members, representing the humanities, social sciences, STEM, and professional disciplines. The focus groups and interviews addressed  personal, pedagogical, and professional motivations for devising  service-learning opportunities for students; impact of service-learning on the faculty/student relationship; benefits for faculty; benefits for students; relevance of service-learning assignments to class content; means of assessing service-learning experiences; what faculty hope to glean from their assessments; what faculty thought service-learning assessment should look like; the challenges and rewards of designing service-learning experiences; and ways in which faculty could receive greater institutional support for their work.

While these conversations revealed a fair amount of overlap in faculty motivations and perceptions of student benefit, they also highlighted a real diversity of methods by which student learning is evaluated. Some faculty felt that it would be difficult to capture student learning with service-learning because of the experiential nature of this learning. Others believed that students become more aware of the impact of these experiences on their personal and professional lives once they graduated and would be better documented by alumni surveys. Yet others believed that service-learning assessment should place the greatest focus on student growth and development, rather than seeing it as product- or outcome-driven.

One interesting point the focus groups revealed was the variability of clarification that was given to students concerning the meaning and purpose of service-learning projects in general. Some faculty did not explain to students that the projects are even called service-learning. This revelation was valuable to the faculty co-coordinator and the director of Service and Citizenship because we now know to communicate with faculty the significance of discussing service-learning with students in ways that are explicit and that capture their attention. It is difficult to determine the efficacy of a program if students are unfamiliar with the language that describes their community engagement.

In Spring 2013, our assessment work group had some change in membership as a faculty member in math (and a newly appointed faculty co-coordinator of service-learning) replaced a faculty member in psychology. We reviewed the focus group report for guidance in developing questions for students and devised a tentative implementation plan for a fall pilot. In late February, we sent out a preliminary letter for our fall pilot to a range of faculty for feedback. The letter was intended to invite faculty to obtain student responses to five-open ended questions relating to their service-learning project experiences. In our letter, we detailed the proposed implementation process. Following faculty reactions and suggestions, we made some changes in wording and adapted the implementation process of the pilot plan, providing faculty who participate in the fall pilot with an opportunity to add questions that are more closely aligned with course content. Many faculty members in the initial focus groups, as well as those who responded to the initial pilot plan, did not want participation in this process to be mandated, so in future missives, we made it clear the fall pilot is voluntary for both faculty and students.

The Dean of Academic Affairs had made the support of our spring work contingent on bringing in a service-learning assessment consultant after our spring work was completed to review our progress and make suggestions. We invited Dr. Arthur Keene, an anthropologist at UMass Amherst, who has been involved in service-learning for over two decades, helped develop a Citizen Scholars Program for undergraduates there, and more recently, has written about, and been involved in, state-wide conversations about service-learning assessment. Dr. Keene visited with us in early May and concurred that we were on track. He reassured us that our questions for students are fine and suggested that we put the questions online in order to elicit less rushed and potentially more thoughtful student responses. He reiterated repeatedly that the questions we pose should be driven by what we want to know and why we want to know it, and predicted that what we want to know is likely to change over time. He also commented that our assessment practices would be ineffective if faculty do not feel genuine ownership of the practices and the process, and that fostering a sense of ownership takes time. Likewise, he argued that a practice as varied, complex and dynamic as service-learning cannot easily be reduced to a single metric or measure, advancing that multiple methods might produce the most meaningful results. Together, we discussed the role of student stories as evidence of student learning.

With these caveats in mind, we made minor modifications in our initial pilot plan, began to actively recruit faculty to participate in the pilot this fall, and arrived at a consensus regarding the next stage of our journey. Our summer proposal entailed perusing the articles and surveys which Dr. Keene had provided us. We hoped that this material might generate further questions, help guide future conversations with faculty, and offer us some ideas for direct assessment measures. Our first summer meeting entailed a discussion of A Crucible Moment because it encompassed the widest scope of any of our readings, calling upon institutions of higher education to evaluate both their curricular and co-curricular programs with regard to the promotion of “civic-mindedness.” We liked the fact that the values, knowledge, experiences, and skills that the publication underscored did not hinge narrowly on professional training but highlighted lifelong, active engagement as members of multiple publics. Nonetheless, we felt that the definition and prerequisites for “civic mindedness” could have been further elaborated. Moreover, we agreed that it was more difficult to identify how the STEM disciplines could conform to this ideal of “civic mindedness.” These concerns simply confirmed Dr. Keene’s observation that terms such as “civic” or “citizenship” have varied meanings and cannot be taken at face value; hence, if we want to claim that our college prepares students to be good citizens, as MCLA does, we need to consider what such a claim means to us, taking into account different disciplinary perspectives on citizenship, as well as a diverse range of possible contributions to honing this citizenship.

In two subsequent summer meetings, we discussed the materials that Dr. Keene had shared with us further. We decided to ask faculty who are participating in the fall pilot if they would be willing to send us any stories students incorporate in class assignments that reveal something pivotal about the nature of the learning that may occur with service-learning (see Polin and Keene 2010). Moreover, we are hoping to continue to think about the best ways to elicit student stories in the future as these stories can provide some context for understanding how something is learned rather than simply what is learned. Likewise, we were intrigued with the idea of “identifying qualities that differentiated levels of student performance” rather than specific outcomes because they would more likely capture learning processes rather than end results (see Shapiro 2012: 46). Another idea with which we were intrigued was to train students who have completed service-learning classes to conduct student focus groups, exit interviews or alumni surveys with students and/or alumni who have engaged in service-learning. Such training would foster student leadership and possibly generate greater trust with, and more authentic responses from, interviewees than would be derived from those in positions of authority (see Addes and Keene 2005).

August 18, 2014

The faculty who participated in the Fall pilot 2013 were located in departments such as computer science, environmental studies, fine and performing arts, psychology, and sociology, providing us with a diverse disciplinary sample. In addition to the online questions that were posed to students derived from this sample, we  also sent out a more general email to faculty who had  planned to do service-learning in any of their classes in the last academic year to suggest two questions that they may want to raise with their own students to provide them with greater feedback on their own courses. The two questions were as follows:

1. Were you sufficiently prepared for your service-learning project?
2. What could be done to make the servicing-learning project of greater benefit to students in the future?

The responses to these two questions, if used, were only viewed by the individual faculty members who chose to make such queries. Only the responses to the online survey were reviewed by the Service-learning Work Group in the spring of 2014.

In order to pursue other threads of our summer 2013 conversations, we requested that departments volunteer to set aside one department meeting in Fall 2013 to discuss what civic engagement and “civic-mindedness” meant or might mean from the lenses of their own disciplines. We hoped that departments would want to facilitate their own discussions and send us summary reports; we did offer to facilitate these dialogues as desired, however. The goal of this request  was to have the reports sensitize us to diverse disciplinary civic-engagement perspectives and practices, providing our work with further direction and depth.       

In Spring 2014, the Service-learning Work Group met twice, once to review the student survey results and once to review the feedback on civic engagement that we received from different departments. A relatively small number of students responded to the surveys. Of those who did respond, students seemed to spend more time answering questions regarding how they saw themselves as benefitting others and how they personally gained from the experience. In our group, we discussed the under-emphasis by students on what they had learned from the community. In order to attempt to remedy this issue, we invited Dr. Arthur Keene back in April to facilitate a workshop to interested faculty on how to better prepare students for specific service-learning experiences before they engage with the community. He provided us with a wealth of handouts  on how to create connections with community partners, define goals of relevance to class content, articulate those links within our syllabi and handouts, and communicate the goals and approaches with students face-to-face once they arrive in our classrooms. Our hope is that if, as faculty, if we are more successful at communicating the reciprocal nature of service-learning (thinking about community members as educators), students will be able to more clearly identify what they learned from community partners and not simply underscore what they see themselves doing for others.

As per the civic engagement responses, we received reponses from six different departments. Each department defined civic engagement in ways that were shaped by their disciplinary foci and concerns. Some stated that civic engagement permeated every aspect of their discipline (values, content and pedagogy, not all of which focus on service-learning per se). Some emphasized that the practical applications of their disciplines might benefit students in other disciplines. Some related civic engagement to non-course related forums departments organize for the broader campus community. Others expressed concern that there not be a singular focus on service-learning as a high-impact pedagogy since committed faculty apply any number of approaches to inspiring students. Overall, I think what we took away from these responses is the importance of allowing assessment techniques to be molded to disciplinary interests rather than being overly generic, the importance of understanding that there is no monolithic definition of civic engagement, and the value of being receptive to diverse pedagogies rather than promoting service-learning as the only approach to providing students with high-impact experiences.