Shirlee Geiger on PCC Faculty Learning Asessment

SHIRLEE GEIGER: Hello, my name is Shirlee Geiger, and I…I’m talking to you today in my capacity as chair of the Faculty Learning Assessment Council This is a council that was set up in 2008 by Sylvia Gray The leadership of the council rotates through faculty members here at PCC, so I have been the chair [in] 2010 and 2011 I’m here to tell you my story, the story of how I came to be very excited about assessment, and a part of setting up this class, this hybrid class on program or discipline assessment at PCC My story starts in the Fall of 2008 There I was, happily an adjunct faculty member teaching philosophy Sylvia Gray had agreed to head up the newly organized Learning Assessment Council, and she asked me to participate to give adjunct faculty input I had been very happy to never join any committees at PCC I thought that was part of the joy of being adjunct But I liked Sylvia a lot And I had a friend, a very good friend, who was a 3rd grade teacher in the Portland public school system, and I had been hearing from her for several years now about how the No Child Left Behind act had ruined her fun Indeed, she took early retirement because she was so disgusted with how she believed the No Child Left Behind act had ruined education in America today So, the accountability movement that’s what the Learning Assessment Council was formed to respond to…the accountability movement, I translated into No Child Left Behind comes to a college or university near you Indeed, in the late 90s there was a report from a commission it’s called the Spellings Commission Report that was extending the requirements for accountability to higher education The commission suggested, strongly, that colleges and universities in the United States needed to measure outcomes to compare the value they were adding into students’ lives And I was thinking that that was going to ruin my fun as a teacher at PCC the way it had ruined my friend’s fun as a 3rd grade teacher So I just wanted to be prepared for what was coming down the turnpike Now that first year on the assessment council, we were charged with a year of inquiry, a year of study, and I must say that much of the first part of that year, I was sullen, cynical, bitter and defended But a few things happened to shift my thinking, and I want to share them with you today The first shift in my thinking came from reading a short piece called “The Assessment Manifesto” by Rick Stiggins You will have a link to this in the hybrid class Now, I’m a person who came of age in the 1970s, and an assessment manifesto, well, “manifesto” is sort of my genre So this short, energetic piece was the first thing I read in a pile of reading about assessment that really spun me around One of the things Rick Stiggins said in “The Assessment Manifesto” is every teacher has a kind of picture or theory of grading They have a picture of assessment, and it’s often times not articulated Well, of course as soon as he said that, I had to articulate my theory of assessment, my theory of grading, and here it is in a nutshell: I viewed grading as the least fun part of teaching philosophy, and I wanted to do my very best to make sure the requirement that I turn in grades at the end of a quarter not mess up my fun or the students’ learning But on the other hand, I had an audience in mind when I was grading When I was recording those A’s and B’s and C’s and occasional D’s over my shoulder I had an imaginary colleague, a member of my SAC, and I kept in mind that at some shadowy point in the future someone, some colleague, might ask me to justify my grades So my audience was that potential colleague who was questioning my professional judgment So that is how I approached grading Now, Rick Stiggins had a different theory of grading, a different model of grading And his first big distinction had to do with the audience: who is the assessing for? Stiggins ways the first, and indeed the most important, audience of assessment information has to be the student That is, the student needs to be able to allocate him or herself

as an educational resource They need to know what to spend more time on; be freed up; you’ve got that down, go over here So he said that faculty members, instructors, teachers, we owe our students This is like an obligation…a serious, moral obligation…to have fast, accurate, frequent feedback so they know how they’re doing The other thing he said which really resonated with me is when we tell someone that they have not learned what we had hoped they had learned we need to make sure we deliver that information in a way that is not shaming So the fast, accurate, frequent feedback to the student says: this is what I think you’ve got; this is what I think you have not yet understood And if you want to understand it, it is my professional judgment that this would be a good route for you to take Students need to know how they’re doing so they can make informed decisions about what to do next, and the accuracy of that feedback becomes really important The, the formative assessment is assessment that is done in the middle of an educational process It is done to let somebody know what they should do next And the idea of formative assessment, and the priority of formative assessment, distinguishes it from grading, which in the assessment world is known as “summative assessment.” Summative assessment is done at the end of an educational process to report out the results, usually to an external audience So the distinction between formative and summative assessment processes started sinking into my brain and really changed what I thought I should be doing as a teacher The notion that the student is the primary and most important audience of all assessment activities, and that the most energy I should be putting into assessing is formative, has completely changed my education practices So that first year of the existence of the Learning Assessment Council, 2008-2009, we were charged with recommending to the college at the end of that year how Portland Community College should respond to this new demand for accountability One of the things that we did during that year, which produced the second big change in my thinking, was we decided to do a little survey of PCC faculty and ask them some questions to see where they were in terms of understanding the change in the expectation of Higher Ed We asked faculty if they were familiar with the notion of an outcome, a learning outcome, that should be assessed, and specifically we asked them if they were familiar with PCC’s core outcomes We discovered that a lot of faculty were…well, like me Busy working in their classrooms, trying to be the best teacher they could be, but not paying attention to the broader context in which they were doing the work We threw in a question in that survey that had to do with what PCC could provide faculty at this college that would make their job easier or more effective What did they wish they had that they didn’t have that would make a difference in their ability to work as educators? And over and over, in rather poignant and touching ways, instructors said what they wish they had was connected; that this was a college, but what they missed was collegiality Over and over, we heard faculty members say they felt isolated They were working as hard as they could but they were working in an isolated situation They were hungry for feedback and connection Well, as we were thinking about this in the Learning Assessment Council, we thought that perhaps there was a beneficial overlap, because assessment of the sort we were being charged to do was quite different from the kind of evaluations that had been done previously Quite often faculty members had evaluations of their work done as part of hiring or promoting or some sort of granting of tenure That is, faculty were used to being assessed as individual faculty members Likewise, as teachers, we were used to assessing, that is grading, the performance of individual students But the change at PCC was from an “I”, how am I doing,

to a “we”, how are we doing That is, we were being…we are being held accountable for our collective learning outcomes The PCC core outcomes are a particular kind of promise that if a student leaves PCC with a degree or a certificate Completely regardless of how they found their way to meeting the degree requirements, which classes they took, and from which instructor, nonetheless, we were promising that they would be meeting these college outcomes If we were going to assess whether our students were meeting our outcomes, we needed to collaborate; we need to create a new sense of “we.” How are we doing? So the Learning Assessment Council asked that faculty, through their subject area committees, through their SACs, work together at assessments to discover how their programs were doing f or their students I am self motivated I work well independently I do not need a lot of supervision, and I actually don’t want a lot of supervision Many teachers, I think, are highly skilled at that kind of self directed, self motivated responsible and independent work But if we are going to be able to meet the changed demands of program discipline and institutional assessment, we have got to learn how to collaborate We have got to break out of the isolation imposed on us by our individual classroom walls and, and turn toward one another, learning how to construct a “we”, and ask the question, “How are we doing?” We so far have not had as much practice working in teams, and so the Learning Assessment Council came to believe that we had to set up some vehicles, some roots, for people to brush up on their collaboration skills So, the next big shift in my thinking in that year of inquiry on the Learning Assessment Council, I went from thinking that collaboration was going to be a nice sort of bonus or side benefit of the accountability movement, the demand for assessment I went from thinking that it was, you know, a nice extra to thinking that is was incredibly necessary; that it was something that we were going to have to do if we were going to be able to succeed Well, that whole notion of success in higher education brings up the context in which many of us are laboring And that is this conversation about whether higher education needs reform; whether it is failing; whether it is doing such a bad job that it needs to be totally changed And the question of whether people in higher education are failing their students and the taxpayers, well, it makes it a bit hard to go to work every day with a smile on your face But after a bit, I began to think it made sense to worry about whether higher education was up for the job because one of the things that has happened in the last decade, maybe two decades, is that the job higher education is being asked to do has changed in very fundamental ways And I heard a person speak about this change in the job requirements, change in the job demands, in terms of what he called “The Pyramid of the Impossible.” So, the Pyramid of the Impossible comes from three requirements for higher education, now the three new pieces of our job demand, and the first one is that higher education is being asked to open its doors ever wider That is, to increase our access to ever more students And these are students from classes from categories, from groups, that were previously excluded It’s hard to gear up for this many new students, and to make sure that we have enough sections for them, and people to process their registration; advisors and counsellors to help them figure out what to take But this is only one side of the pyramid We are asked to increase our access and simultaneously increase the rate of students who leave our colleges and universities with degrees and certificates Now, one easy way to open our doors wider

and make sure a higher percentage of students leave with degrees and certificates is if we would make the degrees and certificates easy to get, if we would water down our standard But if the degrees are watered down, they will lose their legitimacy out there in the world So: more students in, some of those requiring additional resources that were not needed before Remedial work, counseling, extra language work for people entering with English not their native language So ever greater access and a higher rate of completion with a meaningful degree Oh! But here’s the third part of this pyramid We are going to have fewer resources per student We are supposed to do this work with less money instead of more The idea that we grant greater access, provide a higher rate of completion, and we do this with less money, well, if you stand back and think about those three together, it’s not surprising that there are questions about how effective higher education is being I think it is a good thing to keep in mind when people are talking about how education is failing our population, it’s not clear that this job can be done But if it’s doable, it is because we are going to enter an era of unprecedented collaboration among educators No longer can we afford to be isolated in our classrooms each of us innovating to do the best job that we can We need to start sharing amongst ourselves what we discovered, what works, if we have any chance whatsoever of meeting this new request to be effective But as faculty members have talked to each other, I have noticed a piece of the conversation that bothered me a lot And the more I thought about it, the more it seemed to me that it was one of those cases of people using the same language but with different defintions So, with respect to this idea that we are going to be graduating higher rates of our students, I had many instructors talk to me over the course of rotating around with the Learning Assessment Council, and they said something to me like this: You know, when I was in college, either my graduate experience or undergraduate with the senior level classes, I had teachers say something like this: they said hey, students, look to the right and look to the left At the end of this year, only one of those three people will still be here That is, a major function of higher education was to serve as a gatekeeper, as some people would pass through, and in the ideal of a meritocracy, those would be the ones who had earned it But other people would wash out And grading, especially grading on a curve, serves the function of figuring out who to let go forward and who to let wash out This kind of grading, or summative assessment, is called “norm referenced.” Ideally, we have assessment instruments that produce that bell-shaped curve That is, we figure out who are the exceptionally wonderful, who are the exceptionally horrible, and who is average And depending on how few positions there are we only dip into the exceptionally wonderful, and exceptionally, exceptionally wonderful, to let them go through But I remind you with the Pyramid of the Impossible we are not allowed to wash people out We are being asked to get everybody through, as close to 100% as possible And in order to do that, we need to change our thinking from assessment instruments that are norm referenced to assessment instruments that are criterion referenced In criterion based assessments each student, or each student performance, is compared to a standard, a standard that doesn’t change And the assessment gives a yes or a no answer; either this student met that standard or they didn’t You can have a good assessment if it’s criterion referenced and have everybody fail But it’s also possible to have a good assessment if it’s criterion based and have everybody succeed Indeed, if we have criterion based assessments,

and we are effective at education, we want as close to 100% of our students to succeed as possible This is not because we have taken the average and made it “A”, watering down our standards; rather, it’s a different kind of standard People are not being compared to each other It’s only if we have criterion based assessments that we will be able to meet this new demand of higher rates of our students meeting the standard This is a really big change between norm referenced and criterion referenced, and as faculty come to understand it, there’ll be less confusion about what it is we are doing and less consternation over the idea that we are watering down our standards So, by the end of that year of inquiry on the Learning Assessment Council 2008 to 2009, my head was seriously spinning I had thought I had been a good teacher I had longed to be a good teacher But many, many, many ways of thinking, that I had had for my teaching career, they were spun around; they were changed It was a particular kind of vertigo And after awhile I remembered I had like a class in graduate school where they talked about this, and it was the class in paradigm shifts [laughs] And I said, oh, I’m experiencing a paradigm shift Now paradigm shift is, by definition, disconcerting and disorienting, but I wanted to try to get a little bit firmer sense of what we were doing, and so I analyzed the change in the paradigm, and I decided it’s really a three-stepper There are three steps involved in going from the old way to think about teaching to the new way of thinking about teaching If we are going to be effective I think it would be helpful for all of us to understand these three steps Now, the first step may be the hardest At least for me, I discover over and over that as much as I aspire to it, I don’t really have it down That’s because I think of myself as a teacher, and as a teacher, my question has been “what shall I teach today?” I create an outline of ideas that I want to convey, the things I want to say Indeed, I remember when I was a new teacher of philosophy, I had this idea that my job as a teacher was to say everything on my list, one time, right [laughs] I just had to say it right one time and my job would be done Slowly I realized that me saying it one time right did not mean a student heard it and understood it that one time, so I got the notion that I was supposed to repeat myself, maybe in multiple ways But all of this is still a paradigm that people now call “covering the material.” This is a covering the material paradigm and it’s teacher-centric It’s from my point of view, what’s I’m doing, what I’m saying My order The first big step in this new paradigm is to move from teacher-centric thinking to student centered thinking Instead of what shall I teach today the question is, what do I want my students to learn today? This is the movement from teaching goals to learning outcomes I remember at PCC, going to a SAC meeting, a subject area committee meeting, where we were busy changing all of our language from course goals to course outcomes, and I don’t think we understood what we were doing That is, we were complying with an administrative request that felt to us kind of silly, just like a hoop to be jumping through And maybe it’s good that we didn’t know what we were doing because this is a huge shift in higher education, from teacher-centric to student centered: what do I want my students to learn, the movement to student learning outcomes One thing I would like to say as I started to understand that difference is that thinking about the process in terms of student learning outcomes is really somewhat liberating That is, different teachers can arrange really different sets of experiences You don’t have to teach alike The question is whether those two different ways of approaching or setting up a classroom are equally effective at the learning outcomes

If what we’re asking is, what did the student learn, not what did the teacher teach, it leaves open a possibility for incredible innovation and diversity in terms of how those outcomes are met So the first step in the paradigm shift, the first part of my vertigo, was going from teacher centered to student centered activities Teacher-centric to learning-centric thinking on my part But now the second step here, the second step in this paradigm shift, is once you’re thinking about outcomes…the student, what the student is achieving the, the challenge is to think of outcomes not just in your classroom I’m a philosophy teacher and have been for a really long time, and part of that is because I’m a deep believer in the value of critical thinking But the accountability movement asks me to think about what difference I think I am making in people’s lives, not just in my classroom, but in the rest of their lives And I guess always I had the idea that, from taking a critical thinking class, a student would become better at thinking critically Not just in my classroom, not just when they aced the final exam, but in their life as a citizen – in their communities, perhaps in their neighborhood association In their families In their work life If the difference I’m trying to make in somebody’s life is not just in my classroom, but outside the classroom, in their life out there, the question becomes: how can I tell if I’m making that difference? Now, summative assessments are in the classroom, but if we at PCC are saying people leaving with degrees or certificates have increased their critical thinking, well, I’d like to know, for example, if there’s higher voting rates among graduates than those who don’t go to school I would like to know if there’s more civic engagement or participation I would lke to know if in their workplaces they are more likely to challenge the status quo, innovate, come up with new ideas That’s what it means to think about the outcomes out there We are not going to be able to directly assess those as instructors But we can allocate resources, institutional resources, to follow our students into their lives and see if we are really making the difference we say that we are making Well, the third step in the accountability movement, the third step in changing the education of…the paradigm of higher education, is once you have articulated the outcomes out there ask yourself: how could you tell? Ask for a metric Ask how we could find out if the difference in people’s lives we want to be making, well, if we’re actually making that We need to know if we are succeeding Now, some of the pressure on higher education I have come to think is appropriate It’s because what we are doing is really important I believe that all of us are living in precarious times The problems that are facing humanity, collective problems, are really, really big Now, I have been told sometimes that I’m a bit dramatic Some people say even melodramatic But in a melodramatic sort of way, I think the desire of educators to make a difference in the world is coupled at this point in the history of the Earth with incredible need for that difference But I want you to think for a moment about the promise we’re making to our students about what our core outcomes say is the difference we are making in people’s lives We have six core outcomes and I’m just going to jumble them all together We at Portland Community College are promising that students who graduate from PCC with a degree or certificate will be able to collaborate together across the traditional divides of culture, ethnicity, religion, identity, and come together around our communal issues We say they will have acquired a willingness to walk toward our collective social and environmental problems instead of running away We say they will have a sense of themselves within the larger universe

from the practice of self-reflection, self awareness They will be aware of their own talents They will have had a chance to articulate their own values and ideals, their aspirations And with this base they will be able to work with others, exercising creative problem solving, thinking critically, using those communication/collaboration skills to work together to solve the huge collective problems my generation has left them Oh, and we say they will have all that and be able to earn a living consistent with their own values and sense of self Ladies and gentlemen, I believe humankind is facing some enormous problems right now I believe these are incredibly precarious times And the reason there’s additional pressure on higher education and higher educators is because our ability to keep our promise to our students could be the difference between the possibility of a future for humans on this Earth and that possibility being closed We must be adequate to our promises