Integrate Authentic Assessment with Traditional Assessment

TAAA

In the “Making the Grade: The Role of Assessment in Authentic Learning”, Marilyn Lombardi compared authentic assessment with traditional assessment. According to the information provided in Table 1, traditional assessments seem to have few positive characteristics. Thus, a movement from traditional assessment to authentic assessment improves teaching and learning (see more from Grant Wiggins’s “A True Test: Toward More Authentic and Equitable Assessment” and “The Case for Authentic Assessment“). From my experience, some mix of the two seem to be more appropriate and beneficial.

As an instructor of an introductory Remote Sensing course, I applied traditional assessment tools (multiple-choice quizzes, short-answer exams, and essays) and authentic assessment (hands-on experiments, computer labs, and class projects). The combination of the two improves teaching and learning. Quizzes, which only have five multiple choice questions covering basic concepts and key ideas of a former lecture each time, help me measure students’ acquisition of knowledge and my teaching quality. When students made mistakes, they didn’t acquire the knowledge because of their misunderstanding or my unclear explanation. Then I can find another way to explain the concept clearly. The relationship between the time used to answer each question and accuracy of the question can reflect students’ learning in some way —shorter time with higher accuracy means students are likely acquire the knowledge, longer time with lower or even higher accuracy means students are not familiar with the knowledge, and shorter tiem with lower accuracy means students do not care. Hands-on experiments and computer labs, which are designed to introduce various useful equipments, software, and skills, help students apply the knowledge to solve real-world problems. Then students developed their own class projects related to their interests and I talked more about the topics they focused but not well covered in former classes. Authentic assessment complements traditional assessment but it increases my workload. My advisor is always trying to find a better way to mix the two for a large class.

The Important Stuff Isn’t on the Test

Students of my generation, that is, the generation who grew up as participants in No child Left Behind, the height of the standardized testing boom, are sick and tired of it. Ever since I was in 2nd Grade, I had to take a test for about 6 hours per day for a week; I had to take a Writing Assessment in 4th, 7th, and 10th Grades; I had 3 AP exams, in addition to the SATs and ACTs (twice), I even took a driving examination (again, twice, because, as I learned that day, it IS important to come to a complete stop at a STOP sign). I got into college as an undergrad, and had some reprieve…until I applied to graduate school and had to take the GRE. I’m just about done with tests, with people constantly feeling as though I have to measure up to a pre-conceived standard, that, as a result I am pitted in competition with all of my peers to see where we each measure up.

Sometimes we are so focused on the test, the final product, that we forget to learn anything. Teachers, whether due to incentives, regulations, or poor instruction, are sometimes stuck teaching linearly, sort of one-way teaching (teacher to student). Sometimes they open it up and it becomes two-way teaching (teacher to student and back–discussion). Why can’t we find more ways to teach that are harnessing more than memorization and study skills, such as what Eric Liu and Scott Noppe-Brandon talk about in their text Imagination First, going so far as to say that imagination “is…what makes education relevant–to everyone.” Allowing students the freedom to explore as well as the discipline to work toward a goal is an effective means of learning.

Here are some things I learned outside of the tradition classroom and standardized testing:

How to hold a baby.
How to tie a necktie (bowties still elude me)
How to read and write (technically this is cheating because I would have been taught in school, but already knew how when school started).
How to change the oil in my car.
How to change a tire on my car.
How to tell a joke.
How to throw a fastball.
How to bake cookies.
How to kiss.
How to talk to girls. (Not necessarily in this order…)

Most of these things are skills that I will use at various times in my life, and I would argue that they are just as important as what we learn in class. The difference is that I learned them by doing, by putting myself out there, into the world and experiencing them not only with my mind, but with my hands and my heart. We learn and use what we learn in a variety of ways; why shouldn’t we be assessed in a variety of ways as well? Perhaps, instead of asking “Will this be on the test?” we should be asking “What’s the best way to learn this?”

I

Marilyn Lombardi’s article made me realize that I not only enjoy Public Administration as a field of study, but because it offers such amazing opportunities to create learning with my students in a way that is truly different from other courses (not to mention how different from how I learned). Now, I am imagining shaping an intro to public administration class that is scenario based, focused on learning the traditional theories in applied, contemporary situations. Maybe have students conduct analysis of actual policy issues for a real public officials or agencies. The logistics of such an assessment  process are still a bit fuzzy, but the potential is exciting.

I am also intrigued by the concept of co-creating assessment rubrics with the students. While it could be difficult in to do in a large undergrad class, I’m not convinced it is impossible. Of course, the smaller honors or graduate seminars are much easier to imagine. Has anyone tried this with their class? What are your experiences? What went well or not so well?

The research portfolio is a method that has recently been introduced for master’s students in our department. I am still getting my head around the dynamics of the portfolio, but the way in which Lombardi presents the concept, I wonder if we think about the portfolio not just as an assessment tool for a single class, but whether it could document the work throughout a student’s degree program. Would this long-term assessment give more meaning to the degree in totality?

Now I’m a Believer

I found the readings and videos for this week to be very interesting.  It made me reflect on my own education and the impact that grades had on it.  Basically, not much.  Grades always came pretty easy for me, I don’t remember ever being overly stressed about them nor even really caring for them.

But despite the minimal negative impacts that grades had on my education, I have been convinced that they are an archaic and ineffective aspect of school.  Something ingrained into the system and done more out of habit than any thought-out reason as to their purpose.

I am actually in the process of reading Kohn’s What does it mean to be well-educated? (I actually have the full book).  He covers many aspects of contemporary pedagogy but also elaborates on some of his criticisms of standardized testing and traditional grading schemes.  One thing I found particularly insightful was that grades are not the same as evaluations and evaluations don’t necessarily have to be numerical.  I also liked the idea of involving students in the design of evaluations and the assignment of grades.  This is something I have been doing in my own class this semester (though I was doing partly because of my own inexperience).   

As an engineer, I still like the idea of being able to put numbers on things.  It feels more objective to me.  But just because something is easy to measure numerically doesn’t mean its worth measuring.  Likewise, numeric grades make things easier for the educator but diminish the education experience for the student – at least compared to a more in-depth and personal evaluation.

Grade Expectations

My world is largely subjective. The performing arts thrives in ambiguity. It feeds off of intrinsic value in a way that many of the hard sciences does not. We often measure success in the performing arts through emotional connection, talent and ability of actors and musicians, visually compelling sets and design. Administratively we measure success in the number of tickets sold for a performance, the volume of community engagement, the amount of dollars contributed because of an artist or opportunity.

The point is success means different things for different people. The definition varies depending on individual viewpoints of the individuals involved. Success may be something different if you’re an artist, if you’re an administrator, of if you’re an audience member. The important thing is deciding what is your personal definition of success and how it influences your day-to-day decisions in achieving your goals.

The reason I bring this up centers around our current model of assessment within higher education. Now given that I am an artist, I tend to be biased towards the ideas that the arts have this ability to solve everything, including the way we approach assessment. Assessment is really nothing more than a system of rewards and punishments. Incentive is given to students who perform specific tasks in a specific way that ultimately results in a binary judgment of the results. You’re either right or wrong, this is good or bad, it’s positive or negative. The intrinsic nature of the arts has the ability to remove the reward system from the equation which changes the binary system of judgment so suddenly work isn’t good or bad, it just is.

This aligns with what Dan Pink explains in his Ted Talk, The Puzzle of Motivation, which highlights the opposition of using rewards as a form of motivation for cognitive workers. The research in this area indicates that using rewards as a form of motivation causes an inverse reaction to employees who work in the knowledge economy. Essentially, the greater the reward, the poorer the performance. Pink explains when creativity is incentivized it actually narrows focus rather than broadening the range of possibilities. As a result, growth in the knowledge sector is often stifled by a system of rewards rather than expanded as a result of freedom, autonomy, and self direction. If the system of incentive is counter intuitive towards creativity within the knowledge economy, it means that assessment is also counter intuitive since assessment is really another form of incentive.

Bringing it back to the arts world, how do you appropriately assess work that is largely subjective? How do you grade a monologue or a cello solo in a non-subjective way? How do you determine appropriate assessment of a ballet solo or a documentary student film? Is it counter intuitive to do so considering learning in the arts is a series of applied technical skills mixed with the development of raw talent and determination? This is an interesting topic and one that forces me to wonder, what if we allowed students to award themselves with the grade they think they deserve and then require them to live up to their own expectations? As always feel free to leave your comments below. Feel free to discuss how you define success in your field.

Tests: Efficacy on a swing-set

In third grade, letter grades were something I only paid any attention to once a year and the end of the year ceremony when I got my report card. I was a B student. The teacher gave us each a cheesy picture of a bunch of kids playing on a playground and swing set under a big cartoon sun. We had just learned about flash floods with our science teacher – terrible surprising things that catch people off guard. The more I looked at that cheesy (I loved the word cheesy) smilie face of a sun, the more I could picture the footage of flash floods, full of before and after shots. So soon the kids were kayaking over to their neighbors’ houses, designing the new rules of an underwater playground, and going on daring expeditions to retrieve canned goods from their basements, swimming and kayaking around the house. If I had thought more about grades at the time I might have even put some of it on paper. But I doubt I would have crafted a story that excited me enough that I would actually remember it any other way. I don’t think I was as upset about turning in an almost empty page as I was that they hadn’t given me enough time to finish. I was kind of jealous of my friends who had written out their stories, but secretly I still thought my story was the best.

I think there can be a tradeoff between motivation and completion. I never struggled with motivation. I did struggle with completion. I was a bit of a perfectionist.

I designed a desire for grades – I was not forced into one. After third grade I didn’t have grades – a smattering of tests but never course grades – until I was in high school. In high school biology I found I needed tests. I loved biology. A test for me allowed me to say I am finished with a chapter and it was time to move on. It gave me closure. Which is I think, the best and worst part about testing. Testing can lead to an “I’m finished” attitude. If you need to foster curiosity and motivation, this is the counterproductive. But testing was important for me, because I had the opposite problem. I would absorb everything in my biology text, memorize it, read it over and over, and the test was the thing that would convince me it was actually time to move on. It helped progress and gave me a sense of completion.   And even though it is a two-sided coin, a sense of completion – and a deadline for completion – can be a really good thing.

The Omnipotent Grade?

As a teacher, the constant struggle of “how can we make them want to learn?” haunts us before every lecture. We want our students to be excited to come to class, crave the material, and walk away happy with this epic newfound knowledge in hand. To put it simply, we want them to want it.

Intrinsic motivation, as described in Dan Pink’s TedTalk, is what we strive for as educators. A go-getter attitude with an air of excitement is what we hope to see in every student that walks through our doors. Unfortunately, when this doesn’t happen, we tend to do one of two things: blame it on them or on ourselves. We may come away believing that they aren’t a good student, we were never meant to teach, and our material was, even after hours of preparation, lack-luster. The problem with this line of thinking is that, more often than not, this lack of intrinsic motivation is not due to either of the aforementioned individuals. Instead, it is due to the omnipresent extrinsic motivation of the omnipotent grade.

As Alfie Kohn suggests in “The Case Against Grades,” this obsession over the grade (more often tied to a fear of failure and/or overwhelming need to “succeed”) can be a true hindrance on our students. Not only does it force them to worry about a number, it actually prevents them from getting excited to learn. There are many students who really want to learn, but due to many factors (e.g., lack of sleep, complicated material, instructor teaching style), they actually lose all focus on learning simply because they have their eyes on the external prize.

The argument over the failure or success of assessment has been a long standing one. However, I think a huge factor that needs to be taken into consideration when making this argument is our students’ motivation. What is truly motivating them? Is their desire to learn, or is it the number on a test? To say that assessment is a system-flaw would be an understatement.  In the larger sense, assessment could be potentially damaging to students’ intrinsic motivation in an irrevocable way. Thus, the question is: how do we, as teachers, increase intrinsic motivation? How do we make our students want to learn while the system is telling them they have to learn?

Assessing assessments

  • Letter grade: E
  • Textual description: The need and relevance of assessments have been quite extensively discussed over the years. There is a growing consent against the currently existing summative assessment system that we commonly use in schools and colleges. People have also advocated against formative grading systems, standards-based grading, and grading-cum-feedback systems. In short, grading appears, and truly so, to be a hinderance to a meaningful learning experience. Hence, I grade the current quality of assessment to be E (or for those who dislike lettered-grades and want labels: “Execution-lacking”)

Many would find the textual description to be more meaningful than letter grade in the above case. However, it is quite apparent that I tried to justify the grade without adding much information in the description. If “Assessment” were to see the grade and feedback I gave, it would not be able to use it for progress.

We have to go to the root of the issue to find answers to how “Assessment” can improve itself in the coming days. The fundamental question, at least to me, seems to be, “What is the purpose of education?” If it is to be prepared for industrial work where a baseline knowledge is expected and where the work mostly involves low-cognitive effort, meager independent or divergent thinking, or requires only the search for information without much meaning-making then a graded assessment system could be useful. We would put a label on the quality of the individual’s knowledge repository.

However, if the purpose of education is not merely to know more but rather, as Margaret Ammons puts it, “to create a learning society”, then graded assessments would not useful. In fact, with this view about education, all assessments that do not support students in becoming lifelong learners would pointless. On the other hand, a holistic assessment that focuses more on the effort and the process, and which is used as a tool to support students to learn further would be useful.

Moreover, assessments should not be used for conformity – either to conform students to a standard or to an expected performance. As advocated in my previous blog post, autonomy and personalization are important to motivate learners. The design of assessment also affords an opportunity to further enforce the idea of autonomy and personalization. People learn things at different pace, have varied interests, and are motivated for different sets of goals. To wear the same lens to evaluate such diverse range of people, especially by fixating on standards and learning expectations, would be fallacious. An effective assessment would require negotiation between each individual learner and the learning-facilitator whereby the learner decides what they want to learn, how they want to learn, and how will they prove their mastery (of the knowledge body). Based on this negotiation, the facilitator would provide feedbacks to the student. These feedbacks would be presented throughout the learning time and would involve both formal written notes and informal conversational feedback.

However, such change in the way we assess students would require a substantive change in the social structure. Letter (or numbers) make it easy to objectively evaluate people (given that we are willing to overlook the shallowness of the letter). Kohn argues that narrative reports have been useful to evaluate people for admission to colleges. But still, colleges in the United States still depend on additional standardized testing. Moreover, outside the United States, most higher educational institutions especially in Asia depend on numbers/letters to decide admission. Beyond admission, grades are also quite extensively used by potential employers. If we are to move away from grades and pointless assessment systems then our dependence on grades and objective evaluation needs to be removed.

Additionally, teachers need to be willing to let go of the idea of graded assessment and be firm on their approach of a more holistic assessment system. Professional development training for teachers and additional policy changes that supports the removal of grading systems may be needed. Only when anecdotal examples of schools forgoing old-style assessments, such as the story of Jim Drier, becomes commonplace stories would a revolutionary change in education – one that aligns with the current needs of society – take place.

  • Better assessment: “Assessment”, you have a somewhat old mindset of objectively evaluating other people. You need to grow and adapt to the changing needs of the society and the scope of education. As we negotiated earlier, you were supposed to more involved in the process that the learner is situated in and less concerned in the ultimate outcome. You have made some progress over the years but there is still a lot to be desired particularly in your execution. We also had discussed the need for you to be more supportive in the learning process rather than being abstract and decontextualized. In this too, you have made some progress but more needs to be done – you have to move out of letters/grades/rankings and present detailed description of how the learner can do better.
    There are certain things like the society’s structure and school systems’ rigid mindset that is hindering your progress – which you have made it clear to me. Rest assured, that we are working on correcting those problems so that you can become better in the near future. You should keep on transforming yourself and moving forward, if possible, with a greater pace than what you have done so far.

The colossal divide between grades and learning: A college experience

Like most freshman entering college, I did not know what I wanted to study. But to play my cards safely, I went ahead and completed required courses for pre-med “just incase” I still did not know what to do upon graduation. BIG MISTAKE! I learned a lot about myself and other friends who were on the pre-med path, with one general constant: by focusing almost entirely on their grades, they hindered their learning capacity. It wasn’t about ‘learning’ biology yet, they maintained. It was only about getting into medical school, where all the “book theory” in the first two years still don’t prepare you for anything except other exams. It was only during rotations in 3rd/4th year medical school, where you visit patients and take a hand-on approach where medical learning begins. Anyone who has a friend in medical school can attest to this narrative (what a waste of time, right? except they got $$$….so sad these are our doctors!)

My roommate was an interesting case-and-point. He simply knew how to make an A, no matter the content, material or structure of the class (Organic Chem, German film, econ, social theory, all A’s!). One time I asked him how he did it. I knew he worked hard, but he did not strike me as someone particularly marked with high acumen. Aside from working hard, re-writing his homework, taking meticulous notes, etc, he made an effort to visit professors and compliment them. “Do this a few times a semester” he advised, “and you will have an A”.

WHAT?!

I DID NOT JUST HEAR THAT…(or perhaps I was thinking, “Gee, why didn’t you tell me this earlier dude?”)

My roommate uncovered the subjective element in grading, the human element, that I had so easily forgotten due to my unconscious subservience as a student towards my mythic-god like professors. Don’t get me wrong, he worked really hard. But it was clear to me that he tapped into a resource that pushed him above the rest of us.

I’m certain he doesn’t recall or care much about what he learned in college, but for me, it has been a recurrent nightmare. All those pre-med courses could have been better spent in courses I actually cared about. Luckily, my senior year I met a sociology professor who got me up to speed with “learning”, writing and arguing a cohesive paper, and most importantly, to get me to NOT CARE ABOUT THE GRADE. I would add that this professor was notorious on campus for (1) giving too much reading similar to graduate level courses and (2) never giving an A to a student (there have been rumors that a select few received 3.7 over his 20+ year career teaching).

The words of Dan Pink and Alfie Kohn really struck a cord with me. Replacing grades with narrative assessments has seemed to have a fairly positive effect, especially given the fact that elementary and middle school grades are not considered for college applications. I have been scared from the learning v grading divide. It is a visceral experience that has essentially made me no longer care about grading and only towards learning. Some of my most memorable interactions with people have been those who have learned a skill or language without schooling. When asked how they did so, the reply was usually something like “The school called life taught me!” Somethings we simply learn with time, and adolescents going through high school and college already have so much on their plates. How can we provide narrative reports to students/parents with administrative support from universities? The quality of Letters of Recommendation remain the strongest reason for acceptance into universities. Why not make this a practice for grading?

I’m sure many of you have been on the unfortunate side of the learning/grading divide as well. Comments please!

Grades: A new four-letter word.

Hello. My name is Cody, and I survived being graded.

Not only did I survive grades, but I have come to appreciate them. Apostasy in contemporary pedagogy circles, I know. Before you click away, I will remind my readers that my posts so far regularly feature my admission of a sickly, broken education system, both K-12 and in higher ed. I, too, want to see reformation happen, but my end goals are different than what we have read and watched in class so far. So much of what we have covered seems to be coated with sensationalism and revolution, yet, I daresay, there is a lot of momentum to be captured and altered. Instead of thinking about our efforts linearly, in which progress can only be made via a sharp departure from the status quo, we should think radially, in which varying degrees of change can be made without losing ground and with vastly different trajectories.

If people like Alfie Kohn were to have their way, it seems the slate would be wiped clean; a system failed and forgotten. Because there is much wrong with our current grading schemes, we must get rid of it. May I encourage us to instead rethink the purpose of grades and how we evaluate students using a grading system?

Grades are merely one part of a students greater portfolio. One part. I have yet to experience or hear of an instance in my circles in which GPA has been the guiding factor for an individual’s success in the job market. However, I am all too familiar with the role it plays in evaluated potential employees or incoming graduate students. Again, it is one part.

As an example, my undergraduate degree is in Animal Sciences. My final GPA at graduation was deplorable. If I was to be solely evaluated according to my GPA, I would never have gotten into graduate school. It turns out that working in the horticulture department and making connections with the faculty made much more of a difference than my grades ever could. The professors I worked with knew my work ethic and my knowledge in the field. I had no issue getting into the program because I had many references vouching for me.

This brings me to my next point: Grades help to guide. Or at least they should. In my case, it was abundantly clear that animal sciences were not my passion, nor did I have aptitude for chemistry (organic, biochem, etc) and genetics, as reflected by my performance. Instead of kicking against the goads, I took class feedback, i.e. grades, to heart. The classes that I performed well in and understood on a deeper level are responsible for me successfully choosing my master’s program and now PhD program. My master’s GPA was much closer to 4.0, and my PhD GPA is higher still. I take these as indicators that I am on the right path. My aptitudes and desires are aligned with my professional path. Grades can be good.

I am in full agreement that our grading system needs revamping (in large part because of grade inflation and its current ineffectiveness), but stop short of calling for its removal altogether. Grades are one part of evaluation, both of oneself and by a potential employer. School performance helps others understand your ability to rise to a task and achieve goals. Again, grades are just one part of a diverse portfolio.

Before leaving this post, I do want to comment on something that irks me in some of our readings. Speaking of Alfie Kohn again, much of the contemporary pedagogy rhetoric aims to belittle those that are not onboard the progressive train. Case in point, in “The Case Against Grades,” Kohn uses such statements as: “Why tests are not a particularly useful way to assess student learning (at least the kind that matters), and what thoughtful educators do instead, are questions that must wait for another day.”1 Or elsewhere alluding to “responsible” educators as those that agree with his viewpoints or the “best” teachers as those who don’t give tests. It is counterproductive to insult or insinuate against those that you are trying to persuade.

————————————-

1Kohn, A. 2011. The Case Against Grades. Effective Grading Practices 69(3):28-33

1 2 3 4