Category: Week 5 – Assessment

It’s very competitive to get in a top ranking university in China. The admission is only based on one examination (college entrance examination), and there is only one chance to take the examination every year. If you cannot get a good score, you have to be back to high school and wait another year. It’s so cruel and you will never want to fail the exam. There are lots of debates about college admission in China because it seems not reasonable to make decisions based on one examination. But it is how it works now. Based on this situation, grading system is so popular and almost every middle school and high school use the grading and ranking system to evaluate students’ academic achievements.

 

I have very complicated feeling for grading. When I was in high school, every students received a tiny piece of note after midterm and final exams. On the note showed the score and ranking of each subject in the class and the whole grade. And the school asked us to show the note to parents and got them sign the note. I always feel extremely anxious before I got the score ranking note. If I got a nice ranking, it would be very encouraging since the hard work got the gains. But if it is not very satisfying score, I always had a lot of emotions aroused. For one thing, it’s disappointing because it proved a poor performance to some extent. Secondly, there was pressure from my parents. Moreover, everyone is worried about the college entrance examination and want to be completely prepared for it. Overall, I am not big fun of this kind of assessment method. But I cannot say I hate it because it could help me know how I was doing for each subject and make a specific goal for next time.

 

I must say some learning process is not fun at all. Especially for the subject which there are right answers for every questions. Then the teachers need to score the homework and exams. But what happened to the subject which there are no correct or wrong answers? “Learning scientists confirm that it is relatively simple to test for subject matter content recall and difficult to assess independent critical thinking and creativity.” As illustrated in Marilyn M. Lombardi’s article. I used to be reluctant to accept the scores on thesis about history or culture topic. Because there are no correct viewpoint. I do not think the teacher or anyone has the right to say it is good or not.

 

Grading system improve the performance for some people in some subject. On the other hand, it also reduce the motivation for those who cannot adapt to the system. It is like a giant monster which is chasing after me. Sometimes it makes me want to study and work harder to gain a better ranking. But sometimes it makes the whole thing so scary. For the teachers, I would suggest use it carefully and deferentially.

This week’s readings along with the two Dan Pink videos made me think about the role of incentives in human life. It is widely accepted in Economics literature that incentives leads to a positive impact of performance (See Note 1). The authors show that the impact of monetary incentives lead to a positive impact on performance. Recent research has, however shown that this does not hold in all cases. The negative impact of incentives has been demonstrated in pro-social behavior (Note 2), online communities (Note 3) among others. A variety of explanations have been forwarded to explain the negative impacts of incentives including signaling, types of markets, driving out of intrinsic motivations etc. In light of this evidence, Dan Pink revisits a study conducted by Ariely et. al. (2009) (Note 4 for anyone who wants to take a look at the study). Participants of the field experiments conducted in India were assigned to tasks that involved creativity, memory or motor and skills. Participants in the low and mid incentives were found to perform better than those in the high incentive condition. The authors also established that a propensity to choke was not the causal factor behind this phenomenon (Shout out to Carlos Mantilla who brought this up in class).

At this point I want to point out the two key takeaways from this study. Firstly, the authors show that there is a non-monotonic relationship between performance and reward (specifically very high rewards leads to diminished performance). Secondly, for tasks involving cognitive skills, high levels of incentive decreased performance.

Although taking the lessons of this study and applying it directly to assessments in classrooms is a bit of a stretch but I’ll try anyways. My personal experience with grades is that they have to be used appropriately. I do not like to be told and reminded daily about my grades and my rank in class (a system prevalent in most Asian countries). On the other hand in classes where grading scheme is simple, I have found that I tend to work less and take away fewer things from the course. A case of no incentives and very high incentives (or at least sensitivity to incentives), in my opinion, are equally bad. Being an ex-student of economics, I cannot argue for doing away with all forms of incentives. However what I would argue for is to make people and organizations less dependent on numbers generated through tests as a measure of quality. A quality of a student cannot be gauged simply through a number assigned to him/her. The scores assigned to us stay with us throughout our lives and in many cases have a larger impact than we realize. In India, student suicide claimed the life of over 48,000 individuals between 2010-2015 (Note 5). Academic failure was one of the primary reasons. Data collected from American students suggest that, about one-third suffered from depression (the study does not specifically point to grades as a causal factor) (Note 6).

All I am saying is we should not disincentivize people’s lives in order to provide incentives in their academic ones.

Notes:

1. Ehrenberg, R. G., & Bognanno, M. L. (1990). Do tournaments have incentive effects? Journal of Political Economy, 98(6), 1307-1324. doi:10.1086/261736
2. Ariely, D., Bracha, A., & Meier, S. (2009). Doing good or doing well? Image motivation and monetary incentives in behaving prosocially. The American economic review, 99(1), 544-555.
3. Sun, Y., Dong, X., & McIntyre, S. (2017). Motivation of User-Generated Content: Social Connectedness Moderates the Effects of Monetary Rewards. Marketing Science.
4. Ariely, D., Gneezy, U., Loewenstein, G., & Mazar, N. (2009). Large stakes and big mistakes. The Review of Economic Studies, 76(2), 451-469.
5. http://www.hindustantimes.com/health-and-fitness/every-hour-one-student-commits-suicide-in-india/story-7UFFhSs6h1HNgrNO60FZ2O.html
6. http://www.apa.org/monitor/2014/09/cover-pressure.aspx

 

P.S. This was a late post worthy of a red card.

Source: http://www.glitters20.com/quotes/category/funny/funny-soccer/page/3/

I want to call attention to a paper that is quite related to this week’s readings:

• Rogers, Todd, and Avi Feller. “Discouraged by peer excellence: Exposure to exemplary peer performance causes quitting.” Psychological science 27.3 (2016): 365-374.

It is actually discussed in more layman’s terms here (good for me as I am far from a psychologist).

We saw in most of the readings, and the TED talk, that the traditional carrot and stick method harms creativity and performance in anything other than mechanistic tasks. Also, grades force students to narrow their focus onto achieving high marks rather than learning, and can even average grades can discourage students expecting (especially if the peers do better). The Rogers et al. (2016) paper goes beyond that, suggesting that even without grades, seeing their peers do better on a graded assignment harms performance.

The authors actually ran two experiments for this paper. In the first, they collected grade data from a MOOC they offered. The course assignments included writing essays, as well as reading and grading their peers’ essays. This allowed the authors to track the “quality” of each individual as well as the quality of each peer essay read by each individual. They found that students exposed to excellent mid-term essays generally turned in less impressive final essays. These same students were even less likely to finish the course than those who graded poor peer essays. Their second experiment involved writing and grading SAT essay questions, and again those exposed to high quality writing reevaluated their own skill and were less confident in their abilities. Moreover, they were more likely to feel that essays were an unfair measure of skill, but still felt ashamed by their lack of success. The conclusion they drew was that even encountering superior work can have an incredibly demotivating effect.

To be honest, I have felt this a lot. Obviously imposter syndrome is a big part of graduate school, and we are bombarded with exceptional work all the time (“read this Nature paper while you’re trying to publish your masters work”). But I would suggest it is even worse in an interdisciplinary PhD program. Most of my colleagues have different backgrounds, and have many skills far beyond my own; I’ll never compete with the computer scientists in writing code or the statisticians in model design. Being forced to face this routinely, especially in our department seminars, is often crushing. I spend half of those seminars barely able to follow along, even when listening to junior colleagues – it took a year before I realized some of the other students were just as lost when I gave my talks.

Accordingly, even if we eliminate grades in favor of more detailed evaluations, the competitiveness and the feeling of self-doubt will still haunt many students. It makes no difference if it is a graduate seminar, or a grade-school arithmetic problem on the chalk board. I don’t know how we can get away with this.

Incidentally, about 2% of the 150,000 registered students completed the MOOC, which lends support to the opposite argument: a lack of incentive to finish (and penalty for failure to do so) totally guts motivation.

Second point, Alfie Kohn’s essay was fantastic. I want to agree with virtually every point, but as one might have guessed, I feel stirrings of pessimism. One must never forget that a student’s entire career depends on these grades. You can eliminate letter grades, and replace them with the analytical grid Dr. Elbow described, or you can use detailed assessments. All good in theory, but if an employer, college, grad school is going to base hiring or admissions decisions on those assessments, you can be assured that the student will focus on optimizing them, rather than learning.

In centuries past, it may have been the case that most students attended college pursuing a classical education simply for their own benefit. But today’s middle-class is generally built upon these degrees. The quality of the school and the “rank” of the student are literally of life-affecting importance. Until this changes, students will always prioritize ranking over learning. I don’t agree with Elbow that they want to be ranked, it is simply inevitable.

There are few things in life that confound me more than humans choosing to do something – often repetitively – when they know the outcome will be unfair, unreasonable, unrewarding and/or unrevealing. Grading performance in an academic setting is one of those things. Alfie Kohn’s arguments for re-thinking assessment are both sound (research-based) and logical.  One particular revelation struck me as being at the heart of the matter: Maehr and Midgley’s (1996) observation that “an overemphasis on assessment can actually undermine the pursuit of excellence.” Kohn himself summarized the issue even more clearly: “the more students are led to focus on how well they’re doing, the less engaged they tend to be with what they’re doing.” Isn’t this completely counter to what is intended to be accomplished in the education endeavor?

This observation is reflected in a few of the blog posts others wrote reflecting on this topic:

Vanessa Guerram did an interesting post on the difference between educating for the workforce (filling a need of society) versus educating to empower [the individual].

The notion that somehow economic security trumps individual fulfillment has confounded me for years. And as Vanessa eloquently acknowledges “if education systems focus on students’ learning experiences, education will be about empowering students so they can find the tools they need to make their difference in the world.”

If students are able to focus on becoming experienced – possessing both skills and understanding – won’t they be productive citizens and contribute to the greater good of society?

Jaci Drapeau finds Kohn’s arguments limited and more synergy with Elbow’s argument for “more evaluation” which focuses on growth of students’ abilities or sophistication.

I appreciate Jaci’s reading of Elbow. Through her blog post the clarity of the language used – evaluation versus assessment – was more clear after contemplating it from her perspective. Evaluation, in Elbow’s work, is akin to an apprentice relationship: the student learns from a master and receives guidance, crtique, and challenges along the way to refining one’s craft/understanding. Through the experiences one learns and becomes more capable of applying understanding to new situations, problems or innovation.

Lauren Kennedy questions the feasibility of an alternative evaluation system (narrative) in the context of a system that is based upon summative assessment and grades quantified in numbers.

Lauren’s contemplation of where would the education system be without some sort of number-based assessment system was also clarifying. It helped to see the existing system as serving its own needs rather than those of the individual “engaged … in what they’re doing.” (Kohn

Becoming a real _______ [can be filled with any occupation or title] requires the development of the skills, mindset and demonstrated proficiencies that are expected within the field(s). And, while we have established systems where grades represent progress toward becoming real, they rarely reflect ‘real’ anything.

Feedback (both positive and negative) is essential to progressing and developing one’s craft.  Few humans exist and work simply for the benefit of themselves. Effective feedback can be liberating for the perfectionist and the individual who is stuck in an unproductive process: it can be the catalyst for new perspective that leads to original insight. Feedback that leads to an informed evaluation of a student’s progress – their growth – through certain carefully crafted exercises/projects/artifacts intended to develop one’s skills, thinking and articulation should be the aim of any educational endeavor. Grades mean next to nothing to anyone involved in the ‘earning’ or ‘distribution’ if there is no meaningful and intentional feedback.

 

____________________________

Kohn, A. (2011). The case against grades. Educational Leadership, 69(3), 28-33. Retrieved from:  http://www.alfiekohn.org/article/case-grades/

Maher, M. L., & Midgley, C. (1996). Transforming school culture. CO: Westview.

Within my teaching program, we have talked a lot about the differences in assessment versus grades. Typically, they are generally thought of as being interchangeable, but they actually mean two very different things. I believe that we should use assessment as a means of seeing where people are at with the material and the concepts being taught. Maybe that is complimented with a grade and maybe it isn’t. However, I don’t think that assessment should always have a grade attached to it. Assessment also may look a lot of different ways. It could be as simple as “write down two things you learned today” or as casual as walking by student groups and listening to what they are discussing.

When it comes to grades, we get into this conversation of what is fair and what is not? I have had classes where the professor curved the class and just as many students got curved down as those that benefited from the curve. Should a student’s grade ever suffer because of how others in the class perform? It can especially be difficult to not incorporate generality into grading when you have a larger class. It isn’t possible to get to know each of the students and to witness their efforts (or lack thereof). When grades are involved, you also take away a person’s ability to freely make mistakes. And that is disappointing when we often learn the very most from our mistakes.

My favorite point made in The Case Against Grades is “grades create a preference for the easiest possible task.” I find myself experiencing this on a regular basis. If the assignment says, “write five pages,” there may be times when I literally write five pages because that is merely what I need to do to get the assignment done. I have to admit that although I wasn’t too keen on blogging for this class, Dr. Nelson has kept her promise that we have freedom within the assignment. This has made it a lot more enjoyable and I find myself not dreading my homework Sundays.

The What Motivates Us video was very interesting. The statement about treating people like people rather than machines and horses really stuck with me. I feel like that is how to be a successful teacher in the classroom. Students typically want to be seen as humans rather than just another body in a seat. Although students have responsibilities they need to be held accountable for, it is still important that a professor recognizes that students have real lives outside of the classroom. Exploring one’s purpose is not an easy feat. The word “purpose” goes so far beyond the classroom.

 

Photos

https://i.ytimg.com/vi/ZlOBdBPrmh0/hqdefault .jpg

https://pics.me.me/what-ifi-told-you-that-getting-good-grades-doesntmean-that-13915710.png

The theme of this week is assessment. This is a topic that brings out many diverse viewpoints, but it does seem that the majority of people agree that what we are doing right now is far from ideal. With traditional methods of assessing student performance, teachers promote the competitive culture of the modern education system and actually demotivate students. I can personally say that I am one of those students who was never really motivated by grades, much to the dismay of my mother. I would often get poor grades in classes, even though I would have high test scores. I never really understood why people were so motivated by grades. I enjoyed learning and often would loose points on assignments that I had done, but just never turned in for a grade. I thought I was a misfit in a group of students who all shared some unspoken understanding/level of content with the way that we were evaluated. It wasn’t until college that I really started to appreciate that I was part of the vast majority of students who really don’t perform well within the methods that are traditionally used to motivate students. In reading the article by Alfie Kohn I thought the quote about grades and testing was rather accurate,

Collecting information doesn’t require tests, and sharing that information doesn’t require grades.  In fact, students would be a lot better off without either of these relics from a less enlightened age.

Alfie Kohn, “The Case Against Grades”

Furthermore, even if these traditional assessment methods didn’t impede learning, they are also very poor indicators of the skills and competencies that students need in the modern workforce. Right now our education system produces students who are really good at memorizing content and regurgitating it in a a well defined setting. In a modern work setting, people need to be able to think critically and solve problems in settings that are often very loosely defined.

The perspective that Alfie Kohn brings to the table in the article, “The Case Against Grades” is very insightful. I really appreciated the fact that he tackled the issue from a practical perspective, focusing on tools and techniques that educators can use to actively begin to transform their classrooms. Change can often seem difficult and jumping into a new teaching and assessment style can seem like a really big undertaking, but I felt that Kohn did a very good job of providing guidance and examples so that teachers can approach the change confidently, without feeling too overwhelmed. I was amazed at finding out how long research has shown that traditional assessment does not produce desirable results. It is time that we begin to shift away from these outdated and ineffective techniques.

I empathize with Kohn’s argument that grades are not necessary with the potential to distract from the larger questions of learning or deeper engagement. Of course some kind of reporting/assessment is required, but the positions against grades overlook their fundamental utility as springboards to larger questions, tasks, and education. For me, grades are merely a piece in my toolkit to foster reflection and meaningful engagement with questions beyond mere content.

Kohn’s position on the need to ditch grades on the surface seems compelling, but firsthand experience in my courses suggests otherwise. Rather, grades play an important role in shaping my pedagogy, namely less as a disciplinary tool (i.e., to impose a rationalized order), but as a starting point from which students are encouraged to move beyond. Students want – and need – to perform well in school (and this means assessment), but this is not the sole motivation. No, grades are one of the means to the end I seek – student investment in and reflection on wider questions and issues related to the readings, activities, and even assignments.

Grades are merely one incentive in many, but they are not what holds attention. The actual structure and/or content of the assignments, readings, papers, and tests are what sparks and maintains student interest, and thus offer avenues for deeper engagement. Per our discussion at the last session, I planned an “extra credit” film screening for students in my Arab-Israeli Dispute course this evening. I told students they would earn two (2) extra credit points (out of a total 200 points for the course), if they a) show up (and stay for two hours) and b) write a basic one-page response to the film: Jaffa, The Orange’s Clockwork (a critical analysis of the history, symbolism, advertisements, politics, and historiography of the storied Jaffa Orange).

Now, I underscored to them that the primary reason for the screening was student engagement – I emphasized that the bonus points were merely an added incentive, the real benefit they would gain from the screening was the post-screening discussion. Again, I told them they would need to stay for the entire 2 hours (the film is 1.5 hours), meaning we had thirty minutes for discussion. Several of the students expressed interest, but only twelve (out of 40) showed up. Nevertheless, those who did engaged in a lively discussion and reflection on the meaning of the film to their own interests, the course content, and their lives. In those moments I saw the student facade drop to give way to curiosity, self-reflection, teasing out patterns and connecting aspects of the film to their own lived experiences. One of the students said she would “never look at an orange or any fruit in an advertisement in the same way” after the screening. While this may be hyperbole, the main point was moving beyond course content for wider engagement.

OK, the symbolic two extra credit points brought them to the screening (or perhaps curiosity, I will never know and they may not, either (humans are complex)), but that premise was the foundation for a meaningful engagement. This helped me to realize that grades are not an exclusive incentive, but may be wielded as a salient entry point into wider explorations and deeper engagements that correspond to their lived experiences. While I sought to build this in to all of the assignments, I encountered this firsthand this evening.

I think our task, then, is less to teach “imagination” to our students, but rather to reimagine our own role as teachers who do not just wield grades as disciplinary tools (to impose an order on students), but as avenues to facilitate meaningful student engagements (or “imagination” as a reflexive tool), and to grade accordingly based on those engagements. I recognize this entails a bit more work (for instructors), but ask yourself: what was a meaningful class when you were an undergraduate? What kind of assignment would you like to complete? What are the lessons you hope your students take away from your course? While I fully realize using grades as a basic foundational incentive from which to build toward deeper engagements will not work all of the time or would be more successful in disciplines premised on subjective engagement, the use of grades provides a foundational incentive from which to inspire is an important consideration for student empowerment. To do so, we need less of the absolutist banishment of grades, and more reflection on the ways that grades – and other activities or class sessions – provide gateways to meaningful student engagement that goes beyond course content.

Since the beginning of the GEDI course, we have learned several pedagogical practices to apply as current TA’s or for our future carriers. Form my personal experience, answering the questions What?, Why?, and How? have helped me to understand different concepts and apply theories into real life practices (personal and profesional).

So far, I have enough materials to start answering the questions for the What and Why pedagogical science have the potential to transform educational teaching practices.  In this blog I will share some found responses during the past 5 weeks, and leave open other questions with the purpose to find more answers.

What and Why?:

(To apply)

• Try over and over again till you achieve the objective (Baby Gorge video) to break the barriers of the unkown.
• Facilitate experiential learning to support students in applying their knowledge and conceptual understanding to real-world problems and situations.
• Use innovative tools to engage the imagination, such as: digital sources, social media, blogs, twitter, videos, etc. to engage.
• Promote curiosity, practices related to noticing new things and drawing distinction, new ideas, to encourage mindful and avoid mindless learning, among students.
• Avoid incentives when rules are not clear or there are not a single solution or clear tsk . On the contrary, motivate to achieve autonomy on learning, promote desire of getting better and better to reach mastery, and seed purpose to for something larger.
• Think about the grading system and evade diminishing student’s interest, preference of the easiest tasks and low quality thinking, that often performances are to reach a certain grade (extrinsic). Contrarily, motivate desire to learn for its own sake (intrinsic).

How?:

(To get it done)

The hands-on moment! Answering to this will help to apply theories and concepts in authentic circumstances. Literature suggests some answers, However I still have doubts under this question.

• How to defeat the grading system? How to assess students performance being fair? How to ensure the desired level of knowledge among the students? How to create the right environment to impulse students creativity and desire to learn for its own sake?

Any thoughts????

About the Author

Sofia Rincon Gallardo Patino, is the secretary of the LAIGSA (Latina American and Iberic Graduate Students Association) organization at Virginia Tech. 

 

I am very passionate about this week’s topics. To me, while grades play an important role in the education system, their role can also be misinterpreted to generate negative outcomes from the educational system. Grades have always been a good part of my education, and I have always link my progress of a class to the grades I make. Sometimes people complain that grades are not a reflection about how much you know or understand the class material, or how much you have studied int he process.

To me, grades play an important role in assessing how much material that your instructor wants you to, are you able to retrieve in a specific manner. i think grades, in general, are a good self indicator of your knowledge or an assessment of where do you need to work more. Getting a B in a test, to me, indicates that there is some material that I have not learn it yet, or that I need to revise it again. To me, this is a self-reflection process; which means that you can reflect and learn from a grade to improve in your next challenges. I think that grades help one-self to address issues for improvement.

On the other hand, grades have taken a different role, and they have become the way to evaluate and compare the final knowledge/outcome of an individual in comparison to the rest of the classmates. Nowadays we have distributions about how many A’s or B’s should a class have. It may seem that grades have taken over the overall education assessment.

Grades should play as an external motivator agent for the individual to self-assessment rather than for group placing. There is a case in Quito, where in 2000, an elementary school tried and experimented a new methodology where motivation came from the thrill of playing and not so much for the grades of tests. This was an experimental school where kids learned about math by playing to go grocery shopping, or learned language and grammar by writing letters to Santa, or play that their job is to respond to petitions, etc. This was a successful experiment, and the school became famous for it pedagogical methods. Perhaps it was too futuristic because no other schools adopted such methodology, but that school was recognized for its positive approaches to student learning.

Similarly in the real life work. There is the case of the Chilean State Bank that replaced the Human-resources department for a Happiness department. The goal was to reduce the turnover they were facing due to stress and other negative effects typical of banking jobs. More info here (oops, ti is in Spanish): https://massnegocios.com/rodrigo-rojas-foncillas-gerente-de-felicidad-bancoestado-microempresas-s-a-chile/. The results were splendid! The new purpose was to understand people and support them as part of a family, and not to treat them as workers who must finish specific tasks in a certain time.

Perhaps it is time to rethink grades from rewards that “narrow our view and let us focus to achieve it quicker; into creative, conceptual kind of concept.

Ah, assessment. While complaining about my two-semester assessment course all of last year, I actually appreciate it today. I don’t quite grasp assessment enough to make a career out of it, but I know now that a career in higher education requires an understanding of assessment. And I don’t just mean knowing different research methods or spitting out definitions of external and internal validity. I mean understanding that assessment can be flawed, biased, and not necessarily helpful to student learning.

Assessment also comes in many forms. It’s grades, GPA, student demographic data, student engagement, student satisfaction, student involvement, etc. While it’s critical to assess all of these things and more, it’s important for educators to fully comprehend such assessments. To me, the biggest concern with any form of assessment is that a student is producing some outcome, whether it’s a paper to be graded or a answering questions for a qualitative interview, and that outcome can’t ever fully explain student learning. I get that it’s near impossible to perfect measuring something like learning, but I know we can do better.

I’ve loved readings and videos this week about motivation, because so much of learning comes down to motivation. A student may get a perfect score on a test, but if they weren’t genuinely motivated to learn the subject, it’s likely that they’ll forget all the information as soon as the test is turned in. When we’re “learning” to pass a test, it all comes down to memorization. Sure, grades are a motivating factor for many students, myself included. I work hard for the grade, but I work harder because I truly love learning and I want to benefit from it.

I’ll end with a personal assessment/motivation story: Sophomore year of college, I decided to minor in Economics. I hate math and I’m not good at it. But the conversations I had about economics in other classes intrigued me, and I wanted to pursue it. So I took my first class with a professor who I’d heard was extremely difficult but a great teacher. After struggling the whole semester, and ending with a B, I was confident I could keep going with economics. I suffered through the math components of my other econ classes, but still enjoyed learning the subject. My final semester of econ was coming up and I wanted to take one more class with that challenging professor. People tried really hard to talk me out of it, telling me most people fail or end up dropping. Well, I got a C in the class, barely. But the way that professor taught was more about facing a challenge as best you can than it was about acing the class. He graded tests fairly, and even gave credit to explanations of formulas when you couldn’t remember it. He showed me that learning (and grading) shouldn’t reward regurgitating information, but rather an appreciation of effort and thought. It killed my GPA, but it solidified my love of learning. Thanks Dr. Moul!