Best,
Sergio

Although evaluating the quality of the work of teachers – as well as that of cops – is very difficult, I agree with you that this doesn’t mean we should just forget about even trying to measure it. My point was mostly that an arbitrary system of assessment is in many cases not “better than nothing” because very often it will not just be “approximate” or “imperfect” but actually consistently punish good teachers and reward bad ones. So we have to be careful about it. Many “systems of evalutation of merit” exist just because some semblance of objectivity is desired or required, but then what often happens is that someone just conjures some metric out of their hats and voila – problem “solved”.

Many institutions, specially universities, use “student evaluations” for the purpose of judging the quality of teachers, but those have their own problems.

In my experience, the real quality of the teaching goes mostly unevaluated in any meaningful way for most teachers most of the time.

One very modern possibility (which many teachers *strongly* reject, which may actually be a good sign that there’s something here) would be to record videos of the lectures by default and make them publicly available (or at least available within the institution). Somewhat like nowadays more and more police cars have cameras on them recording all the action all the time. I wonder how the quality of teaching would change if that were the norm.

Best,
Sergio

Thanks for the comments! Your input is particularly valuable since you have a hands-on experience of what you’re talking about; you’re part of the team of people in whose hands this nation’s education system actually rests.

In my experience I have observed that unfortunately no matter how hard I try, there are some students who *will* unconditionally fail to learn even the most basic material. Now, we can all sit down and argue until we’re blue with ideologues who “know” that everyone is equal whether this this is due bad parenting, lack of interest or bad luck in the gene lottery, but in the end of the day teachers don’t have the power to magically make everyone into an omnicultural insighftul renaissance citizen. In fact this is a completely unrealistic expectation that taxes both teachers and students and prevents real learning from occurring. People are different and they need to be treated differently. The single most damaging aspect of current education systems in my opinion is collectivization. We should instead be helping people to become “all they can be” (sorry for the cliché) in a way that is relevant and appropriate to their needs, not an instrument for constantly harassing and alienating 50% of society by telling them that they’re not good enough because they don’t care or are unable to keep pace with arbitrary intellectual demands made of them while we bore and hold back the other 50% who would be easily able to fly much higher much faster.

We’re like doctors trying to helps minds instead of bodies but when someone comes to the emergency room with a mix of terminal cancer and ebola virus there’s little we can do. Specially when most of the time we’re constrained to giving the same medicine and treatment to everyone! In fact, the teachers who *care* are immensely disturbed by their impotence to help more people, more effectively. But very often it’s completely beyond our control. And unfortunately, unlike what happens with doctors, the results of quackery in education are not so easily observable as a stream of dead bodies; instead we get a stream of dead minds. And a “dead mind” is not easily measurable by any standardized test; people are *not* equally intelligent or intellectually capable. A person can however very well be helped to become more aware and intellectually self-reliant without ever becoming a scientist. This is much more a matter of attitude and insight than of intelligence. This is why *people* are so important in education; unless a teacher can *inspire* a student, the effectiveness of any fancy superstructure around it will very probably be nil. Again, if it was just a matter of throwing information around, it’s already out there and easily available.

So in summary, yes, I agree with you. Measuring quality of education using final grades is very iffy. In fact I would say it’s absurd. In practice it results in punishing idealistic teachers and creating a perverse pressure to just give A’s and B’s to everyone.

Best,
Sergio

Using exams as a way to evaluate teacher effectiveness is very two sided. (This is akin to the proposals brought forth by the Department of Education to use “merit pay” instead of “seniority pay” for teachers.) I work in an urban, Title-1 school where we have been labeled as a “failing school district”. Our failure rates are very high. On the Algebra 1 state-wide assessment, over 85% of the Algebra students that took that exam, failed. While this shows that students were very ill prepared, blame cannot be solely on placed on the teacher. A nurturing environment starts at home. Parents instill certain values about education. When those values are not instilled, we see higher failure rates. The community is not very involved in my school. This is evident during parent-teacher conference nights where we get less than 25% attendance from parents. In fact, every year, I get about 1 – 2 parents who come in to talk about their child. Usually, it’s the parent of one of my students who is excelling (again, the idea of parents caring about their child’s school work.)

Over all, I would say that most of my colleagues are very hard working, motivated, and dedicated teachers. One of my colleagues, who is an outstanding teacher, has been identified as a good “disciplinarian”. She has excellent classroom management skills and is a very effective teacher. As a result, the administration schedules the most problematic students and special needs student into her classes. She does many hands on activities, projects, research projects, etc. and keeps her kids in as best a line as one can imagine given her population. She does, however, has a very high failure rate. Sometimes, student don’t turn in assignments, do poorly on tests, etc. Calling home is minimally effective since some students don’t even live with their parents. They might live with an older sibling, grandmother who is too old to deal with rowdy teenagers, etc… you get the idea. Now, there are other teachers, but few, who do nothing in class. They read the newspaper, surf the internet, and let the students out of class 20 minutes before the bell rings. (To quote one: “How much do I have to pay you to take my kids today? I don’t feel like putting up with them.”) Many of these teachers’ students are making all A’s and B’s. This is not an example of effective teaching practices and tenure protects teachers like these from getting fired. So, according to test results and the rules of “Merit Pay”, one would deem these educators as “Effective Teachers” since their students are “succeeding” on paper. The other teacher with difficult and special needs students would be deemed a “failing teacher” because her students are not succeeding and meeting the standard she sets in class. She would not be worthy of this “Merit Pay”. It is without a doubt that that students in her class are learning a lot more than in the other classes led by the ineffective teachers.

At the same time… there are teachers who do fail every student for whatever reason. Perhaps it is a power trip? I have seen teachers talk about how “stupid” the students are and some of them take great pleasure in failing most of the students. Tests are extremely difficult and homework assignments must be no different – kind of like a college Professor that does not care about pedagogy, but about simply putting the information out there and the ones who are “smart enough” will understand it and regurgitate it on a test.

In my own experiences, there have been a few instances where all of my students did very poorly on a test as a class. When that does happen, it makes me realize that I was not successful at teaching a certain topic. Maybe I went too fast? Maybe I didn’t go over enough examples? Whatever the case is, I stop, review and go over the subject, and let the students have a re-take. If every student were to fail an exam in my class, it is 95% likely that it is my fault. Sure, there are some students who never study and don’t care. With those students, I find the same common denominator – education and school is not set as a priority at home. The lack of parental support really does have undesirable outcomes in a student’s education. Given their background, a good education is probably the main factor in helping the student go on to have a better / more successful life than that of the parents.

In my opinion, *this* is the real work that a teacher should be doing. Not as an “information sprinkler” but as someone who helps other human beings to raise their level or personal awareness. Of course someone who has studied a subject deeply is much better equipped to do so, but this is not done through harassing the students to perform as many tedious and meaningless tasks as you can. In fact, this will most likely produce the opposite effect, the one you mention : functional illiteracy, which is rampant not only among school children but also in colleges and even among graduate students I would dare to say. People grow up in general thinking that the task of a reader ends at decoding symbols into words and sentences and then attributing some kind of very basic and shallow semantic interpretation to it. Then when given a text like say Hamlet to read they are able to decode things like “hmm, now this guy’s dead father appeared as a ghost” and “hmm, now this guy killed some other guy” but still they are completely, utterly, hopelessly unable to “get” what the point is that the author is trying to make, what the plot is, or in fact anything beyond the most basic, disconnected, shallow level of understanding. There is a score of reasons for that, and unfortunately it seems to me that in practice schools in general *encourage* and *reinforce* this result by forcing students to “read” without really understanding and to routinely write about things they don’t even remotely comprehend or care about. And yes, they still have no idea what they’re doing when they finish the assignment.

This reminds me of the work of Mortimer Adler and in particular of his famous How To Read a Book, and this is the reason why I think that *the* point of education is enabling critical thinking.

Best,
Sergio

I put “study” in quotes because a large problem that I encounter is that students DO NOT know how to study. (In fact, I didn’t figure this out until recently.) Students have absolutely no idea how to find a strategy that works for them. I see many students staring at paragraphs in a textbook (in all subjects) hoping to learn through osmosis. They read or re-read texts and have no idea how to comprehend the meaning of what they’re reading. If they do math problems, they do so with great difficulty only to accomplish the assignment from the teacher. They still have no idea what they’re doing when they finish the assignment.

So, this brings up the question of how do we teach students how to study? Different strategies work for different people. How do we teach students to find what works for them?

Well, you have a good point there. The exams do serve multiple purposes, and the one you mention has to be addressed in some way. Unfortunately it’s very difficult to derive clear conclusions about the other side of education – the one complementary to the student’s efforts – from performance on exams. Low performance across the board does indicate problems, but where? The problem could be with the broader structure around the course – maybe the students are being required to take it without having the adequate background. But the problem could also be with the syllabus of the course itself – too much (or badly structured) material, for example. The problem could also be that the exams themselves were not very well designed. And of course, the problem could be that for a variety of reasons the teacher failed to deliver the material properly in the lectures.

My point is that it’s very hard to decide exactly what is going on, and many of these factors are not even under the teacher’s control. Very often, however, for a variety of reasons, it will be assumed by default that it’s the teacher’s fault. Unfortunately, in most academic settings – at least in the US – it is not admissible to just fail lots of people and then say for example “the problem is that students are reaching this course without an appropriate background” or “the problem is that this syllabus includes too much material”. You’re supposed to somehow “do your job”, i.e. deliver a grading sheet that contains mostly – ideally only – passing grades at the end. Anything other than that will probably create a political problem and if you go that way it’s often the case that you’ll be perceived as a troublemaker or even as incompetent instead of the status quo being altered – something that often not even the institution administrators have the liberty to do.

So in summary – although you raise a good point there, and although I agree that the teacher should indeed pay attention to what the exam results are saying about their effectiveness as teachers, and about the effectiveness and quality of the course and so on, most of the time it’s almost impossible for anyone, looking from the outside, to come to any reasonable conclusion about the quality of a course based on the resulting final grade sheet. In most realistic situations, when faced with appalling grades, teachers are often faced with the pragmatic necessity to adjust the final grades so that they are more politically palatable to everyone involved and then just move on. Unless students are given standardized tests held by third parties, I would not rely very heavily on such grades to assess the quality of a course. The fact that grades are supposed to fulfill so many different – often conflicting – objectives at the same time complicates their interpretation.

So just to recap, to include your observation – exams can be used with following objectives in mind, among others :
1) assessing how much students met certain pedagogical goals…
1a) for the purpose of passing or failing students
1b) for the purpose of comparing and raking students
1c) for the purpose of evaluating the effectiveness / quality of the course
2) helping the students to meet certain pedagogical goals…
2a) by making the pedagogical goals more concrete and tangible (i.e. being able to solve the problems in the exam)
2b) by giving feedback to students about how successful they were in achieving the course’s objectives
2c) by giving students a personal baseline against which to measure future improvement

Which underlines the point that designing a good exam is really not trivial.

Thanks for the comments!

Best,
Sergio

Yes, I firmly believe in strategies like letting students “try the same question again” or “show that now you know how to do this”. Part of the reason is that no matter how detailed a syllabus is, a student only really “gets” what they should have learned from the course when they see a test – but then it’s in general too late to go back and calibrate their efforts towards mastering that content. I absolutely do not share the view of many teachers that students should be kept more or less “in the dark” about what will be evaluated in an exam – in fact I believe this strategy to be immensely counterproductive. Answering that “the exam will be all-inclusive” when it obviously and necessarily won’t in the hopes that the students will magically master the whole textbook only manages to alienate them and to frustrate efforts of any deeper understanding. This creates a situation in which they can’t afford to sit down and think hard about any particular topic because that would mean ignoring some other topic entirely. So by doing this we end up with students who have a very superficial grasp of disconnected ideas instead of any real understanding of anything. It is my belief that fair is fair – we should tell them exactly what it is that they are expected to know and then give them as many opportunities as is practical to show that they managed to learn it – without forcing them to take the course all over again or anything like that.

For example, to get a driver’s license you need to take a driving test. Regardless of how many driving lessons you took, you still have to pass the test, and conversely, if you don’t pass the test, you are not *forced* to take more driving lessons; it’s up to you to do whatever it is that will help you to learn how to drive – or at least to develop the skills that are tested in the exam. The test itself is not a “secret” – in fact it’s very public and standardized – you know exactly what it is that you are preparing for. You just have to learn how to do it. So that’s my take on exams – the more students can concentrate on learning to do something instead of trying to guess what will be in the exam, the better. And if they can’t get it the first time, they should have an opportunity to be tested again on the same material. Any good exam should not rely too heavily on the student not knowing the exact topic on which they they will be tested. I don’t think that any question that could be solved by a cursory examination of the textbook is good at all.

There are also other factors at play here, like (as you mention) boosting the student’s confidence that their efforts are actually resulting in something tangible – namely, the ability to correctly solve problems that they couldn’t tackle before. But how can they have this kind of feedback if testing is a one-shot deal? You study, you take a test, you manage to solve some problems, you fumble others – and then you move along? Where is the incentive to better yourself, to actually improve anything? Being given a chance to improve on previous results gives you a much better baseline against which to measure the usefulness of spending energy trying to learn.

As for homework, that’s a completely different game. Unfortunately, the usual situation that students face with homework is as follows : delivering very low quality work that doesn’t involve any real attempt to learn or understand anything is not only very easy but highly rewarded on average. Of course, there are always students who are brilliant, very dedicated and/or adamantly honest that will submit insightful high quality assignments, as there are also cases of students whose work is so flawed that it must be rejected. But most of the time, the opportunities and incentives for just delivering *something*, *anything*, and not learn anything in the process are just too great. The issue is compounded by the fact that producing low quality work, although pedagogically useless (in some cases worse than useless I would argue), still takes time, time that is taken away from actually trying to understand the subject. So it seems to me that letting to student choose how they will spend their time is better, and then we assess if they are meeting the course’s goals with exams. In summary, I am not a big fan of mandatory homework assignments as a tool for either assesment or to encourage learning.

Once again thanks for the comments!

Best,
Sergio