Piecing together the PSLE puzzle

There has been a lot of debate over the PSLE lately and people were wondering if this sacred cow would finally be slaughtered, until PM came out and put an end to that speculation. He said that instead, MOE will look at ways to reduce the pressure on this one single exam.

I've long been meaning to write a piece on the PSLE and this is as good a time as any.

The dastardly bell curve

To me, the main reason why PSLE is so stressful is the bell curve. There's a lot of misconception and confusion as to how the PSLE t-score is calculated so I'll try to simplify it here.

Each subject - English, Chinese, Maths and Science - is weighted equally at 100.  Your t-score, however, is calculated on a bell curve.  What this means is that your t-score for each subject is based on your relative score to the cohort and if you are right smack in the middle (exactly average), you will get a t-score of 50 points for that subject.

For example, if for that year's Science paper, the cohort's average was 75/100 and you score exactly 75/100, you will earn 50 points.  If you perform better than or worse than 75/100, you will score more than or less than 50 points respectively.  How much more or less depends on the standard deviation which reflects the spread of marks across the cohort.

If you want a more detailed explanation on how t-score is calculated, here's quite a good one.

What this means is that if your child is Joe Average and he scores 50 points for every subject, he will get a total t-score of 200.  Which doesn't sound like much to most parents, I know.  They don't realise that by scoring more than 200, their child is already above average. That's why I feel that PSLE is harder than many parents think. We hear of the 260 scorers so often that we don't realise these are by far a small minority and we start imposing unrealistic expectations on our own kids.

Using t-score is a fairer form of assessment than using raw scores because it takes into account variables like difficulty of the papers. However, adopting the bell curve gives rise to very undesirable traits.  It's as ruthless as Jack Welch's business approach to GE - rank every employee and every year, get rid of the ones at the bottom. It's based on the principle that in any sample population, there will always be someone under performing.

I disagree with it in the business context and I find it detrimental in the school context. Ranking fosters competition and discourages collaboration because it becomes a race where it's about outrunning your rivals rather than performing to the best of your ability.  I'd rather score an A and have my schoolmates score Bs than score an A* along with everyone else. This is what we want to teach 12-year-olds? That I'm a champ if I pass and my neighbour fails? Talk about screwing teamwork and cameraderie.

We are encouraging kiasuism by creating scarcity, explained very well here by an economist blogger.

Even putting aside the ugly competitiveness, the bell curve has severe limitations. Anecdotally, we know that kids are getting "smarter" in the sense that parents are getting more involved and investing more resources into their kids' education. Kids are studying harder, always looking to beat the exam system. By this logic, as a general population, our kids should actually be learning more in this generation than the last. I'm pretty confident the average student today will outperform the average student 20 years ago.

The bell curve does not reflect this as it merely imposes the same distribution on every population. In other words, I suspect that cohorts have been improving as a whole over time but we will never know for sure as the bell curve cannot measure between cohorts.

Why is this important? Simply because the PSLE t-score then is NOT indicative of real ability, only relative ability. Yet the cut-off point for the Express Stream, has remained at 188 for a long time now (I don't have the information for when this score was introduced). This is an arbitrary figure at best - a child who scores below 188 today may be more suited for the Express Stream than one who scored below 188 ten years ago. If our education system is as effective as we like to claim, should we not make room for the possibility that more kids are learning at an acceptable level?

In an environment where kids tend to internalise the t-score as a measure of his or her ability, the bell curve generates a lot of anxiety and can fuel low self-esteem.  All this from a mechanism which, when you break it down, exists primarily to issue you with a queue number to choose your secondary school.

Call for more transparency

The PSLE maths papers were on Friday and Andre came home looking a little deflated, saying the Paper 2 was quite tough. When I say a paper is tough, I mean it is of a similar level of difficulty to those of certain branded schools known for their difficult papers. I want to qualify this because I personally get miffed when I hear that some kids come out of a challenging exam declaring that it was "so easy". It demoralises other kids who may have struggled (or maybe that's their intent) and increases the anxiety of parents. Defining something as hard or easy based on whether you personally can do it is extremely narcissistic. Somebody should tell these folks that standards do not begin and end with them.   

This brings me to my other point that if MOE wants to make the PSLE less stressful, they should start making it more transparent. I find that much of the PSLE is shrouded in mystery, like it's some major national secret. The uncertainty contributes exponentially to the stress because parents don't know what to expect.

Maths has always been one of those subjects where parents fear a suddenly difficult paper. I believe this is the reason why the maths standards have accelerated so much over the years. In an effort to beat the PSLE, schools have been introducing more and more non-standard questions, some of which are simply beyond the conceptual level of many 12-year-olds.

The acceleration is not without cause. In 2009 (Lesley-Anne's PSLE year), the maths paper was unexpectedly difficult, leading to much hue and cry. I suspect it's because 2009 was the first year of the new maths exam format - 2 papers and calculators allowed for Paper 2 and MOE didn't calibrate the difficulty appropriately. 

If MOE had admitted this, I would have accepted it as teething problems. Instead, MOE chose to dish out the standard response to protesting parents, along the lines of "the standard of the paper was equivalent to that of other years'. Parents are just being kancheong because they don't see the whole picture." (Paraphrasing, of course). In other words, we're myopic. And over-reacting.

Usually following such complaints, when the results are released, mainstream media would inevitably report that the complaining parent's kid got A* in maths afterall, making the parent look foolish. I used to think the same way - wah so kiasu. Still A* what, why so kancheong.

It was only after Lesley-Anne's PSLE that it dawned on me what a sham it was. Here's why: in all the practice papers that Lesley-Anne did for maths that year, she NEVER scored more than 90 (which is the stated absolute mark for an A*). In fact, for some of the top school papers, she scored in the range of 70s.

She came home from the PSLE maths exam crying because she said it was MORE difficult than the top school papers. In fact, she left three 4-mark questions blank. You do the math. Even discounting the other mistakes she would most certainly have made, there was no way she could have scored more than 90.  Yet, here's the bombshell - she scored an A*.

Impossible? Hers was not the only case. I know many of her schoolmates experienced the same situation. I think when faced with this, parents are so delighted by the A* they prefer not to question the inconsistency.

There's only one probable explanation for this - that besides moderating the t-score, MOE also moderates the grades. It was like a light bulb went off in my head and set my thinking on a whole new trajectory.  In other words, the cut off score for an A* can vary each year, depending on how well the cohort performs. It's just a matter of deciding what percentage of kids get an A*.

By moderating grades, MOE is effectively hiding your true score and covering their bases. They have always claimed that papers are of an equal standard each year by saying that the number of A/A* achieved remains constant.  But if you control the number of A/A* to begin with, well, then the argument is circular and invalid.

There is no way of proving that I'm right and as far as I'm aware, MOE's official statement is that an A* is 91 and above. But I know that as a whole, parents are not dumb. If many kids across the island reportedly come out of an exam crying, saying an exam was difficult, it is unlikely to be a coincidence and shouldn't be dismissed so patronisingly. If MOE wants to insist that they are right, then I say, prove it by releasing the raw scores for the 2008, 2009 and 2010 maths papers. I would love to know what percentage of kids actually scored less than 50/100 and more than 90/100 as well as the cohort average for the maths exam in 2009. Versus 2008 and 2010.

Do you know that you cannot find a complete PSLE paper anywhere? When you buy the book of past PSLE papers, the questions are strangely broken up by topics, not by year. In fact when you do the questions, you don't know what year each question was from. Again, what's the big secret? What's the harm in letting kids practise the full papers by year? The only reason I can think of is so that people can't gauge and dispute the difficulty of each year's paper.

The uncertainty means there's confusion all around. After the 2009 PSLE maths fiasco, parents went into panic mode. Then the 2010 and 2011 math papers were supposedly easy.  Parents and teachers didn't know what to expect. Do you just learn what you need to? Or prepare for a really tough paper?  As it turns out, this year's was more difficult. I don't think it's anything like 2009's but still, the inconsistency is unsettling.

Educate with continuity in mind

The PSLE is supposed to be a gauge of standards, to prepare you for secondary school. However, I find a huge disconnect between what is taught in primary and secondary school.  For example, in maths, primary school stresses the model method and systematically dissuades the use of algebra. The day you leave primary school, you don't touch models. Ever. Instead, secondary school maths uses, guess what? Algebra.

In primary school composition, you write boring, mundane stories, mostly about accidents, while peppered with lots of fancy, multi-syllabic words. In secondary school, accidents magically become non-existent and you suddenly have to write about things like the ills of computer games.

Many kids who are weak in English and had their t-scores pulled up primarily because of good marks in Chinese, find themselves lost in secondary school because Chinese is only one subject, whereas you need a strong grasp of English to do well in many subjects, especially the humanities.  But of course we all know what happened when MOE tried to reduce the weightage for Chinese at PSLE a couple of years ago.

It's no wonder that many kids who score well for PSLE end up struggling in secondary school. I find it frustrating that we spend so much time preparing our kids for the PSLE which is in essence, merely a placement test. And as it turns out, it's not even a very good placement test.

Now that the authorities have said PSLE is here to stay, I think MOE needs to take a long, hard look at correcting its flaws. By tomorrow, my kids would be done with this trial (hooray!) but for the sake of other kids who have yet to face this challenge, I sincerely hope there is change for the better.