Thursday, June 14, 2012

On Entrance Exams


First, a couple of fairly uncontroversial observations.

  1. There will always be stiff competition for entry into elite colleges -- even if India creates 50 IIts, there will still be a scramble for a seat at, say, IIT-B (going by current trends). With stiff competition, there will always be a need for some filtering mechanism.

  2. A mechanism that considers many aspects of a student's academic (and even non-academic) record, skills, and preparation is better than one that relies on his/her performance in one high-stakes exam.

* * *

JEE is a bad exam in many ways -- this post from six years ago for some of the details. The most important badness is in its insistence on ranking students -- an exercise with little or no statistical basis that produces noisy results with poor reproducibility.

Unfortunately, the new mechanism proposed by the government does not allow us to move away from the statistically bankrupt idea of ranking students. Even if the two proposed national exams (JEE-Main and JEE-Advanced) are standardized, can their percentile scores be expressed with a precision of one in a million?

And, the proposed mechanism is rigid in fixing the relative weights for the three components, and using this rigid formula for ranking students. From this point on, there's no difference at all between the new mechanism and AIEEE (or JEE).

We can do better.

* * *

Here's one possibility:

Let's say there are N exams in all -- and students get a percentile score in each of them. Now, let the individual institutions choose their weights for each of the N exams.

An institution need not stop there -- it can even insist on ensuring a diverse student body by creating state-wise quotas, and quotas for students from poor families, and for women (in IITs) or men (at St. Stephen's).

Each institution can design its policy so that truly exceptional students (top rankers in National Olympiads, top athletes and sportspersons, accomplished artists, ...) are admitted (as long as they fulfill a certain minimum threshold, and as long as these special talents are backed by objective measures).

This sort of stuff would be horrendously complicated if it were to be implemented manually; with computers, it's just a matter of coding an institution's admissions criteria and let them do the hard work.

* * *

Articulating a vision for your institution's student body and devising an admissions policy that helps you get there are at the core of academic freedom and autonomy.

* * *

All our top institutions have a duty to participate in these policy debates to steer them to a broadly acceptable outcome which gives institutions a viable way to exercise their autonomy.

* * *

Ah, yes, one-handed typing is a bloody pain!

8 Comments:

  1. pramod said...

    I'm not convinced any fixed formula that weights a bunch of semi-random numbers is all that much better than the current system.

    I think what many of us really want is an admission system that exercises judgement in such a manner that it results in a diverse, vibrant and well-rounded student body. I think this reliance on test scores, weighting, ranking and whatnot is fundamentally incompatible with this.

    That said, perhaps given the scale of the problem we are dealing with in India, it might be the case that nothing will really do better.

  2. Eosbuddy said...

    The problem, in my view, runs deeper. The standard education system does not teach subjects in their essence (i.e. to solve problems). Typically, a scientific problem (in say physics) may have elements from atomic physics, electricity and magnetism and mechanics bunched together. The system teaches the student each of these units separately without paying attention to making the students grasp the concepts holistically. The pattern of JEE is in this format (at the end of the day, it really doesn't matter whether students are ranked or is based on a certain percentile - there is some mode of cut-off and the scrambling is to make it within this limit). By moving away from the past pattern of JEE, one is entrenching further.

    The fault is actually with the rest of the system and the govt is instead, figuratively saying, ..."rotating the bus to change the wheels".

  3. Pratik Ray said...
    This comment has been removed by the author.
  4. Pratik Ray said...

    Must say I like your idea much better than any of the existing systems or the one proposed by Kapil ji.

    Sorry to read about your hand. Hope it heals fast.

  5. Raj said...

    Your system is way too complicated. I can't imagine the number of law suits during the admission time. You should stop blaming only IITs for the system. India is not ready for what you are suggesting. BTW, can you tell me which computer application you think can do this so easily? I would like to ask who deserves Bharth Ratna more: World chess champion or World cup cricket winner with 100 int'l centuries.

  6. Ungrateful Alive said...

    Prudence entails scaling up and maintaining society's capacity for teaching and training youngsters with the number of said youngsters. (In fact, prudence entails stabilizing the number and fraction of said youngsters in the population.) Indians are anything but prudent and chronically underprovision and overallocate resources. H. G. Well famously said "Civilization is a race between disaster and education" and, in India, disaster has won hands down. With the current level of supply/demand ratio, no amount of rearranging the deckchair on the Titanic will matter.

  7. Vijay said...

    “ This sort of stuff would be horrendously complicated if it were to be implemented manually; with computers, it's just a matter of coding an institution's admissions criteria and let them do the hard work.”
    Dear Abi
    How do US undergrad institutions sift through stuff? Not always by giving weights and numbers and adding, I suspect: There is a very substantial human-interface, sometimes with early-stage triaging farmed out, if I am not mistaken.
    There is an element in US admissions that is also similar to what you suggest. As an applicant, I load all my SAT scores, school scores, tests and essays, my musical and social-service quests, my minority status etc. on to the common applications site: This can be accessed by all colleges I apply too. In addition, each college I apply may demand specific other material. That too is uploaded to this site with access to those colleges only. Georgia Tech uses information to take a decision in a rapid IIT-JEE type manner. Others a mix. Yet others want to get you over for an interview. For a few million dollars, the Bush-Inderesan option is always available somewhere too :-)

    With close to 300 million humans under 15 years of age, we (India) need to have an umbrella philosophy of access under which a thousand systems can flourish. Happy to be corrected!

  8. Dheeraj Sanghi said...

    @Abi, the software is easy to develop. Last year, I was working with two students to develop the same. We had to give it up because of something non-technical. But a naive implementation would be to have the CCB counselling software and JEE counselling software (and similar other softwares from other universities) run on the same server and share a database.