Impending question difficulty and mastery score calibration

Tomorrow morning, I am planning to run a full recalibration of the question difficulties used by the mastery system, together with a full recalculation of everyone’s mastery scores based on these updated difficulties.

When the mastery system was first introduced, the difficulty of many questions necessarily had to begin with my own estimates. That was a reasonable starting point, but now that a much larger number of students have attempted the questions, we have enough real performance data to increasingly let the model estimate question difficulty from how students actually perform.

After all, you probably don't want your mastery score to depend purely on what I happen to think is difficult — however impeccable my judgement may occasionally appear to me. 😊

I expect to carry this out at around 6–7am UK time tomorrow morning. In theory, there should be no disruption to the site at all, but I am deliberately doing it at a quieter time just in case something unexpected happens.

Please expect your mastery score to change a little after the recalibration, although I do not expect the change to be substantial. Whether it moves up or down, just remember that the change should be taking it in the direction of a more accurate estimate! 😊

I also plan to publish model-implied grade boundaries for both the Tyler Tutoring mock papers and my own JZ Maths mock papers tomorrow, so you will be able to get a rough idea of how a raw score on each paper corresponds to the current TMUA score scale.