Getting ready for EQAO

I wrote this 4 years ago in a school magazine I was working on.  At the time, I thought that blogging wasn’t for me.   I have been thinking about it since I read a tweet by @rrmurry : “Letting everyone know ahead of time. Standardized test scores this Spring will be VERY LOW. Non-election year & provide “data” to destroy.”  I don’t know if he is correct; we’ll have to wait and see but I just thought I’d post this to see if it was still relevant:

Response to Chapter 6 — by Sharon Murphy “No-one has ever gown taller as a result of being measured.” By Patrick Tucker

Sharon Murphy describes how, while standardized testing has always been with us, it was traditionally considered one piece of student and school assessment (pp. 145).  In the nineties, as a result of pressure from conservative elements in Canada, standardized testing was pushed to its current “high stakes” position where it governs many aspects of school life — from action plans to improve at the school level to graduation in the form of the “literacy exit examination” in grade 10 (pp. 145-146).  The remainder of the chapter describes the 6 lessons Murphy argues should be learned from our experience with standardized test (particularly the EQAO testing):

1. Neither standards nor standardize testing mean excellence or are a guarantee of excellence
2. Test results that are reported numerically, despite the cautions of the test developers, take on a life of their own.
3. Invariably the media will misuse information from standardized testing to manufacture news.
4. In a time of globalization, business interests and business ways of thinking have infused public policy and contributed to the move toward standardized testing.
5. The consequences of standardized testing can have a negative impact on the quality of education…and the effects can be particularly detrimental to children whose race, culture or first language is not that of the majority.
6. The inappropriate implementation and interpretation of standardized testing has allowed politicians to misguide the public, a consequence of which is the destabilization of the education system.

In her article, Sharon Murphy presents that there are 2 ways to standardize a test: “a) how the test is administered…  b) how the test is constructed – usually, standardized test are scored by comparing the performance of persons taking the test to the performance of a comparator group who initially took the test (Murphy, page 148).”  In regards to the EQAO neither of these standards seems to be rigorously pursued. 

Though both methods of standardization seem lax in regards to the EQAO test, my primary concern is the second standard.  EQAO has no norms other than the curriculum; however, I would argue, standardized concepts doesn’t equate to standardized measurement, how the question is asked, will affect determined achievement. 

I’m not talking about the unnatural conditions of the administration; the difference in the rules from everyday instruction, the up to 12 hours of testing done in 3 or 4 days, nor the societal pressure to excel.  I am more concerned with the fluctuations of mastery that will be observed based on the wording of each question.  Without norms, there is no way to accurately judge whether the results are meaningful.

Let us, for example, take black as a concept.  My 2 year old has mastered this concept; he knows what black is.  Now let’s image we wanted to assess that – to see if I am failing my son in some way (deliberate negative assumption).  Instead of unnatural conditions let’s suppose, I am playing with my son (to be fair of course — rather than having him write a test without assistance or motivation – only pressure); while playing, I hold up a black object, perhaps his favourite “Beep, Beep,” and ask one of the following:

1.  Is this black?

2.  What colour is this: pink, red, orange or black?

3.  Is this purple?

4.  What colour is this?

5.  What colour is this: black, dark blue, midnight grey, or nighttime blue?

6.  Is this midnight grey?

7.  Is this a shade?

8.  Is this a shade or a colour?

9.  How many letters does the colour of this object have?

10. Is this colour the absence or absorption of all visible thermal radiation?

 

The questions, while assess mastery of the same concept, become increasingly more difficult.  Even though my child confidently and independently understands black, he wouldn’t be able to demonstrate that mastery on many of these questions.  Granted this is a problem for all evaluations and assessment, but it is a problem that needs to me more adequately addressed by EQAO more than most tests for 2 reasons.  One, their test holds a particularly powerful place in our society and in our educational system.  Problems with test become bigger as the test becomes bigger.  Secondly, EQAO doesn’t just measure achievement, but it attempts to assert patterns or change over time.  This is particularly difficult when your questions change from year to year and are not normed.  While the concepts stay the same, the questions affect the calibration of the results.  This is not, to my knowledge, acknowledged by the EQAO system.

There are some questions on this year’s EQAO that seem particularly difficult.  Two math questions, in particular, stand out as more difficult then anything I could find in any of the boards approved textbook lists.  I worry a little that this was done consciously and that the resulting lower scores (a predicted result of more difficult questions) will be used to justify a push to increase our math component.  I don’t mean to suggest that increase attention to math is a bad thing for our students but, should that happen, we have to face the reality that our system cruelly exposed students to questions that were too difficult to ‘help’ younger students by forcing change into a system, and not help the ones taking the test.  It seems inconsistent with many of our core beliefs to abandon a group of students, perhaps even damage them to some small degree, to potentially help others.

EQAO has the ability to manipulate perceived achievement (consciously or not) because how the question is asked matters greatly; this is why true, meaningful, standardized tests take the time to create norms.  As a consequence, EQAO is not adequate in projecting trends; because each test is different, you cannot compare the results in any acceptable way; yet we, as a system do, and make policy based on those comparisons.   This problem is something that we really should address as an education system.  How do we measure, not student success, but system success over time?  As yet we haven’t, as a system, had anything even approaching this discussion.  How and when we do, might properly be the focus of the EQAO office.

comments?