A Testing Experience

I’m sitting in class at the moment with a group of Year 10 students as they do the NSW Board of Studies exam for computing skills. For those that don’t know, the Year 10 Computer Skills Test, or CST10, is a NSW government initiative to introduce standardised testing across the state to measure the ability of our 15 and 16 year old kids to confidently use computers. Every school in NSW has been required over the last few years to ensure that technology skills are integrated – or at least included – as part of the standard curriculum delivery. Our school (and I imagine most schools) have taken an approach where we have looked at the skills indicators (the specific list of computing skills that need to be assessed) and shared them out amongst the various key learning areas according to what we think are the most likely candidates to cover them in an integrated way. So, for example, our science classes try to include database use, maths integrates spreadsheets, word processing is obviously included in a number of subjects, HSIE do quite a bit with presentations, and so on. The basic idea is that by farming out the specific skills to various subjects and making their integration a mandatory requirement of curriculum delivery, that all students should get exposed to all the necessary skills by the time they get to this point in their school life. We also build in some redundancy, so that each skillset is covered by more than one key learning area.

It’s a good idea in theory. Trouble is, in theory, theory and reality are the same thing. In reality, they aren’t.

Since its inception in 2002 (I think?) the Board of Studies has been offering a “pen and paper” version of the test as well as an online version. This is designed to cater to those school who have bandwidth issues, are in rural areas, or for whatever reason decide they don’t want to do the online version. Of course, it should be obvious that a pen and paper test for computing skills is going to be pretty limited in scope. It’s all multiple guess questions, with screenshots of a generic computer GUI and fairly basic questions about how to interact with it. I was actually on the original writing committee for this test and found that we had to continually water things down – questions had to be platform agnostic (couldn’t be too Windows, Mac or Linux specific), application agnostic (couldn’t be too Microsoft Office, OpenOffice, Appleworks, or iWork specific), vague enough that they could be asked in a multiple choice format, and general enough that all kids could have a fair chance at answering them, not just the ones that did computing studies… In other words, the test had to cover all the basics but not really any of the specifics. As you can imagine, the questions that eventually emerge from that process are rather vanilla in nature. The fact that many schools still deliver the test on paper doesn’t really do much to improve the relevance of the testing process.

Mind you, the online version of the test is really no better in the sense that it is just an online version of the paper test… new bottle, same old wine. The online test has the same stimulating screen shots, same thrilling multiple choice question…  it’s nothing more than a whizzbang screen-based version of what is already available on paper, except it take a lot more effort, time and resources to administer. I’d love to see CST10 develop into an actual interactive test where kids could interact directly with simulated apps in a way that provides something close to a genuine user experience…  Don’t ask a kid how to put a border on a table by showing a static image and giving 4 choices about where you’d click to do it – give them a simulated version of a table app and ask them to SHOW YOU how they would do it.

Of course, this is hard to do. It would take huge resources to develop a test like this, and quite frankly I think it would raise all sorts of technical issues… as it was, we had some serious login issues this morning as 80,000 students across the state all tried banging on the Board of Studies servers at the same time. There needs to be a lot of further development to support this sort of load if we are ever to get online testing to work in a big way.

There are still technical issues to solve. But mainly, I’d like to see us solve some of the pedagogical issues of assessing this way. Whether we end up doing via paper or a screen, taking something as practical as computing skills and assessing them using a static, multiple choice method like this is, quite frankly, an insult to our kids.

A Testing Experience by Chris Betcher is licensed under a Creative Commons Attribution 4.0 International License.