???Assessment measures the breadth and depth of learning. It has been criticized as being inaccurate and unreliable, and for distorting both teaching and the curriculum; it is also true that assessment results are notoriously poor at predicting future performance. And yet society and teachers are unable to manage without it. In the right hands, assessment can inspire, motivate, and provide the feedback which is essential for targeting prompt corrective help. But it can also lead us to ignore what cannot easily be measured.??™ (Geoff Petty, Teaching Today, Nelson Thornes, page 449).
In unit 101 we discussed the initial assessment of learners and identified why it was important to assess learners at the beginning of a course of study, ascertain their current knowledge or skills level on the subject and use that information to consider what approaches could best be used to manage the learner??™s progress. This logically leads on to the continual assessment of course work, sometimes culminating in a summative assessment at the end of the course.
Assessment serves many different purposes. It can grade the attainment of learners, help to select candidates for jobs or future courses, contribute to evidence on the effectiveness of courses and teachers, and provide a long term goal for learners. But this applies mainly in the final or summative assessment of a course, which aims to sum up the learners??™ achievements.
The main use of assessment for teachers is the ongoing or formative assessment. This is used throughout the course to form judgments on whether, and to what extent, learning has been successful. Also to pinpoint difficulties so that remedial action can be taken. As summative and formative assessment have very different aims, they are usually carried out in radically different ways.
At this point we need to discuss and understand the difference between norm-referenced assessment and criterion-referenced assessment.
A typical example of norm-referenced assessment would be A-level and GCSE grades. Norm-referenced assessment compares candidates and rewards the high scorers with the best grades. The marks show how the candidate did against the ???norm??™ or average.
Many years ago I applied for a job with British Gas in the computer department and I was asked to come in for an assessment test. What I didn??™t know when I arrived was that there were another 299 candidates all simultaneously taking the same test in the same amphitheatre. The prospect of that would have frightened me in those days as I??™m sure it did many of my peers, but somehow, I was one of the six successful candidates for that year??™s intake. Petty suggests that at least a couple of hundred candidates are needed for accurate norm-referenced results although ???average??™ marks and grades can change from year to year subject to the difficulty of the test set. Norm-referencing is appropriate for developmental rather than mastery objectives.
By contrast, criterion-referenced assessment measures what the candidate can do, awarding a pass if they can do it, and a fail if they cannot. The driving test is a good example. This method of assessment is reliable only if the criteria are well defined ??“ for example, in a checklist, or a list of competences, or a mark scheme. Otherwise different markers will apply different standards, or the same marker may apply different standards on different days to different candidates. Criterion- referencing is appropriate for mastery objectives.
Formative Assessment: using assessment to improve learning
Researchers have compared teaching in the USA with teaching in Japan and elsewhere in Asia. The Asian teachers were much more effective. The researchers concluded that a major reason for that was that Western teachers emphasized aptitude and ability, while the Asian teachers put emphasis on effort and persistence. Another reason suggested was that Asian teachers were allowed more preparation time out of the classroom. (Scientific American, December 1992).
I work in a highly technical field and because of the nature of that work my young and ambitious learners are first looking for understanding of the basic concepts followed by more detailed mastery materials. Although they take a massive amount of time to prepare and test, I produce online multiple choice tests. Introductory tests start with a minimum of ten questions. The idea being to demonstrate how the test works ??“ that is ??“ a question is asked and four answers are presented to the user. The user must select one of the four answers. If the correct answer is chosen, the user is informed of it and the test moves on to the next question. If the user selects an incorrect answer the user is notified of this and the correct answer is displayed. The user can examine the correct answer and take time considering it before moving on to the next question.
Teachers/trainers are often only available during the learning sessions, before class and after class for a short period, and normally only during term time. Questions put online in this format offer learners great flexibility in how and when they can use these materials for revision.
???It is often said that students are not really motivated to learn until they revise for an examination.??™ (Reece and Walker, 2000, p411).
I do not always believe this to be the case. We all live and work under different circumstances. It is unfair to compare a student who works twenty hours a week in a factory to pay his way through college with a student of comfortable means living at home with supportive parents. Generalizations of this nature are subjective and yield little benefit to the discussion.
The most important thing is to make these tools available for regular revision throughout the learning programme. It is good for self-esteem and confidence and gives learners an insight in how to manage online examinations which have become the summative tools of choice in the computer industry.
The ten question introductory tests filter out into larger and more comprehensive tests on a plethora of disciplines. Some modules like Unix printing have fifty questions while modules like shell programming have a dedicated one hundred questions assigned. The beauty of this method is that through the visualization of the answer it assists the memory in identifying complex structures. This method relies heavily on effort and persistence and even with limited access to a Unix computer, the user develops beneficial skills. See appendix 2.
To avoid exclusion of those that do not have regular access to the World Wide Web, each multiple choice question can be printed off. Indeed, by deliberately getting the answer wrong and printing the page with correct answer, learners can review these pages on the train or the bus. Our learners can be incredibly resourceful when needs must.
I have used these materials very successfully in preparing for professional exams which use the same principles for summative assessment.
Formative assessment must be informative assessment
Case study 2
Petty suggests a simple model from which all learners can benefit from formative assessment:
??? Clear goals
??? A medal ??“ positive feedback on what was done well
??? A mission ??“ areas for improvement ??“ and how to achieve that improvement
In a voluntary capacity, I run a speakers course to help people develop their communication skills so that they have the confidence and skills to stand up in any situation and deliver a speech or presentation coherently. A speakers??™ club is a motivational and supportive environment whose principles are underpinned by Maslow??™s hierarchy of needs as well as sitting more than comfortably with Petty??™s virtuous and vicious circles of motivation. (Geoff Petty, Teaching Today, Nelson Thornes, page 47).
The club has a simple philosophy in terms of its membership ??“ nobody is discriminated against in terms of age, gender, race, colour, religion or disability. Our training must be accessible to people from all walks of life. We have what is called the Speakers??™ Guide, our road map to success. The guide suggests a course of ten speeches over a period of time. Each new speech must exhibit the techniques learnt in previous assignments. The ten levels are as follows:
2. speaking with conviction
3. construction ??“ an introduction, a main body and a conclusion
4 gestures and body language
5. use of voice ??“ pace, pitch, pause and projection
6. vocabulary and painting word pictures
7. effective use of notes
9. audience rapport
10. the masterpiece ??“ a speech on a serious subject utilizing all the speaking skills learnt previously.
The first speech called the ice-breaker has three clear goals:
??? Eye contact
Whenever anybody stands up in front of an audience, preparation is vital if the speaker??™s message is to be conveyed coherently.
The speaker must be audible. The best preparation and the most appropriate words to convey that message are wasted if the audience cannot hear those words.
Finally, eye contact must be maintained to form that connection with the audience.
One speech – three clear goals. During the speech, an evaluator takes notes (on a specially designed sheet for that level ??“ see appendix 1) and assesses the speech against the success criteria.
When the speech has been delivered, it is the evaluator??™s role to highlight all of the speaker??™s strong points, identify areas for improvement and make suggestions as to what methods/techniques should be utilized to strengthen the speaker??™s performance next time. It is also the evaluator??™s role to decide if the speaker met the success criteria for the assignment and announce that the speaker should progress to the next level, or if the speaker should retake this assignment bearing in mind the suggested feedback.
Interestingly, people can speak on any subject of their choice ??“ the evaluator only comments on the speaking techniques deployed and not on the subject matter ??“ to do so could generate a fruitless subjective debate would create a negative atmosphere, not in keeping with the spirit of a speaker??™s club.
This is a case of clear goals, a medal and a mission. It is a tried and trusted method spanning back over fifty years and one from which I have personally benefited. Receiving positive feedback straight after a speaking assignment gives a massive motivational boost and spurs progression. After the successful level five speech a speaker is awarded the Interim Certificate of Achievement and after level ten the full Certificate of Achievement. It??™s an incredibly positive and supportive environment, one from which little acorns can turn into mighty oaks.
Internal & External assessment
The City and Guilds 7407 course has an interesting assessment mechanism. Assignments are set, assignments are returned and written feedback is received specifying if the assignment was passed or if it required additional input. Beyond that, I found the tutorial sessions most beneficial. This is 10 month course and on average there are ten to fifteen learners in attendance at the sessions. This puts a lot of strain on the teachers as many of the learners usually have small and sometimes large matters to address with the teachers before and after class as well as in the break. The tutorials have been useful to be able to develop more of a one-to-one relationship with the teachers and discuss matters that one would not normally choose to raise in class. There have been times on this long course of study that I felt somewhat jaded ??“ not so much with the course ??“ but with work, family and business development which sometimes left me with little or no time for study. I found the tutorials really put some spring back into my step, they were highly motivational.
I understand that Laima is the lead teacher on this course and that she moderates Gerry and Shirley to ensure that collectively they have a consistent policy on marking learners??™ work. This consistency is vitally important to any function where the nature of the marking is considered somewhat subjective. It would be unfair (to all the learners) if student A passed while omitting X,Y and Z from a piece of work, while student B was failed for that very reason. Consistency of marking must be available and accessible to all. It gives confidence to the learners knowing that they are treated equally and that the process of marking is transparent.
In February, one of the sessions for this course was attended by a representative of the City and Guilds organization. The purpose was to meet some of the students, review the standard of work so far completed and observe the delivery of a session. Again, this form of moderation gives comfort to the student, knowing that somebody values what we are studying, as well as comparing, maintaining and improving standards throughout colleges in this part of the country. It would be cruel if somebody failed at Bromley, but had they for example handed the same work in at Epsom have passed. The process must have integrity and credibility for it to have value.
Geoff Petty – Teaching Today, Nelson Thornes, page 449.
Excerpt from Scientific American, December 1992.
(Geoff Petty, Teaching Today, Nelson Thornes, page 47).
(Reece and Walker, 2000, p411).
ASC Speakers Guide evaluation sheet
Appendix 2. Online resources: