International Testing: What purposes does it serve? : Professor Jim Tognolini International Testing: What purposes does it serve?
The International Programs : The International Programs The Program for International Student Assessment (PISA)
Trends in International Mathematics and Science Study (TIMSS)
PISA : PISA Reading, Mathematics and Science Literacy of 15 year old students across 40 countries
Purpose: How well students are prepared for life beyond school by focussing on how students apply the knowledge and skills that they have learnt in class, to problems presented in real-life contexts.
Paper-and-pencil tests ( computer testing being introduced) with both multiple choice and extended questions.
Background questionnaires about students and about schools.
PISA : PISA Testing carried out in 3-year cycles starting in 2000.
Testing carried out in the primary language of instruction.
More information about the frameworks can be found in the OECD publication: The PISA 2003 Assessment Framework
TIMSS : TIMSS Assesses student knowledge and ability in mathematics and science in Grades 4 and 8.
Purpose: TIMSS is designed to measure trends in students’ knowledge and abilities in mathematics and science and to investigate the cultural environments, teaching practices, curriculum goals and institutional environments that are related to achievement.
Primarily paper-and-pencil tests ( computer testing being introduced) with both multiple choice and extended questions.
Background questionnaires about students and about schools.
TIMSS : TIMSS Testing carried out in 4-year cycles starting in 1994/1995 with 40 countries (now 50 countries in 2002/2003).
Testing carried out in the primary language of instruction.
TIMSS : TIMSS
Purposes of International Tests : Purposes of International Tests High quality objective data for comparisons across time and over countries
Complement the data from national testing programs
Bring together expertise that advances assessment theory and practice
Contributes to the understanding of relationships between correlates of achievement
Contribute to the development of measurement practice that benefits local systems and what happens in the classroom.
Measurement of length : Measurement of length
Measurement of heat : Measurement of heat 40° C 35° C 25° C 30° C 9:00 am 12 noon 3:00 pm
Measurement of mathematics : Measurement of mathematics 2 + 2 = 17 + 29 = 76 –29 = What is the difference between the largest and second largest numbers that can be made from the following digits: 3,7,4 and 6? How can you calculate the area of this shape?
Constructing a test that operationally defines the scale : Constructing a test that operationally defines the scale Test constructors and teachers are challenged by the need to
write items that enable students at different stages along the scale to demonstrate that they have enough of the subject (construct) to correctly answer the item;
ensure that the items are assessing the outcomes for the particular location on the scale;
ensure that as the items are being written, the ones that are intended to be located further towards the top of the scale are, in fact, harder than those that are located toward the bottom of the scale; and,
ensure that the reason that the items are harder is a function of the property/construct that is being measured and not a function of some other extraneous feature (validity).
Slide13 : Mathematics Scale
Slide14 : Mathematics Scale
Slide15 : Mathematics Scale (Easiest Item)
Slide16 : Mathematics Scale (Hardest Item)
Slide17 : Student Report for Reading on the LANNA Test
Slide18 : Class Performance in Mathematics Test
Slide19 : Class Performance in Mathematics Test
(Achievement Order)
Slide20 : Class Performance in Mathematics Test
(Achievement and Difficulty Order)
Types of information : Types of information Item 6 handled badly
Slide22 : Class Performance in Mathematics Test
(Achievement and Difficulty Order)
Types of information : Types of information Item 6 handled badly
Items 11 and 12 hardest
Slide24 : Class Performance in Mathematics Test
(Achievement and Difficulty Order)
Types of information : Types of information Item 6 handled badly
Items 11 and 12 hardest
Overall diverse class: group work with poorer students being supported by the more able.
Slide26 : Class Performance in Mathematics Test
(Achievement and Difficulty Order)
Types of information : Types of information Item 6 handled badly
Items 11 and 12 hardest
Overall diverse class: group work with poorer students being supported by the more able.
Overall pattern is consistent
Slide28 : Class Performance in Mathematics Test
(Achievement and Difficulty Order)
Types of information : Types of information Item 6 handled badly
Items 11 and 12 hardest
Overall diverse class: group work with poorer students being supported by the more able.
Overall pattern is consistent
Student 9 – strange result with easy question wrong.
Slide30 : Class Performance in Mathematics Test
(Achievement and Difficulty Order)
Types of information : Types of information Item 6 handled badly
Items 11 and 12 hardest
Overall diverse class: group work with poorer students being supported by the more able.
Student 9 – strange result with easy question wrong.
Overall pattern is consistent
Student Number 20
Slide32 : Class Performance in Mathematics Test
(Achievement and Difficulty Order)
Slide33 : Class Performance in Mathematics Test
Student Constructed Scale : Student Constructed Scale 1 3 4 5 9 10 2 8 7 6 12 11
Student Constructed Scale : Student Constructed Scale 1 3 4 5 9 10 2 8 7 6 12 11
Student Constructed Scale : Student Constructed Scale 1 3 4 5 9 10 2 8 7 6 12 11
Marking Guidelines are Scales : Marking Guidelines are Scales
Rating Scales : Rating Scales
Future Research and Development : Future Research and Development Professional development programs on interrogating data
Online technology
Online programs for taking raw data and producing feedback
Research on how teachers make on-balanced judgements using evidence captured from various assessments; what evidence they use to support their decisions; how they include decisions based upon evidence in to their final judgments about student performance; and, how these data are then reported to students and parents.