Avoiding the False Negative: Placing Students Into Mathematics Courses According to Their Abilities : Avoiding the False Negative: Placing Students Into Mathematics Courses According to Their Abilities Margaret J. Hager, EdD
AMATYC 32nd Annual Conference
Cincinnati, Ohio
November 2, 2006
My Motivation : My Motivation Many stakeholders believed the advising and placement systems needed improvement
It was important to find out the perceptions of the institution so that recommendations made could be successful
Webb’s Chapter “focuses on theory building through a critical evaluation of current efforts in mathematical assessment” (p. 661).
Relevant Research : Relevant Research Many studies were quantitative in nature
Reliability of placement tests: Bridgeman & Wendler (1989), Johnson (1984), Latterell & Regal (2003), Sawyer (1996), Sworder (1986)
Cut-off scores: Abraham (1986), Hughes & Nelson (1990); Morante (1987)
Use of Multiple Measures: Armstrong (2000), Askt & Hirsch (1991),Bridgeman & Wendler (1989); Hoyt (1999), Jenkins (1989), Jones (1997); Jue (1993); Morante (1987)
Relevant Research (Continued) : Relevant Research (Continued) Revak, Frickenstein, & Cribb (2000)
A small amount of studying prior to taking the placement test resulted in significantly higher placement scores.
Relevant Research (Continued) : Relevant Research (Continued) “An assessment involves a situation, responses to that situation, analyses of the responses, interpretation of the results, and interaction among all of these. . . .” (Webb, 1992, p. 680).
Problem Statement : Problem Statement Why do so many students place into Preparatory mathematics courses even after multiple years of high school mathematics preparation?
Problem Statement (Continued) : Problem Statement (Continued) Are there any recommendations that might be made to the mathematics placement process to help ensure proper placement of students into mathematics courses?
Problem Statement (Continued) : Problem Statement (Continued) Guiding Questions:
1. Why do students not practice before taking the mathematics placement test?
2. Why do students who place into a lower level mathematics course often choose to take a higher-level mathematics course?
3. What other measures of student preparedness for mathematics courses are possible in addition to the one placement test score?
4. Should mathematics placement be mandatory?
Method : Method Student Participants (15):
Semi-structured focus groups and individual interviews
Current students
Students at Orientations
Students appealing their mathematics placement
Method (Continued) : Method (Continued) Advisors/Administrators (A/A) (20):
Semi-structured individual interviews
Faculty and Staff Advisors
Placement Test Coordinator
Assistant Dean of Student Services
Placement Test Proctor
Division Chairs
Method (Continued) : Method (Continued) Further Data Collection
Unobtrusive Placement Testing Observations
Document Collection
Compass/ESL Manual & Practice Test packet
Orientation & Placement Test Proctor Scripts
Various Reports/memos on placement test issues
Method (Continued) : Method (Continued) Data Analysis
Individual/focus Group answers coded
Triangulated with field notes and documents
Results : Results Compass/ESL Process
Computer-adaptive
Students instructed to use only the on-line limited function calculator
Average of 7 questions per testing domain
All students began in Algebra domain
Students not able to check their work
Average time spent was approx. 16 minutes
Results (Continued) : Results (Continued) Accuracy of Mathematics Placement Test
Majority of the students and A/A believed the results were inaccurate
Results (Continued) : Results (Continued) Misconceptions Regarding the Placement Process
A common belief of A/A was that students did not need to study before taking the test
Results (Continued) : Results (Continued) Mathematics Placement Appeal Process
46 in pilot, 27 judged, 81.5% successful
However, there was no formal process that required students to appeal
Results (Continued) : Results (Continued) Six-month retake policy with intervention
Most A/A did not realize the six-month rule was in place
Most A/A believed it was too long of a time-frame
Students were not informed of the need for intervention
Results (Continued) : Results (Continued) Practicing Before Test
Most students did not practice (many wished they had)
Web site with arithmetic problems
http://mathematics.clc.uc.edu/hager/index.html
Need of practice packet
Results (Continued) : Results (Continued) Calculator Use
Instructed to use limited function on-line version
Students wanted their own hand-held calculator
Compass has an approved list of calculators
Results (Continued) : Results (Continued) Multiple Measures
Implementation was a concern
Time and effort in gathering data
Current testing was easy to administer
Results were almost immediate
Results (Continued) : Results (Continued) Mandatory Placement
Many A/A believed this would make their advising easier
Must keep the placement appeal process
Will be pursued if web system can block students from skipping any pre-requisite courses
Discussion : Discussion Guiding questions
1. Why do students not practice before taking the mathematics placement test?
Students had no information on what to study
They were not made aware of the importance of the test and therefore, did not take mathematics placement test seriously
Discussion (Continued) : Discussion (Continued) 2. Why do students who place into a lower level mathematics course often choose to take a higher-level mathematics course?
They did not believe the results of the mathematics placement test to be reflective of their ability
Discussion (Continued) : Discussion (Continued)
3. What other measures of student preparedness for mathematics courses are possible in addition to the one placement test score?
ACT/SAT scores • HS math grades
Motivation to do well • HS GPA
Use multiple measures for students near cut-off scores or requesting appeal
Discussion (Continued) : Discussion (Continued) 4. Should mathematics placement be mandatory?
If implemented, it would make A/A job easier.
A/A inherently trusted the mathematics placement recommendation
Web-based registration system unable to handle it
Appeal process should continue
Recommendations : Recommendations Changes in mathematics placement process
Increase number of questions per testing domain to the maximum number (average of 13)
Have students choose the domain appropriate to their level of preparedness
Recommendations (Continued) : Recommendations (Continued)
Allow students to use a calculator listed on the Compass/ESL approved list of calculators
Allow students to retake the test per the guidelines of Compass/ESL Manual
Allow students an additional or optional paper/pencil, machine-scored test as an additional measure of their ability
Recommendations (Continued) : Recommendations (Continued) Five-week co-requisite course
Allows students an option to spending 10 weeks in a course when they only need some review:
An additional one-credit hour course co-requisite with the Introductory Algebra I course
Recommendations (Continued) : Recommendations (Continued) Information Sheets
Describe computer-adaptability
Strong suggestion of reviewing prior to testing
Where to get practice information
List of approved calculators
How to choose which domain of questions they should begin in
How to interpret results of test
Description of the retake and appeal policies
Recommendations (Continued) : Recommendations (Continued) Further recommendations
Pursue mandatory placement implementation
Follow students
What course they tested into, what they took, how well they did
Add Algebra section/questions to practice web site
Limitations : Limitations Limitations
Generalizability: Cannot generalize to other colleges
Low number of placement testing observations
Not all students tested were interviewed
Future Research : Future Research Future Research
Follow-up with student placements
Follow-up with effectiveness of recommendations
Research why students skip pre-requisite courses
Thank you : Thank you