A Joint MERLOT and European View of Multimedia in Physics : A Joint MERLOT and European View of Multimedia in Physics EPS Multimedia Working Group
MERLOT/Physics MERLOT International Conference 2006
Ottawa Canada, 8/10/2006
Participants : Participants EUPEN Working Group
M. Benedict (Hun), E. Debowska (Pol), B. Eckert (Ger), S. Geiner-Valkier (Neth), H.J. Jodl (Ger), L. Mathelitsch (Aus), I Ruddock (UK), E. Sassi (It), R. Sporken (Bel),
MERLOT
T. Bradfield (OK), T. Colbert (GA), L. Keiner (SC), B. Mason (OK), T. Meldor (LA), J. Rauber (SD), S. Sen (NY), P. Sheldon (VA)
Reviews : Reviews Are we measuring something?
And if so, what?
Comparison... : Comparison... History
Goals & Methods
Criteria
Results
EPS History : EPS History 2002: EUPEN Working Group to Define Quality Standards. Example: Quantum Mechanics (MPTL-7, Parma)
2003: Apply Review Process. Example: Optics (MPTL-8, Prague)
2004: Classical Mechanics (MPTL-9, Graz)
2005: Thermal Physics (MPTL-10, Berlin)
2006: Electricity & Magnetism (MPTL-11, Szeged)
MERLOT Review History : MERLOT Review History In 1996/7 the California State University embarked on a...
Goals & Methods : Goals & Methods Present the high quality stuff, Recognize authors, Identify gaps EPS –
Work within one topic
Collect many resources
Select the best
Apply criteria for full review of best
Write up results MERLOT –
Search & submissions
Rank materials for review
Apply criteria for full review of best
Review and publish
Review Criteria (Rubrics) : Review Criteria (Rubrics) EPS –
Motivation
Content
Method MERLOT –
Quality of Content
Effectiveness as a Learning Tool
Usability
Review Criteria (Rubrics) : Review Criteria (Rubrics) EPS –
Motivation
User Friendly: Easy to start, understand, control, and is documented
Attractive: Appealing, interactive, and interesting
Clear Description and Context
Content
Method MERLOT –
Quality
Effectiveness
Usability
Understandable: Runs easily
Intuitive: Attractive, controlled, input/output
Feedback: Clear communication
Documented
Review Criteria (Rubrics) : Review Criteria (Rubrics) EPS –
Motivation
Content
Relevant: Important topic and media use
Scope: Broad and “Profound” topic
Correct: Content accurate and models indicated
Method MERLOT –
Quality
Correct Models: Numerical, textual, notation
Important Topics: Standard or unique
Conceptual: Understand parameters
Effective Graphics
Flexible: Multiple uses
Effectiveness
Usability
Review Criteria (Rubrics) : Review Criteria (Rubrics) EPS –
Motivation
Content
Method
Flexible: Broad audience and topics
Matching Target: Correct level, background, objectives
Realization: Media is well used
Documentation: Operation, references, teaching process MERLOT –
Quality
Effectiveness
Learner: Level, challenge, control
Relevant Knowledge: Learning goals, application
Experience: Dynamic, flexible, interactive, progress
Feedback: Clear, immediate, positive
Usability
Review Scales : Review Scales EPS –
+2 to -2
Indicated for each sub-question
Averaged MERLOT –
1 – Major Problems
2 – Works, but has errors
3 – Works well, useful
4 – Excellent, notable features
5 – Outstanding and unique
Agreement? : Agreement? MERLOT Reviewers: Average Absolute Value of Difference between Reviewers (Scale of 1 – 5, Reviews 3 – 5, 135 items)
Quality: 0.56 (3.8)
Effectiveness: 0.60 (3.65)
Usability: 0.69 (3.9)
Total: 0.57 (3.72)
Agreement? : Agreement? MERLOT Reviewers: Average Difference between Reviewers, Time Dependence
Quality: 0.57 (00 – 03), 0.55 (04 – 06)
Effectiveness: 0.7 (00 – 03), 0.5 (04 – 06)
Usability: 0.7 (00 – 03), 0.69 (04 – 06)
Total: 0.6 (00 – 03), 0.53 (04 – 06)
Agreement? : Agreement? MERLOT Reviewers: Average Difference between Reviewers, Score Dependence
(N5 = 20, N4 = 61, N3 = 39)
Agreement? : Agreement?
For MERLOT Reviewers
Average Difference in Total Scores: 0.4 – 0.5
2002 Quantum Mech : 2002 Quantum Mech EPS: ~30 items, 7 recommended
MERLOT:
2 award winners, 5, 4 (2), 2 not reviewed = 0 (1 in German
Joffre QM (2003 MERLOT Classic)
de Raedt QM (No MERLOT Review)
General Recommendations
Results from Workshop
2003 Optics : 2003 Optics EPS: ~250 items, 5 Recommended
MERLOT:
5, “4.5” (2), 0 (2)
Paul Falstad Math & Physics Applets
Report & Recommendations
2004 Mechanics, Different Lists : 2004 Mechanics, Different Lists EPS: ~250 items
7 Recommended
Merlot: 5 (2), “4.5” (2), 4, 3, 0 (German)
MERLOT: ~700 items
4 Other Recommendations
EPS: Rec (not listed), High quality, 0 (2)
Chaos Collection
Report & Recommendations
2005 Thermal Physics, Joint : 2005 Thermal Physics, Joint ~150 items, 55 Reviewed
1 Highly Recommended EPS & MERLOT
(Engineering)
7 Recommended
1 Disagreement
Expert System for Thermodynamics
Stat Mech in Intro Physics
Report & Recommendations
2006 Elect & Magn, Joint : 2006 Elect & Magn, Joint 950 items, 200 Reviewed
6 Highly Recommended EPS & MERLOT
6 Recommended by both
5 Disagreements (Some may not have been reviewed)
MIT “TEAL” (OCW)
Physlet Tutorial Pages
Comments & Conclusions : Comments & Conclusions Good agreement w/ independent reviews
Quality rating for larger collections
Many topics repeated
Many Simulations, not video experiments
Instructional aspects rare
How to put into a class rarer
References : References http://physik.uni-graz.at/MPTL9/proceedings/ProcSporkenMason.pdf
http://pen.physik.uni-kl.de/w_jodl/MPTL/MPTL10/contributions/mathelitsch/Rep_Recom_Thermodyn_2005.pdf