Well-designed tests are one of two kinds:
- multiple-choice (or true/false, etc.) questions with unambiguous answers, like the SAT
- short answer or essay questions where you actually get to express a thought in writing
I have a class this semester that has quizzes that are not well designed.
First of all, they are mostly multiple choice, but the instructions are as follows:
Choose a, b, c, d, or all that apply.
Wha? That makes 16 possible answers for every question, only one of which is “right”.
On top of that, the answers are often ambiguous. If one interpretation of the answer would make it right, is that enough to make it “apply”, or should it be the only or primary interpretation of the answer?
For example:
A bibliographic record
a. a surrogate record
b. a metadata record
c. a description of an information package
d. a catalog card
What is the appropriate relationship between the question (”A bibliographic record”) and the correct responses? Should they be equivalent terms (a,b)? Definitions (c)? Examples (d)?
I circled all four, and wrote next to card catalog “It’s an subset—do the terms have to be perfectly equivalent?” I got it marked wrong; next to a, b, and c was written “intellectual info” and next to d was written “a thing”.
Well yeah, I knew that. In a short answer question, I could have said it beautifully. Unfortunately, the structure of the quiz (”or all that apply”) forces me to consider each multiple choice answer as a true/false statement. A surrogate record is a bibliographic record? True. A metadata record is a bibliographic record? True. A description of an information package is a bibliographic record? True. A catalog card is a bibliographic record? True. Oops, I mean… not true? There’s little room for nuance in a binary choice.
Likewise,
Chirographic refers to
a. physical handwriting
b. the “shadow style” used by Leonardo da Vinci
c. hand-typed manuscripts
d. manuscripts
I chose a; a and d together was the “correct” answer. How am I to interpret “manuscript”? I have several good dictionaries that have definitions of manuscript that include typewritten material, so not all manuscripts are hand-written—and the question reinforces this by including choice c, hand-typed manuscripts. So, does d mean all manuscripts, or just the hand-written ones, which could be properly considered chirographic?
Sigh.
In the end, it doesn’t really matter; these are worth a tiny portion of my grade. But really, they’re awfully designed.