Do Your Multiple-Choice Questions Drive Deeper Learning?
Consider this question.
To drive deeper learning, multiple-choice questions must: (select the best answer)
a. Ask participants to recall content from the course.
b. Ask participants to analyze the meaning of unfamiliar words.
c. Assess whether participants met the learning objectives.
d. Assess whether participants listened to all of the videos.
If you're not sure of the answer, you’re not alone. Research shows that multiple-choice questions are difficult to write well. Difficult, that is, until you know and have practiced the evidence-based tactics that make them clear and valid for their intended purpose.
Too many multiple-choice questions are unclear, frustrating people who try to answer them and making results less valid. Too many multiple-choice questions measure only recall of course content, which is not usually what we need to measure. We must write questions so they effectively measure achievement of the learning objectives. (Therefore, the answer to the multiple-choice question above is c.)
Writing well-written, valid multiple-choice questions that measure achievement of the learning objectives is a critical skill for people who design instruction. And yet research shows that most multiple-choice questions are poorly written. This is a big problem because poorly written learning objectives provide bad information (garbage in garbage out) and can lead to legal problems.
Who is this course for?
We designed the Write Learning Assessments for Deeper Learning course specifically for workplace-learning professionals and others who need to write valid job-related assessments. This includes instructional designers, trainers, instructors, content experts, technical writers, and others. In this course, we use actionable research tactics for writing valid multiple-choice questions that measure learning objectives. The course is best suited for people who have some experience designing adult instruction and writing multiple-choice questions.
If writing multiple-choice assessments is something you need to do and you don't know the critical evidence-based tactics, you need this course! Take the quiz to see what you know and don't know about multiple-choice questions.
What does this course include?
The Write Learning Assessments for Deeper Learning course is equivalent to a 3-day classroom workshop, but we spread it out over 3 weeks to give you more time to process and practice. And it’s online, so no travel or travel expenses, plus you get better access to Dr. Shank and your classmates!
The learning objectives for this course are:
-- Make sure your learning objectives are written correctly.
-- Analyze which actions improve assessment validity and reliability.
-- Evaluate how to best assess different levels of learning objectives.
-- Analyze multiple-choice questions to identify flaws that should be fixed.
-- Write appropriate multiple-choice items that measure given objectives.
The course includes elements designed to improve your skills including:
-- 4 live online sessions with Patti and your classmates on important insights, problems, commonly asked questions, and practice issues
-- Ongoing asynchronous discussions to ask and discuss questions, and share insights
-- Feedback from Patti and your classmates
-- Access to live-session recordings, in case you can’t attend live or you want to review
-- Patti’s assessment-writing manual (only available through this course)
-- A master job aid to help you write more effective multiple-choice questions
-- Practice evaluating and writing learning objectives so you can more easily write aligned multiple-choice questions
-- Practice writing multiple-choice questions that assess achievement of learning objectives
-- Three quizzes to check your understanding and help you process and remember critical insights
What's different about THIS course?
Here’s what’s different about this course: Patti. Real practice. Feedback. Support.
Patti Shank, PhD is an acknowledged leader in the use of evidence-based tactics for deeper learning in the workplace. She reads the research, interprets it for use in workplace learning, and helps people leverage the most critical insights in an easy-to-apply and practical way. And she has 12+ years of experience teaching in the virtual classroom.
If you need to write understandable and valid multiple-choice questions, this is the course you need. You’ll learn when to use multiple-choice questions, when other assessment types are more appropriate, and how to design understandable and valid multiple-choice questions.
Supporting Patti’s courses are:
-- Karen Hyder, with her 20 years of virtual-classroom training, coaching and production work
-- Chris Benz, with his 30 years of designing and producing successful live online events, training materials, and technical-communication products
What's the big deal about "validity?"
The reason the word “validity” keeps coming up is because it is one of the most important issues when writing multiple-choice questions. Validity is a critical outcome of the assessment process. Valid questions require clear writing and clear ties to learning objectives. When they are valid, questions can measure deeper levels of learning and create deeper processing, which improves ability to apply. Assessments that are less valid invite frustration, irrelevant results, and legal battles. It’s far better to do a good job with this key skill than risk these bad outcomes.
Why should I enroll as early as possible?
#1: Early access: As soon as you enroll, you’ll get access to the course manual, which gives you a head start on reading and processing important concepts. The first week of the course is the “official” time allotted for participants to read and ask questions, but past participants said they wanted more time. Register early so you have extra time to read the manual!
#2: Secure your spot: For maximum interaction, we purposely limit enrollment, so if you wait too long, you might miss out. Secure your spot by enrolling early. Enroll now!
What are the dates and times?
The next course starts on January 20, 2020 and ends on February 7, 2020. The four live sessions are on the following dates and times:
Live sessions are recorded but we encourage you to attend live whenever possible. Except for the live sessions, you can participate when it works best for you.
|Monday, January 20||11:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time|
|Friday, January 24||11:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time|
|Friday, January 31||11:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time|
|Friday, February 7||11:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time|
How much time do I need?
Previous participants tell us the course took about 25 hours (YMMV) for the reading, live sessions, and self-paced course activities. Week 1 has a fair amount of reading (course manual). If you enroll early, we'll send the reading to you before the course starts so you can spread out the reading time. To gain the maximum benefit, take the course when you can dedicate this much time and effort. Except for the live sessions, you can participate in the course activities at the times that work best for you.
Can I share the materials and live/recorded sessions with my team/friends/pets?
Course materials are copyrighted and each enrollment permits only the person enrolling to access the course, materials, and activities. But it’s a smart idea to train your entire team and give them access to everything the course has to offer!
Can you deliver a private class just for my team?
We can and we do! Please contact us at firstname.lastname@example.org to discuss. We offer discounts for three or more people from the same organization (paid at the same time) attending the same public class, or for 10 or more people enrolling in a private class for their organization.
What's my investment?
Register now! Enrollment is limited for maximum interaction, and we expect the course to fill up quickly. We currently accept credit cards and PayPal for payment. To see if other payment arrangements are possible, please contact us at email@example.com.
How do I convince my workplace to pay for this course?
Assessment writing is a critical skill but is rarely taught to learning practitioners. As research shows, many workplace-learning assessments are poorly written and invalid for the desired use.
Poorly written and invalid assessments do not provide accurate information about the effectiveness of the instruction. Invalid assessments can create morale and legal problems. The bottom line is that everyone who builds assessments must be able to design valid assessments!
Can I get a refund?
To offer personal attention, this course has limited enrollment. Once you register, we are holding your spot. As a result, we are unable to offer refunds. However, you can substitute another person in your organization up to three days before the course starts.
Take the next step. Ready to register? Don’t delay! Enrollment is limited for maximum interaction and we expect the course to fill up quickly. We currently accept credit cards and PayPal. To see if another payment arrangement is possible, please contact us at firstname.lastname@example.org. To attend, we require that payments be paid in full before the course start date.
Not quite ready to register? Sign up for the Deeper Learning At Work mailing list to stay informed about future offerings of this course, upcoming Deeper Learning At Work (DL@W) courses, webinars, and conference presentations by Patti and other DL@W thought leaders. List members will also have access to ongoing information about DL@W principles, strategies, tactics, and research.