Do Your Multiple-Choice Questions Drive Deeper Learning?

Consider this question.

To drive deeper learning, multiple-choice questions must: (select the best answer)
a.   Ask participants to recall content from the course.
b.   Ask participants to analyze the meaning of unfamiliar words.
c.   Assess whether participants met the learning objectives.
d.   Assess whether participants listened to all of the videos.

If you're not sure of the answer, you’re not alone. Research shows that multiple-choice questions are difficult to write well. Difficult, that is, until you know and have practiced the evidence-based tactics that make them clear and valid for their intended purpose.  

Too many multiple-choice questions are unclear, frustrating people who try to answer them and making results less valid. Too many multiple-choice questions measure only recall of course content, which is not usually what we need to measure. We must write questions so they effectively measure achievement of the learning objectives. (Therefore, the answer to the multiple-choice question above is c.)

Writing well-written, valid multiple-choice questions that measure achievement of the learning objectives is a critical skill for people who design instruction. And yet research shows that most multiple-choice questions are poorly written. This is a big problem because poorly written learning objectives provide bad information (garbage in garbage out) and can lead to legal problems.


Who is this course for?
We designed the Write Learning Assessments for Deeper Learning course specifically for workplace-learning professionals and others who need to write valid job-related assessments. This includes instructional designers, trainers, instructors, content experts, technical writers, and others. In this course, we use actionable research tactics for writing valid multiple-choice questions that measure learning objectives. The course is best suited for people who have some experience designing adult instruction and writing multiple-choice questions.

If writing multiple-choice assessments is something you need to do and you don't know the critical evidence-based tactics, you need this course! Take the quiz  to see what you know and don't know about multiple-choice questions.

What does this course include?
The Write Learning Assessments for Deeper Learning course is equivalent to a 3-day classroom workshop, but we spread it out over 3 weeks to give you more time to process and practice. And it’s online, so no travel or travel expenses, plus you get better access to Dr. Shank and your classmates!

The learning objectives for this course are:
 
  --  Make sure your learning objectives are written correctly.
  --  Analyze which actions improve assessment validity and reliability.
  --  Evaluate how to best assess different levels of learning objectives.
  --  Analyze multiple-choice questions to identify flaws that should be fixed.
  --  Write appropriate multiple-choice items that measure given objectives.


The course includes elements designed to improve your skills including:

  --  4 live online sessions with Patti and your classmates on important insights, problems, commonly asked questions, and practice issues
  --  Ongoing asynchronous discussions to ask and discuss questions, and share insights
  --  Feedback from Patti and your classmates
  --  Access to live-session recordings, in case you can’t attend live or you want to review  

  --  Patti’s assessment-writing manual (only available through this course)
  --  A master job aid to help you write more effective multiple-choice questions
  --  Practice evaluating and writing learning objectives so you can more easily write aligned multiple-choice questions
  --  Practice writing multiple-choice questions that assess achievement of learning objectives

  --  Three quizzes to check your understanding and help you process and remember critical insights



What's different about THIS course?

Here’s what’s different about this course: Patti. Real practice. Feedback. Support.

Patti Shank, PhD is an acknowledged leader in the use of evidence-based tactics for deeper learning in the workplace. She reads the research, interprets it for use in workplace learning, and helps people leverage the most critical insights in an easy-to-apply and practical way. And she has 12+ years of experience teaching in the virtual classroom.

If you need to write understandable and valid multiple-choice questions, this is the course you need. You’ll learn when to use multiple-choice questions, when other assessment types are more appropriate, and how to design understandable and valid multiple-choice questions.

Supporting Patti’s courses are:
  --  Karen Hyder, with her 20 years of virtual-classroom training, coaching and production work
  --  Chris Benz, with his 30 years of designing and producing successful live online events, training materials, and technical-communication products


What's the big deal about "validity?"
The reason the word “validity” keeps coming up is because it is one of the most important issues when writing multiple-choice questions. Validity is a critical outcome of the assessment process. Valid questions require clear writing and clear ties to learning objectives. When they are valid, questions can measure deeper levels of learning and create deeper processing, which improves ability to apply. Assessments that are less valid invite frustration, irrelevant results, and legal battles. It’s far better to do a good job with this key skill than risk these bad outcomes.


Why should I enroll as early as possible?
#1: Early access: As soon as you enroll, you’ll get access to the course manual, which gives you a head start on reading and processing important concepts. The first week of the course is the “official” time allotted for participants to read and ask questions, but past participants said they wanted more time. Register early so you have extra time to read the manual! 

#2: Secure your spot: For maximum interaction, we purposely limit enrollment, so if you wait too long, you might miss out. Secure your spot by enrolling early.  Enroll now!


What are the dates and times?
The next course starts on January 20, 2020 and ends on February 7, 2020. The four live sessions are on the following dates and times:
Monday, January 2011:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time
Friday, January 2411:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time
Friday, January 3111:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time
Friday, February 711:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time


Live sessions are recorded but we encourage you to attend live whenever possible. Except for the live sessions, you can participate when it works best for you.

How much time do I need?
Previous participants tell us the course took about 25 hours (YMMV) for the reading, live sessions, and self-paced course activities. Week 1 has a fair amount of reading (course manual). If you enroll early, we'll send the reading to you before the course starts so you can spread out the reading time. To gain the maximum benefit, take the course when you can dedicate this much time and effort. Except for the live sessions, you can participate in the course activities at the times that work best for you.

Can I share the materials and live/recorded sessions with my team/friends/pets?
Course materials are copyrighted and each enrollment permits only the person enrolling to access the course, materials, and activities. But it’s a smart idea to train your entire team and give them access to everything the course has to offer!

Can you deliver a private class just for my team?
We can and we do! Please contact us at info@deeperlearningatwork.com to discuss. We offer discounts for three or more people from the same organization (paid at the same time) attending the same public class, or for 10 or more people enrolling in a private class for their organization.



What's my investment? 
US $1,095

Register now! Enrollment is limited for maximum interaction, and we expect the course to fill up quickly. We currently accept credit cards and PayPal for payment. To see if other payment arrangements are possible, please contact us at info@deeperlearningatwork.com.

How do I convince my workplace to pay for this course?
Assessment writing is a critical skill but is rarely taught to learning practitioners. As research shows, many workplace-learning assessments are poorly written and invalid for the desired use. 

Poorly written and invalid assessments do not provide accurate information about the effectiveness of the instruction. Invalid assessments can create morale and legal problems. The bottom line is that everyone who builds assessments must be able to design valid assessments!

Can I get a refund?
To offer personal attention, this course has limited enrollment. Once you register, we are holding your spot. As a result, we are unable to offer refunds. However, you can substitute another person in your organization up to three days before the course starts.


Take the next step. 
Ready to register? Don’t delay! Enrollment is limited for maximum interaction and we expect the course to fill up quickly. We currently accept credit cards and PayPal. To see if another payment arrangement is possible, please contact us at info@deeperlearningatwork.com. To attend, we require that payments be paid in full before the course start date.

Not quite ready to register? Sign up for the
Deeper Learning At Work mailing list to stay informed about future offerings of this course, upcoming Deeper Learning At Work (DL@W) courses, webinars, and conference presentations by Patti and other DL@W thought leaders. List members will also have access to ongoing information about DL@W principles, strategies, tactics, and research.

Questions?
Please contact us at info@deeperlearningatwork.com




Multiple-choice questions are everywhere in learning, but they can be fiendishly difficult to write well. Nobody is doing a better job than Patti of distilling out research-based principles for writing well, and turning them into practical, actionable guidelines. Highly recommended.

Julie Dirksen, Author, Design for How People Learn

I joined Patti’s course this July and loved it! Her content matches her facilitation; both excellent. I would totally recommend it.

Taruna Goel, Learning & Performance Specialist, Vancouver, BC, Canada

This training gave me insight into the many factors involved in creating effective multiple-choice questions. It has prepared me to better advise instructors on whether multiple-choice questions are the best tool, and how they can be crafted to generate reliable, valid data.

Anna Landes Benz, Program and Faculty Developer, private university, MN, USA

The weekly live sessions opened my mind and upped my question-writing game. Patti knows her topic and added the wow factor that I was looking for by offering real-time feedback to participants.

Comments from other participants helped me see different perspectives. I strongly recommend this course for anybody who must create multiple-choice question learning assessments.

Peter Sage, Trainer, iLAB, South Africa

Patti’s course on writing assessments has been invaluable in helping me write effective, credible assessments for my online learning courses and assessments. There is a wealth of well-researched detail about what exactly goes into writing an effective multiple-choice question and how these questions relate to the bigger picture of effective learning. I strongly recommend this course to anyone who wants to make their learning more effective as well as anyone who needs to write effective assessments.

Nick Tchan, Instructional designer, NSW Health, Australia

Write Learning Assessments modules:

Patti Shank

Patti Shank, PhD is a learning expert, researcher, author, and writer who is listed as one of the Top 10 Most Influential People in eLearning internationally. She is the author of Write and Organize for Deeper Learning, Practice and Feedback for Deeper Learning, and Manage Memory for Deeper Learning. These books offer practical tactics for improving outcomes based on training and other research. You can find Patti's articles on eLearning Industry and elsewhere. Website: pattishank.com.

Course Pricing