Writing (Fair) Questions to Test Software Knowledge

My most recent project has me writing questions to evaluate a student’s software knowledge of the fundamentals of Autodesk Inventor. The questions could be used as a pre-assessment to identify gaps to determine training requirements, or they could be used to evaluate software knowledge after attending training. In either situation, the questions needed to cover the content taught in our Introduction to Solid Modeling learning guide. With topics that span 28 chapters in this learning guide, you can probably imagine that I am writing a LOT of questions. I’d like to share some tips for writing good (and, maybe more importantly, fair) questions.  

For these questions, I focused on multi-answer, matching, and reorder questions and avoided True/False questions for the simple fact that having a choice between only two answers means a 50% chance that a guess would be correct. And because my goal was to use the questions to test pre- or post-training knowledge, I tried to design questions that require application and analysis of knowledge rather than simple recall or making a guess.  

My tips for writing both the question and its answers are as follows: 

Tips for Writing the Question 

1. Ensure the question is relevant to the learning objectives for the content you are testing on. 
2. Avoid tricky or overly complex wording that could mislead the test taker.  
3. Eliminate unnecessary information that could distract from the main point of the question. 
4. Use clear and straightforward language to ensure understanding.  
5. Ensure consistency in writing style between questions. For example, use either “Which of the following are true” or “Which of the following are false,” not both. 
6. Focus on a single idea that avoids complex scenarios. 
7. Include an image to support the question without giving away answers or making the question confusing. 

Tips for Writing the Answers  

1. Maintain a consistent structure for all answer choices in terms of both their length and grammatical structure. This helps avoid giving hints on the correct answer. 
2. Ensure that the distractors (wrong answers) are plausible. Use answers that reflect what students commonly misunderstand or mistakes they might make when using the software.  
3. Be cautious with words like "always" or "never" in answer choices, as they often provide hints on an incorrect answer. 
4. Arrange the answers in a random order to prevent patterns that could give away the correct answer. 

I hope that this helps should you be tasked with writing questions for any knowledge assessment, whether for CAD or for any other skill. 

About the Author

Manager – Learning Content Development<br><br>Trained in Instructional Design, Jennifer uses her skills to develop instructor-led and web-based training products as well as knowledge profiling tools. Jennifer has achieved the Autodesk Certified Professional certification for Inventor and is also recognized as an Autodesk Certified Instructor (ACI). She enjoys teaching the training courses that she authors and is also very skilled in providing technical support to end-users. Jennifer holds a Bachelor of Engineering Degree as well as a Bachelor of Science in Mathematics from Dalhousie University.

Follow on Linkedin More Content by Jennifer MacMillan