MARTest (Marine Corps App.) for Exam Administration
MARTest is a Marine Corps application for online exam administration, built as part of a larger distance learning initiative, covering the test-taking end of MCI (Marine Corps Institute) courseware. One of the problems it aimed to solve was security, enabled by letting exam authors mark certain questions mandatory (must appear in all tests taken by students for this exam). When a student takes a test, questions are randomly generated, first presenting all mandatory questions, and then choosing from the pool of elective ones to round out the required number for each exam.
The system allows exams to be built out of many question types, generally impossible under the previous bubble sheet system (with which MARTest also interfaced). Questions were grouped under Learning Objectives (material the Marine was expected to know after completing a course), and could be:
- Multiple Answer/Multiple Choice - multiple answers may be selected;
- Matching - matching items between two columns; and
- Essay - students type a small essay response.
A weighting system is used to account for these question and answer types. Each question has an associated integer weight associated, marking its importance to the test and the Marine's assumed knowledge. Each answer choice has a floating-point weight value between -1 and 1. Correct answers are 1.0; incorrect answers are 0. Answers which could be interpreted as "valid, but not quite correct" could weigh in somewhere in between. Negative values are primarily used for the Multiple Answer Multiple Choice questions, where the student is presented an interface of checkboxes for the answers. To countermand the marking of all available checkboxes, negative weights are used on extremely wrong answers. The total value of all answers is multiplied against the questions weight. If the result is less than zero, then the score for that question is zero.
Essay questions require a Subject Matter Expert (SME) for grading. The SME is able to grade responses without knowing the weight of the question in regards to the overall test. The system also interfaces to older bubble-sheet based tests sent out as part of the courseware.
MARTest fits into a large Oracle setup, keeping most of the unique information in its own tablespace yet fully able to interact with certain other tables as necessary, making it a significant part of a much larger objective. By integrating with both the old bubble-sheet tests and the other, Oracle-based new modules, MARTest delivers its value while keeping transition costs low and non-disruptive.
Challenges
- Flexible approach needed for authoring tests, processing completed tests, and establishing advanced grading criteria
- Scale system to global test-taking while retaining standards of quality
- Interface with existing test processing systems
Approach
- Zope application server running at MCI, connected to Oracle
- Zope-based web tests deployed worldwide through secure web browsers
Benefits
- Greatly reduced cost for administering and processing tests
- Faster deployment of new tests and changes to existing tests
- Higher quality through random selection of test questions · Immediate feedback to student and staff on test results
Using Zope Corporation to enhance our web platform was an
attractive option for us, because the flexibility and the Open
Source nature of the software allowed for easier integration, as
well as easier project collaboration between the two groups
involved. This is important when you are building capabilities
such as a rule-based personalization engine, for which Zope
clearly understood our expectations, and delivered in a timely
and cost effective manner.
- David Lee, Project Head, Hostway Corporation