Timed Repetitive Quizzes (TRQs)

Timed Repetitive Quizzing is an online activity that challenges students at the knowledge, comprehension and application levels of cognitive thought. In some ways, it is similar to B.F. Skinner's "Teaching Machine" developed in the 1950s and described in the 6-minute video below.

Skinner's Teaching Machine had the following features:

1. The reward system centers on freeing the student from anxiety and making learning "pleasurable."
2. Students work "at their own pace."
3. Students follow a "program" via a large number of small steps.
4. Questions came from all levels of cognitive thought.
5. Skinner stated that students learn twice as fast.

A troubling assertion made by some in the teaching profession, was that a machine could eventually replace the teacher in the classroom. Authur C. Clarke puts some perspective on this when he said "if a teacher can be replaced by a machine, they should be."

We (at Chem21) do not envision a machine replacing a teacher, however, the Chem21 teaching machine is infinitely better than a teacher at helping students place foundational facts into their long-term memory. When facts are quickly accessible, the teacher and student can travel faster and farther down the path of learning.

The animation below uses the learning of cations and ions to demonstrate the rationale and theory of Timed Repetitive Quizzing. Data evaluation shows that ~ 100 facts (100 cations and anions) can be placed in a learner's long-term memory in 6 hours of engagement spaced over an academic year. Click the Reload button to start the animation.

Elements Of An Effective Teaching Machine (or Teacher)

Before building a "teaching machine," it's essential to identify and include the core elements of teaching and learning. Effective teaching leads to learning, which occurs when:

1. new knowledge is introduced and reinforced until it becomes part of long-term memory,
2. existing memory constructs are strengthened for longer retention, or
3. connections between memory constructs are formed.

Learning requires changes to long-term memory - if nothing has been added to long-term memory, then nothing has been learned. Traditionally, memory building has relied on students' intrinsic motivation, limiting exploration of external motivators. Given today's technological advancements,

Is it possible that external motivators may enhance both the efficiency and enjoyment
of learning beyond what can be achieved through a student's intrinsic motivation alone?

The answer is

The TRQ external motivator introduces three 'forces' that propel students down the path of learning:

• Cognitively-simple low-stakes assignments - students with the "willing to work" attitude will complete these small assignments since they possess the mental acuity and lack a reasonable excuse. In fact, experience shows that over 90% of students earn the available points on TRQ assignments which corresponds to a consistent transfer of information into long-term memory.
• Guaranteed success - correct effort, or continued effort, on these cognitively-simple assignments yields a perfect score . . . . which translates to  100%  Success - a feeling loved by all.
• Time - if students are given several ways to complete an assignment and one is much faster, most will choose the quicker path to save time. If the fastest route also requires recalling information from long-term memory, students will be motivated to learn for efficiency. This time-saving effect becomes significant with repeated (15 times) spaced (1 academic year) assignments that strengthens knowledge and creates new memory connections.

We will revisit these "forces" in a few moments, but first let's investigate the essential components of an effective learning plan. A New York Times article entitled The Trouble With Homework (9/10/2001 by Annie Murphy Paul) describes several effective study techniques that have documented abilities to create long-term memories and aid in their retention and retrieval.

1. Spaced Repetition - information is presented in small chunks at increasingly larger time intervals to create and maintain long-term memory constructs.
2. Retrieval Practice - is the active process of retrieving a memory from long-term storage. This activity strengthens memories and extends their lifespan. Retrieval practice tests both the quality of the stored information and speed required to recover the information. Research suggests that retrieval practice is a much more effective learning tool than rereading or note-taking. Combining spaced repetition with retrieval practice makes testing a powerful tool for building and reinforcing memories. While these first two techniques create an exceptional learning plan, interleaving knowledge has the effect of "tricking" the brain so that it remembers more information and for a longer period of time.
3. Interleaving - presenting information through varied methods prompts greater cognitive effort. Interleaving may involve studying content from Chapter 1, proceeding to Chapters 2 and 3, and then revisiting Chapter 1, or integrating Chemistry and Spanish flashcards within one session. This approach facilitates novel associations that enhance retrieval efficiency when accessing information from long-term memory. Known as "cognitive disfluency," this increased mental demand signals the brain that the acquired knowledge is valuable. Interleaving demonstrates notable effectiveness; for example, a study published in the Journal of Applied Research in Memory and Cognition showed that math students who practiced interleaved problems achieved scores twice as high as those practicing grouped or massed problems.

Would you like to use spaced repetition, retrieval practice, and interleaving
to help students build strong memories and reach their goals faster?

If the answer is meet the TRQ Teaching Machine. The TRQ program contains the small-step, anxiety-free, pleasurable aspects found in Skinner's approach with the following features:

1. A timer - internet delivery to an unmonitored learner requires a timer to prevent the learner from searching for answers. The timer ensures students have only a few seconds to respond, creating a more effective learning environment where answers must come from memory storage. The use of a timer limits the questions to instant recall, 4-8 second answer time . . . . no calculations, common domain knowledge and one obvious correct answer.
2. External Rewards
   • Earning course credit - students earn credit for the effort they invest. Few things are more discouraging than spending three hours studying for a quiz and receiving a grade that does not reflect the effort. With TRQs, students are more willing to accept the work required because they know they will receive a guaranteed portion of the course grade for completing them. This creates a clear "contract" with the student where they complete many small assessments that steadily contribute to their grade. This structure is important because it centers on testing - an activity most people do not naturally enjoy, yet one that is highly effective for strengthening long-term memory.
   • Saving time functions as a reward that causes students to work faster than their normal pace. Students complete a TRQ set in one of three time-dependent modes:
     • Path 1 is the shortest path and indicates that the student has stable, easily retrievable memory structures in long-term memory . . . . the student has expert-level retrieval.
     • Path 2 is slightly longer and indicates memory structures that are well developed and highly retrievable. The student demonstrates near–expert-level retrieval.
     • Path 3 is the longest path and indicates that the student is learning and developing memory structures. Retrieval is slower and less consistent, reflecting novice-level retrieval.
   • Assignment completion - rewards the learner with feelings of success and accomplishment. The R in TRQ represents Repeated. Once a student attains "success" on a subset of knowledge, they will repeatedly experience "success" throughout the academic year. A TRQ will be completed daily at first and then at progressively longer intervals to keep the information in the learned state. For teachers, this repetition is essential because it strengthens the memory structures that support critical thinking and deeper exploration. For students, repetition creates an incentive to complete TRQs through Path 1 to save increasing amounts of time as repetitions accumulate (~ 15 per year). The use of a timer places an important constraint on the design of TRQ questions - the content must be in the lowest levels of cognition. The question and answer choices must be succinctly presented so the learner can view the presented material and select an answer within 6 seconds.

     If a goal of education is to move novices toward expert performance, then
     students should develop the ability to answer TRQ questions like an expert.

The TRQ program does not supplant the teacher. Instead, it manages a low-stakes, high-reward, learning activity that creates expert-level memory constructs. Now students achieve the full benefit of the classroom activities that promote connections between the TRQ-created memory constructs.

TRQ Overview

TRQs are low-stakes, high-reward learning activities that incorporate spaced repetition, retrieval practice, and interleaving. Teachers have long suggested these proven study techniques to build foundational knowledge, but suggestions are rarely followed with the intensity needed because students prioritize graded assignments over optional strategies. To transmit the idea that course knowledge is essential for deep understanding, we recommend allotting 10% of the course grade per 2000 domain questions. For general and organic chemistry (4000 - 5000 questions), it's 20% of the grade. For biology, it may be 40% - 50% . . . . for language courses, 60% - 75%.

TRQs alleviate frustration. Student frustration occurs when a task (homework problem) cannot be accomplished without help. Help may be a book, AI, an instructor, a teaching assistant or another student. For some students, the "help option" is the one that most quickly provides the answer. In this scenario, the student may earn homework credit but an answer without the memory constructs to understand it only delays frustration . . . . it reappears at the exam.

A common response to gaps in basic knowledge is adaptive learning. When a student misses a higher-level question, the system drills down to identify the missing knowledge and notifies the student. The student must then learn that information and try a similar question again, repeating the cycle. In this model, students often learn through repeated failure.

TRQs take a different approach - a "flipped" form of adaptation. Instead of diagnosing gaps after failure, TRQs build foundational knowledge first. With this knowledge in long-term memory, students are better prepared to answer higher-level questions that require connecting multiple ideas.

Adaptive systems are built on learning from mistakes, while TRQs focus on learning through repeated success.

Quiz Completion

TRQs are active learning assignments that build foundational knowledge in an entire class of students. TRQ completion signals that the student has taken one of the small learning steps described by Skinner. There are three TRQ completion paths:

1. Path 1: The student submits a perfect quiz within the quiz time.
2. Path 2: The student submits y quizzes with at least x correct answers within the quiz time.
3. Path 3: The student accumulates the required total number of correct answers across multiple quizzes, counting only quizzes that exceed the minimum correct threshold.

Completion through any path awards the same number of points.

This trimodal structure is widely accepted by students. Proficient students (Path 1) avoid unnecessary repetition, while developing students (Path 3) strengthen long-term memory through repeated correct retrieval. Effective memorization requires frequent interaction with new information in the first week, followed by periodic review over the following weeks and month. The spacing of information retrieval effectively maintains the knowledge in the learned state and fosters deeper engagement that builds Higher Order Thinking Skills.

Quiz Customization

Instructors can customize the TRQs with the parameters shown to the left.

Mouseover (or mousedown on touch devices) each link to see additional information in the tooltip.

For the parameters shown, when a student clicks the "Take Quiz" button, 10 questions will appear on the quiz. Students will have 50 seconds to qualify for Option 1 or 2. If the student gets 10/10 questions correct in less than 50 seconds (Option 1), they receive 10 points and are "finished for the day" (Set 1 Complete). If the student gets 8/10 or 9/10 questions correct in less than 50 seconds, they have satisfied ⅓ of Option 2. The student would need to retake the quiz (answer 10 new questions from the question pool) two more times and answer 8/10 or 9/10 questions correctly in less than 50 seconds to complete Option 2, receive 10 points and be "finished for the day" (Set 1 Complete). If a student correctly answers 5 - 9 questions OR 10 questions in more than 50 seconds, the number of correct answers will be added to a "totalCorrect variable" . . . . when this variable equals 75, the student receives 10 points and is "finished for the day" (Set 1 Complete).

After the student has completed Set 1, they must wait 12 hours (Intermission Between Sets) before starting SET 2. After completing SET 2, the student must wait 12 hours before starting SET 3. Students must complete a total of 6 Sets (Times Set Is Repeated) for this assignment. Students can earn 60 points on this assignment (6 SETS × 10 points / SET = 60 points)

Accessibility:

Although TRQs must be timed to be effective as an online assignment, the time can be extended for a student needing more time. For some students, the timing aspect is incompatible with their specific learning style and a different assessment tool (i.e. weekly quiz) must be used to gauge the student's proficiency over the TRQ material.

For more information on the efficacy of TRQs in various subjects, click TRQ Research.
To view sample TRQ questions, click General Chemistry TRQ Questions.