Testing effect

File:Anki flashcard question and answer math.gif

Flashcards are an application of the testing effect. Here, flashcard software Anki is used to review a mathematical formula through active recall. First, only the question is displayed. Then the answer is displayed too, for verification.

The testing effect (also known as retrieval practice, active recall, practice testing, or test-enhanced learning)<ref>Template:Cite journal</ref><ref name=":15">Template:Cite journal</ref><ref>{{#invoke:citation/CS1|citation

|CitationClass=web }}</ref> suggests long-term memory is increased when part of the learning period is devoted to retrieving information from memory.<ref>Template:Cite book</ref> It is different from the more general practice effect, defined in the APA Dictionary of Psychology as "any change or improvement that results from practice or repetition of task items or activities."<ref>Template:Cite APA Dictionary</ref>

Cognitive psychologists are working with educators to look at how to take advantage of tests—not as an assessment tool, but as a teaching tool <ref>Template:Cite journal</ref> since testing prior knowledge is more beneficial for learning when compared to only reading or passively studying material (even more so when the test is more challenging for memory).<ref>{{#invoke:citation/CS1|citation |CitationClass=web }}</ref>

HistoryEdit

Before much experimental evidence had been collected, the utility of testing was already evident to some perceptive observers including Francis Bacon who discussed it as a learning strategy as early as 1620.<ref>Template:Cite book</ref>

"Hence if you read a piece of text through twenty times, you will not learn it by heart so easily as if you read it ten times while attempting to recite it from time to time and consulting the text when your memory fails."

Towards the end of the 17th Century, John Locke made a similar observation regarding the importance of repeated retrieval for retention in his 1689 book "An Essay Concerning Human Understanding".

"But concerning the ideas themselves, it is easy to remark, that those that are oftenest refreshed (amongst which are those that are conveyed into the mind by more ways than one) by a frequent return of the objects or actions that produce them, fix themselves best in the memory, and remain clearest and longest there."<ref>Template:Cite book</ref>

Towards the end of the 19th century, Harvard psychologist William James described the testing effect in the following section of his 1890 book "The Principles of Psychology"

"A curious peculiarity of our memory is that things are impressed better by active than by passive repetition. I mean that in learning (by heart, for example), when we almost know the piece, it pays better to wait and recollect by an effort from within, than to look at the book again. If we recover the words in the former way, we shall probably know them the next time; if in the latter way, we shall very likely need the book once more." <ref>Template:Cite book</ref>

The first documented empirical studies on the testing effect were published in 1909 by Edwina E. Abbott <ref>Template:Cite journal</ref><ref>Template:Cite book</ref> which was followed up by research into the transfer and retrieval of prior learning.<ref>{{#invoke:citation/CS1|citation

|CitationClass=web

}}</ref><ref>Template:Cite journal</ref> In his 1932 book Psychology of Study, C. A. Mace said:

"On the matter of sheer repetitive drill there is another principle of the highest importance: Active repetition is very much more effective than passive repetition. ... there are two ways of introducing further repetitions. We may re-read this list: this is passive repetition. We may recall it to mind without reference to the text before forgetting has begun: this is active repetition. It has been found that when acts of reading and acts of recall alternate, i.e., when every reading is followed by an attempt to recall the items, the efficiency of learning and retention is enormously enhanced." <ref>Template:Cite book</ref>

Studies in retrieval practice started in 1987 by John. L Richards, who published his findings in a newspaper in New York. Template:Citation needed Much of the confusion around early studies could have been due to constrained approaches not accounting for context.<ref name=":02">Template:Cite journal</ref> In more recent research with contributions from Hal Pashler, Henry Roediger and many others, testing knowledge can produce better learning,<ref>Template:Cite journal</ref><ref name=":10">Template:Cite journal</ref><ref>Template:Cite journal</ref> transfer,<ref name=":6">Template:Cite journal</ref> and retrieval <ref name=":5">Template:Cite journal</ref> results when compared to other forms of study <ref name=":10" /> that often use recognition <ref>Template:Cite journal</ref> like re-reading <ref>Template:Cite journal</ref> or highlighting.<ref>Template:Cite journal</ref>

Retrieval practiceEdit

In recent research, storage strength (how well an item is learned) and retrieval strength (how well an item can be retrieved)<ref name=":7">Bjork, Robert A., and Elizabeth L. Bjork. "A new theory of disuse and an old theory of stimulus fluctuation." _From learning processes to cognitive processes: Essays in honor of William K. Estes_ 2 (1992): 35-67.</ref> have become separate measures for retrieval practice.<ref name=":8">Template:Cite journal</ref> Retrieval strength (also known as recall accuracy) is typically higher for restudied words when tested immediately after practice, whereas tested words were higher as time moves on.<ref name=":9">Template:Cite journal</ref> This suggests using tests is more beneficial for long-term memory and retrieval<ref>Template:Cite journal</ref><ref>Template:Cite journal</ref> which some authors believe is due to limited retrieval success during practice<ref name=":8" /><ref name=":9" /><ref>Template:Cite journal</ref> supporting the idea that tests are learning opportunities.<ref>Template:Cite journal</ref>

Functional magnetic resonance imaging suggests that retrieval practice strengthens subsequent retention of learning through a "dual action" affecting the anterior and posterior hippocampus regions of the brain.<ref>Template:Cite journal</ref> This could support findings that individual differences in personality traits or with working memory capacity, don't seem to have any negative impacts of the testing effect,<ref>Template:Cite journal</ref> with a greater impact for lower ability individuals.<ref>Template:Cite journal</ref>

Despite some doubting knowledge transfer across a topic when testing<ref>Template:Cite journal</ref> with some studies showing contradictory evidence<ref name=":11">Template:Cite journal</ref> suggesting recognition was better than recall,<ref>Template:Cite journal</ref> inferential thinking has been supported<ref>Template:Cite journal</ref> and the transfer of learning is at its strongest with application of theory to practice, inference questions, medical education,<ref>Template:Cite journal</ref> and problems involving medical diagnosis.<ref>Template:Cite journal</ref> The transfer can occur across domains,<ref name=":02" /> paradigms,<ref>Template:Cite journal</ref> and help retention for material not on a final test.<ref>Template:Cite journal</ref> Using retrieval practices also produces less forgetting than studying and restudying<ref>Template:Cite journal</ref> while helping to identify misconceptions and errors<ref>Template:Cite journal</ref><ref>Template:Cite journal</ref><ref>Template:Cite journal</ref> with effects lasting years.<ref>{{#invoke:citation/CS1|citation |CitationClass=web }}</ref>

Repeated testingEdit

{{#invoke:Labelled list hatnote|labelledList|Main article|Main articles|Main page|Main pages}} Repeated testing have shown statistical significance<ref>Template:Cite journal</ref> and results getting better than repeated studying<ref name=":12">Template:Cite journal</ref><ref name=":4">Template:Cite journal</ref> which could be due to testing creating multiple retrieval routes for memory,<ref>Template:Cite journal</ref> allowing individuals to form lasting connections between items,<ref>Template:Cite journal</ref> or blocking information together<ref name=":82">Template:Cite journal</ref> which can help with memory retention<ref>Template:Cite journal</ref> and schema recall.<ref>Template:Cite journal</ref> Using spaced repetition has shown an increase on the testing effect<ref>Template:Cite journal</ref><ref>Template:Cite journal</ref> with a greater impact with a delay in testing,<ref>Template:Cite journal</ref> but the delay could lead to forgetting<ref>Template:Cite journal</ref> or retrieval-induced forgetting.

Delaying the test after a session can have a greater impact<ref name=":14">Template:Cite journal</ref> so studying in the day should be tested in the evening with a delay, but studying in the evening should have an immediate test due the effect sleep has on memory.<ref>Template:Cite journal</ref> Despite divided attention being thought to decrease the testing effect, if it is from a different medium it could enhance the effect.<ref name=":11" />

The rate of forgetting is not affected by the speed<ref>{{#invoke:citation/CS1|citation |CitationClass=web }}</ref> or degree of learning<ref>Template:Cite journal</ref> but by the type of practice involved.<ref name=":14" />

Test difficultyEdit

{{#invoke:Labelled list hatnote|labelledList|Main article|Main articles|Main page|Main pages}} According to the retrieval effort hypothesis, "difficult but successful retrievals are better for memory than easier successful retrievals" which supports the idea of finding a desirable difficulty within the retrieval practice considering our memory biases.<ref name=":7" /> Learning a language was better when using unfamiliar words compared to familiar words, supporting higher difficulty resulting in greater learning.<ref name=":13">Template:Cite journal</ref> The difficulty relates to the likelihood of forgetting<ref>Template:Cite journal</ref> as the harder it is to remember, the more likely you are to remember and retain the information<ref>Template:Cite journal</ref> supporting the notion that more effort is required for longer lasting retention<ref>Template:Cite journal</ref> similar to the depth of processing at encoding.<ref name=":32">Template:Cite journal</ref> Therefore, lack of effort from students studying could be a factor that reduces its efficiency.<ref name=":12"/>

Increased difficulty shows decreased initial performance but increased performance on harder tests in the future, so retention and transfer suffer less when training is difficult.<ref name=":82"/> Even unsuccessful retrieval can enhance learning,<ref>Template:Cite journal</ref> as creating the thought helps with retention<ref>Template:Cite journal</ref> due to the generation effect.<ref>Template:Cite journal</ref><ref>Template:Cite journal</ref> Like with processing time, it is the qualitative nature of the information that determines retention.<ref name=":32" />

Getting feedback helps with learning<ref>Template:Cite journal</ref> but finding a desirable difficulty for the test combined with feedback<ref name=":3">Template:Cite journal</ref> is more beneficial than studying or testing without feedback.<ref>Template:Cite journal</ref><ref>Template:Cite journal</ref> The Read, Recite, Review method<ref>Template:Cite book</ref> has been proposed as a method to combine retrieval practice with feedback.<ref>Template:Cite journal</ref>

Test formatEdit

The test format doesn't seem to impact the results as it is the process of retrieval that aids the learning<ref name=":0">Template:Cite journal</ref> but transfer-appropriate processing suggests that if the encoding of information is through a format similar to the retrieval format then the test results are likely to be higher, with a mismatch causing lower results.<ref name=":2">Template:Cite journal</ref> However, when short-answer tests or essays are used <ref>Template:Cite journal</ref><ref>Template:Cite journal</ref> greater gains in results are seen when compared to multiple-choice test <ref>Template:Cite journal</ref>

Cued recall can make retrieval easier<ref>Template:Cite journal</ref> as it reduces the required retrieval strength from an individual which can help short term results,<ref>Template:Cite journal</ref> but can hinder long term retrieval overtime due to reduced retrieval demand during practice.<ref>Template:Cite journal</ref> Quicker learning can reduce the rate of forgetting for a short period of time, but the effect doesn't last as long as more effortful retrieval.<ref>{{#invoke:citation/CS1|citation |CitationClass=web }}</ref> Cueing can be seen when encoding new information overlaps with prior knowledge making retrieval easier<ref>Template:Cite journal</ref><ref>Template:Cite book</ref> or from a visual or auditory aid.

Prior knowledge seems to increase the impact of retrieval practice,<ref>Template:Cite journal</ref> but should not be seen as a boundary condition as individuals with higher prior knowledge and individuals with lower prior knowledge both benefit.<ref>Template:Cite journal</ref> Pre-testing can be used to get greater results,<ref>Template:Cite journal</ref><ref>Template:Cite journal</ref> and the post-testing can be used to facilitate learning and memory of newly studied information, known as the forward testing effect.<ref>Template:Cite journal</ref> Pre-test or practice test accuracy doesn't predict post test results as time affects forgetting <ref name=":62">Template:Citation</ref>

Pre-testing effectEdit

The pre-testing effect, also known as errorful generation or pre-questioning, is a related but distinct category where testing material before the material has been learned appears to lead to better subsequent learning performance than would have been the case without the pre-test, provided that feedback is given as to the correct answers once the pre-testing phase is completed or further study is undertaken. Pre-testing has been shown to aid learning in both laboratory. and classroom settings.<ref>Template:Cite journal</ref> In terms of specific examples, pre-testing appears to be a beneficial strategy in language learning,<ref name=":13" /><ref name=":82" /> science classrooms generally,<ref>Template:Cite journal</ref> and specifically with lower ability learners in Chemistry.<ref>Template:Cite journal</ref> Pre-testing also seems to be a good way of introducing a lecture series and reduces mind-wandering during lectures.<ref>Template:Cite journal</ref> However, while some studies show that it does not seem to be as effective as post testing overall,<ref>Template:Cite journal</ref> others show that it is at least as effective as post-testing.<ref>Template:Cite journal</ref> The pre-testing effect does appear to be more target focused on the specific material to be learned and should not be seen as correlated with more generalised curiosity.<ref>Template:Cite journal</ref> While the strategy has been demonstrated to have learning benefits across different age groups and subject matters, it also appears to be more suited for more concrete material such as learning facts and concepts. It can be used with a variety of materials, including reading passages, videos, and live lectures.<ref>{{#invoke:citation/CS1|citation |CitationClass=web }}</ref>

Practice methodsEdit

When compared to concept mapping alone, retrieval practice is more beneficial,<ref>{{#invoke:citation/CS1|citation |CitationClass=web }}</ref> despite students not seeing retrieval practice as a useful learning tool.<ref>Template:Cite journal</ref> When combined, learner performance was increased, suggesting concept mapping is a tool that should be combined with retrieval practice<ref>Template:Cite journal</ref> alongside other non-verbal responses.<ref>Template:Cite journal</ref> Retrieval helps with mental organization<ref>{{#invoke:citation/CS1|citation |CitationClass=web }}</ref> which can work well with concept mapping. Multimedia testing can be used<ref>Template:Cite journal</ref> alongside flashcards as a method of retrieval practice but removing cards too early can result in lower long term retention.<ref>Template:Cite journal</ref> Individuals may not correctly interpret the outcome of practice cards<ref>{{#invoke:citation/CS1|citation |CitationClass=web }}</ref> contributing to dropped cards which impact future retrieval attempts <ref name=":72">Template:Cite journal</ref> therefore resulting in lower results due to increased forgetting.<ref name=":14" />

It is advised that students,<ref name=":16">Template:Cite journal</ref> people in care units<ref>Template:Cite journal</ref> and teaching professionals<ref>Template:Cite journal</ref><ref>Template:Cite book</ref><ref>Template:Cite book</ref><ref name=":17" /> use distributed<ref>Template:Cite journal</ref> retrieval practice<ref>Gurung, R.A., 2005. How do students really study (and does it matter). Education, 39, pp.323-340.</ref> with feedback to aid their studies.<ref>Template:Cite journal</ref> Interleaved practice, self-explanation,<ref name=":15" /> and elaborative interrogation<ref name=":16" /> can be useful but need more research.<ref>Template:Cite journal</ref> Summarization can be useful for individuals trained how to use to get the most from it.<ref>Template:Cite journal</ref> Keyword mnemonics and imagery for text have been somewhat helpful but the effects are often short lived.<ref name=":42">Template:Cite journal</ref><ref name=":16" /> However, if each of these methods are integrated with retrieval elements the testing effect is more likely to occur.

Test benefitsEdit

A list of benefits of retrieval practice.<ref name=":92">Template:Citation</ref>

Aids later retention
Identifies knowledge gaps
Aids future related learning
Prevents interference from prior material in future learning
Aids transfer of knowledge to new contexts
Aids knowledge organization
Aids retrieval of untested information
Improves metacognitive monitoring
Provides feedback to instructors
Frequent testing encourages study intentions

QuizzesEdit

A meta-analysis found the following links between frequent low-stakes quizzes in real classes and improved student academic performance:<ref>Template:Cite journal</ref>

There was an association between the use of quizzes and academic performance.
This association was stronger in psychology classes
This association was stronger in all classes when quiz performance could improve class grades.
Students doing well on quizzes tended to lead to students doing well on final exams
Regular quizzing increased the chances of students passing classes

Transfer of learningEdit

Learning using retrieval practice appears to be one of the most effective methods for promoting transfer of learning. In particular the following three techniques have been identified as particularly beneficial for transfer especially when combined with feedback: i) Implementing broad rather than narrow retrieval exercises ii) Encouraging meaningful explanations of concepts or topics iii) Using a variety of complexity and formats with questions such as retrieval questions that require inference.<ref>Template:Cite journal</ref>

ConsiderationsEdit

Complex materialsEdit

Some researchers have applied aspects of cognitive load theory to suggest the testing effect may disappear with increasing task difficulty due to increased element interactivity.<ref>Template:Cite journal</ref> This has been addressed in the literature with studies that show complex learning is benefitted by retrieval practice.<ref>Template:Cite journal</ref> Further research has demonstrated that higher-order retrieval does not need to be based on a lower-level factual recall, and that from the beginning of the learning period, both should be combined for best effect.<ref>Template:Cite journal</ref>

Future researchEdit

It has been suggested that as most studies on the impact of retrieval practice were conducted in WEIRD countries, this could cause a bias which should be explored in further studies.<ref name=":17">Template:Cite journal</ref>

ReferencesEdit

Template:Reflist Template:Education Template:Spaced repetition