The role of working memory abilities in lecture note taking

background image

The role of working memory abilities in lecture note-taking

Dung C. Bui

, Joel Myerson

Department of Psychology, Washington University in St. Louis St. Louis, MO 63130, United States

a b s t r a c t

a r t i c l e i n f o

Article history:
Received 3 October 2013
Received in revised form 25 March 2014
Accepted 11 May 2014

Keywords:
Note-taking
Working memory
Individual differences

The utility of lecture note-taking is well documented, with most studies dedicated to understanding how to max-
imize the bene

fits of note-taking. Far less attention has been focused on understanding the cognitive processes

that underlie note-taking and how the bene

fits of note-taking vary with individual differences in the ability to

carry out these processes. One cognitive ability that has been hypothesized to be important for note-taking is
working memory: the ability to temporarily store and manipulate limited amounts of information. The current
paper addresses why working memory is important for lecture note-taking and reviews studies that have exam-
ined the relationship between individual differences in working memory abilities and individual differences in
note-taking. There is currently a lack of consensus regarding the nature of this relationship, and this review ad-
dresses possible reasons for what may appear to be inconsistent results, including differences in how working
memory and its role in note-taking have been assessed, note-taking modality, and individual differences in
note-taking strategy.

© 2014 Elsevier Inc. All rights reserved.

1. Introduction

The process of note-taking is familiar to just about everyone. Al-

though note-taking occurs in both academic and non-academic con-
texts, the positive consequences of note-taking are most clearly
evident in educational situations where students are evaluated on the
basis of how much information they can retain from lectures. Indeed,
note-taking has long been linked to positive test performance (e.g.,

Armbruster, 2009; Crawford, 1925

). This relationship is not lost on stu-

dents, who acknowledge lecture note-taking as a crucial component of
the educational experience (

Dunkel & Davy, 1989

). In fact, lecturing

constitutes more than 80% of college instructors' teaching methods
(

Wirt et al., 2001

), and therefore it should not be surprising that nearly

all college students take notes in class (

Palmatier & Bennett, 1974; Van

Meter, Yokoi, & Pressley, 1994

), even when they are not explicitly told

to do so by the instructor (

Williams & Eggert, 2002

).

2. A brief overview of lecture note-taking research

DiVesta and Gray (1972)

proposed that note-taking facilitates learn-

ing in two important ways, providing not just what these authors
termed an external storage bene

fit, but providing in addition what they

termed an encoding bene

fit. More specifically, they argued that note-

taking does not just help by recording lecture information for us to

restudy later; importantly, note-taking also helps at the time of the lec-
ture by promoting the encoding of information in ways that facilitate
later retrieval (e.g., by encouraging deeper processing of lecture infor-
mation, as suggested by

Kiewra, 1985

). DiVesta and Gray's seminal

paper stimulated considerable research on note-taking concerned
with assessing the independent contributions of encoding and storage
to the overall effects of lecture note-taking and with determining
which of these processes plays a larger role in driving the bene

fits of

note-taking.

In most of the studies exploring the encoding bene

fit, students lis-

tened to a lecture and were randomly assigned to groups which either
took notes during the lecture or just listened without taking notes. In
order to isolate the encoding bene

fit, students in these studies were

not allowed to review their notes prior to being tested for their memory
of the lecture material. A review by

Kiewra (1985)

identi

fied 56 such

studies, of which 33 found a bene

ficial effect of note-taking. In short, a

signi

ficant effect was observed in most cases, but the evidence for an

encoding bene

fit from note-taking was far from unanimous. Moreover,

although knowing that an effect is observed 59% of the time speaks di-
rectly to its replicability, it provides only indirect evidence of the size
of the effect. After all, effects can be inconsistent, perhaps because of un-
speci

fied moderating variables, and yet be large when they occur.

To address these issues,

Kobayashi (2005)

conducted a meta-

analysis of studies that compared no note-taking to note-taking without
restudy. Overall, Kobayashi found a small positive effect of note-taking
(Cohen's d = .26), consistent with the results of

Kiewra's (1985)

re-

view. Importantly, however, the largest effect sizes were observed for
free recall tests, whereas the smallest effects (not counting cases
where the type of test could not be determined) were for recognition

Learning and Individual Differences 33 (2014) 12

–22

☆ We thank Sandy Hale, Geoff Maddox, and John Nestojko for helpful comments at var-

ious stages of this work.

⁎ Corresponding authors.

E-mail addresses:

dcbui@wustl.edu

(D.C. Bui),

jmyerson@wustl.edu

(J. Myerson).

http://dx.doi.org/10.1016/j.lindif.2014.05.002

1041-6080/© 2014 Elsevier Inc. All rights reserved.

Contents lists available at

ScienceDirect

Learning and Individual Differences

j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / l i n d i f

background image

tests (average Cohen's ds for free recall and recognition tests of .55 and
.18, respectively).

Examining a number of other potential moderator variables,

Kobayashi observed that the encoding bene

fit was smaller when the

presentation mode could be visually distracting, as in the case of an ac-
tual or

filmed lecturer, compared to an auditory recording. Noting that

previous research indicated that presentation mode does not affect
learning from lectures when notes are not taken, Kobayashi concluded
that in studies of note-taking, presentation mode may moderate the
encoding bene

fit because writing requires visual attention (e.g., to pre-

vent going off of the page), and thus attention to other visual stimuli
may limit the ability to take handwritten notes.

Kobayashi's (2005)

interpretation is consistent with research show-

ing that note-quantity is a powerful predictor of test performance even
when students are not allowed to restudy their notes (e.g.,

Aiken,

Thomas, & Shennum, 1975; Fisher & Harris, 1973

). For example,

Bui,

Myerson, and Hale (2013)

, following up on prior

findings showing

that (except in novices) typing is usually faster than handwriting
(

Brown, 1988

), had participants take lecture notes either by typing on

a computer keyboard or by writing them. When participants were
told to try and transcribe a lecture, typing using a computer not only
led to greater note-quantity compared to taking handwritten notes, it
also led to better memory for the lecture material. Moreover, similar ef-
fects of note quantity are obtained when students are allowed to study
their notes (e.g.,

Crawford, 1925; Kiewra & Benton, 1988; Kiewra,

Benton, Kim, Risch, & Christensen, 1995; Nye, Crooks, Powley, & Tripp,
1984

). For example, in two experiments in which study time was

given,

Peverly et al. (2007)

found that transcription speed, as measured

by both an adapted version of the alphabet task (

Berninger, Mizokawa,

& Bragg, 1991

) and the Woodcock

–Johnson Writing Fluency subtest

(

Woodcock & Johnson, 1989

), was a signi

ficant predictor of notes,

which in turn was predictive of test performance.

Evidence for an external storage bene

fit is robust (for a recent review,

see

Kobayashi, 2006

), and relative to the encoding bene

fit, appear to be

more reliable. In studies examining the storage bene

fit, students typically

listen to a lecture while taking notes. Afterwards, some students are
allowed to review their notes, whereas others are not. However, all stu-
dents are then tested for their memory of the lecture material. In the
same review by

Kiewra (1985)

that examined the encoding bene

fit, 17

of the 22 identi

fied studies found that reviewing notes resulted in higher

test performance, a

finding subsequently replicated by

Kiewra et al.

(1991)

.

The higher degree of consensus among relevant studies regarding

the external storage bene

fit (77%) compared to the encoding benefit

(59%) in the literature reviewed by

Kiewra (1985)

raises the question

as to which function is more important. Whereas some studies found
the encoding function to be more bene

ficial (e.g.,

Annis & Davis, 1975;

Barnett, DiVesta, & Rogozinski, 1981

), others reported the external stor-

age function was more important (e.g.,

Fisher & Harris, 1973; Howe,

1970; Kiewra et al., 1991; Rickards & Friedman, 1978

). However, be-

cause utilizing both aspects of note-taking in conjunction appears to
be a more potent learning tool than either aspect on its own (e.g.,

Fisher & Harris, 1973; Kiewra, DuBois, Christensen, Kim, & Lindberg,
1989

),

Kiewra (1985)

pointed out that from the perspective of advanc-

ing educational instruction, it may serve little purpose to focus solely on
comparing each component's contribution.

3. Cognitive demands in lecture note-taking

Despite its bene

fits, lecture note-taking can be cognitively demand-

ing, as it typically involves students having to pay attention to a lecture,
temporarily holding onto the information provided while simulta-
neously organizing that information, and then having to write it down
before it is forgotten. Perhaps as a result, students may adopt different
note-taking strategies whose effectiveness can vary for a number of rea-
sons, among them being individual differences in cognitive ability. That

is, the degree of ef

ficiency with which certain cognitive operations can

be performed varies from one individual to another, and these individ-
ual differences in

fluence how well people are able to perform a complex

task such as note-taking.

One cognitive ability that seems like it should be important for lec-

ture note-taking is working memory, which has been de

fined as the

ability to temporarily hold and manipulate limited amounts of informa-
tion (

Baddeley, 1986, 2007

). Early conceptualizations of working mem-

ory tended to focus on short-term storage and rehearsal (e.g.,

Atkinson

& Shiffrin, 1968

). Newer conceptualizations, however, cover much more

than this, and accordingly, working memory has been extensively stud-
ied under conditions that require not just maintaining items in memory,
but also coordinating and switching back and forth between multiple
tasks (e.g.,

Baddeley, Chincotta, & Adlam, 2001; Engle, Tuholski,

Laughlin, & Conway, 1999

). Such multi-tasking is obviously a funda-

mental aspect of lecture note-taking, and indeed, holding onto informa-
tion while multi-tasking is, at least for some researchers, the very
essence of working memory (

Engle et al., 1999

).

It should be noted, however, that the term working memory has been

used in quite different ways by different researchers. For example, some
cognitive neuroscientists, particularly neurophysiologists, have studied
working memory using tasks that require temporary maintenance of
only a single item (e.g., a spatial location or to-be-remembered response;
for a review, see

Goldman-Rakic, 1996

). Other cognitive neuroscientists,

particularly those using neuroimaging, have used n-back tasks that re-
quire constant updating of information about the most recent n items
(for a review, see

Owen, McMillan, Laird, & Bullmore, 2005

). Experimen-

tal psychologists and individual-differences researchers have studied
working memory using both traditional memory span tasks and com-
plex span tasks that interleave irrelevant processing tasks with presenta-
tion of to-be-remembered items (

Conway et al., 2005

).

Both the difference, if any, between the abilities tapped by simple

and complex span tasks and the role of these abilities in higher-order
cognition remain controversial (e.g.,

Colom, Rebollo, Abad, & Shih,

2006; Engle et al., 1999; Unsworth & Engle, 2007b

). In addition,

n-back tasks and complex span tasks have proved to be only weakly cor-
related (

Redick & Lindsey, 2013

). As a result, we have chosen to use a

broad de

finition of working memory here, and to review the literature

that examines a variety of functions (e.g., the storage, forgetting, and
transformation of temporarily stored information) that are included in
current models of working memory, even if the tasks used to assess
these functions tap only one aspect of what some researchers would
consider working memory.

Perhaps the most well-known model of working memory is that of

Baddeley (1986

;

Baddeley & Hitch, 1974)

, who proposed that the work-

ing memory system includes not only content-speci

fic storage compo-

nents (the phonological loop and visuo-spatial sketchpad for verbal
and visuospatial information, respectively), but also a processing com-
ponent (the central executive) that performs a wide range of functions,
including directing attention to relevant information, inhibiting irrele-
vant information and/or actions, and coordinating cognitive processes
when more than one task must be done at the same time. More recently,

Baddeley (2000)

added a new component, the episodic buffer, to his

model to allow for the interaction between the two storage compo-
nents, as well as to account for the contributions of long-term memory
to performance on working memory tasks.

Baddeley's (1986, 2007)

model has been successful in explaining

many

findings in the short-term and working memory literature, as

well as in stimulating further research. More recently, however, other
models have emerged that provide alternative accounts of the processes
that underlie working memory function. These models differ with
regards to issues such as the contribution of long-term memory to
working memory function, the nature of working memory's limited ca-
pacity, and the role of attention in working memory (for reviews of var-
ious models and theories, see

Conway, Jarrold, Kane, Miyake, & Towse,

2007; Miyake & Shah, 1999

).

13

D.C. Bui, J. Myerson / Learning and Individual Differences 33 (2014) 12

–22

background image

For example,

Unsworth and Engle (2007a)

proposed that individual

differences in working memory ability re

flect differences in the ability

to retrieve information under situations of high interference. Following

Cowan (1988, 1995)

, they argued that the amount of information we

can actively maintain in our focus of attention, or primary memory, is
limited to about four items, and that additional items must be retrieved
from what they termed secondary memory. Working memory tasks like
reading span (

Daneman & Carpenter, 1980

), in which one must remem-

ber the last words of a series of sentences, depend on retrieving items
from secondary memory because the processing of each new sentence
displaces the last word of the previous sentence from primary memory.
Unsworth and Engle posited that individuals with better working mem-
ory ability have higher spans largely because they are better able to use
cues to search secondary memory for these displaced items.

Central to almost all models of working memory are two ideas crit-

ical to understanding individual differences in note-taking: First, the
storage capacity of working memory is limited, and second, working
memory functions consist of not just temporary storage, but also the
manipulation and/or transformation of what is stored, and the mainte-
nance of temporarily stored information when attention is shifted to
performance of other tasks. These additional functions have been de-
scribed as being executive in nature, in that they encompass processes
involved in the control of other cognitive processes.

The emergence of executive functions has been shown to corre-

spond to the development of the frontal lobes (

De Luca & Leventer,

2008; Fuster, 2002

), and individual differences in executive function

have been hypothesized to underlie the relation between working
memory and higher cognitive functions (

Kane & Engle, 2002

). Multiple

working memory tasks have been developed that are hypothesized to
assess executive functions, and these tasks have proved to be good pre-
dictors of higher order abilities such as reading comprehension
(

Daneman & Carpenter, 1980; Daneman & Merikle, 1996

),

fluid intelli-

gence (

Engle et al., 1999; Kane & Engle, 2002

), and complex learning

(

Shute, 1991; Tamez, Myerson, & Hale, 2012

). Like working memory,

however, executive functions are not monolithic, but comprise different
abilities, and thus it is possible that, in addition to storage abilities, a
given working memory task may tap more than one executive function.

4. Individual differences in working memory and note-taking

The idea that working memory and lecture note-taking tap similar

cognitive processes has high face validity. For example, the ability to
temporarily store and manipulate information is part of the de

finition

of working memory (

Baddeley, 1986

) and would appear to be very im-

portant in note-taking as well. When students take notes during a lec-
ture, they often need to hold onto what the instructor is saying while
they try and organize it and paraphrase it more succinctly. Another pro-
cess that working memory and note-taking ability both seem to rely on is
task switching. One type of task commonly used to assess working mem-
ory ability, the complex span task (e.g., counting span, operation span),
requires individuals to continuously switch back and forth between a
memory task and a processing task (

Case, Kurland, & Goldberg, 1982;

Turner & Engle, 1989

). Similar demands are placed on students when

they take notes during a lecture and have to switch back and forth be-
tween listening to the instructor and writing down their notes, except
when they can do both simultaneously, in which case they are using an-
other ability tapped by some working memory tasks

–dual-task coordi-

nation (

Baddeley, 1986

).

Despite the apparently similar cognitive demands of working mem-

ory and note-taking tasks, to date there have been relatively few empir-
ical studies examining the link between them, and the studies that have
examined this relationship have produced mixed results. Some studies
have reported that working memory and note-taking abilities are corre-
lated (e.g.,

Hadwin, Kirby, & Woodhouse, 1999; Kiewra, Benton, &

Lewis, 1987; McIntyre, 1992

), whereas other studies have failed to sup-

port this relationship (e.g.,

Peverly et al., 2013

). The present review will

describe the studies that have explored the relationship between work-
ing memory and lecture note-taking, and discuss possible reasons for
some of the reported inconsistencies.

To begin with, studies examining the link between working memory

and note-taking have measured working memory ability in a variety of
ways, and performance on working memory tasks appears to depend on
multiple abilities (e.g.,

Conway et al., 2007; Hale et al., 2011; Oberauer,

Süß, Wilhelm, & Wittman, 2003

). In studies involving note-taking,

working memory has been measured using short-term memory tasks
that focus on temporary storage ability, information-processing tasks
that assess the ability to reorganize information, and complex span
tasks that get at the ability to multi-task while holding onto new infor-
mation. The

findings obtained using each of these approaches have im-

plications for different aspects of the relationship between working
memory and note-taking.

4.1. Short-term storage and forgetting

One of the

first studies to explore the relationship between working

memory and note-taking was conducted by

DiVesta and Gray (1973)

,

who were interested in the effects of thematic relatedness and continuity
on lecture note-taking. In their study, participants were instructed to
take notes while listening to six 5-min lecture segments. A free recall
test was administered immediately after the last segment ended, followed
by a true-false test. Participants returned a week later to take a second
true-false test with test items not included on the previous test. During
the second session, DiVesta and Gray also administered a Brown

–Peterson

task (

Brown, 1958; Peterson & Peterson, 1959

), which measures short-

term storage, a construct that has been shown to be highly correlated
with measures of working memory (

Engle et al., 1999; Unsworth &

Engle, 2007b

). In this task, participants counted down by threes from

some large number during a retention interval. Scores on this task provide
a measure of the rate of forgetting over the short-term.

In the condition most similar to students' actual classroom experi-

ence, a single 30-min lecture was simply divided into six successive seg-
ments. Under these circumstances, the scores on the Brown

–Peterson

task for participants who took notes were positively correlated with
their performance on the delayed true-false tests, whereas for those
who heard the same lecture but did not take notes, the correlation
was close to zero. Based on these results,

DiVesta and Gray (1973)

con-

cluded that individuals with greater memory spans bene

fit more from

note-taking than those of lower ability.

Because working memory necessarily involves short-term storage,

DiVesta and Gray's (1973)

study can be interpreted as being one of

the

first to suggest that individual differences in working memory

may play a role in lecture note-taking. However, two things should be
noted. First, although the results of the study suggest that the bene

fits

one obtains from taking notes depend on one's working memory ability,
the study does not provide any insight into exactly how working mem-
ory ability in

fluences the note-taking process. Such insight requires ac-

tually examining participants' lecture notes in order to see in what ways
they differ. For example, higher working memory ability individuals
may simply take more notes than lower ability individuals. Second,
DiVesta and Gray claimed to have found a relationship between individ-
uals' memory span and the bene

fits they obtain from note-taking, but

the Brown

–Peterson task that they used measures the rate of forgetting

from short-term memory rather than memory span. Unfortunately, to
the best of our knowledge, to date no studies have directly examined
the relationship between note-taking and memory span as measured
by traditional short-term memory tests (e.g., digit span, word span).

4.2. Processing and reorganizing information

Researchers soon moved beyond measures of short-term storage

and forgetting in their effort to explain individual differences in note-
taking ability. In particular,

Benton, Kraft, Glover, and Plake (1984)

14

D.C. Bui, J. Myerson / Learning and Individual Differences 33 (2014) 12

–22

background image

developed measures in

fluenced by the theory of discourse processing

proposed by

van Dijk and Kintsch (1983)

. According to this theory,

readers build three different mental representations of a discourse: a
surface form representation of the discourse, a

“text” base representa-

tion that captures the propositional structure of the text, and a situa-
tional model of the situation to which the text refers. Deeper
comprehension is hypothesized to require not only a coherent text
base, but also a coherent situation model.

Based on

van Dijk and Kintsch's (1983)

theory,

Benton et al. (1984)

developed three measures of the ability to reorganize information, an
ability critical to building mental representations, of which two have
been used in note-taking research: the word-reordering task and the
sentence-reordering task. In the word-reordering task, participants
view sentences in which the words are out of order, and they are re-
quired to rearrange them to create meaningful sentences. In the
sentence-reordering task, participants view paragraphs in which the
sentences are out of order, and they are required to rearrange the
scrambled sentences to make meaningful paragraphs. Although these
tasks were initially developed in order to study differences in writing
ability, scores on both the sentence-reordering task and the word-
reordering task have since been used to assess individual differences
in the process of manipulating information, which is believed to play a
crucial role in both working memory and lecture note-taking.

Kiewra et al. (1987)

were the

first to examine how well performance

on one of these information-processing tasks could predict note-taking
in lectures. They conducted their study over a four week period during
which the students in an Educational Psychology course were given a
series of twelve 50-min lectures. Immediately following the eighth lec-
ture, the students' notes were collected and photocopied, and then
returned two days later. Five days after the students got their notes
back, they were given a multiple-choice test on that lecture, and then
three weeks later, they were given a course exam that covered all
twelve lectures. Scores on the sentence-reordering task (administered
prior to the

first lecture) correlated positively with both the number

of words and the number of idea units in students' notes, which in
turn predicted performance on the course exam covering all twelve lec-
tures, although not the test on just the eighth lecture. These results are
consistent with the hypothesis that the ability to hold and organize in-
formation plays a critical role in taking lecture notes, and are at least
somewhat supportive of the idea that note-taking ability predicts reten-
tion of lecture information.

Kiewra and Benton (1988)

used a somewhat similar design, the

major difference being that this time the word-reordering task was
used as a working memory measure. The results of this study were sim-
ilar to those previously reported by

Kiewra et al. (1987)

. More speci

fi-

cally, scores on the word-reordering task correlated positively with
note-quantity measures, which in turn predicted performance on a
course exam covering six subsequent lectures. Unlike in their previous
study, note-quantity also predicted performance on a lecture-speci

fic

test, although in this case students got their notes back and studied
them immediately before the test. Finally,

McIntyre (1992)

adminis-

tered both the sentence-reordering and word-reordering tasks to stu-
dents in an Introductory Psychology course, and reported that a
composite score on the two tasks predicted note-taking effectiveness,
a note-quantity measure calculated based on the numbers of main
idea units and details in lecture notes. Note-taking effectiveness, in
turn, predicted performance on a lecture-speci

fic test as well as on the

course midterm.

Taken together, these three studies demonstrate that the ability to

manipulate information is an important factor in individual differences
in effective note-taking, as re

flected in various measures of note-

quantity. It should be noted, however, that the processing tasks in-
volved reorganizing information that was visible to the participants,
rather than information in working memory, and the extent to which
scores on the word- and sentence-reordering tasks correlate with
other measures of working memory is unknown. That said, the word-

and sentence-reordering tasks are important in part because they high-
light a cognitive function that has received relatively little attention in
the cognitive literature despite the fact that the generally accepted def-
inition of working memory includes not just the ability to store informa-
tion temporarily, but also the ability to manipulate information. In fact,
one of the reasons why temporary storage is believed to be important
for higher cognitive functions is because it makes such manipulation
possible (

Baddeley, 1986, 2007

).

4.3. Storage while processing

In addition to storage and manipulation, working memory has been

hypothesized to involve a number of executive functions, including at-
tentional control, task switching, and multi-tasking. Therefore, it is im-
portant to explore the relationship between working memory and
note-taking using working memory tasks that tap these functions. One
such family of working memory tasks are what

Engle et al. (1999)

termed complex span tasks (e.g., reading span, counting span, operation
span) in order to distinguish them from simple span tasks (e.g., digit
span, word span) that only assess temporary storage. The de

fining char-

acteristic of complex span tasks is that the presentation of to-be-
remembered items alternates with the processing of information that
is irrelevant to the memory task. In the original form of the reading
span task, for example, an individual reads a series of sentences but
only needs to remember the

final word of each sentence (

Daneman &

Carpenter, 1980

).

Cohn, Cohn, and Bradley (1995)

were the

first to include complex

span tasks in a study of the relationships among working memory, lec-
ture note-taking, and learning. Working memory was measured by two
different complex span tasks (reading span and operation span), as well
as a simple span task (word span). Students took notes on a lecture on
economic principles, after which they completed a multiple-choice
test. Data were analyzed using multiple regression models in which in
addition to working memory, student attributes such as GPA and SAT
scores were predictors. Although a composite score on the three mem-
ory span tasks was a signi

ficant predictor of test performance, working

memory did not predict note-quantity when student attributes were
statistically controlled. The zero-order correlation between working
memory and note-quantity was not reported.

The

Cohn et al. (1995)

findings are hard to interpret because of the

student attributes being controlled for in the multiple regression analy-
ses. One of them, SAT score, is itself known to be correlated with working
memory (e.g.,

Engle et al., 1999

), a

finding that Engle et al. interpreted as

indicating that working memory ability was a major determinant of per-
formance on the SAT. With regards to GPA, another student attribute
controlled for in Cohn et al.'s analyses, working memory ability is also be-
lieved to play a causal role in educational attainment (

Alloway &

Alloway, 2010; St Clair-Thompson & Gathercole, 2006

). Because differ-

ences in SAT and GPA are most likely to be the effects rather than the
causes of differences in working memory, the

finding that controlling

for these attributes eliminated the signi

ficant relationship between

working memory and note-taking sheds little light on the mechanism
underlying the relation between working memory and note-taking.

Unlike

Cohn et al. (1995)

,

Hadwin et al. (1999)

reported the direct

relationship between working memory and note quality as well on
the effect of statistically controlling for other individual difference mea-
sures. In the Hadwin et al. study, participants took notes on a lecture on
theories of evolution and DNA research. Working memory ability was
measured by a reading span task and note content was assessed using
a measure that gave more weight to higher-level information than to
lower-level information. Working memory ability, as measured by a
reading span task, predicted participants' note content. Although this
relationship was weakened when verbal ability and prior knowledge
were included in the regression model, the interpretation of this

finding,

like the interpretation of the Cohn et al. results, is clouded by the fact

15

D.C. Bui, J. Myerson / Learning and Individual Differences 33 (2014) 12

–22

background image

that the variables being controlled for are more likely to be conse-
quences than causes of individual differences in working memory.

In an important series of studies, Peverly and his colleagues (

Peverly,

Garner, & Vekaria, 2014; Peverly et al., 2007, 2013

) explored the rela-

tionships among lecture note-taking, handwriting speed, working
memory, and other variables. Handwriting speed was a consistent pre-
dictor of note quality, a

finding that Peverly and colleagues interpreted

as re

flecting an important role for individual differences in the speed of

accessing verbal codes. The relationship between working memory, as
measured by the listening span task, an auditory version of the reading
span task (

Daneman & Carpenter, 1980

), and note quality, as measured

by how much participants elaborated on the idea units in their notes,
was inconsistent when group administration of the working memory
task was used (

Peverly et al., 2007, 2013

). In

Peverly et al.'s (2014)

most recent study, however, where the working memory task was ad-
ministered individually, working memory was a signi

ficant predictor

of note quality. The zero-order correlations of note quality with hand-
writing speed and sustained attention (as measured by the Lottery sub-
test of the Test of Everyday Attention;

Robertson, Ward, Ridgeway, &

Nimmo-Smith, 1994

) were signi

ficant, but note quality was not corre-

lated with executive attention (as measured by the Stroop task).

When

Peverly et al. (2014)

used hierarchical regression to analyze

their results, note quality was the only signi

ficant predictor of test per-

formance. In interpreting these results, Peverely et al. noted similarities
to cognitive cascade models relating developmental changes in process-
ing speed to changes in working memory (

Fry & Hale, 1996; Kail &

Salthouse, 1994

). Indeed, it would appear that their data would be

well described by a cascade model in which taken together with greater
sustained attention leads to better lecture note-taking, and thereby to
better test performance.

It should be noted that the studies reviewed so far all examined the

taking of handwritten notes, yet

Peverly et al. (2014)

reported that 50%

of the respondents in an ongoing survey from their laboratory of under-
graduates from different universities indicated that they take lecture
notes with a laptop computer. In the one study to examine the role of
working memory in note- taking using computers,

Bui et al. (2013)

found that working memory, assessed with a reading span task, predict-
ed note-quantity when participants were instructed to take organized
notes. Working memory did not predict note-quantity, however,
when participants were instructed to try and transcribe the lecture by
recording everything the lecturer said. Interestingly, working memory
predicted delayed test performance regardless of the note-taking strat-
egy that was used, but when the testing was immediate, working mem-
ory predicted test performance only for those who took organized
notes. These

findings, which are discussed in greater detail below, sug-

gest that

finer-grained analyses based on differences between samples

(

Peverly et al., 2007

) and note-taking modalities and strategies (

Bui

et al., 2013

), among other factors, are needed to determine under

what conditions working memory ability predicts note-taking and test
performance, and under what conditions it does not. Accordingly, we
next consider various factors that may modulate the relationships
among these variables.

5. Factors affecting the role of working memory

To brie

fly summarize the findings reviewed so far, positive relation-

ships among working memory abilities, lecture note-taking, and test
performance have been found when working memory is measured
using both short-term memory tasks and complex span tasks, as well
as when it is measured using information-processing tasks that require
reorganizing information. These different types of tasks may measure
different working memory abilities, however, and evidence is emerging
that the strategies students use in lecture note-taking may affect wheth-
er working memory predicts note-taking and test performance (

Bui

et al., 2013

). It is likely that other factors are involved as well, at least

some of which are more or less under the control of instructors (e.g.,

lecture pace and length). Finally, technological innovations may be in
the process of altering the relationships among working memory,
note-taking, and learning, even as researchers and educators try to un-
derstand them.

5.1. Working memory tasks

Whether or not a relationship is observed between working memo-

ry and note-taking ability appears to be partly determined by exactly
what tasks are used to assess working memory ability and how they
are administered. Signi

ficant correlations between working memory

and note-taking have been found consistently when working memory
is assessed using tasks that require participants to reorganize informa-
tion (

Kiewra & Benton, 1988; Kiewra et al., 1987; McIntyre, 1992

).

Mixed results have been obtained with group-administration of com-
plex span tasks (

Peverly et al., 2007, 2013

). However, signi

ficant corre-

lations have been reported more consistently when complex span tasks
are individually administered (

Bui et al., 2013; Hadwin et al., 1999;

Peverly et al., 2014

), although as discussed in the following section,

note-taking strategy may also play a role (

Bui et al., 2013

).

Complex span tasks (e.g., reading span) have been the most frequently

used measures of working memory in studies of individual differences in
cognitive ability (

Conway et al., 2005

), whereas information-processing

tasks (e.g., word and sentence reordering) like those used by Kiewra
and others (

Kiewra & Benton, 1988; Kiewra et al., 1987; McIntyre,

1992

) that focus on the ability to reorganize information have rarely

been used, despite the fact that according to

Baddeley (1986, 2007)

, ma-

nipulating information is one of the de

fining characteristics of working

memory. Although the word- and sentence-reordering tasks do assess
the relatively neglected manipulation aspect of working memory, the
storage aspect of these tasks is minimal because the items to be reordered
remain visible. In contrast, complex span tasks depend partly on storage
capacity, but they are de

fined by the need to multi-task, maintaining in-

formation while alternating between processing incoming information
and performing an irrelevant secondary task. Indeed, it is this multi-
tasking aspect, with its dependence on executive processes, which has
often been assumed to underlie the relationship between working mem-
ory and

fluid intelligence (e.g.,

Engle et al., 1999

), although researchers'

views on this point may be changing (e.g.,

Colom et al., 2006; Friedman

et al., 2006; Redick & Lindsey, 2013

; Unsworth & Engle, 2007a).

Surprisingly little is known about the role of storage capacity in

note-taking. The Brown

–Peterson task used by

DiVesta and Gray

(1973)

assesses forgetting, not storage, although they described as a

memory span measure. In fact, simple span tasks that focus on tempo-
rary storage (e.g., digit span, word span) have hardly been used in re-
search on note-taking. The one case that we know of is the word span
task in

Cohn et al. (1995)

, but they also administered two complex

span tasks and did not report correlations for the three span tasks sep-
arately. To the best of our knowledge, only two tasks have been devised
that attempt to assess both the storage and manipulation aspects of
working memory, the alphabet span task (

Craik, 1986

) and the letter-

number sequencing task (Wechsler Adult Intelligence Scale, WAIS-IV;

Wechsler, 2008

), neither of which has been used in research on lecture

note-taking. Studies that assess the combined contributions of tempo-
rary storage and manipulation to note-taking or that compare their con-
tributions could potentially shed considerable light on the bases for
individual differences in note-taking ability, as might studies that com-
pare the contributions of the storage and executive, multi-tasking as-
pects of complex span tasks.

One of the reasons that the working memory construct has largely

replaced the short-term memory construct in contemporary cognitive
psychology is because of the richness of the former construct, which
subsumes the storage function of short-term memory and adds addi-
tional components to form a working memory system. Previous studies
(

Kiewra & Benton, 1988; Kiewra et al., 1987; McIntyre, 1992

) clearly

establish the ability to manipulate information as an important

16

D.C. Bui, J. Myerson / Learning and Individual Differences 33 (2014) 12

–22

background image

determinant of individual differences in note-taking, but future re-
search would do well to examine the roles that all of the various compo-
nent abilities, separately and in combination, play in note-taking as part
of the effort to understand the relationship between individual differ-
ences in working memory abilities and academic performance.

5.2. Note-taking strategy

For the most part, researchers examining the relationship between

working memory and note-taking have not instructed their participants
on how to take their notes. Indeed, relatively few students ever receive
formal instruction in note-taking. Perhaps as a result, different students
may use different note-taking strategies, and students may alter the
strategy they use from one course or lecture situation to the next
(

Piolat, Olive, & Kellogg, 2005; Van Meter et al., 1994

). Moreover,

note-taking strategies may differ in their reliance on working memory
(

Bui et al., 2013

), and thus variability in note-taking strategies can po-

tentially mask a correlation between working memory and note-
taking that would have been observed if all participants had used strat-
egies that are dependent on working memory.

For example, consider the results for those who used a computer to

take notes in the study by

Bui et al. (2013)

discussed previously. Half of

these participants were told to try and transcribe the lecture, recording
everything that the lecturer said, and the other half were told to take or-
ganized notes, paraphrasing what was said. As mentioned previously,
when participants in the Bui et al. study were told to organize their
notes, working memory ability was a signi

ficant predictor of note-

taking. From the standpoint of understanding the role of note-taking
strategies, however, what is important is that this was only true when
participants took organized notes. For the participants who were told
to transcribe the lecture, working memory ability was not a signi

ficant

predictor of note-taking. These results suggest that the speci

fic note-

taking strategies that students use can dictate whether or not working
memory ability will play a mediating role in note-taking performance.
These strategies may vary not only across individuals, but also across
samples and situations (for a review, see

Piolat et al., 2005

), complicat-

ing efforts to understand the relationship between working memory
and lecture note-taking.

According to

Piolat et al. (2005)

, different note-taking strategies

place differential demands on executive and/or attentional processes
like those involved in working memory tasks. Much of the evidence
for this view comes from studies using a dual-task paradigm. As may
be seen in

Fig. 1

, participants are much faster to respond to a tone

while simply reading or copying text than when they are composing, re-
vising, or translating text. Piolat et al. hypothesized that this is because
tasks like composing and translating require much more cognitive ef-
fort, in that they place a much greater demand on executive/attentional
resources than merely reading or copying. Similarly, they argued, lec-
ture note-taking takes much more cognitive effort than merely copying
text, and nearly as much as composing or translating text, again as evi-
denced by a comparison of response times. Response times to a tone are

also longer when taking lecture notes than when taking notes while
reading, which may be attributed to the severe time pressure involved
in not falling behind the lecturer.

In addition,

Piolat et al. (2005)

argued that these long response times

re

flect the fact that note-taking is typically not the simple transcription

of information that is heard or read, but rather is a much more compli-
cated process that depends heavily on the executive functions involved
in managing multiple cognitive processes concurrently (

Engle et al.,

1999

). They suggested further that the speci

fic strategy used in a partic-

ular situation probably depends on the skills and abilities of the note-
taker. Simply transcribing information might seem to be a suboptimal
strategy, particularly from a depth-of-processing perspective (

Craik &

Tulving, 1975; Kiewra, 1985

), yet it is reported to be common under

certain circumstances (

Van Meter et al., 1994

). Moreover, as we shall

show in the next section, simple transcription can lead to greater
note-quantity and better test performance than organized note-taking
when students are using computers (

Bui et al., 2013

). Indeed, it is likely

that no single note-taking strategy is best for all students or all situa-
tions. As a result, it may be important to expose students to a variety
of note-taking strategies in order to allow them to make informed deci-
sions as to which strategies work best for them and when.

5.3. Lecture pace

Although students can choose which note-taking strategy to use,

one factor in lectures that they usually are unable to control is the
speed at which the instructor lectures. Little work has been directed to-
wards understanding how lecture pace affects students' retention of lec-
ture information when taking notes, which is surprising given that pace
should directly impact the amount of information that students are able
to record. Students often report that rapidly delivered lectures prevent
them from using their preferred methods of note-taking (e.g.,

Van

Meter et al., 1994

). As

Piolat et al. (2005)

pointed out, the average

speaking rate exceeds the average writing rate, and those students
who are already having dif

ficulty trying to take notes when lectures

are delivered at an average speaking rate would be at an even greater
disadvantage in faster paced lectures.

The

first study to explore the effects of lecture pace was

Peters

(1972)

, who had participants either listen to a normally paced lecture

(146 words per minute) or a fast paced lecture (202 words per minute).
Peters failed to

find an overall effect of lecture pace on test performance,

but for individuals who scored low on listening ef

ficiency (measured as

recall of a list of de

finitions), performance on a multiple-choice test was

better when they were not required to take notes, or when they took
notes and the lecture pace was normal, than when they tried to take
notes on a lecture delivered at a fast pace. These results suggest that in-
dividual differences in a student's listening ability interact with the rate
of a lecture's presentation in determining whether note-taking provides
either a bene

fit or detriment to performance on a test of comprehension

of the lecture material.

The effect of lecture pace on note-taking was also examined by

Aiken

et al. (1975)

. The lecture was divided into four segments with breaks in

between the segments, and in addition to pace, Aiken et al. also varied
whether participants took notes while listening to the lecture or during
the breaks. Test performance was better when the speech rate was nor-
mal than when it was fast. Interestingly, test performance was best for
participants who took their notes during the breaks and immediately
following the lecture, compared to those who took their notes while lis-
tening to the lecture. Allowing breaks for note-taking effectively dimin-
ishes the divided attention and multi-tasking demands of traditional
note-taking, and may thus weaken the relationship between note-
taking and working memory ability, at least as assessed by complex
span tasks.

To the best of our knowledge, no studies have followed up on the

findings by

Peters (1972)

and

Aiken et al. (1975)

regarding the interac-

tion of lecture pace and individual differences in ability, although

Ruhl

Fig. 1. Dual-task measures of the cognitive effort (in milliseconds) required by different
kinds of information processing.
Adapted from Fig. 4 of Piolat et al. (2005).

17

D.C. Bui, J. Myerson / Learning and Individual Differences 33 (2014) 12

–22

background image

and Suritsky (1995)

have successfully used pauses to enhance lecture

note-taking by learning disabled college students.

Salthouse (1996)

proposed two mechanisms to explain the well-established relationships
among individual differences in processing speed, working memory and
higher order cognition, and his processing speed theory would appear
to be directly applicable to note-taking. His limited time mechanism,
which operates exactly as the name suggests, captures the deleterious
effects of rapid lecture pace and predicts that slower processers will
be particularly affected. This prediction is supported by the consistent
finding that the difference between young and older adults' immediate
recall of spoken words and sentences increases as a function of speech
rate: Older adults, who are much slower processors, show much larger
decreases in recall than young adults, and this is exacerbated by
increases in sentence complexity (e.g.,

Wing

field, Peelle, & Grossman,

2003; Wing

field, Poon, Lomabardi, & Lowe, 1985

). Similarly, one

would expect that increases in lecture pace and the dif

ficulty of the ma-

terial would have more adverse effects on immediate recall and note-
taking by slower young adults.

Salthouse (1996)

also postulated a simultaneity mechanism that cap-

tures the interaction between processing speed and forgetting rate: The
longer it takes one to perform task, the longer one has to remember the
information required by that task. Thus, for example, slower individuals
attempting to take organized notes are more likely to forget some of the
information they are trying to organize than faster individuals. It may be
noted that both the limited time and simultaneity mechanisms operate
here, so that slower processors are doubly vulnerable. It should be
pointed out that in the case of note-taking, Salthouse's theory predicts
that being physically slow at recording information can be just as dele-
terious as being mentally slow to process it. This, in turn, suggests that
an intervention that speeds up the process of recording information
could have much the same bene

fit as becoming a faster information

processor. Fortunately, it may be much easier and more practical to
speed up recording than processing, which brings us to the topic of
note-taking with computers,

5.4. Note-taking modality

The

first researchers to study lecture note-taking probably could not

have envisioned a classroom where note-taking would be done using
computers. However, rapid advancements in technology have resulted
in computers increasingly being incorporated into students' classroom
learning experiences. Furthermore, the combination of the portability
and

flexibility of laptop and tablet computers has resulted in a steady in-

crease not only in the percentage of college students who own a com-
puter (

Smith & Caruso, 2010

), but also in students' preference for

taking notes using their computers (

Efaw, Hampton, Martinez, &

Smith, 2004

). In addition to the convenience that computers provide,

taking notes with a computer may also increase transcription speed
during lecture note-taking because typing is typically faster than hand-
writing (e.g.,

Horne, Ferrier, Singleton, & Read, 2011; Rogers &

Case-Smith, 2002

). As a result of the increase in transcription speed

that is possible, using computers may offer students the opportunity
to take more lecture notes, and given that note-quantity is a good pre-
dictor of test performance (e.g.,

Bui et al., 2013; Kiewra & Benton,

1988; Kiewra et al., 1995

), computer note-taking may result in better

scores on tests that assess memory for lecture material.

It should also be pointed out that the role of working memory in lec-

ture note-taking may differ depending on whether the notes are hand-
written or typed into a computer.

Olive and Piolat (2002)

have argued

that handwriting places extra cognitive demands on note-takers be-
cause they not only have to execute the motor movements needed to
write down information, but they also have to monitor the spatial posi-
tion of their hands in order to make sure that the letters and words will
be appropriately spaced and that the next words will

fit on the current

line, all while keeping track of what the lecturer is currently saying. In
contrast, taking notes with a computer eliminates some of these

cognitive demands because word processors automatically maintain
consistent spatial alignment.

Bui et al. (2013)

had participants either take notes with a pencil or

pen or type them into a computer. As may be seen in

Fig. 2

, using a com-

puter resulted in participants including a larger proportion of the idea
units from the lecture in their notes, particularly when they were
using a transcription strategy (i.e., trying to record everything that
was said) rather than following the more traditional practice of taking
organized notes. Notably, the greater note quantity observed when par-
ticipants used a computer was not simply the result of their including
more unimportant details. Their notes also contained more important
details, and because note quantity is a means and not an end in itself,
it should be pointed out that the computer users also recalled more im-
portant details on a subsequent test than those who took notes by hand.

Although note-taking strategy had no effect on either the note-

quantity or test performance of those who took notes by hand, for par-
ticipants who took notes with a computer, using the transcription strat-
egy resulted in their both recording more ideas units in their notes and
recalling more idea units later. This interaction between note-taking
strategy and modality may be clearly seen in

Fig. 2

, and it illustrates

what may become a signi

ficant issue in note-taking research: The fac-

tors affecting note-taking and its relation to test performance may be
different depending on whether notes are taken by hand or with a
computer.

Further, the

Bui et al. (2013)

findings suggest that when the

computer's potential for increasing note-quantity is realized, as when
students use a transcription strategy, it may change the effects of
other important factors. For example, when there was no opportunity

Note-quantity

Proportion of Idea Units

0.0

0.2

0.4

0.6

0.8

Test performance

Proportion of Idea Units

0.0

0.2

0.4

0.6

0.8

Hand

Computer

Hand

Computer

Organize Transcribe

Organize Transcribe

Organize Transcribe

Organize Transcribe

Fig. 2. Proportion of idea units included in notes and recalled on a later test as a function of
note-taking strategy and modality.
Data from

Bui et al. (2013)

.

18

D.C. Bui, J. Myerson / Learning and Individual Differences 33 (2014) 12

–22

background image

for studying one's notes, those who took organized notes showed much
less forgetting over a 24-h delay than those using a transcription strate-
gy; if participants had an opportunity to study their notes, however, then
those who used a transcription strategy bene

fited greatly from this op-

portunity whereas those who took organized notes did not. In fact, tran-
scribing one's notes and then going over them immediately after the
lecture resulted in much better performance on a delayed test than tak-
ing and studying organized notes. These

findings may legitimize a much

maligned note-taking strategy, but they also provide further support for
the idea, suggested above, that

findings from handwritten note-taking

may not necessarily generalize to note-taking with computers.

In the

Bui et al. (2013)

study, this may be seen in the fact that the

transcription strategy produces qualitatively as well as quantitatively
different effects on note-taking depending on whether notes are taken
by hand or by computer (see

Table 1

). Whereas transcribed notes had

the same total number of idea units as organized notes when notes
were taken by hand, they had more unimportant details but fewer
main ideas and less important details, just as proponents of organized
note-taking might have predicted. When a computer was used, howev-
er, taking transcribed notes increased the total number of idea units
without sacri

ficing either the number of main ideas or important

details.

For the current effort, we revisited the data from the

Bui et al. (2013)

study, combining the data from all of the participants who took notes
using a computer. This effort yielded a total of 76 participants across
all three experiments who tried to transcribe the lecture and 76 who
took organized notes. As may be seen in

Fig. 3

, low span individuals

(i.e., those whose scores on a reading span task were in the bottom
quartile) who used a transcription strategy not only had as many idea
units in their notes, on average, as high span individuals (i.e., those
whose scores were in the top quartile) using the same strategy, they
also had signi

ficantly more idea units in their notes than high span indi-

viduals who took organized notes (43% vs. 36%).

The conditions under which the transcription strategy leads to better

test performance remain to be determined: It does lead to better test
performance when notes are taken with a computer but not when
they are handwritten, and it leads to better test performance when
the test is given immediately after the lecture but not when the test is
delayed unless students are given the opportunity to study their notes
(

Bui et al., 2013

). Clearly, the constraints on the transcription strategy

need to be determined before recommending its general use by stu-
dents who do not have good working memory ability. Nevertheless,
what does seem clear is that questions concerning working memory,
note-taking strategies, and learning may have different answers de-
pending on the note-taking modality involved.

Computers have the potential to serve as powerful tools for helping

students take lecture notes, and take more of them, which in turn can
improve their test performance. Importantly, students are now given a
means by which they can take notes while relying less on cognitive abil-
ities (e.g., working memory) that otherwise are critical for carrying out
this task. Put another way, computers may help student users of

fload

some of the cognitive demands present in lecture note-taking. For
those who are strong in the cognitive abilities on which note-taking

traditionally relies, this may not make much of a difference, but for
those who are average or below-average, this use of computers may
help level the educational playing

field.

The expanded role of computers in education will likely have a vari-

ety of consequences. For example, a recent study by

Barrett et al. (2014)

raises the possibility that students who take lecture notes using a com-
puter do better on exams if they can respond to questions on their com-
puters than if they have to write their answers by hand. Although the
Barrett et al. study was a small one, their results are consistent with
the principle of encoding speci

ficity (

Tulving & Thomson, 1973

), and

are an important reminder of the sometimes unintended results of tech-
nological innovation.

Finally, the results of the

Bui et al. (2013)

study highlight the impor-

tance of studying note quality as well as note quantity: Note-taking mo-
dality (i.e., using a computer versus taking notes by hand) interacted
with strategy to produce qualitatively different effects on note contents.
Measuring qualitative differences in note-taking bene

fits from careful

analysis of lecture contents as well as note contents, and Bui et al.
were fortunate to be able to use a text previously analyzed by

Rawson

and Kintsch (2005)

in terms of main ideas and both important and un-

important details, just as Peverly and his colleagues (e.g.,

Peverly et al.,

2007

) based their analyses of note quality on a lecture previously ana-

lyzed by

Brobst (1996)

. Future research on note-taking will likely

need to compare handwritten notes with those taking with a variety
of digital instruments, including tablets and laptops as well as other de-
vices (e.g., wearable devices) that are only beginning to appear. We ex-
pect that the potential bene

fits of analyzing qualitative differences in

the contents of individuals' notes will outweigh the effort that will be
required in conducting careful, qualitative analyses of the contents of
both lectures and notes on topics taken from many different disciplines.

6. Concluding remarks

The notion that working memory ability is important for lecture

note-taking has been around since at least the 1970s (e.g.,

DiVesta &

Gray, 1973

). After all, note-taking involves recording (writing or typing)

either a verbatim or transformed (e.g., summarized) version of what has
just been said, all the while continuing to process and maintain the in-
formation that is currently being said. This, of course, is exactly the
kind of function that

Baddeley (1986)

envisioned when he described

working memory as a system that provides temporary storage and ma-
nipulation of the information necessary for complex cognitive tasks, be-
cause note-taking, especially taking organized notes, is de

finitely a

complex cognitive task.

Despite the intuitive appeal of this idea, however, studies of the rela-

tionship between working memory, note-taking, and test performance

Table 1
Proportion of idea units in notes (standard deviations in parentheses) in Exp. 1 of

Bui et al.

(2013).

Group

Overall

Main

Important Details

Unimportant Details

Hand

Organize

.28 (.12)

.54 (.15)

.41 (.08)

.20 (.16)

Transcribe

.28 (.10)

.46 (.16)

.33 (.12)

.24 (.12)

Computer

Organize

.34 (.13)

.50 (.16)

.48 (.10)

.26 (.17)

Transcribe

.44 (.12)

.59 (.15)

.53 (.13)

.41 (.14)

Note. Numbers indicate the mean proportions of the total of 125 idea units, 8 main points,
15 important details, and 16 unimportant details in the passage.

Working Memory (z-score)

-3

-2

-1

0

1

2

Notes (% Idea Units)

0

10

20

30

40

50

60

70

Organize

Transcribe

Fig. 3. Regression lines for quantity of notes as a function of working memory ability for
the groups using the transcribe and organize note-taking strategies.
Data from

Bui et al. (2013)

.

19

D.C. Bui, J. Myerson / Learning and Individual Differences 33 (2014) 12

–22

background image

have sometimes produced what appear to be inconsistent results. As we
have shown, however, some of this inconsistency may be due to
lumping together zero-order correlations with multiple regression re-
sults, at least some it is likely attributable to the way in which working
memory was assessed, and another portion may be attributable to
variation in note-taking strategy and differences between samples.
Any inconsistency, moreover, must be interpreted in the context of
changing views of individual differences in working memory. Although

Baddeley (1986)

originally described working memory in terms of a

multi-component system, until relatively recently, individual differ-
ences researchers tended to focus on just one of Baddeley's components,
the central executive, as being the essence of a unitary working memory
ability (

Engle et al., 1999; Kane & Engle, 2002

). From this perspective,

complex span tasks were seen as the way to measure this unitary con-
struct. Subsequent research, however, has challenged this view, and
there is an emerging consensus that performance of even this one
type of working memory task may involve a number of abilities, and
that other types of tasks may tap other abilities (e.g.,

Hale et al., 2011;

Oberauer et al., 2003; Redick & Lindsey, 2013; Unsworth & Spillers,
2010

).

Consider, for example, n-back tasks, which many cognitive neurosci-

entists have seen as capturing the essence of working memory, a role
once played by complex span tasks for many psychologists. n-back
tasks focus on the need to continually update what is temporarily
being stored, something which is required in lecture note-taking al-
though to date, such tasks have not been used in note-taking research.
Perhaps surprisingly, complex span tasks and n-back tasks have turned
out to be only weakly correlated (for a meta-analytic review, see

Redick

& Lindsey, 2013

), even though both types of tasks are correlated with

fluid intelligence. Moreover, most of the intelligence-related variance
that each type of working memory task explains is unique (

Kane,

Conway, Miura, & Col

flesh, 2007

), and training that improves perfor-

mance on one type of task results does not result in transfer to tasks
of the other type (e.g.,

Jaeggi, Buschkuehl, Jonides, & Perrig, 2008;

Lilienthal, Tamez, Shelton, Myerson, & Hale, 2013

).

Because no single task captures the essence of working memory and

different tasks tap different sets of working memory abilities whose
contributions may vary across individuals, strategies, and situations, it
is perhaps not surprising if some inconsistencies are observed. Clearly,
the job of future research will be try to discover what underlies the com-
plex relationship between working memory and note-taking even as re-
searchers try to understand the nature of working memory itself.

One reason why this may be important for educational purposes is

that although evidence is accumulating that general working memory
ability cannot be improved through training (for a recent review, see

Shipstead, Redick, & Engle, 2012

), certain component skills (e.g.,

updating) can be improved by training (e.g.,

Dahlin, Neely, Larsson,

Bäckman, & Nyberg, 2008; Lilienthal et al., 2013

). Thus, to the extent

that speci

fic weaknesses in students' note-taking can be identified, it

may be possible to use training to remediate these de

ficits, and a clearer

understanding of the relationship between working memory and note-
taking may help in identifying which skills to target. For example, as re-
search using reordering tasks has shown, the ability to manipulate in-
formation is a key component of both working memory and taking
organized notes (e.g.,

Kiewra & Benton, 1988; McIntyre, 1992

). If the fu-

ture studies

find that information-manipulation can be improved with

training and that such improvements actually bene

fit note-taking, for

example, then analogous training research with other processes com-
mon to both working memory and note-taking could follow, potentially
opening up new avenues for the application of cognitive research to ed-
ucational interventions.

Further, recent research suggests that the cognitive demands of

note-taking may be changing alongside technological advancements,
and students may need help in adapting to these new demands. For ex-
ample, not only has the number of students who own portable com-
puters increased dramatically (

Smith & Caruso, 2010

), but so has the

proportion of students who take lecture notes using these computers
(

Efaw et al., 2004

). Because computerized note-taking is faster than

hand-writing notes, it has the potential to at least partially alleviate
the cognitive demands imposed by factors such as lecture pace, but
whether or not this occurs may depend on what note-taking strategies
students use (

Bui et al., 2013

).

In addition, online learning is becoming increasingly popular even as

it is changing. The

first massive open online course (MOOC) was offered

at the University of Manitoba in 2008 (

Mackness, Mak, Fai, & Williams,

2010

), and a number of major universities (including Stanford, Princeton,

the University of Pennsylvania, and the University of Michigan) are
partnering to offer further MOOCs (

Pappano, 2012

). Having access to lec-

tures on demand may allow students to listen to a lecture in chunks, rath-
er than all at once. This may be particularly useful in situations where
note-taking is very cognitively demanding. Instead of missing important
points in the lecture, students can simply pause or repeat critical lecture
segments. Research that could provide instructors and students with
guidance regarding optimal segment durations would be very valuable.
The results of one early study suggest that taking notes during pauses be-
tween segments may be more effective than taking notes continuously
(

Aiken et al., 1975

), perhaps because it minimizes the divided attention

and multi-tasking demands associated with traditional note-taking.
Such possibilities clearly seem worth exploring.

Both the ways in which information can be accessed by students and

the technology available to them for recording and organizing this infor-
mation are changing, and researchers and educators must consider the
cognitive demands presented by these novel situations and how stu-
dents with different sets of strengths and weaknesses might best cope
with these demands. We believe that one way to begin is by trying to
more fully understand the demands on their multiple working memory
abilities when they try to record and organize information.

References

Aiken, E. G., Thomas, G. S., & Shennum, W. A. (1975).

Memory for a lecture: Effects of

notes, lecture rates, and informational density. Journal of Educational Psychology, 67,
430

–444.

Alloway, T. P., & Alloway, R. G. (2010).

Investigating the predictive roles of working memory

and IQ in academic attainment. Journal of Experimental Child Psychology, 106, 20

–29.

Annis, L. F., & Davis, J. K. (1975).

Effect of encoding and an external memory device on

notetaking. Journal of Experimental Education, 44, 44

–46.

Armbruster, B. B. (2009).

Taking notes from lectures. In R. F. Flippo, & D. C. Caverly (Eds.),

Handbook of college reading and study strategy research (pp. 220

–248) (2nd ed.).

New York, NY: Routledge.

Atkinson, R. C., & Shiffrin, R. M. (1968).

Human memory: A proposed system and its con-

trol processes. The psychology of learning and motivation, 2, 89

–195.

Baddeley, A. D. (1986).

Working memory. Oxford, England: Oxford University Press.

Baddeley, A. D. (2000).

The episodic buffer: A new component of working memory?

Trends in Cognitive Sciences, 4, 417

–423.

Baddeley, A. D. (2007).

Working memory, thought, and action. Oxford, England: Oxford

University Press.

Baddeley, A. D., Chincotta, D., & Adlam, A. (2001).

Working memory and the control of action:

Evidence from task switching. Journal of Experimental Psychology: General, 130, 641

–657.

Baddeley, A. D., & Hitch, G. (1974).

Working memory. In G. H. Bower (Ed.), The psychology

of learning and motivation: Advances in research and theory, Vol. 8. (pp. 47

–89). New

York: Academic Press.

Barnett, J. E., DiVesta, F. J., & Rogozinski, J. T. (1981).

What is learned in note taking?

Journal of Educational Psychology, 73, 181

–192.

Barrett, M. E., Swan, A. B., Mamikonian, A., Ghajoyan, I., Kramarova, O., & Youmans, R. J.

(2014).

Technology in note taking and assessment: The effects of congruence on stu-

dent performance. International Journal of Instruction, 7, 49

–57.

Benton, S. L., Kraft, R. G., Glover, J. A., & Plake, B. S. (1984).

Cognitive capacity differences

among writers. Journal of Educational Psychology, 76, 820

–834.

Berninger, V. W., Mizokawa, D., & Bragg, R. (1991).

Theory based diagnosis of remediation

of writing disabilities. Journal of School Psychology, 29, 57

–79.

Brobst, K. E. (1996).

The process of integrating information from two sources, lecture and

text. (Doctoral dissertation, Teachers College, Columbia University, 1996).
Dissertation Abstracts International, 57, 217.

Brown, J. (1958).

Some tests of the decay theory of immediate memory. Quarterly Journal

of Experimental Psychology, 10, 12

–21.

Brown, C. M. (1988).

Comparison of typing and handwriting in

“two-finger typists”.

Proceedings of the Human Factors Society, USA, 32. (pp. 381

–385).

Bui, D. C., Myerson, J., & Hale, S. (2013).

Note-taking with computers: Exploring alterna-

tive strategies for improved recall. Journal of Educational Psychology, 105, 299

–309.

Case, R., Kurland, D. M., & Goldberg, J. (1982).

Operational ef

ficiency and the growth of

short-term memory span. Journal of Experimental Child Psychology, 33, 386

–404.

20

D.C. Bui, J. Myerson / Learning and Individual Differences 33 (2014) 12

–22

background image

Cohn, E., Cohn, S., & Bradley, J. (1995).

Notetaking, working memory, and learning in prin-

ciples of economics. Research in Economic Education, 26, 291

–307.

Colom, R., Rebollo, I., Abad, F. J., & Shih, P. C. (2006).

Complex span tasks, simple span tasks,

and cognitive abilities: A reanalysis of key studies. Memory & Cognition, 34, 158

–171.

Conway, A. R. A., Jarrold, C., Kane, M. J., Miyake, A., & Towse, J. N. (2007).

Variation in work-

ing memory. New York: Oxford University Press.

Conway, A. R. A., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., & Engle, R. W.

(2005).

Working memory span tasks: A methodological review and user's guide.

Psychonomic Bulletin & Review, 12, 769

–786.

Cowan, N. (1988).

Evolving conceptions of memory storage, selective attention, and their

mutual constraints within the human information processing system. Psychological
Bulletin, 104, 163

–191.

Cowan, N. (1995).

Attention and memory: An integrated framework. Oxford, England: Ox-

ford University Press.

Craik, F. I. M. (1986).

A functional account of age differences in memory. In F. Klix, & H.

Hagendorf (Eds.), Human memory and cognitive capabilities (pp. 409

–420).

Amsterdam: Elsevier.

Craik, F. I. M., & Tulving, E. (1975).

Depth of processing and the retention of words in ep-

isodic memory. Journal of Experimental Psychology: General, 104, 268

–294.

Crawford, C. C. (1925).

Some experimental studies on the results of college note-taking.

Journal of Educational Research, 12, 379

–386.

Dahlin, E., Neely, A. S., Larsson, A., Bäckman, L., & Nyberg, L. (2008).

Transfer of learning

after updating training mediated by the striatum. Science, 320, 1510

–1512.

Daneman, M., & Carpenter, P. A. (1980).

Individual differences in working memory and

reading. Journal of Verbal Learning and Verbal Behavior, 19, 450

–466.

Daneman, M., & Merikle, P. M. (1996).

Working memory and language comprehension: A

meta-analysis. Psychonomic Bulletin & Review, 3, 422

–433.

De Luca, C. R., & Leventer, R. J. (2008).

Developmental trajectories of executive functions

across the lifespan. In V. Anderson, R. Jacobs, & P. J. Anderson (Eds.), Executive func-
tions and the frontal lobes: A lifespan perspective (pp. 3

–21). New York: Taylor &

Francis.

DiVesta, F. J., & Gray, G. S. (1972).

Listening and note taking. Journal of Educational

Psychology, 63, 8

–14.

DiVesta, F. J., & Gray, G. S. (1973).

Listening and note taking: II. Immediate and delayed

recall as functions of variations in thematic continuity, note taking, and length of lis-
tening

–review intervals. Journal of Educational Psychology, 64, 278–287.

Dunkel, P., & Davy, S. (1989).

The heuristic of lecture notetaking: Perceptions of American

& international students regarding the value & practice of notetaking. English for
Speci

fic Purposes, 8, 33–50.

Efaw, J., Hampton, S., Martinez, S., & Smith, S. (2004).

Miracle or menace: Teaching and

learning with laptop computers in the classroom. EDUCAUSE Quarterly Magazine,
27(3), 10

–18.

Engle, R. W., Tuholski, S. W., Laughlin, J. E., & Conway, A. R. A. (1999).

Working memory,

short-term memory, and general

fluid intelligence: A latent-variable approach.

Journal of Experimental Psychology: General, 128, 309

–331.

Fisher, J. L., & Harris, M. B. (1973).

Effect of note taking and review on recall. Journal of

Educational Psychology, 65, 321

–325.

Friedman, N. P., Miyake, A., Corley, R. P., Young, S. E., DeFries, J. C., & Hewitt, J. K. (2006).

Not all executive functions are related to intelligence. Psychological Science, 17,
172

–179.

Fry, A. F., & Hale, S. (1996).

Processing speed, working memory, and

fluid intelligence: Ev-

idence for a developmental cascade. Psychological Science, 7, 237

–241.

Fuster, J. M. (2002).

Frontal lobe and cognitive development. Journal of Neurocytology,

31(3

–5), 373–385.

Goldman-Rakic, P. S. (1996).

Regional and cellular fractionation of working memory.

Proceedings of the National Academy of Sciences, 93, 13473

–13480.

Hadwin, A. F., Kirby, J. R., & Woodhouse, R. A. (1999).

Individual differences in notetaking,

summarization, and learning from lectures. The Alberta Journal of Educational
Research, 45, 1

–17.

Hale, S., Rose, N. S., Myerson, J., Strube, M. J., Sommers, M., Tye-Murray, N., et al. (2011).

The structure of working memory abilities across the adult life span. Psychology and
Aging, 26, 92

–110.

Horne, J., Ferrier, J., Singleton, C., & Read, C. (2011).

Computerised assessment of hand-

writing and typing speed. Educational and Child Psychology, 28(2), 52.

Howe, M. J. (1970).

Notetaking strategy, review and long-term retention of verbal infor-

mation. Journal of Educational Research, 63, 285.

Jaeggi, S. M., Buschkuehl, M., Jonides, J., & Perrig, W. J. (2008).

Improving

fluid intelligence

with training on working memory. Proceedings of the National Academy of Sciences of
the United States of America, 105, 6829

–6833.

Kail, R., & Salthouse, T. A. (1994).

Processing speed as a mental capacity. Acta Psychologica,

86, 199

–225.

Kane, M. J., Conway, A. R., Miura, T. K., & Col

flesh, G. J. (2007).

Working memory, attention

control, and the N-back task: A question of construct validity. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 33, 615

–622.

Kane, E., & Engle, R. W. (2002).

The role of prefrontal cortex in working-memory capacity,

executive attention, and general

fluid intelligence: An individual-differences perspec-

tive. Psychonomic Bulletin & Review, 9, 637

–671.

Kiewra, K. A. (1985).

Investigating notetaking and review: A depth of processing alterna-

tive. Educational Psychologist, 20, 23

–32.

Kiewra, K. A., & Benton, S. L. (1988).

The relationship between information processing

ability and notetaking. Contemporary Educational Psychology, 13, 33

–44.

Kiewra, K. A., Benton, S. L., Kim, S., Risch, N., & Christensen, M. (1995).

Effects of note-

taking format and study technique on recall and relational performance.
Contemporary Educational Psychology, 20, 172

–187.

Kiewra, K. A., Benton, S. L., & Lewis, L. B. (1987).

Qualitative aspects of notetaking and

their relationship with information-processing ability and academic achievement.
Journal of Instructional Psychology, 14, 110

–117.

Kiewra, K. A., DuBois, N. F., Christensen, M., Kim, S., & Lindberg, N. (1989).

A more equita-

ble account of the note-taking functions in learning from lecture and from text.
Instructional Science, 18, 217

–232.

Kiewra, K. A., DuBois, N. F., Christian, D., McShane, A., Meyerhoffer, M., & Roskelley, D.

(1991).

Note-taking functions and techniques. Journal of Educational Psychology, 83,

240

–245.

Kobayashi, K. (2005).

What limits the encoding effect of note-taking? A meta-analytic ex-

amination. Contemporary Educational Psychology, 30, 242

–262.

Kobayashi, K. (2006).

Combined effects of note-taking/-reviewing on learning and the en-

hancement through interventions: A meta-analytic review. Educational Psychology,
26, 459

–477.

Lilienthal, L., Tamez, E., Shelton, J. T., Myerson, J., & Hale, S. (2013).

Dual n-back training in-

creases the capacity of the focus of attention. Psychonomic Bulletin & Review, 20, 135

–141.

Mackness, J., Mak, Sui, Fai, J., & Williams, R. (2010). The ideals and reality of participating

in a MOOC. Networked Learning Conference (pp. 266

–274) (Retrieved from

http://

www.lancs.ac.uk/fss/organisations/netlc/past/nlc2010/abstracts/Mackness.html

).

McIntyre, S. (1992).

Lecture notetaking, information processing, and academic achieve-

ment. Journal of College Reading and Learning, 25, 7

–17.

Miyake, A., & Shah, P. (Eds.). (1999).

Models of working memory: Mechanisms of active

maintenance and executive control. New York: Cambridge University Press.

Nye, P. A., Crooks, T. J., Powley, M., & Tripp, G. (1984).

Student note-taking related to uni-

versity examination performance. Higher Education, 13, 85

–97.

Oberauer, K., Süß, H. M., Wilhelm, O., & Wittman, W. W. (2003).

The multiple faces of

working memory: Storage, processing, supervision, and coordination. Intelligence,
31, 167

–193.

Olive, T., & Piolat, A. (2002).

Suppressing visual feedback in written composition: Effects

on processing demands and coordination of the writing processes. International
Journal of Psychology, 37, 209

–218.

Owen, A. M., McMillan, K. M., Laird, A. R., & Bullmore, E. (2005).

N

‐back working memory

paradigm: A meta

‐analysis of normative functional neuroimaging studies. Human

Brain Mapping, 25, 46

–59.

Palmatier, R. A., & Bennett, J. M. (1974).

Notetaking habits of college students. Journal of

Reading, 18, 215

–218.

Pappano, L. (2012).

The year of the MOOC. The New York Times, 2(12), 2012.

Peters, D. L. (1972).

Effects of note taking and rate of presentation on short-term objective

test performance. Journal of Educational Psychology, 63, 276

–280.

Peterson, L. R., & Peterson, M. J. (1959).

Short-term retention of individual verbal items.

Journal of Experimental Psychology, 58, 193

–198.

Peverly, S. T., Garner, J. K., & Vekaria, P. C. (2014).

Both handwriting speed and selective

attention are important to lecture note-taking. Reading and Writing, 27, 1

–30.

Peverly, S. T., Ramaswamy, V., Brown, C., Sumowsky, J., Alidoost, M., & Garner, J. (2007).

What predicts skill in lecture note taking? Journal of Educational Psychology, 99,
167

–180.

Peverly, S. T., Vekaria, P. C., Reddington, L. A., Sumowski, J. F., Johnson, K. R., & Ramsay, C. M.

(2013).

The relationship of handwriting speed, working memory, language compre-

hension and outlines to lecture note-taking and test-taking among college students.
Applied Cognitive Psychology, 27, 115

–126.

Piolat, A., Olive, T., & Kellogg, R. T. (2005).

Cognitive effort during note taking. Applied

Cognitive Psychology, 19, 291

–312.

Rawson, K. A., & Kintsch, W. (2005).

Rereading effects depend on time of test. Journal of

Education Psychology, 97, 70

–80.

Redick, T. S., & Lindsey, D. R. (2013).

Complex span and n-back measures of working

memory: A meta-analysis. Psychonomic Bulletin & Review, 20, 1102

–1113.

Rickards, J. P., & Friedman, F. (1978).

The encoding versus the external storage hypothesis

in note taking. Contemporary Educational Psychology, 3, 136

–143.

Robertson, I. H., Ward, T., Ridgeway, V., & Nimmo-Smith, I. (1994).

The Test of Everyday At-

tention: TEA. Bury St. Edmunds: Thames Valley Test Company.

Rogers, J., & Case-Smith, J. (2002).

Relationship between handwriting and keyboarding

performance of sixth-grade students. American Journal of Occupational Therapy, 56,
34

–39.

Ruhl, K. L., & Suritsky, S. (1995).

The pause procedure and/or an outline: Effect on imme-

diate free recall and lecture notes taken by college students with learning disabilities.
Learning Disability Quarterly, 18, 2

–11.

Salthouse, T. A. (1996).

The processing-speed theory of adult age differences in cognition.

Psychological Review, 103, 403

–428.

Shipstead, Z., Redick, T. S., & Engle, R. W. (2012).

Is working memory training effective?

Psychological Bulletin, 138, 628

–654.

Shute, V. J. (1991).

Who is likely to acquire programming skills? Journal of Educational

Computing Research, 7, 1

–24.

Smith, S. D., & Caruso, J. B. (2010). The ECAR Study of Undergraduate Students and Infor-

mation Technology, 2010. Research study, Vol. 6, . Boulder, CO: EDUCAUSE Center for
Applied Research (available from

http://www.educause.edu/ecar

).

St Clair-Thompson, H. L., & Gathercole, S. E. (2006).

Executive functions and achievements

in school: Shifting, updating, inhibition, and working memory. The Quarterly Journal
of Experimental Psychology, 59, 745

–759.

Tamez, E., Myerson, J., & Hale, S. (2012).

Contributions of associative learning to age and

individual differences in

fluid intelligence. Intelligence, 40, 518–529.

Tulving, E., & Thomson, D. M. (1973).

Encoding speci

ficity and retrieval processes in epi-

sodic memory. Psychological Review, 80, 352

–373.

Turner, M. L., & Engle, R. W. (1989).

Is working memory capacity task dependent? Journal

of Memory and Language, 28, 127

–154.

21

D.C. Bui, J. Myerson / Learning and Individual Differences 33 (2014) 12

–22

background image

Unsworth, N., & Engle, R. W. (2007a).

The nature of individual differences in working

memory capacity: Active maintenance in primary memory and controlled search
from secondary memory. Psychological Review, 114, 104

–132.

Unsworth, N., & Engle, R. W. (2007b).

On the division of short-term and working memo-

ry: An examination of simple and complex span and their relation to higher order
abilities. Psychological Bulletin, 133, 1038

–1066.

Unsworth, N., & Spillers, G. J. (2010).

Working memory capacity: Attention control, sec-

ondary memory, or both? A direct test of the dual-component model. Journal of
Memory and Language, 62, 392

–406.

van Dijk, T. A., & Kintsch, W. (1983).

Strategies of discourse comprehension. New York: Ac-

ademic Press.

Van Meter, P., Yokoi, L., & Pressley, M. (1994).

College students' theory of note-taking de-

rived from their perceptions of note-taking. Journal of Educational Psychology, 86,
323

–338.

Wechsler, D. (2008).

Wechsler Adult Intelligence Scale-Fourth Edition. San Antonio, TX:

Pearson Assessment.

Williams, R. L., & Eggert, A. C. (2002).

Notetaking in college classes: Student patterns and

instructional strategies. The Journal of General Education, 51, 173

–199.

Wing

field, A., Peelle, J. E., & Grossman, M. (2003).

Speech rate and syntactic complexity as

multiplicative factors in speech comprehension by young and older adults. Aging,
Neuropsychology, and Cognition, 10, 310

–322.

Wing

field, A., Poon, L. W., Lomabardi, L., & Lowe, D. (1985).

Speed of processing in normal

aging: Effects of speech rate, linguistic structure, and processing time. The Journal of
Gerontology, 40, 570

–585.

Wirt, J., Choy, S., Greald, D., Provasnik, S., Rooney, P., & Watanabe, S. (2001).

The condition

of education. Washington, DC: Government Printing Of

fice (NCES Publication No.

2001072).

Woodcock, R. W., & Johnson, M. B. (1989).

Woodcock-Johnson Psychoeducational Battery

Revised. Itasca, IL: Riverside.

22

D.C. Bui, J. Myerson / Learning and Individual Differences 33 (2014) 12

–22


Document Outline


Wyszukiwarka

Podobne podstrony:
Ehrman; The Role Of New Testament Manuscripts In Early Christian Studies Lecture 1 Of 2
Ehrman; The Role of New Testament Manuscripts in Early Christian Studies Lecture 2 of 2
Kałuska, Angelika The role of non verbal communication in second language learner and native speake
The role of child sexual abuse in the etiology of suicide and non suicidal self injury
Fourth Lecture Universal Corporatism The Role of Intellectuals in the Modern World
Newell, Shanks On the Role of Recognition in Decision Making
Morimoto, Iida, Sakagami The role of refections from behind the listener in spatial reflection
86 1225 1236 Machinability of Martensitic Steels in Milling and the Role of Hardness
Illiad, The Role of Greek Gods in the Novel
The Role of the Teacher in Methods (1)
THE ROLE OF CATHARSISI IN RENAISSANCE PLAYS - Wstęp do literaturoznastwa, FILOLOGIA ANGIELSKA
The Role of Women in the Church
The Role of the Teacher in Teaching Methods
The Role of The Japanese Emperor in the Meiji Restoration
Newell, Shanks On the Role of Recognition in Decision Making
Morimoto, Iida, Sakagami The role of refections from behind the listener in spatial reflection
Aggarwal And Conroy Price Discovery In Initial Public Offerings And The Role Of The Lead Underwriter

więcej podobnych podstron