Evaluating the Emergence of Reverse Intraverbals
in Children with Autism
Alicia C. Allan & Jason C. Vladescu &
April N. Kisamore & Sharon A. Reeve &
Tina M. Sidener
Published online: 21 November 2014
# Association for Behavior Analysis International 2014
Abstract Verbal behavior plays a fundamental role in the development of complex
social and communication skills. Many children diagnosed with autism spectrum
disorder exhibit profound deficiencies in intraverbal repertoires and the development
of social relationships. Recent studies that investigated the effects of intraverbal training
on the emergence of reverse intraverbals produced mixed results (e.g., Perez-Gonzalez
et al., Journal of Applied Behavior Analysis 40:697–701, 2007)). In the current study, a
multiple-probe design across four participants with autism was used to evaluate the
effects of intraverbal training on the emergence of reverse intraverbals. Intraverbal
training consisted of multiple exemplars taught concurrently, bidirectional stimulusresponse teaching formats, general case analysis, reinforcement, and a constant prompt
delay (CPD) procedure. Participants were trained on intraverbal targets and probes
were conducted to assess emergence of untaught reverse intraverbals. Three participants demonstrated the emergence of reverse intraverbals as a result of the intraverbal
training procedures. Social validity and maintenance of target responses and emergent
reverse intraverbals were assessed.
Keywords Autism . Emergence . Intraverbal training . Verbal behavior
Functional intraverbal skills play a pivotal role in social interactions and provide the
foundation for more advanced communication skills such as describing or recalling
events, solving problems, requesting assistance, asking questions, categorizing, and
telling stories (Ingvarsson and Hollobaugh 2011; Ingvarsson and Le 2011; Kisamore
et al. 2011; Miguel et al. 2005; Sundberg and Sundberg 2011). Appropriate verbal
Analysis Verbal Behav (2015) 31:59–75
DOI 10.1007/s40616-014-0025-8
This article is based on a thesis submitted by the first author, under the supervision of the second author, at
Caldwell University in partial fulfillment for the requirements of a Master’s in Applied Behavior Analysis.
A. C. Allan : J. C. Vladescu (*) : A. N. Kisamore : S. A. Reeve : T. M. Sidener
Department of Applied Behavior Analysis, Caldwell University, 120 Bloomfield Avenue, Caldwell, NJ
07006, USA
e-mail: [email protected]
interactions typically lead to other opportunities for social reinforcement in various
social or academic settings (e.g., participating in classroom discussion, joining clubs).
Tact-to-intraverbal (Goldsmith et al. 2007), echoic-to-intraverbal (Bloh 2008;
Ingvarsson and Le 2011), receptive-to-intraverbal (Bloh 2008), and textual-tointraverbal (Finkel and Williams 2001; Vedora et al. 2009) procedures for transferring
stimulus control have proven effective for establishing intraverbals. Results of comparison studies indicate that echoic-to-intraverbal is more efficient than tact-tointraverbal transfer for some learners (Ingvarsson and Le 2011), tact-to-intraverbal is
more efficient than echoic-to-intraverbal transfer for other learners (Ingvarsson and
Hollobaugh 2011), and that textual-to-intraverbal is more efficient than echoic-tointraverbal for some learners (Finkel and Williams 2001; Vedora et al. 2009). The
efficiency of each procedure is likely a product of participant learning history (Coon
and Miguel 2012).
Given the vast number of intraverbals in typical social exchanges, identifying
teaching procedures that result in emergent verbal responses may save instruction time,
a commodity that is often at a premium for individuals with autism spectrum disorders
(ASDs). A response is said to emerge when it has not been directly taught and
reinforced (Perez-Gonzalez et al. 2008; Sidman and Tailby 1982). For example, a child
is taught the echoic response “moo” when presented with the antecedent verbal
stimulus “What does a cow say?” Then, without further training, the child emits the
verbal response “cow” when presented with the antecedent verbal stimulus, “What
animal says moo?”
Perez-Gonzalez et al. (2007) investigated the emergence of a symmetrical
intraverbal relation (i.e., the B-A relation) following training of an A-B relation with
two children diagnosed with pervasive developmental disorder (PDD). They referred to
the symmetrical intraverbal relation as the “reversible intraverbal relation.”
Experimenters developed several sets of targets consisting of pairs of unrelated
intraverbals (e.g., “Name the opposite of cold” [hot] and “Name the opposite of long”
[short]) referred to as “original intraverbals” and their reverse relations (e.g., “Name the
opposite of hot” [cold] and “Name the opposite of short” [long]). For each set of
targets, the experimenters followed a multistep process. Baseline was conducted in step
1, intraverbal training was initiated in step 2, and probe trials were conducted in step 3.
Training of reverse intraverbals was conducted in step 4 using identical procedures to
step 2. Results showed that neither participant produced the target reverse intraverbals
until they had a history of those relations being directly taught.
One potential limitation of Perez-Gonzalez et al. (2007) was the number of components comprising the antecedent verbal stimulus. It is unclear which component in the
antecedent verbal stimuli (e.g., “What’s the opposite of cold?” [hot]) may have
controlled responding. For example, only the stimulus “cold” could have evoked the
correct response (i.e., “hot”), whereas the stimulus “opposite” may not have exerted
appropriate stimulus control over the participants’ responding. For further discussion
regarding complex antecedent verbal stimuli, the reader should see Axe (2008),
Michael et al. (2011), and Sundberg and Sundberg (2011). Additional research is
needed to evaluate whether individuals might demonstrate emergent responses when
simple antecedent verbal stimuli are used. As such, we chose to use single words as
antecedent verbal stimuli in the current study to prevent interference by responses that
may have already been under the control of potential contextual stimuli.
60 Analysis Verbal Behav (2015) 31:59–75
Another possible explanation for the participants’ failure to demonstrate emergence
prior to a history of instruction of reverse intraverbals in Perez-Gonzalez et al. (2007) is
that the number of intraverbal targets taught in each set may have been insufficient to
produce the desired results. It is possible that training with multiple A-B relation
examples concurrently (rather than the pair of targets used in Perez-Gonzalez et al.)
would have facilitated generalization to subsequent reverse (B-A) intraverbal relations
(Rosales et al. 2011; Stokes and Baer 1977). Several researchers have employed this
strategy to teach a variety of skills to children with autism including daily living skills,
sharing, and offering assistance (e.g., Horner et al. 1987; Marzullo-Kerth et al. 2011;
Reeve et al. 2007). The findings of these studies support Stokes and Baer’s (1977)
assertion that generalization to untrained responses or stimulus conditions can be
programmed through the training of sufficient exemplars.
Multiple exemplars, however, may not be enough to promote an optimal level of
generalized responding. Some instructional protocols include the full range of stimulus
variations and response requirements in the generalization setting (Cooper et al. 2007).
For example, Sprague and Horner (1984) demonstrated that this general case strategy
promoted the general use of vending machines for six high school-aged participants.
Given these findings and the lack of research on the potential utility of general case
analysis in the verbal behavior domain, investigating whether general case analysis may
facilitate generalization of intraverbal responses seems warranted. For example, if
typical instruction involves training intraverbals using a similar antecedent verbal
stimulus (e.g., “The [animal] says [sound]” as exemplified by “The cow says”
[moo]), individuals may struggle to respond appropriately when the reverse is tested
(e.g., “[Sound] says the [animal]” as exemplified by “Moo says the” [cow]) because
they have not been exposed to that antecedent stimulus format.
Therefore, in contrast to the study of Perez-Gonzalez et al. (2007) that trained pairs
of intraverbals, the present study implemented multiple-exemplar training (Cooper
et al. 2007); that is, ten exemplars (e.g., city-state and state-city relations) were trained
concurrently to facilitate generalization to untrained responses (Stokes and Baer 1977).
In addition, a general case strategy consisting of bidirectional teaching formats (i.e.,
both A-B and B-A relations) was used to expose participants to a larger range of stimuli
and responses. It was predicted that implementation of both multiple-exemplar training
and general case analysis as described would produce the desired results Thus, the
purpose of the study was to extend previous research (Perez-Gonzalez et al. 2007) on
the emergence of reverse intraverbals in children with autism by (a) using bidirectional
stimulus-response teaching formats, (b) teaching multiple exemplars concurrently with
a general case strategy to identify a full range of exemplars, (c) using single-word
antecedent verbal stimuli, (d) assessing maintenance, and (e) assessing social validity of
emergent intraverbal responding.
Method
Participants
To be eligible for participation in this study, participants had to demonstrate the
following prerequisite skills: (a) attending, defined as sitting upright with hands on
Analysis Verbal Behav (2015) 31:59–75 61
the table or lap, feet on the floor, and the body positioned toward the experimenter; (b)
eye contact, defined as looking at the experimenter in the eye when an antecedent
verbal stimulus was presented; (c) generalized vocal imitation, defined as providing an
echoic response following a novel antecedent verbal stimulus; (d) tacting, defined as
identifying an object or action when presented with a picture depicting that object or
action (50–100 tacts; Sundberg 2008); and (e) receptive discrimination, defined as
identifying an object or action when presented with a three-picture array and instructed
to point to or touch the photo depicting that object or action (50–100 objects or actions;
Sundberg 2008). Additionally, participants had to demonstrate deficient intraverbal
repertoires and low levels of problem behavior. Evaluations to determine if potential
participants met inclusion criteria were conducted prior to the study by the staff at the
participants’ school. The first four participants who met the prerequisite skills and
inclusion criteria were enrolled in the study. None of the selected participants had a
previous history of explicitly taught reverse intraverbals in their verbal behavior
programs.
Four males who attended a private school for individuals with ASDs participated.
All participants had received an independent diagnosis of autism by a physician at least
5 years prior to enrollment in the study. Additionally, parents of the participants
completed The Gilliam Autism Rating Scale—Second Edition (GARS-2; Gilliam
1995) to document the occurrence of behaviors characteristic of ASD. Parental ratings
for all participants indicated a high probability of autism.
Ray was 18 years old and had received intervention based on the principles of
applied behavior analysis (ABA) since age 2 years. Ray obtained a standard score of 71
on the Expressive Vocabulary Test—Second Edition (EVT-2; Williams 2007) and
performed all or nearly all skills through level 3 on the mand, tact, listener, and
intraverbal subtests from the Verbal Behavior Milestones Assessment and Placement
Program (VB-MAPP; Sundberg 2008). Additionally, he performed all skills on the
echoic subtest from the VB-MAPP.
Eric was 9 years old and had received intervention based on the principles of ABA
since age 6 years. He had a standard score of 78 on the EVT-2. On the VB-MAPP, Eric
demonstrated the majority of skills through level 3 for the tact, listener, and intraverbal
subtests. He demonstrated all skills on the echoic subtest and was performing at level 2
for the mand subtest.
Alan was 12 years old and had received ABA-based intervention since age
4 years. On the EVT-2, Alan obtained a standard score of 75. He performed all
or nearly all skills through level 3 on the mand, tact, listener, and intraverbal
subtests on the VB-MAPP. He also demonstrated all skills on the echoic subtest
of the VB-MAPP.
Tony was 9 years old and had received ABA-based intervention since age 7 years.
Tony scored 51 on the EVT-2 and demonstrated some skills from levels 1 to 2 on the
mand, tact, listener, echoic, and intraverbal subtests of the VB-MAPP.
Setting and Materials
All baseline, treatment, and maintenance sessions were conducted in the participants’
school. The designated areas contained desks, phones, computers, filing cabinets, a
copy machine, a long table, and chairs.
62 Analysis Verbal Behav (2015) 31:59–75
Laminated two-dimensional pictures depicting the outline of a map of the USA and
European countries, as well as football team logos, animals, and common objects were
used during tact and listener pretests and during baseline and training sessions in
mastered task trials. Tokens, token boards, and edibles were also used during training.
A laptop computer was used to provide echoic prompts during intraverbal training and
a small video camera was used to record some sessions. Instruction was delivered in a
one-to-one format with the experimenter seated across from the participant during all
pretest and experimental sessions.
Response Measurement, Interobserver Agreement, and Procedural Integrity
The experimenter served as the primary observer and scored the occurrence of the
following dependent variables: correct unprompted responses, incorrect unprompted
responses, correct prompted responses, and incorrect prompted responses. Correct
unprompted responses were defined as the participant providing the target response
within 5 s of the experimenter presenting the antecedent verbal stimulus. An incorrect
unprompted or prompted response was scored if the participant provided an error of
commission (i.e., incorrect answer) or omission (i.e., did not respond) within 5 s of the
presentation of the antecedent verbal stimulus or the echoic prompt, respectively. A
correct prompted response was scored if the participant provided a target response
within 5 s of the presentation of the echoic prompt.
A secondary observer collected data for interobserver agreement (IOA)
purposes during a minimum of 32 % of sessions. IOA data were calculated
on a trial-by-trial basis. The number of agreements were divided by the number
of agreements plus disagreements and multiplied by 100. Mean reliability was
100 % for all participants across sessions during the first evaluation and
replication for Eric and Ray.
An independent observer collected procedural integrity data during a minimum of
32 % of sessions. The observer scored each of the following components as correct or
incorrect: (a) presentation of antecedent stimuli, (b) delivery of vocal prompt within the
predetermined interval, (c) implementation of the error correction procedure, and (d)
appropriate reinforcer delivery. All procedural steps for a trial had to be implemented
accurately for the entire trial to be scored correct. Procedural integrity was 100 % for all
four participants across sessions in the first evaluation and 100 % for both participants
across sessions in the replication.
A second observer also collected procedural integrity data during a minimum of
16 % of sessions for IOA purposes. IOA on procedural integrity was 100 % for all
participants across sessions in the first evaluation and 100 % for both participants
across sessions in the replication.
Preference Assessment and Reinforcement Systems
Preference and reinforcer assessments were conducted with the participants at the time
of their initial enrollment in their current school placement, prior to their enrollment in
the current study. A motivational system (i.e., token economy) was established for each
individual (see below for a description of each participant’s token economy) based on
these assessments. Training with token economies was initiated for each participant
Analysis Verbal Behav (2015) 31:59–75 63
upon placement in the school, and token economies had been used with each participant since that training. Thus, all participants had long histories with token economies.
Ray’s token economy was a monetary value system that involved dollar and cent
amounts written on a small white board upon completion of various activities. A list of
preferred snacks and activities was taped to the back of the white board and Ray could
exchange the money he earned at various times throughout the day for these preferred
items. Eric and Alan’s token economies consisted of them earning dimes contingent on
correct responses during their academic and social skills programs. Once 20 dimes
were earned, Eric and Alan could exchange them for a wide range of preferred activities
or edibles. Tony’s token economy involved earning pennies for correct responses
during academic programs. Once he earned ten pennies, Tony could exchange them
for a preferred item. The experimenter used each participant’s token economy during
the current evaluation. Ray exchanged the money he accrued on his white board for a
preferred activity or snack at the conclusion of each session. Eric, Alan, and Tony
exchanged their respective tokens at the conclusion of each session for preferred
activities that were restricted to this study. Additionally, edibles identified through
teacher surveys were used during training for Eric, Alan, and Tony to reinforce correct
unprompted responses. These edibles were restricted for use in the current study.
Pre-experimental Procedure
A pool of potential targets were selected for each participant by the clinical director at
the participants’ school, and behavior analysts assigned to oversee the participants’
individualized educational programs in terms of social relevance and academic skill.
For example, NFL teams were selected for Ray because he enjoyed watching football
with his family. States and cities were selected for Eric and Alan based on their
advanced academic skills, and simple word associations were identified for Tony based
on his tact and listener repertoire. The experimenter conducted intraverbal, tact, and
listener pretests prior to assigning stimuli and beginning the evaluation. Once pretests
were completed, intraverbals were assigned to teaching conditions (see below for
additional information).
Intraverbal Pretest Intraverbal pretests were conducted to determine if target
intraverbals were already in the participants’ repertoires. During the pretest, target
intraverbals were probed using multiple formats, twice with a fill-in-the blank format
(e.g., “Cowboys” [Dallas] and “Dallas” [Cowboys]) and twice with a question format
(e.g., “Which football team is from Dallas?” [Cowboys] and “The Cowboys football
team is from what city?” [Dallas]).
The experimenter presented the antecedent verbal stimulus and allowed 5 s for a
response. No feedback was provided for correct or incorrect responses. Praise and
tokens were provided for appropriate sitting and attending about every three trials.
Following the intraverbal pretests, we conducted tact and listener pretests to ensure
that participants could correctly identify at least one relevant component of the potential
intraverbal target. For potential targets involving states and countries, we only required
the participant be able to identify the state or country as a speaker and listener, but did
not require a correct response for the relevant city. For potential targets involving
football teams, we required the participant be able to correct identify the team name
64 Analysis Verbal Behav (2015) 31:59–75
(e.g., “Cowboys”) when shown the team’s logo. For potential targets involving associations, we required the participant be able to identify at least one object (e.g., socks)
from the pair (e.g., socks [feet]). Correct identification of one relevant component was
defined as responding correctly in three out of four trials. If the participant could not
identify a relevant component as a speaker and listener, the component was trained or
the potential target was discarded. We required this as an attempt to make acquisition of
intraverbal targets more functional for the participants.
Tact Pretest The experimenter held up a card and presented the antecedent verbal
stimulus (e.g., “What is it?”). Participants had 5 s to respond. No feedback was
provided for correct or incorrect responses. Praise and tokens were provided for
appropriate sitting and attending about every three trials.
Listener Pretest The experimenter placed a three-card array (e.g., laminated maps of
states) in front of the participant and presented the antecedent verbal stimulus (e.g.,
“touch Florida”). Participants had 5 s to respond. The position of the target card was
randomly rotated across trials. No feedback was provided for correct or incorrect
responses. Praise and tokens were provided for appropriate sitting and attending about
every three trials.
Assignment of Targets Ten intraverbal targets were identified for the first evaluation in
the following categories: (a) football teams and cities for Ray, (b) states and cities for
Alan and Eric, and (c) objects and places for Tony. For the replication conducted with
Eric and Ray, ten European countries and cities were selected as intraverbal targets.
Using a general case strategy construct, the intraverbal targets in each group were
arranged to expose the participants to the bidirectional stimulus-response formats and
facilitate emergence of reverse intraverbals. Using the cities and states example, the five
targets in group A were taught by presenting the state as the antecedent verbal stimulus
(e.g., “Florida”) and the five targets in group B were taught by presenting the city as the
antecedent verbal stimulus (e.g., “Seattle”). During probe trials, the reverse relations
were tested for each group; that is, the five targets in group A were probed by
presenting the city as the antecedent verbal stimulus (e.g., “Seattle”) and the five targets
in group B were probed by presenting the state as the antecedent verbal stimulus (e.g.,
“Florida”). The targets were counterbalanced across participants for Eric and Alan in
the first evaluation and for Eric and Ray during the replication. The intraverbal targets
were assigned to two groups (group A and group B) of five targets based on the number
of syllables in each; that is, the antecedent verbal stimuli and the target responses in
group A contained approximately the same number of syllables as the antecedent
verbal stimuli and the target responses in group B. This arrangement was intended to
equate response effort across groups. Table 1 shows the bidirectional teaching and
probe formats for intraverbal and reverse intraverbal targets for each participant in the
first evaluation and the replication.
Identification of Mastered Targets Prior to the current evaluation, a list of potential
mastered tacts and intraverbals was compiled for each participant from previously
mastered tact and intraverbal school programs. These mastered tasks were used to
ensure participants contacted a high level of reinforcement during sessions across
Analysis Verbal Behav (2015) 31:59–75 65
Table 1 Bidirectional teaching and probe formats for intraverbal and reverse intraverbal targets
Intraverbals teach and probe assignments for Ray Intraverbals teach and probe assignments for Eric
Group A—teach (cities) Group A—probe (teams) Group A—teach (states) Group A—probe (cities)
Baltimore (Ravens) Ravens (Baltimore) Florida (Miami) Miami (Florida)
Carolina (Panthers) Panthers (Carolina) Idaho (Boise) Boise (Idaho)
Denver (Broncos) Broncos (Denver) Virginia (Williamsburg) Williamsburg (Virginia)
Oakland (Raiders) Raiders (Oakland) Tennessee
(Chattanooga)
Chattanooga
(Tennessee)
Pittsburg (Steelers) Steelers (Pittsburg) Nevada (Reno) Reno (Nevada)
Group B—teach (teams) Group B—probe (cities) Group B—teach (cities) Group B—probe (states)
Cowboys (Dallas) Dallas (Cowboys) Chicago (Illinois) Illinois (Chicago)
Lions (Detroit) Detroit (Lions) Seattle (Washington) Washington (Seattle)
Seahawks (Seattle) Seattle (Seahawks) Helena (Montana) Montana (Helena)
Texans (Houston) Houston (Texans) Malibu (California) California (Malibu)
Vikings (Minnesota) Minnesota (Vikings) Houston (Texas) Texas (Houston)
Intraverbals teach and probe assignments for Alan Intraverbals teach and probe assignments for Tony
Group A—teach (cities) Group A—probe (states) Group A (teach) Group A (probe reverse)
Miami (Florida) Florida (Miami) Beach (sand) Sand (beach)
Boise (Idaho) Idaho (Boise) Gloves (hands) Hands (gloves)
Williamsburg (Virginia) Virginia (Williamsburg) Hat (head) Head (hat)
Chattanooga
(Tennessee)
Tennessee (Chattanooga) Bird (feather) Feather (bird)
Reno (Nevada) Nevada (Reno) Umbrella (rain) Rain (umbrella)
Group B—teach (states) Group B—probe (cities) Group B (teach) Group B (probe reverse)
Illinois (Chicago) Chicago (Illinois) Snow (sled) Sled (snow)
Washington (Seattle) Seattle (Washington) Socks (feet) Feet (socks)
Montana (Helena) Helena (Montana) Juice (cup) Cup (juice)
California (Malibu) Malibu (California) Car (garage) Garage (car)
Texas (Houston) Houston (Texas) Library (books) Books (library)
Intraverbals teach and probe assignments for Eric’s
replication
Intraverbals teach and probe assignments for Ray’s
replication
Group A—teach
(countries)
Group A—probe (cities) Group A—teach (cities) Group A—probe
(countries)
Spain (Madrid) Madrid (Spain) Madrid (Spain) Spain (Madrid)
Germany (Berlin) Berlin (Germany) Berlin (Germany) Germany (Berlin)
England (York) York (England) Lucerne (Switzerland) Switzerland (Lucerne)
Finland (Helsinki) Helsinki (Finland) Helsinki (Finland) Finland (Helsinki)
Italy (Venice) Venice (Italy) Venice (Italy) Italy (Venice)
Group B—teach (cities) Group B—probe (countries) Group B—teach (countries) Group B—probe (cities)
Paris (France) France (Paris) Greece (Athens) Athens (Greece)
Copenhagen (Denmark) Denmark (Copenhagen) Denmark (Copenhagen) Copenhagen (Denmark)
Athens (Greece) Greece (Athens) Poland (Warsaw) Warsaw (Poland)
Tralee (Ireland) Ireland (Tralee) Norway (Oslo) Oslo (Norway)
Oslo (Norway) Norway (Oslo) Ireland (Tralee) Tralee (Ireland)
66 Analysis Verbal Behav (2015) 31:59–75
conditions (see below for further information). For each participant, mastery on these
programs was defined as 90 to 100 % correct unprompted responding for two consecutive sessions. To ensure maintenance, potential mastered tacts and intraverbal targets
were presented to each participant, and the tacts and intraverbals to which the participants responded correctly in three out of four trials were selected for use during
mastered task trials. Mastered intraverbals were used for Ray, Eric, and Alan.
Mastered tacts were used for Tony as his intraverbal repertoire was limited.
Experimental Design and General Procedure
A multiple-probe design across participants was used to evaluate the effects of
intraverbal training on the acquisition of intraverbals and emergence of reverse
intraverbals. The evaluation was conducted twice with Ray and Eric for replication
purposes. The experimenter conducted one session per day, 5 days a week. Each
session consisted of either 40 trials (i.e., 30 intraverbal trials and ten mastered tact or
intraverbal trials) or 50 trials (i.e., 30 intraverbal trials, ten mastered tact or intraverbal
trials, and ten reverse intraverbal probe trials).1 These 40- or 50-trial sessions were
alternated such that reverse intraverbal probe trials were conducted every other session.
The experimenter conducted intraverbal training until the participants’ correct unprompted responding reached 100 % for two consecutive sessions. Following mastery
of intraverbals, reverse intraverbal probe trials were conducted during every session (as
opposed to every other session) to provide the participants additional opportunities to
demonstrate mastery of the reverse intraverbals. Training of reverse intraverbals was
implemented if a flat trend in responding to reverse intraverbals was observed for five
consecutive sessions following mastery of intraverbals. Direct training of reverse
intraverbals was only necessary for Tony.
Baseline During intraverbal trials and during reverse intraverbal probe trials, the
experimenter presented the antecedent verbal stimulus (e.g., “Florida”) and the participant had 5 s to respond. No feedback was provided for correct or incorrect responses
during these trials. During trials of mastered tasks, the experimenter presented an
antecedent verbal stimulus (e.g., “Birds build nests in the” [trees]) or held a photograph
of an animal or a sport and presented the antecedent verbal stimulus (e.g., “What is
this?”). Praise and tokens were provided for correct responses on a fixed ratio (FR 1)
only during mastered task trials. Trials of mastered tasks were presented in each session
in accordance to the 3-to-1 ratio specified in the “Experimental Design and General
Procedure” section above (i.e., ten mastered tact or intraverbal trials were interspersed
with 30 intraverbal trials). Baseline continued until participants demonstrated 0 %
correct unprompted responding for a minimum of four consecutive sessions.
Intraverbal Training During intraverbal trials, the first session included trials with a 0-s
prompt delay. During these trials, the experimenter presented the antecedent verbal
1 Intraverbal trials refer to trials in which the participants were required to respond and feedback was provided
based on the participants’ response. Reverse intraverbal probe trials refer to trials in which the participant was
not required to respond to and no consequences were provided if a response was provided.
Analysis Verbal Behav (2015) 31:59–75 67
stimulus (e.g., “Florida”) and immediately provided an echoic prompt (e.g., “Miami”).
Praise, tokens, and edibles (except for Ray) were provided for correct prompted
responses. Beginning with the second session, the prompt delay was increased to 5 s
and praise/tokens/edibles were delivered for correct unprompted responses only.
The experimenter implemented an error correction (EC) procedure following incorrect unprompted responses during intraverbal trials. The EC procedure consisted of the
experimenter providing an echoic prompt and allowing the participant the opportunity
to respond. Then, the experimenter re-presented the trial until the participant engaged in
a correct unprompted response. Praise and tokens/edibles were provided during EC
trials following correct unprompted responses. Error correction data are not included in
the session data.
During training, the echoic prompts were prerecorded on and presented using a
laptop computer (Stevenson et al. 2000). A prerecorded male voice was used to deliver
prompts in an attempt to help the participants discriminate between the antecedent
verbal stimulus and the echoic prompt. To provide the prompt, the experimenter cued
and played the recorded prompt from the laptop.
During intraverbal trials, the schedule of reinforcement for correct unprompted
responses was thinned from FR 1 to VR 3 when the participants reached 90 % correct
unprompted responding; that is, an edible or a token was delivered following each
correct unprompted response until the participants’ correct unprompted responding
reached 90 %. At that time, an edible or token was delivered every three correct
unprompted responses on average. This strategy was implemented to prepare participants for extinction conditions during probe trials of reverse intraverbals when edibles,
tokens, or praise was not provided contingent on correct unprompted responses.
During reverse intraverbal probe trials, the experimenter presented the antecedent
verbal stimulus (e.g., “Miami”) and the participant had 5 s to respond. No feedback or
differential consequences were provided.
During trials of mastered tasks, the experimenter presented an antecedent verbal
stimulus (e.g., “Birds build nests in the [trees]”) or held a photograph of an animal or
sport and presented the antecedent verbal stimulus (e.g., “What is this?”). Praise,
tokens, and edibles were provided for correct responding an FR 1 schedule during
mastered task trials until participants demonstrated 50 % correct unprompted
responding during intraverbal trials. This was done to ensure a high density of
reinforcement during training. Once a participant demonstrated at least 50 % correct
unprompted responding during intraverbal trials, praise and tokens/edibles were no
longer provided during mastered task trials. At that point, praise and tokens/edibles
were provided only for correct unprompted responses on an FR 1 schedule which was
thinned as described above.
Reverse Intraverbal Training Reverse intraverbal training was composed of sessions
consisting of three reverse intraverbal probe trials interspersed with seven mastered task
trials for a total of ten trials per session. The format of the reverse intraverbal training
trials and the mastered task trials was the same as described in the “Intraverbal
Training” section. Mastery criterion was 100 % correct unprompted responding for
two consecutive sessions. Following mastery of a trained reverse intraverbal, five probe
sessions were conducted to provide the participant with additional opportunities to
demonstrate emergence of the remaining untrained reverse intraverbals. Reverse
68 Analysis Verbal Behav (2015) 31:59–75
intraverbal training was only necessary for Tony and these data are not included in the
session data.
Maintenance Maintenance of intraverbals was evaluated after mastery-level
responding was demonstrated during intraverbal teaching trials and reverse intraverbal
probe trials. As in baseline, sessions consisted of 50 trials (30 intraverbal trials, ten
mastered tact or intraverbal trials, and ten reverse intraverbal probe trials). No feedback
was provided for correct or incorrect responses during any of these trials. Praise and
tokens were provided only for appropriate sitting and attending. Maintenance sessions
were conducted every 3 days for Ray, Eric, and Alan. Tony was available for three
maintenance sessions, during which he demonstrated mastery-level responding with
both intraverbals and reverse intraverbals.
Social Validity Nine items derived from the Treatment Acceptability Rating Form
(TARF; Martens et al. 1985) were compiled to assess social validity. A group of
teachers and parents was provided with written protocols used in the current evaluation
and asked to rate the items using five-point Likert-type scale (1 = strongly disagree, 5 =
strongly agree). The items included effectiveness in producing untrained responses,
normative comparison to age-matched peers, efficiency, and overall benefits to the
participants.
Results
Figures 1 and 2 depict the percentage of correct unprompted responses for intraverbals
and reverse intraverbals for all participants during baseline, training, and maintenance
conditions in the first and replication evaluations. During baseline, all participants’
correct responses were at zero for intraverbals and reverse intraverbals.
Ray’s responding in intraverbal and reverse intraverbal trials during his first evaluation is displayed in Fig. 1 (first panel). After training was implemented, Ray demonstrated mastery-level responding for intraverbal targets in 13 training sessions (390
training trials without error correction trials) and required 11 probe sessions to achieve
mastery-level responding for the reverse intraverbals without direct training. Ray
continued to demonstrate mastery-level responding for intraverbals and reverse
intraverbals during maintenance. Figure 2 (second panel) depicts the replication results
for Ray. He demonstrated mastery-level responding for intraverbals in seven training
sessions (210 training trials without error correction trials). Similar to his first evaluation, Ray acquired the reverse intraverbals without direct training, requiring seven
probe sessions to demonstrate mastery. During maintenance, he demonstrated high
levels of correct unprompted responses.
Figure 1 (second panel) depicts Eric’s correct unprompted responding in intraverbal
and reverse intraverbal trials during his first evaluation. Eric mastered intraverbals in
ten training sessions (300 training trials without error correction trials) and reverse
intraverbals in six probe sessions following implementation of training. Acquisition of
reverse intraverbals occurred without direct training. Eric continued to demonstrate
mastery-level responding for intraverbals and reverse intraverbals during maintenance.
Analysis Verbal Behav (2015) 31:59–75 69
Fig. 1 Percentage of correct unprompted intraverbals and reverse intraverbals for all four participants. Each
filled data point is computed from three presentations of ten intraverbals; each open data point is computed
from one presentation of ten reverse intraverbals. The small arrows in the bottom panel indicate training of
reverse intraverbals for Tony
Fig. 2 Percentage of correct unprompted intraverbals and reverse intraverbals for Eric and Ray during
intrasubject replication. Each filled data point is computed from three presentations of ten intraverbals; each
open data point is computed from one presentation of ten reverse intraverbals
70 Analysis Verbal Behav (2015) 31:59–75
Figure 2 (first panel) shows the replication results for Eric. Correct unprompted
responding increased very quickly once training was initiated. Eric required six training
sessions (180 training trials without error correction trials) to master intraverbals and
four probe sessions to master reverse intraverbals. As in his first evaluation, Eric
acquired the reverse intraverbals without direct training. During maintenance, he
demonstrated mastery-level responding of correct unprompted intraverbals and reverse
intraverbals.
Results for Alan are depicted in Fig. 1 (third panel). Alan demonstrated
mastery level of intraverbal responding in 13 training sessions (390 training
trials without error correction trials). Following mastery of intraverbals, he
demonstrated low levels of correct responding in reverse intraverbal trials.
Correct unprompted responding in reverse intraverbal trials began to increase
during his 20th session and mastery was reached in session 30. Alan demonstrated high levels of correct unprompted responding for intraverbals and
reverse intraverbals during maintenance.
Figure 1 (fourth panel) displays Tony’s responding in intraverbal and reverse
intraverbal trials. Following an increasing trend, correct unprompted responding in
intraverbal trials leveled off at 90 % for six consecutive sessions (i.e., one of the ten
intraverbals was not acquired). At this time, a 0-s prompt delay was reinstated for the
intraverbal target for which Tony consistently emitted an incorrect unprompted response. After two sessions with trials conducted at 0-s prompt delay, the prompt delay
was increased to 5 s. Tony demonstrated mastery-level responding for all intraverbals in
training session 30 (900 training trials without error correction trials). Following
mastery of intraverbals, he demonstrated minimal correct responses during reverse
intraverbal probes; he responded correctly to one reverse intraverbal for five consecutive sessions. Reverse intraverbal training was implemented in session 36. Following
mastery-level responding of a trained reverse intraverbal, five probe sessions were
conducted to provide Tony with additional opportunities to demonstrate emergence of
the remaining reverse intraverbals. It was necessary to teach Tony all nine reverse
intraverbals to which he responded incorrectly during probe trials. Three reverse
intraverbals were taught individually and the remaining six reverse intraverbals were
taught in pairs. Tony demonstrated mastery-level responding for reverse intraverbals in
session 63.
To provide further details regarding training duration for each participant, we
calculated the total number of training trials, including error correction trials, required
for each participant to reach the mastery criterion. In the first evaluation, Ray required
512 training trials, Eric 465 training trials, Alan 541 training trials, and Tony 1270
training trials to reach the mastery criterion. During the replication, Eric and Ray
required 237 and 340 training trials, respectively.
Results for the social validity measure show scores averaging between 4.0
and 4.5 across the nine items surveyed. Raters assessed the intervention as
being effective in producing untrained verbal responses, appropriate for children
with moderate to high functioning verbal skills; efficient; and simple to implement. Scores also indicate that the intraverbal responses selected for training are
comparable to intraverbal responses that are present in the verbal repertoires of
age-matched peers. Overall, raters found the intervention to be beneficial for the
participants.
Analysis Verbal Behav (2015) 31:59–75 71
Discussion
The present study investigated the effects of concurrent multiple-exemplar training,
bidirectional stimulus-response teaching formats, reinforcement, and single-word antecedent verbal stimuli on the acquisition of intraverbals and the emergence of reverse
intraverbals. Overall, results suggest that the protocols used in the present evaluation
are effective in producing the desired results for some children. Ray, Eric, and Alan
demonstrated mastery-level responding for intraverbal targets and emergence of reverse
intraverbals following implementation of intraverbal training. Ray and Eric began
responding correctly in reverse intraverbal trials simultaneously with correct unprompted responding of intraverbals. In contrast, Alan demonstrated emergence of reverse
intraverbals following mastery of taught intraverbals. Examples of errors during training and probe trials for Ray, Alan, and Eric included “Africa” when responding to the
antecedent verbal stimulus “Lions” (i.e., Detroit Lions), “Pacific Ocean” when
responding to the antecedent verbal stimulus “Oakland,” and unintelligible vocalizations (Ray); inappropriate echoic responses (Eric); and “I don’t know,” incorrect state/
city names, and errors of omission (Alan).
During the replication evaluation, Eric and Ray demonstrated faster acquisition of
intraverbal targets and emergence of reverse intraverbals than in the first evaluation,
providing additional evidence as to the effectiveness of the protocols used in the present
study. Although we attempted to equate intraverbal targets within evaluations, we did
not equate them across evaluations. Thus, Ray and Eric may have acquired the
intraverbals more rapidly in the replication evaluation because the targets were less
difficult than those selected for the first evaluation. On the other hand, the faster
acquisition of intraverbals and emergence of reverse intraverbals could be attributed
to the participants’ recent instructional history. If so, it could be concluded that repeated
exposure to intraverbal training may promote mastery-level responding of reverse
intraverbals and more rapid acquisition of subsequent intraverbals and reverse
intraverbals for some children (e.g., Perez-Gonzalez et al. 2008). Future studies should
consider controlling for the difficulty of targets across evaluations to investigate the
potential effects of recent instructional history and the difficulty of targets on rate of
acquisition.
The protocol used in this study was not effective in producing the desired
results for Tony. Although he demonstrated mastery-level responding for intraverbal
targets, albeit at a slower rate than the other participants, he failed to demonstrate
emergence of reverse intraverbals even after training of reverse intraverbals was
initiated; that is, following training of each reverse intraverbal, Tony continued to
respond incorrectly to the remaining untaught reverse intraverbals. Tony’s errors
during intraverbal training consisted of errors of omission or inappropriate echoic
responses (i.e., imitating the antecedent verbal stimulus). It should be noted that
Tony engaged in frequent vocal and motor stereotypy throughout the day, and
these competing behaviors may have interfered with instruction. In addition, Tony’s
verbal repertoire was the most limited among the four participants as demonstrated
by his EVT-2 and VB-MAPP scores. Outcomes for Tony are consistent with the
results of some participants from previous research (Perez-Gonzalez et al. 2008;
Petursdottir et al. 2008a, b) in that he failed to demonstrate the emergence of the
symmetrical intraverbal relation (B-A).
72 Analysis Verbal Behav (2015) 31:59–75
The use of single words as antecedent verbal stimuli could be considered a potential
limitation because they were provided out of context. Thus, it could be argued that the
target responses may have been in the participants’ repertoires and incorrect responses
may have resulted from the lack of additional stimuli. This is unlikely, however,
because all four participants responded incorrectly during intraverbal pretest trials
which the antecedent verbal stimulus was presented within context (i.e., question and
fill-in-the blank formats).
Another potential limitation should be noted in the social validity assessment.
Reviewers were asked to rate the intervention based on a description of protocols. It
is possible that the raters could not obtain sufficient information from the written
descriptions and they might have provided different scores had they been given the
opportunity to view video-taped sessions. Future studies could address this by showing
raters video clips demonstrating the intervention and having them complete ratings
based on these clips.
Overall, the findings from the present evaluation do not support previous research
(Petursdottir et al. 2008a, b) in which intraverbal training did not result in the emergence of reverse intraverbals. However, results from the three participants in the present
study who demonstrated emergence of reverse intraverbals following intraverbal training extend the findings by Perez-Gonzalez et al. (2007). That study showed that
teaching additional sets of reverse intraverbals did facilitate the emergence of novel
reverse intraverbals in subsequent trials for both participants. Despite incorrect
responding during intraverbal pretests across participants, the relatively advanced
intraverbal repertoires demonstrated by Ray, Eric, and Alan bring into question to what
extent the novel aspects of the intervention in this study contributed to the positive
results. It is possible that these participants would have demonstrated emergence of
reverse intraverbals using teaching procedures that have failed to produce positive
results in the past with participants whose intraverbal repertoires were possibly less
advanced. This possibility should be addressed in future research.
In addition, results from the three participants from the present evaluation who
demonstrated emergence of reverse intraverbals following intraverbal training support
previous research (Marzullo-Kerth et al. 2011; Reeve et al. 2007; Sprague and Horner
1984; Stokes and Baer 1977) in demonstrating that training multiple exemplars concurrently in conjunction with a bidirectional stimulus-response teaching format (i.e.,
general case strategy) appears to be an effective training package that may promote
emergence of reverse intraverbals for some children. Future studies could continue this
line of research to determine the optimal number of exemplars to train concurrently.
Perhaps the most intriguing results stem from Alan’s unexpected feedback in the
early stages of the current investigation. His comments may help shed light on covert
behavior (Skinner 1957) that may have exerted control over responding. Prior to the
sessions during which emergence of reverse intraverbals began to occur, Alan said on a
few occasions, “Oh, that one goes with…” when an antecedent verbal stimulus was
presented, then emitted the correct response (e.g., “Illinois”). Based on this observation,
it could be argued that Alan was beginning to identify the symmetrical relation between
the reverse intraverbals and the intraverbals as a result of the bidirectional teaching
formats used in the present evaluation.
Considering the variety and complexity of stimuli controlling verbal behavior,
learning appropriate verbal exchanges can be a daunting task for individuals with
Analysis Verbal Behav (2015) 31:59–75 73
autism and other developmental disabilities (e.g., Finkel and Williams 2001; Sundberg
and Sundberg 2011; Taylor and Harris 1995; Weiner 2005). Thus, interventions
designed to increase intraverbal repertoires and promote generalization to untrained
verbal relations seem like a worthwhile effort, as it would be prohibitive to teach
children with autism all the potential intraverbal responses required for advanced verbal
exchanges.
Although adequate experimental control was demonstrated in the current evaluation,
training was introduced at the same time for the third and fourth participants due to time
constraints. Had we introduced training for these participants individually, a more clear
demonstration of control might have been achieved. Future research could sequentially
introduce training across participants to increase experimental control. Future research
should also investigate whether additional language assessments may reveal any
minimal participant characteristics that may be necessary for the emergence of
intraverbal responses. Additionally, additional research is needed to evaluate how the
procedures used in the current study could be used to teach individuals with autism to
demonstrate emergent responses to more complex antecedent verbal stimuli (e.g.,
intraverbal conditional discriminations).
Acknowledgments We thank Paul Argott for his feedback regarding various aspects of this study.
References
Axe, J. B. (2008). Conditional discrimination in the intraverbal relation: a review and recommendations for
future research. The Analysis of Verbal Behavior, 24, 159–174.
Bloh, C. (2008). Assessing transfer of stimulus control procedures across learners with autism. The Analysis of
Verbal Behavior, 24, 87–101.
Coon, J. T., & Miguel, C. F. (2012). The role of increased exposure to transfer-of-stimulus-control procedures
on the acquisition of intraverbal behavior. Journal of Applied Behavior Analysis, 45, 657–666. doi:10.
1901/jaba.2012.45-657.
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis (2nd ed.). Upper Saddle
River: Pearson Prentice Hall.
Finkel, A. S., & Williams, R. L. (2001). A comparison of textual and echoic prompts on the acquisition of
intraverbal behavior in a six-year old boy with autism. The Analysis of Verbal Behavior, 18, 61–70.
Gilliam, J. E. (1995). Gilliam autism rating scale (2nd ed.). Austin: Pro-Ed.
Goldsmith, T. R., LeBlanc, L. A., & Sautter, R. A. (2007). Teaching intraverbal behavior to children with
autism. Research in Autism Spectrum Disorders, 1, 1–13. doi:10.1016/j.rasd.2006.07.001.
Horner, R. H., Williams, J. A., & Steveley, J. D. (1987). Acquisition of generalized telephone use by students
with moderate and severe mental retardation. Research in Developmental Disabilities, 8, 229–247. doi:10.
1016/0891-4222(87)90006-0.
Ingvarsson, E. T., & Hollobaugh, T. (2011). Acquisition of intraverbal behavior: teaching children with autism
to mand for answers to questions. Journal of Applied Behavior Analysis, 43, 1–17. doi:10.1901/jaba.
2010.43-1.
Ingvarsson, E. T., & Le, D. D. (2011). Further evaluation of prompting tactics for establishing intraverbal
responding in children with autism. The Analysis of Verbal Behavior, 27, 75–93.
Kisamore, A. N., Carr, J. E., & LeBlanc, L. A. (2011). Training preschool children to use visual imagining as a
problem-solving strategy for complex categorization skills. Journal of Applied Behavior Analysis, 44,
255–278. doi:10.1901/jaba.2011.44-255.
Martens, B. K., Witt, J. C., Elliot, S. N., & Darveaux, D. X. (1985). Teacher judgments concerning the
acceptability of school-based interventions. Professional Psychology: Research and Practice, 16, 191–
198. doi:10.1037/0735-7028.16.2.191.
74 Analysis Verbal Behav (2015) 31:59–75
Marzullo-Kerth, D., Reeve, S. A., & Reeve, K. F. (2011). Using multiple-exemplar training to teach a
generalized repertoire of sharing to children with autism. Journal of Applied Behavior Analysis, 44,
279–294. doi:10.1901/jaba.2011.44-279.
Michael, J., Palmer, D. C., & Sundberg, M. L. (2011). The multiple control of verbal behavior. The Analysis of
Verbal Behavior, 27, 3–22.
Miguel, C. F., Petursdottir, A. I., & Carr, J. E. (2005). The effects of multiple-tact and receptive-discrimination
training on the acquisition of intraverbal behavior. The Analysis of Verbal Behavior, 21, 27–41.
Perez-Gonzalez, L. A., Garcia-Asenjo, L., Williams, G., & Carnerero, J. J. (2007). Emergence of intraverbal
antonyms in children with pervasive development disorder. Journal of Applied Behavior Analysis, 40,
697–701. doi:10.1901/jaba.2007.697701.
Perez-Gonzalez, L. A., Herszlikowicz, K., & Williams, G. (2008). Stimulus relations analysis and the
emergence of novel intraverbals. The Psychological Record, 58, 95–129.
Petursdottir, A. I., Carr, J. E., Lechago, S. A., & Almason, S. M. (2008a). An evaluation of intraverbal training
and listener training for teaching categorization skills. Journal of Applied Behavior Analysis, 41, 53–68.
doi:10.1901/jaba.2008.41-53.
Petursdottir, A. I., Olafsdottir, A. R., & Aradottir, B. (2008b). The effects of tact and listener training on the
emergence of bidirectional intraverbal relations. Journal of Applied Behavior Analysis, 41, 411–415. doi:
10.1901/jaba.2008.41-411.
Reeve, S. A., Reeve, K. F., Townsend, D. B., & Poulson, C. L. (2007). Establishing a generalized repertoire of
helping behavior in children with autism. Journal of Applied Behavior Analysis, 40, 123–136. doi:10.
1901/jaba.2007.11-05.
Rosales, R., Rehfeldt, R. A., & Lovett, S. (2011). Effects of multiple exemplar training on the emergence of
derived relations in preschool children learning a second language. The Analysis of Verbal Behavior, 27,
61–74.
Sidman, M., & Tailby, W. (1982). Conditional discrimination vs. matching to sample: an expansion of the
testing paradigm. Journal of Experimental Analysis of Behavior, 37, 5–2. doi:10.1901/jeab.1982.37-5.
Skinner, B. F. (1957). Verbal behavior. Cambridge: Prentice-Hall.
Sprague, J. R., & Horner, R. H. (1984). The effects of single instance, multiple instance, and general case
training on generalized vending machine use by moderately and severely handicapped students. Journal
of Applied Behavior Analysis, 17, 273–278. doi:10.1901/jaba.1984.17-273.
Stevenson, C. L., Krantz, P. J., & McClannahan, L. E. (2000). Social interaction skills for children with
autism: a script-fading procedure for nonreaders. Behavioral Interventions, 15, 1–20. doi:10.1002/(SICI)
1099-078X(200001/03)15.
Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization. Journal of Applied Behavior
Analysis, 10, 349–367. doi:10.1901/jaba.1977.10-349.
Sundberg, M. L. (2008). VB-MAPP. Verbal behavior milestones assessment and placement program.
Concord: AVB.
Sundberg, M. L., & Sundberg, C. A. (2011). Intraverbal behavior and verbal conditional discrimination in
typically developing children and children with autism. The Analysis of Verbal Behavior, 27, 23–43.
Taylor, B. A., & Harris, S. L. (1995). Teaching children with autism to seek information: acquisition of novel
information and generalization of responding. Journal of Applied Behavior Analysis, 28, 3–14. doi:10.
1901/jaba.1995.28-3.
Vedora, J., Meunier, L., & Mackay, H. (2009). Teaching intraverbal behavior to children with autism: a
comparison of textual and echoic prompts. The Analysis of Verbal Behavior, 25, 79–86.
Weiner, J. S. (2005). Peer-mediated conversational repair in students with moderate and severe disabilities.
Research & Practice for Persons with Severe Disabilities, 30, 26–37. doi:10.2511/rpsd.30.1.26.
Williams, K. T. (2007). Expressive vocabulary test (2nd ed.). Minneapolis: Pearson.
Analysis Verbal Behav (2015) 31:59–75 75
