This
work is licensed under a Creative Commons
Attribution-ShareAlike 4.0 International License
A paper inspired by the the 2007 TCC online conference: Blending Reality and Multimedia in Ubiquitous Learning
Category: Technology use to enhance communication and collaboration
In a piece presented at TCC in 2003, my work built on the recognition that there is value in creating and designing "exercises that enable students to conceive of themselves as creators, custodians and commentators." In that presentation, "Of Drugs, Messages and Time", I invited co-exploration of the analogies between drug administration and instructional design. The remarks and suggestions recorded here build on the "helping hands" metaphor that concluded the 2003 paper.
The presentation briefly touches upon Marcel Jousse's research on memory work and "verbo-moteurs" and sets it alongside an expanded view of Brenda Laurel's field work on computers and theatre. This exploration of theory is then followed by description of three techniques and a discussion of how a practice of managing information loads can evolve out of attention to micro-events.
...Before moving on to practical applications and advice, it is worthwhile taking a brief theoretical excursus. Such a little trek will provide an opportunity to tease out some of the principles that can be generalized and hence applied to a manifold of circumstances.
The route is quite simple. First a consideration of Miller's concept of "chunking". Then, from Laurel's work on theatre and computers, the great desirability of "staging affordances" . And finally via Jousse's research into the memory work of "verbo-motoeurs" readers are invited to contemplate the complementary activities of chunking and staging as they are aligned with simple bodily processes. This last stretch may prove to require a little hyper-jump. But we begin by tiny steps.
Let us recall George Miller's famous seven plus or minus two (Ref 4). To begin, we take the minimalist approach and begin with five. Five is the number of digits usually found on a human hand: one thumb and four fingers. The hand or its equivalent in terms of body imagery provides us with a measuring device. The "two" we reserve for operating upon the five.
To draw out some of the implications of Miller's fascinating number, let us here juxtapose two key passages from his article. The first aligns the notion of "information" with that of "variance"; the second, re-labels "chunking" as "recoding". The key to reading the passage about variance and information is understanding the relationship between "making an observation" and "getting information":
When we have a large variance, we are very ignorant about what is going to happen. If we are very ignorant, then when we make the observation it gives us a lot of information. On the other hand, if the variance is very small, we know in advance how our observation must come out, so we get little information from making the observation.
A close re-reading of the passage reveals that variance is observer-dependant. We have, we hold, variance. On the other hand, through observation, we get information. There is a negative feedback loop between the getting of information and the holding of variance; the more of one, the less of the other.
Now let us re-read Miller's take on chunking:
In the jargon of communication theory, this process would be called recoding. The input is given in a code that contains many chunks with few bits per chunk. The operator recodes the input into another code that contains fewer chunks with more bits per chunk. There are many ways to do this recoding, but probably the simplest is to group the input events, apply a new name to the group, and then remember the new name rather than the original input events.
The concern here is getting signal through noise. The simplicity of tree-like structure of chunking provides an elegant means to do so especially in a mono-channel setting. At first glance, chunking is a parsing plus a naming. If re-coding is read as attention splitting, then chunking as a grouping activity is also a mapping: a set of names is mapped to a set of groupings. Where do the names come from? Off-stage.
Off-stage is a zone from which noise can be introduced. Names before they become Labels are Noise. The noise of names provides a set of affordances for chunking. For example, take a set of numbers, colours or letters. Consider them before and after they become attached as labels to elements in a list. Names that have become labels provide the affordances of container-like behaviours.
Implied in Miller's article is the importance of attention splitting. I like to think of that aspect as the plus or minus two. In a multimedia setting, learners and teachers can add or subtract channels. Or in the language of cognitive science, devote attention to sensory modalities, uniquely or in mixed combinations. This attention is like a pre-fab affordance for the processing of information.
In re-reading Miller, we learnt that names are important for achieving chunking. By way of that re-reading we found that the origin of names can be located in a zone of offstage affordances. We also learnt that a name is a sort of pre-label.
Let us now reread Brenda Laurel's summary (Ref 3) and speculations about the power of storytelling for engaging game players and computer users. As we do so, let us hold in mind the analogies gleaned from rereading Miller and be guided by the following two suppositions:
With Laurel we can dwell briefly upon the ubiquity of stories and their manifold forms, and pick up her question:
Stories were made up about existing narratives or from whole cloth. Stories could be told, written, drawn, theatrically performed, or improvised. How can this finding be translated into computer-based game-play?
and pick up once again the question of a gain and place it again alongside the considerations on character:
Of course, games are not the only computer-based spaces in which narrative construction occurs! All but the most procedural activities can be seen to have a narrative arc. To re-cast the central observation of Computers as Theatre, good experience design provides affordances for narrative construction of a particular type: a story of a successful or delightful action with a beginning, middle and end, where the interactor is typically the central character.
Let us expand the application to "the most procedural activities" and minimize the character loading of the narrative construction. Copy and paste. Middle beginning and end: one sets a mark, then sets another mark, then commands a copy be produced of what is between the marks; one chooses an insertion point, then yanks a copy of what has been copied, then accepts or rejects the "pasting". There are sometimes in the play of this micro-theatre surprises: both successful and delightful are in store. There is at times a gap between what one thinks is copied and what one wishes to be pasted (there is a lot of delightful magic to (un)successful WYSIWYG transfers). In terms of stories and stages, there is flex in what is being told and the demarcation of the place from where it is being told. The important point here is to recognize a kinship between Miller's remarks about the activity of "observation" yielding more or less information depending upon initial perception of variance and the potential to read Laurel's opportunities for narrative construction as affordances in themselves.
If one considers narrative construction as the building of affordances, whether such opportunities are built or not is moot if there is a social context for the reception of the constructing or building. At a bare minimum such a social context is supplied by the learner in conversation with if not their multiple selves then their faculties as personified. Such intrapsychic conversations are the mark of the ZPD (Zone of Proximal Development) [for an elegant in-context demonstration and discussion of ZPD, See Zhang (Ref 5).]
It is through this aspect of personification, we can re-visit Laurel's insistence on character focus. For Laurel the interactor is a central character. For Jousse (Ref 1) there is always at play in processing a triple set of faculties of Kantian hue (((feeling, understanding and will))). It is worth considering the central character or interactor to be engaged in parallel processing of affects, representations and actions. Take for example where Jousse quotes Luquet to the effect that the not-to-be-too-rigoursly-demarcated faculties play a round of moving from foreground to background and back:
Il ne faut pas consideérer comme profonde, comme correspondant à une réalité véritable, la distinction dans l'âme de trois facultés. Même sans donner à ce mot le sens abusif et réaliste que semblaient lui donner les éclectiques, les groupes de phénomènes dont elles sont comme l'étiquette commune ne présentent pas de caractères nettement tranchés permettant de tracer entre eux une ligne de démarcation précise. Il n'y a pas des phénomènes affectifs qui ne seraient qu'affectifs, des phénomènes représentatifs qui ne seraient que représentatifs, etc., mais tout phénomène affectif par exemple est en même temps représentatifs et actif. D'une manière générale, tout état de la conscience est à la fois affectif, intellectuel et actif; il n'y a qu'une différence de dosage dans les proportions relatives de ces éléments : chaque état de conscience comprend, outre le phénomène affectif ou représentatif ou actif qui est au premier plan, des éléments psychiques appartenant aux deux autres classes, plus reculés ou moins visibles, mais qui n'en sont pas moins toujours présents (LUQUET [Idées générales de psychologie, Paris, 1906 ] : 82-83)
Much of Jousse's book is a tissue of quotations. He demonstrates what he describes. Similar to the micro-theatre case of the copy and paste procedure, parsing is at play. The selection from Lucquet builds in adequate redundancy and if read closely one would notice an invariance in the ordering of the principle terms: (1) (2) (3). I draw this to your attention because Jousse's sensitivity to patterns is as he notes on the first page a psychology of recitation:
Nos recherches ont porté sour la Mémoire verbo-motrice rythmique, sur ce que notre Professeur au Collège de France, M. le Dr Pierre Janet, appellerait la Psychologie de la Récitation.
Jousse's great contribution to social linguistics is to underscore the importance of gesture in language. It follows that with a helping hand from the presence of gesture, we can view recitation as not only as the oral delivery of what is remembered but also the re-siting.
Verbal and motor segments are aligned through parsing. They are re-sited. It appears to be a process similar to what was suggested above in the discussion of Miller and multiple channels. The motor sequence is taken to be the name of the verbal sequence and vice versa the verbal sequence names the motor sequence. Similar to Laurel's staging of affordances, the verbo-motor processes, like the relation between the telling and the told, possess a dual orientation. One can be a source of noise for the other.
But what purpose does the introduction of noise serve? If noise is akin to variance and surprise is akin to information then the answer becomes clear: one holds on to noise (silence as per John Cage) as a source of variance in order to get information (delightful and successful surprises).
We are now in a position to hypothesize. Learners can switch attention between plus or minus two sensory modalities (in addition to the one that is the focus of foreground attention). Attention switching can be accelerated to the point of self-induced minor information overload which produces the appropriate level of cognitive dissonance that is appropriately resolved into consonance with a re-parsing of the sequence associated with at least one of the channels or sensory modalities. [We are here of course extending the concept of cognitive dissonance/consonance as proposed by L. Festinger in A Theory of Cognitive Dissonance (1962)]
In processing terms it is feasible to describe stories and sequences as stacks. Stacks are order-sensitive. They are usually processed by a rule such as Last In First Out (LIFO).
A stack can be split (middle can shift). Stack order can be reversed (end becomes beginning). These operations thus produce from the perspective of processing a different stack.
The sensory modalities present information in a sequential fashion. Or the human apparatus accepts the information in such a sequential fashion.
A stack of pairings between two or more sensory modalities can be split and recombined (c.f. twice behaved behaviours theorized by Richard Schechner).
Stacks can be replicated. Multi-media processing plays with replicating stacks.
Learners can be invited to sub-vocalize what is read on screen. This is an evident application of the verbo-motor dynamic between gesture and speech. It fosters retention.
Learners can be invited to activate voice synthesis while they are reading on screen. The eye moves at a different rate than the ear processes the sounds. This can lead to interesting desynchronisations. What gets retained is what is encoded with appropriate redundancy. For example a log of a MOO or MUD session reduplicates the say commands:
This is fun!
You say : This is fun!
in such a fashion that when read back by voice synthesis a rhythmic repetition comes into play.
Desynchronizations can also produce pleasant shocks when eye and ear are operating in rapid switching. Collisions productive of sense making can occur. Similar results can be obtained by layering sound: an audio track is on playback while voice synthesis is used to read off the screen.
Also while composing or authoring, an audio track playback interrupted by alert sounds, provides a rich stimulus for behavioural learning. This is especially true if the alert sounds are supplemented by visual stimuli such as flashes or dimming. In time these accidental occurrences can be self-induced and provide a means of "programming" the bodily response to an information stream and thus improve recall or jamming.
The visual field itself can be the object of split attention. Consider the on-screen cursor as a species of stylus which is itself a type of conducting baton. Consider the shape-shifting that occurs when the cursor passes in and out of an active window. Consider the click that realigns insertion point (usually a blinking vertical bar) with the cursor-in-window (often an I-beam). Consider the delightful agitation produced by tabbing through windows and that set of clicks. At the level of hardware, each of these gestures is also accompanied by aural feedback.
Rhythmic activity, vital for parsing and re-parsing, lies in the control of the user-learner. It's in the bones. It's in the drumming. Keyboard, screen and table top become a resounding board.
So what is one to do with a noise maker? How can one use for pedagogical purposes the Vibratron potential inherent in the hardware and software interaction?
A bit of attention to timing and to John Cage (on silence) provide a hint. Clock settings can be used to create from the buzz of activity a random event generator. By analogy with coin tossing: the flip is of a certain duration and at the end of the duration one reads the motif on the face of the coin (in order to take a decision). Time-limitations regulate the random event generators that are the conjunction of humans and machines.
A clock built into the computer can be set to announce the hour or the quarter hour. Likewise timers that are not built-in can be set to announce the passage of time. Time here is an ally in the control of information flow (and overload). Periodic reminders of time elapsed provide a break and help bring the attention to the mix of eye, ear and touch. Adjustments can then be made for the next period.
U is the logical symbol for Union. An upside down U is the sign for Intersection. Ubiquity is all about intersecting unions. Discuss in relation to mixes and segregation of sensory modalites. Please make use of a hand to point, count and mark a bit of beat. Tap into the generation of joyful noise. Have fun!
For more on parsing as discovery process, see Sprezzatura and Blogcraft (Ref 2).
This
work is licensed under a Creative Commons
Attribution-ShareAlike 4.0 International License