Children’s perceptions of multimodal texts
After a project which used the complex picturebook The Red Tree with a class of nine- and ten-year-old bilingual children to develop inference and deduction, the teacher, Leanne, and Jane, an adviser, asked the children to define a multimodal text:
Tasnia: It can be all types of things. You can read in different ways, like you can read pictures, scan the pictures in your head, you can read from Braille like blind people do and you can read from different languages and that’s what a multimodal text means to me.
Tania: Something that all people can look at – the pictures can be written in all different languages like in The Red Tree book. I think the colours show how the person is really feeling in the story and it helps you understand the feelings…
Tasnia: …and it creates emotions.
Muhammad: When you look at something and you look at the similarities and differences between the words and the pictures..
Rashida: I think it’s looking at both ... books with more words than pictures and more pictures than words and saying … which one do you prefer more? The Red Tree’s got more pictures than words because with pictures you can really feel the emotion. With words you can, but with pictures you can actually see the people’s faces… I never thought you could really read pictures. I thought they were just there to make it look pretty.. Jane came along and explained to us and we can read books like this … She taught us that you have to put the words and the pictures together. Really go into it … go into the picture world and really analyse it... you’ve got to really look at that person’s eyes and think and think.
When asked how they would explain to a younger reader how to read a complex picturebook, their explanations of the process showed sophisticated understanding of how these texts work:
Tasnia: If they had a picture book I would ask them: ‘What do you think is going on in the picture?’ and then say if there’s a person on a swing it might be One day a girl went to the park with her family and they had a picnic in the park and then if the next page was dark and it had a witch they could say The witch came and destroyed everything…
Tania: Look at the pictures carefully. If they can’t read they have to look at it really carefully and look at the colours and what the person is doing – the facial expression and the body language. You have to look at the picture and the colours and the person’s face really carefully to ask yourself a few questions: How is the person feeling? What sort of state is he or she in? or maybe a few more questions. But I think that they should look at the picture really carefully... the colours and the face represent something.
Rashida: You connect with the picture and put yourself in that person’s shoes and ask yourself: ‘How would you feel if you were that girl?’ Connect with the picture and look at the emotion under the picture… If they made up their own words to the pictures, to think ‘This is how the person feels’ and if they were that person: ‘How would you say it?’ – then look at the words themselves and think ‘Does the book say it in a different way?’…
(Extract from Bednall, Jane and Cranston, Leanne (2008) The most wonderful adventure… going beyond the literal. English 4–11, Spring.
Reference
Tan, Shaun (2012) The Red Tree. Sydney: Lothian Children’s Books. ISBN 9780734411372.
Case study: Starting from sound
Isaac Manley has been working with multimodal texts for some time with his class of seven and eight year olds in a north east England industrial town. He describes how he used sound as a way in to looking closely at film:
My class has recently read and enjoyed annotating Anthony Browne’s The Tunnel, spending time poring over the images of the forest, and noting the picture hints of the different interests of the girl and boy in the images, discussing what the images added to the narrative and the role played by the words. Earlier in the year, using film as a basis for planning narrative, we watched The Wrong Trousers, a short Wallace and Gromit film, several times and noted the use of filmic devices like close up and camera angle as well as directorial decisions about increasing tension through length of shot and cutting to a different scene.
Much of our multimodal work so far has focused on images and words so for this two week unit on traditional tales, I planned to start from sound. I wanted to use a wordless film to introduce first person narrative and I wanted my class to make their own sound recordings telling a traditional tale from different points of view with sound effects. I started with the video of The Steadfast Tin Soldier from Fantasia 2000. I muted the sound, and played the sequence starting with the rain falling over the town until the tin soldier gives the ballerina a rose, watched by the malevolent jack-in-the-box. I asked the class what was going on in the video so far and they identified that it was a dark winter’s night, that the jack-in-the-box was a nasty character and that the soldier had fallen in love with the ballerina. I showed the section again, asking them to think about what they thought the atmosphere was like as the clip went on. They thought that it might be a bit soft when the rain was falling and scary when the jack-in-the-box appeared and ‘sweet’ and ‘dreamy’ when the soldier gave the ballerina the rose. The class normally works in discussion trios, so I asked each group to think about the kind of music that might accompany this section of the film and choose from a range of musical instruments to make a suitable sound track. Two groups volunteered to share their music, using tambourines for the rain, sticks and scrapers plus a drum beat for the jack-in-the-box and bells and xylophone for the soldier and the ballerina. After they had played their music sound tracks to the video, we discussed how suitable the tone and tempo had been for each section. Finally, I played the unmuted film, which uses Shostakovich’s second piano concerto as its soundtrack, with its steady beat building to a marching crescendo as the tin soldiers hop out of the box then fades to a more reflective tone as the soldier hands the ballerina the rose. The children talked about how suitable they thought the music was for each section and although it was quite different from their own sound tracks, they thought it fitted the images.
In the second session we focused on the section after the soldier and ballerina have danced together to the point where he falls out of the window and, in their trios, the class noted how the film maker used sound to heighten the atmosphere. We began to collect words to describe the atmosphere of the fight between the jack-in-the-box and the soldier. Finding language to describe atmosphere is quite difficult because it’s easy to fall back on ‘scary’ or ‘creepy, so I gave the children the class set of the Primary Thesaurus to do some ‘word research’ and they came up with ‘threatening’, ‘terrifying’, ‘intimidating’ and ‘fearsome’ amongst others and ‘sinister’ for the jack-in-the-box . We collected the words on the wall and added to the lists as the week’s work went on. I replayed the section again, explaining that film makers cut from one face to another to show character and after deciding in their trios which of the three main characters each of them would be, they noted the facial expressions – even wooden toys have expressions! – and I asked them particularly to notice the eyes (Disney is noted for using eyes in his animations to show emotion). To end the session the trios role played the thoughts of the three main protagonists: the soldier, the ballerina and the jack-in-the-box.
The third session followed the soldier’s journey through the sewer, noting the use of camera angle and colour as well as the music, to heighten the sense of fear. I gave the class five minutes to speed-write the soldier’s thoughts as he is taken along the dangerous route, thinks he sees the ballerina’s shadow and is surrounded by the rats. After sharing in their trios, I asked them to think how the ballerina or the jack-in-the-box were feeling and again gave them five minutes to write. Layla imagined she was the jack-in-the-box: Heh heh, I’ve got rid of that horrid little soldier he won’t be around any more so I can have the ballerina all to myself. Lisha thought the ballerina might be thinking: What could I do? I was stuck under that glass. I’ll never see him again. That wicked jack-in-the-box I’ll get my own back on him.
In the following session we returned to looking at how the film director uses point of view and I asked them, ‘Why do you think the director used close-ups and focused on hands?’ By now the class were quite familiar with analysing film and came to the conclusion it was because the tin soldier was ‘out of the picture’ as he’d been swallowed the fish and the other two main characters were still in the playroom so there was no need to see the faces of the fishermen, the cook and the little boy because they weren’t involved in the emotions of the story. To tie up this part of the work, I asked them to help me reprise the events of the story, and made a flow chart of episodes to keep as a basis for next week’s work:
Midnight: the toys come to life. The jack-in-the-box wants to be the friend of the ballerina. The soldiers come out of their box.
The soldier with one leg sees the ballerina and falls in love with her, gives her a rose and they dance together, watched by the jack-in-the-box.
The jack-in-the-box is jealous, imprisons the ballerina under a wine glass and throws bricks at the soldier until he falls out of the window.
The soldier falls into the gutter where the water sweeps him into the sewer, surrounded by rats. He is swallowed by a fish.
The fishermen catch the fish, sell it to the cook, and the soldier falls out of its mouth. The little boy picks him up and returns him to the box of soldiers.
The ballerina is delighted to see him, the jack-in-the-box is furious and attacks him again but this time the jack-in-the-box falls in the fire and the soldier and the ballerina can be together.
At the start of the following week, I explained to the class that there was a different ending to the story, introducing them to Naomi Lewis’ picturebook version of The Steadfast Tin Soldier which has dramatic illustrations by P.J. Lynch. I read the story to them, showing them the striking images, emphasising the points of view taken, and after the ending, we discussed what they thought of the original version. Opinion was divided. As the week went on, we revisited the flow chart of the narrative and noted where there were differences. They noticed more detail about the different toys: the jack-in-the-box is a goblin in the written story; the book shows the soldier’s precipitous fall from the window; there is less frightening detail as he goes through the sewer; and, of course, the ending where in the book both the soldier and the ballerina are burned in the fire, made a great discussion point. In their trios, the class chose which character they were going to be and which of the six episodes they were going to focus on. They wrote their scripts and worked out the sound track they were going to use and recorded them.
I asked one of my colleagues who has a class of five and six year olds if my class could share their stories with them in small groups and as they did this, they really enjoyed watching the younger children’s faces. For my part, I’d been able to see how well each of my class had understood the nuances of the story, were able to write convincing first person narratives, and had expressed their response to the story through words and music.
Books and video
Andersen, Hans Christian (retold by Naomi Lewis, illustrated by P.J. Lynch) (2005) The Steadfast Tin Soldier. Andersen Press. ISBN 9781842704431.
Browne, Anthony (2008) The Tunnel. Walker Books. ISBN 9781406313291.
Collins Dictionaries (illustrated Maria Herbert-Liew) (2018) Primary Thesaurus. Collins. ISBN 9780008222055.
Nick Park (1993) The Wrong Trousers (Film). Aardman Animations.
The Steadfast Tin Soldier in Fantasia 2000. Walt Disney Pictures. (https://www.bing.com/videos/search?q=Fantasia+2000+the+Steadfast+Tin+Soldier&&view=detail&mid=A06ADE12315A853D01D8A06ADE12315A853D01D8&&FORM=VRDGAR&ru=%2Fvideos%2Fsearch%3Fq%3DFantasia%2B2000%2Bthe%2BSteadfast%2BTin%2BSoldier%26FORM%3DVDMHRS).
Case study: Creating authors of non-fiction: writing and design
Inspired by ideas she had read on twitter from another teacher, Eleanor Falcon planned for her class of nine and ten year olds in an English midlands city to create multimodal information books. She describes the work:
As so many teachers do, I scroll through Twitter for ideas and inspiration and I came across this post by Emma Stanley, an upper primary teacher, about the power of a double page spread in a multimodal picture book:
One day, I decided to bring design into my writing lessons and I’ve never looked back. A double page spread is about more than spreading writing across two pages; it is about crafting and presenting a text that is visually appealing for the reader. Audience and purpose should be at the heart of all pieces of writing, and what better way to engage the reader than by drawing them in with a visually stimulating text?
(https://missstanleyyr6.wordpress.com/2019/04/17/the-power-of-a-double-page-spread/)
Not usually one to rush into school on a Monday morning, I wanted to get started right away. My class had been learning about climate change and were especially inspired by Greta Thunberg and the book Greta and the Giants: inspired by Greta Thunberg's stand to save the world by Zoe Tucker.
I provided the pupils with an audience (the class of seven and eight year olds) and a purpose (to educate and inspire children) and we began the process of creating a double-page spread. We have long subscribed to good-quality children’s magazines like Aquila and Scoop! so I laid copies of these out on tables for the pupils to explore. Discussions were enthusiastic and centred on design: layout including font and graphics, use of colour, organisation of information.
The next step was to begin planning the children’s own multimodal information texts. As writers of non-fiction, the class knew that they needed to ignite the readers’ interest and convey enthusiasm for the subject to the audience. They also worked collaboratively to share ideas and concepts to ensure that they continuously had ‘audience’ at the forefront of their work. They sketched out initial designs; planned the sections they felt were most important; planned colour schemes and at this point, the children were desperate to get started!
Figure W17.3.1
Save the planet double page spread
Figure 1 shows Alex’s double page spread. The title SAVE THE PLANET uses green for the word ‘planet’ to underline the message of a desirable state of the environment. The double page spread is surrounded by inserts giving information: a fact box on the top left explains what a carbon footprint is; at the top right hand corner there is a collection of facts including the effects of plastic on animals and the environment, drought and the unnecessary use of electricity; at the bottom right a ‘Did You Know’ oval gives Greta Thunberg’s full name. On the left hand page, almost centrally, an image of the world with one half (‘before??’) green and blue signifies a healthy planet with the other half (‘after??’) coloured red and orange as the planet burns in the heat of global warming. This small image makes more of an impression than the number of words it has taken to describe it here. A small pie chart shows how much plastic is used in relation to other materials and a blue rain cloud image surrounded by red – another hint of the contrast between a healthy and unhealthy environment – gives advice on ‘Ways to reduce your carbon footprint’. On the facing page a large footprint shape encloses information about what will happen to the earth if carbon emissions are not reduced ending with a warning of: ‘floods, droughts, tornadoes, hurricanes and tsunamis.’ On the extreme right is a map clipped from printed material showing the effects of global warming on five most affected countries. This young writer’s choices of when to use words and when to use images, decisions on layout, colour, and different font sizes show a sophisticated understanding of how images can carry persuasive messages as much as words and how some information can best be conveyed by words.
Later we focused on Shackleton’s journey to the South Pole and used William Grill’s wonderful Shackleton’s Journey as a key text. Mia, Ben and Lily reflect the cool images of the book in their choices of colour. Mia used design layout to compare two different regions of the world with vignette symbols and colour to enhance the topic (Figure 17.2). Ben produced a biography of Shackleton (Figure 17.3) and chose to illustrate the route taken by The Endurance, using mapping lines and small maps to plot the journey. Lily chose to trace over the map of the intended route of Endurance as a way of comparing the intended with the actual route (Figure 17.3). As the pupils have become more familiar with this style of teaching, so has their confidence grown and they are keen to experiment and challenge one another with suggestions and ideas.
Figure W17.3.2
Mia’s comparison of the Arctic and the Antarctic
Figure W17.3.3
Ben’s biography of Shackleton
Figure W17.3.4
Lily’s intended route of the Endurance
Using multimodal design has now become automatic in my class, and I see more and more the pupils’ ability to make choices of mode for their own purposes as authors.
(A fuller version of this case study can be found in English 4–11 Summer 2021 pp.2–4.)
Children’s books
Tucker, Zoe (2019) Greta and the Giants Frances Lincoln Children’s Books. ISBN 9780711253759.
Grill, William (2014) Shackleton’s Journey 1. Flying Eye Books. ISBN 9781909263109.