Moving across screens

Welcome to Learning Event 6 (#LE6): Moving across screens.

(Need more info about Learning Events in general? Visit the Learning Commons for a full description of this series.)

Learning Event 6 (#LE6) focuses on the challenges and opportunities as we increasingly move across screens for teaching, learning, and assessment. 

The Internet is the dominant text of our generation. We use it to read, write, communicate, and socialize, and it has changed what counts as “text.” Texts have traditionally been said to act as linguistic and non-linguistic signs requiring interpretation to communicate meaning. Moving to digital, social spaces, there is an opportunity to expand our view of “text” to include visual, digital, and other multimodal forms of expressing information and content. 

Too often educators privilege one form of text over another. Many times this takes the form of printed text in a paper book or periodical.  A broader view of text takes into account the different textual cues, as well as images, audio, graphic design elements, and structural components. 

Educators must identify instructional opportunities to accommodate these shifts in text, and help prepare students to interrogate these texts as they interact with them. This includes helping learners understand connections made in course content, and critique other forms of text as they read and synthesize across multiple modes of communication. This means that students in classes may consume content across a variety of sources, including electronic text, video, audio podcasts, infographics, and other media.


Multimodality. Multimodality is an interdisciplinary approach that frames communication and representation to be more than about language. It has been developed over the past decade to address questions about how changes in literacy, communication, and other fields have changed in relation to new media and technologies.

Multimodal perspectives on learning and meaning-making rest on the basic assumption that meaning is made through many representational and communicational resources (semiotic resources), not just the use of textual language. Meaning-making in these contexts relates the ways in which people consider, understand, or make sense of the world. From a multimodal perspective, modes, such as audio, images, video, and gestures are semiotic resources that have different affordances, or potentials for making-meaning. This means that the different modes have unique capacities to express and represent things based on their material, physical and environmental components. In plain english, this means that a YouTube video can explain things differently than an audio podcast, or handbook chapter. 

It is important to recognize that learning does not occur in a silo, but is enriched by social, political, cultural, and other contexts. Texts are written and presented from a specific standpoint and contain a particular ideological and ontological focus. For the most part, there is no way to read, write, view, or speak a text from an impartial, or neutral position. As a result, learners need to be taught that these contexts and contingencies exist within texts, and be challenged to respond to texts in a critical manner. Learners must be taught to question a text and interrogate it to understand who created the text, controls the power, determines these power structures, and is not heard in the text. This includes asking questions “about language and power, about people and lifestyle, about morality and ethics, about who is advantaged by the way things are, and who is disadvantaged” (Comber, 2001, p. 271).

Check out the materials presented below to learn and engage more!


Multimodality and Digital Technologies in the Classroom – Carey Jewitt. “It explores how the use of digital technologies in the school reconfigures and redesigns the pedagogic landscape of the classroom, with particular attention to changes to the specificity of image and writing and their relationships. More generally it demonstrates the potential of a multimodal perspective for understanding the complex relationships between representations, digital technologies and knowledge.”

Engaging students through multimodal learning environments: The journey continues – Michael Sankey, Dawn Birch, Michael Gardiner. “Given these findings, the importance of improving student progression and retention, and engendering a joy of learning, leading to life-long learning, educators should be encouraged to continue to explore the use of educational technology and multimedia for developing multiple representations of content.”

Multimodal Communication in the University: Surveying Faculty Across Disciplines  – (full paper here) Gwendolynne Reid, Robin Snead, Keon Pettiway & Brent Simoneaux. “At issue are “modes” of communication, with each mode being a different means of communicating information. For example, speech, the written word, sound, physical gestures and graphic images are all different modes of communication. Multimodal communication is communication that takes advantage of multiple modes, such as a PowerPoint presentation or a television commercial.”


Gunther Kress – What is Multimodality? (10:24)
Cope & Kalantzis on Multimodality in Meaning Making (12:23)
Cope & Kalantzis on Design & Multimodality (8:13)


Do you provide opportunities for students search and sift across multimodal texts while engaged in the meaning-making process?


Provide opportunities for students to engage in meaning-making and produce evidence of knowledge construction through a variety of digital modes.

Make it happen:

To prepare students for the ever-expanding digital and multimodal world, we need self-programmable learners who are able to move across different spaces, identities, and arguments with network fluidity. Our students do not need to know all the answers; they need to be able to curate texts and share answers while engaged in the meaning-making process.

This research by Jalal Nouri examines multimodal design, meaning-making, and knowledge construction during self-study in a higher ed classroom. Findings suggest the preferences learners have as they engage with learning content, as well as practices they engage in as they construct meaning.

To assist with learning content in classes, students indicated they utilized the following forms of pre-existing digital learning materials to support self-studies. The quotes below are taken from the publication and help give context to the use of these resources:

Using pre-existing video material – Participant Interview quote: “It is easier to understand things in video format and books are sometimes boring. When I do not understand what the book is talking about, or my teacher’s explanations, I find video lectures on the subject. I follow Youtube and I’ve subscribe to some scientific channels. In that way I get to see the subject from another perspective or explained differently or better.”

Using pre-existing digital texts – Participant Interview quote: “I like digital texts better because you have it on your computer so you don’t need to carry it with you if you go to school, not having to bring the physical thing with you. And most likely I can read it easily from everywhere at any time I want, like home, school or in the bus using my phone. It’s also easy to get and it’s also an easy way to learn something fast, especially if it’s just something that the lecturer just mentions and you just want to get some more information on it, then it’s a good way to just skim it and get a good overview of the area.”

Using pre-existing digital images – Participant Interview quote: “If I don’t understand some term or other word or important terms it is easy to Google images that can help me grasp them, and understand the terms.”

Using pre-existing audio – Participant Interview quote: “Well I actually listen to YouTube video clips on my phone. It is a video, but I listen to the audio in the video clip because I’m at the gym and cannot look at the video. And I learn better when I listen or look than reading.”

To engage in meaning-making, students indicated that they engaged in:

  • Writing digital texts – Students writing summaries of lecture materials, course literature, video materials, and student notes for learning purposes;
  • Taking pictures – Students taking pictures of lecture slides, drawings, notes, brainstorming, and textual content for learning purposes;
  • Making audio recordings – Students audio recording lectures, group discussions, and subject summaries for learning purposes;
  • Making video recordings – Students recording themselves or others for learning purposes.

We would love to hear about what you created or implemented as a result of this Learning Experience! Please send an email to if you have something to share!