Developing Metamodal Awareness Frank Serafini

Download PDF

Abstract

Bringing together diverse perspectives on children’s literature and literacy development offers new avenues for expanding what it means to be literate, the skills and processes necessary for full participation in today’s society, an acknowledgement that being literate requires more than the ability to encode and decode written language, and challenges written language as the dominant form of representation and communication. In this article, it is asserted children and young adults developing as readers would benefit from an introduction to the multiple ways humans represent and communicate ideas, identities, and ideologies, and by developing a basic awareness of how various semiotic systems work and are used across a range of literary and informational texts and contexts. Developing metamodal awareness in emerging and fluent readers across textual, semiotic, and critical dimensions is essential for expanding the strategies and approaches used to navigate and make sense of the wide range of multimodal texts available in today’s complex communicational environment.

Keywords: children’s literature, metamodal awareness, multimodality, picturebooks, semiotics

Dr Frank Serafini is Professor of Literacy Education and Children’s Literature at Arizona State University and has published extensively in the fields of literacy education, multimodality, and children’s literature. Current research projects include multimodal research designs, the instantiation of narrative in multimodal forms, and the relationship of words and images in children’s literature.

 

Introduction

Penning a distinction that became renowned, John Rowe Townsend (1969) asserted the existence of unavoidable disciplinary divisions between book people, scholars interested in children’s literature as literary phenomena worthy of study in its own right, and child people, scholars interested in how children made sense of these texts and the curricular frameworks that support children’s literacy development (p. 417). Apol (1998) further suggested children’s literature scholars are ‘caught straddling worlds, one foot planted in English departments, the other in colleges of education, negotiating a tricky balance between theory and practice, between texts and readers’ (p. 32). Given the technological, multimodal, digital, and transmedial influences on forms of representation and communication that pervade the theoretical terrain in which children’s literature scholars now find themselves, there may be more academic worlds worth straddling.

In addition to book people and child people, children’s literature scholars need to situate themselves as multimodal people, attending to the multimodal nature of texts designed for children, specifically picturebooks, illustrated novels, and comics. Further, children’s literature scholars would also benefit by situating themselves as digital people, considering the impact digital technologies have had on children’s literature, specifically picturebook apps, transmedial narratives, and the mediation of children’s literature through digital platforms and devices.

While there are certainly other worlds to be straddled and perspectives worth considering, the addition of multimodal and digital dimensions is not meant to complicate scholarly inquiry or blur the boundaries among disciplines, although siloed perspectives do represent a challenge for children’s literature scholarship. Rather, these additional worlds and perspectives are intended to address the constantly changing nature of what is meant by the term text, emerging perspectives on the concept of modes, referring to socially embedded, material and semiotic resources for making meaning, used in the creation of narrative and informational texts, and the changing world in which texts for children and young adults are designed, produced, and embedded.

Comparable to Townsend’s (1969) original distinction between children’s literature as the focus of literary analysis or as a pedagogical instrument or approach, the Children’s Literature in English Language Education (CLELE) journal straddles similar worlds and pushes the boundaries of both language and literature education from multiple perspectives. Bland (2024) has asserted, ‘[t]aken together, language and literature education research demonstrates that linguistic, aesthetic and sociocultural concerns are interdisciplinary, and are often contextualized within wider critical (sometimes socially polarizing) themes’ (p. ii). Children’s literature scholars and language educators alike need to cross traditional disciplinary boundaries for both researching literature as analogue, digital, and multimodal ensembles and for understandings its role in English language pedagogical frameworks. Such endeavours are necessary for making sense of the multimodal, transmedial, and intercultural nature of the texts children experience in our global societies and improving the quality of language education in various educational settings. Specifically, book people and child people need to move beyond language as the primary mode of representation and communication to address and analyze the myriad ways visual images, digital platforms, multimodal ensembles, and transmedial spaces influence the texts young readers encounter (Serafini, 2023a; Towndrow, Nelson, & Yusuf, 2013).

 

From Metalinguistic to Metamodal Awareness

The term metalanguage has been defined as the use of language to refer back on itself, as an overarching language or metalinguistic perspective to discuss and analyze the functions and structures of language (Berry, 2005). The concept of metalinguistic awareness has been primarily associated with the fields of linguistics and literacy education, serving as a language about language or as an alternative description that calls attention to how language works. It is a reflective form of discourse designed to understand how language works and how it is organized ideationally, compositionally, rhetorically, and interpersonally.

The New London Group (1996) recommended metalanguages to describe patterns of meaning initially required for linguistically dominant (written and oral language) texts, might also be used for understanding visual, audio, gestural, spatial, and multimodal texts. From these beginnings, it has been important to consider how metalanguages are not only associated with linguistic forms of representation and communication, but how they are used to develop insights, perspectives, and vocabularies for understanding a more diverse range of semiotic resources and multimodal phenomena.

Since the publication of Reading Images: The Grammar of Visual Design (Kress & van Leeuwen, 1996), researchers and educators have worked to expand the concept of metalinguistic awareness beyond its original focus on written and oral language to consider the semiotic resources used in visual images, digital technologies, and multimodal texts. Literacy educators have called for expanding the use of metalanguages to address the visual arts (Barton, 2019), digital literacies (Mills, 2016), visual and multimodal literacies (Lim, 2018), and music (Dinham, 2020). It has been deemed essential to recognize specific arts-based and visual literacies in addition to traditional language-based literacies in order to more intentionally, strategically, and comprehensively support children’s development of a range of literacies across digital and multimodal educational contexts (Barton, Burke, & Freebody, 2022).

The word awareness refers to a knowledge or perception of a situation or fact and suggests a well-informed interest in a particular situation or development. Awareness also involves both attention to, and interest in, a particular object, event, interaction or space. More specifically, metalinguistic awareness is a type of meta-awareness of the structures and functions of written and oral language that may provide the foundation for a reflective perspective on other modalities and forms of representation and communication and how they operate.

The term metamodal discourse was initially proposed by Fagerston, Holmsten, & Cunningham (2010) to refer to the ‘explicit naming or overtly attending to the modes of communication in use’ and ‘the co-construction of meaning via a combination of linguistic, nonlinguistic or even para-linguistic resources’ (p. 149). Although this work frames additional modalities in terms of what they are not (nonlinguistic), equates modes with specific channels of communication, and is primarily focused on a psycholinguistic approach to multimodal communication, it serves as a call for expanding perspectives on metalinguistic awareness beyond its original focus on written and oral language in research on multimodal communication and children’s literature.

Supporting students’ and teachers’ capabilities for appreciating and understanding multimodal phenomena requires a more expansive form of metamodal discourse or, specifically, a metamodal awareness that calls attention to the range of modalities and semiotic resources used in contemporary forms of representation and communication beyond language-based systems. Metamodal awareness requires participants to consider not only what a particular form of representation and communication potentially means, but how it means in particular contexts, spaces, and interactions. In other words, attention to both signifiers (modes of representation or the expression plane) and signifieds (potential meanings or the content plane) is necessary for understanding how various forms of multimodal representation and communication operate (Jappy, 2013; Lim, 2004) and for supporting readers engaged in a variety of roles, literacy practices, and meaning making experiences (Serafini, 2012).

In this article, I will explore the concept of metamodal awareness for accessing, navigating, reading, and analyzing picturebooks and other forms of multimodal texts intended for children and young adult readers. This requires first, a rethinking of the concept of metalinguistic awareness to include the additional modalities featured in multimodal children’s literature, and second, the development of a pedagogical framework for structuring the application of metamodal awareness and its development among emerging and fluent readers.

 

A Metamodal Awareness Framework

Much of the research connecting multimodality and children’s literature has focused on the modes, graphic design features, and semiotic resources used in the design and production of picturebooks, illustrated novels, comics, digital narratives, and other multimodal ensembles (Serafini & Pantaleo, 2022). The development of metamodal awareness for calling readers’ attention to the ways that various semiotic resources and modalities operate provides a platform for developing readers as active participants in the reading process and establishes a type of awareness that goes beyond basic strategies focused on decoding linguistic elements to consider multiple systems for representing and communicating potential meanings.

In addition to developing a metamodal awareness of the basic modalities and semiotic resources used in picturebook design and construction, a metamodal awareness framework needs to include pedagogical approaches for helping children and young adult readers notice and consider how various semiotic resources work in the texts they encounter, and the ways potential meanings are constructed. Literacy educators need to consider not only the structures of multimodal texts themselves but the various roles and literacy practices readers of multimodal and transmedial texts are asked to implement across social, cultural, and historical contexts (Serafini, 2012).

The four perspectives presented in the opening of this article, namely book, child, multimodal, and digital serve as a foundation for the development of a pedagogical framework to address the challenges of developing metamodal awareness and critically proficient readers in the twenty-first century. A critical multimodal literacies pedagogy (Serafini, 2023b) posits the integration of critical theories, digital literacy, and multimodal perspectives and offers a framework for reconceptualizing literacy education and developing a metamodal awareness that fosters more active and critical readers. Foundationally, a metamodal awareness pedagogical framework focuses on three forms of awareness, including 1) textual awareness – what constitutes a text and what texts are made available for readers, 2) semiotic awareness – how texts work in the process of representing and communicating narratives and information, and 3) critical awareness – the impact of sociocultural, historical, economic, and political influences on the design and construction of multimodal texts themselves and readers’ interpretations and experiences of such texts.

 

Textual awareness

To begin, textual awareness is an understanding of what the concept of text means, an inventory of the basic types of texts that are available for readers, and the types of texts that may interest a diverse population of readers. Textual awareness is an awareness of the range of texts available, with the concept of text understood in an expansive sense of the term, and how various texts can be accessed and navigated in digital and analogue contexts. Initially, it is important to understand and demonstrate the texts children and young adult readers encounter are more than language-based entities. Although scholarship in children’s literature has traditionally focused on linguistically dominant texts, the term text will be used in this framework as any form of representation and communication that consists of multiple modalities including visual images, videos, music, animation, sculpture and three-dimensional forms in analogue, digital, and transmedial forms.

Definitions of children’s literature have traditionally focused on paper-based artifacts that ranged from predominantly language-based texts, such as novels and chapter books, to forms that included or featured visual art, like picturebooks. Early definitions of texts for children were concerned with analogue qualities like the drama of the turning page and the relationships between printed words and illustrations (Bader, 1976; Nikolajeva & Scott, 2000). However, an expanded definition of texts for children would need to conceptualize a text as a multimodal, digital, and cultural artifact embedded in an array of discursive and literary practices. Contemporary definitions of children’s literature have needed to focus on the multimodal aspects of picturebooks and comics, the technologies of digital platforms and devices like picturebook apps and e-readers, and the potential of emerging transmedial narratives (Kummerling-Meibauer, 2018).

Multimodal texts intended for children and young adult readers include a wide range of modalities, technologies, sensory channels, and layouts that are produced and distributed across a range of formats and platforms made available for navigating specific types of texts. This expanded definition also suggests differences among fictional, expository, rhetorical, poetic, and encyclopaedic genres. By expanding the concept of text, literacy educators are making room for a plethora of literary possibilities for today’s readers across digital and analogue contexts, providing increased access and more choices for helping readers connect their interests and passions.

A primary challenge in supporting emergent and adolescent readers is helping them come to know what texts are available, what texts are sanctioned and privileged by schools and educational institutions, what texts are considered outside the traditional canon, and what texts are deemed appropriate for consumption at particular stages of one’s life and educational journey. In recent years, the number of books and digital resources challenged by parents, caregivers, and other stakeholders on political, religious, and ethical grounds has skyrocketed. According to statistics published by the American Library Association, since 2016, the number of banned books by local and statewide agencies has expanded one hundred-fold or more. The more texts for children are made inaccessible or deemed inappropriate for students to access in school or public libraries, the more children and caregivers will have to find other means for accessing the types of texts that are of interest, and challenge educators to rethink traditional norms.

It is important to understand that as children begin to locate and access texts across digital, audio, video-based and augmented reality platforms they begin to develop an identity as a reader and have broader opportunities to locate and access the types of texts they need and enjoy beyond those mandated in the school curriculum. Knowing what texts are available, appropriate, sanctioned, and censored is part of learning to read and a primary component of textual awareness. The ability to identify, locate, and access a wide variety of texts across digital and analogue platforms and the opportunity to choose for oneself what is to be read is another essential aspect of textual awareness and offers increased possibilities that readers will find a space for themselves and identify as readers in today’s multimodal world.

 

Semiotic awareness

Semiotic awareness focuses on how texts work and the various semiotic resources and modalities that are used in their design, construction, and dissemination. It also focuses on how various modes, genres, formats, platforms, and structures are used to represent and communicate potential meanings. Towndrow et al. (2013) initially posited the concept of semiotic awareness as ‘an alertness to the representational possibilities that any form can afford, in which contexts and for whom and how and why’ (p. 20). This conceptualization focused on the affordances of particular semiotic modes and resources and the design choices used in actualizing intended and potential meanings. In other words, semiotic awareness is an awareness of the ways people use words, images, design elements, layouts, and technologies to make sense of the world and represent and communicate potential meanings in particular social, cultural, historical, and political contexts.

Lim and Toh (2020) suggested that multimodal semiotic awareness helps students understand how multimodal texts are made meaningful, and the contributions and interplay of specific semiotic modes in selected multimodal ensembles. Developing a metamodal awareness for understanding how multimodal texts work, extends the concept of semiotic awareness to fully embrace a wide range of text types, modalities, genres, layouts and formats, and diverse semiotic resources used to transform and construct textual artifacts, experiences, and performances.

Extending this work more specifically within children’s literature scholarship, semiotic awareness includes: 1) how different modes and semiotic resources work in picturebooks and other forms of children’s and young adult literature, 2) the affordances, limitations, and materials aspects of particular modes and semiotic resources used in these types of texts, 3) the array of metalanguages used to describe aspects of multimodal texts and literacy practices, 4) the basic concepts of visual grammar, narrative structures, graphic design elements, 5) an understanding of how texts for children and young adults are designed, produced, and disseminated, and 6) how semiotic work is performed across a range modalities, platforms, and devices in analogue and digital contexts.

There are numerous literary and linguistic structures that have been part of traditional literature for children and reading lessons for many years. A focus on plot, character, setting, and theme as part of English language instruction is common across many pedagogical approaches to literature education. However, as the texts that children and young adults encounter are transformed across modalities, platforms, and technologies, children will need to attend to additional semiotic resources that go beyond traditional literary features and structures. Attention to visual grammar (Kress & van Leeuwen, 1996), peritextual features (Pantaleo, 2003), metafictive devices (Pantaleo, 2014), digital platforms and applications (Serafini, Kachorsky, & Aguilera, 2015), the affordances of visual and linguistic modalities (Serafini, 2014), and transmedial narratives (Ryan & Thon, 2014) is deemed necessary to move forward.

 

Critical awareness

Critical awareness is an awareness of the ideological aspects of texts and the metacognitive and sociocultural aspects involved in the process of reading and interpretation. It is an understanding that all texts serve to empower certain groups of people while simultaneously undermining the power of other groups. It is an awareness of how texts operate in particular social, cultural, historical, political, and economic spaces, and how semiotic resources are used in service of particular interests. In addition, critical awareness views reading comprehension as a social process of generating meanings in transactions with multimodal phenomena from a variety of perspectives to meet and challenge the requirements of particular social, cultural, historical, political, and economic contexts (Serafini, 2023b). In similar fashion to theories of critical literacy, critical awareness requires ‘understanding the relationship between texts, meaning making and power in order to undertake transformative social action that contributes to the achievement of a more equitable social order’ (Janks & Vasquez, 2011, p. 1).

Theories of critical literacy, digital literacies, and critical media literacy are directly connected to the concept of critical awareness calling attention to the roles that digital media, in particular picturebook apps and social media, play in the life of today’s students. Burnett and Merchant (2019) offered a framework to address critical perspectives on digital literacies that focused on 1) practices – what students do with digital media, 2) identities – how student identities and digital media are connected, and 3) networks – the connections students make with other students and affinity groups. This framework called attention to aspects of critical awareness that go beyond analogue technologies and sets of learning skills to consider the contexts in which comprehension and literacies occur.

The process of becoming critically aware involves looking at how texts work in service of certain groups and problematizing certain interpretations over others. It is based on contextualizing texts in social and cultural settings and understanding that universal, neutral, innocent, or value-free readings are not available and that all texts are cultural and social artifacts constructed by specific people for specific purposes. Critical awareness is a form of socially embedded literacy practice designed to ‘enhance both people’s agency over their life trajectories and communities’ intellectual, cultural, and semiotic resources in multimediated economies’ (Luke & Freebody, 1999, p. 2).

While there may be no universal approach for developing a sense of critical awareness, issues such as diversity, privilege, equity, marginalization, and power are deeply connected to the theories and pedagogies supporting this concept. Critical awareness is a perspective or stance, ‘that seeks to unsettle and contest taken-for-granted cultural and social practices, and to use literacy to spark deep questioning and imagine more just futures’ (Spina, Comber & Jeffries, 2023, p. 217).

 

Developing Textual, Semiotic, and Critical Awareness

Pedagogical frameworks have been posited for helping readers understand how language works and for developing a vocabulary or metalanguage to enrich the understandings of the underlying compositional and syntactical structures of written language (Callow, 2023). Using a metalanguage to describe the linguistic parts of a text provides teachers with a way of explicitly teaching about a particular type of written text, genre, or literacy practice. Expanding these ideas about developing metalanguages focusing on linguistic elements to ways of thinking about the modalities and semiotic resources associated with multimodal ensembles in digital and analogue contexts is the primary foundation for developing metamodal awareness.

Addressing the three dimensions of the framework for metamodal awareness involves noticing the various types of children’s literature available and attending to how modes and semiotic resources serve the narrative or informational text. In conjunction with the process of noticing, readers need to begin to name or construct a vocabulary for discussing and understanding the affordances of various modes, semiotic resources, materials, and technologies. To name something is to identify it and to separate it from the rest of one’s experiences and this process is a precursor for understanding and the ability to use various resources for representing and communicating ideas, identities and ideologies in one’s own multimodal designs. In addition to the textual and semiotic awareness dimensions, the critical awareness dimension requires readers to consider the social, cultural, and historical contexts in which these texts are embedded and the associated issues of power, agency, and equity.

 

Developing textual awareness

In order to help develop a textual awareness in readers today, teachers need to abdicate their traditional role as the ‘arbiters of meaning’ and adopt a more facilitative stance, serving as curators or docents (Serafini & Youngs, 2006) to provide access to and guide children through the maze of texts made available for them to experience. Rather than providing a list of required readings for readers to adopt, these lists serve to take away an essential component of being a reader, namely choosing what one reads.

Teachers need to help readers develop their own textual preferences and interests rather than simply adhere to traditional lists and reading mandates. In order to serve as literary docents, teachers should strive to keep up-to-date with current children’s and young adult literature publications, demonstrate how to browse through available texts and digital platforms, help students make more appropriate selections aligned with their own interests, conduct book talks as often as possible, read aloud a wide variety of narrative and information texts, and provide increased access to digital texts as well as paper-based novels and picturebooks.

Foundational supports for fostering textual awareness begin by providing extended amounts of time to read and explore available texts, allowing readers access to a wider range of texts across digital and analogue platforms, supporting more students’ choices in what they are interested in reading, and fostering the love of reading in a variety of contexts and settings. As students become more aware of the kinds of texts that are available, they begin to develop their own identities as readers and become more empowered in the process.

 

Developing semiotic awareness

Semiotic awareness is initiated by noticing and attending to modes and semiotic resources that have traditionally been overlooked by language dominant pedagogies. While calling attention to how language works and developing a vocabulary or metalanguage for describing the structures and features of language is important, it is also important to call attention to other modes, designs, platforms, devices, and structures of the multimodal and digital texts experienced by today’s  readers. Moving beyond the basic distinctions between words and images, attention to the typographic, design, compositional, and technological elements of children’s literature expands readers’ metamodal awareness and enhances their transactions with contemporary texts across digital and analogue formats and platforms (Serafini, 2014).

Originally proposed by Towndrow et al. (2013), the concept of semiotic awareness was developed to consider the affordances of a variety of semiotic resources and the semiotic work these resources accomplish while embedded in social practices, material resources, and sociocultural contexts. In addition, Bezemer and Cowan (2020) used the terms engagement, interpretation, and signification to move beyond traditional perspectives on reading and expand their understandings of reading and students’ meaning making from a social semiotic perspective. Semiotic awareness has sought to be more inclusive of additional modalities and strategies for comprehending multimodal texts than some of the narrower views of reading with its focus on decoding processes.

Drawing upon systemic functional theories and approaches, Lim (2018) recognized a multimodal turn in literacy education and called attention to the ways modes other than written language are used in the reading process. Based on the Learning by Design model developed originally by Cope and Kalantzis (2015), Lim (2018) proposed a pedagogical framework that featured situated practice, overt instruction, critical framing, and transformed practice, so that through ‘collective negotiation, argumentation, and citing of evidence to defend their interpretation, students [would] develop a more robust analysis of the multimodal texts they view’ (p. 7).

By shifting the emphasis of instruction from decoding and comprehending written language to a multimodal perspective on meaning-making, teachers are invited to help young readers consider the ways multiple modalities are used in the construction of narrative and informational texts, including the use of visual images, sound effects, music, and gestures in multimodal and digital ensembles. In addition to the theoretical shifts this requires, it also requires an expanded set of reading and comprehension strategies than those proposed by reading researchers working from a linguistically dominant perspective (Duke & Pearson, 2002).

 

Developing critical awareness

Critical awareness posits a pedagogical space that provides access and opportunities for teachers to foster the skills, dispositions, and identities necessary for carrying out emancipatory agendas (Comber, 2001; Luke, 1995). Although critical theories and critical literacies have not been associated with universal instructional approaches and are seen more as a stance or perspective on literacy, making these theories accessible for classroom teachers and other educators is an essential part of expanding teachers’ instructional repertoires to include critical awareness as part of their pedagogical framework.

Lewison, Flint and Van Sluys (2002) offered a framework for approaching multimodal texts from a critical perspective. Their framework incorporated four dimensions: 1) disrupting the commonplace, 2) interrogating multiple viewpoints, 3) focusing on sociopolitical issues, and 4) taking action and promoting social justice. This dimension was intended to provide teachers and literacy educators with a framework for taking a critical stance and working towards more culturally sustaining and relevant pedagogies (Ladson-Billings, 2014; Paris, 2012).

Simpson (1996) proposed working towards critical understandings by asking different types of questions concerning the children’s literature her students were reading, and to help them become more aware of how texts worked upon them and reduce their susceptibility for manipulation through the texts experienced. Questions concerning who is represented and/or missing from a story, how the illustrations might change one’s understandings, the consideration of potential messages offered by the author or illustrator, and the ways one’s own interpretations might conflict with other interpretations of the story were proposed. In short, Simpson (1996) supported a variety of ways for her students to begin asking critical questions in order to develop their critical awareness of the texts they encountered.

Cappello, Wiseman and Turner (2019) proposed a framework for supporting critical multimodal literacy which also included four dimensions, namely 1) communicating and learning with multimodal tools, 2) restorying, representing, and redesigning, 3) acknowledging and shifting power relations, and 4) leveraging multimodal resources to critique and transform sociopolitical realities. Their work complicates the tension between dominant narratives and the stories retold and re-represented by children from traditionally marginalized groups. Based on a previously established tripartite framework for visual and multimodal interpretation, these teacher educators developed a methodology for analyzing contemporary texts, specifically student-made multimodal texts, by building on theories of critical literacy and multimodal analysis.

More recently, Nash (2021) has drawn upon critical literacy’s emphasis on interrogating subjectivity to focus readers’ critical awareness on the ways they construct their own meanings with texts. Nash (2021) proposed four reading practices for approaching online reading and inquiry from a critical perspective, including 1) digital inquiry, 2) navigation, 3) evaluation, and 4) synthesis (p. 716). These pedagogical examples that serve to foster a sense of critical awareness involve asking new questions of texts, holding one’s interpretations tentatively to consider other points of view, interrogating sociocultural messages, and reconsidering the ways different modalities, in particular visual images, design elements, and digital platforms, work to serve certain groups of people and marginalize others.

 

Concluding Remarks

The evolution of multimodal and multiliteracies curricular frameworks from a focus on cognitively based visual literacies (Debes, 1968), and critical thinking skills (Leithwood, 2006) to frameworks embracing semiotics, social semiotics, and critical theories, provides an avenue for rethinking behaviourist, cognitive, and psycholinguistic models of literacy pedagogy to include critical and ideological frames for expanding views of literacy education. A shift from a primary focus on language and metalinguistic awareness to a metamodal awareness across textual, semiotic, and critical dimensions provides an avenue to move beyond linguistic imperialism (McDonald, 2013) to critically examine the roles that modes other than written language play in the narratives and informational texts students experience in today’s digital and analogue environments. Semiotic and social semiotic perspectives on reading challenge the myopic views of linguistically dominant frameworks to expand our understandings of the multiple modalities and contexts of contemporary representational and communicational platforms, formats, devices, and structures.

An awareness of not only the features and structures of the texts designed and produced for emerging and adolescent readers, but the social, cultural, and historical contexts and environments in which these texts are produced and consumed, requires merging critical theories with theories of literacy education and children’s literature. By crossing traditional theoretical boundaries, scholars working to understand the ways semiotic, multimodal, digital, and critical theories support research and pedagogies connected to types of children’s and young adult literature acknowledge transdisciplinary perspectives for illuminating new avenues for understanding.

The texts that students experience in today’s visual and digital environments have expanded beyond written language and thus require new frameworks for addressing additional modalities, structures, and technologies. It is important to understand the ways children are constructed as readers by the reading practices made available to them across a variety of educational settings, and how these reading practices privilege some interpretations, groups, and individuals over others. Textual, semiotic, and critical awareness of the types of texts that are made available, how the semiotic resources embedded in multimodal texts work, and the interests these complex texts serve are essential aspects of contemporary literacy education.

Considering children’s literature from semiotic, multimodal, and critical perspectives offers new ways of analyzing and experiencing the potential of emerging technologies, the affordances of visual and multimodal resources and layouts, and the complexities inherent in combining words and images in texts for emerging and adolescent readers. Breaking out of traditional perspectives and academic boundaries allows literacy educators to straddle new worlds and consider innovative research perspectives and pedagogical frameworks.

 

 

References

Apol, L. (1998). ‘But what does this have to do with kids?’: Literary theory and children’s literature in the teacher education classroom. Journal of Children’s Literature, 24(2), 32–46.

Bader, B. (1976). American picturebooks from Noah’s ark to the beast within. Macmillan.

Barton, G. (2019). Developing literacy and the arts in schools. Routledge. https://doi.org/10.4324/9780429316067

Barton, G., Burke, K., & Freebody, P. (2022). Disciplinary literacies in the arts: Semiotic explorations of teachers’ use of multimodal and aesthetic metalanguage. International Journal of Education & the Arts, 23(8), 1–22.

Berry, R. (2005). Making the most of metalanguage. Language Awareness, 14(1), 3–20. https://doi.org/10.1080/09658410508668817

Bezemer, J., & Cowan, K. (2020). Exploring reading in social semiotics: Theory and methods. Education 3-13, 49(1), 107–118. https://doi.org/10.1080/03004279.2020.1824706

Bland, J. (2024). Positioning children’s literature in English language education within English language and literature education. Children’s Literature in English Language Education, 12(1), ii–v. https://clelejournal.org/editorial-janice-bland-8/

Burnett, C., & Merchant, G. (2019). Revisiting critical literacy in the digital age. The Reading Teacher, 73(3), 263–266. https://doi.org/10.1002/trtr.1858

Callow, J. (2023). The shape of text to come: How image, text and other modes work (2nd ed.). Primary English Teaching Association Australia.

Cappello, M., Wiseman, A. M., & Turner, J. D. (2019). Framing equitable practices: Potential of critical multimodal research. Literacy Research: Theory, Method, and Practice, 68, 205–225. https://doi.org/10.1177/2381336919870274

Comber, B. (2001). Negotiating critical literacies. School Talk, 6(3), 1–3. https://doi.org/10.4324/9781410600288

Cope, B., & Kalantzis, M. (2015). A pedagogy of multiliteracies: Learning by design. Palgrave Macmillan. https://doi.org/10.1057/9781137539724

Debes, J. L. (1968). Some foundations for visual literacy. Audiovisual Instruction, 13, 961–964.

Dinham, J. (2020). Delivering authentic arts education (4th ed.). Cengage.

Duke, N., & Pearson, P. D. (2002). Effective practices for developing reading comprehension. In A. Farstrup & S. J. Samuels (Eds.), What research has to say about reading instruction (pp. 205–242). International Reading Association. https://doi.org/10.1598/0872071774.10

Fagerston, K. B., Holmsten, E., & Cunningham, U. (2010). Multimodal communication and meta-modal discourse. In R. Taiwo (Ed.), Handbook of research on discourse behavior and digital communication: Language structures and social interaction (Vol. I, pp. 145–162). Information Science Reference. https://doi.org/10.4018/978-1-61520-773-2.ch009

Janks, H., & Vasquez, V. (2011). Editorial: Critical literacy revisited: Writing as critique. English Teaching: Practice and Critique, 10(1), 1–6.

Jappy, T. (2013). Introduction to Piercean visual semiotics. Bloomsbury.

Kress, G., & van Leeuwen, T. (1996). Reading images: The grammar of visual design. Routledge.

 

Kummerling-Meibauer, B. (Ed.). (2018). The Routledge companion to picturebooks. Routledge. https://doi.org/10.4324/9781315722986

Ladson-Billings, G. (2014). Culturally relevant pedagogy 2.0: a.k.a.the remix. Harvard Educational Review, 84(1), 74–84. https://doi.org/10.17763/haer.84.1.p2rj131485484751

Leithwood, K. A. (2006). Teaching for deep understanding: What every educator should know. Corwin Press.

Lewison, M., Flint, A. S., & Van Sluys, K. (2002). Taking on critical literacy: The journey of newcomers and novices. Language Arts, 79(5), 382–392. https://doi.org/10.58680/la2002255

Lim, F. V. (2004). Problematising ‘semiotic resource’. In E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on multimodality (pp. 51–63). John Benjamins.

Lim, F. V. (2018). Developing a systemic functional approach to teach multimodal literacy. Functional Linguistics, 5(13), 1–17. https://doi.org/10.1186/s40554-018-0066-8

Lim, F. V., & Toh, W. (2020). How to teach digital reading? Journal of Information Literacy, 14(2), 1–20. https://doi.org/10.11645/14.2.2701

Luke, A. (1995). When basic skills and information processing just aren’t enough: Rethinking reading in new times. Teachers College Record, 97(1), 95–115. https://doi.org/10.1177/016146819509700105

Luke, A., & Freebody, P. (1999). Further notes on the four resources model. Reading Online. http://www.readingonline.org/research/lukefreebody.html

McDonald, E. (2013). Embodiment and meaning: Moving beyond linguistic imperialism in social semiotics. Social Semiotics, 23(3), 318–334. https://doi.org/10.1080/10350330.2012.719730

Mills, K. A. (2016). Literacy theories for the digital age: Social, critical, multimodal, spatial, material, and sensory lenses. Multilingual Matters. https://doi.org/10.21832/9781783094639

Nash, B. (2021). Constructing meaning online: Teaching critical reading in a post-truth era. The Reading Teacher, 74(6), 713–722. https://doi.org/10.1002/trtr.1980

New London Group (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60–92.

Nikolajeva, M., & Scott, C. (2000). The dynamics of picturebook communication. Children’s Literature in Education, 31(4), 225–239. https://doi.org/10.1023/A:1026426902123

Pantaleo, S. (2003). ‘Godzilla lives in New York’: Grade 1 students and the peritextual features of picture books. Journal of Children’s Literature, 29(2), 66–77.

Pantaleo, S. (2014). The metafictive nature of postmodern picturebooks. The Reading Teacher, 67(5), 324–332. https://doi.org/10.1002/trtr.1233

Paris, D. (2012). Culturally sustaining pedagogy: A needed change in stance, terminology, and practice. Educational Researcher, 41(3), 93–97. https://doi.org/10.3102/0013189X12441244

Ryan, M.-L., & Thon, J.-N. (Eds.). (2014). Storyworlds across media: Towards a media-conscious narratology. University of Nebraska Press. https://doi.org/10.2307/j.ctt1d9nkdg

Serafini, F. (2012). Expanding the four resources model: Reading visual and multimodal texts. Pedagogies: An International Journal, 7(2), 150–164. https://doi.org/10.1080/1554480X.2012.656347

Serafini, F. (2014). Reading the visual: An introduction to teaching multimodal literacy. Teachers College Press.

Serafini, F. (2023a). How multimodality matters in children’s literature scholarship. The Australian Journal of Language and Literacy, 46, 1–20. https://doi.org/10.1007/s44020-023-00046-2

Serafini, F. (2023b). Implications of multimodal theories for literacy research and pedagogy. In E. D. Romero, J. Bobkina, S. Stefanova, & C. Herrero (Eds.), Rethinking multimodal literacy in theory and practice (pp. 17–35). Peter Lang.

Serafini, F., Kachorsky, D., & Aguilera, E. (2015). Picturebooks 2.0: Transmedial features across narrative platforms. Journal of Children’s Literature, 41(2), 16–24.

Serafini, F., & Pantaleo, S. (2022). The evolution of children’s literature: Metafictive devices and digital mediation. In D. Yaden, & T. Rogers (Eds.), International encyclopedia of education (4th ed., Vol. 10, pp. 620–632). Elsevier. https://doi.org/10.1016/B978-0-12-818630-5.07070-6

Serafini, F., & Youngs, S. (2006). Around the reading workshop in 180 days: A month by month guide to quality instruction. Heinemann.

Simpson, A. (1996). Critical questions: Whose questions? The Reading Teacher, 50(2), 118–127.

Spina, N., Comber, B., & Jeffries, M. (2023). Critical literacies & social justice: Social justice approaches in schools. International Encyclopedia of Education, 10, 217–224. https://doi.org/10.1016/B978-0-12-818630-5.07014-7

Towndrow, P. A., Nelson, M. E., & Yusuf, W. F. B. M. (2013). Squaring literacy assessment with multimodal design: An analytic case for semiotic awareness. Journal of Literacy Research, 45(4), 327–355. https://doi.org/10.1177/1086296X13504155

Townsend, J. R. (1969). The present state of children’s literature. In S. Egoff, G. T. Stubbs, & L. F. Ashley (Eds.), Only connect: Readings on children’s literature (pp. 417–418). Oxford University Press.