Building Bridges for Multimodal Research.
International Perspectives on Theories and Practices of Multimodal Analysis.
Sprache - Medien - Innovationen. Frankfurt am Main: Peter Lang.
Multimodality is one of the most influential semiotic theories for analysing media artefacts. The concepts of this theory are, however, heterogeneously widespread, especially with regard to approaches in Germany on the one hand and those within the international context on the other. The book takes these differences as a starting point and aims at activating their discussion and opening new approaches to the analysis of multimodal documents. It features contributions by researchers from more than 15 nations and various disciplines all dealing with the notion of multimodality. It includes theoretical reflections on the term and its research, thouhts about methodological, empirical, and experimental approaches as well as diverse example analyses of various multimodal artefacts.
1 Bridging the Gap between Here and There: Combining Multimodal Analysis from International Perspectives
I THEORETICAL REFLECTIONS ON THE NOTION AND WORK OF MULTIMODALITY
2 Issues in Multimodality: Reflecting on Definitions, Transcription, and Analysis
3 From Text Linguistics to Multimodality: Mapping Concepts and Methods across Domains
4 Cognitively Oriented Semiotics as a Common Descriptive Framework for Pictorial and Verbal Representation
II METHODS FOR MULTIMODALITY RESEARCH
5 Spotlight on the Image: Adding the Missing Visual Piece to the Multimodal Puzzle
6 Visual Rhetoric and Cognitive Semantics: The Relevance of Entrenched Conceptual Patterns for the Reconstruction of Visual and Multimodal Arguments
7 Conflicts and Pictures: A Study of Images and Agonality in Discourse
8 Multimodality in Perspective: Creating a Synergy of the Discourse Historical Approach and the Framework of Visual Grammar
Shaimaa El Naggar
9 Structure and Multimodal Texts
III EMPIRICAL AND EXPERIMENTAL APPROACHES TO MULTIMODAL ANALYSIS
10 Automatic Classification of Iconic Images Based on a Multimodal Model: An Interdisciplinary Project.
Simone Ponzetto, Hartmut Wessler, Lydia Weiland, Stephan Kopf, Wolfgang Effelsberg, Heiner Stuckenschmidt
11 On the Use of Different Modalities in Political Communication: Evidence from German Election Manifestos
Marc Debus, Heiner Stuckenschmidt, Hartmut Wessler
12 Seeing the Unforeseen: Eye-Tracking Reading Paths in Multimodal Webpages
Martin Kaltenbacher, Thomas Kaltenbacher
13 An Experimental Approach to Multimodality: How Musical and Architectural Styles Interact in Aesthetic Perception
Martin Siefkes, Emanuele Arielli
14 Narrative Process Annotation of Comic Strips for Corpus Analysis
IV EXAMPLE ANALYSES OF VARIOUS MULTIMODAL ARTEFACTS
15 Combining Computer Vision and Multimodal Analysis: A Case Study of Layout Symmetry in Bilingual In-Flight Magazines
16 Embodied Meaning in Audio-Visuals: First Steps Towards a Notion of Mode
17 Documentary Film as Multimodal Argumentation: Arguing Audio-Visually About the 2008 Financial Crisis
V SHORT MEMOS ON MULTIMODAL ANALYSES
18 Reflecting on a Gap between Polyphony and Multimodality in Online Media Formats
19 When Here is Now and There is Then: Bridging the Gap in Time with “Sumer Is Icumen In”
20 Visual Literacy in EAP: The Dialogue between Reader/Designer Features and Multimodal Text
List of Contributors