Building Bridges for Multimodal Research.
International Perspectives on Theories and Practices of Multimodal Analysis.
Sprache - Medien - Innovationen. Frankfurt am Main: Peter Lang.
2015.

Multimodality is one of the most influential semiotic theories for analysing media artefacts. The concepts of this theory are, however, heterogeneously widespread, especially with regard to approaches in Germany on the one hand and those within the international context on the other. The book takes these differences as a starting point and aims at activating their discussion and opening new approaches to the analysis of multimodal documents. It features contributions by researchers from more than 15 nations and various disciplines all dealing with the notion of multimodality. It includes theoretical reflections on the term and its research, thouhts about methodological, empirical, and experimental approaches as well as diverse example analyses of various multimodal artefacts.

The book somewhere else:
Publisher's Website
Amazon

Contents

Preface

1 Bridging the Gap between Here and There: Combining Multimodal Analysis from International Perspectives
Janina Wildfeuer

I THEORETICAL REFLECTIONS ON THE NOTION AND WORK OF MULTIMODALITY
2 Issues in Multimodality: Reflecting on Definitions, Transcription, and Analysis
Diane Mavers

3 From Text Linguistics to Multimodality: Mapping Concepts and Methods across Domains
Hartmut Stöckl

4 Cognitively Oriented Semiotics as a Common Descriptive Framework for Pictorial and Verbal Representation
Alina Kwiatkowska

II METHODS FOR MULTIMODALITY RESEARCH
5 Spotlight on the Image: Adding the Missing Visual Piece to the Multimodal Puzzle
Ognyan Seizov

6 Visual Rhetoric and Cognitive Semantics: The Relevance of Entrenched Conceptual Patterns for the Reconstruction of Visual and Multimodal Arguments
Sandra Handl

7 Conflicts and Pictures: A Study of Images and Agonality in Discourse
Anna Mattfeldt

8 Multimodality in Perspective: Creating a Synergy of the Discourse Historical Approach and the Framework of Visual Grammar
Shaimaa El Naggar

9 Structure and Multimodal Texts
Wendy Bowcher

III EMPIRICAL AND EXPERIMENTAL APPROACHES TO MULTIMODAL ANALYSIS
10 Automatic Classification of Iconic Images Based on a Multimodal Model: An Interdisciplinary Project.
Simone Ponzetto, Hartmut Wessler, Lydia Weiland, Stephan Kopf, Wolfgang Effelsberg, Heiner Stuckenschmidt

11 On the Use of Different Modalities in Political Communication: Evidence from German Election Manifestos
Marc Debus, Heiner Stuckenschmidt, Hartmut Wessler

12 Seeing the Unforeseen: Eye-Tracking Reading Paths in Multimodal Webpages
Martin Kaltenbacher, Thomas Kaltenbacher

13 An Experimental Approach to Multimodality: How Musical and Architectural Styles Interact in Aesthetic Perception
Martin Siefkes, Emanuele Arielli

14 Narrative Process Annotation of Comic Strips for Corpus Analysis
Elisa Vales

IV EXAMPLE ANALYSES OF VARIOUS MULTIMODAL ARTEFACTS
15 Combining Computer Vision and Multimodal Analysis: A Case Study of Layout Symmetry in Bilingual In-Flight Magazines
Tuomo Hiippala

16 Embodied Meaning in Audio-Visuals: First Steps Towards a Notion of Mode
Christina Schmitt

17 Documentary Film as Multimodal Argumentation: Arguing Audio-Visually About the 2008 Financial Crisis
Assimakis Tseronis

V SHORT MEMOS ON MULTIMODAL ANALYSES
18 Reflecting on a Gap between Polyphony and Multimodality in Online Media Formats
Jan Krasni

19 When Here is Now and There is Then: Bridging the Gap in Time with “Sumer Is Icumen In”
Kate Maxwell

20 Visual Literacy in EAP: The Dialogue between Reader/Designer Features and Multimodal Text
Dorra Moalla

List of Contributors