Recurrent semiosis in the multimodal discourse of drama and the prospects of AI development

Loginova, Elena Georgievna
PhD (Philology), Associate Professor, Department of Germanic Languages and Language Teaching Methodology, Ryazan State University named for S. Yesenin, Ryazan, Russia
e.loginova@365.rsu.edu.ru


Abstract
The study presents the results of the cognitive-semiotic analysis of gestures in the discourse of drama. By the discourse of drama we understand a semiotic continuum that includes a basic discourse (a play) and its functional derivative (a theatrical performance), the latter being interpreted as an example of recurrent semiosis, in which different semiotic systems (verbal and non-verbal) and different semiotic modes (visual, auditory, tactile, kinetic, etc.) are involved.
In the discussion proposed here we assume that the interplay of speech and co-speech gestures can make the content salient and thus trigger off the amplification of meaning or bring about the emergence of new semantic projections. Since the focus of a theatrical performance is on visualization and intensification it accounts for gestural patterns enhancing the expressiveness of the verbal utterance due to the effect of cross-activation between different semiotic systems and sensory modes. That can be of some practical use for AI to mimic the appropriate communicative patterns, particularly for the subfield of deep learning.
The analysis was carried out with help of the monomodal and multimodal datasets, namely, the plays by A. Vampilov, A. Kazantsev, N. Sadur and the theatrical interpretations of these plays. To verify the hypothesis, the parametric analysis and the diagraph analysis accompanied by and statistical processing were used.
The results obtained made it possible to define the cognitive technologies which account for emotive charge and the amplification of meaning projections in the process of recurrent semiosis. It is also determined that there are certain types of verbal signs that can act as triggers to activate gestural modality.
The present study of co-occurring conceptualizations through verbal and gestural components which emerge during the communication on stage can broaden our understanding of embodied meaning construal and as such serve as big data for training AI solutions.

Keywords: modality, monomodality, multimodality, sign, semiosis, gesture, trigger, interpretation, recurrence, recurrent semiosis, communication on stage