Abstracts Category : Other

Attention and Dual Coding Theory: an Interaction Model Using Subtitles as a Paradigm

by Moreno Andreu Oliver

We have conducted two experiments on subtitling and its presentation as part of theHBB4ALL European project. Nowadays, the tendency for content consumption is shiftingtowards an on demand dynamic performed on an array of different devices. With that inmind, not only did we explore the effect the audio language has on the way we processcontent but also the ways in which it is affected when using different devices to consumecontent.In the first experiment we explored the relation between attention, subtitles and informationprocessing. We presented the participants with two sets of clips, one with audio in Englishand the other with audio in Norwegian in order to explore the effect on the reading ofsubtitles (in Polish). As predicted by the Dual Coding Theory (Paivio, 1986) multiple channelinformation processing is found to be a strong facilitator towards better performance incontent comprehension. Moreover, attention resource distribution is also found to be alteredby the experimental conditions. Resource allocation and demand evaluation mechanisms(Kahneman, 1973) are considered to be the main causes of those differences.In the second experiment the focus is more applied. A comparison was made betweeninformation processing on different devices in order to explore how reading and processingbehaviour changes from platform to platform. To assess whether or not there are differencesbetween the different platforms, we presented a set of clips in an unknown language andcompared the results of comprehension and visual questionnaires as well as eye trackingreadings. No significant differences were found across devices in content comprehension. Itis not necessary to adapt subtitles to fit these devices as we are capable of compensating forany differences in their characteristics.The following general discussion considers the findings of the two experiments to be relevantto the understanding of our attentional and learning system. In both experimental situations,our cognitive processes are able to retrieve the necessary mechanisms to compensate for thedifferences made in the experimental manipulation.Advisors/Committee Members: [emailprotected] (authoremail), true (authoremailshow), Soler Vilageliu, Olga (director), true (authorsendemail).