A Semantic Associative Computation Method for Automatic Decorative-Multimedia Creation with 'Kansei' Information

Kiyoki, Y. and Chen, X.

In the design of multimedia systems, one of the important issues is how to deal with 'Kansei' of human beings. The concept of 'Kansei' in Japanese includes several meanings on sensitive recognition, such as 'impression', 'human senses', 'feelings', 'sensitivity', 'psychological reaction' and 'physiological reaction'. This paper presents a new concept of 'automatic decorative-multimedia creation' and a semantic associative computation method. This method realizes automatic main-media decoration with dynamic sub-media data selection for representing main-media as decorative multimedia. The aim of this method is to create a new field of 'automatic decorative-media art' with 'semantic associative computing'. This paper defines an 'automatic media decoration model' with semantic spaces and media-decoration functions. Automatic media decoration is realized by applying the Mathematical Model of Meaning (MMM) to a media-transmission space for computing semantic correlations between main-media objects and sub-media. The process of this dynamic media decoration method consists of the following functions: (1) Extraction of semantic 'Kansei' features of 'main-media object', such as music, image and video. (2) Mapping of the main-media object onto the media-transmission space between main-media and sub-media. (3) Semantic associative computation of correlations between the main-media object and the features of the sub-media space by MMM, and creating a vector of the main-media object with the features of the sub-media space. (4) Mapping of the vector of the main-media object to the sub-media space, and semantic associative computing between the main-media object and sub-media data. (5) Semantic ranking of sub-media objects as the result of the semantic associative computation, and selects one of the sub-media objects with high correlation values to the target main-media object. (6) Automatic rendering of the target main-media object with the selected sub-media object for decorating the main-media presentation. This paper shows several significant applications of the semantic associative computation method for 'automatic decorative-media creation'.

Cite as: Kiyoki, Y. and Chen, X. (2009). A Semantic Associative Computation Method for Automatic Decorative-Multimedia Creation with 'Kansei' Information. In Proc. Sixth Asia-Pacific Conference on Conceptual Modelling (APCCM 2009), Wellington, New Zealand. CRPIT, 96. Kirchberg, M. and Link, S., Eds. ACS. 7-15.

(from crpit.com) (local if available)