Lip-synchronization: a method without the use of visemes

Abstract

In the present article we describe a simplified method to resolve the problem of lip synchronization with a previously recorded audio file. The algorithm receives as input the audio file, in a standard format (WAV) and returns as result a text file that contains average amplitude values corresponding to a predefined fraction of a second. In order to achieve the synchronization, the values such defined are correlated with lip animation sequences. The adopted solution allows an automatic production process that is, on one hand, the analysis of an arbitrary audio file recorded by a user and, on other hand, the association of the file with a sequence of animation files. The method was successfully employed in a large scale educational project in Mexico.