JP2019057116A - Lip sink processing program, recording media and lip sink processing method - Google Patents

Abstract

To relieve a processing load of lip sink processing.SOLUTION: An information processor 3 is functioned as an operation pattern acquisition processing part 13 for acquiring a mouth-like operation pattern of a character corresponding to a predetermined voice sequence, sound volume level acquisition processing part 15 for acquiring a sound volume level of a voice uttered from a character, an application rate determination processing part 17 for determining, as an application rate, a degree of the magnitude of an operation pattern applied to the character on the basis of the sound volume level; and an operation pattern execution processing part 19 for executing an operation pattern with a magnitude based on the application rate according to the vocalization of the character.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a lip sync processing program, a recording medium on which the lip sync processing program is recorded, and a lip sync processing method.

  In recent years, animation by computer graphics has been used in various video content fields such as games, television programs, and movies. In such an animation, a lip sync technique for synchronizing the utterance of a character to appear and the lip shape is used.

  For example, in the game device described in Patent Document 1, the sound of the player captured by the sound input means is frequency-analyzed by the sound analysis means, and the sound is discriminated from the sound pressure distribution characteristics for each frequency component in the audible frequency band. The voice synchronization means changes the lip shape of the character according to the sound.

Japanese Patent Laid-Open No. 2002-351489

  The above-described prior art has a problem that processing load is heavy because complicated arithmetic processing is required to analyze the voice.

  The present invention has been made in view of such problems, and an object thereof is to provide a lip sync processing program, a recording medium, and a lip sync processing method capable of reducing the processing load.

  In order to achieve the above object, a lip sync processing program according to the present invention causes an information processing apparatus to utter a motion pattern acquisition processing unit that acquires a mouth-shaped motion pattern of an object corresponding to a predetermined audio string, A volume level acquisition processing unit that acquires a volume level of sound, an application rate determination processing unit that determines, as an application rate, a degree of the size of the motion pattern to be applied to the object based on the volume level, and according to the utterance of the object The operation pattern is made to function as an operation pattern execution processing unit that executes the operation pattern with a size based on the application rate.

  In order to achieve the above object, a recording medium of the present invention is a recording medium readable by an information processing apparatus on which the lip sync processing program is recorded.

  In order to achieve the above object, a lip sync processing method according to the present invention is a lip sync processing method executed by an information processing apparatus, and acquires a mouth-shaped motion pattern of an object corresponding to a predetermined audio sequence. A step of obtaining a volume level of sound to be uttered by the object, a step of determining a degree of the size of the motion pattern to be applied to the object based on the volume level as an application rate, Executing the motion pattern in a size based on the application rate in accordance with the utterance.

  According to the present invention, the processing load of lip sync processing can be reduced.

It is a system configuration figure showing an example of the whole game system composition concerning one embodiment. It is a block diagram showing an example of a functional structure of information processing apparatus. It is explanatory drawing for demonstrating an example of an audio | voice waveform and an application rate. It is explanatory drawing for demonstrating the example of the magnitude | size of the opening / closing operation | movement of a character's mouth based on an application rate. It is explanatory drawing for demonstrating the example with which the expression which laughed and the movement pattern of the mouth shape were synthesize | combined. It is explanatory drawing for demonstrating the example with which the expression pattern which cried and the movement pattern of mouth shape were synthesize | combined. It is explanatory drawing for demonstrating the example with which the angry facial expression and the movement pattern of the mouth shape were synthesize | combined. It is a flowchart showing an example of the process sequence which information processing apparatus performs. It is a block diagram showing an example of the hardware constitutions of information processing apparatus. It is a system block diagram showing an example of the whole structure of the game system which concerns on the modification provided with the audio | voice input apparatus.

<1. Background of the Invention>
First, before describing an embodiment of the present invention, the background of the present invention will be described.

  In recent years, animation by CG (computer graphics) has been used in various video content fields such as games, theaters, online distribution, TV programs, movies and the like. In such an animation, a lip sync technique for synchronizing the voice uttered by the appearing character and the mouth shape is used.

  In general, in the lip sync technology, a plurality of types of mouth shapes corresponding to one sound (for example, a vowel or a combination of vowels and consonants) produced by a character are prepared in advance, and the mouth shape is applied to each sound. Change. In this case, since it is necessary to discriminate the sound, for example, a complicated calculation process for analyzing the input voice is required, which increases the processing load. Further, when creating animation data, for example, it is necessary to manually input vowel information, which increases the burden of creating animation data.

  Therefore, as a result of earnest research, the inventors of the present application have used a mouth-shaped motion pattern corresponding to a predetermined voice sequence without discriminating the voice uttered by the character. It was thought that it is possible to synchronize utterance and mouth shape. As a result, it is possible to significantly reduce the processing load by eliminating the need for voice analysis, and to reduce the burden of creating animation data by eliminating the trouble of inputting vowel information and the like. The details will be described below.

<2. Overall configuration of game system>
An embodiment of the present invention will be described with reference to the drawings. In this embodiment, the case where the lip sync processing program of the present invention is applied to a game will be described, but the application target is not limited to a game.

  First, an example of the overall configuration of the game system 1 according to the present embodiment will be described with reference to FIG. As shown in FIG. 1, the game system 1 includes an information processing device 3, a controller 5, and a display device 7. Each of the controller 5 and the display device 7 is communicably connected to the information processing device 3. In addition, although FIG. 1 illustrates the case of being connected by wire, it may be connected wirelessly.

  The information processing apparatus 3 is a stationary game machine, for example. However, the present invention is not limited to this, and for example, a portable game machine that is integrally provided with an input unit, a display unit, and the like may be used. In addition to game machines, for example, those manufactured and sold as computers such as server computers, desktop computers, notebook computers, tablet computers, mobile phones, smartphones, fablets, etc. As such, it may be manufactured and sold as a telephone.

  The player performs various operation inputs using the controller 5. In the example illustrated in FIG. 1, the controller 5 includes, for example, a cross key 9 and a plurality of buttons 11. The controller 5 may include, for example, a joystick or a touch pad instead of or in addition to the above.

  In the present embodiment, a case where a game program which is an example of a lip sync processing program is executed by the information processing apparatus 3 will be described.

<3. Functional configuration of information processing apparatus>
Next, an example of a functional configuration of the information processing apparatus 3 will be described with reference to FIGS. 2 and 3 to 7.

  As illustrated in FIG. 2, the information processing apparatus 3 includes an operation pattern acquisition processing unit 13, a volume level acquisition processing unit 15, an application rate determination processing unit 17, an operation pattern execution processing unit 19, and a reset processing unit 21. , A smoothing processing unit 23 and a behavioral synthesis processing unit 25.

  The motion pattern acquisition processing unit 13 acquires a mouth-shaped motion pattern of a character (an example of an object) corresponding to a predetermined voice string. The predetermined speech string is composed of, for example, a plurality of vowels arranged in a random order. In addition, it is good also as a predetermined order instead of random. Further, not only vowels but also combinations of vowels and consonants may be included.

  The movement pattern is a series of mouth-shaped movements when the character utters the above-mentioned voice sequence, and the pattern varies depending on the content of the voice uttered by the character, although the magnitude of the movement varies depending on the application rate described later. It will never be done. The motion pattern is generated so that the motion corresponding to the voice sequence is started from a state in which the mouth shape is closed, and the portion corresponding to the voice sequence is repeated while the character is speaking.

  As the motion pattern, one pattern is set for each character. For example, depending on the character's personality (personality, ability, etc.) May be set differently. A plurality of patterns may be set for each character.

  The type of the character is not particularly limited as long as it is an object having a mouth capable of uttering action. For example, a human character, an animal character other than a human, a virtual biological character other than a human or an animal is used. Or an object other than a living thing. In the present embodiment, a case where the object is a human character will be described as an example as shown in FIGS.

  The operation pattern is recorded in the ROM 103, the RAM 105, the recording device 117, and the like (see FIG. 9 described later) of the information processing device 3 by executing the game program. The operation pattern acquisition processing unit 13 reads out and acquires an operation pattern from them. The operation pattern may be acquired from an external server or the like.

  The volume level acquisition processing unit 15 acquires a volume level of voice to be uttered by the character. The acquisition of the volume level is performed continuously at a predetermined time interval, for example. The volume level is included in the audio information together with the audio content (line) and the like, and is recorded in the ROM 103, the RAM 105, the recording device 117, and the like of the information processing apparatus 3 by executing the game program. The volume level acquisition processing unit 15 reads and acquires the volume level from them. When voice is input in real time from a voice input unit such as a microphone, the volume level acquisition processing unit 15 acquires a volume level based on the volume of the input voice (see FIG. 10 described later).

  An example of a speech waveform is shown in the upper part of FIG. In FIG. 3, the amplitude of the speech waveform corresponds to the volume level. In the example shown in FIG. 3, utterance is started at time T1 (beginning of speaking), the volume is relatively high from time T1 to T2, the volume is relatively low from time T2 to T3, and at time T3. Speaking ends (end of talk).

  Returning to FIG. 2, the application rate determination processing unit 17 determines the degree of the size of the motion pattern applied to the character as the application rate based on the volume level. The application rate is determined by a numerical value of 0% to 100%, for example. When the numerical value is 0%, the character's mouth shape is closed, and as the numerical value increases, the motion pattern reflected in the character increases in magnitude. At 100%. The application rate determination processing unit 17 determines the application rate to be 0% when the volume level is 0 (including the case near 0), increases the application rate as the volume level increases, and the volume level is set in advance. When the value exceeds the set maximum value, the application rate is determined to be 100%.

  Note that, as shown in the upper part of FIG. 3, the volume level may fluctuate rapidly. For this reason, for example, when the application rate is determined in accordance with the value of the sound volume level as it is, the size of the character's mouth shape changes rapidly, resulting in an unnatural motion.

  Therefore, in this embodiment, the smoothing processing unit 23 smoothes the volume level. The volume level smoothing process is continuously performed every time the volume level is acquired in accordance with the volume level acquisition interval. The smoothing method is not particularly limited. For example, an average value of a volume level acquired at a certain time point and one or a plurality of volume levels acquired before that time point is calculated, and the sound volume level at that time point is calculated. A moving average method can be considered. A method other than moving average may be employed. The application rate determination processing unit 17 described above determines the application rate based on the volume level smoothed by the smoothing processing unit 23.

  FIG. 3 shows an example of the application rate. The application rate shown in FIG. 3 is determined by the application rate determination processing unit 17 based on the smoothed sound volume level after the speech waveform shown in the upper part is smoothed by the smoothing processing unit 23. As shown in FIG. 3, since the volume level is 0 before time T1, the application rate is also 0. After that, the utterance is started at time T1, and the volume level starts to increase rapidly, but the application rate increases gently (not sharply) due to the smoothing of the volume level, and becomes relatively large until time T2. Become. After that, the sound volume suddenly decreases at time T2, but the application rate decreases gently (not steeply) due to the smoothing of the sound volume level, and remains relatively small until time T3. Thereafter, when the utterance ends at time T3 and the volume level falls to 0, the application rate also becomes 0.

  In the above, the volume level is smoothed and the application rate is determined according to the smoothed volume level. However, for example, the application rate is determined without smoothing the volume level, and the determined application rate is determined. May be smoothed.

  Returning to FIG. 2, the motion pattern execution processing unit 19 executes the motion pattern in a size based on the application rate in accordance with the utterance of the character. Thereby, only the volume level of the voice uttered by the character can be input, and the magnitude of movement for opening and closing the mouth of the character can be changed.

  FIG. 4 shows an example of the magnitude of the opening / closing operation of the character's mouth based on the application rate. As shown in FIG. 4, the opening / closing operation of the character's mouth increases when the application rate is large (when the volume level is high), and decreases when the application rate is low (when the volume level is low). Note that although the size of the mouth opening / closing operation varies depending on the variation in the application rate, the repeated motion pattern is common, so even if the application rate varies, a series of mouth-shaped movements (corresponding to a predetermined audio sequence) The movement itself does not change.

  The motion pattern is repeatedly executed by the motion pattern execution processing unit 19. However, when the application rate varies while being simply repeated, for example, the application rate starts from 0 when the mouth shape starts to close in the motion pattern. There is a possibility that an unnatural motion such as starting to rise (that is, the mouth shape starts to close when the character starts speaking) may occur.

  Therefore, in the present embodiment, the reset processing unit 21 resets the start position of the operation pattern so that the operation pattern is executed from the beginning when the sound volume level starts to increase from 0 (including the case near 0). . As described above, since the motion pattern is generated so that the motion corresponding to the predetermined voice sequence is started from the state where the mouth shape is closed, the motion pattern of the motion pattern is started at the timing when the volume level starts to increase from 0 in this way. By resetting the start position, it is possible to express a natural mouth-shaped motion synchronized with the timing of the start of speaking of the character (start of utterance).

  Returning to FIG. 2, the behavioral synthesis processing unit 25 synthesizes the mouth-shaped motion pattern of the character executed by the motion pattern execution processing unit 19 and the facial expression motion of the character. As for the facial expression of the character, a separate motion pattern is set according to the emotion of the character. The behavioral synthesis processing unit 25 can synthesize the emotional expression based on the facial expression and the opening / closing operation of the mouth according to the voice by synthesizing the facial motion pattern and the mouth-shaped motion pattern.

  FIGS. 5 to 7 show examples in which facial expressions and mouth-shaped motion patterns are combined. In the example shown in FIG. 5, a mouth-shaped motion pattern is synthesized with a laughing face. In the example shown in FIG. 6, a mouth-shaped motion pattern is synthesized with the crying face. In the example shown in FIG. 7, a mouth-shaped motion pattern is synthesized with an angry face. Thereby, it becomes possible to combine the emotion expression by the facial expression of the character and the lip sync, and the reality of the character can be improved.

  Note that the application rate may be changed according to the facial expression to be combined, such as setting a higher application rate for a laughing face or an angry face and setting a lower application rate for a crying face.

  Note that the processing in each processing unit described above is not limited to the example of sharing of these processing, and may be processed by, for example, a smaller number of processing units (for example, one processing unit), Further, it may be processed by a further subdivided processing unit. The functions of each processing unit described above are implemented by a game program that is executed by a CPU 101 (see FIG. 9 described later), for example, a part of which is a dedicated integrated circuit such as an ASIC or FPGA, or the like. It may be implemented by an actual device such as an electric circuit.

<4. Processing procedure executed by information processing apparatus>
Next, an example of a processing procedure executed by the CPU 101 of the information processing apparatus 3 will be described with reference to FIG.

  In step S <b> 5, the information processing apparatus 3 uses the motion pattern acquisition processing unit 13 to acquire a mouth-shaped motion pattern of a character corresponding to a predetermined voice string. In addition, when a plurality of characters utter, an action pattern is acquired for each character.

  In step S <b> 10, the information processing apparatus 3 starts the acquisition of the volume level of the voice to be uttered by the character by the volume level acquisition processing unit 15. Thereafter, the acquisition of the volume level is continuously performed at predetermined time intervals.

  In step S <b> 15, the information processing apparatus 3 starts smoothing the volume level acquired in step S <b> 10 by the smoothing processing unit 23. Thereafter, the sound volume level smoothing process is continuously performed every time the sound volume level is acquired in accordance with the sound volume level acquisition interval.

  In step S <b> 20, the information processing apparatus 3 determines whether or not the volume level has started to increase from 0 (including the case in the vicinity of 0. The same applies hereinafter) by the reset processing unit 21. If the volume level remains 0 (step S20: NO), the process proceeds to step S25.

  In step S <b> 25, the information processing apparatus 3 determines the application rate to be 0 by the application rate determination processing unit 17.

  In step S30, the information processing apparatus 3 sets a flag F indicating whether or not the application rate is 0 to “0” indicating that the application rate is 0. Thereafter, the process proceeds to step S55 described later.

  On the other hand, if the volume level has increased from 0 in step S20 (step S20: YES), the process proceeds to step S35.

  In step S35, the information processing apparatus 3 causes the reset processing unit 21 to reset the start position of the operation pattern so that the operation pattern is executed from the beginning.

  In step S <b> 40, the information processing apparatus 3 determines the application rate according to the smoothed volume level by the application rate determination processing unit 17.

  In step S <b> 45, the information processing apparatus 3 determines whether the application rate has dropped to 0 by the application rate determination processing unit 17. This is synonymous with determining whether or not the sound volume level has dropped to 0 (including the case in the vicinity of 0). When the application rate drops to 0 (step S45: YES), the process proceeds to step S30 described above, and the flag F is set to “0”. On the other hand, when the application rate is not 0 (step S45: NO), the process proceeds to step S50.

  In step S50, the information processing apparatus 3 sets a flag F indicating whether or not the application rate is 0 to “1” indicating that the application rate is not 0.

  In step S55, the information processing apparatus 3 causes the motion pattern execution processing unit 19 to execute the motion pattern in accordance with the application rate determined in step S25 or step S40 according to the voice of the character.

  In step S60, the information processing apparatus 3 uses the behavioral synthesis processing unit 25 to synthesize the facial expression motion of the character and the character mouth shape motion pattern executed in step S55.

  In step S65, the information processing apparatus 3 determines whether or not the flag F described above is “0” indicating that the application rate is 0. If the flag F is not “0” (step S65: NO), the process returns to the previous step S40, and until the flag F becomes “0” (until the application rate becomes 0), steps S40 to S40. Repeat S65. On the other hand, when the flag F is “0” (step S65: YES), the process proceeds to step S70.

  In step S <b> 70, the information processing apparatus 3 determines whether or not to end the lip sync process, for example, when the movie in the game ends or when the game execution ends. If the lip sync process is not terminated (step S70: NO), the process returns to the previous step S20, and steps S20 to S30 and steps S55 to S70 are repeated until the volume level increases from 0. When the lip sync process is to be ended (step S70: YES), this flow is ended.

  In addition, the process procedure mentioned above is an example, Comprising: At least one part of the said procedure may be deleted or changed, and procedures other than the above may be added. In addition, the order of at least a part of the above procedure may be changed, and a plurality of procedures may be combined into a single procedure.

<5. Hardware configuration of information processing apparatus>
Next, an example of the hardware configuration of the information processing apparatus 3 that implements each processing unit implemented by the program executed by the CPU 101 described above will be described with reference to FIG.

  As shown in FIG. 9, the information processing apparatus 3 includes, for example, a CPU 101, a ROM 103, a RAM 105, a GPU 106, a dedicated integrated circuit 107 constructed for a specific application such as an ASIC or FPGA, and an input device 113. An output device 115, a recording device 117, a drive 119, a connection port 121, and a communication device 123. These components are connected to each other through a bus 109, an input / output interface 111, and the like so that signals can be transmitted to each other.

  For example, the game program can be recorded in the ROM 103, the RAM 105, the recording device 117, or the like.

  In addition, the game program is temporarily or permanently (non-temporarily) recorded on a removable recording medium 125 such as a magnetic disk such as a flexible disk, various CDs, MO disks, and optical disks such as a DVD, and semiconductor memory. You can also keep it. Such a recording medium 125 can also be provided as so-called package software. In this case, the game program recorded in these recording media 125 may be read by the drive 119 and recorded in the recording device 117 via the input / output interface 111, the bus 109, or the like.

  In addition, the game program can be recorded on, for example, a download site, another computer, another recording device, or the like (not shown). In this case, the game program is transferred via a network NW such as a LAN or the Internet, and the communication device 123 receives this program. The program received by the communication device 123 may be recorded in the recording device 117 via the input / output interface 111, the bus 109, or the like.

  Further, the game program can be recorded in an appropriate external connection device 127, for example. In this case, the game program may be transferred via an appropriate connection port 121 and recorded in the recording device 117 via the input / output interface 111, the bus 109, or the like.

  Then, the CPU 101 executes various processes according to the program recorded in the recording device 117, thereby realizing the processes by the operation pattern acquisition processing unit 13, the volume level acquisition processing unit 15, and the like. At this time, for example, the CPU 101 may directly read and execute the program from the recording device 117 or may be executed after being once loaded into the RAM 105. Further, for example, when receiving the program via the communication device 123, the drive 119, and the connection port 121, the CPU 101 may directly execute the received program without recording it in the recording device 117.

  Further, the CPU 101 may perform various processes based on signals and information input from the input device 113 including the controller 5 described above, such as a mouse, a keyboard, and a microphone (not shown), as necessary. .

  The GPU 106 performs processing for image display such as rendering processing in accordance with an instruction from the CPU 101.

  Then, the CPU 101 and the GPU 106 output the result of executing the above processing from the output device 115 including the display device 7 and the audio output unit described above, for example. Further, the CPU 101 and the GPU 106 may transmit the processing result via the communication device 123 or the connection port 121 as necessary, or may record the processing result in the recording device 117 or the recording medium 125.

<6. Effects of the embodiment>
The game program according to the present embodiment causes the information processing device 3 to acquire a motion pattern acquisition processing unit 13 that acquires a mouth-shaped motion pattern of a character corresponding to a predetermined sound sequence, and a volume that acquires a sound volume level to be uttered by the character. Level acquisition processing unit 15, application rate determination processing unit 17 that determines the degree of the size of the motion pattern applied to the character based on the volume level as an application rate, and the size based on the application rate of the motion pattern according to the character's utterance The function is executed as an operation pattern execution processing unit 19 to be executed.

  As described above, in the present embodiment, a mouth-shaped motion pattern of a character corresponding to a predetermined voice sequence is prepared in advance, and the magnitude of the motion is changed according to the volume level of the voice uttered by the character. Execute the operation pattern. This eliminates the need for vowel information and the like, for example, eliminates the need for complicated arithmetic processing for analyzing speech, and the processing load on the information processing apparatus 3 can be greatly reduced. As a result, since it becomes easy to apply to real-time voice input, it can be applied to, for example, chat, theater, online distribution, and the like. Also, in creating CG animation data, for example, the labor of manually inputting vowel information can be saved, so that the burden of creating animation data can be reduced.

  In the present embodiment, in particular, the motion pattern is generated so as to start the motion corresponding to the voice sequence from the state where the mouth shape of the character is closed, and the game program causes the information processing device 3 to have a volume level. When it starts to rise from 0 or near 0, it further functions as a reset processing unit 21 that resets the start position of the operation pattern so that the operation pattern is executed from the beginning.

  Thereby, for example, it is possible to avoid an unnatural motion such that the mouth shape starts to close at the timing when the character starts to speak. Therefore, it is possible to express a natural mouth-shaped motion synchronized with the timing of the character's start of speaking (start of utterance).

  In the present embodiment, in particular, the application rate determination processing unit 17 determines the application rate to 0 when the volume level drops to 0 or close to 0.

  Thus, the mouth shape can be closed at the timing when the character finishes speaking. Therefore, it is possible to express a natural mouth-shaped movement synchronized with the timing of the character's talking end (speech end).

  In the present embodiment, in particular, the game program causes the information processing apparatus 3 to further function as the smoothing processing unit 23 that smoothes the volume level acquired by the volume level acquisition processing unit 15, thereby applying the application rate determination process. The unit 17 determines the application rate based on the volume level smoothed by the smoothing processing unit 23.

  For example, when the application rate is determined according to the value of the volume level as it is, when the volume level changes abruptly, the size of the character's mouth shape changes abruptly, resulting in an unnatural action. End up.

  According to this embodiment, since the application rate is determined after smoothing the volume level as described above, an unnatural operation due to a sudden change in the volume level can be suppressed, and the user's uncomfortable feeling can be reduced.

  Particularly in the present embodiment, the game program causes the information processing apparatus 3 to synthesize a character mouth shape motion pattern executed by the motion pattern execution processing unit 19 and a character facial expression motion. 25, further function.

  As a result, it is possible to combine emotional expressions such as smiles, crying faces, angry faces, etc. of the character with lip sync, and improve the reality of the character.

<7. Modified example>
The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit and technical idea of the present invention.

  As described above, the present invention can significantly reduce the processing load of the information processing apparatus 3, and therefore is suitable for real-time voice input. For example, as shown in FIG. 10, a voice input device 27 such as a microphone may be connected to the information processing device 3, and the voice of the player may be input in real time to utter a character appearing in a game or the like. In this case, the volume level acquisition processing unit 15 acquires the volume level based on the volume of the player's voice input from the voice input device 27. Other processes are the same as in the above embodiment.

  In the above description, the case where the lip sync processing program of the present invention is applied to a game has been described as an example. However, the application target of the present invention is not limited to a game. For example, the present invention can be applied to various video contents such as a TV program and a movie, and can also be applied to chat, 3D theater, online distribution, and the like.

  In addition to those already described above, the methods according to the above-described embodiments and modifications may be used in appropriate combination. In addition, although not illustrated one by one, the above-mentioned embodiment and each modification are implemented with various modifications within a range not departing from the gist thereof.

DESCRIPTION OF SYMBOLS 3 Information processing apparatus 13 Action pattern acquisition process part 15 Volume level acquisition process part 17 Application rate determination process part 19 Action pattern execution process part 21 Reset process part 23 Smoothing process part 25 Action synthesis process part 125 Recording medium

Claims (7)

Information processing device
An action pattern acquisition processing unit for acquiring an action pattern of the mouth shape of the object corresponding to a predetermined voice sequence;
A volume level acquisition processing unit for acquiring a volume level of sound to be uttered by the object;
An application rate determination processing unit that determines, as an application rate, a degree of the size of the motion pattern to be applied to the object based on the volume level;
An action pattern execution processing unit for executing the action pattern in a size based on the application rate in accordance with the utterance of the object;
Lip sync processing program to function as The operation pattern is
It is generated so as to start an operation corresponding to the voice sequence from a state in which the mouth shape of the object is closed,
The information processing apparatus;
A reset processing unit that resets the start position of the operation pattern so that the operation pattern is executed from the beginning when the volume level starts to increase from 0 or near 0;
To further function as,
The lip sync processing program according to claim 1. The application rate determination processing unit
When the volume level drops to 0 or near 0, the application rate is determined to be 0;
The lip sync processing program according to claim 1 or 2. The information processing apparatus;
A smoothing processing unit for smoothing the volume level acquired by the volume level acquisition processing unit;
Further function as
The application rate determination processing unit
Determining the application rate based on the volume level smoothed by the smoothing processing unit;
The lip sync processing program according to any one of claims 1 to 3. The information processing apparatus;
A behavioral synthesis processing unit that synthesizes the motion pattern of the mouth shape of the object executed by the motion pattern execution processing unit and the motion of the facial expression of the object;
To further function as,
The lip sync processing program according to any one of claims 1 to 4.   6. A recording medium readable by an information processing apparatus, on which the lip sync processing program according to claim 1 is recorded. A lip sync processing method executed by an information processing apparatus,
Obtaining a movement pattern of the mouth shape of the object corresponding to a predetermined voice sequence;
Obtaining a volume level of sound to be uttered by the object;
Determining, as an application rate, a degree of the size of the motion pattern to be applied to the object based on the volume level;
Executing the motion pattern in a size based on the application rate according to the utterance of the object;
A lip sync processing method.

Images