JP2012050791A - Character display device, character display method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a character display device and the like suitable for a player to enjoy conversation while visually attracting the user by controlling characters to move according to user's voice.SOLUTION: In the character display device 200, a storage part 201 stores a direction of a character to be disposed in a virtual space; a display part 202 displays on a screen an image where the character is disposed in the virtual space in the stored direction; a detecting part 203 detects voice from the user; a determination part 204 determines a change in direction of the character based on the volume of the detected voice; and an updating part 205 updates the stored direction by the determined change.

Description

  The present invention relates to a character display device, a character display method, and a program suitable for enjoying a conversation comfortably while appealing to the user's vision by causing a character to perform an action according to the user's voice.

  Traditionally, in the fields of games, chats, social network services, etc., users select and set a character (sometimes called an “avatar”) in the virtual world onto which they are projected. The method of making a user feel familiar by displaying on other terminals and other people's terminals is widely used.

  In this case, a method has been developed to change the character's shape, posture, and facial expression, making it easier for the user to recognize that the character is displayed there, and at the same time giving the user a sense of realism and familiarity. Yes.

  For example, Patent Document 1 captures a minute change of a mouse or a controller that appears in an unconscious movement of a user, that is, a situation where the user is nervous or excited, and reflects it in the movement of the character. Thus, a technique is disclosed in which a user and a character are closely linked to give the character an impression of being alive.

JP 2008-233946 A

  On the other hand, scenes where avatars such as those described above are used are often scenes where a plurality of users who are separated from each other have a conversation on the network in a network game, web chat, or the like. That is, by displaying an avatar on a game monitor or the like, friendliness can be added to the conversation between users.

  When users have a conversation with each other using such an avatar, there is a demand for the avatar to take an action corresponding to the voice when the user has a conversation. In other words, conversations between users at remote locations are usually performed only by voice or characters, but appealing to the user's vision by letting the avatar take actions according to the actions that the user speaks. However, it will be possible to have more realistic conversations.

  SUMMARY OF THE INVENTION The present invention solves the above-described problems. A character display device and character suitable for comfortably enjoying conversation while appealing to the user's vision by causing the character to perform an action according to the user's voice. It is an object to provide a display method and a program.

  In order to achieve the above object, a character display device according to a first aspect of the present invention includes a storage unit, a display unit, a detection unit, a determination unit, and an update unit.

  In the storage unit, the direction of the character arranged in the virtual space is stored.

  That is, for example, a three-dimensional virtual space is prepared, and characters are arranged at predetermined positions in the virtual space. At that time, the characters are arranged in, for example, a front direction, a 30 ° direction from the front to the left, and a 45 ° direction from the front to the right, and the direction is stored in the storage unit.

  The display unit displays an image representing a state in which the character is arranged in the virtual space in the stored direction on the screen.

  That is, if the orientation stored in the storage unit is leftward, an image representing the state of the character placed leftward in the virtual space is generated and displayed on a screen such as a monitor. Similarly, if the orientation stored in the storage unit is front-facing, an image showing the state of the character placed in front in the virtual space is displayed. If the stored orientation is right-facing, the image is directed right in the virtual space. An image representing the state of the character placed in is displayed on the screen.

  The detection unit detects voice from the user.

  That is, a voice or sound uttered by a user is captured using a device that converts sound such as a microphone into an electrical signal. Here, a predetermined threshold volume may be provided so as not to react to voices from other than the user and other small noises, and detection may be performed only when the captured sound exceeds the threshold volume.

  The determining unit determines a change in the direction of the character based on the detected sound volume.

  That is, the sound volume averaged at a predetermined sampling period is acquired from the sound detected by the detection unit. Then, a change in the direction of the character is determined from the acquired volume based on a predetermined conversion formula. Here, the “change in direction” is the amount of change in which the direction of the character arranged in the virtual space changes. For example, when a character changes from a state in which the character is arranged in the 30 ° direction to the left from the front in the virtual space to a state in which the character is arranged in the 45 ° direction to the left, the “change in direction” That is, “15 ° to the left”. Such a change in direction is typically set to a larger amount as the detected voice is larger.

  The update unit updates the stored orientation according to the determined change.

  That is, the direction of the character in the virtual space stored in the storage unit is updated based on the change determined by the determination unit. For example, if the determined change is a change in the direction of 90 ° from the front to the left, the character arranged in the direction of 90 ° from the front to the left is newly stored in the storage unit. If the determined change is a change in the direction of 30 ° from the front to the right, the character arranged in the direction of 30 ° from the front to the right is newly stored in the storage unit.

  With such a configuration, the direction of the displayed character changes based on the voice uttered by the user. As a result, the user can enjoy a conversation with a sense of familiarity and presence while viewing an image of a character that reacts to voice.

In the character display device of the present invention,
The detector detects the sound with a monaural microphone,
The range that the character can take is between the front direction and the horizontal direction of either the left or right angle,
The determination unit determines the change such that the larger the volume of the detected voice is, the closer the character's direction is to the front.
It is also possible.

  Here, the monaural microphone means a microphone that cannot or cannot determine whether the sound is detected from the left or right direction. That is, the monaural microphone here has, for example, a voice detection part at one position, and does not distinguish between the voice detection from the left direction and the voice detection from the right direction. It is.

  At this time, for example, consider a case where the range that the character can take is predetermined between the front direction and the horizontal direction 30 ° left from the front. In this case, when no sound is detected, the character is arranged in the horizontal direction 30 ° left from the front in the virtual space, and when the sound starts to be detected by the monaural microphone, the character direction is changed from the horizontal direction 30 ° to the left. Gradually begin to change to the front. Then, as the detected voice increases, the direction changes from the horizontal direction of 30 ° left to the direction of 20 ° left, 10 ° left, and 0 ° left (that is, front).

In the character display device of the present invention,
The detection unit detects the sound with a stereo microphone,
The determination unit determines a change in the direction of the character based on a difference between left and right sound volumes detected by the stereo microphone.
It is also possible.

  Here, the stereo microphone means a microphone that can determine whether the sound is detected from the left or right direction. That is, the stereo microphone here has, for example, a part for detecting sound independently at two positions on the left and right, or a part for detecting sound from the left and right directions even at one position. By doing so, it is possible to distinguish between the sound detected by the left part (microphone) and the sound detected by the right part (microphone).

  At this time, for example, if the user speaks directly to the stereo microphone, the sound of the same volume is detected by the left and right microphones of the stereo microphone. Also, if the user emits sound from the left side of the stereo microphone, the volume of the left microphone of the stereo microphone will be higher than that of the right microphone, while if the user emits sound from the right side of the stereo microphone. The volume of the right microphone of the stereo microphone is larger than the volume of the left microphone. Here, the determining unit determines the change in the direction of the character based on the difference between the left and right volume.

In the character display device of the present invention,
The range that the character's orientation can take is between the horizontal orientation of the left and right angles including the front orientation,
The decision part
(A) The smaller the difference between the left and right volume of the sound detected by the stereo microphone, the closer the character is to the front,
(B) The higher the left volume of the sound detected by the stereo microphone is, the closer the character is to the left side at a predetermined angle,
(C) The larger the right volume of the sound detected by the stereo microphone is, the closer the character's orientation is to the right side at a predetermined angle.
To determine the change,
It is also possible.

That is,
(A) The fact that the difference between the left and right sound volumes detected by the stereo microphone is small means that the user is at an equal distance from the left and right microphones of the stereo microphone, that is, the user is facing the stereo microphone. Usually considered. Therefore, the character display device of the present invention simulates the positional relationship between the user and the stereo microphone, and brings the character direction closer to the front direction.
On the other hand,
(B) The left volume of the sound detected by the stereo microphone is larger than the right volume, which means that the user is close to the left microphone of the stereo microphone, that is, the user speaks from the left side of the stereo microphone. It is usually considered that the situation is. Therefore, the character display device of the present invention simulates the positional relationship between the user and the stereo microphone, and turns the character sideways at a predetermined angle on the left side, for example, sideways at an angle such as 45 ° left or 30 ° left. Move closer to.
Furthermore,
(C) The right volume of the sound detected by the stereo microphone is larger than the left volume, which means that the user is close to the right microphone of the stereo microphone, that is, the user speaks from the right side of the stereo microphone. It is usually considered that the situation is. Therefore, the character display device of the present invention simulates the positional relationship between such a user and a stereo microphone, and turns the character sideways at a predetermined angle on the right, for example, sideways at an angle such as 45 ° to the right or 30 ° to the right. Move closer to.

In the character display device of the present invention,
The decision part
(P) determining a left-right orientation change of the character based on a first volume averaged from a detected voice by a predetermined first section length;
(Q) determining a change in the vertical direction of the character based on a second sound volume averaged from the detected voice by a predetermined second section length;
It is also possible.

That is, when acquiring the sound volume from the sound detected by the detection unit, the determination unit averages two different section lengths, that is, sampling periods, and acquires two types of sound volume from each. For example, consider a case where the predetermined second section length is set to a longer section length than the predetermined first section length. At this time,
(P) When the user's voice contains many high frequency components, that is, when the user's voice contains many high-frequency components, the first volume increases. Then, based on the first volume, a change in the left / right orientation of the character is determined. For example, the larger the first volume, the larger the left / right direction of the character is determined to change.
On the other hand,
(Q) When the user's voice contains many low frequency components, that is, when the user's voice contains many low frequency components, the second volume increases. Then, based on the second volume, a change in the vertical direction of the character is determined. For example, the larger the second volume is, the larger the vertical direction of the character is determined to change.

  With such a configuration, the character display device of the present invention can change two different directions of characters in the screen from one voice uttered by one user. As a result, more complex actions can be taken by the characters in the screen, and the presence and familiarity can be enhanced.

In the character display device of the present invention,
The update unit updates the stored orientation while keeping the character's line of sight toward the front.
It is also possible.

  In other words, in a situation where the character's orientation changes and a character that faces sideways instead of facing the front is displayed on the screen, the user who looks at the screen does not feel that the character is speaking to the user It is thought that things will increase. Therefore, the character display device of the present invention always keeps the character's line of sight toward the front even when the character is turned sideways. As a result, it is possible to display a screen that speaks to the user even on the character facing sideways.

In the character display device of the present invention,
The update unit updates the stored orientation so that the angular acceleration does not exceed a predetermined threshold.
It is also possible.

  In other words, if a character placed in a fixed orientation in the virtual space changes suddenly when it is changed to a different orientation by the update unit, it will be unnatural for the player viewing the generated character display screen. It is likely to be uncomfortable. In order to avoid this, a predetermined threshold value is provided for the angular acceleration when the direction changes, and the direction can be changed smoothly so that the user can comfortably view the character display screen.

  In order to achieve the above object, a character display method according to a second aspect of the present invention is a character display method executed by a character display device including a storage unit, a display unit, a detection unit, a determination unit, and an update unit. A display process, a detection process, a determination process, and an update process.

The storage unit stores the orientations of characters arranged in the virtual space.
In the display step, the display unit displays an image representing a state in which the character is arranged in the virtual space in the stored direction on the screen.
In the detection step, the detection unit detects voice from the user.
In the determining step, the determining unit determines a change in the character direction based on the detected sound volume.
In the update process, the update unit updates the stored orientation according to the determined change.

  In order to achieve the above object, a program according to a third aspect of the present invention is configured to cause a computer to function as the above character display device and to cause the computer to execute each step of the above character display method.

  The program of the present invention can be recorded on a computer-readable information storage medium such as a compact disk, flexible disk, hard disk, magneto-optical disk, digital video disk, magnetic tape, and semiconductor memory.

  The program can be distributed and sold via a computer communication network independently of the computer on which the program is executed. The information storage medium can be distributed and sold independently from the computer.

  According to the present invention, a character display device, a character display method, and a program suitable for enjoying a conversation comfortably while appealing to the user's vision by causing the character to perform an operation according to the user's voice are provided. be able to.

It is a mimetic diagram showing an outline composition of a typical information processor with which a character display concerning an embodiment of the present invention is realized. It is a schematic diagram which shows the general | schematic structure which concerns on the character display apparatus of this invention. It is the figure which showed the structural example of the display screen of a character display apparatus. It is a figure showing the mode in the virtual space of a character display apparatus. It is a figure showing a mode that the direction of a character image changes based on the audio | voice from a user. It is a figure showing a mode that the direction of the virtual character in virtual space changes. 4 is a graph showing a relationship between a detected sound volume and an angle for determining the direction of a character in the first embodiment. 3 is a flowchart illustrating a flow of processing according to the character display device of the first embodiment. (A) is a figure showing a mode that a user is uttering from a different direction toward a monaural microphone. (B) is a figure showing a mode that a user is uttering from a different direction toward a stereo microphone. (A), (b) is the figure which showed the mode of the virtual space and a display screen when an audio | voice is detected from the left direction in the character display apparatus of Embodiment 2. FIG. (A), (b) is the figure which showed the mode of the virtual space and a display screen when an audio | voice is detected from the right direction in the character display apparatus of Embodiment 2. FIG. 10 is a graph showing a relationship between a detected sound volume and an angle for determining a character direction in the second embodiment. 10 is a flowchart illustrating a flow of processing according to the character display device of the second embodiment. (A), (b) is a figure showing signs that a character image changes based on a voice from a user in Embodiment 3. (A) is the figure which showed the relationship between the 1st volume in Embodiment 3, and the angle which determines the left-right direction of a character. (B) is the figure which showed the relationship between the 2nd volume in Embodiment 3, and the angle which determines the up-and-down direction of a character. 14 is a flowchart illustrating a flow of processing according to the character display device of the third embodiment.

  Embodiments of the present invention will be described below. In the following, for ease of understanding, an embodiment in which the present invention is realized using a game information processing device will be described. However, the embodiment described below is for explanation, and the present invention is described. It does not limit the range. Therefore, those skilled in the art can employ embodiments in which these elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention.

(Embodiment 1)
FIG. 1 is a schematic diagram showing a schematic configuration of a typical information processing device in which a character display device according to an embodiment of the present invention is realized. Hereinafter, a description will be given with reference to FIG.

  The information processing apparatus 100 includes a CPU (Central Processing Unit) 101, a ROM 102, a RAM (Random Access Memory) 103, an interface 104, a controller 105, an external memory 106, an image processing unit 107, and a DVD-ROM. (Digital Versatile Disc ROM) drive 108, NIC (Network Interface Card) 109, audio processing unit 110, and microphone 111.

  A DVD-ROM storing a game program and data is loaded into the DVD-ROM drive 108, and the information processing apparatus 100 is turned on to execute the program. Realized.

  The CPU 101 controls the overall operation of the information processing apparatus 100 and is connected to each component to exchange control signals and data. Further, the CPU 101 uses arithmetic operations such as addition / subtraction / multiplication / division, logical sum, logical product, etc. using an ALU (Arithmetic Logic Unit) (not shown) for a storage area called a register (not shown) that can be accessed at high speed. , Logic operations such as logical negation, bit operations such as bit sum, bit product, bit inversion, bit shift, and bit rotation can be performed. In addition, the CPU 101 itself is configured so that saturation operations such as addition / subtraction / multiplication / division for multimedia processing, vector operations such as trigonometric functions, etc. can be performed at a high speed, and those provided with a coprocessor. There is.

  The ROM 102 records an IPL (Initial Program Loader) that is executed immediately after the power is turned on, and when this is executed, the program recorded on the DVD-ROM is read out to the RAM 103 and execution by the CPU 101 is started. The The ROM 102 stores an operating system program and various data necessary for operation control of the entire information processing apparatus 100.

  The RAM 103 is for temporarily storing data and programs, and holds programs and data read from the DVD-ROM and other data necessary for game progress and chat communication. In addition, the CPU 101 provides a variable area in the RAM 103, performs an operation by directly operating the ALU on the value stored in the variable, and temporarily stores the value stored in the RAM 103 in a register. Performs an operation on the register and writes the result back to the memory.

  The controller 105 connected via the interface 104 receives an operation input performed when the user executes the game.

  The external memory 106 detachably connected via the interface 104 stores data indicating game play status (past results, etc.), data indicating the progress of the game, and log of chat communication in a network battle ( Data) is stored in a rewritable manner. The user can record these data in the external memory 106 as appropriate by inputting an instruction via the controller 105.

  A DVD-ROM mounted on the DVD-ROM drive 108 stores a program for realizing the game and image data and audio data associated with the game. Under the control of the CPU 101, the DVD-ROM drive 108 performs a reading process on the DVD-ROM loaded therein, reads out necessary programs and data, and these are temporarily stored in the RAM 103 or the like.

  The image processing unit 107 processes the data read from the DVD-ROM by an image arithmetic processor (not shown) included in the CPU 101 or the image processing unit 107, and then processes the processed data on a frame memory ( (Not shown). The image information recorded in the frame memory is converted into a video signal at a predetermined synchronization timing and output to a monitor (not shown) connected to the image processing unit 107. Thereby, various image displays are possible.

  The image calculation processor can execute a two-dimensional image overlay calculation, a transmission calculation such as α blending, and various saturation calculations at high speed.

  Also, polygon information arranged in the virtual three-dimensional space and added with various texture information is rendered by the Z buffer method, and the polygon arranged in the virtual three-dimensional space from the predetermined viewpoint position is determined in the direction of the predetermined line of sight It is also possible to perform high-speed execution of operations for obtaining rendered images.

  Further, the CPU 101 and the image arithmetic processor cooperate to draw a character string as a two-dimensional image in the frame memory or draw it on the surface of each polygon according to the font information that defines the character shape. Is possible. The font information is recorded in the ROM 102, but it is also possible to use dedicated font information recorded in the DVD-ROM.

  The NIC 109 is used to connect the information processing apparatus 100 to a computer communication network (not shown) such as the Internet, and is based on the 10BASE-T / 100BASE-T standard used when configuring a LAN (Local Area Network). To connect to the Internet using an analog modem, ISDN (Integrated Services Digital Network) modem, ADSL (Asymmetric Digital Subscriber Line) modem, cable television line A cable modem or the like and an interface (not shown) that mediates between these and the CPU 101 are configured.

  The audio processing unit 110 converts audio data read from the DVD-ROM into an analog audio signal and outputs the analog audio signal from a speaker (not shown) connected thereto. Further, under the control of the CPU 101, sound effects and music data to be generated during the progress of the game are generated, and sound corresponding to this is output from the speaker.

  When the audio data recorded on the DVD-ROM is MIDI data, the audio processing unit 110 refers to the sound source data included in the audio data and converts the MIDI data into PCM data. If the compressed audio data is in ADPCM format or Ogg Vorbis format, it is expanded and converted to PCM data. The PCM data can be output by performing D / A (Digital / Analog) conversion at a timing corresponding to the sampling frequency and outputting it to a speaker.

  Furthermore, a microphone 111 can be connected to the information processing apparatus 100 via the interface 104. In this case, the analog signal from the microphone 111 is subjected to A / D conversion at an appropriate sampling frequency so that processing such as mixing in the sound processing unit 110 can be performed as a PCM format digital signal.

  When the information processing apparatus 100 is used as a karaoke apparatus, audio data read from a DVD-ROM or audio data acquired from a computer communication network via the NIC 109 is used as accompaniment data, and audio input from the microphone 111 is used. The voice processing unit 110 mixes the accompaniment data and the singing data with the data as the singing data, and outputs it from the speaker. Moreover, it is possible to output sound using headphones (not shown) or earphones (not shown) instead of the speakers.

  In addition, the information processing apparatus 100 uses a large-capacity external storage device such as a hard disk so as to perform the same function as the ROM 102, the RAM 103, the external memory 106, the DVD-ROM mounted on the DVD-ROM drive 108, and the like. You may comprise.

  Further, it is possible to adopt a form in which a keyboard for accepting a character string editing input from a user, a mouse for accepting various position designations and selection inputs, and the like are connected. In addition, a general-purpose personal computer can be used instead of the information processing apparatus 100 of the present embodiment.

  The information processing apparatus 100 described above corresponds to a so-called “consumer video game apparatus”, but the present invention can be realized as long as it performs image processing to display a virtual space. Therefore, the present invention can be realized on various computers such as a mobile phone, a portable game device, a karaoke apparatus, and a general business computer.

  For example, a general computer includes an CPU, a RAM, a ROM, a DVD-ROM drive, and an NIC as in the information processing apparatus 100, and an image processing unit having a simpler function than the information processing apparatus 100. In addition to having a hard disk as an external storage device, a flexible disk, a magneto-optical disk, a magnetic tape, and the like can be used. Further, not the controller 105 but a keyboard or a mouse is used as an input device.

  FIG. 2 is a schematic diagram showing a schematic configuration according to the character display device of the present invention. The character display device 200 of the present invention is typically realized by the information processing device 100 described above. Hereinafter, the general configuration will be described with reference to this figure.

  The character display device 200 includes a storage unit 201, a display unit 202, a detection unit 203, a determination unit 204, and an update unit 205.

  In the storage unit 201, the direction of the character arranged in the virtual space is stored.

  That is, in this embodiment, a three-dimensional virtual space is prepared, and characters are arranged there in a predetermined direction. The predetermined orientation is updated by the update unit 205 described later, and the character arranged in the updated orientation is stored in the storage unit 201 each time. For example, the RAM 103 or the like functions as such a storage unit 201.

  The display unit 202 displays an image representing a state in which the character is arranged in the virtual space in the stored direction on the screen.

  That is, an image obtained by viewing a character arranged in the virtual space in the orientation stored in the storage unit 201 from a virtual camera arranged in a predetermined position in the virtual space is generated, and the image is displayed on the character display device 200. On the screen. For example, the image processing unit 107 functions as such a display unit 202.

  The detection unit 203 detects voice from the user.

  That is, a voice uttered by the user is accepted as an input, and the accepted voice is supplied to the determination unit 204. For example, the microphone 111 connected to the interface 104 functions as such a detection unit 203.

  The determination unit 204 determines the change in the character direction based on the detected sound volume.

  That is, the sound volume is acquired from the sound detected by the detection unit 203 from the user at a predetermined sampling frequency. Then, based on the acquired volume, an amount for changing the direction of the character in the virtual space is determined. For example, the CPU 101 functions as such a determination unit 204 in cooperation with the RAM 103 or the like.

  The update unit 205 updates the stored orientation according to the determined change.

  In other words, the change determined by the determination unit 204 is received, and the direction of the character stored in the storage unit 201 is updated. For example, the CPU 101 functions as such an update unit 205 in cooperation with the RAM 103 or the like.

  FIG. 3 is a diagram showing a configuration example of a display screen according to the character display device 200 of the present invention. The display unit 202 typically displays a display screen 300 as shown in the figure on the monitor through the image display unit 107 of the information processing apparatus 100.

  As shown in this figure, the display screen 300 includes a character image 301 and an image of a message window 302. If necessary, other background images (in this figure, the character image 301 stands up) constituting the virtual world. And images of houses arranged along the road). For the generation of these images, it is possible to apply the same graphics technique as that used in ordinary games and chat systems.

  The character image 301 is a so-called alternation of users in the virtual world. When users in remote locations communicate with each other on the network, the character image 301 is displayed on the display screen 300 to display the other user. It is intended to make people feel friendly and to be able to communicate smoothly. In this figure, only one character image 301 is displayed on the display screen 300. However, when a large number of users are communicating on the network, a large number of character images corresponding to each of the large number of users are displayed. 301 may be arranged in the display screen 300.

  The message window 302 displays a message when the users communicate with each other on the network in characters. The displayed message may be generated by analyzing the user's voice detected by the detection unit 203 and converting it into characters, or the user's voice detected by the detection unit 203 Alternatively, it may be a message input by the user using an input device such as a keyboard. In the present embodiment, the message window 302 may not necessarily be displayed, and the users may communicate with each other only by the operation of the voice and the character image 301.

  In order to generate such a character image 301 on the display screen 300, the character display device 200 of this embodiment prepares a three-dimensional virtual space in the storage unit 201, and a virtual character and a virtual camera in the virtual space. Is generated, and an image representing the state of the virtual character photographed from the virtual camera is generated and displayed in the display screen 300. Hereinafter, a description will be given with reference to FIG.

  FIG. 4 is a diagram showing a state in the virtual space according to the character display device 200 of the present invention. This figure shows a state of the virtual space 400 viewed from above the virtual character 401 arranged in the three-dimensional virtual space 400. A virtual camera 402 is further arranged in the virtual space 400 and is directed toward the virtual character 401.

  The virtual camera 402 is used to generate the character image 301 displayed on the display screen 300 in FIG. 3, and the display unit 202 is based on the state of the virtual character 401 viewed from the virtual camera 402. An image is generated and displayed on the display screen 300 in FIG. That is, for example, if the virtual character 401 in the virtual space 400 is arranged to face the virtual camera 402, the virtual camera 402 generates a front-facing character image 301 and displays it on the display screen 300. To do. If the virtual character 401 in the virtual space 400 is arranged so as to face left with respect to the virtual camera 402, the virtual camera 402 generates a character image 301 facing left and displays it on the display screen 300. . Further, if the virtual character 401 in the virtual space 400 is arranged so as to face right with respect to the virtual camera 402, the virtual camera 402 generates a character image 301 facing right and displays it on the display screen 300. .

  Specifically, in FIG. 4, the virtual character 401 is arranged with the direction of the virtual camera 402 being 0 ° (front direction) and facing leftward from the virtual camera 402 toward the virtual character 401 by a predetermined angle 403. . In other words, the virtual camera 402 projects the virtual character 401 to the left, and the character image 301 generated based on the virtual character 401 is viewed by the user viewing the display screen 300 as shown in FIG. It will be in the state which has turned to the left direction toward from.

  In this embodiment, the character image 301 in the display screen 300 is generated based on the state of the virtual character 401 in various directions in the virtual space 400 as described above. The orientation of the virtual character 401 changes when the detection unit 203 detects voice from the user. Hereinafter, a description will be given with reference to FIGS. 5 and 6.

  FIG. 5 is a diagram showing how the orientation of the character image 301 changes based on the voice from the user. In the display screen 300 of this figure, a character image 301, a message window 302, and other background images are displayed as in FIG. Furthermore, in this figure, a monaural microphone 502 is connected to the display unit 202 of the character display device 200 displaying such a display screen 300, and a user 501 who is viewing the display screen 300 can view the monaural microphone 502. The situation where he says “Hello !!” is shown.

  That is, here, the detection unit 203 of the character display device 200 detects the sound from the user 501 with the monaural microphone 502. Note that the monaural microphone 502 in FIG. 5 is written as being connected to the display unit 202 by wire, but is not limited thereto, and may be connected by wireless communication.

  FIG. 5 shows that the orientation of the character image 301 displayed in the display screen 300 is changed from the left direction to the front direction as the user 501 speaks to the monaural microphone 502. Yes. That is, while the character image 301 is facing left in FIG. 3, the character image 301 changes so as to face the front with respect to the user 501 who is viewing the display screen 300 in this figure.

  In the message window 302 of FIG. 5, the conversation content 304 “Hello !!” is displayed based on the voice “Hello !!” uttered by the user 501. Such conversation content 304 is displayed by, for example, analyzing voice spoken by the user 501 toward the monaural microphone 502 and converting it into characters.

  Furthermore, FIG. 6 is a diagram illustrating a state in which the orientation of the virtual character 401 in the virtual space 400 changes. This figure represents the state of the virtual space 400 viewed from above the virtual character 401 arranged in the three-dimensional virtual space 400, as in FIG. However, the virtual character 401 is turned to the left by a predetermined angle 403 with respect to the virtual camera 402 in FIG.

  That is, FIG. 6 shows that the voice of the user 501 as shown in FIG. 5 is detected by the detection unit 203, that is, the monaural microphone 502, and the orientation of the virtual character 401 in the virtual space 400 changes to the front. It shows how to do. Then, the virtual camera 402 displays the state of the virtual character 401 that has been changed to the front direction as described above, and the display unit 202 generates a front-side character image 301 as shown in FIG. To do.

  As described above, in the present embodiment, in a state in which no sound is detected by the monaural microphone 502, the virtual character 401 is arranged sideways at a predetermined angle 403 with respect to the virtual camera 402 as shown in FIG. The character image 301 facing left is displayed in the display screen 300 as shown in FIG. On the other hand, when sound is detected by the monaural microphone 502, the virtual character 401 changes to face the virtual camera 402 as shown in FIG. A character image 301 facing the front is displayed as shown in FIG.

  Here, in the present embodiment, the orientations of the virtual character 401 and the character image 301 do not change at random, but change based on the sound volume detected by the monaural microphone 502. Specifically, in the present embodiment, the range in which the orientation of the virtual character 401 in the virtual space 400 can be taken is between the front orientation with respect to the virtual camera 402 and the lateral orientation of the predetermined angle 403 from the front to the left. To do. Then, the detection unit 203 of the character display device 200 detects the voice from the user 501 with the monaural microphone 502, and the determination unit 204 increases the volume of the voice detected by the detection unit 203 as the volume of the virtual character 401 increases. The change is determined so that the direction approaches the front direction, that is, the direction of the character image 301 displayed in the display screen 300 approaches the front direction. The relationship between the sound volume and the direction will be described in detail with reference to FIG. 7 below.

  FIG. 7 is a graph showing the relationship between the volume of the sound detected by the detection unit 203 and the angle that determines the direction of the virtual character 401 in this embodiment. In this figure, the abscissa indicates the detected sound volume, the ordinate indicates the angle that determines the orientation of the virtual character 401, the detected volume increases toward the right, and the orientation of the virtual character 401 increases toward the left. It represents that.

  In the graph of this figure, when the volume of the detected voice is 0, the angle that determines the orientation of the virtual character 401 is a predetermined leftward predetermined angle 403. Then, as the volume of the detected voice gradually increases, the angle that determines the direction of the virtual character 401 gradually approaches the front direction. When the detected sound volume reaches a predetermined threshold volume, the angle reaches 0 °, and the virtual character 401 faces the virtual camera 402 directly in front. In the graph of this figure, even when a sound having a sound volume equal to or higher than a predetermined threshold sound volume is detected, it remains in the front direction.

  That is, in this embodiment, if the user 501 detects no sound, the character image 301 is displayed facing left as shown in FIG. 3, but the user 501 speaks to the monaural microphone 502. When emitted, the character image 301 gradually changes its orientation. Then, the greater the volume of the detected voice, the closer the orientation of the character image 301 is to the front. Finally, when the detected volume reaches a predetermined threshold volume, the character image 301 turns to the front as shown in FIG.

  In FIGS. 3 to 7, the virtual character 401 is facing the predetermined angle 403 facing left when no sound is detected. However, the present embodiment is not limited to this. For example, the virtual character 401 is facing right. Even if it exists, it may be left diagonally downward.

  In the present embodiment, when the orientation of the virtual character 401 is changed, the line of sight of the character image 301 in the display screen 300 is changed while facing the front. That is, the update unit 205 updates the orientation stored in the storage unit 201 while keeping the line of sight of the character image 301 facing the front.

  Specifically, a virtual pupil 404 is separately prepared in the virtual character 401 in the virtual space 400 in FIGS. 4 and 6. Then, even if the updating unit 205 updates the direction of the virtual character 401, the position is adjusted so that only the virtual pupil 404 is always facing the direction of the virtual camera 402. Specifically, in FIG. 4, the virtual character 401 is directed leftward by a predetermined angle 403 with respect to the virtual camera 402, and in FIG. 6, the virtual character 401 is directly facing the virtual camera 402. The virtual pupil 404 also faces the virtual camera 402 in FIG.

  An image generated by the virtual camera 402 looking at the virtual character 401 and the virtual pupil 404 is displayed in the display screen 300 of FIGS. 3 and 5. Specifically, in FIG. 3, the character image 301 is in a state in which the entire body faces leftward, but only the pupil image 303 faces front. That is, the line of sight of the character image 301 faces the front. On the other hand, in FIG. 5, the character image 301 has the whole body facing the front, and the pupil image 303 is also facing the front.

  As described above, in the present embodiment, by adjusting the position of the virtual pupil 404, the pupil image 303 always faces the front regardless of the orientation of the entire body of the character image 301. As a result, when users in remote locations are having a conversation while looking at the character image 301 corresponding to the other user, even if the character image 301 is facing sideways, the other party ignores himself / herself. Instead of facing in the other direction, you can make it look like you are talking to yourself, and you can increase the sense of presence in the conversation.

  In the present embodiment, when the orientation of the virtual character 401 is changed, the angular acceleration is changed so as not to exceed a predetermined threshold. That is, the update unit 205 updates the orientation stored in the storage unit 201 so that the angular acceleration does not exceed a predetermined threshold.

  In other words, the focus is particularly given here when the orientation of the virtual character 401 that has been fixed in a certain direction begins to change or when the orientation ends. That is, the direction of the virtual character 401 suddenly starts to move at a large angular velocity or suddenly stops, and the direction of the character 301 on the display screen 300 is more smoothly changed when the direction changes with a so-called viscous movement. It is thought that it feels more natural to the user who is. Therefore, in this embodiment, a predetermined upper limit threshold is provided for the angular acceleration at which the orientation of the virtual character 401 changes.

  FIG. 8 is a flowchart showing a flow of processing according to the character display device 200 of Embodiment 1 of the present invention. The processing flow realized in the character display device 200 in the present embodiment described so far will be summarized below.

  When this process is started, the CPU 101 of the character display device 200 initializes various information prepared in the RAM 103, such as information stored in the storage unit 201 (step S801).

  Next, the character display device 200 displays a leftward character image 301 having a predetermined angle 403 on the display screen 300 (step S802). Here, the display unit 202 generates an image to be displayed on the display screen 300, waits for a vertical synchronization interrupt performed at a predetermined cycle, and then displays the image on the display screen 300. Specifically, as shown in FIG. 4, a virtual character 401 is arranged in the virtual space 400 in a state where the virtual character 401 is directed to the left by a predetermined angle 403 with respect to the virtual camera 402, and the display unit 202 An image viewed from the virtual camera 402 is generated, and a character image 301 facing left as shown in FIG. 3 is displayed on the display screen 300 together with other images.

  And the character display apparatus 200 determines whether the audio | voice was detected from the user 501 (step S803). That is, the detection unit 203 receives and detects a voice emitted from the user 501. If it is determined that no detection has been made (step S803; NO), the process of this flowchart returns to step S802, and a process of displaying a horizontally oriented character image 301 at a predetermined angle 403 on the display screen 300 is performed. That is, as long as no sound is detected by the detection unit 203, the character display device 200 according to the present embodiment continues to display the character image 301 in the horizontal direction of the predetermined angle 403, that is, in the initial direction.

  On the other hand, when it is determined that the sound is detected by the detection unit 203 (step S803; YES), the character display device 200 determines a change in the orientation of the virtual character 401 based on the detected sound volume (step S804). ). That is, the determination unit 204 obtains a volume from the detected voice, and obtains an angle at which the direction of the virtual character 401 in the virtual space 400 should be changed based on the volume value.

  In the present embodiment, the angle whose direction should be changed is obtained from the relationship between the volume and the angle shown in FIG. That is, as the volume increases, the direction approaches from the lateral direction of the predetermined angle 403 to the front direction, and when a predetermined threshold volume or higher is finally detected, the virtual character 401 faces the virtual camera 402 directly in front. Become.

  Furthermore, the character display device 200 according to the present embodiment updates the orientation of the virtual character 401 according to the determined change (step S805). That is, the update unit 205 actually updates the orientation of the virtual character 401 in the virtual space 400 by the angle that changes the orientation of the virtual character 401 determined by the determination unit 204. For example, when it is determined by the determining unit 204 that the orientation of the virtual character 401 is changed to the front direction, the virtual character 401 is oriented so as to face the virtual camera 402 as shown in FIG. Can be changed.

  Here, in the present embodiment, when the orientation of the virtual character 401 is updated, it is determined whether or not the angular acceleration has exceeded a predetermined threshold (step S806). That is, in step S809, which will be described later, the character image 301 whose direction has been updated is displayed on the display screen 300 after waiting for a predetermined vertical synchronization interrupt, and the change in the direction displayed there has been displayed so far. It is determined whether the degree of change (angular acceleration) is too large compared to the change in orientation.

  And when it determines with having exceeded (step S806; YES), the update part 205 adjusts so that angular acceleration may not exceed a predetermined threshold value, and updates direction (step S807). That is, here, the update is performed such that the angle is changed less per unit time than the direction updated in step S805. Thereafter, the process proceeds to step S808. On the other hand, when it is determined that the angular acceleration does not exceed the predetermined threshold (step S806; NO), nothing is performed here, and the process proceeds to the next step S808.

  In the present embodiment, the position of the virtual pupil 404 of the virtual character 401 is further adjusted so that the line of sight of the character image 301 faces the front (step S808). That is, the update unit 205 adjusts the position of the virtual pupil 404 in the virtual character 401 in the virtual space 400 so that the line of sight is directed toward the virtual camera 402 no matter which direction the virtual character 401 is facing.

  Then, the updated character image 301 is displayed on the display screen 300 (step S809). Here, as in step S <b> 802, the display unit 202 generates an image to be displayed on the display screen 300, waits for a vertical synchronization interrupt performed at a predetermined cycle, and then displays the image on the display screen 300. Specifically, the display unit 202 generates an image representing the state of the virtual character 401 in the virtual space 400 updated in the processing so far, and displays the image on the display screen 300.

  Thereafter, the process of the character display device 200 according to the present embodiment returns to step S803, and determines again whether or not voice is detected from the user 501. That is, as long as the detection unit 203 detects the sound from the user 501, the character display device 200 according to the present embodiment repeats the processing from step S804 to S809 for each vertical synchronization. On the other hand, if no sound is detected in step S803, the process returns to step S802, and the character image 301 in the horizontal direction of the predetermined angle 403, that is, in the initial direction is displayed again.

  With such a configuration, the character display device 200 according to the present embodiment changes the direction of the character image 301 displayed in the display screen 300 based on the volume of the sound from the user 501, and the larger the volume, Move closer to the front. That is, when the user 501 speaks to the monaural microphone 502, the character image 301 associated with the user 501 changes from the horizontal direction to the front direction.

  As a result, when users in distant places communicate with each other, the user does not talk only by voice or voice and characters, but looks at the character image 301 that changes direction in response to the voice. You will be able to enjoy the conversation.

(Embodiment 2)
Next, Embodiment 2 of the present invention will be described. In the first embodiment, as the sound volume of the user 501 detected by the monaural microphone 502 is larger, the direction of the character image 301 displayed in the display screen 300 is closer to the front. In the present embodiment, the user's voice is detected by the stereo microphone, and the direction of the character image 301 displayed in the display screen 300 is changed based on the detected difference in volume between the left and right voices of the stereo microphone. In other words, in the present embodiment, the detection unit 203 detects the sound with the stereo microphone, and the determination unit 204 changes the direction of the virtual character 401 based on the difference between the left and right sound volumes detected by the stereo microphone. decide.

  FIG. 9 is a diagram illustrating a state in which the user utters from different directions toward the microphone in order to explain the comparison between the monaural microphone and the stereo microphone. In particular, FIG. 9A shows the position of the user 501 changing between the left position 510a and the right position 510b with respect to the monaural microphone 502 and FIG. It is the figure showing a mode that it was made to utter.

  As shown in FIG. 9A, when the sound is detected by the monaural microphone 502, a change due to the position of the user 501 cannot be captured. That is, for example, when the user 501 makes a sound from the left position 510a and when the user utters a sound from the right position 510b, both of them make a sound with the same magnitude at the same distance from the monaural microphone 502. If this is the case, only the sound of the same volume is detected in either case, and the difference is not detected. For this reason, the character display device 200 according to the first embodiment using the monaural microphone 502 can detect the volume of the detected sound, whether the sound is detected from the left direction or the sound detected from the right direction. Similarly, the orientation of the virtual character 401 is changed.

  On the other hand, in FIG.9 (b), the stereo microphone 503 has a microphone in two places, the left microphone 504a and the right microphone 504b. Therefore, it is possible to determine whether the detected sound is, for example, the sound that the user 501 utters from the left position 510a or the sound that the user 501 utters from the right position 510b. Specifically, when the user 501 is uttering from the left position 510a toward the stereo microphone 503, the volume of the sound detected by the left microphone 504a is the volume of the sound detected by the right microphone 504b. Bigger than. On the other hand, when the user 501 is uttering from the right position 510b to the stereo microphone 503, the sound volume detected by the right microphone 504b is greater than the sound volume detected by the left microphone 504a. Also grows. Although not shown in FIG. 9B, when the user 501 is speaking from the front with respect to the stereo microphone 503, the left and right microphones 504a and 504b detect the same volume.

  9B, the left microphone 504a and the right microphone 504b are arranged at two different positions on the left and right, but the stereo microphone 503 in the present embodiment is not limited to this, and the left microphone 503a can be viewed from the left. It may be of a type in which a microphone that captures the sound of the above and a microphone that captures the sound from the right direction are arranged in one place.

  In FIGS. 9A and 9B, the user 501 at the left position 510a and the user 501 at the right position 510b are uttered by the same user 501 changing positions. However, different users 501 may utter at the same time. Even in the latter case, similarly to the above, the monaural microphone 502 cannot determine which position the detected sound is from the user 501, and the stereo microphone 503 has left and right microphones 504 a, It is possible to determine in which position the voice is from the user 501 based on the difference in volume of the voice detected in 504b.

  In the present embodiment, the left and right orientations of the character image 301 in the display screen 300 are changed using the difference in volume obtained by the left and right microphones 504a and 504b. Therefore, as in the first embodiment, a virtual space 400 is prepared in the storage unit 201, a virtual character 401 is arranged therein, and the left and right orientations of the virtual character 401 are obtained by the left and right microphones 504a and 504b. It is changed based on the difference in volume that is generated. Here, the orientation of the virtual character 401 changes between the front orientation and the left orientation of the predetermined angle 403 in the first embodiment, but in this embodiment, the orientation changes from the front orientation to the left and right directions.

More specifically, in the present embodiment, the range that the orientation of the virtual character 401 can take is between the horizontal orientations of the left and right predetermined angles 403 including the front orientation,
(A) The smaller the difference between the left and right volume of the sound detected by the stereo microphone 503, the closer the virtual character 401 is to the front,
(B) As the left volume of the sound detected by the stereo microphone 503 is larger than the right volume, the direction of the virtual character 401 approaches the horizontal direction of the left predetermined angle 403,
(C) The larger the right volume of the sound detected by the stereo microphone 503 is, the closer the virtual character 401 is to the horizontal direction of the predetermined angle 403 on the right.
To determine the change. Hereinafter, a description will be given with reference to FIGS. 10A and 10B.

  FIG. 10A shows a state of the virtual space 400 (FIG. 10A (a)) and the display screen 300 (FIG. 10A (b)) when sound is detected from the left direction in the character display device 200 of the present embodiment. It is a figure. On the other hand, FIG. 10B shows a virtual space 400 (FIG. 10B (a)) and a display screen 300 (FIG. 10B (b)) when sound is detected from the right direction in the character display device 200 of the present embodiment. It is the figure which showed a mode. The virtual space 400 and the display screen 300 in FIGS. 10A and 10B are composed of the same components as those described in the first embodiment. However, here, the stereo microphone 503 is connected to the display unit 202 of the character display device 200 displaying the display screen 300, and the user 501 who is viewing the display screen 300 is directed toward the stereo microphone 503. The situation is saying “Hello !!”. Then, based on the sound detected by the stereo microphone 503, the possible orientations of the virtual character 401 and the character image 301 change to the left and right.

  First, in FIG. 10A (b), the user 501 utters “Hello !!” from the left side of the stereo microphone 503. In this situation, the volume detected by the left microphone 504a of the stereo microphone 503 is larger than the volume detected by the right microphone 504b. Therefore, the orientation of the virtual character 401 in FIG. 10A (a) is directed to the left with respect to the virtual camera 402. Then, the character image 301 in the display screen 300 generated based on the virtual character 401 facing left is turned left as shown in FIG. 10A (b).

  On the other hand, in FIG. 10B (b), the user 501 utters “Hello !!” from the right side of the stereo microphone 503. In this situation, the volume detected by the right microphone 504b of the stereo microphone 503 is larger than the volume detected by the left microphone 504a. Therefore, the orientation of the virtual character 401 in FIG. 10B (a) is directed to the right with respect to the virtual camera 402. Then, the character image 301 in the display screen 300 generated based on the virtual character 401 facing right is turned right as shown in FIG. 10B (b).

  That is, when the user 501 has a conversation from the left side of the stereo microphone 503, the character image 301 can also turn to the left, and it can appear as if the character is having a conversation from the left side. Similarly, when the user 501 has a conversation from the right side of the stereo microphone 503, the character image 301 can be turned to the right, as if the character is having a conversation from the right side.

  In the present embodiment, as in the first embodiment, when the orientation of the virtual character 401 is changed, the line of sight of the character image 301 in the display screen 300 may be changed while facing the front. That is, the virtual pupil 404 is prepared in the virtual character 401 in FIG. 10A (a) and FIG. 10B (a), and the virtual pupil 404 is virtual even if the virtual character 401 faces left or right with respect to the virtual camera 402. It arrange | positions so that the direction of the camera 402 may face. As a result, in FIG. 10A (b) and FIG. 10B (b), the character image 301 displayed in the display screen 300 remains with the pupil image 303 facing forward, that is, the line of sight regardless of the orientation of the entire body. It remains facing the front.

  FIG. 11 is a graph showing the relationship between the detected sound volume and the angle that determines the orientation of the virtual character 401 in the present embodiment. The change in the orientation of the virtual character 401 based on the detected voice as described in FIGS. 10A and 10B will be described more quantitatively with reference to this drawing.

  In FIG. 11, the horizontal axis shows the difference between the left and right volume detected by the stereo microphone 503 as a value obtained by subtracting the volume of the left microphone 504a from the volume of the right microphone 504b. The volume of the microphone 504b is larger than the volume of the left microphone 504a, and the volume of the left microphone 504a is larger than the volume of the right microphone 504b toward the left side. The vertical axis indicates the angle that determines the orientation of the virtual character 401. The orientation of the virtual character 401 increases toward the left as the upper side of the graph, and increases toward the right as the lower side. The virtual character 401 in the present embodiment takes an angle direction as shown in this figure based on the left and right sound volumes detected by the stereo microphone 503.

  That is, first, when there is no difference in volume detected by the left and right microphones 504a and 504b, the angle that determines the orientation of the virtual character 401 is 0 °, that is, the front direction. On the other hand, when the volume detected by the left microphone 504a is larger than the volume detected by the right microphone 504b, the orientation of the virtual character 401 turns to the left at an angle proportional to the volume difference. Conversely, when the volume detected by the right microphone 504b is larger than the volume detected by the left microphone 504a, the virtual character 401 turns to the right at an angle proportional to the volume difference. When the left and right volume difference reaches a predetermined threshold volume, the virtual character 401 is directed leftward or rightward at a predetermined angle 403. In the present embodiment, the virtual character 401 remains left or right at a predetermined angle 403 even when the difference in volume is greater.

  With such a configuration, the direction of the character image 301 displayed on the display screen 300 can be changed by reflecting the direction in which the user 501 speaks toward the stereo microphone 503.

  FIG. 12 is a flowchart showing a process flow according to the character display device 200 of the second embodiment of the present invention. The flow of processing in the present embodiment that has been described so far will be described below. This flowchart is basically the same as that of the flowchart (FIG. 8) in the first embodiment, but there is a difference in the processing of step S802 and step S804 in FIG. 8 (portion surrounded by a dotted line in FIG. 12). is there.

  That is, the character display device 200 according to the present embodiment starts this processing and initializes various types of information (step S801), and then the display unit 202 waits for a vertical synchronization interrupt for the front-facing character image 301. Later, it is displayed on the display screen 300 (step S1201). In other words, in the situation where no sound is detected by the detection unit 203, the horizontal character image 301 of the predetermined angle 403 is displayed on the display screen 300 in the first embodiment, but the character image 301 displayed in the present embodiment is front-facing. Become.

  Furthermore, in the present embodiment, the detection unit 203 determines whether or not a voice has been detected from the user 501 (step S803), and when it is determined that it has been detected (step S803; YES), the determination unit 204 has been detected. A change in the orientation of the virtual character 401 is determined based on the difference between the left and right sound volumes (step S1202). That is, in the first embodiment, the direction is determined based on the volume of the sound detected by the monaural microphone 502, whereas in this embodiment, the direction is determined based on the difference between the left and right sound volumes detected by the stereo microphone 503. Determine the orientation. At this time, the relationship between the difference between the left and right volume shown in FIG. 11 and the angle that determines the orientation of the virtual character 401 is used.

  In this embodiment, thereafter, as in the first embodiment, the update unit 205 updates the orientation of the virtual character 401, and the display unit 202 displays the updated character image 301 on the display screen 300 after vertical synchronization. The processing up to (steps S805 to S809) is performed, and the process returns to step S803, and the detection of the voice by the detection unit 203 is performed again.

  While the voice from the user 501 is detected, the processing from step S1202 and steps S805 to S809 is repeated for each vertical synchronization, and the character image 301 whose left and right orientations are updated is displayed on the display screen 300. Will do.

  With such a configuration, when the volume of the left microphone 504a detected by the stereo microphone 503 is large, the character image 301 in the display screen 300 faces the left and the character display device 200 in this embodiment faces the right microphone 504b. When the volume is large, the character image 301 turns to the right. In other words, the direction reflecting the relative position of the user 501 with respect to the stereo microphone 503 can be taken in the character image 301 associated with the user 501.

  As a result, when users in remote locations communicate with each other, the user can proceed with the conversation while confirming from which direction the other party is speaking, and the user has a more realistic feeling. You will be able to enjoy the conversation smoothly.

(Embodiment 3)
Next, a third embodiment of the present invention will be described. In the first and second embodiments, the left and right orientations of the character image 301 in the display screen 300 are changed based on the volume of the voice from the user detected by the monaural microphone 502 or the stereo microphone 503. In the present embodiment, two types of volumes are acquired from the user's voice, and not only the left and right direction of the character image 301 but also the vertical direction is changed based on each.

  That is, in Embodiments 1 and 2 so far, when the determination unit 204 acquires the sound volume from the sound detected by the detection unit 203, the sound is acquired by averaging the sound by a predetermined section length. In the present embodiment, two different time lengths, that is, a predetermined first section length and a predetermined second section length are averaged separately to obtain two volumes of a first volume and a second volume, respectively. .

  For example, in the present embodiment, the predetermined first section length is set to a longer section length than the predetermined second section length. In this case, the first sound volume averaged by the predetermined first section length is increased when the low-frequency component, that is, the low-sound component is included in the voice emitted from the user. . On the other hand, the second sound volume averaged by the predetermined second section length is obtained when the sound emitted from the user contains a higher frequency component, that is, a higher sound component. growing.

Based on the first volume and the second volume acquired in this way, the character display device 200 of the present embodiment changes the horizontal direction and the vertical direction of the virtual character 401, respectively. Specifically, the determination unit 204
(P) determining a change in the left-right orientation of the virtual character 401 based on the first volume averaged from the detected voice by a predetermined first section length;
(Q) A change in the vertical direction of the virtual character 401 is determined based on the second sound volume averaged from the detected sound by a predetermined second section length.
Then, as in the first and second embodiments, the update unit 205 updates the orientation stored by the determined left and right and up and down changes, and the display unit 202 updates the left and right and up and down orientations of the virtual character 401. Is generated and displayed as a character image 301 in the display screen 300. Hereinafter, a description will be given with reference to FIG.

  FIG. 13 is a diagram showing how the character image 301 changes based on the voice from the user 501 in the present embodiment. Here, in both FIGS. 13A and 13B, as in FIGS. 3 and 5 in the first embodiment and FIGS. 10A and 10B in the second embodiment, a character image is displayed on the display screen 300. 301, a message window 302, and other background images are displayed. A monaural microphone 502 is connected to the display unit 202 of the character display device 200 displaying the display screen 300.

In FIG. 13A, the user 501 does not emit any sound toward the monaural microphone 502. At this time, the character image 301 is directed leftward when viewed from the user 501 who is viewing the display screen 300, and the head portion of the character image 301 is directed downward. In the present embodiment, it is assumed that the character image 301 is in such an orientation as shown in FIG. 13A in a state where the detection unit 203 has not yet detected any sound, that is, in an initial state. When the detection unit 203 detects voice from here, that is, when the user 501 speaks to the monaural microphone 502, the orientation of the character image 301 changes to the orientation shown in FIG.

  In FIG. 13B, the user 501 utters “Hello!” Toward the monaural microphone 502. At this time, as compared with FIG. 13A, the character image 301 has a left-right orientation facing the front and a head portion facing upward. That is, here, the direction of the character image 301 is changed from the left direction to the front direction by the first sound volume averaged by the predetermined first section length from the detected sound, and the detected sound is further detected. Thus, the orientation of the head portion of the character image 301 changes from downward to upward by the second sound volume averaged by the predetermined second section length. Such a change in the orientation of the virtual character 401 will be described more quantitatively with reference to FIG.

  FIG. 14 is a graph showing the relationship between the sound volume detected by the detection unit 203 and the angle that determines the orientation of the virtual character 401 in the third embodiment. In particular, FIG. 14A is a diagram showing the relationship between the first volume and the angle that determines the left and right direction of the virtual character 401, and FIG. 14B is the second volume and the virtual character 401. It is the figure which showed the relationship with the angle which defines an up-down direction. Here, the horizontal axis represents the first or second volume acquired from the detected voice, and the volume increases toward the right side of the graph. The vertical axis takes an angle that determines the left and right or up and down directions of the virtual character 401. The upper side of the graph indicates that the direction of the virtual character 401 is largely left or up.

  First, in FIG. 14A, when there is no first sound volume averaged by the first section length, the virtual character 401 is leftward at a predetermined angle 403 determined in advance. As the first sound volume gradually increases from here, the direction of the virtual character 401 gradually approaches the front direction. When the first sound volume reaches a predetermined first threshold sound volume, the angle reaches 0 °, and the virtual character 401 faces directly toward the virtual camera 402, that is, the character image 301 faces front in the display screen 300. Will be displayed. Furthermore, even when a sound including a first sound volume that is equal to or higher than a predetermined first threshold sound volume is detected, it remains in the front direction.

  On the other hand, in FIG. 14B, when there is no second volume averaged by the second section length, the virtual character 401 is in a predetermined downward angle. From here, the virtual character 401 gradually turns upward as the second volume gradually increases. When the second sound volume reaches a predetermined second threshold sound volume, the angle reaches a predetermined upward angle, and the character image 301 facing upward is displayed in the display screen 300. Furthermore, even when a sound including a second volume that is equal to or higher than a predetermined second threshold volume is detected, the direction remains at a predetermined upward angle.

  Note that the detection unit 203 in the present embodiment is not limited to the monaural microphone 502, and may be a stereo microphone 503. When the stereo microphone 503 is used, the first sound volume averaged by the first section length and the first section averaged by the second section length from the sounds detected by the left and right microphones 504a and 504b. Get the volume of 2. For example, the left and right direction and the vertical direction of the character image 301 are changed based on the difference between the first volume and the second volume between the left and right microphones 504a and 504b. be able to.

  In this way, not only the left and right direction is changed to the front direction in response to the user 501 uttering voice, but also by changing the vertical direction upward, the character image 301 speaks with his head raised. This makes it possible to have a conversation with a sense of reality for a user who is having a conversation.

  FIG. 15 is a flowchart showing a flow of processing according to the character display device 200 of Embodiment 3 of the present invention. The flow of processing in the present embodiment that has been described so far will be described below. This flowchart basically follows the flowchart of the first embodiment (FIG. 8), but replaces the processing of steps S802, S804, and S805 in FIG. The part enclosed with a dotted line at) enters.

  That is, the character display device 200 according to the present embodiment starts the process and initializes various types of information (step S801), and then the display unit 202 is a character having a predetermined angle 403 leftward and a predetermined downward angle. The image 301 is displayed on the display screen 300 after waiting for the vertical synchronization interrupt (step S1501). That is, in a situation where no sound is detected by the detection unit 203, in this embodiment, a character image 301 having an orientation as shown in FIG. 13A is first displayed as an initial state.

  Furthermore, in the present embodiment, the detection unit 203 determines whether or not a voice is detected from the user 501 (step S803), and when it is determined that it is detected (step S803; YES), the determination unit 204 is first detected. The first sound volume averaged by the predetermined first section length is acquired from the obtained sound (step S1502). Here, the first section (the section length is T1) taken for averaging is a section between time t1-T1 and time t1, where the current time is t1. That is, the first sound volume is obtained by averaging the detected voices for the time of the past T1 with reference to the current time t1.

  Then, based on the obtained first volume, a change in the left / right orientation of the virtual character 401 is determined (step S1503). In this determination of the change in direction, the relationship between the first volume and the angle that determines the left and right direction of the virtual character 401 shown in FIG.

  Subsequently, a second sound volume averaged by a predetermined second section length is acquired from the detected sound (step S1504). Here, the second section (the section length is T2) taken for averaging is a section between time t2-T2 and time t2, where the current time is t2. In other words, the detected sound is averaged over the past T2 with reference to the current time t2, thereby obtaining the second volume.

  Then, based on the obtained second volume, a change in the vertical direction of the virtual character 401 is determined (step S1505). In this determination of the change in direction, the relationship between the second volume and the angle that determines the vertical direction of the virtual character 401 shown in FIG. 14B is used.

  When the change in the horizontal and vertical orientations of the virtual character 401 is determined in this way, the character display device 200 according to the present embodiment further includes the update unit 205 in which the virtual character 401 has the horizontal and vertical orientations determined by the determined changes. Is updated (step S1506).

  Note that although the processing in steps S1502 and S1504 has been described as being performed in chronological order, they may be performed simultaneously. In other words, the process of averaging from the detected voice by the predetermined first section length T1 and the process of averaging by the predetermined second section length T2 are based on the same time t0 as time t0-T1. May be averaged from a section from time t0 to time t0 and a section from time t0-T2 to time t0.

  In the present embodiment, as in the first embodiment, the character image 301 with the updated orientation displayed on the display unit 202 is displayed from the process of determining whether the angular acceleration has exceeded a predetermined threshold when the orientation is updated. Processing (steps S806 to S809) from waiting for synchronization until display on the display screen 300 is performed, the process returns to step S803, and the detection of the sound by the detection unit 203 is performed again.

  And while the audio | voice from the user 501 is detected, the process to step S1502-S1506 and step S806-S809 is repeated. That is, for each vertical synchronization, two volume levels averaged by two section lengths (T1 and T2) are acquired, and a character image 301 whose left and right and up and down directions are updated based on the two volumes is displayed on the display screen 300. become.

  With such a configuration, the character display device 200 according to the present embodiment can change two different directions of left and right and up and down of the character in the screen from one voice uttered by one user.

  As a result, the character image 301 associated with the user 501 can be made to perform more complicated movements, and the user can enjoy a conversation while enhancing a sense of presence and a sense of familiarity.

  As described above, a character display device, a character display method, and a program suitable for enjoying a conversation comfortably while appealing to the user's vision by causing the character to perform an action according to the user's voice are provided. be able to.

100 Information processing apparatus 101 CPU
102 ROM
103 RAM
104 Interface 105 Controller 106 External Memory 107 Image Processing Unit 108 DVD-ROM Drive 109 NIC
DESCRIPTION OF SYMBOLS 110 Audio | voice processing part 111 Microphone 200 Character display apparatus 201 Storage part 202 Display part 203 Detection part 204 Determination part 205 Update part 300 Display screen 301 Character image 302 Message window 303 Pupil image 304 Conversation content 400 Virtual space 401 Virtual character 402 Virtual camera 403 Predetermined angle 404 Virtual pupil 501 User 502 Monaural microphone 503 Stereo microphone 504a Left microphone 504b Right microphone 510a Left position 510b Right position

Claims (9)

A storage unit that stores the orientation of characters placed in the virtual space,
A display unit for displaying an image representing a state in which the character is arranged in the virtual space in the stored orientation on a screen;
A detection unit for detecting voice from the user,
A determining unit that determines a change in the orientation of the character based on the volume of the detected voice;
A character display device, comprising: an update unit that updates the stored orientation according to the determined change. The character display device according to claim 1,
The detection unit detects sound with a monaural microphone,
The range of the orientation of the character is between the front direction and the horizontal direction at a predetermined angle on either side,
The determination unit determines the change such that the larger the volume of the detected sound is, the closer the character is to the front direction.
Character display device characterized by that. The character display device according to claim 1,
The detection unit detects sound with a stereo microphone,
The determining unit determines a change in the orientation of the character based on a difference between left and right sound volumes detected by the stereo microphone;
Character display device characterized by that. The character display device according to claim 3,
The range of the orientation of the character is between the lateral orientation of each predetermined angle including the front orientation,
The determination unit
(A) The smaller the difference between the left and right volume of the sound detected by the stereo microphone, the closer the direction of the character is to the front direction,
(B) The larger the left volume of the sound detected by the stereo microphone is, the closer the direction of the character is to the left side of the predetermined angle,
(C) The larger the right volume of the sound detected by the stereo microphone is, the larger the left volume is, the closer the character is to the right side at a predetermined angle.
To determine the change,
Character display device characterized by that. The character display device according to any one of claims 1 to 4,
The determination unit
(P) determining a left-right orientation change of the character based on a first volume averaged from the detected voice by a predetermined first section length;
(Q) determining a change in the vertical direction of the character based on a second sound volume averaged from the detected voice by a predetermined second section length;
Character display device characterized by that. A character display device according to any one of claims 1 to 5,
The update unit updates the stored orientation while keeping the line of sight of the character facing the front.
Character display device characterized by that. A character display device according to any one of claims 1 to 6,
The update unit updates the stored orientation so that angular acceleration does not exceed a predetermined threshold;
Character display device characterized by that. A character display method executed by a character display device including a storage unit, a display unit, a detection unit, a determination unit, and an update unit,
The storage unit stores the orientation of characters arranged in the virtual space,
A display step in which the display unit displays an image representing a state in which the character is arranged in the virtual space in the stored orientation on a screen;
A detection step in which the detection unit detects voice from a user;
A determining step in which the determining unit determines a change in the direction of the character based on the volume of the detected voice;
The character display method comprising: an update step in which the update unit updates the stored orientation according to the determined change. Computer
A storage unit that stores the orientation of characters placed in the virtual space,
A display unit for displaying an image representing a state in which the character is arranged in the virtual space in the stored orientation on a screen;
A detection unit for detecting voice from the user,
A determining unit that determines a change in the orientation of the character based on the volume of the detected voice;
A program that functions as an updating unit that updates the stored orientation according to the determined change.

Images