JP3391050B2 - Voice virtual localization system - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an audio virtual localization system that localizes audio corresponding to a virtual position in a virtual space virtualized by an operator. Conventionally, by connecting a plurality of virtual space simulators (virtual space forming means) by communication, a virtual space common to each virtual space simulator is formed, and each virtual space simulator is operated. There is known a virtual space simulator system in which the operators can share the same virtual space. A virtual space formed by a virtual space simulator is formed by a three-dimensional space in which a so-called virtual reality can be experienced. In each virtual space simulator, for example, when an operator operates a joystick or the like, the operator on the virtual space can take an arbitrary action, and at the same time, a sound corresponding to the action is emitted. [0004] However, in the virtual space simulator system connected by the conventional communication, the sound is only patterned from the source set in advance by the virtual space simulator. However, there is a problem in that the actual voice generated by the operator and the image displaying the three-dimensional space are not related and the reality of the virtual space is lacking. The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to associate sound in a virtual space by associating actual sound generated by an operator with an image displaying a three-dimensional space. It is to provide a sound virtual localization system that improves the reality of the virtual space. [0006] The above object is achieved by a virtual space forming means for forming a virtual space that is virtualized by an operator,
A voice input means for inputting a voice uttered by the operator;
Communication means for performing data transmission between the virtual space forming means and voice transmission for sending voice data for each predetermined unit transmission time based on the voice together with virtual data of the virtual space for every predetermined unit transmission time in data transmission And a voice virtual localization system characterized in that the voice of the operator input to the voice input means is localized at a virtual position in the virtual space. According to the present invention, the voice data based on the voice uttered by the operator input to the voice input means is transmitted to the virtual space virtualized by the operator formed in the virtual space forming means. Since the data is transmitted every predetermined unit transmission time together with virtual data in the virtual space every predetermined unit transmission time in the data transmission, the voice uttered by the operator is localized at the appropriate virtual position in the virtual space. Thus, the data is output to other virtual space forming means, and at the same time, it is linked to an image based on the virtual data. DESCRIPTION OF THE PREFERRED EMBODIMENTS An audio virtual localization system according to an embodiment of the present invention will be described below with reference to the drawings. As shown in FIG. 1, the sound virtual localization system 10 is provided for each virtual space simulator 11, a virtual space simulator (virtual space forming means) 11, a simulator group 12 provided with a plurality of virtual space simulators 11, and each. Microphone (voice input means) 13, communication means 14, and voice sending means 15
And a host computer 16. This voice virtual localization system 10 forms a common virtual space for each virtual space simulator 11 constituting the simulator group 12, and each operator who operates each virtual space simulator 11 has the same virtual space. You can share space. Virtual space simulator 11
As shown in FIG. 2, the CPU 11a as the control means
CPU 11 based on the input virtual data
By the arithmetic processing of a, a video output and an audio output for displaying an image of a virtual space consisting of a three-dimensional (3-dimention) space are output. This virtual space allows you to experience so-called virtual reality. The virtual data consists of position data in the virtual space of the virtual operator, which is issued by the operator, state data such as a desired moving direction and moving speed, or a desired action type. It is possible to make a virtual operator take a desired action. In each virtual space simulator 11, position data and state data are input via the decoder 17a when an operator operates input means such as the joystick 17 or the like. Similarly, position data and state data can be input via the A / D converter 18a and the like by the position sensor 18 that detects the displacement state of the operator (see FIG. 2). The image information is output as a three-dimensional image to the head mounted display 19b or the like via the image output device 19a. Similarly, the audio data is output as audio to the speaker 20c or the like via the audio output device 20a and the virtual space audio localization calculation device 20b (FIG. 2).
reference). The simulator group 12 includes a plurality of virtual space simulators 11 and a host computer 16 to which each virtual space simulator 11 is connected via communication means 14 (see FIG. 1). Data can be input / output to / from the computer 16. The microphone 13 is provided, for example, for each head-mounted display 19b worn by the operator, and collects sound produced by the operator who operates the virtual space simulator 11. The collected voice becomes voice data digitized through the noise filter 13a and the like and the A / D converter 13b. The communication means 14 is, for example, ISD
N, a telephone line, and the like, and the voice sending means 15 that is a data output unit from the virtual space simulator 11 and the host computer 16 are connected to each other, and the virtual space simulator 11
A data transmission path for transmitting data between each other is formed. The voice sending means 15 is used for sending data.
The data string is divided into a certain size and transmitted for each packet as a divided portion. For example, one field of data of 1/60 seconds which is a predetermined unit transmission time is defined as one packet. Therefore, for each field, the CPU 11
Along with the position data a and the state data b obtained by the arithmetic processing of a, the voice data c obtained by dividing the voice data input at any time via the microphone 13 for each field
Is sent. In the figure, d is a header (HEADER), and e
Is the checksum (CHECK SUM) or parity (PARI)
TY). That is, the audio data c for each predetermined unit transmission time is transmitted together with the position data a and the status data b for every predetermined unit transmission time in the data transmission. Therefore, since the voice data c can be collected in real time and reproduced in real time, the voice uttered by the operator is output after being localized at an appropriate position in the virtual space, and at the same time, Images can also be linked using data a and status data b. The voice sending means 15 has a compression / decompression function for compressing and decompressing voice data, and is necessary when sending the voice data c for each field together with the position data a and the status data b. Accordingly, the audio data c can be compressed, and the input compressed audio data can be decompressed and returned to the original state. Next, the operation of the voice virtual localization system will be described. First, as shown in FIG. 4, each operator
For example, each person mounts his / her head mounted display 19b or the like on a chair or the like, and becomes a virtual space experienceable state by the virtual space simulator 11 to constitute the simulator group 12. Each operator operates the joystick 17 and the like, and the head mounted display 19
Experience the virtual state on the virtual space displayed as an image on b etc. At this time, common virtual data is sent to the respective virtual space simulators 11 via the host computer 16, and each operator receives each virtual space simulator 1.
1 share the same virtual space. Subsequently, when the operator in the virtual state utters voice,
After being collected by the microphone 13, it is digitized and becomes voice data c divided into fields, and the position data a and the status data b together with the position data a and status data b (see FIG. 3) from the voice sending means 15 through the communication means 14. It is transmitted to the host computer 16. At the time of data transmission, the audio data c is compressed as necessary to be transmitted together with the position data a and the status data b. By the way, when voice data is digitized and compressed and communicated by a necessary amount, more effective three-dimensionalization is possible. In other words,
In particular, the more the high frequency that corresponds to the reflected wave, the clearer the position when localized, so by sending accurate data with complete high frequency components without sound quality degradation,
The signal-to-noise ratio and frequency characteristics necessary for three-dimensionalization on the virtual space simulator 11 can be obtained. The host computer 16 to which the audio data c for each frame is input transfers the audio data c for each frame to the other virtual space simulator 11 through the communication means 14 in the same manner. The audio data c for each frame input to the other virtual space simulator 11 is decompressed and returned to the original state when compressed by the audio sending means 15, and then the position data a and the state data b
And audio data c, and these data are input to the audio output device 20a and the virtual space audio localization calculation device 20b. The input audio data c is localized at an appropriate position in the virtual space by the audio output device 20a and the virtual space audio localization calculation device 20 based on the position data a and the state data b. It is output to the speaker 20c or the like. Accordingly, the voice uttered by the operator is localized at a position in the appropriate virtual space and then output to the other virtual space simulator 11, and naturally, the head is simultaneously based on the position data a and the state data b. The image is linked to the image output to the mounted display 19b or the like. As described above, the localization of the sound and the position of the image in the virtual space are linked in real time, and a plurality of operators can communicate the images and sounds at the respective positions (localization) in the same virtual space by communication. From being able to enjoy
Participants can talk with each other in the virtual space. In addition, more effective three-dimensionalization is possible without deterioration of the voice. Therefore, since the voice can be localized on the virtual space by associating the actual voice generated by the operator with the image displaying the three-dimensional space, only the image and the sound generated from the system side are shown. As a result, the reality of the virtual space and the communication can be improved compared to the conventional virtual space simulator in which the actual voice is separated from the system. In addition, it is possible to output the sound by subjecting it to sound field processing (for example, echo processing). In this case, the reality can be further improved. The present invention is not limited to the above-described embodiments, and various modifications are possible. For example, the virtual space simulators 11 may be directly connected without using the host computer 16. As described above, according to the present invention, the voice is localized on the virtual space by associating the actual voice generated by the operator with the image displaying the three-dimensional space, The reality of virtual space can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an explanatory diagram of an audio virtual localization system according to an embodiment of the present invention. FIG. 2 is a schematic configuration explanatory diagram of a virtual space simulator. FIG. 3 shows position data a, state data b, and audio data c.
It is explanatory drawing of 1 frame which accommodated. FIG. 4 is an explanatory diagram in which a simulator group is configured by each operator wearing a head-mounted display and a microphone. DESCRIPTION OF SYMBOLS 10 ... Sound virtual localization system 11 ... Virtual space simulator (virtual space forming means) 11a ... CPU 12 ... Simulator group 13 ... Microphone (voice input means) 13a ... Noise filter 13b ... A / D converter 14 ... Communication means 15 ... Audio sending means 16 ... Host computer 17 ... Joystick 17a ... Decoder 18 ... Position sensor 18a ... A / D converter 19a ... Image output device 19b ... Head mounted display 20a ... Audio output device 20b ... Virtual space Audio localization calculator 20c ... speaker a ... position data b ... status data c ... audio data d ... header e ... checksum

Continued on front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) G09B 9/00-9/56 A63F 9/00-13/12 G10K 15/00 H04N 5/76 H04S 1/00 G06F 17 / 00

Claims (1)

(57) Claims 1. A virtual space forming means for forming a virtual space virtualized by an operator, a voice input means for inputting a voice uttered by the operator, and the virtual space Communication means for performing data transmission between the forming means, and voice sending means for sending voice data for each predetermined unit transmission time based on the voice together with virtual data in the virtual space for each predetermined unit transmission time in data transmission. A voice virtual localization system characterized in that the voice of the operator input to the voice input means is localized at a virtual position in the virtual space.