JPH06121311A - Virtual reality space sharing system - Google Patents

Abstract

PURPOSE:To obtain a system in which one virtual reality space can be shared by plural terminals in order to realize a communication with a presence. CONSTITUTION:Plural terminals T having virtual reality space preparing means connected with a communication line (ISDN) perform an access to one virtual reality space preparing part (CS) connected to the communication line. Thus, the information of the plural terminals T can be synthesized into one by the CS, the synthesized information can be outputted to the plural terminals T, and a virtual reality space can be prepared by the virtual reality space preparing means based on the synthesized information.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a virtual reality space service system in which a plurality of people communicate by using one virtual reality space using a communication line.

[0002]

2. Description of the Related Art Conventionally, when a conference is held from a plurality of distant places, a television is arranged in a conference room, a distant person is photographed by a camera, the photographed image is transferred through a communication line, and the conference is performed. Broadcast on the TV set on the spot. On the other hand, a person in the conference hall is photographed by a camera, similarly transferred using a communication line, and broadcast on a television placed at a distant person.
A video conferencing system is known in which a person from a distance can participate in a conference by these operations.

[0003]

However, in this video conference system, the equipment becomes large due to the cameras for photographing humans and the televisions installed at each venue. Further, when a plurality of people access, it is divided and displayed as shown in FIG. 10, and the sense of presence is lost. Further, since the image data is transferred, there is a problem that the amount of data becomes enormous. An object of the present invention is to provide a virtual reality in which a plurality of people gather in a room created in a virtual reality space as shown in FIG. It is to provide a space service system.

[0004]

According to a first aspect of the present invention, there is provided a virtual reality space creating means for creating a virtual reality space, and a plurality of terminal means connected to a communication line and having the virtual reality space creating means. , Is connected to the communication line, transmits / receives to / from the plurality of terminal means, collects information sent from the plurality of terminal means, and outputs the combined information to each terminal means. And one information providing means.

It is desirable that the terminal means further includes a motion detecting means for detecting a human motion and a motion data transferring means for transferring the motion data detected by the motion detecting means.

The motion detecting means further stores a voice detecting means for detecting a voice, a voice recognizing means for recognizing the voice detected by the voice detecting means, and a mouth movement corresponding to a voiced voice. It is desirable to have one storage means and a first determination means for determining the mouth movement stored in the first storage means based on the voice recognized by the voice recognition means.

The action detecting means further includes a language recognizing means for recognizing a language based on the voice recognized by the voice recognizing means, and a second storing means for storing a face action corresponding to the language. It is desirable to have a second discriminating means for discriminating the motion of the face stored in the second storage means based on the language recognized by the language recognizing means.

It is desirable that the motion detecting means further has a vector amount detecting means for detecting a vector amount of the motion of each part of the body.

The means of the invention according to claim 6 is 1
It has a means for a plurality of terminals to use one virtual reality space.

[0010]

The service information is reproduced in the virtual reality space by using a communication line from a plurality of terminals to access one service space and service information.

[0011]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a conceptual diagram of a virtual reality space service system of the present invention. In FIG. 1, each terminal T has a device connected to a communication line (ISDN) and creating a virtual reality space. In addition, CS
(Community Stage) is connected, and each terminal T can access this CS to use the information of each one terminal T. When each terminal T reproduces this service information by the virtual reality space creation device, a realistic meeting can be performed as if a plurality of people were communicating at one place.

The data to be transmitted / received is not the image data itself, but the human motion data. That is, the terminal T has the image data corresponding to the ID of each terminal T, and the image is made to appear to be moving by synthesizing the moving image based on the motion data transmitted to the image data. . Further, the terminal T which has not registered the ID displays the image data in a predetermined format. Further, the line connection between each terminal T and CS is performed according to a predetermined procedure.

2 and 3 show an embodiment of the present invention. FIG. 1 shows an example of a terminal, for example, a helmet 1 is put on the head and a glove 2 is put on the hand. This helmet 1
Has a detection unit and a notification unit. The detection unit detects the movement of the head, and includes a vector sensor that detects the movement direction and the movement amount, and a microphone that detects a voice. The notification unit is configured by a display device that displays an image that should normally be seen by the eyes, separate images displayed on the left and right eyes, a stereoscopic effect, and a speaker that emits sound. Further, the glove 2 is a detection unit, and is provided with a vector sensor that detects a motion of a hand and a vector sensor that detects a motion of a finger.

It should be noted that, for example, as shown in FIG. 3, the above-mentioned display device has a display as seen by one's own eyes, and it looks as if a conference is being held at the same place.

Each of the detecting section and the notifying section is connected to the terminal body, and the terminal body recognizes and
Based on the data detected by the detection unit, operation data and voice data are created and transmitted to the CS connected through the communication line. Also, image data and audio data are created based on the operation data and audio data received through the communication line, and output to the notification unit.

In the above-mentioned motion data, the motion of the head, the motion of the hand, and the motion of the finger are detected by the detection unit. Furthermore, basic motions of each part of the body that accompany the motions of the head and hands are also created as motion data. In addition, voice recognition is performed on the voice detected by the microphone, and the mouth movement data is created by referring to the knowledge base from the recognized voice. Furthermore, language recognition is performed from the recognized voice, and the motion data of the face is created by referring to the knowledge base from the recognized language. As a result, body motion data, mouth motion data, and face motion data are created as motion data.

As for transmission / reception data, as shown in FIG. 4, a header, operation data and voice data are transmitted / received using a time division digital multiplexer (TDM). The header contains the ID of each terminal T, the user ID, the time synchronization data, and the like.

5 and 6 are block diagrams of the virtual reality space service system. FIG. 5 is a block diagram of the terminal T side,
Vector sensor 10, microphone 11, speaker 12, display unit 13, knowledge base 14, microcomputer 15, voice codec 16, motion data companding unit 17, voice data companding unit 1
8 and TDM 19.

First, the transmission will be described. The vector sensor 10 detects movements of the head, hands, and fingers, and outputs the movement direction and movement amount of each part to the microcomputer 15. Further, upon receiving the voice data from the microphone 11, the voice codec 16 converts the voice into a digital signal and outputs it to the voice data companding unit 18 and also to the microcomputer 15.

The microcomputer 15 will be described later with reference to the movement direction and movement amount of each other body portion which accompanies the movement of each portion of the body detected based on the movement direction and movement amount of each body portion from the vector sensor 10. Guess by referring to the knowledge base 14. In addition, voice recognition and language recognition are performed while referring to a knowledge base 14 described later based on the voice data taken in through the microphone 11, and the knowledge base 14 described later is also used.
With reference to, the mouth movement and the face movement are inferred based on the voice recognized speech and the language recognized language. Then, these motion data (the motion of each part of the body, the motion of the mouth, the motion of the face) are output to the motion data companding unit 17.

The knowledge base 14 infers motions of other parts of the body associated with the motions of the respective parts of the body detected as described above, voice recognition, language recognition, and mouth and face motions based on these recognition data. It is a database for doing.

In response to the operation data from the microcomputer 15, the operation data companding unit 17 compresses the operation data,
It outputs to 19. In addition, the voice data from the voice codec 16 is compressed by the voice data companding unit 18 and the TDM1.
9 is output. These compressed operation data and voice data are time-divisionally digitally multiplexed by the TDM 19 and a terminal ID as a header and a user I at a predetermined timing position.
D, time synchronization data, etc. are added and output to ISDN.

Next, the reception will be described. The time-division-multiplexed data as shown in FIG. 4 is sent from the ISDN to the TDM 19 at a predetermined timing position. The data at this timing position is taken out and separated into a header, motion data, and voice data. The header and motion data are output to the motion data companding unit 17, and the voice data is output to the voice data companding unit 18. . The operation data companding unit 17 expands the header and the operation data from the TDM 19 and outputs them to the microcomputer 15. Further, the audio data companding unit 18 expands the audio data from the TDM 19 and outputs it to the audio codec 16.

In the microcomputer 15, the operation data companding unit 17
From the header and operation data from, the operation data from the user ID and time synchronization data of this header,
Detect which user's data and which time. In the headers sent, the same time synchronization data is detected, and the motion data with this header makes it possible to reconstruct and display the image in which the moving part of the previous image of each user is re-displayed. The images input to the left and right eyes are changed and output to the display unit 13. A control signal is output to the voice codec 16 at this timing (that is, the timing of the time-synchronized data), and the voice codec 16 converts the voice data sent from the voice data companding unit 18 into analog voice data, and outputs the analog voice data from the microcomputer 15. It outputs to the speaker 12 at the timing of the control signal.

Next, the CS side will be described. Figure 6 shows CS
FIG. 3 is a block diagram on the side, which is composed of an exchange 100 and is connected to an ISDN by transmission lines corresponding to a plurality of terminals. Then, the data (header, operation data, voice data) of each terminal sent from the ISDN is output to a transmission path other than the transmission path to which the data is sent. That is, the data of one terminal is output to each of the other terminals.

7, 8 and 9 are operation flowcharts of the virtual reality space service system, FIGS. 7 and 8 are operation flowcharts of transmission / reception on the terminal T side, and FIG. 9 is operation flowcharts on the CS side. .

The following operation is performed after connecting each terminal to CS by a predetermined procedure.

FIG. 7 shows a transmission process of the terminal T. The motion direction and motion amount of the head, hand, and finger are detected by each vector sensor 10 (step S1), and the motion is accompanied based on the motion direction and motion amount of the head, hand, and finger detected by the vector sensor 10. The motion of each part of the other body is estimated and motion data is created for each part (step S2). On the other hand, voice data is detected by the microphone 11 (step S3), and voice recognition is performed based on the detected voice data (step S).
4). The mouth movement is estimated by performing voice recognition, and mouth movement data is created (step S5). Furthermore, language recognition is performed based on the voice recognized in step S5 (step S6). The face motion corresponding to the recognized language is estimated by performing language recognition, and face motion data is created (step S7).
Then, time-division digital multiplexing processing is performed by attaching a header to these operation data and voice data (step S8). It is transmitted by this.

FIG. 8 shows a receiving process of one terminal. Received data (header, operation data, voice data) corresponding to each other terminal by time division digital multiplexing is taken out and separated (step S11). Time matching is performed according to the time synchronization data in the header of the data corresponding to each terminal (step S1).
2) An image to be output to the display device is created based on the operation data (step S13). Further, the image data is output to the display device, and at the same time, the audio data is output from the audio codec 16 to the speaker 12 (step S14).

By the above operation, the image as shown in FIG. 3 is displayed on the display device arranged in front of the user's eyes.

FIG. 9 shows the processing on the CS side. The data transmitted from each terminal via ISDN is received (step S21), and the data transmitted from the transmission path corresponding to each terminal is grouped into another transmission path at the same timing. Time-division digital multiplexing processing is performed (step S22) and transmitted.

As described above, according to the present invention, there is a sense of realism as if a conference is being held at the same place, and since the image data itself is not transferred, it is transferred as operation data. It can be transferred in a small amount and efficiently.

In this embodiment, the transmission line is ISDN, but the present invention is not limited to this. The transmission method used is time division digital multiplexing, but the present invention is not limited to this. Further, the present system can be applied not only to the conference system but also to a communication system, a game and the like.

[0034]

As described in detail above, according to the present invention, since one virtual reality space is accessed by a plurality of terminals, it is possible to carry out realistic communication. Further, since the motion of the person is detected and the motion data is transferred instead of transferring the image data itself, the amount of data is small with respect to the image data, and the image data can be transferred efficiently.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an embodiment of the present invention.

FIG. 2 is a conceptual diagram of a usage example of a terminal according to an embodiment of the present invention.

FIG. 3 is a conceptual diagram showing a display example of a terminal according to an embodiment of the present invention.

FIG. 4 is a configuration diagram of transfer data according to an embodiment of the present invention.

FIG. 5 is a block diagram of a terminal according to an embodiment of the present invention.

FIG. 6 is a block diagram of a CS according to an embodiment of the present invention.

FIG. 7 is an operation flowchart showing a transmission process of the terminal according to the embodiment of the present invention.

FIG. 8 is an operation flowchart showing a reception process of the terminal according to the embodiment of the present invention.

FIG. 9 is an operational flowchart showing signal processing of CS according to an embodiment of the present invention.

FIG. 10 is a conceptual diagram showing a conventional example.

FIG. 11 is a conceptual diagram showing the present invention.

[Explanation of symbols]

 10 ... Vector sensor 11 ... Microphone 12 ... Speaker 13 ... Display unit 14 ... Knowledge base 15 ... Microcomputer 16 ... Voice codec 17 ... Operation data companding unit 18. ..Voice data companding unit 19 ... TDM 100 ...

Claims (6)

[Claims] 1. A virtual reality space creating means for creating a virtual reality space, a plurality of terminal means connected to a communication line and having the virtual reality space creating means, and transmission / reception with the plurality of terminal means connected to the communication line. And collecting at least one piece of information sent from the plurality of terminal means, and outputting at least one piece of information to each terminal means. Virtual reality space service system. 2. The terminal means further comprises motion detecting means for detecting a human motion, and motion data transferring means for transferring motion data detected by the motion detecting means. Item 1. The virtual reality space service system according to Item 1. 3. The motion detecting means further stores a voice detecting means for detecting a voice, a voice recognizing means for recognizing the voice detected by the voice detecting means, and a mouth movement corresponding to a voiced voice. A first storage means for determining the movement of the mouth stored in the first storage means based on the voice recognized by the voice recognition means.
The virtual reality space service system according to claim 2, further comprising: 4. The motion detecting means further includes a language recognizing means for recognizing a language based on the voice recognized by the voice recognizing means, and a second memory storing a face motion corresponding to the language. Second means for determining a motion of the face stored in the second storage means based on the language recognized by the language recognition means.
The virtual reality space service system according to claim 2, further comprising: 5. The virtual reality space service system according to claim 2, wherein the motion detecting means further comprises vector quantity detecting means for detecting a vector quantity of a motion of each part of the body. 6. A virtual reality space service system in which one virtual reality space is used by a plurality of terminals.