JPH09244845A - Three-dimensional virtual space display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To give extent to virtual environment by saving the trouble of content generation when data constituting the virtual environment are prepared to the three-dimensional space display method which provides the virtual environment wherein plural persons join in a three-dimensionally displayed virtual space. SOLUTION: One of plural computers is decided as a server 2 and the remaining computes are connected as clients 1 by a network. etc. Each client 1 specifies and sends a face image and position information, or a real world image to the server 2. The server 2 send the position information and face image together to all clients or the real world image together with a table wherein a discrimination flag is set. Each client 1 arranges the face image at an avarter as user's double in the virtual environment or uses the real world image in the world of the virtual environment by arranging a background, objects, etc.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention creates a three-dimensionally displayed virtual space such as a town or a house on a computer such as a personal computer (PC) or a workstation (WS), a computer network or a public network. Involved in the technical field of building a virtual environment in which multiple people participate via
The present invention relates to a three-dimensional virtual space display technology in which a real-world image input by a participant is captured and used in a virtual environment.

[0002]

2. Description of the Related Art Conventionally, a three-dimensionally displayed virtual space such as a town or a house is created on a computer such as a PC or WS, and a virtual space in which a plurality of people (users) participate in this virtual space. There is technology to build the environment. In a virtual environment in which a plurality of people (users) participate in such a three-dimensionally displayed virtual space, it is necessary to prepare data such as backgrounds and objects for constructing the virtual environment in advance.

[0003]

However, in the above-mentioned conventional virtual space display, since it is necessary to previously create the data of the objects constituting the virtual environment, it is possible to use the problem that it takes time and effort to create the content. Since the data for constructing the virtual environment is limited, there is a problem that the virtual environment is not widened.

The present invention is intended to solve such a problem, and an object thereof is to allow a plurality of people to participate in a virtual space three-dimensionally displayed on a computer such as a PC or WS. When implementing a virtual environment, it is possible to save time and effort for content creation and to expand the virtual environment. 3
An object of the present invention is to provide a three-dimensional virtual space display method.

[0005]

In order to achieve the above object, the first aspect of the present invention provides a person image as an alter ego and a person image of another user in a three-dimensional virtual space on a computer. The three-dimensional virtual space display method realized by arranging the images includes the steps of inputting an image of a real environment and using the input image in a three-dimensional virtual space on the computer. The three-dimensional virtual space display method is as follows.

A second invention of the present invention is, in the three-dimensional virtual space display method of the first invention, that there are a plurality of computers having a three-dimensional virtual space, and the plurality of computers are connected via a network. In the step of inputting the image of the real environment, there is a process of distributing the input image of the real environment to the plurality of computers via a network, and the input image is stored in a three-dimensional virtual space on the computer. The step of capturing and using is characterized by including a process of capturing and using the distributed image in a three-dimensional virtual space on the distributed computer. The second aspect of the present invention is suitable for constructing a virtual environment by a computer system connected to a network.

A third invention of the present invention is the method for displaying a three-dimensional virtual space according to the second invention, wherein one of the plurality of computers is a server device and the rest are terminal devices, and the server device is a terminal device. The user's person image and position information are collected and distributed to the terminal devices, and the person image as their own image and the person image of another user are arranged in the three-dimensional virtual space on each terminal device, and input. In the process of distributing the image of the real environment to the plurality of computers via the network, the process of transmitting the image of the real environment input at the terminal device to the server device and the process of transmitting the image of the real environment transmitted from the terminal device to the server. The device distributes to a plurality of terminal devices, and in the process of using the distributed image in a three-dimensional virtual space on the distributed computer, the distributed image It characterized by having a process to be used is taken into three-dimensional virtual space of the distributed terminal devices. The third aspect of the invention is suitable in that one server unit manages user information in a unified manner.

The fourth invention of the present invention is the method of displaying a three-dimensional virtual space according to the third invention, wherein in the process of transmitting the image of the real environment input by the terminal device to the server device, the image of the real environment is displayed. In the process in which the image is transmitted together with the image type information for distinguishing between the person image and the person image, and the image of the real environment transmitted from the terminal device is distributed to the plurality of terminal devices by the server device, each image and each image type information are created. In the process of transmitting the table and using the distributed image in the three-dimensional virtual space of the distributed terminal device, the image arrangement is selected based on the image type information of the table distributed from the server device. It is characterized by doing. The fourth aspect of the invention is suitable in that it facilitates the image drawing process in the terminal device.

Further, a fifth invention of the present invention is the above-mentioned second invention.
In the three-dimensional virtual space display method of the invention described above, one of the plurality of computers is a server device and the other is a terminal device, and the server device collects the user's personal image and position information from the terminal device and distributes them to the terminal device. A process of arranging a person image as one's own self and a person image of another user in a three-dimensional virtual space on each terminal device, and distributing the input image of the real environment to a plurality of computers via a network Then, a process of forming a new path from a specific terminal device that inputs an image of a real environment to a terminal device that intends to distribute the image, and the process of directly forming the new path from the specific terminal device to the terminal device that has formed the new path. And a step of distributing an image of a real environment. This fifth invention is
It is suitable for quick communication between terminal devices.

According to the present invention described above, when a virtual environment in which a plurality of people participate in a virtual space displayed three-dimensionally on a computer such as a PC or WS, a real world input by one participant is realized. By capturing the image of the real environment into the virtual environment and using it as a part of the world of the virtual environment, it is easy to create new objects in the virtual environment and save the trouble of content creation. It is possible to expand the virtual environment by removing barriers and expanding the range of communication. Conventional technologies include:
There is no three-dimensional virtual space display method that realizes such a virtual environment.

[0011]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 shows a system configuration for realizing a virtual environment to which the present invention is applied. As shown in FIG. 1, this system is composed of a plurality of computers (PS, WS, etc.). One of these computers serves as the server 2 that manages the position information of the user in the virtual environment, and the rest. N of the clients 1-1 for users,
1-2, ... 1-n (hereinafter referred to as “1” when represented) are connected to each other by a network, a public line, or the like. The client 1 includes position information that changes according to an operation such as movement of the user in the virtual environment,
For example, a face image of the user is transmitted to the server 2 as a person image. The server 2 collects the position information and face images sent from the client 1, and transmits the position information and face image data for all users to all clients.

FIG. 2 shows an image diagram of a usage example of the present invention. Here, the client that captures and transmits the real-world image is referred to as the transmitting client, and the other clients that receive the image are referred to as the receiving clients. As shown in FIG. 2, the originating client captures a real-world image with a camera attached to the machine. The receiving client can handle the image input by the transmitting client in the virtual space realized on the computer in the same manner as other objects.

As a mechanism for the transmitting client to transmit an image, there are two methods of (1) via the server and (2) newly establishing a communication path. The processing flow in the case of (1) is shown in FIG. 4, and the processing flow in the case of (2) is shown in FIG. The contents of each process will be described later.

As a user interface for using the real-world image in the virtual space, a method of replacing all the backgrounds of the virtual environment with the received real-world image, and pasting it on an object in the virtual space is considered. To be

Further, when a camera is used, the only information input from the real world is a video, but it is possible to obtain additional information from an object in the real world by combining it with another sensor. As an example of implementation that does not depend on whether the image transfer between the sending client and the receiving client is via a server or a new communication path is established, the image is sent from the sensor attached to the book by communicating with the staff at the library window. It is possible to search for books and find related documents while receiving information. An image diagram of its use is shown in FIG. In FIG. 3, the library search user is at the place where the book is located, and the library staff is at a remote place. The user inputs the image of the bookshelf into the virtual environment and shares it with the staff. In the virtual environment, the clerk can point to the book that the user is looking for, teach the location of the book, and retrieve information related to the book recognized by communicating with the sensor.

<< Embodiment Example 1: Flow of processing in case of passing through server >> A flow chart of processing in the first embodiment of the present invention is as shown in FIG.

The method according to the first embodiment for transmitting the real-world image input from the transmitting client to other participants via the server is as follows. The server 2 transmits from each client 1 as shown in FIG. It is realized by using the mechanism that delivers the received face image.

Originally, the server 2 collects face images sent from the client 1 and sends them to the client. However, in the configuration of the present invention, the data sent from the server 2 is sent from the client 1. For example, a table summarizing the type flags of the video resources is created as shown in FIG. 6, and whether or not the input video from the client 1 is the face image of the user (in the case of no, it means the real world video). ) Is identified by the type flag.

First, the processing on the client 1 side in the first embodiment is as follows.

(1) Input Switching The video input from the client 1 may be a face image of the user in addition to the real world video.

Since the face image is processed so as to be attached to the avatar which is the alter ego in the virtual space of the user,
It needs to be distinguished from the real-world image used as a new object in the virtual space.

The client 1 confirms that the transferred image is not the user's self-portrait by the user's explicit operation such as the direction change of the camera and the button click.
Notify.

This corresponds to S1 in FIG.

(2) Image Input The user inputs an image of his / her surroundings by the camera attached to the machine of the client 1.

This corresponds to S2 in FIG.

(3) Image transmission The input image is transmitted to the server 2.

This corresponds to S3 in FIG.

(4) Image Receiving The image input from all the clients put together by the server 2 is received.

In addition to the video, the input video from each transmission client is identified by receiving the correspondence table (FIG. 6) indicating whether or not the image is the face image of the user.

This corresponds to S4 in FIG.

(5) Based on the image transmitted from the drawing server 2 and the correspondence table, the image from the identified transmission client is extracted.

If the face image is determined, it is drawn on the avatars of other participants based on the position information of the participants.

An image judged to be a real world image is drawn on a background or an object in the virtual space.

This corresponds to S5 in FIG.

Next, the processing on the server side in the first embodiment will be described below.

(1) Image switching By the notification from the client 1, the client 1
It is determined whether the image transmitted from is the user's face image.

If it is a face image, the received image is only delivered to another user, but if it is not a face image, a process for distinguishing it from an image from another client is performed.

This corresponds to S6 in FIG.

(2) Set a video type flag.

When it is determined that the image transmitted from the client 1 is not a face image, as shown in FIG. 6, the flag of the corresponding client portion in the correspondence table is set to input the image from another client. And identify.

This corresponds to S7 in FIG.

(3) Image reception The image is received from the client 1.

This corresponds to S8 in FIG.

(4) Creation of transmission image table After receiving images from all clients, a table of image data to be transmitted to the clients is created again.

This corresponds to S9 in FIG.

(5) Image transmission The table created by (4) on the server 2 side and containing the images from all clients is transmitted to each client side.

At this time, the client table created in (2) for identifying the image contents is transmitted together with the image data table.

This corresponds to S10 in FIG.

<< Embodiment 2: Processing flow for communication between clients >> FIG. 5 is a flow chart of the processing for delivering and using an image in the real world by the method of the second embodiment of the present invention. is there.

Normally, the face images of all the participants are transferred via the server, but by the operation of a specific client 1, the client 1 is transferred between the n-ground clients sharing the real-world image. Opens a new closed communication path and sends the surrounding real-world image to n people. Also, the end operation closes the connection between the clients and returns to normal face image transfer via the server.

First, the processing on the transmitting client 1 side is as follows.

(1) Input Switching The transmission client 1 inputs that the input image from the camera is not the face image of the user by explicitly changing the direction of the camera or clicking a button.

This corresponds to S11 in FIG.

(2) Creation of a new path When the input from the transmitting client 1 is an image other than a face image, the n-point members who share the image of the real world input from the transmitting client 1 are connected to the existing server. In addition to the communication port of
+1) Make a closed connection to people.

At this time, a limited method is conceivable in which the members to be shared are n persons in the vicinity as seen from the originating client 1 and users existing in a specific place (room) in the virtual space. To be

This corresponds to S12 in FIG.

(3) Background Object Creation An object for drawing the image of the real world input from the transmission client 1 is created in the virtual space.

A method of drawing on the entire background and a posting method of drawing on an object imitating a whiteboard or a window can be considered, but the input image is 2 in the virtual space.
Since it is a three-dimensional image, an object to be drawn is indispensable.

This corresponds to S13 in FIG.

(4) Image Input The user inputs an image of his or her surroundings with the camera attached to the machine of the transmitting client 1.

This corresponds to S14 in FIG.

(5) Image transmission The input image is transmitted to another client 1 '.

This corresponds to S15 in FIG.

(6) Drawing The input image is drawn on the background or object.

Corresponding to S16 in FIG. 5, the processing on the receiving client 1'side is as follows.

(1) Creation of a new path Upon receiving a notification from the originating client 1, a new communication path closed to members sharing the real world video input from the originating client 1 is established.

At this time, a limited method is conceivable in which the members to be shared are n neighbors from the viewpoint of the transmission client 1 and users existing in a specific place (room) in the virtual space. Get

This corresponds to S17 in FIG.

(2) Background Object Creation An object for drawing a real-world image input from the transmission client 1 is created in the virtual space.

A method of drawing on the entire background and a method of presenting by drawing on an object imitating a whiteboard or a window can be considered, but the input image is 2 in the virtual space.
Since it is a three-dimensional image, an object to be drawn is indispensable.

This corresponds to S18 of FIG.

(3) Waiting for reception Waiting for reception of the real world video from the transmission client 1. It is in the state of turning the loop.

This corresponds to S19 of FIG.

(4) Image reception The real world video from the transmission client is received.

This corresponds to S20 in FIG.

(5) Drawing Drawing the received image on the background object.

This corresponds to S21 in FIG.

[0079]

As described above, according to the present invention, a plurality of people participate in a virtual space displayed three-dimensionally on a computer such as a personal computer (PC) or a workstation (WS). It is possible to provide a mechanism for realizing a virtual environment in which the virtual environment is realized and using the real environment image captured from the real world in the virtual environment.

Therefore, it becomes easy to create a new object in the virtual environment, and the trouble of creating the content can be saved. Furthermore, by utilizing the information of the real world in the virtual environment, it is possible to expand the virtual environment by expanding the range of communication.

[Brief description of drawings]

FIG. 1 is a diagram showing a system configuration example for realizing a virtual environment for implementing the present invention.

FIG. 2 is an image diagram illustrating an example of a usage form of the present invention.

FIG. 3 is an image diagram illustrating an example of collection of books in a library as an example of the above-mentioned usage form.

FIG. 4 is a flow chart of processing in the case of passing through a server according to the first embodiment of the present invention.

FIG. 5 is a flow chart of processing when performing inter-client communication according to the second embodiment of the present invention.

FIG. 6 is a diagram showing a configuration example of a table of a correspondence relationship between a client name and video content in the first embodiment example of the present invention.

[Explanation of symbols]

 1-1, 1-1, ..., 1-n ... Client (computer) 2 ... Server (computer)

Claims (5)

[Claims] 1. A three-dimensional virtual space display method realized by arranging a person image as one's own self and a person image of another user in a three-dimensional virtual space on a computer, and inputs an image of a real environment. A three-dimensional virtual space display method, comprising: a step of capturing the input image in a three-dimensional virtual space on the computer for use. 2. A plurality of computers having a three-dimensional virtual space exist, the plurality of computers are connected via a network, and in the step of inputting an image of the real environment, the input image of the real environment is connected to the network. A step of distributing the input image to a three-dimensional virtual space on the computer and using the input image in the three-dimensional virtual space on the computer. The three-dimensional virtual space display method according to claim 1, further comprising a step of capturing and using it in the inside. 3. A plurality of computers, one of which is a server device and the other of which is a terminal device, and the server device collects the user's personal image and position information from the terminal device and distributes them to the terminal device. In the process of distributing the input image of the real environment to multiple computers via the network, the image of the person and the image of the other user are placed in the three-dimensional virtual space. The process of transmitting the image of the real environment to the server device and the process of distributing the image of the real environment transmitted from the terminal device to the plurality of terminal devices by the server device. The process of capturing and using the distributed image in the three-dimensional virtual space on the computer may include the process of capturing and using the distributed image in the three-dimensional virtual space of the distributed terminal device. 3-dimensional virtual space display method according to claim 2, wherein. 4. In the process of transmitting the image of the real environment input at the terminal device to the server device, the image is transmitted together with image type information for distinguishing the image of the real environment from the image of the person, and the real image transmitted from the terminal device. In the process in which the server device distributes the environment image to the plurality of terminal devices, each image and the table created for each image type information are transmitted, and the distributed image is distributed within the three-dimensional virtual space of the distributed terminal device. 4. The three-dimensional virtual space display method according to claim 3, wherein the arrangement of images is selected based on the image type information of the table distributed from the server device in the process of capturing and using the image. 5. A plurality of computers, one of which is a server device and the other of which is a terminal device, the server device collects the user's personal image and position information from the terminal device and distributes them to the terminal device. In the process of distributing the input image of the real environment to multiple computers via the network, the image of the person and the image of the other user are placed in the three-dimensional virtual space. A step of forming a new path from a specific terminal device to which the image is to be distributed to a terminal device that intends to distribute the image, and directly distributing the image of the real environment from the specific terminal device to the terminal device that has formed the new path. 3. The three-dimensional virtual space display method according to claim 2, further comprising: