JPH09233445A - Communication controller - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a simple conversation environment merging a virtual world with a real world. SOLUTION: At least one or more terminal equipments constituted of an input/output part 10 for inputting/outputting information, a communication part 11 for transmitting information and a terminal control part 12 for controlling and managing these respective parts are managed and controlled by a terminal supporter constituted of a virtual world managing part 14 for managing information concerning the virtual world, a sound field managing part 15 for managing information concerning sound field information, a communication part 16 for transmitting information and a terminal support control part 17 for controlling and managing these respective parts. Thus, the plural users share the virtual world through a network so that the environment close to the place of real communication can be obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a communication control device that realizes a conversation environment by fusing a virtual world and a real world.

[0002]

2. Description of the Related Art Conventionally, a virtual world constructed by three-dimensional computer graphics is shared by a large number of users through a network to provide a place of communication that is close to reality, to meet people, to communicate with information. A system has been proposed to provide a new communication environment that supports people's encounters (for example, Tanigawa et al .:
"Voice control method in multi-person participation type virtual environment", IEICE, IEICE Technical Report, IE95-43 (1995)).

Specifically, a variety of voice environments are realized by utilizing the coordinate positions of the user's alter ego in the virtual space transmitted and received between the user terminals and the direction of the line of sight. Even when moving in a virtual space where many users are present, a voice mixing function that allows listening to surrounding sounds without interruption, controlling the start and end of conversation according to the distance between the ghosts and the line of sight, the degree of coincidence of the line of sight, It describes a method in which up to four users can participate in the same conversation, and even if three or more users drop out during the conversation, the conversation between the remaining users is not adversely affected.

[0004]

However, in the conventional technique, the sound is mixed (synthesized) by utilizing the coordinate position of the user's alter ego in the virtual space and the direction of the line of sight. Therefore, it has been impossible for a user's alter ego to speak from behind another user's alter ego in the virtual space, or to call another user's alter ego according to the volume of a user's voice. Also, no consideration was given to the video transmission method. It should be noted that the virtual world targeted by the present invention is not limited to one configured by three-dimensional computer graphics, but one configured by a two-dimensional planar animation, a model of a remotely operable robot or miniature. May be

[0005]

SUMMARY OF THE INVENTION The present invention is a terminal device comprising input / output means for inputting / outputting information, communication means for transmitting information, and terminal control means for controlling and managing each of these means. A virtual world management means for managing information about a virtual world, a sound field management means for managing information about sound field information, a communication means for transmitting information, and a terminal support control means for controlling and managing each of these means. By managing and controlling at least one or more terminal support devices configured by, a virtual world is shared by users via a network, thereby providing an environment close to a real place of communication.

[0006]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a schematic configuration diagram of a centralized management type communication control device according to an embodiment of the present invention. FIG.
As shown in (a), the terminal device includes the following parts.

In the figure, the input / output unit 10 includes a camera, a microphone,
It is composed of an input means such as a scanner, a mouse and a keyboard, and an output means such as a printer and a display. The communication unit 11 includes a coaxial cable, CATV, LAN, modem,
It is a means for transmitting information such as an exchange. Terminal control unit 12
Is a means for controlling and managing the input / output unit 10 and the communication unit 11. As shown in FIG. 1B, the terminal support device is composed of the following parts. In the figure, the virtual world management unit 14 is 3
It is a means for managing information about a virtual world represented on a computer by two-dimensional or two-dimensional computer graphics.

The sound field management unit 15 is means for mixing (synthesizing) sounds input from the terminal device or provided in the terminal support device. The sound field management unit 15 manages the voice synthesis state. The communication unit 16 is means for transmitting information. The terminal support control unit 17 is means for controlling and managing each of these units. The terminal device is managed by the terminal support device, and a sound input from the terminal device or a sound included in the terminal support device is synthesized by the sound field management unit 15 based on the information managed by the virtual world management unit 14, and processed. The sound is transmitted to the target terminal device of (1).

FIG. 2 is a schematic configuration diagram of a communication control device (distributed management type) according to the embodiment of the present invention. FIG.
As shown in (a), the terminal device includes the following parts. The input / output unit 21 in the figure includes input means such as a camera, a microphone, a scanner, a mouse and a keyboard, a printer,
It consists of output means such as a display. Communication unit 22
Is a means for transmitting information such as a coaxial cable, a CATV, a LAN, a modem, an exchange. Virtual World Management Department 23
Is a means for managing information about a virtual world represented on a computer by three-dimensional or two-dimensional computer graphics.

The sound field management unit 24 is means for mixing (synthesizing) sounds input from the terminal device or provided in the terminal support device. The sound field management unit 24 manages the voice synthesis state. In the configuration of FIG. 2, each terminal device is logically connected in a ring shape. Therefore, information of voice and control data exchanged between terminals is transmitted in a bucket brigade type (the physical form of the network may not be a ring type, but may be a bus type network such as Ethernet). ). As a result, in FIG. 2, compared to FIG. 1, the virtual world and the sound field control are managed in a distributed manner, and the load concentration can be reduced. On the other hand, since the information in FIG. 1 is centrally managed, the labor for maintenance can be reduced as compared with FIG.

FIG. 3 is a diagram for explaining the virtual world. The virtual world is represented on a computer by three-dimensional or two-dimensional computer graphics. By sharing a virtual world formed on a computer by a large number of users via a network, it is possible to provide a place close to actual communication, and to meet people and information. In the example of the figure, character A and character B exist in the virtual world, and user A and user B operate their respective characters from their respective terminal devices. The sound sources emitted by character A and character B are the conversation sounds of user A and user B, respectively.
Further, the sound source A and the sound source B are environmental sounds such as a human voice and a running sound of a car. In the virtual environment, there are walls that prevent sound propagation. Information about the virtual world such as characters and walls is managed by the virtual world management department. In the example of the figure, the volume of the sound is represented by a dotted line, and the sound field creation unit synthesizes the sound according to the crossing state of the sounds. Information about the size of the sound source and the position of the sound source is managed by the sound field management unit.

4 to 7 are diagrams showing information managed by the virtual world management unit and the sound field management unit. FIG. 4 shows information about characters managed by the virtual world management unit. In the example of the figure, the character name and the coordinate information in the virtual space where the character exists are managed. In addition to the example shown in the figure, information such as character size, color tone, user name, and terminal device name operating the character may be managed.

FIG. 5 shows information about characters other than those managed in the virtual space. In the example of the figure, information about an obstacle (wall) existing in the virtual space is managed, and information about the arrangement coordinates of the wall and information about transparency indicating how much the sound propagation is disturbed is managed. As information other than the character and the wall, what is targeted in the real world such as buildings, animals, and cars may be targeted in the virtual space.

FIG. 6 shows information managed by the sound field management section. In the example of the figure, information about the sound source name (for example, a character, a car running sound, a voice), the position of the sound source, and the size of the sound source is managed. The sound source is represented by the radius of the circle, and the sound field management unit determines whether or not to mix the sound, depending on the intersecting state of the circles. In the illustrated example, the sounds of the character A, the character B, and the sound source B are mixed (synthesized). Note that the sound intensity may be set freely by the user, and may be a sound source that attenuates according to the distance from the center of the sound source as shown in FIG.

FIG. 8 is a diagram showing the sound field management unit. As shown in the figure, sound synthesis is performed by the sound field creation unit, and if a new crossing state of sound sources is created, the sound field creation unit is created and processed accordingly (in the case of hardware implementation. Is
A new audio mixing processor may allocate for processing). In the illustrated example, the sounds of the character A, the character B, and the sound source B are combined in the sound field creation unit A. The sound field creation unit may synthesize sounds by software or may be realized by hardware.

FIG. 9 is a diagram for explaining the processing procedure performed by the sound field management unit. As a procedure, first, it is checked whether the crossing state of the sound sources has changed due to the movement of the sound source such as a character or the size of the sound source changing (S60).
1). If the newly intersected sound source has already been subjected to sound field creation processing (sound synthesis), a sound field is added, and if not, a new sound field creation processing unit is provided (S602). When the intersection with the sound source that has been intersected so far is canceled, the sound field at which the intersection is resolved is deleted, and the target of the sound field creation processing is 1
When they become one, the sound field creation process is terminated. The situation of these processes is managed by the sound field management unit.

FIG. 10 is a diagram for explaining the information managed by the sound field management unit. The sound field management unit manages the names of the sound production units that are currently operating and the information about the sound source processed by each sound production unit. FIG. 11 is a diagram showing a display form of a video managed by the video management unit.
When there are a plurality of video channels, the following video display modes are available. FIG. 11A shows a method of simultaneously displaying a plurality of videos on a plurality of screens, and FIG.
Shows a method of combining a plurality of videos into one screen and displaying the same, and FIG. 11C shows a method of sequentially switching and displaying the plurality of videos.

The display mode of the image is determined based on the communication capacity that the communication unit can transmit, or based on the operation status of the user's terminal (for example, the degree of load on the terminal operated by the user). You may decide. The video display method and the determination of the display method are performed by the video management unit described later (information such as the number of frames per second of the video and the resolution of the video may also be managed). The target image may be a face of a person who operates the terminal or a document image.

FIG. 12 is a block diagram of a communication control device (centralized management type) capable of video transmission. As shown in FIG. 12A, the terminal device includes the following parts. The input / output unit 91 in the figure includes input means such as a camera, a microphone, a scanner, a mouse and a keyboard, and output means such as a printer and a display. The communication unit 92 is means for transmitting information such as a coaxial cable, a CATV, a LAN, a modem, an exchange. The terminal control unit 93 is means for controlling and managing the input / output unit 91 and the communication unit 92. FIG. 12 (b)
As shown in, the terminal support device is configured by the following parts. The virtual world management unit 94 is means for managing information about a virtual world represented on a computer by three-dimensional or two-dimensional computer graphics.

The sound field management unit 95 is means for mixing (synthesizing) sounds input from the terminal device or provided in the terminal support device. The sound field management unit 95 manages the voice synthesis state. The communication unit 96 is means for transmitting information. The video management unit 97 determines a video display method and a display method. The terminal support control unit 98 is means for controlling and managing each of these units. The terminal device manages the sound / video that is managed by the terminal support device and that is input from the terminal device or the sound / video that is included in the terminal support device based on the information managed by the virtual world management unit 94. The image is synthesized by the image management unit 97, and the sound is transmitted to the terminal device to be processed (FIG. 12 (c)).

FIG. 13 is a diagram showing a schematic configuration of a communication control device (distributed management type) capable of video transmission. FIG.
As shown in (a), the terminal device includes the following parts. The input / output unit 1001 in the figure is composed of input means such as a camera, a microphone, a scanner, a mouse and a keyboard, and output means such as a printer and a display. The video management unit 1005 determines a video display method and a display method. The communication unit 1004 is a coaxial cable, CATV, LA.
It is a means for transmitting information such as N, modem, and exchange.

The virtual world management unit 1002 is means for managing information about a virtual world represented on a computer by three-dimensional or two-dimensional computer graphics.
The sound field management unit 1003 is means for mixing (synthesizing) sounds input from the terminal device or provided in the terminal support device. The sound field management unit 1003 manages the voice synthesis state. In the configuration of FIG. 13, each terminal device is logically connected in a ring shape. Information such as video, audio, and control data exchanged between terminals is transmitted in a bucket brigade manner. Accordingly, in FIG. 13, the virtual world, the sound field control, and the image generation are distributed and processed, so that the concentration of the load can be reduced as compared with FIG. On the other hand, since the information in FIG. 12 is centrally managed, the labor for maintenance can be reduced as compared with FIG.

FIG. 14 is a diagram for explaining the video management unit. As shown in the figure, the management of the video display format is performed by the video creation unit, and if a new crossing state of sound sources is created,
The image creation unit is created and processed accordingly (when implemented by hardware, a new processor is assigned for image display processing). In the illustrated example, the video is transmitted from each of the terminal devices operating the character A and the character B, and the video creating unit A performs the video display processing (see FIG. 11). The video creation unit may be processed by software or may be realized by hardware.

FIG. 15 is a diagram for explaining the information managed by the video management unit. The video management unit manages the video creation unit that manages the video display format, the video to be processed, and information about the video display format.

The sound field management unit manages the information about the direction in which the sound is heard (for example, the volume of the left and right speakers) to express the perspective, or the users who have decided in advance according to the user's request. The sound field synthesis and the image synthesis may always be performed. Specific applications of the present invention include:
Examples include games using the network, shopping, karaoke, meetings, and distance education. Further, the transmission line is not limited to a wired line, and may be a wireless line (for example, PHS, wireless LAN, terrestrial wave, satellite, etc.) considering the use of a portable computer. Finally, even in the case of the centralized management type described in FIGS. 1 and 12, even if the virtual world for displaying the virtual world is held on the terminal device side and the virtual world is created. Good.

[0027]

As described above, according to the present invention, a user can share a virtual world via a network to provide a place for communication that is close to reality, and to help meet people and information. A communication environment can be realized.

[Brief description of drawings]

1A to 1C are schematic configuration diagrams of a communication control device (centralized management type) according to an embodiment of the present invention.

2A and 2B are schematic configuration diagrams of a communication control device (distributed management type) according to an embodiment of the present invention.

FIG. 3 is a diagram showing a virtual world

FIG. 4 is a diagram showing information managed by a virtual world management unit and a sound field management unit.

FIG. 5 is a diagram showing information managed by a virtual world management unit and a sound field management unit.

FIG. 6 is a diagram showing information managed by a virtual world management unit and a sound field management unit.

FIG. 7 is a diagram showing information managed by a virtual world management unit and a sound field management unit.

FIG. 8 is a diagram showing a sound field management unit.

FIG. 9 is a diagram showing a processing procedure performed by a sound field management unit.

FIG. 10 is a diagram showing information managed by a sound field management unit.

11A to 11C are diagrams showing a display form of a video managed by a video management unit.

12A to 12C are configuration diagrams of a communication control device (centralized management type) capable of video transmission.

13A and 13B are schematic configuration diagrams of a communication control device (distributed management type) capable of video transmission.

FIG. 14 is a diagram showing a video management unit.

FIG. 15 is a diagram showing information managed by a video management unit.

[Explanation of symbols]

 10 input / output unit 11 communication unit 12 terminal control unit 13 terminal device 14 virtual world management unit 15 sound field management unit 16 communication unit 17 terminal support control unit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location H04M 3/56 G06F 15/62 Z // G09G 5/00 510 G10K 15/00 M 9466-5K H04L 11/18

Claims (13)

[Claims] 1. A terminal device comprising an input / output unit for inputting / outputting information, a communication unit for transmitting information, and a terminal control unit for controlling and managing each of these units, managing information about a virtual world. A terminal support device including a virtual world management means, a sound field management means for managing information on sound field information, a communication means for transmitting information, and a terminal support control means for controlling and managing each of these means. A communication control device characterized by managing and controlling at least one or more. 2. An input / output unit for inputting / outputting information, a communication unit for transmitting information, a virtual world managing unit for managing information about a virtual world, and a sound field managing unit for managing information about sound field information. And a terminal control means for managing and controlling each of these means. 3. A terminal device comprising an input / output unit for inputting / outputting information, a communication unit for transmitting information, and a terminal control unit for controlling and managing each of these units, managing information about a virtual world. Virtual world management means, sound field management means for managing information about sound field information, video information management means for managing video information, communication means for transmitting information, and control and management of these means. A communication control device, characterized in that at least one terminal support device composed of terminal support control means manages and controls. 4. An input / output unit for inputting / outputting information, a communication unit for transmitting information, a virtual world managing unit for managing information about a virtual world, and a sound field managing unit for managing information about sound field information. And a video management unit for managing video information, and a terminal control unit for managing and controlling each of these units. 5. An obstacle that obstructs sound transmission is provided in the virtual space, and the degree of sound transmission is managed according to the obstacle,
It controls, The communication control apparatus in any one of Claims 1-4. 6. The method according to claim 1, wherein, when there are a plurality of sound sources, whether to synthesize the sound of the sound source is determined based on the size of the sound source. Communication control device. 7. A sound field creating means for performing processing for synthesizing sounds of a sound source is provided each time a sound field is created, and the sound field managing means manages and controls the sound field. 4. The communication control device according to any one of 4 to 4. 8. When a plurality of the communication control devices are present, the information transmission paths between the communication control devices are connected in a ring shape, and the information is transmitted in a bucket relay manner. Alternatively, the communication control device according to item 4. 9. A video creating means for managing the display format of the video information is provided each time a new sound field is generated, and the video managing means manages and controls the video information. The communication control device according to claim 3 or 4. 10. The communication control device according to claim 3, wherein a plurality of images are displayed on a plurality of screens simultaneously, or combined into one screen, or sequentially switched and displayed. 11. The communication control apparatus according to claim 3, wherein the display form of the image is determined based on the communication capacity that the communication means can transmit or the operating condition of the user's terminal. 12. The communication control apparatus according to claim 1, wherein the sound field management means manages information about a direction in which sound is heard. 13. The communication control device according to claim 1, wherein a sound field is synthesized in response to a user's request.