JP2004056161A - Multimedia communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a multimedia communication system for providing a sense to users as if a plurality of spaces geometrically apart from each other were present adjacent to each other. <P>SOLUTION: Servers S2, S1 exchange video and audio information (multimedia data) of opposite rooms A, B with each other, a projector 21 of the room A(B) displays the video image of the opposite room B(A) on a screen 22 and a loudspeaker 23 of the room A(B) sounds the sound of the opposite room B(A) at the same time. Further, an object (mainly a person H1(H2) present in the room A(B)) displayed on the screen 22 is a full size (life size) picture nearly equal to the size of the actual object. Then virtual rooms (shared virtual space) B', A' are respectively formed adjacent to the two rooms A, B geometrically apart from each other, the people H1, H2 respectively present in the respective rooms A, B feel as if the opposite party were in the virtual adjacent rooms B', A' and can share the virtual spaces although the people H1, H2 are present in places apart from each other. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a multimedia communication system for transmitting and receiving multimedia data such as audio and video between a plurality of communication devices.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a multimedia communication system for transmitting and receiving multimedia data such as audio and video between a plurality of communication apparatuses has been used for the purpose of communicating the state of two geographically separated places with video and audio. . Examples of this type of multimedia communication system include a videophone system that simultaneously transmits audio and video using a telephone line, and a moving image captured by a camera and audio collected by a microphone over a network connected to a computer. There are already provided a system for transmitting data to another computer in real time, a system for attaching a video letter obtained by compressing video and audio recorded in a video to an e-mail or the like, and transmitting the same.
[0003]
[Problems to be solved by the invention]
However, the conventional system as described above has the following problems.
[0004]
First, a system that sends and receives a video letter does not transmit video and audio in real time, so it is not possible to grasp the state of the two places in real time, and the people in each place are able to communicate with each other. Could not be realized.
[0005]
In a video and audio transmission system using a videophone system or a computer, the mutual situation is generally transmitted by a video image projected on a small monitor screen and a sound heard from a speaker. Although it is possible to grasp in real time, it has been difficult to give the people in each place the feeling that the other person is right next to each other.
[0006]
The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a multimedia communication system capable of giving a sense as if a plurality of geographically separated spaces exist adjacent to each other. Is to do.
[0007]
[Means for Solving the Problems]
In order to achieve the above object, the invention according to claim 1 includes a plurality of communication devices that are installed in a plurality of geographically separated spaces and exchange multimedia data including audio and video data via a communication line. A plurality of data processing devices installed in each space and respectively processing multimedia data transmitted and received between communication devices, an imaging unit including each space in an imaging region, and a sound collection unit collecting sound in each space Each data processing device comprises: a video presentation unit that presents video information processed by the data processing device in the space; and a sound generation unit that emits audio information processed by the data processing device into the space. The image presentation means is characterized by presenting an image of the subject with a size substantially equal to the actual size.
[0008]
According to a second aspect of the present invention, in the first aspect, a communication device and a data processing device are installed in two specific spaces.
[0009]
According to a third aspect of the present invention, in the first aspect of the present invention, the video presenting means individually presents video information in a plurality of spaces, and the sounding means emits audio information in a plurality of spaces.
[0010]
According to a fourth aspect of the present invention, in the first, second, or third aspect, the communication device constantly exchanges multimedia data.
[0011]
According to a fifth aspect of the present invention, in any one of the first to fourth aspects, one communication device communicates only with a specific communication device.
[0012]
According to a sixth aspect of the present invention, in any one of the first to fifth aspects, each data processing device includes an illuminance measurement unit that measures illuminance of an imaging region by the imaging unit, and the data processing device measures the illuminance by the illuminance measurement unit. The communication device transmits the illuminance information to the communication device of the other party, and the video presenting unit adjusts the brightness of a presentation screen for presenting the video information based on the illuminance information received by the communication device.
[0013]
According to a seventh aspect of the present invention, in any one of the first to sixth aspects, each data processing device includes a volume measuring unit that measures a volume of the sound collecting unit at the time of sound collection. The communication device transmits the volume information measured by the communication device to the communication device of the other party, and the sounding means adjusts the volume at which the voice information is emitted based on the volume information received by the communication device.
[0014]
According to an eighth aspect of the present invention, in the invention according to any one of the first to seventh aspects, the data processing device is provided with a sound source position specifying means for specifying a position of the sound source in the space, and the data processing device is specified by the sound source position specifying means. The generated sound source position information is transmitted to the communication device of the other party by the communication device, and the sound generator reproduces the sound source position based on the sound source position information received by the communication device.
[0015]
According to a ninth aspect of the present invention, in any one of the first to eighth aspects of the present invention, a measurement unit for measuring a physical quantity relating to an environment in a space and a measurement result for displaying a measurement result measured by a measurement unit in another space. And a display means provided in the data processing device.
[0016]
According to a tenth aspect of the present invention, in any one of the first to ninth aspects, an air conditioner having at least a function of adjusting a temperature is installed in a space, and a temperature measuring means for measuring a temperature in the space, The air conditioner control means for controlling the operation of the air conditioner in accordance with the information on the temperature measured by the space temperature measuring means is provided in the data processing device.
[0017]
An invention according to claim 11 is the invention according to any one of claims 1 to 10, wherein an air conditioner having at least a function of adjusting humidity is installed in the space, and humidity measuring means for measuring the humidity in the space is provided. An air conditioner control means for controlling the operation of the air conditioner in accordance with information on the humidity measured by the space humidity measuring means is provided in the data processing device.
[0018]
According to a twelfth aspect of the present invention, in any one of the first to eleventh aspects, there is provided a data in which a contact-type temperature sensor or a tactile sensor installed at a location where an image is presented by an image presentation means is installed in any space. A heating element provided in the processing device and transmitting the temperature data detected by the temperature sensor or the detection data of the tactile sensor together with the multimedia data, and a heating element installed at a location where an image is presented by the image presentation means, and the temperature data or A heating element control means for controlling a heating value of the heating element based on the detection data is provided in a data processing apparatus installed in any other space.
[0019]
According to a thirteenth aspect of the present invention, in any one of the first to twelfth aspects, an agent for providing a service to a user is provided in each data processing device, and the data of each agent is transmitted and received together with the multimedia data. It is characterized in that it is shared between data processing devices.
[0020]
According to a fourteenth aspect of the present invention, in any one of the first to thirteenth aspects, each of the data processing devices includes a biological information measuring unit that measures biological information of a person in a space, and the data processing device is configured to perform the biological information measurement. Each data processing device is provided with a biological information display unit that transmits the biological information measured by the means to the communication device of the other party by the communication device and displays the biological information received by the communication device.
[0021]
BEST MODE FOR CARRYING OUT THE INVENTION
(Embodiment 1)
Hereinafter, Embodiment 1 of the present invention will be described in detail with reference to FIGS.
[0022]
As shown in FIG. 1, a plurality of (two in this embodiment) geographically separated spaces, for example, rooms A and B of two dwellings located at remote locations also serve as a communication device and a data processing device. Servers S1 and S2 are installed, and these two servers S1 and S2 are connected to each other by a communication line Ls. As the communication line Ls connecting the two servers S1 and S2, it is desirable to use a broadband (so-called broadband) line such as xDSL, CATV, or optical fiber suitable for transmitting multimedia data.
[0023]
In each of the rooms A and B, microphones 20 are installed at four corners near the ceiling, and a screen 22 is arranged on one entire wall surface, and a projector 21 is installed behind the screen 22. Further, a pair of speakers 23 is installed at both corners near the screen 22 on the ceiling of each of the rooms A and B, and a television camera 24 is installed behind the wall where the screen 22 is arranged. A small hole 22a is formed in the wall and the screen 22, and the inside of the rooms A and B is imaged by the television camera 24 through the small hole 22a. Here, as shown in FIG. 2, a small hole 22a is provided at the center in the horizontal direction of the screen 22 (wall surface) and near the eye level when the main persons H2 and H1 who use the other rooms A and B stand. Has formed. Further, by adjusting the position of the projector 21, the magnification of the optical system, and the like, the persons H2, H1 and the like in the other rooms A and B are projected on the screen 22 at substantially the same size. Further, an illuminance sensor 25 for measuring the illuminance of the rooms A and B and a temperature sensor 26 for measuring the room temperature are provided on the wall surface facing the screen 22, and the other rooms A and B measured by the temperature sensor 26. A display device 27 for displaying the temperature or the like is installed on another wall surface.
[0024]
On the other hand, the servers S1 and S2 installed in the rooms A and B have the same configuration, and are configured by mounting a dedicated program on a general-purpose computer, and as shown in FIG. , A data processing unit 2 for processing multimedia data transmitted and received by the communication processing unit 1, and a video input unit for capturing a video (video signal) captured by the television camera 24 3, a video output unit 4 for outputting video information to the projector 21, an audio input unit 5 for receiving audio (audio signal) collected by the microphone 20, and an audio information (audio signal) to the speaker 23. It comprises an audio output unit 6, an illuminance input unit 7 for receiving the sensor output of the illuminance sensor 25, a temperature input unit 8 for receiving the sensor output of the temperature sensor 26, and a liquid crystal panel. And a LCD driver 9 to display the temperature information or the like by driving the display device 27. Here, the voices referred to in the present invention include not only human voices but also general sounds heard by humans in daily life, such as sounds of animals, operating sounds of machines, and sounds of wind and rain.
[0025]
The video input unit 3 converts the video signal captured from the television camera 24 into digital video information (video data) and outputs the digital video information to the data processing unit 2, and the audio input unit 5 converts the audio signal captured from each microphone 20 into a digital signal. It is converted into audio information (audio data) and output to the data processing unit 2. The illuminance input unit 7 and the temperature input unit 8 convert the sensor outputs fetched from the illuminance sensor 25 and the temperature sensor 26 into digital illuminance information (illuminance data) and temperature information (temperature data), respectively. Output to The data processing unit 2 captures video information, audio information, illuminance information, and temperature information output from the video input unit 3, the audio input unit 5, the illuminance input unit 7, and the temperature input unit 8, and performs various types of processing described below. And outputs the result to the communication processing unit 1. The communication processing unit 1 appropriately compresses the data of each information received from the data processing unit 2, and transmits the data to the partner servers S2 and S1 via the communication line Ls according to a predetermined communication protocol.
[0026]
Further, the communication processing unit 1 receives the data of each information transmitted from the partner servers S2 and S1 through the communication line Ls, expands the received data as appropriate, and outputs the data to the data processing unit 2. Then, the data processing unit 2 processes the data of each information received from the communication processing unit 1, outputs video information to the projector 21 through the video output unit 4 to project a video on the screen 22, and outputs the video through the audio output unit 6. By outputting voice information to the speaker 23, a voice is generated, and further, temperature information is output to the display device 27 through the LCD drive unit 9, and the temperature information is displayed on the screen of the display device 27. The communication processing unit 1 and the data processing unit 2 always perform each processing without delay, and the multimedia server (including illuminance information and temperature information) is almost real-time between the two servers S1 and S2. Exchange is performed.
[0027]
Thus, between the servers S2 and S1, video and audio information (multimedia data) of the rooms A and B of the other party are transmitted and received, and the images of the rooms A and B of the other party are projected on the screen 22 by the projector 21. The voices of the rooms A and B of the other party are sounded from the speaker 23, and the subjects (mainly persons H1 ′ and H2 ′ in the rooms A and B) projected on the screen 22 by the projector 21 are almost the actual subjects. Since they have the same size (life-size), the state of two geographically separated spaces (rooms A and B) is transmitted in real time by voice and video exchanged between the servers S2 and S1, and the geographically separated spaces are separated. It is possible to give the persons H1 and H2 in the rooms A and B a feeling as if the persons H1 and H2 of the other party are beside each other. Further, in this embodiment, since the temperature information of the rooms A and B of the partner is displayed on the display device 27, it is possible to grasp the environment of the rooms A and B of the partner in detail. However, the information displayed on the display device 27 is not limited to the temperature, but may be any physical quantity (for example, humidity or the like) related to the environment of the rooms A and B.
[0028]
Here, in the present embodiment, various measures described below are taken in order to increase the reality of the image projected on the screen 22 and the sound heard from the speaker 23.
[0029]
First, the data processing unit 2 adjusts the brightness (brightness) of an image projected on the screen 22 to the actual brightness of the rooms A and B of the other party based on the illuminance information included in the received data and the information of the calibration performed when the system is installed. The video information adjusted to match is output to the projector 21 through the video output unit 4. Then, the brightness is adjusted in accordance with the video information received by the projector 21 and the video is projected on the screen 22, so that the brightness in the rooms A and B of the other party can be recognized with a bodily sensation and the reality of the video increases. It is.
[0030]
In the calibration when the system is installed, the brightness of the room is changed in a plurality of steps on the sender side of the image, and in each step, the illuminance measured by the illuminance sensor 25 on the sender side and the illuminance near the screen 22 on the receiver side are changed. The brightness of the projector 21 is adjusted so that the measured value matches. Further, the relation between the illuminance measurement value on the sender side and the luminance of the image projected on the screen 22 is determined by a linear function approximated by a linear function based on the illuminance measurement values on the sender side and the receiver side at the time of calibration, The data processing unit 2 corrects the brightness of the video information based on the illuminance information and outputs the corrected brightness to the projector 21 through the video output unit 4 using the relational expression. As a result, the brightness (brightness) of the image projected on the screen 22 during operation of the system can be made to substantially match the actual brightness of the rooms A and B of the other party.
[0031]
When the system is installed, calibration for sound image localization is also performed. That is, as shown in FIG. 4, each side of the rooms A and B is divided into four equal parts, and the signal sounds are individually separated at a total of 27 points P excluding points on each side among the intersections of the bisectors. A table showing the correspondence between the phase difference between audio signals collected by a plurality of (four) microphones 20 installed in the rooms A and B at that time and the point P at which the signal sound is generated. It is created and stored in the data processing unit 2. That is, when the system is operating, the data processing unit 2 can identify the position of the sound source by comparing the phase difference of the audio signals collected by the microphones 20 with the above-mentioned table. Also serves as. The data processing unit 2 transmits the specified position information of the sound source (sound source position specifying information) to the partner servers S1 and S2 through the communication processing unit 1 together with the video information and the audio information.
[0032]
The data processing unit 2 of the server on the receiving side (for example, S1) performs frequency conversion of the audio signal using, for example, a conventionally known head-related transfer function, and outputs the audio signal after the frequency conversion through the audio output unit 6. By outputting to the speaker 23, there is a virtual room B ′ next to the screen 22 as shown in FIG. 5A, and the sound source position PB of the actual room B (see FIG. 5B). Increases the reality of the sound information reproduced by making the person H1 in the room A feel as if it can be heard from the corresponding sound source position PB 'in the virtual room B' indicated by the sound source position specifying information. be able to.
[0033]
Similarly, in the calibration at the time of system installation, a signal sound having a constant sound pressure (dB) is generated in a plurality of stages near the center of the room on the sender side, and the speaker on the receiver side is generated at each stage. The volume of the speaker 23 is adjusted such that the sound pressure of the sound collected by the microphone 20 when the sound emitted from the microphone 23 is collected by the microphone 20 matches the sound pressure of the sound collected by the microphone 20 on the sender side. Further, the data processing unit obtains a relational expression which approximates the relationship between the volume of the sender side and the volume of the speaker 23 of the receiver side by a linear function based on the sound pressure levels of the sender side and the receiver side at the time of calibration. 2 is stored.
[0034]
On the other hand, when the system is operating, the data of the volume (the sound pressure level of the audio signal input from the microphone 20) measured by the audio input unit 5 is transmitted to the other servers S1 and S2, and the receiving servers S1 and S2 perform data processing. The unit 2 corrects the volume according to the above relational expression and outputs an audio signal to the speaker 23 through the audio output unit 6. As a result, the volume of the sound emitted from the speaker 23 during the operation of the system can be made to substantially match the actual volume of the rooms A and B of the other party.
[0035]
With the configuration described above, virtual rooms (shared virtual spaces) B ′ and A ′ can be formed next to the two rooms A and B that are geographically separated from each other. Persons H1 and H2 in B feel as if each other exists in virtual neighboring rooms B 'and A', and they can virtually share space while actually being away from each other. It becomes. For example, if one room A is an elderly home where one person lives and the other room B is a child's home, the child does not monitor the parent unilaterally as in the conventional monitoring system, but rather the parent (elder). Children can watch each other while making them feel as if they are living together in daily life. Alternatively, if one room A is a hospital room of an inpatient and the other room B is a living room of the patient's home, even if a family member cannot be present in the hospital room, the hospital room and the living room of the home exist as if they are next to each other. The family can watch the state of the patient while feeling as if they are doing it, and the patient can have a sense of security that the family is always beside him. If one room A is a store and the other room B is a police station where police officers are present and a guard is present, the police officers or security officers should constantly monitor the inside of the store and talk with the staff. Can strengthen the security system. Further, by setting the classrooms of the schools in the depopulated area to the rooms A and B, respectively, the number of students can be virtually increased, and the students can easily exchange with each other. Alternatively, if one room A is a sales floor of a store that sells food, and the other room B is a production site of foodstuffs sold in the store, a person who purchases food at the store may be able to use the food production site or It is possible to know the state of the producer and increase the sense of security of the purchaser for the food, and the producer can grasp the reaction of the purchaser and interact with the purchaser.
[0036]
By the way, in the above description, servers S1 and S2, which also serve as a communication device and a data processing device, are installed in rooms A and B of two houses geographically separated from each other, and these two servers S1 and S2 are connected to each other via a communication line Ls. , The two specific spaces (rooms A and B) virtually share the space. However, as shown in FIG. A camera 24 is installed, and multimedia data is exchanged between the room A and servers S2 installed in a plurality of geographically distant spaces (for example, rooms B and C and outdoor places D such as sand beaches). Alternatively, images of up to four spaces may be projected on the wall of one room A and sound may be emitted from the speaker 23. For example, an image of another room C is projected on a wall opposite to a wall on which an image of the room B is projected, and one of the walls A and B is sandwiched between two walls from which the images of the rooms B and C are transferred. In order to project an image of a place D away from C (for example, a sand beach), the servers S1 and S2, the microphone 20, the television camera 24, and the like installed in the rooms A and B are installed in the room C and the place D. do it. However, in a place D such as a sandy beach, only the multimedia data needs to be transmitted to the other party, so that the projector 21 and the screen 22 are unnecessary.
[0037]
Thus, video and audio information is exchanged between the server S1 installed in one room A and each server S2 installed in the plurality of spaces B, C, D, and When the image is projected on the screen 22 and at the same time the sound of the other party's space B to D is generated from the speaker 23, a plurality of geographically separated (in this case, four spaces A to D) states are shared between the servers S1. A plurality of spaces A to D are transmitted in real time by transmitting and receiving voices and images in real time to give a person in each of the geographically separated spaces A to D a sense as if the other person is present beside each other. Can be shared.
[0038]
(Embodiment 2)
FIG. 7 shows a block diagram of the servers S1 and S2 in the present embodiment. However, the same components as those of the servers S1 and S2 in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.
[0039]
The difference between the servers S1 and S2 in the present embodiment from the servers S1 and S2 in the first embodiment is that the humidity input unit 10 that receives the sensor output of the humidity sensor 28 installed indoors, the air conditioner 30 installed indoors, It is provided with a device control unit 29 that controls the operation of the humidifier 31. The air conditioner 30 is a conventionally well-known device having a temperature control function and a dehumidifying function, and the humidifier 31 is a conventionally well-known device that humidifies the interior by blowing air containing moisture. By controlling the operation of the humidifier 31, the indoor temperature and humidity can be adjusted to desired values.
[0040]
The temperature input unit 8 and the humidity input unit 10 convert sensor outputs taken from the temperature sensor 26 and the humidity sensor 28 into digital temperature information (temperature data) and humidity information (humidity data) and output the data to the data processing unit 2. I do. In the data processing unit 2, the video information, the audio information, the illuminance information, the temperature information, and the humidity information output by the video input unit 3, the audio input unit 5, the illuminance input unit 7, the temperature input unit 8, and the humidity input unit 10 are output. It takes in, performs various processes described in the first embodiment, and outputs it to the communication processing unit 1. The communication processing unit 1 appropriately compresses the data of each information received from the data processing unit 2, and transmits the data to the partner servers S2 and S1 via the communication line Ls according to a predetermined communication protocol.
[0041]
Further, the communication processing unit 1 receives the data of each information transmitted from the partner servers S2 and S1 through the communication line Ls, expands the received data as appropriate, and outputs the data to the data processing unit 2. Then, the data processing unit 2 compares the temperature information and the humidity information received from the communication processing unit 1 with the temperature information and the humidity information acquired from the temperature input unit 8 and the humidity input unit 10, and controls the air conditioner 30 so as to match the two. And the control information for controlling the operation of the humidifier 31 is created and output to the device control unit 29. The device control unit 29 having received the control information outputs the control signal generated based on the control information to the air conditioner 30 and the humidifier 31 to control the operation of both, thereby receiving the temperature information and the humidity information. The temperature and the humidity in the room on the side can be matched with the temperature and the humidity in the room on the transmission side, and the temperature environment and the humidity environment on the other side can be grasped with a bodily sensation.
[0042]
Here, when the values of the temperature or the humidity are significantly different between the transmitting side and the receiving side, the data processing unit 2 on the receiving side obtains the temperature and the humidity based on the temperature information and the humidity information on the other side and the temperature information and the humidity information on the receiving side. The average value is calculated, and control information for matching the average value is output to the device control unit 29. As a result, the operation of the air conditioner 30 and the humidifier 31 is controlled by the control signal output from the device control unit 29, and the temperature and humidity in the room on the receiving side are adjusted to match the above average values. The degree of change in the temperature and humidity on the transmitting side can be grasped with a bodily sensation without impairing the comfort on the side.
[0043]
(Embodiment 3)
FIG. 8 shows a block diagram of the servers S1 and S2 in the present embodiment. However, the same components as those of the servers S1 and S2 in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. The difference between the servers S1 and S2 in the present embodiment from the servers S1 and S2 in the first embodiment is that a contact-type temperature sensor 32 installed at a predetermined position on the screen 22 and a temperature input unit that captures the sensor output of the temperature sensor 32 11, a heating element 33 provided at a predetermined location on the screen 22, and a heating element control unit 12 for controlling the amount of heat generated by the heating element 33.
[0044]
The contact-type temperature sensor 32 is a well-known sensor using a resistor such as a thermistor. The contact-type temperature sensor 32 is built in a pad 34 and installed at a predetermined position on the screen 22 as shown in FIG. When the forehead is brought into contact, the temperature (body temperature) is measured. The heating element 33 uses Joule heat generated by energization, and the amount of heat generation is controlled by changing the magnitude of the energizing current. Note that the heating elements 33 are embedded in a matrix in the entire screen 22 as shown in FIG.
[0045]
For example, when the person H2 in one room B makes contact with the forehead on the pad 34 provided on the screen 22, the sensor output of the temperature sensor 32 built in the pad 34 is taken into the temperature input unit 11 of the server S2, The temperature data of the forehead of the person H2 is sent from the temperature input unit 11 to the data processing unit 2. The data processing unit 2 of the server S2 transmits the temperature data sent from the temperature input unit 11 together with other multimedia data from the communication processing unit 1 to the server S1 in the other room A.
[0046]
On the other hand, in the server S1 in the room A, when the temperature data from the server S2 received by the communication processing unit 1 is sent to the data processing unit 2, the data processing unit 2 receives the temperature data of the pad 34 in the other room B, which has been registered in advance. Control data for controlling the amount of heat generated by the heating element 33 corresponding to the position is generated and output to the heating element control unit 12, and the surface temperature of the heating element 33 is adjusted by the heating element control unit 12 to control the surface temperature of the pad 34. Is almost equal to the temperature of the forehead of the person H2 who has contacted. In other words, if the person H1 in the room A touches the part corresponding to the forehead of the other party (person H2) projected on the screen 22, the surface temperature of the heating element 33 becomes almost equal to the temperature of the forehead of the person H2. As a result, the body temperature of the other party (person H2) can be indirectly sensed.
[0047]
As described above, in the present embodiment, the temperature data measured by the contact-type temperature sensor 32 installed at a predetermined location on the screen 22 is transmitted from the server S2 to the server S1, and is installed at the predetermined location on the screen 22 in the server S1 on the receiving side. By controlling the amount of heat generated by the heating element 33 based on the temperature data, the surface temperature of the heating element 33 is made to match the temperature detected by the temperature sensor 32, and the persons H1 and H1 located in separate spaces (rooms A and B) are separated. There is an advantage that the body temperature of H2 can be sensed through the heating element 33 and the physical condition of the other party can be grasped in more detail.
[0048]
(Embodiment 4)
FIG. 11 shows a block diagram of the servers S1 and S2 in the present embodiment. However, the same components as those of the servers S1 and S2 in the third embodiment are denoted by the same reference numerals, and description thereof is omitted. The difference between the servers S1 and S2 according to the present embodiment from the servers S1 and S2 according to the third embodiment is that a tactile sensor 35 installed at a predetermined position on the screen 22 and a tactile input unit 13 that captures a sensor output of the tactile sensor 35 are provided. The point is to prepare.
[0049]
The tactile sensor 35 is a conventionally known sensor using electrostatic induction. As shown in FIG. 12, the tactile sensor 35 is embedded in a matrix adjacent to the heating element 33 over the entire inside of the screen 22.
[0050]
For example, when the person H2 in one room B touches the screen 22 with his / her hand, the sensor output is taken into the tactile input unit 13 from the tactile sensor 35 disposed at the place where the hand of the person H2 touches, and the tactile input is further performed. The position information for specifying the position of the tactile input unit 13 that has performed the sensor output is transmitted from the unit 13 to the data processing unit 2. The data processing unit 2 of the server S2 transmits the position information sent from the tactile input unit 13 together with other multimedia data from the communication processing unit 1 to the server S1 in the other room A.
[0051]
On the other hand, in the server S1 in the room A, when the position information from the server S2 received by the communication processing unit 1 is sent to the data processing unit 2, the position on the screen 22 where the data processing unit 2 is indicated by the position information, Control data for controlling the amount of heat generated by the heating element 33 disposed at the position on the screen 22 touched by the person H2 in the other room B is generated and output to the heating element control unit 12, and the generated data is output from the heating element control unit 12. By adjusting the amount of current, the surface temperature of the heating element 33 is made to match a preset temperature of human skin. Therefore, if the person H1 in the room A touches the place where the partner (person H2) projected on the screen 22 touches the hand, the surface temperature of the heating element 33 becomes the temperature of the human skin. Therefore, it is possible to indirectly sense a temperature close to the body temperature of the other party (person H2).
[0052]
As described above, in the present embodiment, the position information on the screen 22 detected by the tactile sensor 35 provided at a predetermined position on the screen 22 is transmitted from the server S2 to the server S1, and the server 22 on the receiving side performs the clean 22 based on the position information. By controlling the amount of heat generated by the heating element 33 at the corresponding location above, the surface temperature of the heating element 33 is set to a temperature similar to human skin, and the persons H1 and H2 in separate spaces (rooms A and B) are as if each other. It is possible to sense each other's body temperature via the heating element 33 as if they are directly touching each other, increasing the reality of realizing that they are sharing a virtual space, and feeling the presence of the other party more realistically. .
[0053]
(Embodiment 5)
This embodiment is characterized in that agents using pet-type robots R1 and R2 are provided in the servers S1 and S2 as shown in FIG. 13, and the other points are common to the first embodiment. Description is omitted.
[0054]
The robots R1 and R2 in the present embodiment are made by imitating small animals such as dogs and cats, and each part of the head, feet and tail is freely drivable with respect to the trunk. Further, as shown in FIG. 14, the robots R1 and R2 include a control unit 50 composed of peripheral circuits such as a CPU, a memory, and various interfaces, and a communication unit 51 that communicates with the servers S1 and S2 via a signal line Lx. A driving mechanism 52 for driving each of the above-described parts, an imaging unit 53 including a CCD camera or the like, a microphone unit 54 for collecting surrounding sounds, and a speaker unit 55 for outputting sounds such as the sound of small animals, And a contact sensor unit 56 built in the head. The control unit 50 transmits the video information captured by the imaging unit 53, the sound information collected by the microphone unit 54, and the detection information of the contact sensor unit 56 from the communication unit 51 to the servers S1 and S2 via the signal line Lx. At the same time, the driving mechanism 52 is controlled based on the control information transmitted from the servers S1 and S2 via the signal line Lx to drive each part such as the head and the foot, or to generate a cry from the speaker 55. Output.
[0055]
On the other hand, the data processing unit 2 of each of the servers S1 and S2 determines the behavior of the robots R1 and R2 based on the video information, the sound information, the detection information, and the like, and generates control information according to the behavior. An agent program is installed, and control information generated by the agent program is sent to the control unit 50, and the control unit 50 controls the drive mechanism 52 and the like based on the control information, so that the robots R1 and R2 Perform actions such as walking and singing. Note that the data processing units 2 of the servers S1 and S2 exchange information related to the action determination and control of the robots R1 and R2 through the communication processing unit 1 and share the above information about the robots R1 and R2. are doing.
[0056]
For example, when the person H1 in one room A calls the robot R1 or touches the head of the robot R1, the control unit 50 of the robot R1 collects the voice of the person H1 collected by the microphone unit 54 as sound information on the server S1. Or when the contact information of the person H1 is detected by the contact sensor unit 56 incorporated in the head of the robot R1 and the detection information is transmitted to the server S1, the data of the server S1 is transmitted. An agent program installed in the processing unit 2 generates control information for outputting a response to the call or contact of the person H1, for example, a singing "one", and transmits the control information from the communication processing unit 1 via the signal line Lx. To the robot R1, and also to the server S2 in the other room B via the communication line Ls. Then, the control unit 50 of the robot R1 drives the speaker unit 55 based on the control information received from the server S1 to output a singing sound of “one”. Further, the server S2 that has received the control information transmits the control information to the robot R2 located in the room B, and the control unit 50 of the robot R2 causes the microphone unit 54 to output a singing “one” in the same manner. That is, the robots R1 and R2 in the two rooms A and B are configured to take exactly the same action in response to an event (such as a call or a contact) that has occurred in one of the robots R1 or R2.
[0057]
Further, if the person H1 in one room A speaks to the robot R1, "Please go to OO (person H2)", the control unit 50 of the robot R1 collects the person H1 collected by the microphone unit 54. Is transmitted to the server S1 as sound information, and the data processing unit 2 of the server S1 performs a natural language analysis process on the sound information to obtain a keyword included in the sound information (for example, “XX (person H2 (Names to be specified, names, etc.) "and" go "are extracted), and based on those keywords, the action content of moving in the direction approaching the person H2 in the other room B is determined and the server S2 is determined. Send. Further, the data processing unit 2 of the server S1 generates control information for moving the robot R1 in a direction approaching the person H2 ′, that is, the screen 22, based on the determined action content, and transmits the generated control information to the robot R1. When the control unit 50 controls the drive mechanism unit 52 according to the information, the robot S1 moves in a direction approaching the screen 22.
[0058]
On the other hand, in the server S2 of the other room B, the data processing unit 2 controls the robot R2 to confirm the position of the person H2 so that the state inside the room B is imaged by the image pickup unit 53, and further, the image pickup unit 5 For the video information, the position of the person H2 in the room B is specified by performing a pattern matching process of searching for a portion that matches a pattern for identifying the person H2 registered in advance, and the specified position is determined. The action content of moving in the approaching direction is determined. Then, control information for executing the determined action content is generated and transmitted from the data processing unit 2 of the server S2 to the robot R2, and the control unit 50 controls the drive mechanism unit 52 according to the control information. Moves the robot S2 in a direction approaching the person H2.
[0059]
As described above, the actions of the robots R1 and R2 are determined by the agent program installed in the data processing unit 2 of the servers S1 and S2, and the robots R1 and R2 in the two rooms A and B are generated in one of the robots R1 and R2. By controlling the robot to take exactly the same action in response to an event (call, contact, etc.), it is possible to feel as if two rooms A and B share the same robot R1 and R2 virtually. can get.
[0060]
In the present embodiment, the robots R1 and R2 having an entity are illustrated as constituents of the agent. However, for example, a specific image (for example, a character representing a pet or the like) displayed on the screen of the display device 27 is used. The same effect can be obtained by causing the characters displayed on the display devices 27 of the rooms A and B to make the same movement.
[0061]
(Embodiment 6)
In the present embodiment, the robot R2 includes means for measuring the heart rate and the body temperature, which are the biological information of the person H2 in the room B, and the biological information of the person H2 measured by these means is transmitted from the server S2 together with the multimedia data. It is characterized in that the information is transmitted to the server S1 in another room A, and the biological information of the person H2 is displayed on the display device 27 in the server S1 on the receiving side. The other points are the same as those of the fifth embodiment, and the description is omitted.
[0062]
As shown in FIG. 15, the robot R2 according to the present embodiment includes a heart rate measurement unit 57 that measures a human heart rate, and a plurality of body temperature measurement units 58 that measure body temperature. The heart rate measuring unit 57 has a pair of electrodes built in a portion corresponding to the ear of the robot R2, for example, as shown in FIG. The sensor output (heart rate data) is obtained by detecting the potential difference generated between the pair of electrodes and removing the noise component after amplifying the potential difference. The heart rate data is transmitted from the control unit 50 to the server S2 via the communication unit 51, and the data processing unit 2 of the server S2 calculates the heart rate from the time interval of the peak value of the heart rate data obtained in time series, and obtains the obtained heart rate. The number of data is transmitted to the other server S1.
[0063]
The body temperature measuring section 58 is a conventionally known one using a resistor such as a thermistor similarly to the contact-type temperature sensor 32, and as shown in FIG. Each is built in. The control unit 50 transmits the temperature data output from each body temperature measurement unit 58 to the server S2 via the communication unit 51. In the data processing unit 2 of the server S2, when the detection information of any one of the contact sensor units 56 is received, the temperature data of the body temperature measuring unit 58 adjacent to the contact sensor unit 56 is extracted, and from among the plurality of temperature data. An average value of temperature data falling within a range (for example, 34 ° C. to 41 ° C.) corresponding to a human body temperature is obtained as a measured value of the body temperature, and the measured value (body temperature information) is transmitted to the other server S1.
[0064]
On the other hand, in the server S1 on the receiving side, the data processing unit 2 displays the data of the heart rate and the body temperature received from the server S2 on the display device 27, and the person H1 in one room A moves to the other room B. The heart rate and the body temperature of the person H2 can be known, and the people H1 and H2 in the rooms A and B can easily grasp the health condition of each other.
[0065]
【The invention's effect】
The invention according to claim 1 is provided with a plurality of communication devices which are installed in a plurality of geographically separated spaces and exchange multimedia data including audio and video data via a communication line, and which are installed in each space and communicate with each other. A plurality of data processing devices for processing multimedia data transmitted and received between the devices; an imaging unit including each space in an imaging area; a sound collection unit for collecting sound in each space; Each data processing device has a video presenting means for presenting the obtained video information in the space, and a sounding means for emitting audio information processed in the data processing device in the space, and the video presenting means has an actual size. Since the image of the subject is presented in a size approximately equal to the size of the subject, the situation of the geographically distant space is transmitted in real time by voice and video transmitted and received between the communication devices, and the video presenter Indeed for presenting images with substantially equal size to the size of the subject, it can give a feeling as if the plurality of space geographically separated are present adjacent though by.
[0066]
According to a second aspect of the present invention, in the first aspect of the present invention, the communication device and the data processing device are respectively installed in two specific spaces, so that two specific geographically separated spaces exist as if they are adjacent to each other. By giving such a feeling, the two spaces can be virtually shared.
[0067]
According to a third aspect of the present invention, in the first aspect, the video presenting means individually presents video information in a plurality of spaces, and the sounding means emits audio information in a plurality of spaces. By giving a sense as if a plurality of spaces exist adjacent to each other, the plurality of spaces can be virtually shared simultaneously.
[0068]
According to a fourth aspect of the present invention, in the first, second or third aspect of the present invention, since the communication device constantly exchanges multimedia data, it is possible to always know the state of each other's space and increase the presence. .
[0069]
According to a fifth aspect of the present invention, in the invention of any one of the first to fourth aspects, since one communication device communicates only with a specific communication device, two specific geographically separated spaces always exist adjacent to each other. It can be recognized as if it were.
[0070]
According to a sixth aspect of the present invention, in any one of the first to fifth aspects, each data processing device includes an illuminance measurement unit that measures illuminance of an imaging region by the imaging unit, and the data processing device measures the illuminance by the illuminance measurement unit. The communication device transmits the illuminance information to the communication device of the other party, and the video presenting unit adjusts the brightness of the presentation screen for presenting the video information based on the illuminance information received by the communication device. It can be recognized by feeling.
[0071]
According to a seventh aspect of the present invention, in any one of the first to sixth aspects, each data processing device includes a volume measuring unit that measures a volume of the sound collecting unit at the time of sound collection. The communication device transmits the volume information measured by the communication device to the communication device of the other party, and the sounding means adjusts the volume at which the sound information is emitted based on the volume information received by the communication device. It can be recognized by feeling.
[0072]
According to an eighth aspect of the present invention, in the invention according to any one of the first to seventh aspects, the data processing device is provided with a sound source position specifying means for specifying a position of the sound source in the space, and the data processing device is specified by the sound source position specifying means. The transmitted sound source position information is transmitted to the communication device of the other party by the communication device, and the sound generation unit reproduces the sound source position based on the sound source position information received by the communication device. Therefore, the reality of the reproduced sound information can be increased. .
[0073]
According to a ninth aspect of the present invention, in any one of the first to eighth aspects of the present invention, a measurement unit for measuring a physical quantity relating to an environment in a space and a measurement result for displaying a measurement result measured by a measurement unit in another space. Since the data processing device is provided with the display means, the environment of the other party's space can be grasped in detail.
[0074]
According to a tenth aspect of the present invention, in any one of the first to ninth aspects, an air conditioner having at least a function of adjusting a temperature is installed in a space, and a temperature measuring means for measuring a temperature in the space, Since the data processing device includes an air conditioner control unit that controls the operation of the air conditioner according to the information on the temperature measured by the space temperature measurement unit, it is possible to experience and grasp the temperature environment of other spaces. it can.
[0075]
An invention according to claim 11 is the invention according to any one of claims 1 to 10, wherein an air conditioner having at least a function of adjusting humidity is installed in the space, and humidity measuring means for measuring the humidity in the space is provided. Since the data processing device is provided with an air conditioner control unit that controls the operation of the air conditioner according to the information on the humidity measured by the humidity sensor in the space, it is possible to experience and grasp the humidity environment in other spaces. it can.
[0076]
According to a twelfth aspect of the present invention, in any one of the first to eleventh aspects, there is provided a data in which a contact-type temperature sensor or a tactile sensor installed at a location where an image is presented by an image presentation means is installed in any space. A heating element provided in the processing device and transmitting the temperature data detected by the temperature sensor or the detection data of the tactile sensor together with the multimedia data, and a heating element installed at a location where an image is presented by the image presentation means, and the temperature data or Since the heating element control means for controlling the heating value of the heating element based on the detection data is provided in the data processing device installed in any other space, the temperature of the heating element is made to match the temperature detected by the temperature sensor. This makes it possible to sense the body temperature of a person in a distant space via a heating element, and to understand the physical condition of the other person in more detail, or to use a tactile sensor. By setting the temperature of the heating element to about the human body temperature when detecting that a person has touched, it is possible to have a feeling as if people in distant spaces are in direct contact with each other, virtually The reality of the recognition of sharing the space increases, and the presence of the opponent can be felt more realistically.
[0077]
According to a thirteenth aspect of the present invention, in any one of the first to twelfth aspects, an agent for providing a service to a user is provided in each data processing device, and the data of each agent is transmitted and received together with the multimedia data. Since the data processing devices are shared, the agents can be virtually shared in a plurality of spaces, and the sense of virtually sharing the spaces can be increased.
[0078]
According to a fourteenth aspect of the present invention, in any one of the first to thirteenth aspects, each of the data processing devices includes a biological information measuring unit that measures biological information of a person in a space, and the data processing device is configured to perform the biological information measurement. The biological information measured by the means is transmitted to the communication device of the other party by the communication device, and the biological information display means for displaying the biological information received by the communication device is provided in each data processing device. It is possible to easily grasp the health condition of the other party.
[Brief description of the drawings]
FIG. 1 is a system configuration diagram showing a first embodiment.
FIG. 2 is an explanatory diagram illustrating an installation state of a television camera in the above.
FIG. 3 is a block diagram showing a server in the embodiment.
FIGS. 4A and 4B show a room in the above, wherein FIG. 4A is a plan view, FIG. 4B is a front view, and FIG.
FIGS. 5A and 5B are explanatory diagrams of the above.
FIG. 6 is a schematic diagram showing another configuration of the above.
FIG. 7 is a block diagram illustrating a server according to a second embodiment.
FIG. 8 is a block diagram illustrating a server according to a third embodiment.
FIG. 9 is an explanatory diagram illustrating an installation state of the temperature sensor in the above.
FIG. 10 is an explanatory diagram illustrating an installation state of a heating element in Embodiment 1;
FIG. 11 is a block diagram illustrating a server according to a fourth embodiment.
FIG. 12 is an explanatory diagram illustrating the installation state of the heating element and the tactile sensor in the above.
FIG. 13 is a system configuration diagram showing a fifth embodiment.
FIG. 14 is a block diagram of the robot in the above.
FIG. 15 is a block diagram of a robot according to a sixth embodiment.
FIG. 16 is a schematic configuration diagram of the robot in the above.
[Explanation of symbols]
A and B rooms
S1, S2 server
Ls communication line
20 microphones
21 Projector
22 screen
23 Speaker
24 TV camera

Claims (14)

A plurality of communication devices that are installed in a plurality of geographically separated spaces and exchange multimedia data including audio and video data via a communication line, and a multimedia that is installed in each space and exchanges communication data between the communication devices. A plurality of data processing devices for processing data; an imaging unit including each space in an imaging region; a sound collection unit for collecting sound in each space; and video information processed by the data processing device in the space. Each data processing device is provided with a video presenting means for presenting the sound information and a sounding means for emitting sound information processed by the data processing device into a space, and the video presenting means has a size substantially equal to an actual size. A multimedia communication system characterized by presenting an image of a subject. 2. The multimedia communication system according to claim 1, wherein a communication device and a data processing device are respectively installed in two specific spaces. 2. The multimedia communication system according to claim 1, wherein the video presentation unit individually presents video information in a plurality of spaces, and the sounding unit emits audio information in a plurality of spaces. 4. The multimedia communication system according to claim 1, wherein the communication device constantly exchanges multimedia data. The multimedia communication system according to any one of claims 1 to 4, wherein one communication device communicates only with a specific communication device. Each data processing device is provided with an illuminance measurement unit that measures the illuminance of the imaging region by the imaging unit, the data processing device transmits the illuminance information measured by the illuminance measurement unit to the communication device of the other party by the communication device, and the video presentation unit The multimedia communication system according to any one of claims 1 to 5, wherein brightness of a presentation screen for presenting video information is adjusted based on illuminance information received by the communication device. Each data processing device is provided with a sound volume measuring means for measuring the sound volume of the sound collecting device at the time of sound collection, and the data processing device transmits the sound volume information measured by the sound volume measuring device to the communication device of the other party by the communication device, The multimedia communication system according to any one of claims 1 to 6, further comprising: adjusting a volume at which the voice information is emitted based on the volume information received by the communication device. The data processing device includes a sound source position specifying unit that specifies the position of the sound source in the space, the data processing device transmits the sound source position information specified by the sound source position specifying unit to the communication device of the other party by the communication device, The multimedia communication system according to any one of claims 1 to 7, wherein the sound source position is reproduced based on sound source position information received by the communication device. A data processing device comprising: a measuring unit that measures a physical quantity related to an environment in a space; and a measurement result display unit that displays a measurement result measured by a measuring unit in another space. A multimedia communication system according to any one of claims 8 to 13. An air conditioner having at least a function of adjusting the temperature is installed in the space, and the operation of the air conditioner according to temperature information measured by the temperature measurer in the other space and the temperature measuring means for measuring the temperature in the space. The multimedia communication system according to any one of claims 1 to 9, wherein the data processing device includes an air conditioner control unit for controlling the multimedia communication system. At least an air conditioner having the function of adjusting humidity is installed in the space, and the operation of the air conditioner according to the humidity information measured by the humidity measurer for measuring the humidity in the space and the humidity measurer in the other space. The multimedia communication system according to any one of claims 1 to 10, wherein the data processing device includes an air conditioner control unit that controls the air conditioner. A data processing device provided with a contact-type temperature sensor or a tactile sensor installed at a place where an image is presented by the image presenting means in any space, and temperature data detected by the temperature sensor or detection of the tactile sensor The data is transmitted together with the multimedia data, and a heating element installed at a place where an image is presented by the image presentation means, and a heating element control means for controlling a heating value of the heating element based on the temperature data or the detection data. The multimedia communication system according to any one of claims 1 to 11, wherein the multimedia communication system is provided in a data processing device installed in any other space. 13. The data processing device according to claim 1, wherein each data processing device includes an agent that provides a service to a user, and exchanges data of each agent together with multimedia data and shares the data with a plurality of data processing devices. A multimedia communication system according to any one of claims 1 to 3. Each data processing device is provided with a biological information measuring means for measuring biological information of a person in the space, and the data processing device transmits the biological information measured by the biological information measuring means to the communication device of the other party by the communication device, and performs communication. The multimedia communication system according to any one of claims 1 to 13, wherein each data processing device includes a biological information display unit that displays biological information received by the device.

Images