JP4452100B2 - Video communication system - Google Patents

Description

  The present invention relates to a video communication system, and more particularly to a video conference system and a video communication system that can be applied to a dementia or autism treatment apparatus.

Recently, video conferencing systems combining a communication line and a computer are becoming popular. As this video conference system, a pointing cursor with a different color for each terminal is displayed on the same image respectively displayed on a display unit of a terminal installed in a plurality of places, and the pointing A system is known in which a desired position in an image can be specified independently from each terminal using a cursor (see Patent Document 1).
JP-A-8-298654

  However, in the video conference system, since there are always pointing cursors corresponding to the number of terminals in the image in the display unit, when some of the cursors are not used, There was a problem with the cursor getting in the way. That is, in this case, the disturbing cursor acts to hide a part of the image, which becomes an obstacle to communication between the terminals.

  The present invention has been devised by paying attention to the above problems, and an object of the present invention is to provide a video communication system that is useful as a human-to-human communication tool between places separated from each other.

(1) In order to achieve the above object, the present invention provides a video communication system that enables image data communication between at least two sets of camera units.
Each of the camera units includes a first screen and a second screen that are continuously arranged, an operation tool that can be moved by a user's hand, an operation detection unit that detects a position and a posture of the operation tool, Shadow image creation means for creating a shadow image corresponding to movement, first projection means for projecting the shadow image created by the shadow image creation means onto the first screen, and shadow image projected on the first screen And situation imaging means for imaging the situation around it, and second projection means for projecting an image from the situation imaging means in another camera unit onto the second screen,
The shadow image creating means creates a virtual shadow image of the operating tool based on the position and orientation of the operating tool detected by the operation detecting means.

(2) Further, the virtual shadow image is composed of a self-display area extending in the own camera unit, and a counterpart display area extending in the other camera unit connected to the self-display area,
Preferably, the shadow image creating means creates the shadow image by superimposing the self-display area and the counterpart display area created by another camera unit.

(3) Further, each of the camera units further includes object imaging means for imaging an object on the first screen,
The shadow image creating means creates an object shadow image by binarizing the image data picked up by the object image pickup means, and superimposes the object shadow image on the virtual shadow image to thereby convert the shadow image. It is preferable to use the configuration of creating together.

  (4) Further, the first screen is arranged substantially horizontally, while the second screen is arranged to be inclined obliquely upward and outward from one end edge side of the first screen, and the situation image pickup means is provided on the second screen. It is advisable to adopt a configuration in which imaging is possible in the direction along the surface direction from the uppermost end side of the lens.

  According to the configuration of (1), the virtual shadow image of the other party is projected on the first screen, while the virtual shadow image of the other party is projected on the second screen connected to the first screen. , The virtual shadow images of the self and the other party can be displayed, and communication involving the movement of the operation tool is possible between the users of each camera unit. Also, if you keep the operation tool in a certain place away from the camera unit, you can prevent the creation of a virtual shadow image, and when a virtual shadow image is unnecessary, you can easily project it onto the screen. Can be stopped. Therefore, as represented by a video conference system, it is useful as a human communication tool between places distant from each other.

  With the configuration as described in (2) above, the virtual shadow image of the self and the virtual shadow image of the other party can be projected onto the first screen in a superimposed manner, and an object is placed on the first screen. In such a case, the virtual shadow image can be accessed from both sides of the object, and it is possible to effectively discuss or hold an object that exists only in one place.

  According to the configuration of (3), an artificial object shadow image similar to the virtual shadow image is covered on the actual shadow of the object on the first screen, and the actual shadow of the object It is possible to eliminate the feeling of separation of shadows due to the difference in image quality from the virtual shadow image and the presence or absence of jaggies.

  According to the configuration of (4), when the user is present near the first screen so as to face the second screen, the state of the user and the situation on the first screen can be simultaneously imaged. It is possible to project more state of the other party's space on the second screen, which is more useful as a communication tool.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a schematic configuration diagram of a video communication system according to an embodiment of the present invention. In this figure, a video communication system 10 includes first and second camera units 11 and 12 that create predetermined image data connected via an IP network 14 such as the Internet, and images between the camera units 11 and 12. Data communication is possible.

  The first and second camera units 11 and 12 are installed at different locations apart from each other, but have substantially the same configuration. Accordingly, the configuration of the first camera unit 11 will be described in detail below, and the detailed description of the configuration of the second camera unit 12 will be omitted. Hereinafter, the same or equivalent components in the first camera unit 11 and the second camera unit 12 are denoted by the same reference numerals, and the first camera unit 11 side after the numbers. If so, the suffix “A” is added, and if it is on the second camera unit 12 side, the suffix “B” is added.

  In FIG. 1, the description of the frame that supports each member of each camera unit 11, 12 is omitted except for a part in order to avoid complication of the drawing.

  The first camera unit 11 includes a horizontal screen 16A as a first screen on which an object 15A is placed, a facing screen 17A as a second screen continuously disposed on one end edge side of the horizontal screen 16A, and a user's hand. A movable bar-shaped operation tool 19A, a first camera 22A as an object imaging means for imaging the object 15A on the horizontal screen 16A, and a magnetic sensor unit 24A as an operation detection means for detecting the position and orientation of the operation tool 19A. A terminal 26A constituting a shadow image creating means for creating a shadow image corresponding to the movement of the operation tool 19A, and a first projector as a first projecting means for projecting the shadow image created by the terminal 26A onto the horizontal screen 16A 28A, a shadow image projected on the horizontal screen 16A, and a situation imaging hand for imaging the surrounding situation A second camera 30A as is configured by a second projector 32A as a second projection means for projecting an image captured by the second camera 30B of the second camera unit 12 to face the screen 17A.

  The horizontal screen 16 </ b> A is provided in a flat plate shape, and is supported by a leg-shaped frame F and is arranged substantially horizontally.

  The facing screen 17A is provided in a flat plate shape, and is inclined obliquely upward and outward from one end edge side of the horizontal screen 16A. The state of the user 34B on the second camera unit 12 side and the situation on the horizontal screen 16B are displayed on the facing screen 17A. Here, the user 34B in the facing screen 17A appears to face the user 34A, and the horizontal screen 16B in the facing screen 17A appears to be substantially line-symmetric with the existing horizontal screen 16A. The user 34A is set to stand on the edge side of the horizontal screen 16A facing the facing screen 17A.

  The operation tool 19A is operated by the user 34A by hand, and when moved in an arbitrary direction, the size and direction of the virtual shadow image 36A of the operation tool 19A change as will be described later. ing.

  The first camera 22A is constituted by a CCD camera and is installed above the horizontal screen 16A so as to be able to take an image of the state on the horizontal screen 16A. Here, the captured image is used to create a shadow image in the terminal 26A described later. In order to prevent divergence due to an infinite loop between the shadow image projected from the first projector 28A on the horizontal screen 17A and the acquisition of the image on the horizontal screen 16A for creating the shadow image, illustration is omitted. A polarizing filter is attached to the lens of the first camera 22A, and a polarizing film is attached to the upper surface side that is the projection surface of the horizontal screen 17A. That is, the polarizing filter and the polarizing film act so as to cut a shadow image projected from the first projector 28A when an image is acquired by the first camera 22A.

  The magnetic sensor unit 24A includes a transmitter 38A installed at a corner of the horizontal screen 16A, a receiver 39A attached to the distal end side of the operation tool 19A, and a control unit 40A connected to the transmitter 38A and the receiver 39A by wire or wirelessly. It is comprised by. The magnetic sensor unit 24A generates a magnetic field around the transmitter 38A in response to a command from the control unit 40A. From the state of passage of the magnetic field of the receiver 39A, the control unit 40A uses the orthogonal three of the operation tool 19A to which the receiver 39A is attached. A well-known structure that can identify the position in the axial direction and the attitude around the three axes is employed. Therefore, the detailed description of the device structure is omitted below. Note that as the operation detection means, other devices that can detect the position and orientation of the operation tool 19A, such as a three-dimensional position measurement device combining an ultrasonic sensor and a magnetic gyro, may be employed.

  The terminal 26A, based on the image data acquired by the first camera 22A and the position and orientation of the operation tool 19A detected by the magnetic sensor unit 24A, is a blackish virtual shadow corresponding to the operation of the operation tool 19A. A shadow image including the image 36A is created. The terminal 26A transmits data such as the position and orientation of the operation tool 19A, image data related to the created virtual shadow image 36A, and the like to the terminal 26B of the second camera unit 12, and the second camera unit 12. Data such as the position and orientation of the operation tool 19B on the side, image data relating to the virtual shadow image 36B created there, and the like are received from the terminal 26B.

  The first projector 28A is disposed above the horizontal screen 16A, and the shadow image created by the terminal 26A is projected onto the upper surface of the horizontal screen 16A with reflection by the mirror body 42A.

  The second camera 30A is constituted by a CCD camera, and is arranged so as to be able to take an image of the state of the user 34A and the situation on the horizontal screen 16A from the uppermost end side of the facing screen 17A along the surface direction.

  The second projector 32A projects the image of the second camera 30B of the second camera unit 12 from the back side of the facing screen 17A. The communication of the image data between the second projector 32A and the second camera 30B is performed by a known image communication means different from the communication between the terminals 26A and 26B. However, the communication is performed using the terminals 26A and 26B. Also good.

  Next, creation and display of shadow images in the video communication system 10 will be described. In the following, the first camera unit 11 side will be mainly described, but the same applies to the second camera unit 12 side.

  First, as an initial setting, fixed information of the operation tool 19A, that is, information on the length and diameter of the operation tool 19A and the attachment position of the receiver 39A is stored in the terminal 26A by a predetermined input operation. Then, the operating tool 19A is placed at an arbitrary position on the horizontal screen 16A, and the operating tool 19A is imaged by the first camera 22A. At this time, the image of the operation tool 19A is binarized, and the reference position of the operation tool 19A on the horizontal screen 16A is specified. Thereafter, the object 15A is placed on the horizontal screen 16A, and the user 34A holds the operation tool 19A in his / her hand and moves the operation tool 19A in the direction of the object 15B of the other party reflected on the facing screen 17A. Then, the following image processing is performed by the terminal 26A, and the shadow image is created for each frame (still image).

First, the first camera 22A picks up an image of the object 15A installed on the horizontal screen 16A, and binarizes the image data. As shown in FIG. 2A, the object 15A on the horizontal screen 16A is obtained. A black-painted object shadow image 47A is created corresponding to the existence area of the image. As will be described later, the object shadow image 47A is projected from the first projector 28A so as to cover the actual shadow of the object 15A generated by illumination light or the like. Thereby, it is possible to reduce the feeling of separation of the shadow based on the difference in image quality between the actual shadow and the virtual shadow image 44A and the presence or absence of jaggies.
Before and after this, the position and orientation of the operation tool 19A are specified in time series by the magnetic sensor unit 24A, the information thereof, the information on the reference position of the operation tool 19A set initially, and the fixed information Thus, the length, width, position, and angle of the virtual shadow image 36A are determined according to the operation of the operation tool 19A by a predetermined algorithm that causes the full-size operation tool 19A to perform scaling, angular displacement, and the like. That is, with the amount of movement of the operation tool 19A, the length of the virtual shadow image 36A starting from right below the operation tool 19A is extended, and the extending direction of the virtual shadow image 36A is substantially the same as the movement direction of the operation tool 19A. Try to match. When the operation tool 19A is moved in the direction of the object 15B as in this example, the virtual shadow image 36A is created so as to extend in the direction of the object 15B from directly below the operation tool 19A. The virtual shadow image 36A at this time is configured by a self-display area 44A extending into the first camera unit 11 of the self and a counterpart display area 45A extending into the other second camera unit 12 and continuing to the self-display area 44A. Is done. When the operation amount of the operation tool 19A is equal to or less than the predetermined amount, the opponent display area 45A is not created, and the virtual shadow image 36A is displayed in the horizontal screen 16A of the first camera unit 11 and the facing screen of the second camera unit 12. It will extend only within 16B.
Data related to the other party display area 45A is transmitted to the terminal 26B on the second camera unit 12 side of the other party. On the other hand, of the virtual shadow video 36B created by the terminal 26B, data related to the counterpart display area 45B extending in the first camera unit 11 is received by the terminal 26A.
2B, the object shadow image 47A, the self-display area 44A in the virtual shadow image 36A created on the first camera unit 11 side, and the second camera unit 12 side. The other party display area 45B in the created virtual shadow image 36B is overlaid to create a shadow image.

The shadow image created in this way is projected on the horizontal screen 16A from the first projector 28A. On the horizontal screen 16A, a self-display area 44A created based on the intention of the user 34A and a partner-side display area 45B created based on the intention of the other user 34B are displayed. The shadow image to be included is captured by the second camera 30A together with the state of the user 34A, and is displayed on the facing screen 17B on the second camera unit 12 side. On the other hand, the shadow image and the user 34B on the second camera unit 12 side are projected on the facing screen 17A on the first camera unit 12 side.
Accordingly, between each screen 16A, 17A, the self-display area 44A based on the intention of the user 34A and the counterpart display area 45A are connected, and the self-display area 44B and the counterpart display area based on the intention of the counterpart user 34B. 45B is connected, and communication in a remote space can be performed as if it were performed in the same space.

  Therefore, according to such an embodiment, for the objects 15A and 15B installed at different places, the parts of the objects 15A and 15B that are not present at the own place but are present at the other place are required. Accordingly, the virtual shadow images 36A and 36B can be instructed, and the instruction can be displayed on both the camera units 11 and 12, so that it can be effectively used as a communication tool such as a video conference system.

  When creating the virtual shadow images 36A and 36B, the virtual shadow images 36A and 36B that extend longer when the operation tools 19A and 19B are moved to a certain degree of speed may be created. It is useful as a treatment device for patients with dementia and autism. The speed and angular velocity of the operation tool can be obtained by calculating the elapsed time between the frames using a timer from the change in position and orientation of each frame by the magnetic sensor units 24A and 24B.

  In addition, the configuration of each part of the apparatus in the present invention is not limited to the illustrated configuration example, and various modifications are possible as long as substantially the same operation is achieved.

1 is a schematic configuration diagram of a video communication system according to an embodiment of the present invention. (A) is a schematic plan view showing a state of an object shadow image, and (B) is a schematic plan view showing a state in which a virtual shadow image is superimposed on the state of (A).

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image | video communication system 11 1st camera unit 12 2nd camera unit 15A, 15B Object 16A, 16B Horizontal screen (1st screen)
17A, 17B Face-to-face screen (second screen)
19A, 19B Operation tool 22A, 22B First camera (object imaging means)
24A, 24B Magnetic sensor unit (operation detection means)
26A, 26B terminal (shadow image creation means)
28A, 28B First projector (first projection means)
30A, 30B second camera (status imaging means)
32A, 32B Second projector (second projection means)
34A, 34B User 36A, 36B Virtual shadow image 44A, 44B Self-display area 45A, 45B Counterpart display area 47A. 47B Object shadow image

Claims (4)

In a video communication system that enables image data communication between at least two camera units,
Each of the camera units includes a first screen and a second screen that are continuously arranged, an operation tool that can be moved by a user's hand, an operation detection unit that detects a position and a posture of the operation tool, Shadow image creation means for creating a shadow image corresponding to movement, first projection means for projecting the shadow image created by the shadow image creation means onto the first screen, and shadow image projected on the first screen And situation imaging means for imaging the situation around it, and second projection means for projecting an image from the situation imaging means in another camera unit onto the second screen,
The video communication system, wherein the shadow image creating means creates a virtual shadow image of the operating tool based on the position and orientation of the operating tool detected by the operation detecting means. The virtual shadow image is composed of a self-display area extending in the own camera unit, and a counterpart display area extending in the other camera unit connected to the self-display area,
2. The video communication according to claim 1, wherein the shadow image creating means creates the shadow image by superimposing the self-display area and the opponent display area created by another camera unit. system. Each camera unit further includes object imaging means for imaging an object on the first screen,
The shadow image creating means creates an object shadow image by binarizing the image data picked up by the object image pickup means, and superimposes the object shadow image on the virtual shadow image to thereby convert the shadow image. The video communication system according to claim 1, wherein the video communication system is created.   The first screen is disposed substantially horizontally, while the second screen is inclined obliquely outward and upward from one end edge side of the first screen, and the situation imaging means is disposed from the uppermost end side of the second screen. 4. The video communication system according to claim 1, wherein the video communication system is arranged so as to be capable of imaging in a direction along the surface direction.

Images