JP2012010059A - Imaging apparatus and remote communication system using the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus which allows participants to communicate smoothly by looking at eyes of each other with cheap and small-sized structure, and enables participants to freely select a display device to display the participants thereon, and to provide a remote communication system using the same.SOLUTION: An imaging apparatus includes: a camera body 1 integrally comprising a camera 5 to take an image of participants on a local station side, and a camera 6 to take an image of participants on the other station side to be displayed on a display device 13; a supporting member 2 (engaging member 4) to hold the camera body 1 in a state where the imaging direction and the height of the camera body 1 are adjustable; a drive part to drive supporting member 2 so as to change the imaging direction and the height of the camera body 1; a face detection processing part to detect faces of participants from moving images taken by cameras 5 and 6; and a position control part to drive the drive part so as to take moving images of faces of participants by cameras 5 and 6 referring to the detection result from the face detection processing part, and to perform control for positioning the camera body 1.

Description

  The present invention relates to a photographing apparatus capable of performing a smooth dialogue by exchanging images and sounds with a partner while in a remote place, and a remote communication system using the same.

  In recent years, with the development of video processing technology and communication technology, remote communication systems for exchanging images and voices with remote parties such as video conferences, WEB conferences, and videophones have come into practical use. ing. In the future, especially in the business field, the importance of smooth remote communication will inevitably increase with the shift of work styles such as the spread of work and working hours and the spread of telework. In order to conduct a smooth dialogue as well as a face-to-face dialogue in a remote communication system, it is an important factor that participants can talk in a natural state with their eyes on each other.

  A video conference system in which a half mirror is interposed between one participant and a display device that displays the other participant at a remote location is known as a method for realizing line-of-sight coincidence between participants. In this configuration, one participant sees an image of the other participant displayed on the display device via the half mirror. The appearance of one participant is reflected by the half mirror and photographed by the photographing device. The video imaged by the imaging device and transmitted to the other participant is a form in which one participant is facing the other participant. Therefore, it is possible to perform remote communication in which the eyes of the participants are matched.

  Patent Document 1 proposes a technique for synthesizing an image in which a participant has a line of sight matched from video captured by an imaging device. In addition, various technologies have been proposed, such as a technique for displaying characters that generate body movements from speech and matching the lines of sight of participants by interacting while watching each other.

  Japanese Patent Application Laid-Open No. 2004-133620 proposes a photographing device that is attached to the upper part of a display device such as a television receiver. In this photographing apparatus, a camera body is attached to a lower end portion of a support member that hangs downward along a display screen of a display device from a locking member that is locked to an upper portion of the display device. In this photographing apparatus, by arranging the camera body on the eyes of the participants who are viewing the display screen, the participants can talk with each other with the eyes aligned.

  However, the configuration using optical components such as a half mirror has a problem that a large and expensive dedicated display device is required, and the entire device has a large volume. In addition, in order to display a composite video or character on the display device, a dedicated display device having a processing unit for executing this is required. Moreover, the imaging device described in Patent Document 2 cannot be used for a display surface such as a screen projected by a projector or the like, and the display device that displays the participants is limited.

  The present invention has been made in view of the above problems. The purpose of this is to provide a photographing device capable of smoothly selecting a display device for displaying a participant, and allowing participants to have a smooth dialogue with a low-cost and compact configuration. It is to provide a remote communication system used.

In order to solve the above-mentioned problem, the invention of claim 1 is installed between a local station participant participating in remote communication and a display screen on which a video image taken at the other station side is displayed. In a photographing device that photographs a station-side participant and transmits a video to another station, the local station-side photographing unit that photographs the subject on the local station and the other-station side photographing that captures the subject on the other station displayed on the display screen A photographing device main body configured integrally with the imaging device, a support mechanism that supports the photographing device main body so that the photographing direction and height of the photographing device main body can be adjusted, and the photographing direction and height of the photographing device main body are changed. Accordingly, a drive means for driving the support mechanism, a face detection means for detecting the face of the participant from the images photographed by the local station side photographing section and the other station side photographing section, and a detection result from the face detection means Referring to the participant's side photographing unit and the rear photographing unit Is driving the driving means so as to shoot and is characterized in that it comprises a position control means for controlling for positioning of the imaging apparatus main body.
According to a second aspect of the present invention, in the remote communication system for exchanging image / sound information between participants at remote locations, the photographing device according to the first aspect is used as a photographing device for photographing the participant. To do.
In the present invention, the video of the local station participant captured by the local station imaging unit of the imaging apparatus body is transmitted to the other station side and displayed on the display screen installed on the other station side. On the other hand, the local station participant interacts while viewing the other station side participant on the display surface on which the video captured by the other station side imaging device is displayed. At this time, the face detection means detects the face of the local station participant from the video shot by the local station side shooting unit, and detects the face of the other station side participant from the video shot by the other station side shooting unit. To do. Then, the position control means drives the drive means to adjust the photographing direction and height of the photographing unit so that both the local station photographing unit and the other station photographing unit can detect the face. As a result, the main body of the photographing apparatus is positioned approximately on the line of sight connecting the face of the participant on the local station side and the face of the participant on the other station side displayed on the display surface. It is possible to have a smooth dialogue in a matched state. Further, in the present invention, it is not necessary to mount an optical component such as a half mirror in addition to a photographing unit such as a CCD camera in the photographing device body, and a small device configuration is possible. In using a remote communication system using this photographing apparatus, since the type of display device is not selected, an existing display device can be used, and new equipment costs can be reduced.

  The present invention provides an imaging device capable of allowing a participant to perform a smooth dialogue with a low-cost and small configuration while keeping their eyes on each other, and capable of freely selecting a display device for displaying the participant. There is an excellent effect that the remote communication system used can be provided.

1 is a perspective view illustrating an overall configuration of a photographing apparatus according to an embodiment. The perspective view explaining operation | movement of the imaging device. The perspective view explaining the hardware constitutions of the camera main body of the imaging device. The flowchart explaining the positioning process process of the imaging device. The schematic diagram explaining the positioning process process of the imaging device.

  Hereinafter, embodiments of a photographing apparatus to which the present invention is applied and a remote communication system using the same will be described. FIG. 1 is a perspective view illustrating the overall configuration of the photographing apparatus. As shown in FIG. 1, a photographing apparatus 10 includes a camera main body 1 that is a photographing apparatus main body, and a support member 2 that supports the height and photographing direction of the camera main body 1 so as to be adjustable as a support mechanism that supports the camera main body 1. A mounting table 3 that locks the support member 2 and can be mounted at an arbitrary installation location, and a locking member 4 that locks the camera body 1 and the support member 2 are provided. The mounting table 3 has a shape that is mainly assumed to be placed on a desk, and has a mechanism that can stabilize the photographing device 10 when installed. This mechanism may be a method of simply mounting a certain amount of weight on the mounting table 3 in order to stabilize the photographing apparatus 10, or a method of fixing to another member using some kind of locking member.

  The photographing apparatus 10 includes driving means for driving the support member 2, the locking member 4, or the mounting table 3 in order to change the height of the camera body 1 and the photographing direction. 2A, 2B, and 2C are perspective views for explaining the operation of the photographing apparatus. As shown in FIG. 2, as the driving means for driving to change the height and photographing direction of the camera body 1, for example, a driving unit such as a motor is mounted inside the support member 2 or the mounting table 3, and the driving unit Is a mechanism for extending the support member 2 and automatically adjusting the distance between the camera body 1 and the mounting table 3. For example, as shown in FIG. 3, a driving unit 7 such as a motor is mounted on the camera body 1, and the driving unit 7 is connected to a locking member 4 that locks the camera body 1 and the support member 2. For example, the angle (photographing direction) of the camera body 1 is adjusted by the power of the unit 7. It is desirable that the drive unit for moving the camera body 1 up and down and the drive unit for adjusting the angle of the camera body 1 operate simultaneously.

  Next, the configuration of the camera body 1 will be described. FIG. 3 is a perspective view illustrating the hardware configuration of the camera body. As shown in FIG. 3A, the camera body 1 has a substantially rectangular parallelepiped camera casing 1a disposed on the right side in the figure and a left side in the figure, and has a smaller volume than the camera casing 1a. The camera housing 1b has a substantially rectangular parallelepiped shape. The camera housing 1a is provided with a camera 5 as a local-side image capturing unit that captures a video of the local-station participant in the right direction in the figure. The camera housing 1b is provided with a camera 6 that is the other station side photographing unit that photographs an image on the display surface of the display device that is in the left direction in the figure and displays other station side participants. The camera body 1 also receives a signal input from the drive unit 7 that can change the angle of the camera body 1, a wiring unit 8 that transmits a signal from the camera 5 or the camera 6, and a signal input from the wiring unit 8. A control unit 9 for controlling the main body 1 is provided. The camera 5 and the camera 6 transmit the video imaged by each camera to the control unit 9 through the wiring unit 8. The control unit 9 includes a reception unit that receives video, a face detection processing unit that is a face detection unit that performs video processing such as face detection, and a position control unit that is a position control unit for positioning the camera body 1. .

  The face detection processing performed by the face detection processing unit of the control unit 9 converts the video captured by the camera 5 and the camera 6 into luminance data, hue data, and chroma data, and changes the luminance in the vertical direction in the skin color region based on the luminance data. And a person's face area is detected based on a spatial pixel distribution of skin color pixels based on three attribute data (Japanese Patent No. 3561985 (paragraph numbers [0038] to [0058], FIG. 1)). reference). Also, there is a technique that detects a skin color area and facial feature points such as eyes and mouth from the captured image and determines whether the skin color area is a person's face from the positions of the feature points (Japanese Patent Application Laid-Open No. 2004-5384). Gazette (see paragraph numbers [0018] to [0019], FIG. 1)). Furthermore, as a technique that does not use color information in particular, there is a technique that performs matching processing with a face template image using luminance data of a captured image, and determines that the face is a face when their correlation value is sufficiently high. (See Japanese Patent Laid-Open No. 2003-271933 (paragraph numbers [0046] to [0055], FIG. 6)).

  The position control performed by the position control unit of the upper control unit 9 is performed by moving the camera body 1 up and down or changing the angle (photographing direction) so that both the camera 5 and the camera 6 can photograph a face. This is control for positioning the main body 1. FIG. 4 is a flowchart for explaining a positioning process step of the photographing apparatus. For example, as shown in FIG. 4, when the operation of the photographing apparatus is started (step S1), the positioning processing unit can change the angle of the camera body 1 built in the camera body 1. The drive part 7 is driven. At the same time, the camera 5 and the camera 6 transmit the captured video to the control unit 9 through the wiring unit 8. The control unit 9 receives an image transmitted from the camera 5 and the camera 6 through the wiring unit 8. The face detection processing unit of the control unit 9 receives the video transmitted through the wiring unit 8, executes the face recognition first process, and the face of the participant appears in at least one of the images captured by the camera 5 or the camera 6. Is detected (step S2). When the face of the participant is not detected from either of the images taken by the camera 5 or the camera 6 (no in step S2), the drive unit 7 is further driven to change the angle of the camera body 1, and the camera 5 and the shooting location of the camera 6 are changed (step S3). Then, the camera 5 and the camera 6 transmit the captured image to the control unit 9 through the wiring unit 8 again. The face detection processing unit of the control unit 9 receives the video transmitted through the wiring unit 8, executes the face recognition first process again, and the participant's face is included in at least one of the images captured by the camera 5 or the camera 6. It is detected whether it is reflected (step S2). Thereafter, the same operation is repeated until it is detected that the face of the participant is reflected in at least one of the images taken by the camera 5 or the camera 6 (step 1 to step 3).

  When it is detected that the face of the participant is reflected in at least one of the images captured by the camera 5 or the camera 6 (hereinafter, it is assumed that the face of the local participant is detected by the camera 5). (Yes in Step 2), the face detection processing unit of the control unit 9 receives the video transmitted through the wiring unit 8, performs the face recognition second process, and also from the video captured by the other camera 6 on the other station side It is confirmed whether it is detected that the participant's face is reflected (step 4). When it is detected from the video captured by the camera 5 that only the face of the local station participant is reflected (no in step S4), the control unit 9 reflects the face of the local station participant. The camera body 1 mounted inside the support member 2 and the mounting table 3 through the drive unit 7 and the locking member 4 in a state where the shooting location of the camera 5 transmitting the detected video is fixed. The support member 2 for moving up and down is operated (step S5). Then, the camera 5 and the camera 6 transmit the captured image to the control unit 9 through the wiring unit 8 again. The face detection processing unit of the control unit 9 receives the video transmitted through the wiring unit 8, executes the face recognition second process again, and the face of the other station side participant is reflected from the video captured by the camera 6. It is confirmed whether it is detected (step S4). Thereafter, the above-described operation is repeated until it is detected that the face of the participant is reflected in both the images taken by the camera 5 and the camera 6 (steps S4 and S5). When it is detected that the face of the participant is reflected in both of the images taken by the camera 5 and the camera 6 (yes in step S4), the position control unit of the control unit 9 is for moving the camera body 1 up and down. The drive unit 7 for changing the angle of the support mechanism and the camera body 1 is stopped (step 6). This completes the basic operation (step 7).

  Next, a case will be described in which it is no longer possible to detect that a participant's face is shown only from one of the images taken by the camera 5 or the camera 6 because the participant temporarily leaves during the meeting. After the basic operation is completed, when it is no longer detected that the face of the participant is reflected from only one of the images captured by the camera 5 or the camera 6, the imaging device 10 maintains the current state. At that time, the face detection processing unit of the control unit 9 continuously performs the face recognition first process in step 2 and the face recognition second process in step 4 at regular intervals. If it is detected that the participant's face is reflected in both of the images taken by the camera 5 or 6 when the participant returns, the participant's face is reflected again in the proper position. The position control unit finely adjusts the camera body 1 until it is detected. When the arrangement of the participants changes greatly, the local station participant starts up the imaging device 10 again, and performs processing again from the start of operation.

  Next, the case where the photographing apparatus 10 having the above-described configuration is installed to perform remote communication will be specifically described. FIG. As shown in FIG. 5A, first, the photographing apparatus 10 is installed on the conference desk 11 in an environment for performing remote communication. At this time, for example, the imaging device 10 is installed between the display device 13 that displays the other station side participants and the local station participant 12 who sits across the conference desk 11. Assuming that the position of the face 14 of the other station side participant in the display device 13 is the position shown in the figure, the line of sight of the local station participant 12 is as shown by the dotted line A. In FIG. 5, only the local station participant 12, the display device 13, the imaging device 10, and the conference desk 11 are illustrated, but in addition to this, an audio monitor device that acquires the audio of the local station participant, etc. A voice output device that outputs voice acquired on the station side and a communication device that enables transmission and reception of image information and voice information between the local station side participant and the other station side participant are also installed.

  This is activated after installation of the photographing apparatus 10 is completed. When the photographing apparatus 10 is activated, the cameras 5 and 6 and the drive unit 7 built in the camera body 1 are first driven, and the camera 5 and the camera 6 transmit the captured video to the control unit 9 through the wiring unit 8. The control unit 9 receives an image transmitted from the camera 5 and the camera 6 through the wiring unit 8. The control unit 9 receives the video transmitted through the wiring unit 8, executes the face recognition first process, and detects whether the participant's face is reflected in at least one of the images captured by the camera 5 or the camera 6. To do. In the case shown in FIG. 5A, at the start of operation, it cannot be detected that the face of the participant is reflected from both the images taken by the camera 5 and the camera 6. Therefore, the control unit 9 drives the drive unit 7 to change the angle of the camera body 1 and receives the video transmitted from the camera 5 and the camera 6 through the wiring unit 8 again. By driving the drive unit 7, the angle of the camera body 1 is changed, and the face of the participant appears in at least one of the images taken by the camera 5 or 6 in the control unit 9 (face detection processing unit). If it is detected that the face of the participant is reflected from the video taken by the other camera, it is confirmed. In the case shown in FIG. 5B, the control unit 9 can detect the face of the local participant 12 within the shooting range B of the camera 5. However, when the face of the local station participant is detected, the face of the other station side participant cannot be detected from the shooting range C of the other camera 6.

  If it is detected that the face of the participant is shown only from one of the images taken by the camera 5 or 6, the control unit 9 (position control unit) may show the face of the participant. Fix the shooting location of the camera that is sending the detected video. And the control part 9 operates the support mechanism for raising / lowering the support member 2 mounted in the inside of the support member 2 or the mounting base 3 through the drive part 7 and the latching member 4 in this state. In the case shown in FIG. 5C, the face of the local station participant 12 is detected, but the face 14 of the other station side participant cannot be detected. Therefore, the control unit 9 operates a support mechanism mounted in the support member 2 or the mounting table 3 for moving the support member 2 up and down, and extends the support member 2. At the same time, the driving unit 7 capable of changing the angle of the camera body 1 is driven so that the participant on the local station can be accommodated in the video taken by the camera 5 or 6 even when the support member 2 is extended. The angle of the camera body 1 is adjusted.

  The control unit 9 drives a drive unit mounted inside the support member 2 or the mounting table 3 for moving the support member 2 up and down, and the face of the participant is displayed on both the images taken by the camera 5 and the camera 6. Is detected, the operation of the support member 2 and the change of the angle of the camera body 1 are stopped. As shown in FIG. 5D, when the camera body 3 moves on the line of sight A of the local station participant, the face of the participant appears in both the images taken by the camera 5 and the camera 6. If it is detected, the control unit 9 stops the operation of the support member 2 and the change of the angle of the camera body 1. This completes the operation.

  As shown in FIG. 5 (d), when the camera body 1 is installed on the dotted line A, the other station side participant can obtain an image whose line of sight coincides with that of the own station side participant. Communication is possible. Further, in the remote communication system using the photographing apparatus 10, since the type of the display device is not selected, an existing display device can be used, and new equipment costs can be reduced.

Note that the camera body 1 according to the present embodiment is configured such that the camera housing 1a and the camera housing 1b can be moved manually or automatically left and right in the drawing as shown in FIG. 3B. It is connected. Ca and camera housing 1a and the camera housing 1b is integrally formed, the face of the participant to both images taken by the camera 5 and 6 that are to be detected, to match the eyes among participants However, the direction of the camera 6 can be arbitrarily adjusted manually or automatically depending on the user's environment. For example, in the photographing apparatus 10 according to the present embodiment, the camera body 1 has a size of about 4 cm (width) × 1.5 cm (height) × 4 cm (depth), but the camera body 1 is on the dotted line A. In some cases, the local station participant may feel that the camera body 1 is in the way. In such a case, the camera casing 1a and the camera casing 1b can be operated to change the angle of the camera 6 as shown in FIG. It is also possible to perform.

  Moreover, although the imaging device 10 according to the present embodiment has a configuration in which the camera body 1 is attached to the upper portion of the support member 2, the imaging device according to the present invention includes, for example, a camera body. The structure attached to the lower end part of the supporting member suspended below from the ceiling part may be sufficient.

  As described above, according to the imaging apparatus 10 according to the present embodiment, the video of the local station participant captured by the camera 5 that is the local station imaging unit of the camera body 1 is transmitted to the other station side and transmitted to the other station side. It will be displayed on the display surface of the installed display device. On the other hand, the local station-side participant 12 interacts while viewing the other-station-side participant on the display surface of the display device 13 on which the video captured by the other-station-side imaging device is displayed. At this time, in the control unit 9, the face detection processing unit detects the face of the local station participant 12 from the video imaged by the camera 5, and the other station side participant's face 14 from the video imaged by the camera 6. Is detected. Then, the position control unit drives the drive unit so that both the camera 5 and the camera 6 can detect a face, and adjusts the shooting direction and height of the cameras 5 and 6. As a result, the camera body 1 is positioned approximately on the line of sight A connecting the face of the local station participant 12 and the face 14 of the other station side participant displayed on the display surface of the display device 13. Participants can have a smooth dialogue with their eyes aligned. In the present invention, the camera body 1 does not need to be equipped with an optical component such as a half mirror in addition to a photographing unit such as a CCD camera, and a small device configuration is possible. In using the remote communication system using the photographing apparatus 10, since the type of the display device is not selected, the existing display device can be used, and new equipment costs can be reduced.

DESCRIPTION OF SYMBOLS 1 Camera main body 2 Support member 3 Mounting stand 4 Locking member 5 Camera 6 Camera 7 Drive part 8 Wiring part 9 Control part 10 Imaging device

Japanese Patent Laid-Open No. 2005-218103 Japanese Patent No. 3945584

Claims (2)

It is installed between the local station participants participating in remote communication and the display screen on which the video taken by the other station is displayed. In the imaging device that transmits,
An imaging apparatus body in which a local station imaging unit that captures an image of the local station subject and an other station side imaging unit that captures an image of the other station side image displayed on the display surface;
A support mechanism for supporting the photographing apparatus main body so that the photographing direction and height of the photographing apparatus main body can be adjusted;
Drive means for driving the support mechanism to change the photographing direction and height of the photographing apparatus body;
Face detection means for detecting the face of the participant from the video imaged by the local station side imaging unit and the other station side imaging unit;
Referring to the detection result from the face detecting means, the driving means is driven to photograph the participant's face in both the local station side photographing unit and the other station side photographing unit, and positioning of the photographing apparatus main body is performed. An imaging apparatus comprising: position control means for performing control for performing. In a remote communication system that exchanges image and audio information between participants at remote locations,
The photographing apparatus according to claim 1 is used as a photographing apparatus for photographing the participant.

Images