JP2012175136A - Camera system and control method of the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a camera system capable of reducing occurrence of erroneous communication while securing high operability.SOLUTION: An image processing part 16 subjects an image taken by a camera 11 to image processing of processing a prescribed region within the image, thereby generating a processed image. A display part displays the processed image. An operation part receives an operation command indicating the orientation of the camera. A motor control part 18 adjusts the orientation of the camera 11 according to the operation command.

Description

  The present invention relates to a camera system and a control method thereof, and more particularly, to a camera system using a robot apparatus disposed in a remote place and a control method thereof.

In recent years, there has been a need for remote communication to communicate with people in remote areas as movements are increasingly restricted due to pandemic measures due to infectious diseases, CO ₂ reduction and cost reduction. It is growing.

  As means for performing remote communication, an audio device such as a telephone device that communicates by voice, a TV conference system that communicates by voice and images, and the like are used. However, non-verbal communication that does not depend on language such as facial expressions and behaviors cannot be performed with a voice device alone, and thus miscommunications that do not convey accurate intentions increase. In the case of a TV conference system, a certain degree of non-verbal communication can be performed, but there are many cases where the gazes of the participants in the TV conference do not match, and it is not clear who the information was sent to. May feel uncomfortable.

  On the other hand, a robot system has been proposed that can achieve both voice communication and non-verbal communication by using a voice device and a robot. The robot system aims at more accurate communication than the communication using the TV conference system by causing the robot to reproduce the direction of the line of sight and the whispering, which are important in non-verbal communication.

  In a robot system, a remote user who is in a different location from the robot usually operates the robot while watching an image captured by a camera provided in the robot (see Patent Document 1). In such a robot system, if a camera is provided on the head of a humanoid robot, a remote user can move the direction of the head of the robot and point the camera at his or her attention point, The robot's line of sight can be directed to the point of interest. For this reason, a conversation partner can be grasped | ascertained through the robot's eyes | visual_axis (robot head direction).

  However, there is a difference in the appearance between the image captured by the camera equipped with the robot and the image actually seen by the human being at the robot's location, and the difference affects the operation of the robot and is accurate. The intention may not be transmitted to the conversation partner through the robot.

  For example, since the viewing angle of the camera is usually wide to improve the operability of the robot, the image captured by the camera is blurred or not visible to the human eye. It looks very clear. For this reason, a remote user may pay attention to a part where the robot's line of sight is not facing. In this case, the conversation partner does not know the location of the user at the remote location, so the intention of the remote location user cannot be grasped and miscommunication may occur.

  On the other hand, Patent Document 2 describes a robot apparatus that allows a remote user to adjust the viewing angle of a camera. Using this robotic device, if the viewing angle is adjusted so that the image captured by the camera matches the image seen by the human eye, the remote user should pay attention to where the robot's line of sight is not facing Can be reduced.

JP 2004-255552 A JP 2002-46088 A

  However, if the viewing angle of the camera is adjusted and narrowed by using the robot apparatus described in Patent Document 2, it becomes difficult for a remote user to grasp the situation outside the viewing angle, so the operability of the robot is low. There is a problem of becoming. For this reason, the occurrence of miscommunication cannot be reduced while ensuring high operability.

  An object of the present invention is to provide a camera system and a control method therefor that can solve the above-mentioned problem that it is impossible to reduce the occurrence of miscommunication while ensuring high operability. is there.

  The camera system according to the present invention includes a camera, an image processing unit that generates a processed image obtained by performing image processing for processing a predetermined area in the captured image on the captured image captured by the camera, and the processed image. A display unit that displays a command, an operation unit that receives a command indicating the orientation of the camera, and an adjustment unit that adjusts the orientation of the camera according to the command.

  A control method for a camera system according to the present invention is a control method for a camera system including a camera, and performs image processing for processing a predetermined area in the captured image on a captured image captured by the camera. A processed image is generated, the processed image is displayed, a command indicating the orientation of the camera is received, and the orientation of the camera is adjusted according to the command.

  According to the present invention, it is possible to reduce the occurrence of miscommunication while ensuring high operability.

It is a figure which shows the camera system of the 1st Embodiment of this invention. It is a front view which shows an example of the external shape of a robot. It is a figure which shows an example of the functional structure of a robot. It is a figure which shows an example of a functional structure of a user terminal. It is a flowchart for demonstrating an example of operation | movement of a camera system. It is a figure for comparing an original image and a processed image. It is a flowchart for demonstrating the other example of operation | movement of a camera system. It is a figure which shows the other example of a functional structure of a robot.

  Embodiments of the present invention will be described below with reference to the drawings. In the following description, components having the same function may be denoted by the same reference numerals and description thereof may be omitted.

  FIG. 1 is a diagram showing a camera system according to a first embodiment of the present invention. In FIG. 1, the camera system includes a robot 1 and a user terminal 2. The robot 1 and the user terminal 2 are connected to each other via a communication network 3 such as the Internet so that they can communicate with each other.

  The robot 1 is a humanoid robot device. As shown in FIG. 2, the robot 1 has a rotatable head 10, and the direction of the camera 11 matches the direction of the line of sight of the robot 1. A camera 11 is provided at a place where it hits the eyes. The head portion 10 is provided on the body portion 10 '.

  The user terminal 2 is a terminal device for operating the robot 1.

  In the following, it is assumed that the robot 1 is disposed in a conference room, and the user terminal 2 is disposed at the home of a user 4 who is a remote participant of a conference held in the conference room. In addition, the user 4 operates the robot 1 using the user terminal 2 to communicate with a plurality of (three in FIG. 1) conversation partners 5 to 7 who are indoor participants in the conference.

  FIG. 3 is a diagram illustrating a functional configuration of the robot 1. In FIG. 3, the robot 1 includes a head 10, a camera 11, a motor 12, a head angle detection unit 13, a memory write control unit 14, an image memory 15, an image processing unit 16, and an information transmission / reception unit 17. And a motor control unit 18.

  The head 10 is a rotary operation body that can rotate around a predetermined axis. The predetermined axis is not particularly limited. In the present embodiment, it is assumed that the predetermined axis is an axis that extends in a direction from the head 10 toward the body portion 10 ′. In this case, the head 10 moves left and right.

  The camera 11 is an imaging unit that images the surroundings. As the camera 11, a wide-angle camera with a wide viewing angle is used.

  The motor 12 is a drive unit that rotates the head 10. When the motor 12 rotates the head 10, the orientation of the camera 11 changes according to the rotation.

  The head angle detector 13 detects the rotation angle of the head 10 of the robot 1.

  The memory writing control unit 14 adds the rotation angle detected by the head angle detection unit 13 to the captured image captured by the camera 11 and records it in the image memory 15.

  The image processing unit 16 performs image processing for processing a predetermined area in the captured image on the captured image recorded in the image memory 15, and generates a processed image from the captured image. The predetermined areas are left and right areas excluding the central part of the captured image, left and right and upper and lower areas excluding the central part of the captured image, and the like.

  Examples of the image processing include processing for reducing the sharpness of a predetermined area and blurring the inside of the predetermined area, and processing for synthesizing a translucent mask in the predetermined area.

  Further, the image processing may be processing for performing image recognition processing on a predetermined area, specifying a predetermined object (for example, a human) in the predetermined area, and changing the object. Examples of the process for changing the object include a process for making the object only an outline and a process for deforming or abstracting the object. Moreover, as a method of deforming or abstracting the object, a method of approximating the curved portion of the contour of the object with a straight line can be cited.

  Further, the image processing may be processing that creates position information indicating the position of an object in a predetermined area and shows only position information in the predetermined area. The image processing may be a combination of the above examples.

  The information transmission / reception unit 17 transmits the processed image generated by the image processing unit 16 to the user terminal 2. Further, the information transmitting / receiving unit 17 receives an operation command for operating the robot from the user terminal 2. More specifically, the operation command indicates the rotation angle of the head 10, that is, the direction of the camera 11.

  The motor control unit 18 adjusts the direction of the camera 11 (the line of sight of the robot 1) by rotating the head 10 according to the operation command received by the information transmitting / receiving unit 17. More specifically, the motor control unit 18 adjusts the orientation of the camera 11 by driving the motor 12 so that the head 10 rotates by the rotation angle indicated by the operation command. Therefore, the motor 12 and the motor control unit 18 constitute an adjustment unit that adjusts the orientation of the camera 11 in accordance with the operation command.

  FIG. 4 is a diagram illustrating a functional configuration of the user terminal 2. In FIG. 4, the user terminal 2 includes an information transmission / reception unit 21, a display processing unit 22, a display unit 23, an operation unit 24, and an input processing unit 25.

  The information transmitting / receiving unit 21 receives the processed image from the robot 1.

  The display processing unit 22 displays the processed image received by the information transmitting / receiving unit 21 on the display unit 23.

  The operation unit 24 receives an operation command from the user.

  The input processing unit 25 transmits the operation command received by the operation unit 24 to the robot 1 via the information transmission / reception unit 21.

  Next, the operation will be described.

  FIG. 5 is a flowchart for explaining an example of the operation of the camera system when displaying the captured image captured by the camera 11 on the display unit 23.

  First, the camera 11 provided in the head 10 of the robot 1 captures the surroundings to acquire a captured image, and outputs the captured image to the memory write control unit 14 (step S51).

  When receiving the captured image, the memory writing control unit 14 acquires the rotation angle of the head 10 from the head angle detection unit 13 and adds the rotation angle to the captured image, and writes the captured image with the rotation angle added to the image memory 15. (Step S52).

  The image processing unit 16 detects that the captured image has been written in the image memory 15 and acquires the captured image from the image memory 15. The image processing unit 16 performs image processing on the captured image, and outputs a processed image, which is the captured image on which the image processing has been performed, to the information transmitting / receiving unit 17 (step S53).

  FIG. 6 is a diagram illustrating an example of an original image that is a captured image before image processing and a processed image that is a captured image after image processing. As the image processing, processing for adding a translucent mask to a predetermined area is used.

  As shown in FIG. 6, in the original image 61, all the conversation partners 5 to 7 arranged side by side are clearly shown, but in the processed image 62, the conversation partner in the center among the conversation partners 5 to 7. 6 is clearly shown, but the conversation partners 5 and 7 shown in the predetermined area 63 are blurred.

  Returning to the description of FIG. When receiving the processed image, the information transmitting / receiving unit 17 transmits the processed image to the information transmitting / receiving unit 21 of the user terminal 2 via the communication network 3 (step S54).

  When receiving the processed image, the information transmitting / receiving unit 21 outputs the processed image to the display processing unit 22. Upon receiving the processed image, the display processing unit 22 displays the processed image on the display unit 23 (step S55).

  FIG. 7 is a flowchart for explaining an example of the operation of the camera system related to the operation of the robot 1.

  First, the input processing unit 25 checks whether or not an operation command input operation has been performed on the operation unit 24 (step S71).

  When there is an input operation, the input processing unit 25 outputs the operation command to the information transmitting / receiving unit 21. When receiving the operation command, the information transmitting / receiving unit 21 transmits the operation command to the information transmitting / receiving unit 17 of the robot 1 via the communication network 3 (step S72).

  When receiving the operation command, the information transmitting / receiving unit 17 outputs the operation command to the motor control unit 18. When receiving the operation command, the motor control unit 18 drives the motor 12 so that the head 10 rotates by the rotation angle indicated by the operation command (step S73). The head 10 is rotated by the drive of the motor 12, and the orientation of the camera 11 is changed (step S74).

  As described above, according to the present embodiment, the image processing unit 16 generates a processed image by performing image processing for processing a predetermined region in the captured image on the captured image captured by the camera 11. To do. The display unit 23 displays the processed image. The operation unit 24 receives an operation command indicating the direction of the camera. The motor control unit 18 adjusts the orientation of the camera 11 according to the operation command.

  Therefore, since the processed image obtained by processing the predetermined area in the captured image is displayed, it is difficult for the user 4 to accurately grasp the facial expression of the person shown in the predetermined area. For this reason, when the user 4 sees the facial expression of the person shown in the predetermined area, the user 4 needs to input an operation command to point the camera 11 toward the person shown in the predetermined area. Therefore, the camera 11 faces the user 4's attention location, and the conversation partner can grasp the user 4's attention location from the direction of the camera 11. For this reason, the occurrence of miscommunication can be reduced.

  Further, unlike the case where the viewing angle of the camera 11 is adjusted using an optical filter or the like, the user 4 can grasp the presence / absence of a human being in a predetermined area and a rough movement. For this reason, the predetermined area plays a role like a human peripheral vision, and the camera 11 can be moved in accordance with a change in the predetermined area, so that high operability can be ensured.

  In this embodiment, since the camera 11 is provided on the head 10 of the robot 1, the direction of the camera 11 and the line of sight of the robot can be matched. For this reason, the conversation partner can easily and intuitively grasp the part that the user 4 is paying attention to.

  Next, a second embodiment of the present invention will be described.

  FIG. 8 is a diagram illustrating a functional configuration of the robot 1 according to the present embodiment. The robot 1 shown in FIG. 8 further includes a voice acquisition unit 81 and a voice processing unit 82 in addition to the configuration shown in FIG.

  The sound acquisition unit 81 acquires surrounding sound as sound data.

  Based on the audio data acquired by the audio acquisition unit 81, the audio processing unit 82 specifies the sound source direction that is the direction of the audio source indicated by the audio data.

  For example, the sound acquisition unit 81 has a plurality of microphones provided at each of a plurality of positions, and the sound processing unit 82 calculates a time difference between the sound data acquired by each microphone, and based on the time difference, Specify the sound source direction.

  The motor control unit 18 adjusts the direction of the camera 11 by rotating the head 10 by driving the motor 12 according to the sound source direction specified by the sound processing unit 82.

  For example, the motor control unit 18 holds in advance a table indicating the correspondence between the sound source direction and the rotation angle of the head 10. It is assumed that the correspondence between the sound source direction and the rotation angle of the head 10 is determined so that the orientation of the camera 11 faces the sound source direction.

  And the motor control part 18 calculates | requires the rotation angle corresponding to a sound source direction using a table, drives the motor 12 so that the head 10 rotates only the rotation angle, and points the camera 11 to a sound source direction.

  As described above, in the present embodiment, since the orientation of the camera 11 is adjusted according to the sound source direction, the camera 11 can be directed to the face of a speaker who is often watched by the user 4. For this reason, since the direction of the camera 11 can be adjusted without the user 4 inputting an operation command, the operability can be improved.

  In addition, the image processing unit 16 performs image recognition processing on the captured image, and specifies the position of a predetermined detection target (for example, a human face) in the captured image. Then, the motor control unit 18 may adjust the direction of the camera 11 by rotating the head 10 by driving the motor 12 according to the position of the detection target.

  For example, the image processing unit 16 outputs a command indicating the distance and direction from the center point of the captured image to the detection target as the position of the detection target to the motor control unit 18. Based on the distance and direction indicated by the command, the motor control unit 18 obtains a rotation angle at which the camera 11 faces the detection target, and drives the motor 12 so that the head 10 rotates by the rotation angle. The camera 11 is directed toward the detection target.

  In this case, since the orientation of the camera is adjusted according to the position of the detection target object, the camera 11 can be directed to the face of the conversation partner who the user 4 often pays attention to. For this reason, since the direction of the camera 11 can be adjusted without the user 4 inputting an operation command, the operability can be improved.

  In each embodiment described above, the illustrated configuration is merely an example, and the present invention is not limited to the configuration.

  For example, the camera 11 may be composed of a plurality of camera devices instead of the wide-angle camera. In this case, for example, the image processing unit 16 generates a large image by combining a plurality of captured images captured by each camera device, and performs image processing on the large image to generate a processed image.

  The image memory 15 may further record past captured images, and the image processing unit 16 may combine the past captured images in the image memory 15 with the current captured images. At this time, only the captured image may be updated in real time, and the past captured image may be kept constant or may be periodically changed.

  Further, the image processing unit 16 may be provided in the user terminal 2.

DESCRIPTION OF SYMBOLS 1 Robot 2 User terminal 3 Communication network 10 Head 10 'Torso part 11 Camera 12 Motor 13 Head angle detection part 14 Memory write control part 15 Image memory 16 Image processing part 17, 21 Information transmission / reception part 18 Motor control part 22 Display processing part 23 display unit 24 operation unit 25 input processing unit 81 voice acquisition unit 82 voice processing unit

Claims (9)

A camera,
An image processing unit that performs image processing on a captured image captured by the camera to process a predetermined area in the captured image, and generates a processed image;
A display unit for displaying the processed image;
An operation unit for receiving a command indicating the orientation of the camera;
An adjustment unit that adjusts the orientation of the camera in response to the command.   The camera system according to claim 1, wherein the image processing includes a process of reducing a definition of the predetermined area.   The camera system according to claim 1, wherein the image processing includes a process of changing a predetermined object displayed in the predetermined area.   The camera system according to any one of claims 1 to 3, wherein the image processing includes a process of combining a mask with the predetermined area. A robot apparatus having a body part, and a rotatable head provided on the body part;
The camera is provided on the head;
The camera system according to claim 1, wherein the adjustment unit adjusts the orientation of the camera by rotating the head. A terminal device capable of communicating with the robot device;
The robot apparatus includes the adjustment unit and the image processing unit,
The camera system according to claim 5, wherein the terminal device includes the operation unit and the display unit. The robot apparatus further includes a voice acquisition unit and a voice processing unit,
The voice acquisition unit acquires surrounding voice as voice data,
The sound processing unit specifies a direction of a sound source of the sound based on the sound data,
The camera system according to claim 5, wherein the adjustment unit adjusts a direction of the camera according to a direction of the sound source. The image processing unit specifies a position of a predetermined detection target in the captured image,
The camera system according to claim 1, wherein the adjustment unit adjusts a direction of the camera according to a position of the detection target. A control method for a camera system including a camera,
For a captured image captured by the camera, a processed image is generated by performing image processing for processing a predetermined area in the captured image,
Displaying the processed image;
Accepts a command indicating the direction of the camera,
A method of controlling a camera system, wherein the direction of the camera is adjusted in accordance with the command.

Images