JP2006270380A - Image information transmitter - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image information capable of transmitter eliminating the need for the minding of the field angle of a camera when a communication is intended while viewing an image by mutually transmitting their own images and being capable of operating the camera on the opposite side by its own will while respecting a privacy on the opposite side. <P>SOLUTION: A person-image range corresponding to a person is specified from the image acquired by an image input section 10, the field angle of the image acquired by the image input section 10 is controlled by an image-input control section 13 so that the person is captured at the center of the field angle on the basis of the person-image range and a specified range wider than the person-image range is set by an indication from a user including the person image range. When a transmitting image obtained by working an image section excepting the person-image range or the specified range by an image-range selecting section 15 is set on the basis of the person-image range, the transmitting image obtained by working the image section excepting the specified range in the image supplied from the image input section 10 is generated by an input-image working section 12 and the transmitting image is transmitted at an opposite-side terminal. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an information processing apparatus to which a communication function for capturing a self-side image and a partner-side image is added. In particular, privacy protection of the self-side image and manipulation of the partner-side image are possible, and privacy is ensured while ensuring privacy. The present invention relates to an image information transmitting apparatus that enables easy communication.

  There are a videophone system and a video conference system as a system for communicating with a remote party through voice and video. In addition to the main voice of each other, they use a TV camera and a monitor to send their own video to a remote partner, receive and display the video from the partner, and view the partner's appearance. It is something that can be communicated. As a result, it can be said that these systems are close to normal communication compared to existing telephone systems that exchange only voice.

  In such a system, a TV camera is fixed at a fixed position, and an image only within a range captured by the TV camera is transmitted to the other side, and the received image is generally monitored on the receiver side. Is.

  However, in such a system, the viewpoint on the self side is determined by the position of the TV camera on the other side. Therefore, when the other party goes out of the imaging range of the video camera, or when the other party's face is at the end of the imaging range, there is a problem that the other party disappears from the transmitted image.

  Further, when the self side performs operations such as panning / tilting and zooming on the other party's television camera, a place contrary to the intention of the other side is photographed, resulting in inconvenience that the other party's privacy cannot be protected.

  Therefore, as a technique for solving this problem, Patent Document 1 reports a technique called “image information transmission device and image information transmission system”. An outline of this technique will be described below.

  In other words, while respecting the intentions of the self side and the partner side and protecting the privacy, the remote control of the screen can be performed so that the viewpoint can be freely moved within the range permitted by the partner.

Specifically, the sender side preselects a transmission permission range from the captured image, and the receiver side selects and receives an arbitrary image range within the transmission permission range. Thus, the receiver side can freely operate his / her line of sight to some extent.
JP 2001-53947 A

  However, in the case of the system reported in Patent Document 1, images within a range narrower than the imaging range of the television camera are transmitted and received, and the performance of each television camera cannot be fully utilized. Further, when the other party moves, the receiver side must perform an operation for chasing the other party, which is an operation burden on the user.

  As described above, in the conventional technology, when trying to communicate with a remote partner, for example, in the case of a videophone, the imaging range of the TV camera is not always appropriate, for example, when the other party goes out of the angle of view of the camera. In some cases, the other party's TV camera has to be panned and tilted on the receiver side, resulting in a burden on the user.

  In addition, when the other party's video camera is panned / tilted, a background image other than that of the other party is shot, so that the privacy of the other party cannot be ensured.

  The present invention has been made in view of the above, and as its purpose, it is not necessary to worry about the angle of view of the camera when communicating with each other while viewing their images, An object of the present invention is to provide an image information transmitting apparatus capable of operating the other party's camera with his own intention while respecting the other party's privacy.

  In order to solve the above problem, the invention according to claim 1 is an image acquisition unit that captures an image, a person image range specifying unit that specifies a person image range corresponding to a person from the image acquired by the image acquisition unit, Based on the person image range specified by the person image range specifying means, an angle of view control means for controlling the angle of view of the image acquired by the image acquisition means so as to capture the person at the center of the angle of view; and the person image range Based on the person image range specified by the person image range specifying means, the image range selecting means for setting a specified range wider than the person image range including the person image range specified by the specifying means, according to the instruction from the user, Generate a transmission image obtained by processing an image portion outside the human image range in the image supplied from the image acquisition means, or select the image range When the specified range is set by a stage, based on the specified range, an input image processing unit that generates a transmission image obtained by processing an image portion outside the specified range in the image supplied from the image acquisition unit; The gist of the present invention is that the transmission image output from the input image processing means is transmitted to a counterpart terminal.

  According to the image information transmitting apparatus of the present invention, an image acquired by the image acquisition unit is specified so that a person image range corresponding to a person is specified from the acquired image and the person is captured at the center of the angle of view based on the person image range. An image outside the person image range in the image supplied from the image acquisition means is set based on the person image range based on the person image range. Generate a transmission image by processing the part, or if the specified range is set by the image range selection unit, generate a transmission image by processing the image part outside the specified range in the image supplied from the image acquisition unit , By sending the transmission image to the other party's terminal, when trying to communicate while watching each other's own images, the other party is automatically tracked There is no need to worry about the angle of view of the camera, even while respecting the masking to the other side of privacy other than the party image in the imaging range, it is possible to manipulate the other side of the camera in my intention.

  The best mode for carrying out an image information transmitting apparatus according to the present invention will be described below with reference to the drawings. FIG. 1 is a configuration diagram of an image information transmitting apparatus according to an embodiment of the present invention.

  As shown in FIG. 1, the image information transmitting apparatus has two terminals as a minimum number of system components, and the terminals mutually send images on the own terminal side and monitor the image of the other terminal. Yes. In the present embodiment, the configuration of the image information transmitting apparatus is made up of two terminals as shown in FIG. 1, but in actual use, three or more terminals are prepared and the transmission paths are mutually connected. You may connect via.

  The image input unit 10 is a television camera installed in advance on a subject, has a zoom function and a pan / tilt function, and outputs an input image 110 as a signal. Further, a zoom operation and a pan / tilt operation can be performed by an image input control unit 13 described later.

  In addition, the person image range specifying unit 11 detects the range of the person image from the input image 110 using a technique such as image recognition for the input image 110 captured by the image input unit 10, and the person image range 111. Is output as a signal.

  The image input control unit 13 operates the image input unit 10 from the person image range 111 so that the subject is at the center of the angle of view, for example. Thereby, even when the user moves from the front of the camera, the image input control unit 13 can obtain an optimal input image by using the zoom function and the pan / tilt function of the image input unit 10. Thus, the burden on the user can be reduced.

  In addition, the image input control unit 13 receives the camera control information transmitted from the partner side among the information separated by the reception information separation unit 21 via the image input control unit 13, and receives the image input unit 10. Control. This makes it possible to receive and display images intended by each other by using the zoom function of the other camera, for example, when it is desired to look closely at the other person's facial expression.

  Further, the input image processing unit 12 uses the person image range 111 input from the person image range specifying unit 11 for the input image 110 input from the image input unit 10, and the input image from the image input unit 10. For example, mosaic processing is performed on an image outside the range of 110 person images. Further, by using an input from the image transmission range selection unit 15 to be described later, it is possible to transmit the image outside the person image range to the other party. This prevents the other party from seeing the background image that the sender does not intend to, so that privacy can be protected, and areas that the sender may disclose to the other party can be sent to the other party. This increases the degree of freedom for the user.

  The counterpart image control unit 31 outputs a signal for controlling the counterpart image input unit 10 to the counterpart image input control unit 13. An operation unit for the user to input information such as the direction he / she wants to see and zoom magnification to the counterpart terminal, for example, the user of this system performs zoom, pan / tilt operations on the counterpart camera An operation unit (for example, a cross cursor key, a mouse, a joystick, etc.) is prepared, and the input content is output to the transmission information combining unit 14 as a corresponding partner input image control signal.

  The transmission information combining unit 14 combines the transmission image 112 output from the input image processing unit 12 and the partner input image control signal output from the partner image control unit 31 to generate transmission information. The transmission / reception unit 20 sends the combined transmission information output from the transmission information combining unit 14 to the transmission path 310, receives transmission information from the other side transmitted through the transmission path 310, and sends it to the reception information separation unit 21. Output.

  The reception information separation unit 21 separates the reception information received via the transmission path 310, outputs the partner transmission image to the image display unit 40, and outputs the partner image control signal to the image input control unit 13. To do.

  When the image transmission range selection unit 15 may disclose a wider range to the other person than the person image range 111 indicating the range of the person image 112a, for example, FIG. ) And (b). 112b shown in FIGS. 4A and 4B is a transmission image when only the person image range determined by the person image range specifying unit 11 is shown on the other side, and 112c shown in FIG. This is a transmission image when the user sets an arbitrary range.

(Operation of image information transmission system)
Next, the operation of the image information transmitting apparatus will be described in detail with reference to FIGS.

  The input image 110 captured by the image input unit 10 is subjected to a restriction such as mosaic processing outside the person image range by the input image processing unit 12 that has received the person image range 111 output from the person image range specifying unit 11. The transmission image 112 is processed, the transmission image 112 is input to the transmission information combining unit 14, the partner input image control signal output from the partner image control unit 31 and the transmission image 112 are combined to form transmission information, and the transmission / reception unit 20. To the other party via the transmission line 310.

  As shown in FIG. 2A, the input image 110 obtained by the image input unit 10 includes a person image 110 a and a background image 110 b captured at one angle of view, and this input image is input to the person image range specifying unit 11. In this case, the person image range 111 output from the person image range specifying unit 11 is specified outside the person image 110a as shown in FIG. The input image processing unit 12 receives the input image 110 and the person image range 111, and outputs a transmission image 112 in which the background portion 112b of the person image 110a as shown in FIG. 2C is processed into a mosaic pattern. As a result, only the person image 112a is transmitted to the other party as a clear image, and the privacy of the person can be protected.

  When the subject moves to the end of the angle of view of the image input unit 10 or moves away from the image input unit 10, the image input control unit 13 receives the person image range input from the person image range specifying unit 11. The position of the subject is detected using 111, and the subject is automatically tracked by performing operations such as zooming, panning and tilting on the image input unit 10 (see FIG. 3).

  FIG. 3 shows an example of automatic tracking of a subject. 3, 100 indicates a range in which the image input unit 10 can shoot using the zoom and pan / tilt functions, 101 indicates a field angle of the image input unit 10, and 101a indicates a human image. FIG. 3A shows a case where the subject (101a) is located at the center of the angle of view. When the subject (101a) moves from the state of FIG. 3 (a) to the corner of the angle of view as shown in FIG. 3 (b), the person image range specifying unit 11 processes the person image range 111 after the subject movement as input image processing. To the unit 12 and the image input control unit 13. In response to this, the image input control unit 13 performs zoom, pan, and tilt operations so that the angle of view of the image input unit 10 becomes appropriate (FIG. 3C).

  Further, the range in which the input image processing unit 12 performs mask processing such as mosaic processing changes according to the movement of the subject. As a result, the image input unit 10 can capture the subject at the center of the angle of view as long as the subject does not fall outside the range 100 in which the image input unit 10 can shoot using the zoom and pan / tilt functions. Thus, the user does not need to pay attention to the angle of view of his / her camera, and can move freely within the imaging range of the camera, thereby reducing the user's troublesomeness.

  In this way, the image 112 processed by the input image processing unit 12 is sent from the transmission information combining unit 14 to the counterpart terminal via the transmission / reception unit 20 and the transmission line 310.

  In the case of the image information transmitting apparatus shown in the present embodiment, the image input unit 10 can be controlled by an operation from the counterpart terminal as will be described later, and the control is performed by the received information separating unit 21. This is performed by the image input control unit 13 according to the command information. As will be described in detail later, this command information is sent embedded in the image data from the other party.

  For example, when the present system is applied to a TV phone, a TV conference system, or the like, command information is transmitted from the transmission / reception unit 20 by operating the partner image control unit 31 of the terminal A on the self side.

  In other words, the counterpart image control unit 31 generates command information for causing the image input unit 10 ′ of the counterpart terminal B to perform input operations such as zoom, pan / tilt, etc. When command information such as zoom, pan / tilt, etc. is output from the image control unit 31 to the image input unit 10 ′ of the counterpart terminal B, this information is transmitted to another signal (the above input image processing unit) by the transmission information synthesis unit 14. 12 is combined with the transmission image 112) output by the transmission / reception unit 12 and then output to the transmission path 310 from the transmission / reception unit 20.

  On the other hand, in the counterpart terminal B, transmission information from the counterpart side sent via the transmission path 310 is received by the transmission / reception unit 20 ′, and this is given to the reception information separation unit 21 ′ to separate the information. By this separation process, the command information included in the information transmitted from the terminal A and the image information are separated. Of the separated information, the reception information separating unit 21 'gives the image information to the image display unit 40' to display it as an image, and the command information is given to the image input control unit 13 'to display the image of the image input unit 10'. Control the corners. As a result, for example, if the content of the command information is “designating panning of the camera”, the image input control unit 13 ′ pans the image input unit 10 ′, and the input image 110 obtained as a result thereof. 'Is subjected to masking processing such as mosaic processing outside the human image range by the human image range specifying unit 11' and the input image processing unit 12 ', and this image information is transmitted from the input image processing unit 12' to the transmission information combining unit. 14 'and the transmission / reception unit 20' are sent to the other party via the transmission line 310. Thereby, on the terminal A side, an image obtained by panning the image input unit 10 ′ on the terminal B side can be received and observed by the image display unit 40 of the own system.

  As a result, the self-side terminal sends zoom and pan / tilt operation commands to the other-side terminal, so that the user can freely move his gaze while respecting the privacy of the other side. It becomes possible to see.

  In the case where only the person image range determined by the person image range specifying unit 11 is shown on the other side with respect to the person image range 111 indicating the range of the person image 112a, the image transmission range selecting unit 15 is shown in FIG. ), The transmission image 112b obtained by mosaicing the background of the person image 112a is output from the input image processing unit 12.

  Further, when the range arbitrarily set by the user by the image transmission range selection unit 15 becomes wider than the human image range 111 determined by the human image range specifying unit 11, as shown in FIG. The person image 112a in the person image range 113 determined by the person image range specifying unit 11 is released together with the background, and the image transmission range is selected outside the person image range 113 determined by the person image range specifying unit 11. A transmission image 112 c in which a portion inside the range selected by the unit 15 is subjected to mosaic processing is output from the input image processing unit 12 to the transmission information combining unit 14.

  Thus, for example, by making the image transmission range wider than the person image range, the privacy protection restrictions can be relaxed by the other party, and the user can show not only his own image but also any area to the other party Is possible. Thereby, depending on the partner, the entire shooting range of the camera is disclosed. Alternatively, it is possible to make a selection such that a range slightly wider than the person image range is disclosed, or the face of the self-image is not disclosed. In order to select an image transmission range, for example, a frame line is displayed on the image display unit 40 on which an image is displayed using a pointing device such as a cross key or a mouse, and the size of the frame line is displayed. The position can be determined by the pointing device, and the user can set an area considered appropriate for protecting his / her privacy by the image transmission range selection unit 15, and the input image For example, mosaic processing may be performed on the processing unit 12 for a region outside the range selected by the image transmission range selection unit 15 and a region other than the person image range. Further, the range selected by the image transmission range selection unit 15 may be set so as to follow the person image range specified by the person image range specification unit 11. Thereby, for example, it is possible to show the other party what the user has in his hand.

  In addition, the image input control unit 13 has two types of input, that is, input from the person image range specifying unit 11 and input from the reception information separating unit 21, and the user may set which is prioritized.

  In addition, it is possible to provide a configuration for automatically setting the image transmission range at the initial time by providing a section for selecting the image transmission range at the initial time according to the additional information of the communication partner. The effort to set the image transmission range can be saved, and furthermore, the close friend and the first meeting salesman can be distinguished and dealt with, so that privacy can be more rationally protected.

  For these reasons, according to the image information transmitting apparatus of the present invention, the mental resistance and annoyance at the initial introduction of “communication using images”, which cannot be wiped off as a sensory trap, are generally provided. Can be reduced.

  Although various embodiments have been described above, in short, the present invention can capture images of each other by the image input unit, send them to the other side, and display them on the image display unit provided on each side for use in dialogue. In the image information transmitting apparatus configured as described above, by using an imaging range having a zoom, pan / tilt function for each image input unit, it is possible to freely determine its own line of sight, and the image input unit obtained By specifying the person image range from the image, it becomes possible to automatically capture each other's own images at an appropriate position within the imaging range, and perform processing such as mosaic processing outside the person image range. By transmitting to the partner terminal after applying the image information, it is possible to obtain an image information transmitting apparatus capable of proceeding with the conversation while protecting each other's privacy.

  According to the present invention, in a conventional video conference system or videophone system, it is impossible to move one's line of sight according to one's intention only by looking at a screen sent from the other party, or one's own line of sight This is a technology that compensates for the disadvantage of not being able to protect the privacy of the other party when it is moved with the intention of the user. While respecting each intention and maintaining privacy, it is possible to provide a technology that can move one's line of sight according to one's own intention.

  In particular, in the image information transmitting apparatus of the present invention, since a person image range can be specified from a captured image and the camera can be tracked thereby, there is no problem even if the self side moves freely within the imaging range of the camera. Accordingly, there is no need to worry about the angle of view of the camera as in the prior art, and the subject can be placed in communication.

  In addition, by using a camera equipped with zoom and pan / tilt functions, it is possible to send a high-quality image rather than a means of photographing a wide range with a wide-angle lens camera and transmitting a part of the image to the other party. .

  In addition, since the image transmission range can be changed depending on the other party, or can be set from time to time, this enables flexible communication.

  Therefore, the present invention is a communication in which the speakers can see each other's desired images and can protect the privacy on the sender side, such as communication in which they meet at the same place and face each other. A system can be provided.

  In addition, this invention is not limited to the Example mentioned above, A various deformation | transformation can be implemented.

  In the present invention, when communicating with each other by sending their own images and watching the images, the pan / tilt function of the video camera is dynamically controlled so that the subject is always properly imaged. Mosaic and other processing can be performed outside the human image range, and the video cameras on the other side can be panned and tilted to reduce the burden on the user and protect privacy. And a technique for enabling the user to obtain a desired image.

  As an important technique for reducing the burden on the user and ensuring privacy, the present invention specifies a person image range, dynamically tracks the subject, and is outside the person image range in the shooting range. By applying mosaic processing to the other party, you can protect the other party's privacy even if you zoom, pan, or tilt the other party's camera when you want to see the other person's facial expression or see the other person's clothes. And an image desired by the user can be obtained.

  The image information transmitting apparatus of the present invention can be applied to a video phone, a video conference system, a video chat, and the like.

It is a block block diagram which shows the Example of the image information transmitter of this invention. It is a figure for demonstrating the person image range specific example in the Example of this invention. It is a figure for demonstrating the example which performs zoom, pan, and tilt operation of the image input part in the Example of this invention. It is a figure for demonstrating the example which selects an arbitrary range in the image transmission range selection part in the Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image input part 11 Person image range specific | specification part 12 Input image processing part 13 Image input control part 14 Transmission information synthetic | combination part 15 Image transmission range selection part 20 Transmission / reception part 21 Reception information separation part 31 Counterpart image control part 40 Image display part 100 Image Range in which the input unit can shoot using the zoom and pan / tilt functions 101 Angle of view of the image input unit 110 Input image 111 Input image range 112 Transmission image 210 Reception image 310 Transmission path

Claims (1)

Image acquisition means for capturing images;
A person image range specifying means for specifying a person image range corresponding to a person from the image acquired by the image acquisition means;
An angle-of-view control means for controlling the angle of view of the image acquired by the image acquisition means so as to capture the person at the center of the angle of view based on the person image range specified by the person image range specifying means;
Image range selection means for setting a specified range including the person image range specified by the person image range specifying means and wider than the person image range by an instruction from the user;
Based on the person image range specified by the person image range specifying unit, a transmission image is generated by processing an image portion outside the person image range in the image supplied from the image acquiring unit, or the image range selecting unit When the specified range is set by the input image processing means for generating a transmission image obtained by processing the image portion outside the specified range in the image supplied from the image acquisition means based on the specified range;
An image information transmitting apparatus comprising: a transmitting unit that transmits the transmission image output from the input image processing unit to a counterpart terminal.

Images