JP2007019671A - Image communication system and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image communication system in which a concealment method can be set easily for each individual in detail. <P>SOLUTION: The image communication system comprises an apparatus 10 for photographing a single image or photographing images continuously, a device 30 for replacing a portion of the image by other image, and a device 50 for transmitting an acquired image to other point. The photographing apparatus 10 has a means 101 for recognizing the region of a figure from a photographed image, and a means 102 for clipping the image of a region recognized by the figure region recognizing means 101. The image replacing device 30 replaces the image extracted by the figure region clipping means 102 by other image, and the image transmitting means 50 transmits the image replaced by the image replacing device 30. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image communication system used for videophones, video conferences, and the like, and more particularly to an image communication system that can conceal an arbitrary person image and transmit it to the other party and a program thereof.

In recent years, with the expansion of broadband communication lines, the opportunity to transmit and receive moving images over the Internet and the like has greatly increased, and videophone / videoconferencing systems etc. have been implemented using such video transmission functions. Sometimes.
For example, in a project that is being developed at multiple locations, video conferences can be established by installing computers equipped with photographing devices in the conference rooms, break rooms, and lounges at each location. This makes it possible to facilitate smooth communication between existing persons, and is effective as a system that supports large-scale projects and the like that are implemented at multiple sites.

However, when images including such videos are communicated, information related to personal attributes such as the face and appearance of people is exchanged, so the handling of image information is restricted from the standpoint of personal information protection. May be required.
That is, it is not preferable to always transmit all information photographed by the photographing apparatus in terms of privacy and security, and some concealment processing of image information may be required depending on the situation.
Therefore, conventionally, an image processing method, an image processing apparatus, an image communication apparatus, and the like that can hide only a portion that the user wants to hide by a simple operation while protecting the privacy of the person to be photographed have been proposed (for example, , See Patent Document 1).

JP 2004-056488 A (page 1-2, FIG. 17)

However, in the conventionally proposed image processing method and the like, when there are a plurality of persons in the image, it is impossible to specify individual persons other than the user judging by looking at the image. .
For this reason, it is difficult to set the concealment method in detail for each individual, and in order to set the concealment area, the user must manually set the area range while looking at the screen. There were problems such as difficulty and annoyance.

  The present invention has been proposed in order to solve the above-described problems of the prior art, and human images in a captured image used in a videophone, a video conference system, etc., for each arbitrary individual. An object of the present invention is to provide an image communication system that can easily set the details of the display method and the like.

  In order to achieve the above object, an image communication system according to the present invention includes, as described in claim 1, an imaging device that captures a transmission image, an image replacement device that replaces a part of the transmission image with another image, An image transmission device that transmits a transmission image to another device, wherein the photographing device is recognized by a person region recognition unit that recognizes a person region from the photographed transmission image and the person region recognition unit. A person area cutout unit that extracts an image of the area, and the image replacement device includes a person area replacement unit that replaces the image extracted by the person area cutout unit with a predetermined replacement image, and the image transmission apparatus. However, the configuration is such that the image subjected to the replacement processing by the person region replacement means of the image replacement device is transmitted.

According to the image communication system of the present invention having such a configuration, it is possible to recognize and extract only a person image from a captured transmission image as a transmission image transmitted / received in a videophone / video conference or the like. it can.
In addition, when there are a plurality of persons in the captured transmission image, the images of the persons can be recognized and extracted individually.

Accordingly, it is possible to set, for each individual, a determination related to a person display method, such as necessity of concealing a person image and selection of a replacement image related to concealment, according to the nature and situation of the video conference.
Therefore, before transmitting a captured image, each person included in the image can be subjected to different image processing, and an image communication system having various variations in the person display method and concealment method and Provision is possible.

  According to a second aspect of the present invention, there is provided an image communication system including a person recognizing device that holds personal attribute information and identifies a person by collation, and the person recognizing device includes the photographing device. A personal identification unit for recognizing and identifying an individual person from the shooting range, and the image replacement device replaces the image according to the attribute information of the individual identified by the personal identification unit. .

According to the image communication system of the present invention having such a configuration, by holding personal attribute information for specifying a person by the person recognition device, for example, RFID (Radio Frequency Identification), face recognition, etc. By using various person recognition methods, it is possible to specify the person in the shooting range and set the necessity of concealment and selection of a replacement image used for concealment for each individual.
As a result, it is possible to more easily and reliably determine the person display method.

  The image communication system according to the present invention further includes a position recognition camera that recognizes the position of the person when the image including the person photographed as the transmission image is photographed from above. The recognition camera includes a person area recognition unit that recognizes a person area from an image captured by the position recognition camera, a person area cutout unit that extracts an image of the area recognized by the person area recognition unit, and the person area cutout. Human position recognition means for recognizing the position of the region extracted by the means and converting it into predetermined position information, and the photographing device extracts the region extracted by the photographing device by the position recognition camera. A plurality of person area association means for associating the obtained area and / or position information, and the image replacement device is obtained by the person position recognition means. It is constituted to replace an image in accordance with the position information.

According to the image communication system of the present invention having such a configuration, the position recognition camera can identify and recognize the position of the person in the image captured by the image capturing apparatus that captures the transmission image.
Accordingly, it is possible to make a determination relating to a person display method, such as necessity of concealment for each person and selection of a replacement image for concealment, according to the position of the person.
In addition, the above determination can be automatically controlled in accordance with the data value of the position information, which makes it possible to construct and provide a more convenient image communication system.

  According to the image communication system of the present invention, as described in claim 4, the photographing apparatus is configured to acquire attribute information of each individual held by the person recognition apparatus in the person recognition apparatus. In this configuration, the person area extracted by the photographing apparatus is associated with the person area.

  In the image communication system of the present invention, as described in claim 5, the photographing apparatus has the attribute information of each individual held in the person recognition apparatus by the person area association unit between the multiple images. In this configuration, the person area and / or position information extracted by the position recognition camera is associated.

According to the image communication system of the present invention having such a configuration, it is possible to integrally display the attribute information and position information of the person in the person area extracted from the image photographed by the photographing apparatus. .
As a result, the user can confirm these pieces of information only by looking at one screen, and can easily make a judgment regarding the person display such as necessity of concealment and selection of a replacement image for concealment based on the information. .
Furthermore, the above determination can be automatically controlled according to the range and value of the data constituting the personal attribute information, and it is possible to construct and provide a more convenient image communication system. .

  In the image communication system of the present invention, as described in claim 6, the imaging device and / or the position-recognition camera extracts from one captured image when capturing continuous images. An overlap detection means is provided for detecting an overlap between the person area and the person area extracted from the one image captured thereafter.

According to the image communication system of the present invention having such a configuration, movement of a person can be tracked by detecting an overlap of person areas in a transmission image taken by a photographing apparatus.
This makes it possible to always recognize and extract a person's region even when shooting a moving image.
Therefore, the image communication system according to the present invention can be widely used as a transmission image regardless of whether it is a still image or a moving image, and a video conference system excellent in convenience, versatility, and expandability can be realized.

  According to a seventh aspect of the present invention, there is provided an image processing program, a photographing device for photographing a transmission image, an image replacement device for replacing a part of the transmission image with another image, An image processing program for causing an image transmission apparatus to transmit to an apparatus to function as an image communication system, wherein a computer constituting the imaging apparatus recognizes a person area from a captured image , Functioning as a person area cutout means for extracting an image of the area recognized by the person area recognition means, and causing the computer constituting the image replacement device to use the image extracted by the person area cutout means as another image. In addition to functioning as a person area replacement means to be replaced, a computer constituting the image transmitting apparatus is used by the person area replacement means. Means for transmitting the image after being replaced is a program for functioning as a.

Thus, the present invention can also be provided as a program.
Thus, the present invention can be realized by installing a program in a device having a photographing function and a communication function, for example, a portable information terminal, a video phone, a video conference device, a personal computer, etc. Can be provided as an excellent image processing program.

According to the image communication system of the present invention, it is possible to set the details of a person image in a photographed image used in a videophone, a video conference system, etc. for each arbitrary individual in which the display method is easily overcome.
As a result, the necessity of concealing human images, selection of replacement images related to concealment, etc. can be set for each individual, preventing inadvertent leakage of personal information, etc. easily and reliably, and convenience and convenience An excellent video conference system can be realized.

A preferred embodiment of the present invention will be described below with reference to FIGS.
Here, the image communication system of the present invention described below is realized by processing, means, and functions executed by a computer in accordance with instructions of a program (software). The program sends a command to each component of the computer to perform predetermined processing and functions as shown below. That is, each processing / means in the image communication system of the present embodiment is realized by specific means in which a program and a computer cooperate.
Note that all or part of the program is provided by, for example, a magnetic disk, optical disk, semiconductor memory, or any other computer-readable recording medium, and the program read from the recording medium is installed in the computer and executed. The The program can also be loaded and executed directly on a computer through a communication line without using a recording medium.

[First embodiment]
FIG. 1 is a block diagram showing an outline of an image communication system according to the first embodiment of the present invention.
The image communication system 1 according to the present embodiment is installed in a conference room, a break room, a common room, etc. of each base in a project that is developed at a plurality of bases, for example. It is an image communication system for conducting a conference.
Specifically, the image communication system 1 of the present embodiment includes an imaging device 10, a position recognition camera 20, an image replacement device 30, a moving image compression device 40, an image transmission device 50, a display method setting storage device 60, and avatar information storage. The apparatus 70, the display method input apparatus 80, and the person recognition apparatus 90 are configured to be installed at each site where a video conference or the like is performed.

The photographing device 10 is photographing means for photographing an image that is a basis of a transmission image transmitted to the other party.
A plurality of the position recognition cameras 20 are installed above a person such as a ceiling, are photographing means for photographing the top of the human head, and the person position recognition means 203 recognizes the position of the person.
The image replacement device 30 is a means for performing an image replacement process for replacing a part of the image captured by the image capturing device 10 with another image (avatar information) stored in the avatar information storage device 70.

The moving image compression device 40 is a means for compressing the image data of the image photographed by the photographing device 10 and replaced by the image replacement device 30, thereby enabling efficient data transmission.
The image transmission device 50 is a transmission unit that transmits the image data compressed by the moving image compression device 40 to another point designated in advance.
The display method input device 80 is an input means for a user to input and set a display method, and includes a touch panel or the like.
When performing the concealment process by image replacement, the user can search and select an image used for the replacement from the avatar information storage device 70.

The avatar information storage device 70 stores an image used for the replacement and provides a necessary image (avatar information) according to a predetermined setting, and is configured by a storage medium such as a memory or a hard disk.
The avatar information can be subjected to concealment processing rich in variations by storing various images such as a person's cartoon and background.
The display method setting is stored in the display method setting storage device 60.
The person recognition device 90 is means for recognizing and specifying a person by various methods such as RFID and face recognition.
The person recognition apparatus 90 stores personal attribute information in a storage medium such as its own memory or hard disk in advance, and is used for specifying a person.

Next, the overall operation of the image communication system according to the first embodiment configured as described above will be described with reference to FIGS.
First, in this system, an image that is a basis of a transmission image that is actually transmitted to the other party is captured by the imaging device 10.
A plurality of position recognition cameras 20 are installed above a person such as a ceiling to photograph the human head.
The image taken by the position recognition camera 20 is used for detecting the position of the person in the real world and also for tracking the same person, so that the occlusion (missing information) does not occur at a position as high as possible. Install.

Next, the person area recognition means 101 and 201 recognize the person area in the captured image.
Specifically, after performing background difference processing (processing for extracting only a person image by subtracting only the background image from the image) from the images captured by the imaging device 10 and the position recognition camera 20, noise is generated by expansion / contraction processing or the like. Remove.
An image is a set of pixels, and each pixel has values such as luminance, color difference, red, green, and blue. For this reason, normal addition and subtraction can be performed. Assuming that an image in which no person exists is a “background image” and an image in the presence of a person is “person + background image”, “person + background image” to “background image” as shown in Equation 1 below. By subtracting and taking the absolute value, a “person area” is obtained.
Formula 1: | "person + background image"-"background image" | = "person area"

Next, a labeling process (a process of grouping and numbering which person's area each pixel of the person image extracted by the background difference process belongs to) is binarized. Hold as an image.
Even if the “person area” is defined as a portion other than 0 on the pixel, it cannot be determined which pixel is which person. Therefore, by renumbering pixels other than 0, it is possible to determine which person region each pixel belongs to.
Usually, it can be executed by determining one pixel in a certain person area and reassigning the same number to a non-zero pixel in the vicinity thereof by a serial number.

Specifically, the pixel of the person area is 1 and the pixel of the background area is 0 (zero).
Thereafter, the person area cutout means 102 and 202 extract a person area from the photographed image.
Then, an image in which only one portion is extracted from the binary image information created by the person area recognition means 101 and 201 is created.

Next, the position of the person photographed by the position recognition camera 20 is recognized by the person position recognition means 203.
Specifically, the position of the center of gravity on the image of the person area extracted by the person area cutout unit 202 is converted into real world position information using stereo parallax, and the position information of the person is calculated.
When two or more cameras are used to photograph the same object, the image is taken from the relationship between the position of the object on the image obtained from each camera and the actual position of each camera (three-dimensional position in the real world). If the position of the selected object on the image is known, the actual position can be calculated. This is position information acquisition using stereo parallax.
If one of the three-dimensional positions (X, Y, Z) of the photographed object is known (for example, the height of the floor surface is fixed), two cameras are not required and one camera is used. The actual position of the photographed object can be calculated from the position on the photographed image. This is position information acquisition by monocular vision.

Then, the inter-image person area association means 103 associates the image photographed by the photographing device 10 with the image photographed by the position recognition camera 20 and the calculated position information.
Specifically, as shown in FIG. 2, the person area extracted by the person area cutout unit 202 is associated with the person area extracted by the person area cutout unit 102 using projective transformation.
According to the projective transformation, it is possible to know where an object on an image captured by one camera is projected on an image captured by a second camera, based on the same principle as the stereo parallax described above. If this principle is used, if a person is specified on one image, it becomes possible to know which person area is the person area on the second and subsequent images.

Note that it is possible to associate attribute information such as the name of the person specified by the person recognition device 90 together with the position information or instead of the position information.
As described above, the person areas are recognized and extracted from the images taken by the photographing apparatus 10 and the position recognition camera 20, the areas are made to correspond to each other, and the position information and attribute information corresponding to the person areas are assigned. I am doing so.

In addition, in order to cope with the movement of a person in a moving image, in this embodiment, as shown in FIG. 3, the overlapping of human regions between frame images (between n frames and n + 1 frames shown in FIG. 3) is detected. By doing so, the movement of the person is tracked.
This makes it possible to accurately recognize and extract a person's area even when shooting a moving image. Therefore, in this system, it is possible to target a transmission image regardless of a still image or a moving image.

Next, the image replacement device 30 receives a part of the image photographed by the photographing device 10 and the avatar information storage device based on the setting of the display method setting storage device 60 through the input by the display method input device 80. The avatar information stored in 70 is replaced and drawing is performed.
Here, the image communication system according to the present embodiment can perform control for automatically setting a display method in accordance with the position information, for example.
In addition, when occlusion occurs, it is possible to set a drawing with a more restrictive person.

Specifically, as shown in FIG. 4, since the person A has avatar setting, it is replaced with another image and displayed.
The person B is not displayed because of the non-display setting, or is displayed transparently by using a background image.
Here, the person C has a display setting, but since the occlusion has occurred with the person A, it may be set in advance so that the avatar display is strongly restricted, that is, displayed by a predetermined replacement image. Is possible.
Although the person D is set to be non-displayed, the display method can be automatically controlled according to the position of the person, such as displaying a raw image because the person D is in front of the photographing apparatus 10.
Then, the image obtained through the above process is compressed by the moving image compression apparatus 40, transmitted to the target point (the other party system) by the image transmission system 50, and a videophone or a video conference is performed. .

As described above, according to the image communication system 1 of the present embodiment, only the image of a person is recognized and extracted from the images captured by the image capturing device 10.
Further, even when there are a plurality of persons in the photographed image, the images of the persons are individually recognized and extracted.
For this reason, it is possible to determine whether or not to conceal for each person, and to set what image to replace when concealing.

The replacement image can also be set for each individual after the person recognition device 90 identifies a person in the shooting range.
Furthermore, the replacement image can be set based on the position information of the person acquired by the position recognition camera 20.
Therefore, before transmitting a captured image, each person included in the image can be subjected to different image processing, and an image communication system having various variations in the display method and concealment method of the person can be constructed. And can be provided.

[Second Embodiment]
Next, an image communication system according to a second embodiment of the present invention will be described with reference to FIG.
FIG. 5 is a block diagram showing an outline of an image communication system according to the second embodiment of the present invention.
Unlike the first embodiment described above, this embodiment does not include the person recognition device 90. Therefore, position information such as the distance from the personal screen is used to determine the person display method.

For example, a person who is more than a certain distance from a predetermined reference point is hidden, a person who is within a predetermined distance range is displayed as an avatar, and a person who is within a certain distance range from the reference point is displayed in a raw drawing. By making such settings, automatic control can be performed.
As described above, according to the image communication system of the second embodiment of the present invention, not only the same effect as the first embodiment is exhibited but also the configuration that does not include the person recognition device 90 is relatively simple. It is possible to construct a simple system and to provide an inexpensive system.

The image communication system according to the present invention has been described with reference to the preferred embodiment. However, the image communication system according to the present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the present invention. It goes without saying that implementation is possible.
For example, in the image communication system of the present invention, various means are incorporated in the photographing apparatus 10 or the position recognition camera 20, but these means are incorporated in an external image processing apparatus or the like and connected to each other for cooperation. There may be.
Further, in the present embodiment, the image communication system that targets a person and conceals it for transmission has been described. However, the application range can be further expanded by targeting an object.
For example, the scope of application of the system is further expanded by targeting personal information such as name and address, national secrets and trade secrets.

  The present invention can be suitably used for an imaging apparatus having a communication function.

It is a block diagram which shows the outline of the image communication system which concerns on 1st embodiment of this invention. It is a figure which shows matching between the several images in 1st embodiment of this invention. It is a figure which shows the principle regarding the person area tracking method which concerns on 1st embodiment of this invention. It is a figure regarding the person display method and its setting which concern on 1st embodiment of this invention. It is a block diagram which shows the outline of the image communication system which concerns on 1st embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image communication system 10 Image pick-up device 101 Person area recognition means 102 Person area cutout means 103 Interpersonal area | region correlation means between multiple images 20 Position recognition apparatus 201 Person area recognition means 202 Person area cutout means 203 Person position recognition means 30 Image replacement apparatus 40 Moving image compression device 50 Image transmission device 60 Display method setting storage device 70 Avatar information storage device 80 Display method input device 90 Person recognition device 901 Personal identification means

Claims (7)

An imaging device that captures a transmission image, an image replacement device that replaces a part of the transmission image with another image, and an image transmission device that transmits the transmission image to another device,
The imaging device
A human area recognition means for recognizing a human area from the captured transmission image;
A person area clipping means for extracting an image of the area recognized by the person area recognition means,
The image replacement device comprises a person area replacement means for replacing the image extracted by the person area cutout means with a predetermined replacement image;
An image communication system, wherein the image transmission device transmits an image subjected to replacement processing by a human region replacement unit of the image replacement device. Possessing personal attribute information, equipped with a person recognition device to identify a person by matching,
This person recognition device
Recognizing and identifying individual persons from the photographing range of the photographing apparatus,
2. The image communication system according to claim 1, wherein the image replacement device replaces an image according to personal attribute information specified by the individual specifying means. A position recognition camera for recognizing the position of a person by photographing an image including a person photographed as a transmission image from above;
The position recognition camera is
A person area recognition means for recognizing a person area from a photographed image;
A person area clipping means for extracting an image of the area recognized by the person area recognition means;
Recognizing the position of the area extracted by the person area cutout means and converting the position into predetermined position information,
The imaging device
A plurality of person area correlation means for associating the area extracted by the position recognition camera and / or position information with the area extracted by the photographing apparatus;
3. The image communication system according to claim 1, wherein the image replacement device replaces an image according to position information obtained by the person position recognition means. The imaging device
4. The image communication according to claim 3, wherein the inter-multi-person person area association means associates the attribute information of each individual held in the person recognition apparatus with the person area extracted by the photographing apparatus. system. The imaging device
The multi-image person area association means associates attribute information of each individual held in the person recognition apparatus with a person area and / or position information extracted by the position recognition camera. Item 5. The image communication system according to Item 3 or 4. The imaging device and / or the position recognition camera is
In the case of capturing continuous images, the image processing apparatus includes an overlap detection unit that detects an overlap between a person region extracted from one captured image and a person region extracted from the one captured image. The image communication system according to any one of claims 1 to 5. An image for causing an imaging device that captures a transmission image, an image replacement device that replaces a part of the transmission image to another image, and an image transmission device that transmits the transmission image to another device as an image communication system A processing program,
A computer constituting the photographing apparatus;
While functioning as a person area recognition means for recognizing a person area from the captured image, a person area cutout means for extracting an image of the area recognized by the person area recognition means,
Causing the computer constituting the image replacement device to function as a person region replacement unit that replaces the image extracted by the person region cutout unit with another image;
An image processing program for causing a computer constituting the image transmitting apparatus to function as means for transmitting an image subjected to replacement processing by the person region replacement means.

Images