JP7004460B2 - Image processing device, image processing method and image processing program - Google Patents

Description

The present invention relates to an image processing technique.

A foreground extraction technique for extracting an image area (foreground) of an object that does not exist in the background image by comparing a background image prepared in advance with a reference image is known. As a technique for extracting the foreground, there is a method of taking a difference for each pixel based on the brightness and extracting a portion having a large difference as the foreground. Further, as a technique for extracting the foreground, there is a method of calculating the similarity of the luminance distribution for each local region of the image and extracting the portion having a low similarity as the foreground.

When using these foreground extraction technologies for images taken outdoors, the brightness value of the background changes due to changes in sunshine conditions such as sunlight and shadows, so areas that should not be the foreground are extracted as the foreground. Mis-extraction occurs.
In particular, in an environment where a structure such as a building is included in the background, a specific part of the background structure, such as the edge of the structure or the surface of a glossy material such as metal or a smooth material, strongly reflects sunlight. The local brightness distribution pattern changes even in the background area where the foreground does not exist. Since such a partial change in the background is extracted as a change in brightness as in the foreground, it is not possible to distinguish between the area where reflection is shown and the area in the foreground by the conventional method, and the area that is not originally the foreground is mistaken. It may be the foreground.

As a technique related to the present invention, there are a technique disclosed in Patent Document 1 and a technique disclosed in Patent Document 2.

JP-A-2015-46811 Japanese Unexamined Patent Publication No. 2001-209808

In the technique disclosed in Patent Document 1, attention is paid to the property that the textures of the background image and the photographed image are similar in the region where the shadow derived from the shape of the structure is erroneously extracted, and the portion having a high degree of texture similarity. By excluding the above, erroneous extraction is reduced.
However, for this purpose, it is premised that the texture can be sufficiently extracted from the captured image and the background image of the erroneously extracted area, and if the image has overexposure or underexposure, the texture cannot be sufficiently extracted. , The foreground is erroneously extracted.

In the technique disclosed in Patent Document 2, for the problem that the shadow of an object is erroneously extracted as the foreground, edge detection is performed on the captured image to extract the outline of the foreground object, and the erroneously extracted shadow area and the photograph are taken. By comparing the edges of the images and removing the foreground region away from the edges of the captured image, erroneous extraction is suppressed.
However, erroneous extraction of the foreground due to the reflection of light by the edges of the structure occurs on the contour of the object reflected in the background image and the captured image. It is not possible to distinguish between the foreground existing in the area and the reflection of the structure, and it is not possible to suppress erroneous extraction.

An object of the present invention is to solve such a problem. More specifically, it is a main object of the present invention to accurately separate the reflection region and the foreground region in the original image in which the foreground object is shown overlapping a part of the background object.

The image processing apparatus according to the present invention is
A foreground candidate image generation unit that generates a foreground candidate image showing a foreground candidate that is a candidate for the area of the foreground object from an original image in which the foreground object as the foreground overlaps with a part of the background object as the background.
A background edge image showing the edge of the background object is acquired, and the foreground candidate image and the background edge image are used to show a state in which light is reflected on the background object included in the foreground candidate image. It has a region separating portion that separates the reflection region and the region of the foreground object.

According to the present invention, the reflection region and the foreground region can be accurately separated.

The figure which shows the functional structure example of the image processing apparatus which concerns on Embodiment 1. FIG. The figure which shows the hardware configuration example of the image processing apparatus which concerns on Embodiment 1. FIG. The flowchart which shows the operation example of the image processing apparatus which concerns on Embodiment 1. The flowchart which shows the detail of the extraction separation process which concerns on Embodiment 1. The flowchart which shows the detail of the overlap edge image generation processing which concerns on Embodiment 1. The flowchart which shows the detail of the area extraction processing which concerns on Embodiment 1. The figure which shows the example of the original image which concerns on Embodiment 1. The figure which shows the example of the background object which concerns on Embodiment 1. FIG. The figure which shows the example of the foreground candidate image which concerns on Embodiment 1. The figure which shows the example of the background edge image which concerns on Embodiment 1. FIG. The figure which shows the example of the overlap edge image which concerns on Embodiment 1. FIG. The figure which shows the example of the reflection area image which concerns on Embodiment 1. FIG. The figure which shows the example of the foreground extraction image which concerns on Embodiment 1. The figure which shows the functional configuration example of the image processing apparatus which concerns on Embodiment 2. The flowchart which shows the operation example of the image processing apparatus which concerns on Embodiment 2. The flowchart which shows the detail of the 2nd overlap edge image generation processing which concerns on Embodiment 2. The figure which shows the example of the whole edge image which concerns on Embodiment 2. The figure which shows the example of the 1st overlap edge image which concerns on Embodiment 2. The flowchart which shows the operation example of the image processing apparatus which concerns on Embodiment 3. The flowchart which shows the detail of the 2nd overlap edge image generation processing which concerns on Embodiment 3.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description and drawings of the embodiments, those having the same reference numerals indicate the same parts or corresponding parts.

Embodiment 1.
In this embodiment and subsequent embodiments, the foreground is extracted in order to accurately extract the foreground even in an environment where reflection is generated at the edge of the structure due to local sunshine such as outdoors. Among them, an image processing device that distinguishes between a reflective portion at an edge portion of a structure and a correct foreground portion will be described.
In the image processing apparatus according to the present embodiment and subsequent embodiments, when the reflection of the edge portion of the structure is erroneously extracted as the foreground, the region has a shape along the contour of the structure. Of the foreground areas extracted by focusing on the existing properties and using the edge information extracted from the background image to determine the part of each area of the extracted foreground that is along the edge of the structure. Determine the component derived from the reflection of the edge portion of the structure.

*** Explanation of configuration ***
FIG. 1 shows an example of a functional configuration of the image processing apparatus 20 according to the first embodiment. The image processing device 20 is connected to a camera 10 that captures and distributes an image.
The operation performed by the image processing device 20 corresponds to an image processing method and an image processing program.

The video reception / decoding unit 201 decodes the compressed video stream delivered from the camera 10 and converts it into a video frame. The video frame corresponds to the original image.
In the video frame, it is assumed that the foreground object, which is the foreground, is shown overlapping with a part of the background object, which is the background.

The foreground candidate image generation unit 202 generates a foreground candidate image from the video frame.
The foreground candidate image is an image showing the foreground candidate. Foreground candidates are candidates for the foreground area. Foreground candidates may include reflection areas within the background as well as areas of the foreground. The reflection area is an area showing a state in which light is reflected on the background object.

The area separation unit 203 acquires a foreground candidate image from the foreground candidate image generation unit 202 and acquires a background edge image from the storage unit 204. Then, the region separation unit 203 separates the reflection region and the foreground object region included in the foreground candidate image by using the foreground candidate image and the background edge image.
More specifically, the region separation unit 203 performs a luminance gradient comparison between the foreground candidate image and the background edge image, and shows an edge portion of the background object of the background edge image that overlaps with the foreground candidate. Generate duplicate edge images. Then, the area separation unit 203 performs a logical product operation of the overlapping edge image and the foreground candidate image, and extracts the reflection area from the foreground candidate image. Further, the area separation unit 203 subtracts the overlapping edge image from the foreground candidate image to extract the area of the foreground object.
Further, the region separation unit 203 outputs a reflection region image in which the reflection region is shown and a foreground extraction image in which the foreground object region is shown to the output unit 205.

The storage unit 204 stores analysis parameters and a background edge image.

The output unit 205 outputs the reflection region image and the foreground extraction image to the outside. For example, the output unit 205 outputs the reflection region image and the foreground extraction image to a display device connected to the image processing device 20.

FIG. 2 shows a hardware configuration example of the image processing apparatus 20 according to the present embodiment.

The image processing device 20 according to the present embodiment is a computer.
The image processing device 20 includes a processor 6101, a memory 6102, a network interface 6103, and a storage device 6104 as hardware.
The storage device 6104 stores a program that realizes the functions of the video reception / decoding unit 201, the foreground candidate image generation unit 202, the area separation unit 203, and the output unit 205.
These programs are loaded from the storage device 6104 into the memory 6102. Then, the processor 6101 executes these programs to operate the video reception / decoding unit 201, the foreground candidate image generation unit 202, the area separation unit 203, and the output unit 205.
FIG. 2 schematically shows a state in which the processor 6101 is executing a program that realizes the functions of the video reception / decoding unit 201, the foreground candidate image generation unit 202, the area separation unit 203, and the output unit 205.
Network interface 6103 receives a video stream from the network.

*** Explanation of operation ***
Next, an operation example of the image processing apparatus 20 according to the present embodiment will be described with reference to the flowchart of FIG.

First, in the analysis parameter reading process (step ST01), the foreground candidate image generation unit 202 acquires the analysis parameter from the storage unit 204.
The analysis parameter is a parameter used by the foreground candidate image generation unit 202 to generate the foreground candidate image from the video frame. The analysis parameters include parameters such as a threshold value used for extracting the foreground candidate.

In the video stream reception process (step ST02), the video reception decoding unit 201 receives the compressed video stream from the camera 10 via the network interface 6103. Then, the video reception / decoding unit 201 decodes the video frame from the compressed video stream.

In the extraction separation process (step ST03), the foreground candidate image generation unit 202 generates a foreground candidate image from the video frame, and the area separation unit 203 separates the reflection region and the foreground object region, and the reflection region image and the foreground. Generate an extracted image.
The details of the extraction / separation process (step ST03) will be described later.

In the output process (step ST04), the output unit 205 outputs the reflection region image and the foreground extraction image.

While the compressed video stream is being transmitted from the camera 10, steps ST02 to ST04 are repeated.

Next, the details of the extraction / separation process (step ST03) will be described with reference to FIG.

In the foreground candidate image generation process (step ST11), the foreground candidate image generation unit 202 generates a foreground candidate image from the video frame using the analysis parameters.
The foreground candidate image generation unit 202 extracts as a foreground candidate an area having different attributes from the area of the background object (hereinafter referred to as the background area) of the video frame.
For example, the foreground candidate image generation unit 202 takes the difference between the brightness of the background region and the brightness of other regions in the video frame. Then, the foreground candidate image generation unit 202 sets the pixel whose difference is equal to or greater than the threshold value as the foreground candidate. Further, the foreground candidate image generation unit 202 may use another region in the video frame having a luminance distribution pattern different from the luminance distribution pattern in the vicinity of the pixel of interest in the background region as the foreground candidate. Further, the foreground candidate image generation unit 202 generates a foreground candidate for each video frame, accumulates the foreground candidates generated for each video frame for several frames, and selects a foreground candidate from the foreground candidates for several frames. You may.

In the present embodiment, it is assumed that the original image shown in FIG. 7 is obtained as a video frame.
In the original image of FIG. 7, the foreground object 51 and the background object 52 are shown. The foreground object 51 overlaps a part of the background object 52. Further, the background object 52 has a reflection region 520. The reflection region 520 represents a state in which sunlight or the like is reflected on the background object 52.
The background object 52 has the shape shown in FIG. 8 except for the overlap with the foreground object 51.

The foreground candidate image generation unit 202 generates the foreground candidate image shown in FIG. 9 from the original image of FIG. 7. In the foreground candidate image shown in FIG. 9, not only the region 501 of the true foreground object but also the reflection region 520 of FIG. 7 is included in the foreground candidate as the reflection region 502.
The region separation unit 203 separates the region 501 of the true foreground object and the reflection region 502 included in the foreground candidate image (FIG. 9).

Returning to FIG. 4, in the overlapping edge image generation process (step ST12), the area separation unit 203 acquires the foreground candidate image from the foreground candidate image generation unit 202, acquires the background edge image from the storage unit 204, and obtains the foreground candidate image. And the background edge image are compared with each other to generate an overlapping edge image. The overlapping edge image is an image showing an edge portion of the background object of the background edge image that overlaps with the foreground candidate.
For example, when the foreground candidate image and the background edge image have the same edge shape and direction, the region separation unit 203 determines that they overlap.

The details of the overlapping edge image generation process (step ST12) will be described with reference to FIG.
Further, in the following, it is assumed that the area separation unit 203 acquires the background edge image illustrated in FIG. 10 from the storage unit 204 with respect to the background object of FIG.
It is assumed that the edge information is extracted from the background image (for example, FIG. 8) in advance, and the background edge image (FIG. 10) is stored in the storage unit 204 at the time of step ST12. Specifically, a general edge detection algorithm such as a Canny filter, a Laplacian filter, or a Sobel filter is used for extracting the edge information.

The region separation unit 203 calculates the luminance gradient by applying a sobel filter in a specific direction to the foreground candidate image (FIG. 9) and the background edge image (FIG. 10) (step ST22), and the luminance gradient is 0 for both. The region other than the above is set as the luminance gradient matching region (step ST23). Further, the region separation unit 203 sets the direction in which the sobel filter is applied in various directions such as 0 ° (horizontal), 90 ° (vertical), and 45 ° until the luminance gradient comparison is performed in all the preset directions. Step ST22 and step ST23 are carried out while changing (step ST21).
After the luminance gradient comparison in all directions is completed (NO in step ST21), the region separation unit 203 integrates the luminance gradient matching regions extracted in each direction to generate a similarity distribution image (step ST24). .. Each pixel value of this similarity distribution image is defined as the respective similarity.
Then, the region separation unit 203 binarizes the similarity distribution image at an arbitrary threshold value.
The region separation unit 203 uses a region where the pixel value of the similarity distribution image is non-zero as an overlapping region, and expands the similarity distribution image to obtain an overlapping edge image (FIG. 11).
As a result of performing the processing of FIG. 5 on the foreground candidate image (FIG. 9) and the background edge image (FIG. 10), the overlapping edge image shown in FIG. 11 is obtained.

Returning to FIG. 4, in the region extraction process (step ST13), the region separation unit 203 separates the reflection region 502 and the true foreground region 501 from the foreground candidate image (FIG. 9).

The details of the region extraction process (step ST13) will be described with reference to FIG.

The region separation unit 203 generates a reflection region image by performing a logical product operation between the foreground candidate image (FIG. 9) and the overlapping edge image (FIG. 11) (step ST31).
For example, the region separation unit 203 performs a logical product operation of the foreground candidate image of FIG. 9 and the overlapping edge image of FIG. 11 to generate the reflection region image of FIG. In the reflection area image of FIG. 12, only the reflection area 502 of the foreground candidate image of FIG. 9 is shown.

Next, the region separation unit 203 subtracts the overlapping edge image (FIG. 11) from the foreground candidate image (FIG. 9) to generate a foreground extraction image (step ST32).
For example, the region separation unit 203 subtracts the overlapping edge image of FIG. 11 from the foreground candidate image of FIG. 9 to generate the foreground extraction image of FIG. In the foreground extraction image of FIG. 13, only the region 501 of the true foreground object of the foreground candidate image of FIG. 9 is shown.

The region separation unit 203 can extract a region in which the pixel value of the foreground extraction image (FIG. 13) is non-zero as a foreground region. Further, the region separation unit 203 can separate the region where the pixel value of the reflection region image (FIG. 12) is non-zero as the reflection region.

Returning to FIG. 3, finally, in the output process (ST04), the output unit 205 outputs the reflection region image (FIG. 12) and the foreground extraction image (FIG. 13) to the outside.

*** Explanation of the effect of the embodiment ***
As described above, according to the present embodiment, the reflection region and the foreground object region can be accurately separated.
That is, according to the present embodiment, in the foreground extraction technique for extracting different regions of the background image and the captured image, even in an environment where there is a change in the background caused by reflection of light by a metal or a smooth surface. It is possible to accurately separate and extract the foreground region, which is an object that does not exist in the background, and the region where the background has changed, such as light reflection.

Further, the image processing device 20 according to the present embodiment can be used in a system for comparing two images and detecting a difference. It is especially effective in an environment where the sunshine conditions change. Specific application methods include, for example, detection of left-behind objects such as leftover objects and left-behind objects in an outdoor environment.

Embodiment 2.
In this embodiment, the difference from the first embodiment will be mainly described.
The matters not described below are the same as those in the first embodiment.
The same reference numerals may be given to the same configurations as those in the first embodiment, and the description thereof may be omitted.

In the first embodiment, the extraction / separation process (ST03) is performed when a foreground object appears near the edge of the background object and the apparent shape of the foreground candidate does not change even if the peripheral edge changes due to the foreground object. In the first form, the determination accuracy of the reflection region may be lowered, for example, the region of the foreground object may be erroneously determined as the reflection region.
In the present embodiment, an example of accurately separating the true foreground object region and the reflection region even when the foreground object appears around the edge of the background object will be described.

*** Explanation of configuration ***
FIG. 14 shows an example of a functional configuration of the image processing apparatus 20 according to the present embodiment.
In FIG. 14, the whole edge image generation unit 206 is added to the configuration shown in FIG.

The whole edge image generation unit 206 performs edge detection processing on the video frame which is the original image, and generates a whole edge image showing the edge of the foreground object and the edge of the background object.
For example, the whole edge image generation unit 206 performs edge detection processing on the original image of FIG. 7 and generates the whole edge image illustrated in FIG. 17.
In the whole edge image of FIG. 17, the edge of the foreground object and the edge of the background object are shown.

In the present embodiment, the region separation unit 203 separates the reflection region and the foreground object region by using the entire edge image in addition to the foreground candidate image and the background edge image.
More specifically, the region separation unit 203 acquires the entire edge image (FIG. 17) from the entire edge image generation unit 206. Further, the area separation unit 203 performs a logical product operation of the foreground candidate image (FIG. 9) and the whole edge image (FIG. 17), and sets the foreground candidate among the foreground object edge and the background object edge of the whole edge image. Generates a first overlapping edge image showing overlapping edge portions. FIG. 18 shows an example of a first overlapping edge image.
Further, the region separation unit 203 performs a luminance gradient comparison between the background edge image (FIG. 10) and the first overlapping edge image (FIG. 18), and the first overlapping edge among the edges of the background object of the background edge image. Generates a second overlapping edge image that shows the edges that overlap the edges of the image. The second overlapping edge image is the same as the overlapping edge image shown in the first embodiment, and is the image shown in FIG. In the present embodiment, the image shown in FIG. 11 is referred to as a second overlapping edge image in order to distinguish it from the first overlapping edge image.
Further, the region separation unit 203 performs a logical product operation of the second overlapping edge image (FIG. 11) and the foreground candidate image (FIG. 9) to extract a reflection region from the foreground candidate image. Further, the region separation unit 203 subtracts the second overlapping edge image (FIG. 11) from the foreground candidate image (FIG. 9) to extract the region of the foreground object.

*** Explanation of operation ***
An operation example of the image processing apparatus 20 according to the present embodiment will be described with reference to FIGS. 15 and 16.
FIG. 15 corresponds to FIG. 4 of the first embodiment. FIG. 16 corresponds to FIG. 5 of the first embodiment.

In FIG. 15, the foreground candidate image generation process (step S11) is the same as that described in the first embodiment, and therefore the description thereof will be omitted.

In the whole edge image generation process (step ST14), the whole edge image generation unit 206 generates the whole edge image from the video frame. The method of extracting the edge of the foreground object and the edge of the background object from the video frame may be the same as the method of extracting the edge of the background object described in the first embodiment, or may be another method.

In the first overlapping edge image generation process (step ST15), the region separation unit 203 performs a logical product operation between the foreground candidate image (FIG. 9) and the entire edge image (FIG. 17), and the first overlapping edge image. (FIG. 18) is generated.

In the second overlapping edge generation process (step ST16), the region separation unit 203 performs a luminance gradient comparison between the background edge image (FIG. 10) and the first overlapping edge image (FIG. 18), and the second overlap occurs. An edge image (FIG. 11) is generated.

In the region extraction process (step ST13), the region separation unit 203 separates the reflection region and the foreground object region from the foreground candidate image, as in the first embodiment.

Next, the details of the second overlapping edge image generation process (step ST16) will be described with reference to FIG.

In this embodiment, step ST22 in FIG. 5 is changed to step ST25.
That is, in step ST22 of FIG. 5, the area separation unit 203 calculated the brightness gradient between the background edge image (FIG. 10) and the foreground candidate image (FIG. 9), but in step ST25 of FIG. 16, the area separation is performed. Part 203 includes a background edge image (FIG. 10) and a first overlapping edge image (FIG. 18) (an image obtained by a logical product calculation of a foreground candidate image (FIG. 9) and an overall edge image (FIG. 17)). Calculate the brightness gradient.
Since step ST21, step ST23 and step ST24 are the same as those shown in FIG. 5, the description thereof will be omitted.

*** Explanation of the effect of the embodiment ***
According to the present embodiment, even if the apparent shape of the foreground candidate does not change, the change in the edge in the foreground candidate appears as a change on the overlapping edge image. It is possible to accurately separate the foreground region and the reflection region existing around the edge.

Embodiment 3.
In this embodiment, the difference from the first embodiment will be mainly described.
The matters not described below are the same as those in the first embodiment.
The same reference numerals may be given to the same configurations as those in the first embodiment, and the description thereof may be omitted.

In the first embodiment, the overlapping edge image generation process (step ST12) is performed on the entire video frame at once, and the periphery of the edge having the same luminance gradient is determined as the reflection region. At this time, the luminance gradient is compared for each pixel to generate an overlapping edge image. If a part of the video frame is missing due to the influence of the shooting environment, the foreground candidate image (FIG. 9) is also partially missing. When a part of the foreground candidate image (FIG. 9) is missing, the region to be determined as the reflection region may be partially erroneously determined as the foreground region.
In the present embodiment, in order to prevent such an erroneous determination, the region separation unit 203 divides the foreground region candidate of the foreground candidate image (FIG. 9) into a plurality of blocks, and compares the luminance gradient for each block. conduct.

*** Explanation of operation ***
An operation example of the image processing apparatus 20 according to the present embodiment will be described with reference to FIGS. 19 and 20.
FIG. 19 corresponds to FIG. 4 of the first embodiment. FIG. 20 corresponds to FIG. 5 of the first embodiment.

Since the foreground image candidate extraction process (step ST11) is the same as that described in the first embodiment, the description thereof will be omitted.

In the foreground candidate division process (step ST17), the area separation unit 203 divides the foreground candidate of the foreground candidate image into a plurality of blocks.
In the foreground candidate image of FIG. 9, the region separation unit 203 is divided into, for example, a region 501 of a true foreground object and a reflection region 502.

Then, in the overlapping edge generation process (step ST12a), the region separation unit 203 performs a luminance gradient comparison with the background edge image (FIG. 10) for each block.

The details of the overlapping edge image generation process (step ST12a) will be described with reference to FIG. 20.

In FIG. 20, steps ST21 to ST24 are the same as those shown in the first embodiment, but the region separation unit 203 performs steps ST21 to ST24 in block units of the foreground candidate image.
Further, in the present embodiment, the region separation unit 203 utilizes the ratio of the number of pixels of the luminance gradient matching edge to the number of pixels of the background edge image in the block of the foreground candidate image. The region separation unit 203 determines that the target block is an overlapping region between the foreground candidate image and the background edge image when the ratio of the number of pixels of the luminance gradient matching edge to the number of pixels of the background edge image is equal to or greater than the threshold value (step). ST26).

*** Explanation of the effect of the embodiment ***
According to the present embodiment, even if the edge of the video frame is partially missing, it is possible to determine whether it is a reflection region or a foreground region by using other edges existing in the same block in the foreground candidate. Therefore, even if the edge detection accuracy is lowered, it is possible to prevent the determination accuracy of the reflection region from being lowered.

In the present embodiment, an example in which the processing procedure of the first embodiment is applied to the blocks obtained by the division has been described, but the processing procedure of the second embodiment may be applied to the blocks obtained by the division. good.

Although the embodiments of the present invention have been described above, two or more of these embodiments may be combined and implemented.
Alternatively, one of these embodiments may be partially implemented.
Alternatively, two or more of these embodiments may be partially combined and carried out.
The present invention is not limited to these embodiments, and various modifications can be made as needed.

*** Explanation of hardware configuration ***
Finally, a supplementary explanation of the hardware configuration of the image processing apparatus 20 will be given.
The processor 6101 shown in FIG. 2 is an IC (Integrated Circuit) that performs processing.
The processor 6101 is a CPU (Central Processing Unit), a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), or the like.
The memory 6102 shown in FIG. 2 is a RAM (Random Access Memory).
The storage device 6104 shown in FIG. 2 is a ROM (Read Only Memory), a flash memory, an HDD (Hard Disk Drive), or the like.
The network interface 6103 shown in FIG. 2 is an electronic circuit that executes data communication processing.
The network interface 6103 is, for example, a communication chip or a NIC (Network Interface Card).

An OS (Operating System) is also stored in the storage device 6104.
Then, at least a part of the OS is executed by the processor 6101.
The processor 6101 executes a program that realizes the functions of the video reception / decoding unit 201, the foreground candidate image generation unit 202, the area separation unit 203, the output unit 205, and the overall edge image generation unit 206 while executing at least a part of the OS. ..
When the processor 6101 executes the OS, task management, memory management, file management, communication control, and the like are performed.
Further, at least one of the information, data, signal value, and variable value indicating the processing result of the video reception decoding unit 201, the foreground candidate image generation unit 202, the area separation unit 203, the output unit 205, and the overall edge image generation unit 206 , Memory 6102, storage device 6104, register in processor 6101 and cache memory.
The programs that realize the functions of the video reception / decoding unit 201, the foreground candidate image generation unit 202, the area separation unit 203, the output unit 205, and the overall edge image generation unit 206 are magnetic disks, flexible disks, optical disks, compact disks, and Blu-rays. (Registered trademark) It may be stored in a portable recording medium such as a disc or a DVD. Then, a portable recording medium containing a program that realizes the functions of the video reception / decoding unit 201, the foreground candidate image generation unit 202, the area separation unit 203, the output unit 205, and the overall edge image generation unit 206 is commercially distributed. You may.

Further, the "section" of the video reception / decoding unit 201, the foreground candidate image generation unit 202, the area separation unit 203, the output unit 205, and the overall edge image generation unit 206 is a "circuit" or "process" or "procedure" or "process". May be read as ".
Further, the image processing device 20 may be realized by a processing circuit. The processing circuit is, for example, a logic IC (Integrated Circuit), a GA (Gate Array), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array).
In this case, the video reception / decoding unit 201, the foreground candidate image generation unit 202, the area separation unit 203, the output unit 205, and the overall edge image generation unit 206 are each realized as a part of the processing circuit.

20 image processor, 51 foreground object, 523 background object, 201 video reception / decoding unit, 202 foreground candidate image generation unit, 203 area separation unit, 204 storage unit, 205 output unit, 206 overall edge image generation unit, 501 foreground object Region, 502 Reflection Region, 520 Reflection Region, 6101 Processor, 6102 Memory, 6103 Network Interface, 6104 Storage Device.

Claims (7)

A foreground candidate image generation unit that generates a foreground candidate image showing a foreground candidate that is a candidate for the area of the foreground object from an original image in which the foreground object as the foreground overlaps with a part of the background object as the background.
A background edge image showing the edge of the background object is acquired, and the foreground candidate image and the background edge image are used to show a state in which light is reflected on the background object included in the foreground candidate image. An image processing apparatus having a region separating portion for separating a reflection region and a region of the foreground object. The area separation part
A luminance gradient comparison between the foreground candidate image and the background edge image is performed to generate an overlapping edge image showing an edge portion of the background object of the background edge image that overlaps with the foreground candidate.
A logical product calculation of the overlapping edge image and the foreground candidate image is performed to extract the reflection region from the foreground candidate image.
The image processing apparatus according to claim 1, wherein the overlapping edge image is subtracted from the foreground candidate image to extract a region of the foreground object. The area separation part
An overall edge image showing the edge of the foreground object and the edge of the background object is acquired, and the logical product calculation of the foreground candidate image and the overall edge image is performed to obtain the edge of the foreground object of the overall edge image. And the edge of the background object, the first overlapping edge image showing the edge portion overlapping with the foreground candidate is generated.
A luminance gradient comparison between the background edge image and the first overlapping edge image is performed to show an edge portion of the background object of the background edge image that overlaps with the edge of the first overlapping edge image. Generates a second overlapping edge image that is
A logical product calculation of the second overlapping edge image and the foreground candidate image is performed to extract the reflection region from the foreground candidate image.
The image processing apparatus according to claim 1, wherein the second overlapping edge image is subtracted from the foreground candidate image to extract a region of the foreground object. The area separation part
The candidate for the foreground object area of the foreground candidate image is divided into a plurality of blocks.
The image processing apparatus according to claim 2, wherein the luminance gradient is compared with the background edge image for each block. The area separation part
The candidate for the foreground object area of the foreground candidate image is divided into a plurality of blocks.
The image processing apparatus according to claim 3, wherein the luminance gradient is compared with the entire edge image for each block. A computer generates a foreground candidate image showing a foreground candidate that is a candidate for the area of the foreground object from an original image in which the foreground object as the foreground overlaps with a part of the background object as the background.
The computer acquires a background edge image showing the edge of the background object, and uses the foreground candidate image and the background edge image to reflect light on the background object included in the foreground candidate image. An image processing method for separating a reflection region showing a state of being present and a region of the foreground object. Foreground candidate image generation processing that generates a foreground candidate image showing a foreground candidate that is a candidate for the area of the foreground object from an original image in which the foreground object that becomes the foreground overlaps a part of the background object that is the background.
A background edge image showing the edge of the background object is acquired, and the foreground candidate image and the background edge image are used to show a state in which light is reflected on the background object included in the foreground candidate image. An image processing program that causes a computer to perform an area separation process for separating a reflected area and an area of the foreground object.

Images