JP2002312787A - Image processor, image processing method, recording medium and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correct an area division result by using the pixels of a neighborhood area to be processed. SOLUTION: A memory 271 preserves a pixel value B' (i, j) of a binary picture indicating a corrected area division result outputted from a subject probability calculating part 203 for a whole picture. A neighborhood area corresponding to coordinates to be corrected can be constituted of pixels within a prescribed distance (within a circle having a prescribed radius) from the coordinates (i, j) to be corrected, or pixels in a square-shaped area decided by a preliminarily set constant m, or pixels in an area set by another method. A background correcting part 272 selects only pixels judged as a subject by a background difference calculating part 202 or the subject probability calculating part 203 in the neighborhood area of the pixels of the coordinates (i, j), and corrects the area division result by using a value calculated by weighting and then averaging the selected pixel value with corresponding subject probability.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method, a recording medium, and a program.
In particular, the present invention relates to an image processing apparatus, an image processing method, a recording medium, and a program that are suitable for extracting only a subject area from an image including a background area and a subject area.

[0002]

2. Description of the Related Art For example, three methods have been conventionally proposed as a region dividing method for dividing an image region, for example, when a subject is extracted from image data composed of a background and a subject.

The first method uses only the pixel values of image data. As a typical method, for example, a method of defining similarity of pixel values and repeating area integration,
There is clustering in the feature space.

A second method is to separate and extract a subject area from a background area by using a pixel value and other additional information together. For example, a rough boundary line of the subject area to be extracted is determined. This is a method in which a subject region is accurately extracted by applying image processing only in the vicinity of the boundary line in advance. When this method is applied to a moving image, the region extracted in the first frame is tracked for changes in the region by using motion detection, inter-frame differences, or Snakes (active contour model). A method has been proposed in which a subject region is extracted in advance in a first frame and a final frame, and interpolation is performed on an image between the subject frames based on the subject region extracted in these two frames.

In the second method, as a method of avoiding the intervention of manual operation such as input of a rough boundary line of a subject region to be extracted, for example, an image of only a background not including a subject is taken in advance. There is a background difference method in which an image including a subject is compared with an image of only the background, and the difference is calculated to extract the subject.

A technique called chroma key, which covers a background area other than a subject with a screen of a fixed color at the time of imaging, and extracts an object using this color information, is currently most frequently used in, for example, video production. It is a technique that is.

The third method is a method of calculating distance information and the like using a special imaging device and extracting a subject based on the information. For example, a parallax between images obtained by a plurality of cameras having different viewpoints is used. Many methods using information have been proposed. Also, a method has been proposed in which the position of the imaging surface of one camera is changed without using a plurality of cameras, and the distance is calculated by using a change in blur occurring on an image at that time.

[0008]

As described above, various methods have been used to cut out and extract a subject from the background.

[0009] However, the first method is excellent as a means for extracting a physically uniform local region (for example, a region having certain color information, a boundary line, and the like). This is not a sufficient method for extracting a subject having a certain area as one area.

In the method using parallax information described as the third method, a plurality of cameras are required (or it takes a long time to capture a plurality of images with one camera). In addition, the amount of calculation for calculating the correspondence between images is very large. When blur information is used, the depth of field achievable in the imaging apparatus becomes a problem. Recent imaging devices, especially imaging devices for consumer use,
It tends to be downsized. For this reason, the depth of field is often deep, and in most cases, it is difficult to detect a difference in blur due to a change in the position of the imaging surface.

In the second method, when a method relying on a manual operation is employed, a flexible interface is required, so that it is very difficult to realize it as an application. Further, when a method of temporally tracking an area is employed, errors such as motion detection are likely to accumulate, which is not suitable for processing a long sequence. Furthermore, since motion detection and Snakes require a large amount of computation,
It is very difficult to introduce into applications that process images in real time.

On the other hand, the background subtraction method is relatively simple in operation, and as long as the background area does not change.
A pixel having a large difference value can be expected to be extracted as a subject region with a very high probability, and there is no possibility of accumulation of a temporal error.

However, even in the background subtraction method, when the difference value is small (for example, when the subject has a color portion close to the background), it cannot always be guaranteed that correct area division has been performed. In such a case,
The subject area is likely to be erroneously determined as a background area. Attempts have been made to improve the extraction accuracy of the subject by statistically processing the difference values, but it cannot be said that the conventional method has always achieved satisfactory region division.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and it is possible to accurately extract a subject area by a simple operation using a simple device while utilizing the simplicity of the background subtraction method. Is to be able to do.

[0015]

According to the present invention, there is provided an image processing apparatus comprising: input means for receiving input of captured image data;
Calculated by an image difference calculating means for calculating a difference between a background image consisting of only the background area input by the input means and a target image consisting of the background area and the subject area input by the input means, and the image difference calculating means. Based on a difference between the background image and the target image, a dividing unit that divides the candidate region of the subject region from the target image, and each pixel of the target image is
Probability calculating means for calculating a first probability included in the subject area, and a first correcting means for correcting the candidate area of the subject area divided by the dividing means based on the first probability calculated by the probability calculating means. By determining from the correction means, the first probability calculated by the probability calculation means, and the correlation between the pixels in the vicinity of the predetermined pixel whether or not the predetermined pixel is a pixel included in the subject, And a second correcting means for further correcting the candidate area of the subject area corrected by the first correcting means.

The probability calculating means may calculate the first probability using Bayes estimation.

[0017] The probability calculating means may calculate the first probability based on a weighting coefficient set for each coordinate.

The probability calculating means calculates the first probability by performing a smoothing process on the distribution of the candidate area of the subject area and the candidate area of the background area based on the result of the division by the dividing means. be able to.

[0019] The smoothing process may be a non-linear smoothing process.

[0020] The second correction means uses the correlation between the pixels in the vicinity of the predetermined pixel only in the area determined as not being the candidate area of the subject area as a result of the correction by the first correction means. Can be corrected.

The second correction means sets a background area to a predetermined pixel based on the first probability calculated by the probability calculation means and a correlation of pixels in the vicinity of the predetermined pixel. The second probability indicating the probability and the third probability indicating the probability of being the subject area can be calculated.
Is smaller than the third probability, a predetermined pixel can be determined to be a pixel included in the subject area.

The second probability can take a larger value as the difference between the pixel value of the corresponding coordinate of the target image and the pixel value of the corresponding coordinate of the background image is smaller.

The third probability is calculated by using a value obtained by weighting and averaging the pixel values of pixels having a high first probability other than the predetermined pixel in the neighboring area. be able to.

When the image data input by the input means includes color information, the image difference calculating means may calculate the difference between the background image and the target image for each component of the color information.

When the image data input by the input means includes color information, the second correction means may calculate the second probability and the third probability for each component of the color information. it can.

According to the image processing method of the present invention, there is provided an input control step of controlling input of captured image data, a background image consisting only of a background area whose input is controlled by the processing of the input control step, and an input control step. Based on the difference between the background image and the target image calculated by the processing of the image difference calculation step, and an image difference calculation step of calculating a difference between the target image including the background area and the subject area whose input is controlled by the processing. A dividing step of dividing a candidate region of a subject region from the target image, and each pixel of the target image
A first calculating step of correcting the candidate area of the subject area divided by the processing of the dividing step based on the probability calculation step of calculating the probability of being included in the subject area, and the probability calculated by the processing of the probability calculating step; Step and the probability calculated by the process of the probability calculation step, and from the correlation of the pixels in the vicinity of the predetermined pixel, by determining whether the predetermined pixel is a pixel included in the subject,
And a second correction step of further correcting the candidate area of the subject area corrected by the processing of the first correction step.

The program recorded on the recording medium of the present invention includes an input control step for controlling the input of captured image data, and a background image consisting only of a background area whose input is controlled by the processing of the input control step. An image difference calculating step of calculating a difference between the background image and the subject image whose input is controlled by the processing of the input control step, and a processing of the background image and the target image calculated by the processing of the image difference calculating step. Based on the difference, from the target image, a dividing step of dividing a candidate region of a subject region, a probability calculating step of calculating a probability that each pixel of the target image is included in the subject region, and a process of a probability calculating step A first method of correcting a candidate area of a subject area divided by the processing of the dividing step based on the calculated probability
Correction step, the probability calculated by the process of the probability calculation step, and from the correlation of the pixels in the vicinity of the predetermined pixel, by determining whether the predetermined pixel is a pixel included in the subject, And a second correction step of further correcting the candidate area of the subject area corrected by the processing of the first correction step.

The program according to the present invention comprises an input control step for controlling the input of captured image data, a background image consisting only of a background area whose input has been controlled by the processing of the input control step, and a processing of the input control step. An image difference calculating step of calculating a difference between the target image including the background area and the subject area whose input is controlled, and a target image based on a difference between the background image and the target image calculated by the processing of the image difference calculating step. From the dividing step of dividing the candidate area of the subject area, the probability calculating step of calculating the probability that each pixel of the target image is included in the subject area, and the probability calculated by the processing of the probability calculating step. The first correction step of correcting the candidate area of the subject area divided by the processing of the division step and the processing of the probability calculation step By determining whether the predetermined pixel is a pixel included in the subject based on the calculated probability and the correlation of the pixels in the vicinity of the predetermined pixel, the correction is performed by the processing in the first correction step. And a second correction step for further correcting the candidate area of the subject area.

In the image processing apparatus, the image processing method, and the program according to the present invention, captured image data is input, and a background image consisting only of the input background area, and a target image consisting of the background area and the subject area, Is calculated, and based on the calculated difference between the background image and the target image, the candidate region of the subject region is divided from the target image, and the probability that each pixel of the target image is included in the subject region is calculated. Based on the calculated and calculated probabilities, the candidate regions of the divided subject regions are corrected, and the predetermined pixels are included in the subject based on the calculated probabilities and the correlation of the pixels in the vicinity of the predetermined pixels. By determining whether or not the pixel is a pixel, the corrected candidate region of the subject region is further corrected.

[0030]

Embodiments of the present invention will be described below with reference to the drawings.

First, a first embodiment of the present invention will be described. FIG. 1 is a block diagram showing a configuration of an image processing apparatus 1 to which the present invention is applied.

The image processing apparatus 1 has two operation modes, a background information extraction mode and a subject extraction mode. The user operates an operation unit (not shown) to operate the background information extraction mode. And the subject extraction mode. In the background information extraction mode, a process of extracting information related to a background region in response to an input of an image including only a background is executed. On the other hand, in the subject extraction mode, a process of extracting only a subject region from an image composed of a subject and a background is executed based on the background information extracted in the background information extraction mode.

The imaging unit 11 is, for example, a CCD (Charge Couple).
d Devices) It consists of a camera. Imaging unit 1
1 captures an image under the control of the control unit 13 and converts the image data including the pixel values I (i, j) into the subject area detection unit 1
Output to 2. Here, the pixel value I (i, j) represents a pixel value on the two-dimensional coordinates (i, j) of the captured image data.

The subject area detector 12 includes a switch 21
Background information extraction unit 22, memory 23, subject candidate detection unit 2
4 and a subject determination unit 25, according to the processing of the control unit 13, processes the image data input from the imaging unit 11 and executes a process of detecting a subject area.

In the background information extraction mode, the switch 21 converts the image data of the background image composed of only the background into the background information extraction unit 22 based on the signal input from the control unit 13.
And in the case of the subject extraction mode, the image data including the subject is subjected to subject candidate detection section 24 and subject determination section 25.
To supply.

The control unit 13 controls the operation of the image processing apparatus 1. For example, based on a signal input from an operation unit (not shown), the control unit 13 determines whether the mode is the background information extraction mode or the subject extraction mode, and The switch 21 is switched so that the image data captured at 11 is supplied to a suitable portion of the subject area detection unit 12. Further, a drive 14 is also connected to the control unit 13. A magnetic disk 31, an optical disk 32, a magneto-optical disk 33, and a semiconductor memory 34 are mounted on the drive 14 as necessary, so that data is transferred.

FIG. 2 is a block diagram showing a more detailed configuration of the background information extracting unit 22.

In the background extraction mode, the background image data input via the switch 21 is supplied to the maximum value image generation unit 41, the minimum value image generation unit 42, and the average value image generation unit 43. The maximum value image generation unit 41 calculates the maximum value of the pixel for each coordinate (i, j) of the plurality of input images using Expression (1), and calculates the maximum pixel value Max (i, j) at each coordinate. ) Is generated as background information and output to the memory 23 for storage.

(Equation 1) ... (1)

Here, Ik (i, j) indicates the pixel value at the coordinates (i, j) of the k-th image. Also, MA
X (•) indicates an operation for calculating the maximum value of a plurality of numerical values shown in parentheses.

The minimum value image generation unit 42 calculates the minimum value of the pixel for each coordinate (i, j) of the plurality of input images by the following equation (2).
And the minimum pixel value M at each coordinate
A minimum value image composed of in (i, j) is generated as background information, output to the memory 23 and stored. The average value image generation unit 43 calculates the average value of the pixels at each coordinate (i, j) of the plurality of input images using Expression (3), and calculates the average pixel value Ave (i, j) at each coordinate. ) Is generated as background information and output to the memory 23 for storage.

(Equation 2) ... (2)

(Equation 3) ... (3)

Here, MIN (•) indicates an operation for calculating the minimum value of a plurality of numerical values shown in parentheses. Also, N
Indicates the number of images supplied to the background information extraction unit 22.

The memory 23 receives the input of the background information extracted by the background information extracting unit 22, stores the background information, and supplies the stored background information to the subject candidate detecting unit 24 and the subject determining unit 25. .

FIG. 3 is a block diagram showing a more detailed configuration of the subject candidate detecting section 24. As shown in FIG.

The binarization processing section 52 of the subject candidate detection section 24
Is the pixel value I of the input image data via the switch 21.
(I, j) is supplied. The threshold calculator 51 calculates necessary information (here, the pixel value Max (i, j) of the maximum value image and the pixel value Min (i, j) of the minimum value image) from the background information recorded in the memory 23. Upper threshold value Tm of the pixel value for reading and determining whether it is a background area or a subject
ax and the lower limit threshold value Tmin of the pixel value are calculated.

The upper threshold value Tmax of the pixel value and the lower threshold value T of the pixel value for determining whether it is a background area or a subject.
The method of calculating min may be any method, but in this example, the upper limit threshold value Tmax of the pixel value and the lower limit threshold value Tmin of the pixel value are calculated using Expressions (4) and (5). Then, the calculation result is output to the binarization processing unit 52.

(Equation 4) ... (4)

(Equation 5) ... (5)

The coefficient a in equation (4) and the coefficient b in equation (5)
Are preset coefficients, for example, in order to absorb fluctuations in pixel values due to fluctuations in illumination light, etc.
Usually, when a> 1.0 and b <1.0, it is determined experimentally or empirically.

The binarization processing section 52 receives the threshold value calculated by the threshold value calculation section 51 and stores it in an internal memory (not shown). The binarization processing unit 52 receives the input of the pixel value I (i, j) via the switch 21 and uses the threshold value stored in the internal memory based on the following equation (6).
A binary image including the binarized pixel values B (i, j) is generated and output to the subject determination unit 25.

(Equation 6) ... (6)

That is, the binarization processing unit 52 sets the pixel corresponding to the pixel value within the threshold value range of the input pixel value I (i, j) to 0 as a value indicating that it is a background candidate. And a pixel value B (i, j) of a binary image in which 1 is set as a value indicating that a pixel value outside the threshold value range is a subject candidate.

In FIG. 3, the threshold value calculating section 51 calculates the pixel value Max (i, j) of the maximum value image and the pixel value Min of the minimum value image by using the equations (4) and (5).
By integrating a predetermined coefficient into (i, j), for example, the fluctuation of the pixel value due to the fluctuation of the illumination light is absorbed. However, the threshold value calculation unit 51 is omitted, and the pixel of the maximum value image is omitted. By directly inputting the value Max (i, j) and the pixel value Min (i, j) of the minimum value image to the binarization processing unit 52, the fluctuation cannot be absorbed, but the amount of calculation is reduced instead. You may do it.

In the subject extraction mode, the pixel value I (i, j) is input from the switch 21 to the subject determination unit 25, and the pixel value B of the binary image is supplied from the subject candidate detection unit 24.
(I, j) is input, and the pixel value Max (i, j) of the maximum value image, the pixel value Min (i, j) of the minimum value image, and the pixel value Ave of the average value image are input from the memory 23.
(I, j) is read. FIG. 4 is a block diagram showing a more detailed configuration of the subject determining unit 25.

The background condition probability detecting section 61 reads the background information (here, the pixel value Max of the maximum value image) from the memory 23.
(I, j), the pixel value Min (i, j) of the minimum value image, and the pixel value Ave (i, j) of the average value image are read, and the Gaussian distribution is applied to the background using Expression (7). Probability P that pixel value I (i, j) appears under the condition that there is
(I (i, j) | bg) and calculates the Bayesian estimation
6 is output.

(Equation 7) ... (7)

Here, the probability P (A | B) is the probability that the event A will occur under the condition that the event is the event B.

The coefficient s in the equation (7) is the standard deviation of the Gaussian distribution, and in this example, the pixel value Ma of the maximum value image
x (i, j), the pixel value Min (i, j) of the minimum value image,
And the pixel value Ave (i, j) of the average image,
It is calculated by the following equation (8).

(Equation 8) ... (8)

That is, equation (7) indicates that, in the background information extraction mode, the closer the pixel value is to the pixel value that appeared as the background, the higher the probability of being a background area.

The histogram generation section 62 calculates the pixel value I (i, j) of the input image data and the pixel value B of the binary image.
(I, j), the pixel value B (i, j) of the binary image is received.
j) is 1 (that is, the subject candidate detection unit 24
, The histogram H (I) corresponding to the pixel value I (i, j) corresponding to the coordinates of the pixel considered as the subject candidate
(I, j)) is generated and output to the subject condition probability detection unit 63.

The subject condition probability detection unit 63 calculates the pixel value I (i, j) of the input image data and the histogram H (I
(I, j)), and the probability P (I (i, j) that the pixel value I (i, j) appears under the condition that the subject is a subject.
j) | obj) is calculated by equation (9). P (I (i, j) | obj) = H (I (i, j)) / obj_count (9)

Here, obj_count is the total number of pixels of the subject candidate (that is, the total number of pixels having a pixel value of 1 in the pixel values B (i, j) of the binary image).

The subject candidate detector 24 obtains a subject candidate obtained by the background subtraction method (ie, a binary image composed of pixel values B (i, j) having different values for the subject candidate and the background candidate). In this subject candidate, a pixel having a small difference from the pixel value information extracted in the background information extraction mode does not necessarily belong to the background region, but a pixel having a large difference belongs to the subject region with a considerably high probability. It is extracted using the property that it can be said.
The definition of the probability P (I (i, j) | obj) in equation (9) is based on this property, and the appearance frequency of the pixels corresponding to the subject candidate detected by the subject candidate detection unit 24 is A higher pixel value (that is, a pixel value having a higher numerical value in the histogram) indicates a higher probability of appearing in the subject area.

The multi-value mask generating section 64 calculates, for example, the equation (1) for the pixel value B (i, j) of the input binary image.
0) is performed to generate a multi-valued image having a narrow band composed of the smoothed pixel values M (i, j), and output it to the subject / background probability detection unit 65. M (i, j) = LPF (B (i, j) × C) (10)

Here, in equation (10), the pixel value B
The coefficient C multiplied by (i, j) is a preset constant and determines the accuracy of the probability value in the calculation performed by the subject / background probability detection unit 65 described later. As the LPF, for example, an average filter can be used.

The subject / background probability detection unit 65 calculates the pixel value M (i, i,
j), a probability P (obj) of a subject represented by the following equation (11) and a probability P (bg) of a background represented by the following equation (12) are calculated and output to the Bayes estimation calculation unit 66. I do. P (obj) = M (i, j) / C (11) P (bg) = 1.0-P (obj) (12)

Here, the coefficient C in the equation (11) is calculated by the equation (1)
0) is the same value as the coefficient C. Multi-value mask generator 64
The pixel value M (i, j) of the multi-valued image generated in step (1) is larger as the pixel value (the pixel set to pixel value 1) spatially closer to the subject candidate in the subject candidate detection unit 24 (pixel set to pixel value 1). (Pixel value close to 1). That is, Expressions (11) and (12) define the probability of being a subject and the probability of being a background based on the spatial distribution of subject candidates extracted by the background difference method. The closer it is, the higher the probability of being a subject.

The Bayesian estimation calculating unit 66 calculates the probability P (obj | I (i, j)) that the pixel value I (i, j) belongs to the subject area according to Bayes' theorem based on the input information. I do.

When there are two events A and B, the event A
The probability that the event B will occur under the condition of is represented by the following equation (13). This is called Bayes' theorem.

(Equation 9) ... (13)

That is, the Bayesian estimation calculation unit 66 calculates the probability P (I, j) that the pixel value I (i, j) appears under the condition of being the background, which is input from the background condition probability detection unit 61.
(I, j) | bg), the pixel value I (i, i, j) based on the condition that the subject is input from the subject condition probability detection unit 63.
j) appears, and the probability P (obj) of the subject and the probability P (b) of the background input from the subject / background probability detection unit 65
g) and the pixel value I (i, i,
j) belongs to the subject area P (obj | I (i,
j)) is calculated.

(Equation 10) ... (14)

The Bayesian estimator 66 calculates the equation (1)
If the probability P (obj | I (i, j)) that the pixel value I (i, j) obtained by 4) belongs to the subject area is larger than a predetermined threshold T, the corresponding pixel (i , J) is determined to be a subject area, and if smaller than the threshold T, it is determined to be a background area, and is made up of binarized pixel values O ′ (i, j) according to the following equation (15). A binary image is generated and output to the logical sum operation unit 67.

[Equation 11] ... (15)

The logical sum operation unit 67 includes the subject candidate detection unit 2
In order to reflect the correction based on the Bayesian estimation only for the pixels that are considered as background candidates in FIG.
4, the pixel value B (i, j) of the binary image input from the Bayesian estimator 66 and the pixel value O of the binary image input from the Bayes estimation calculation unit 66.
A logical OR of coordinates' (i, j) is calculated, and a binary image composed of the pixel values O (i, j) obtained as a result is output.

The pixel value B (i, j) of the binary image input from the subject candidate detection unit 24 and the pixel value O ′ (i, j) of the binary image input from the Bayesian estimation calculation unit 66 (That is, 1 when it is determined to be a background, and 0 when it is determined to be a subject), and replaces the logical sum operation unit 67 with each coordinate of the input binary image. An AND operation unit that performs an AND operation may be provided.

In the image processing apparatus 1 described with reference to FIG. 1, the pixel value Max (i,
j), the pixel value Min (i, j) of the minimum value image, and the pixel value Ave (i, j) of the average value image, and the threshold value calculation unit 51 of the subject candidate detection unit 24 stores the pixel value of the maximum value image. Max (i, j) and the pixel value Min of the minimum value image
(I, j) is read, and a threshold value Tmax of the upper limit of the pixel value and a threshold value Tmin of the lower limit of the pixel value for determining whether the pixel is the background area or the subject are calculated using Expressions (4) and (5). However, in the background information extraction mode, the background information extraction unit 22 is caused to execute the calculations of Expressions (4) and (5) in advance, and the memory 23 stores the threshold value Tmax of the pixel value upper limit and the pixel value The lower threshold Tmin may be stored. In that case, the threshold value calculation unit 51 can be omitted, and the calculation time in the subject extraction mode can be reduced.

The standard deviation s described using the equation (8) may be calculated in advance by the background information extraction unit 22 in the background information extraction mode, and may be stored in the memory 23.

The probability P (I (i, j) | bg) that the pixel value I (i, j) appears under the condition of the background is calculated by the subject determining unit 25 by the equation (8). Although it is approximated by a Gaussian distribution of the standard deviation s, it is needless to say that another formula may be used for the calculation of the standard deviation, or another distribution function according to the property of the background image may be used.

The probability P (obj) of the subject is
In addition to equation (11), for example, a non-linear function as shown in the following equation (16) may be used (weight is applied to the subject area or the background area).

(Equation 12) (16) Here, g is a predetermined constant suitable for performing weighting.

Next, a second embodiment of the present invention will be described. FIG. 5 shows an image processing apparatus 71 to which the present invention is applied.
FIG. 3 is a block diagram showing the configuration of FIG. The parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

That is, the image processing apparatus 71 shown in FIG.
1 is basically the same as that of the image processing apparatus 1 of FIG.
Replaces the background information extraction unit 22 with the background information extraction unit 9
1 except that the object area detection unit 1 shown in FIG.
2 has basically the same configuration.

FIG. 6 is a block diagram showing a more detailed configuration of the background information extracting unit 91.

Since the average value image generation unit 43 has been described with reference to FIG. 2, the description is omitted here.

The deviation image generation unit 101 calculates a standard deviation from the pixel values I (i, j) of the plurality of input image data for each coordinate by using the following equation (17). A deviation image whose value is a pixel value Sdv1 (i, j) is generated as background information and output to the memory 23.

(Equation 13) ... (17)

In order to reduce the amount of calculation in the deviation image generation unit 101, the square root may be omitted in the calculation of the equation (17), and the variance value may be stored in the memory 23.

The subject candidate detecting section 24 reads the background information (here, the pixel value Ave of the average image) from the memory 23.
(I, j) and the pixel value Sdv1 (i,
j)) is read. The threshold calculation unit 51 calculates the upper threshold Tmax of the pixel value and the lower threshold Tmin of the pixel value using Expressions (18) and (19), and outputs the calculation result to the binarization processing unit 52. . The binarization processing unit 52 performs the pixel value B of the binary image in the same manner as in the first embodiment.
(I, j) is calculated and output.

[Equation 14] ... (18)

(Equation 15) (19) Here, the coefficient c and the coefficient c ′ are preset constants.

Then, in the background condition probability detecting section 61 of the subject judging section 25, the probability P (I, j) that the pixel value I (i, j) appears under the condition of the background using the above equation (7). (I, j) | bg) is calculated. In this case, the standard deviation s of the Gaussian distribution is calculated using the pixel value Sd of the deviation image stored in the memory 23 without using the equation (8).
Each pixel value of v1 (I, j) can be used.

In consideration of the illumination light and the variation of the imaging parameters of the imaging unit 11, for example, the pixel value S of the deviation image
A preset coefficient may be integrated with each pixel value of dv1 (I, j), and the integration result may be used as the standard deviation of the Gaussian distribution.

Then, similarly to the first embodiment,
A pixel value O (i, j) of a binary image having a pixel value of 0 is generated when the subject is determined to be a background by the subject determination unit 25, and is output when the subject is determined to be a subject. .

According to the second embodiment, the information stored in the memory 23 is the pixel value Ave of the average image.
(I, j) and the pixel value Sdv1 (i, j) of the deviation image
Therefore, the capacity of the memory 23 can be reduced.

Next, a third embodiment of the present invention will be described. FIG. 7 shows an image processing apparatus 11 to which the present invention is applied.
1 is a block diagram showing a configuration of FIG. The parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

That is, the image processing device 111 of FIG.
Instead of the subject area detection unit 12, the subject area detection unit 1
Except that the image processing apparatus 21 is provided, it has basically the same configuration as the image processing apparatus 1 described with reference to FIG.
The object region detection unit 121 is basically the same as the object region detection unit 12 described with reference to FIG. 1 except that a correction coefficient calculation unit 131, a memory 132, and a pixel value correction unit 133 are newly provided. It has a configuration.

In the subject extraction mode, the image data picked up by the image pick-up unit 11 is
Before being input to the subject determination unit 25, the correction coefficient calculation unit 131, the memory 132, and the pixel value correction unit 133
In, the correction of the illumination light and the fluctuation component of the imaging parameter of the imaging unit 11 is performed. The pixel value I (i, j) of the image data imaged by the imaging unit 11 is output to the correction coefficient calculation unit 131 and the memory 132 via the switch 21.

The correction coefficient calculating section 131 calculates the correction coefficient from the input pixel value I (i, j) and the pixel value Ave (i, j) of the average image read from the memory 23 by using the equation (20). p is calculated and output to the pixel value correction unit 133.

(Equation 16) (20) where dif in equation (20) is the input pixel value I
This is the absolute value of the difference between (i, j) and the pixel value Ave (i, j) of the average image, and is represented by the following equation (21).

[Equation 17] ... (21)

Further, L in equation (20) indicates a portion other than the subject region. However, since this region cannot be known in advance, for example, the four corners of the image shown in FIG. In this way, an area in which the subject is unlikely to enter is set in advance.

In the equation (20), T is a threshold value, and the equation (2)
If the absolute value of the difference between the input pixel value I (i, j) shown in 1) and the pixel value Ave (i, j) of the average image exceeds this threshold, it is included in the region L. Coordinates (i,
In j), the subject is regarded as being imaged, and is excluded from the calculation of the correction coefficient.

Vmin and Vmax are constants set in advance, and the input pixel value I (i, j) and
The pixel value Ave (i, j) of the average image is V
Only when the value is not less than min and not more than Vmax, the value is used for calculating the correction coefficient. This is because if the pixel value is very large, it is likely that the pixel is saturated, and if the pixel value is very small, the pixel is strongly affected by noise.
This is because it is not preferable to use any of them for calculating the correction coefficient.

The memory 132 stores the pixel value I (i, j) input via the switch 21 into the correction coefficient calculating unit 131.
Is temporarily stored in accordance with the execution time of the correction coefficient calculation process.

The pixel value correction section 133 is provided with the correction coefficient calculation section 1
31 receives the input of the correction coefficient p calculated,
Reads the pixel value I (i, j) of the input image data from
The correction of each pixel value is executed by the following equation (22). I ′ (i, j) = I (i, j) / p (22)

The corrected image including the corrected pixel values I ′ (i, j) is obtained by the subject candidate detecting section 24 and the subject determining section 2.
5, and the subject area is detected in the same manner as described with reference to FIG. 1, and a binary image including the pixel values O (i, j) is output.

The image processing apparatus 111 described with reference to FIG.
In the formula, for example, the correction coefficient p is calculated by using the calculation formula obtained by exchanging the denominator and the numerator in the calculation formula of the correction coefficient p described by using Expression (20), and the pixel value correction unit 133 outputs The correction coefficient may be added to the pixel value I (i, j) of the read input image data.

In the image processing apparatus 111 of FIG. 7, as in the first embodiment, background information is extracted by using the background information extraction unit 22, and the subject area is detected by using the background information. However, a background information extraction unit 91 is provided in place of the background information extraction unit 22, and the background information is extracted using the background information 91 as in the second embodiment. Alternatively, the subject area may be detected using the background information.

Next, a fourth embodiment of the present invention will be described. FIG. 9 shows an image processing apparatus 14 to which the present invention is applied.
1 is a block diagram showing a configuration of FIG. Image processing device 141
Is particularly suitable for extracting a subject region from a moving image. The parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

That is, the image processing apparatus 141 shown in FIG.
Instead of the subject area detection unit 12, the subject area detection unit 1
Except that the image processing apparatus 51 is provided, the image processing apparatus 1 has basically the same configuration as the image processing apparatus 1 described with reference to FIG.
The object region detection unit 151 is the same as that shown in FIG. 1 except that an object determination unit 161 is provided instead of the object determination unit 25.
Has a configuration basically similar to that of the subject region detection unit 12 described with reference to FIG.

FIG. 10 is a block diagram showing a more detailed configuration of the subject determining unit 161. Parts corresponding to those in FIG. 4 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

First, the pixel value I (i, j) and background information (here, the pixel value Max (i, j) of the maximum value image, the pixel value Min of the minimum value image corresponding to the first image data)
(I, j) and the pixel value Ave (i,
j)) is input to the subject determination unit 161.

For only the first image data, the pixel value B of the binary image output from the subject candidate detecting section 24
(I, j) is directly supplied to the histogram generation unit 62 and the multi-value mask generation unit 64 via the logical sum operation unit 67 and the memory 172. The pixel value I (i, j) of the input image data supplied to the memory 171 is immediately input to the histogram generation unit 62, but the pixel value I (i, j) of the first input image data is obtained. Is still in memory 171
Shall be stored in

The histogram generation section 62 has a memory 171
Pixel value B (i, j) inputted through the memory 1
Using the pixel value I (i, j) input via
In the same manner as the method described with reference to FIG.
(I (i, j)) is generated and output to the subject condition probability detection unit 63 via the memory 173.

The background condition probability detection section 61, the subject condition probability detection section 63, the multi-value mask generation section 64, the subject / background probability detection section 65, the Bayes estimation calculation section 66, and the logical sum calculation section 67 are shown in FIG. A process similar to the process described above is executed, and the pixel value O (i,
Generate a binary image consisting of j). Generated pixel value O
The binary image consisting of (i, j) is output and
The data is supplied to the memory 172.

The histogram generation section 62 has a memory 171
From the pixel value I (i, i, i) corresponding to the first image data.
j), the pixel value O (i, j) of the binary image corresponding to the first image data is read from the memory 172, and the pixel value O (i, j) of the binary image becomes 1. The histogram H (I) related to the coordinates, that is, the pixel value of the coordinates that are regarded as subject candidates in the first image data
(I, j)) is generated and output to the memory 173.

Next, the pixel value I (i, j) corresponding to the second image data is input to the subject determining unit 161.

The memory 171 receives the pixel value I (i, j) corresponding to the second image data and saves (overwrites the pixel value I (i, j) corresponding to the first image data) )
Is done.

The background condition probability detection section 61 executes a process similar to the process described with reference to FIG. 4, and performs pixel value I based on the condition that the second image data is a background.
The probability P (I (i, j) | bg) at which (i, j) appears is calculated and output to the Bayes estimation calculation unit 66.

The subject condition probability detection unit 63 receives the input of the pixel value I (i, j) corresponding to the second image data, and is regarded as a subject candidate from the memory 173 in the first image data. The histogram H (I (i, j)) corresponding to the coordinates thus obtained is read, and using equation (9),
A probability P (I (i, j) | obj) at which the pixel value I (i, j) appears under the condition that the subject is a subject is calculated and output to the Bayes estimation calculation unit 66.

The multi-value mask generation unit 64 reads from the memory 172 the pixel value O (i, j) of the binary image indicating the coordinates regarded as the subject candidate in the first image data, and for example, formula ( By performing a smoothing process as shown in 10), a multi-valued image having a narrow band composed of the smoothed pixel values M (i, j) is generated and output to the subject / background probability detection unit 65.

The subject / background probability detecting section 65 executes the same processing as the processing described with reference to FIG. 4 (the arithmetic processing described using the equations (11) and (12)), and the corresponding pixel is determined. The probability P (obj) of the subject and the probability P (bg) of the background are calculated and output to the Bayesian estimation calculation unit 66.

The Bayes estimation calculation unit 66 and the logical sum calculation unit 67 execute the same processing as the processing described with reference to FIG. 4, and the pixel values O (i,
Generate a binary image consisting of j). Generated pixel value O
The binary image consisting of (i, j) is output and
The data is supplied (overwritten) to the memory 172.

The histogram generation section 62 has a memory 171
From the pixel value I (i, i,
j), the pixel value O (i, j) of the binary image corresponding to the second image data is read from the memory 172, and the pixel value O (i, j) of the binary image becomes 1. The histogram H (I) related to the coordinates, that is, the pixel values of the coordinates of the coordinates regarded as the subject candidates in the second image data
(I, j)) is generated and output to the memory 173.

The pixel value I (i, j) and background information corresponding to the third and subsequent image data are stored in the
61, and the same processing is repeated, so that the subject region in the moving image data composed of a plurality of continuous image data is determined based on the information of the region estimated to be the subject region in the immediately preceding image data. In addition, detection can be performed with high accuracy.

Further, in FIG. 10, the second image data supplied from the subject candidate detecting section 24 for the first image data is shown.
The case where the pixel value B (i, j) of the value image is directly input to the memory 172 to generate information necessary for Bayesian estimation for the first image data has been described. The display may not be performed, and the first image data may be used only for generating information necessary for Bayesian estimation with respect to the second image data.

In FIG. 10, the case where the histogram and the multi-value mask for the immediately preceding input image are used has been described, but either the histogram generator 62 or the multi-value mask generator 64 may be used. , The pixel value B of the binary image supplied from the subject candidate detection unit 24
(I, j) may be directly supplied to generate a histogram or a multi-value mask for the image data that is currently being processed, and use it for Bayesian inference.

In the image processing apparatus 141 of FIG. 9, as in the first embodiment, the background information is extracted by using the background information extracting unit 22, and the correction coefficient is calculated by using the background information. Although the description has been made assuming that the subject area is detected without performing the above, a background information extraction unit 91 is provided in place of the background information extraction unit 22, and the background information extraction unit 91 is provided as in the second embodiment. The background information may be extracted using the information 91, and the subject area may be detected using the background information. Alternatively, the correction coefficient calculation unit described with reference to FIG. 7 may be used as in the third embodiment. 131, a memory 132, and a pixel value correction unit 133 may be provided to calculate a correction coefficient.

In the first to fourth embodiments described above, the case where the pixel value is a single value has been described. Next, as a fifth embodiment, the image processing apparatus 1 shown in FIG. A case where a color image is processed will be described.

The pixels I (i, j) of the image data input from the imaging unit 11 include, for example, a luminance signal Y (i, j),
A plurality of components representing colors, such as a color difference signal U (i, j) and a color difference signal V (i, j), are included.

A luminance signal Y (i, j) and a color difference signal U (i, j)
j) and the color difference signal V (i, j) are output from the background information extraction unit 2 via the switch 21 in the background information extraction mode.
Second, in the subject extraction mode, it is supplied to the subject candidate detection unit 24 and the subject determination unit 25.

In the background extraction mode, the background image data input to the background information extraction unit 22 described with reference to FIG. 2 through the switch 21 is converted into a maximum value image generation unit 41, a minimum value image generation unit 42, The average value image generation unit 43 supplies the image data. Maximum value image generation unit 41, minimum value image generation unit 4
2, and the average value image generation unit 43 calculates the pixel value of each of the components of the luminance signal Y (i, j), the color difference signal U (i, j), and the color difference signal V (i, j) of the image data. A maximum value image including a maximum value, a minimum value image including a minimum pixel value, and an average value image including an average pixel value are generated and output to the memory 23.

The maximum value image generation unit 41 calculates the following equation (23).
From Equation (25), the maximum pixel value MaxY (i, j) for each component of the luminance signal Y (i, j), the color difference signal U (i, j), and the color difference signal V (i, j). ), MaxU
A maximum value image including (i, j) and MaxV (i, j) is generated.

(Equation 18) ... (23)

[Equation 19] ... (24)

(Equation 20) ... (25)

Here, Yk (i, j) represents the value of the luminance signal at the coordinates (i, j) of the k-th image by Uk
(I, j) and Vk (i, j) are the k-th image,
The value of the color difference signal of each element at the coordinates (i, j) is shown. MAX (•) indicates an operation for calculating the maximum value of a plurality of numerical values shown in parentheses.

The minimum value image generation section 42 calculates the following equation (26).
From Expression (28), the minimum pixel value MinY (i, j) for each component of the luminance signal Y (i, j), the color difference signal U (i, j) and the color difference signal V (i, j). ), MinU
A minimum value image composed of (i, j) and MinV (i, j) is generated.

(Equation 21) ... (26)

(Equation 22) ... (27)

(Equation 23) ... (28)

Here, MIN (•) indicates an operation for calculating the minimum value of a plurality of numerical values shown in parentheses.

The average value image generation section 43 calculates the following equation (29).
From Equation (31), the average value AveY (i, j) of the pixel values is used for each of the components of the luminance signal Y (i, j), the color difference signal U (i, j), and the color difference signal V (i, j). ), AveU
An average image composed of (i, j) and AveV (i, j) is generated.

(Equation 24) ... (29)

(Equation 25) ... (30)

(Equation 26) (31) Here, N indicates the number of images supplied to the background information extraction unit 22.

The subject candidate detector 2 described with reference to FIG.
4, in the subject extraction mode, the pixel value I (i, j) of the input image data is supplied via the switch 21, and the necessary information (here, the maximum value The pixel value Max (i,
j) and the pixel value Min (i, j) of the minimum value image are read.

The threshold value calculating section 51 calculates a threshold value used for determining whether the object is a background area or a subject. The threshold calculator 51 calculates a luminance signal Y (i, j) and a color difference signal U (i, j).
For each component of the color difference signal V (i, j), the threshold value Tmax of the upper limit of the pixel value is calculated by using Expressions (32) to (34), and by using Expressions (35) to (37), The threshold value Tmin of the lower limit of the pixel value is calculated and output to the binarization processing unit 52.

[0127]

[Equation 27] ... (32)

[Equation 28] ... (33)

(Equation 29) ... (34)

[Equation 30] ... (35)

(Equation 31) ... (36)

(Equation 32) ... (37)

Here, ay, au, av, by, bu,
And bv are coefficients set in advance, respectively, in order to absorb fluctuations in pixel values due to illumination light or the like,
Usually, ay, au, av> 1.0, by, b
u, bv <1.0 are set.

The binarization processing section 52 receives the input of the pixel value I (i, j) of the input image data via the switch 21,
Threshold Tm represented by equations (32) to (37)
Using the ax and the threshold Tmin, a binary image composed of the pixel values B (i, j) is generated by the following equation (38), and is output to the subject determination unit 25.

[Equation 33] ... (38)

That is, the binarization processing section 52 outputs the luminance signal Y (i, j) of the pixel value I (i, j) of the input image data.
j), the color difference signal U (i, j) and the color difference signal V (i,
If all the components of j) are within the range between the threshold value Tmax and the threshold value Tmin, the corresponding pixel is set to 0 as a value indicating that it is a background candidate, and the luminance signal Y (i, j) and the color difference signal If any component of U (i, j) and the color difference signal V (i, j) is out of the range of the threshold, the corresponding pixel is set to 1 as a value indicating that it is a subject candidate. Generate an image pixel value B (i, j).

The background condition probability detection section 61 of the subject determination section 25 described with reference to FIG.
While receiving the input of (i, j), the background information represented by the equations (23) to (31) is read from the memory 23, and the combination of the color components I (i , J) = (Y (i, j), U (i,
j), V (i, j)), which is the probability of occurrence of P (I,
(I, j) | bg) is calculated using the following equation (39), and is output to the Bayesian inference operation unit.

(Equation 34) ... (39)

Here, P (Y (i, j) | bg), P
(U (i, j) | bg) and P (V (i, j) | b
g) is the probability that each component appears under the condition that it is a background. The background information represented by Expressions (23) to (31) is read from the memory 23, and Expression (7) is obtained.
And by substituting for the pixel value I (i, j) in equation (8).

The histogram generation unit 62 calculates the pixel value I (i, j) of the input image data and the pixel value B of the binary image.
(I, j), the pixel value B (i, j) of the binary image is received.
j) is 1 (ie, three-dimensional histogram H (Y (i, j), U (i,
j), V (i, j)) are generated and output to the subject condition probability detection unit 63.

The subject condition probability detecting section 63 outputs the three-dimensional histogram H (Y
(I, j), U (i, j), V (i, j))
Under the condition of being a subject, a combination of color components I (i, j) = (Y (i, j), U (i, j),
V (i, j)), which is the probability of occurrence of P (I, (i,
j) | obj) is calculated using the following equation (40), and is output to the Bayesian inference operation unit.

(Equation 35) (40) where obj_count is the total number of pixels of the subject candidate (that is, the pixel value B of the binary image) as in the case of Expression (9).
(I, j), the total number of pixels having a pixel value of 1).

Then, the multi-value mask generator 64 sets the subject /
In the background probability detection section 65, the Bayes estimation calculation section 66, and the logical sum calculation section 67, the same processing as the processing described in the first embodiment is executed, and the coordinates determined to be the background are set to 0. Since a binary image O (i, j) having a value of 1 is generated and output by effectively utilizing the color information of the input color image at the coordinates determined to be the subject, It is possible to extract a more accurate subject region.

Here, the case where a color image is processed by the image processing apparatus 1 of FIG. 1 has been described.
Also in the image processing apparatus described with reference to FIG.
Of course, a color image can be processed in a similar manner.

For example, when a color image is processed by the image processing apparatus 71 described with reference to FIG. 5, the deviation image generation unit 101 of the background information extraction unit 91 calculates the deviation image generated using the equation (15). The pixel value Sdv1 (i, j) is
All the components of the luminance signal Y (i, j), the color difference signal U (i, j) and the color difference signal V (i, j) are generated, and the threshold value calculation unit 51 sets the upper limit of the pixel value for each color component. The threshold value Tmax and the lower limit threshold value Tmin of the pixel value may be calculated. At this time, the upper threshold Tm
The constant multiplied to calculate ax and the lower limit threshold value Tmin of the pixel value may be different for each color component.

When a color image is processed in the image processing device 71 described with reference to FIG. 7, the correction coefficient calculating section 131 outputs a luminance signal Y (i, j) and a color difference signal U (i, j).
j) and all the components of the color difference signal V (i, j) are calculated, and the pixel value correction unit 133 is calculated.
Then, using the correction coefficient of each component, the luminance signal Y
(I, j), color difference signal U (i, j) and color difference signal V
What is necessary is just to correct all the components of (i, j).

In the following embodiments, it is assumed that the input image data is a color image. However, even when the input image does not include color information, a plurality of pieces of input information are required. Since the same processing is basically performed only when the color components do not hold, the description thereof is omitted.

Next, a sixth embodiment of the present invention will be described. FIG. 11 shows an image processing apparatus 1 to which the present invention is applied.
It is a block diagram which shows the structure of 81. The parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

That is, the image processing device 181 shown in FIG.
Has basically the same configuration as that described with reference to FIG. 1 except that a subject area detection unit 191 is provided instead of the subject area detection unit 12.

The pixels I (i, j) of the image data input from the imaging unit 11 include, for example, a luminance signal Y (i, j),
A plurality of components representing colors, such as a color difference signal U (i, j) and a color difference signal V (i, j), are included. Luminance signal Y
(I, j), color difference signal U (i, j) and color difference signal V
(I, j) indicates to the background information extraction unit 201 via the switch 21 in the background information extraction mode, and to the background difference calculation unit 202 and the background correction unit 2 in the subject extraction mode.
06 and the memory 205.

The background information extracting section 201 is composed of the average value image generating section 43 described with reference to FIG. 2, and in the background information extracting mode, calculates the average of the pixel values by using equations (29) to (31). Value AveY (i, j), AveU
An average image composed of (i, j) and AveV (i, j) is generated, output to the memory 23 and stored, and in consideration of a variable element such as illumination light, the following equation (41) is used.
To the pixel value Sav2 of the dispersed image using Expression (43).
(I, j) = (Sav2Y (I, j), Sav2U
(I, j), SavV (I, 2j)), and outputs the pixel value Sav2 (I, j) of the dispersed image to the memory 23 for storage.

[Equation 36] ... (41)

(37) ... (42)

(38) ... (43)

In the subject extraction mode, the background difference calculation unit 202 receives the input of the pixel value I (i, j) of the input image data via the switch 21 and
3, the average value image is compared with the pixel value of the input image data according to the following equation (44) to determine whether the pixel at the corresponding coordinate is a subject candidate or a background candidate. Then, a pixel B (i, j) which is 1 if the corresponding pixel is a subject candidate and is 0 if it is a background candidate
Generates a binary image composed of
Output to 3.

[Equation 39] ... (44)

When the subject probability calculation unit 203 calculates the subject probability using the Bayesian inference described above, the background difference calculation unit 202 switches the switch 21 together with the binary image composed of the pixels B (i, j). The pixel value I (i, j) of the input image data input through the
Output to 3.

Here, the threshold values Ty, Tu and Tv of each component used in the equation (44) are, for example, the pixel values Sav of the dispersed image calculated by the background information extraction unit 201.
Using 2 (I, j), the following equations (45) to (47)
Can be calculated by

(Equation 40) ... (45)

[Equation 41] ... (46)

(Equation 42) (47) where ay, au, and av are preset coefficients.

The subject probability calculating section 203 calculates the probability Pobj (i, j) that each coordinate is included in the subject area.
Probability Pobj (i, j) that each coordinate is included in subject area
May be a method using Bayesian estimation theory as described in the first embodiment, or may be another method. For example, due to the nature of the background subtraction method, the pixel that has been set as a subject candidate in the background difference calculation unit 202 has a high probability of actually being a subject, so that the pixel value B (i, j) of the input binary image is 1 For the pixel of, the subject probability Pobj (i, j) may be set to 1.0, and for the other pixels, the subject probability for each region may be set empirically or experimentally.

For example, if the probability that the subject area is located at the center of the image is high, the subject probability based on the coordinates on the image is calculated as follows.
It may be set in advance as shown in FIG. In such a case, a table for referencing the subject probability by the coordinates is prepared in advance in the subject probability calculation unit 203, and the table is referred to according to the position of the pixel to calculate the subject probability Poj (i, j). You. The object probability Pobj (i, j) is output to the memory 204 and stored.

In the case where the subject probability calculation unit 203 obtains the subject probability Pobj (i, j) using the Bayesian inference described above, the subject probability calculation unit 20
3 substantially corresponds to the subject determination unit 25 of FIG.
It has a configuration similar to that of the subject determination unit 161 of 0, and executes similar processing. Then, the subject probability calculation unit 203
The pixel O (I, I,
j) is output as the subject probability Poj (i, j).

In the subject probability calculating section 203, if necessary, the pixel value B of the input binary image
(I, j) can also be corrected. For example, a threshold value of the object probability Pobj (i, j) is set in advance, and for a pixel whose pixel value B (i, j) = 0, the corresponding object probability Pobj (i, j) is smaller than the threshold value. If larger,
The corresponding pixel is regarded as a subject, and the pixel value B ′ (i, i,
j) = 1. Pixel value B ′ corrected as necessary
(I, j) is output to the background correction unit 206 together with the subject probability Pobj (i, j).

The memory 204 includes a subject probability calculating section 203
From the object probability Pobj (i, j) of the image currently being processed
Is received and saved, and the background correction unit 206 sets the subject probability of the previous image as Pprv (i, j).
Output to

The memory 205 has a subject probability calculating section 203.
, Receives and stores the pixel value I (i, j) of the image currently being processed, and outputs the pixel value of the immediately preceding image to the background correction unit 206 as the pixel value Iprv (i, j). .

The background correction unit 206 includes the subject probability calculation unit 2
In the pixel value B ′ (i, j) of the binary image input from step 03, the correction is performed only on the pixel whose value is 0, that is, the pixel regarded as a background candidate.

In order for the background correction unit 206 to determine whether the pixel indicated by the coordinates (i, j) is the background or the subject, it is assumed that the pixel at the corresponding coordinate is the background. And a pixel considered appropriate when the object is assumed to be an object, and it is determined which pixel value I (i, j) actually input is closer to. This is done by comparison.

For pixels considered as background candidates,
A pixel value I′bg = (Y′bg, which is considered to be appropriate when the pixel at the coordinates (i, j) is assumed to be the background.
U′bg, V′bg), and a pixel value I′obj = appropriate assuming that the subject is assumed
The three-dimensional distance Dbg and distance Dobj between each value of (Y'obj, U'obj, V'obj) and the input pixel value are expressed by the following equations (48) and (49). .

[Equation 43] ... (48)

[Equation 44] ... (49)

Then, the background correction unit 206 calculates the expression (48)
The distance Dbg and the distance Dobj calculated using are compared. If the distance Dbj is smaller, the pixel indicated by the coordinates (i, j) is determined to be the subject. If the distance Dbg is smaller, The pixel indicated by the coordinates (i, j) is determined to be the background.

The estimated pixel value I'bg (i, j) of the background is
Since the pixel value Ave (i, j) of the average value image of the background image data stored in the memory 23 can be read and used, it is expressed by Expressions (29) to (31).
Average values AveY (i, j) and AveU (i,
j) and AveV (i, j) are obtained by using the above equation (4).
By substituting into 8), the distance Dbg can be calculated.

However, the estimated pixel value I'o of the subject
Since no corresponding pixel value is given to bj (i, j), the background correction unit 206 determines the pixel value Ip of the immediately preceding input image stored in the memory 204 and the memory 205.
rv (i, j) and the object probability P at each coordinate of the image
Using prv (I, j), the estimated pixel value I ′ of the subject
obj (i, j) is calculated.

FIG. 13 is a block diagram showing a more detailed configuration of background correction section 206.

The background distance calculation unit 211 receives the input of the pixel value I (i, j) of the image data currently being processed and the pixel value Ave (i, j) of the average image, and receives the above equation (48).
Are average values of pixel values AveY (i, j), AveU
The distance Dbg is calculated using the following equation (50) in which (i, j) and AveV (i, j) are substituted, and the correction unit 21
5 is output.

[Equation 45] ... (50)

The previous image distance calculator 212 calculates the pixel value I (i, j) of the image data currently being processed and the pixel value Iprv of the immediately preceding input image stored in the memory 205.
Upon receiving the input of (i, j), the pixel values IprvY (i, j), Iprv of the immediately preceding input image are calculated in the above equation (49).
The distance Dprv is obtained by using the following equation (51) into which U (i, j) and IprvV (i, j) are substituted, and is output to the subject distance calculation unit 214.

[Equation 46] ... (51)

The weight calculating section 213 receives the subject probability Pprv (I, j) of each coordinate of the immediately preceding image, and the subject distance calculating section 214 causes the previous image distance calculating section 212
Is corrected using the following equation (52), and the correction value w for calculating the distance Dobj is output to the subject distance calculation unit 214.

[Equation 47] ... (52)

The subject distance calculation unit 214 receives the input of the distance Dprv shown in the equation (51) from the previous image distance calculation unit 212 and receives the input from the weight calculation unit 213 in the equation (52).
, The correction is performed by multiplying the distance Dprv by the correction value w, and the distance Dobj is calculated as shown in the following equation (53).
5 is output. Dobj = w × Dprv (53)

That is, in the subject distance calculating section 214, the subject probability Pprv of each coordinate of the immediately preceding image is calculated.
When (I, j) is 1, the distance Dobj = the distance Dprv, and the correction is made so that the smaller the subject probability Pprv (I, j), the larger the distance Dobj becomes.

The correction unit 215 includes a subject probability calculation unit 203
, The input of the pixel value B ′ (i, j) of the binary image from the background distance calculation unit 211 and the distance Dbg shown in the equation (50) from the subject distance calculation unit 214 to the equation (5).
The input of the distance Dobj shown in 3) is received. Correction unit 2
Reference numeral 15 denotes a pixel value B '(i, j) of the binary image first.
And if the pixel value B ′ (i, j) = 1,
Without correction, the pixel at the corresponding coordinates (i, j) is determined to be a subject, and a pixel value O (I, j) = 1 is output.

On the other hand, pixel value B ′ (i, j) = 0
, The correction unit 215 calculates the distance Dbg and the distance Dob
j, and when the distance Dobj is smaller than the distance Dbg, the pixel at the corresponding coordinate (i, j) is determined to be a subject, and a pixel value O (I, j) = 1 is output, and the distance D
If obj is greater than the distance Dbg, the pixel at the corresponding coordinate (i, j) is determined to be the background, and the pixel value O
(I, j) = 0 is output.

In the image processing apparatus 181 described with reference to FIG. 11, in the background information extraction mode, the background information extraction unit 201 extracts the background information described using the equations (41) to (43). Has been described as being output to and stored in the memory 23. For example, instead of the background information extraction unit 201, background information is extracted using the background information extraction unit 22 described with reference to FIG. ) Through (31), and using the subject candidate detection unit 24 described with reference to FIG. 2 instead of the background difference calculation unit 202, instead of the calculation of expression (44). By executing the operations of Expressions (32) to (38), a binary image including the pixel values B (I, j) may be obtained.

Also, in the sixth embodiment, as the a priori knowledge for calculating the subject probability Pobj (i, j), the probability that the subject exists on the screen described with reference to FIG. 12 is high. Although the position information is used, for example, the object probability Pobj (i,
j) may be calculated.

Next, a seventh embodiment of the present invention will be described. In the seventh embodiment, the same smoothing processing as that performed by the multi-value mask generation unit 64 described with reference to FIGS. 4 and 10 is added to the image processing in the above-described sixth embodiment. Things.

FIG. 14 is a block diagram showing a configuration of an image processing device 221 to which the present invention is applied. Parts corresponding to those in FIG. 11 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

That is, the image processing device 221 shown in FIG.
Has basically the same configuration as that described with reference to FIG. 11 except that a subject area detection unit 231 is provided instead of the subject area detection unit 191.

The subject area detecting section 231 replaces the subject probability calculating section 203 with the subject probability calculating section 241.
And the memory area 242 is newly provided, and the object region detecting unit 1 described with reference to FIG.
It has basically the same configuration as 91.

Here, the background information extraction unit 201 and the memory 2
3 and the processing of the background difference calculation unit 202 are the same as those in the sixth embodiment, and thus description thereof is omitted.

For the first image data in the subject extraction mode, the background difference calculator 202 and the subject probability calculator 24
1. Background correction unit 206, memory 204, and memory 2.
05 is the background difference calculation unit 20 in the sixth embodiment.
The same processing as that of the second to the background correction unit 206 is performed, and the pixel values O (i,
j) is generated and output. Then, 1
The pixel value O (i, i) of the binary image corresponding to the
j) is input and stored as the pixel value Oprv (i, j) of the binary image corresponding to the immediately preceding image data, which is used to detect the subject area of the next image data.

Object probability calculating section 2 receiving the input of the pixel value B (i, j) of the binary image corresponding to the second image data
41 reads the pixel value Oprv (i, j) of the binary image corresponding to the immediately preceding image data from the memory 242. Then, the subject probability calculation unit 241 uses, for example, Equation (10) described above to calculate the pixel value Oprv (i,
j) is smoothed to generate a multi-valued image composed of pixel values M (i, j). For example, the pixel value Oprv (i,
When j) is smoothed as shown in FIG. 15, the pixel value M (i, j) becomes 2 corresponding to the immediately preceding image data.
In the pixel value Oprv (i, j) of the value image, a position closer to the center point of the part determined to be the subject area has a value closer to 1 (that is, it is determined that the probability of being the subject is higher). ).

The object probability calculating section 241 normalizes the pixel value M (i, j) using the above-described equation (11) to generate the object probability Pobj (i, j),
04. That is, the object probability Pobj (i,
j) indicates that a pixel position Oprv (i, j) of a binary image corresponding to the immediately preceding image data has a higher probability as the position is closer to the center point of a portion determined to be a subject area. Is done.

It should be noted that also in the seventh embodiment, the background information extraction method and the subject probability Pobj (i, j)
May be calculated using any of the methods described in the sixth embodiment, and furthermore, the subject probability Pobj
In calculating (i, j), 2 corresponding to the immediately preceding image data
In the pixel value Oprv (i, j) of the value image, a histogram of the pixel value of the input image and a histogram of the background image corresponding to the pixel determined to be the subject area are generated, and these histograms are compared. Thus, the possibility that a certain color appears in the subject area may be evaluated to calculate the subject probability.

Next, an eighth embodiment of the present invention will be described.

FIG. 16 is a block diagram showing a configuration of an image processing device 251 to which the present invention is applied. Parts corresponding to those in FIG. 11 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

That is, the image processing device 251 shown in FIG.
Has basically the same configuration as that described with reference to FIG. 11 except that a subject area detection unit 261 is provided instead of the subject area detection unit 191.

The subject area detection section 261 is provided with a background correction section 272 instead of the background correction section 206.
1 except that a memory 271 is newly provided.
Since the configuration is basically the same as that of the first subject area detection unit 191, the description thereof is omitted.

The background information extraction unit 201 and the memory 2
3. Background difference calculation unit 202 and subject probability calculation unit 2
The process of step 03 is the same as that of the sixth embodiment, and a description thereof will be omitted.

The subject probability calculating section 203 calculates the probability Pobj that each coordinate is included in the subject area by the above-described processing.
(I, j) is calculated and output to the memory 204. If necessary, the pixel value B (i, j) of the binary image input from the background difference calculation unit 202 is corrected.
The pixel value B ′ (i, j) of the value image is output to the memory 271.

The background correction section 272 needs the pixel values of the area near the coordinates (i, j) to be corrected among the pixel values B ′ (i, j) of the binary image in the correction processing described later. I do. The memory 271 stores the pixel value B ′ of the binary image for the entire screen.
(I, j) is stored, and the pixel value R of the binary image required for the correction is adjusted according to the processing timing of the background correction unit 272.
(I, j) is output or the background correction unit 272 is output.
Reads the pixel value R (i, j) from the memory 271.

The neighborhood area for the coordinates to be corrected is, for example, a pixel within a predetermined distance from the coordinates (i, j) to be corrected (that is, a pixel having a predetermined radius around the coordinates (i, j) to be corrected). Pixel within a circle) or the following equation (5)
As shown in 4), it may be a pixel in a square area determined by a preset constant m, or may be a pixel in an area set by another method.

[Equation 48] ... (54)

Hereinafter, the coordinates in the vicinity area are represented by coordinates (i ′,
j ').

[0187] The background correction section 272 is, like the background correction section 206, provided with the object probability calculation section 203 via the memory 271.
In the pixel value B ′ (i, j) of the binary image input from the above, the correction is performed only on the pixel whose value is 0, that is, the pixel regarded as the background candidate.

The background correction unit 272 calculates the coordinates (i,
The method of determining whether the pixel indicated by j) is the background or the subject is also the same as that of the background correction unit 206, and the above-described equations (48) and (49) are used to calculate the corresponding coordinates. Pixels that are considered appropriate when the pixel is assumed to be the background and pixels that are considered appropriate when the pixel is assumed to be the subject are calculated, and the actually input pixel value I (i , J) by comparing which one is closer.

Therefore, in this case as well, the distance Dbg can be calculated as in the seventh embodiment, but the pixel value corresponding to the estimated pixel value I'obj (i, j) of the subject is given. Not. Therefore, the background correction unit 272 calculates the estimated pixel value I'obj (i, j) of the subject using information near the corresponding coordinates stored in the memory 271, the memory 204, and the memory 205.

FIG. 17 is a block diagram showing a more detailed configuration of the background correction section 272. Parts corresponding to those in FIG. 13 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

As described with reference to FIG. 13, the background distance calculation unit 211 calculates the pixel value I (i, j) of the image data currently being processed and the pixel value Ave (i, j) of the average image.
j), the distance Dbg is calculated using Expression (50), and is output to the correction unit 215.

The subject distance calculation section 281
, The object probability Pobj (i ′, j ′) of the above-described neighboring region (for example, the region represented by Expression (54)) supplied from the memory 204,
And the estimated pixel value I′bg (i) of the background from the following Expressions (55) to (57) using the pixel value I (i ′, j ′) of the above-described neighborhood supplied from the memory 205. ,
j) is obtained.

[Equation 49] ... (55)

[Equation 50] ... (56)

(Equation 51) ... (57)

Here, the subject distance calculation unit 281 calculates the subject probability Pobj (i ′,
j ′) and the pixel value I (i ′, j ′) may be selectively read from the memories 204 and 205.

Equations (55) to (57) represent the coordinates (i,
j) In the neighborhood of the pixel, the background difference calculation unit 20
2 or only the pixels determined to be the subject in the subject probability calculation unit 203 are selected, and the selected pixel values are weighted by the corresponding subject probabilities and then averaged.

The correction unit 215 includes the distance Dbg supplied from the background distance calculation unit 211 and the subject distance calculation unit 281.
The correction is executed by using the distance Dobj supplied from, by the same processing as that described with reference to FIG. 13, and the pixel O (i, j) of the generated binary image is output.

Note that also in the eighth embodiment, the method of extracting background information and the object probability Pobj (i, j)
May be calculated by any of the methods described in the sixth embodiment or the seventh embodiment.

Next, a ninth embodiment of the present invention will be described.

FIG. 18 is a block diagram showing a configuration of an image processing device 291 to which the present invention is applied. Parts corresponding to those in FIG. 16 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

That is, the image processing device 291 shown in FIG.
Has basically the same configuration as the case described with reference to FIG. 16 except that a subject area detection unit 301 is provided instead of the subject area detection unit 261.

The subject area detecting section 301 replaces the subject probability calculating section 203 with the subject probability calculating section 31.
1 and a new memory 312 are provided, and the configuration is basically the same as that described with reference to FIG. Except for the subject probability calculation unit 311 and the memory 312, the same processing as that of the eighth embodiment is executed, and the description is omitted.

The memory 132 has a background difference calculation unit 202
The pixel value B (i, j) of the binary image calculated in is supplied and stored. The memory 312 stores the subject probability calculation unit 3
The pixel value B (i, j) of the stored binary image is output to the subject probability calculation unit 311 at a timing corresponding to the processing executed by the subject probability calculation unit 31 or the subject probability calculation unit 31
The pixel value B (i, j) of the binary image is read by 1.

The subject probability calculating section 311 has a memory 312
, Or necessary information is read out, and a non-linear smoothing process as shown in the following equation (58) is performed to generate a multi-valued image M (i, j).

(Equation 52) ... (58)

Here, C ′ is a predetermined constant, which determines the accuracy of the value of the probability Pobj (i, j) of the subject. Also, dmin is, as shown in FIG. 19, B (i ′,
j ′) = 1 is the distance to the nearest pixel (i ′, j ′), and is calculated by the following equations (59) and (60).

(Equation 53) ... (59)

(Equation 54) (60) Here, the coefficient D is a value preset as the maximum value that dmin can take.

The multi-valued image M generated by equation (58)
(I, j) is normalized by the following equation (61), and is output to the memory 204 as the subject probability Pobj (i, j). Pobj (i, j) = M (i, j) / C ′ (61)

Other processes are the same as those described with reference to the eighth embodiment, and a description thereof will not be repeated.

In the ninth embodiment, the background information extraction method and the subject probability Pobj (i, j)
Any of the methods described in the sixth to eighth embodiments may be used as the calculation method.

In the first to ninth embodiments described above, it goes without saying that the processes executed in the background information extraction mode and the subject extraction mode may be executed in different combinations.

In all the processing described above, a plurality of background images are captured in the background information extraction mode to extract background information. However, only one background image is captured. The processing in the background information extraction mode may be simplified by using the pixel value of each coordinate of the background image.

[0209] The above-described series of processing can be executed by software. The software is a computer in which a program constituting the software is built in dedicated hardware, or a general-purpose personal computer that can execute various functions by installing various programs. For example, it is installed from a recording medium.

As shown in FIG. 1 and the like, this recording medium
Apart from the computer, a magnetic disk 31 (including a floppy (registered trademark) disk) on which the program is recorded and an optical disk 32 (CD-ROM (Compact Disk-Read On) which are distributed to provide the program to the user.
ly Memory), DVD (including Digital Versatile Disk)),
It is composed of a magneto-optical disk 33 (including an MD (Mini-Disk)) or a package medium including a semiconductor memory 34 or the like.

[0211] In this specification, a step of describing a program recorded on a recording medium may be performed in a chronological order according to the described order. This also includes processing executed in parallel or individually.

[0212]

According to the image processing apparatus, the image processing method, and the program of the present invention, a captured image data is input, and a background image consisting only of the input background region, and a background image and a subject region are input. Is calculated, and based on the calculated difference between the background image and the target image, the candidate region of the subject region is divided from the target image, and each pixel of the target image is included in the subject region. Is calculated, and based on the calculated probability, the candidate region of the divided subject region is corrected.From the calculated probability and the correlation of the pixels in the vicinity of the predetermined pixel, the predetermined pixel is calculated. By judging whether or not the pixel is included in the subject, the corrected candidate region of the subject region is further corrected. Feelings By performing correction using, for example, even if it contains a pixel having a background value close to the part of the subject can be extracted accurately subject area.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus to which the present invention has been applied.

FIG. 2 is a block diagram illustrating a more detailed configuration of a background information extraction unit in FIG. 1;

FIG. 3 is a block diagram illustrating a more detailed configuration of a subject candidate detection unit in FIG. 1;

FIG. 4 is a block diagram illustrating a more detailed configuration of a subject determination unit in FIG. 1;

FIG. 5 is a block diagram illustrating a configuration of an image processing apparatus to which the present invention has been applied.

FIG. 6 is a block diagram illustrating a more detailed configuration of a background information extraction unit in FIG. 5;

FIG. 7 is a block diagram illustrating a configuration of an image processing apparatus to which the present invention has been applied.

FIG. 8 is a diagram for describing an example of an area selected as a part other than a subject area.

FIG. 9 is a block diagram illustrating a configuration of an image processing apparatus to which the present invention has been applied.

FIG. 10 is a block diagram illustrating a more detailed configuration of a subject determination unit in FIG. 9;

FIG. 11 is a block diagram illustrating a configuration of an image processing apparatus to which the present invention has been applied.

FIG. 12 is a block diagram for explaining an example of setting a subject probability.

13 is a block diagram illustrating a more detailed configuration of the background correction unit in FIG.

FIG. 14 is a block diagram illustrating a configuration of an image processing apparatus to which the present invention has been applied.

FIG. 15 is a diagram for describing an example of smoothing.

FIG. 16 is a block diagram illustrating a configuration of an image processing apparatus to which the present invention has been applied.

FIG. 17 is a block diagram illustrating a more detailed configuration of a background correction unit in FIG. 16;

FIG. 18 is a block diagram illustrating a configuration of an image processing apparatus to which the present invention has been applied.

FIG. 19 is a diagram illustrating a non-linear smoothing process.

FIG. 20 is a diagram for describing a relationship between a binary image and a subject probability.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 11 imaging part, 12 subject area detection part, 13 control part, 21 switch, 22
Background information extraction section, 23 memory, 24 subject candidate detection section, 25 subject determination section, 41 maximum value image generation section, 42 minimum value image generation section, 43 average value image generation section, 51 threshold value calculation section, 522 value processing section , 6
1 background condition probability detection section, 62 histogram generation section, 63 subject condition probability detection section, 64 multi-value mask generation section, 65 subject / background probability detection section, 66 Bayes estimation calculation section, 67 logical sum calculation section, 71 image processing device , 81 subject area detecting section, 91 background information extracting section, 101 deviation image generating section, 111 image processing apparatus, 121 subject area detecting section, 131 correction coefficient calculating section, 132 memory, 133 pixel value correcting section,
141 image processing device, 151 subject region detection unit, 161 subject determination unit, 171 to 173 memory, 181 image processing device, 191 subject region detection unit, 201 background information extraction unit, 202 background difference calculation unit, 203 subject probability calculation unit, 204, 20
5 memory, 206 background correction unit, 211 background distance calculation unit, 212 previous image distance calculation unit, 213 weight calculation unit, 214 subject distance calculation unit, 215 correction unit, 221 image processing device, 231 subject area detection unit, 241 subject probability Calculator, 242 memory,
251 image processing device, 261 subject area detection unit, 271 memory, 272 background correction unit, 281
Subject distance calculation unit, 291 image processing device, 30
1 subject area detecting section, 311 subject probability calculating section,
312 memory

Claims (14)

[Claims] 1. An input means for receiving input of imaged image data, a background image consisting only of a background area inputted by the input means, and an object consisting of the background area and the subject area inputted by the input means Image difference calculating means for calculating a difference from an image; and dividing the candidate area of the subject area from the target image based on a difference between the background image and the target image calculated by the image difference calculating means. A dividing unit, a probability calculating unit that calculates a first probability that each pixel of the target image is included in the subject area, and the first probability calculated by the probability calculating unit, based on the first probability. First correcting means for correcting the candidate area of the subject area divided by the dividing means, the first probability calculated by the probability calculating means, and the vicinity of a predetermined pixel Determining whether or not the predetermined pixel is a pixel included in the subject from the correlation of the pixels in the area, thereby further correcting the candidate area of the subject area corrected by the first correction unit; An image processing apparatus comprising: a second correction unit. 2. The apparatus according to claim 1, wherein said probability calculating means calculates said first probability using Bayesian estimation.
An image processing apparatus according to claim 1. 3. The image processing apparatus according to claim 1, wherein the probability calculation unit calculates the first probability based on a weighting factor set for each coordinate. 4. The first probability calculating means performs a smoothing process on a distribution of a candidate area of the subject area and a distribution of a candidate area of the background area based on a result of the division by the dividing means. The image processing apparatus according to claim 1, wherein is calculated. 5. The image processing apparatus according to claim 4, wherein the smoothing process is a nonlinear smoothing process. 6. The method according to claim 1, wherein the second correction unit corrects only an area determined as not being a candidate area of the subject area as a result of the correction by the first correction unit, in an area adjacent to the predetermined pixel. The image processing apparatus according to claim 1, wherein the correction is performed based on pixel correlation. 7. The method according to claim 6, wherein the second correction unit is configured to calculate the first probability based on the first probability calculated by the probability calculation unit and a correlation between pixels in an area near the predetermined pixel. Calculating a second probability indicating the probability of being the background area and a third probability indicating the probability of being the subject area;
The image processing apparatus according to claim 1, wherein when the second probability is smaller than the third probability, the predetermined pixel is determined to be a pixel included in the subject area. 8. The method according to claim 1, wherein the second probability takes a larger value as the difference between the pixel value of the corresponding coordinate of the target image and the pixel value of the corresponding coordinate of the background image is smaller. 8. The image processing device according to 7. 9. The third probability uses a value obtained by weighting and averaging a pixel value of a pixel having a high first probability except for the predetermined pixel in the neighboring region. The image processing apparatus according to claim 7, wherein the calculation is performed by: 10. When the image data input by the input unit includes color information, the image difference calculation unit calculates a difference between the background image and the target image for each component of the color information. The image processing apparatus according to claim 1, wherein: 11. When the image data input by the input unit includes color information, the second correction unit includes:
The image processing apparatus according to claim 7, wherein the second probability and the third probability are calculated for each component of the color information. 12. An input control step of controlling input of captured image data; a background image consisting only of a background area whose input is controlled by the processing of the input control step; and an input by the processing of the input control step. An image difference calculating step of calculating a difference between the controlled background image and a target image including the subject area, based on a difference between the background image and the target image calculated by the processing of the image difference calculating step, A dividing step of dividing the candidate region of the subject region from the target image; a probability calculating step of calculating a probability that each pixel of the target image is included in the subject region; and a process of the probability calculating step. A first correcting unit that corrects the candidate area of the subject area divided by the processing of the dividing step based on the calculated probability. A positive step, the probability calculated by the process at the probability calculating step, and from the correlation of the pixels in the vicinity area of a given pixel,
A second correction step of further correcting the candidate area of the subject area corrected by the processing of the first correction step by determining whether the predetermined pixel is a pixel included in the subject; An image processing method comprising: 13. An input control step of controlling input of captured image data; a background image consisting only of a background area whose input is controlled by the processing of the input control step; and an input by the processing of the input control step. An image difference calculating step of calculating a difference between the controlled background image and a target image including the subject area, based on a difference between the background image and the target image calculated by the processing of the image difference calculating step, A dividing step of dividing the candidate region of the subject region from the target image; a probability calculating step of calculating a probability that each pixel of the target image is included in the subject region; and a process of the probability calculating step. A first correcting unit that corrects the candidate area of the subject area divided by the processing of the dividing step based on the calculated probability. A positive step, the probability calculated by the process at the probability calculating step, and from the correlation of the pixels in the vicinity area of a given pixel,
A second correction step of further correcting the candidate area of the subject area corrected by the processing of the first correction step by determining whether the predetermined pixel is a pixel included in the subject; A recording medium on which a computer-readable program is recorded. 14. An input control step of controlling input of captured image data; a background image consisting only of a background area whose input is controlled by the processing of the input control step; and an input by the processing of the input control step. An image difference calculating step of calculating a difference between the controlled background image and a target image including the subject area, based on a difference between the background image and the target image calculated by the processing of the image difference calculating step, A dividing step of dividing the candidate region of the subject region from the target image; a probability calculating step of calculating a probability that each pixel of the target image is included in the subject region; and a process of the probability calculating step. A first correcting unit that corrects the candidate area of the subject area divided by the processing of the dividing step based on the calculated probability. A positive step, the probability calculated by the process at the probability calculating step, and from the correlation of the pixels in the vicinity area of a given pixel,
A second correction step of further correcting the candidate area of the subject area corrected by the processing of the first correction step by determining whether the predetermined pixel is a pixel included in the subject; A program causing a computer to execute a process including: