JP4802253B2 - Image processing apparatus and shadow removal processing program - Google Patents

Description

  The present invention relates to an image processing apparatus having an area processing function for extracting an area including a change pixel from a plurality of time-series images taken by a camera, and a shadow removal process for removing an image of a shadow portion from an image taken by a camera. Regarding the program.

  In an image processing apparatus having an area processing function that extracts a rectangular area including a change pixel from a plurality of time-series images captured by a monocular camera (one camera), a shadow area is included in the extracted area. . With the object estimation and position estimation functions, it is difficult to predict the appearance direction and state of shadows, and if the estimation process is performed using area information including shadow areas, the performance will be extremely reduced. there were.

  As a conventional shadow removal technique in this type of image processing technology, “the background area texture is lost in the object area and the background area texture remains relatively good in the shadow area. Prepare an image, and the shadow area where the texture of the background part remains relatively well and the area of the target object where the texture of the background part is lost are affected by the texture of the background part as an effect on the brightness. There has been a technique of “capturing and performing shadow detection and image processing” (see Patent Document 1). However, this shadow removal technology is a configuration that prepares a background image in advance and detects the shadow area of the target object based on this background image (background texture). In addition, it is necessary to prepare a simple background image in advance, and the use (applicable range) is limited to road monitoring with a fixed shooting area. In addition, for example, the shade density changes drastically due to changes in sunlight, etc., and the background roads and structures are exposed to rain and snow, resulting in large brightness fluctuations in the shadow area and background area of the image. In this case, there is a problem of versatility and reliability that the device is likely to malfunction and therefore cannot be applied to the above estimation function.

JP-A-2005-251132

  As described above, in an image processing apparatus having a regionization processing function that extracts a region including a change pixel from a plurality of time-series images captured by a camera, an object is obtained using region information including a shadow region. When performing the estimation processing, there is a problem that the processing performance is affected, and in the conventional technology that detects shadows by capturing the influence of the texture of the background portion on the brightness, an appropriate standard for shadow detection is required. A means for preparing a background image in advance is required, and there are problems in terms of versatility and reliability.

  The present invention has been made in view of the above circumstances, and an image processing apparatus having a regionization processing function for extracting a region including a change pixel from a plurality of time-series images captured by a camera does not require a background image, An object of the present invention is to provide an image processing apparatus and a shadow removal processing program that can expect a shadow removal process with high reliability and reliability.

The present invention includes an image buffer capable of storing a predetermined number of color images taken in units of frames taken by one camera, and an input image N frames before from the image buffer is used as a reference image, and an input image 2N frames before The current input image is input as a difference target image, a difference area is extracted by comparing the reference image and the difference target image in pixel units, and is not necessary for extracting a moving body area from the difference processed image of the difference area Image processing means for removing noise and processing the difference- processed image of the noise-reduced difference area into a rectangular area on a screen displaying the color image; and the difference area Check whether an area of has been noted picture element is changed pixel, when the change pixel, the luminance reduction ratio between the reference image and the 2N previous frame of the input image Because, when the brightness reduction ratio is within the shadow removal determination range, obtains a similarity difference between the color information of the 2N previous frame of the input image and the reference image, the attention degree of similarity difference is below a threshold extracting a pixel as a first shadow candidate, the current of the input image calculated luminance reduction ratio of the reference image, when the brightness reduction ratio is within the shadow removal determination range, the current of the input image the reference image A shadow removal method extracting means for obtaining a similarity difference with the color information and extracting the pixel of interest whose similarity difference is equal to or less than a threshold as a second shadow candidate; and the first shadow candidate and the second shadow candidate And a shadow removal processing means for removing the target pixel determined as a shadow candidate in any one of the extraction processes from the reference image.

The present invention further includes an image buffer capable of storing a predetermined number of color images taken in units of frames taken by one camera, and inputs the current input image and the input image N frames before from the image buffer, A difference area is extracted by comparing the input image of N and the input image before N frames in a pixel unit, noise unnecessary for extraction of the moving body area is removed from the difference-processed image of the difference area, and the noise removal is performed. the difference processed image of the difference region that is an image processing means for performing processing for the region of the rectangular area on the screen for displaying the color image, said region of has been noted picture element of the differential area checks whether the change pixel, when the change pixel, the N frame before the input image a reference image, obtains a brightness reduction rate input image of the current as the differential target image, If serial brightness reduction ratio is within the shadow removal determination range, the N frame and the previous input image calculated similarity difference between the color information of the current input image, the pixel of interest thereof similarity difference is below a threshold extracting a first shadow candidate, the current input image a reference image, said calculated brightness reduction ratio input image before N frame as a difference target image, when the brightness reduction ratio is within the shadow removal determination range, A shadow removal method extracting means for obtaining a similarity difference between the current input image and the color information of the input image before N frames, and extracting the target pixel whose similarity difference is equal to or less than a threshold as a second shadow candidate; , characterized in that anda shadow removal processing means for removing the target pixel determined as shadow candidate in one of the extraction process of the first shadow candidate and the second shadow candidate from the reference image.

In the image processing apparatus, the shadow removal method extraction unit performs the extraction when the luminance of the corresponding pixel of the difference target image reaches a set minimum threshold for the pixel for which the luminance reduction rate has been calculated. It is characterized by performing .

Further, the present invention provides a computer that realizes shadow removal processing ability for sequentially inputting color images taken by one camera and removing a shadow portion from an input current image and a past image including a changed pixel. A function of inputting an input image of N frames before as a reference image from an image buffer that stores a predetermined number of color images taken in units of frames taken in step 2 as a reference image, and the reference extracting a difference area by comparing pixel by pixel between the image and the differential target image, a function of removing unnecessary noise extraction of a moving object region from the image difference processing of the differential region, which is the noise removing the the differential processing image in the difference region, a function of the area of the rectangular area on the screen for displaying the color image, is the area of the differential area Attention picture element it is checked whether the change pixel, when the change pixel, the 2N frame and the previous input image calculated luminance reduction ratio of the reference image, the brightness reduction ratio is within the shadow removal determination range If it finds a similarity difference between the color information of the 2N previous frame of the input image and the reference image, a function of extracting the pixel of interest thereof similarity difference is below a threshold as a first shadow candidate, When a luminance reduction rate between the corresponding pixel of the current input image and the reference image is obtained, and the luminance reduction rate is within a shadow removal determination range, similarity between color information of the current input image and the reference image seek time difference, determines the pixel of interest thereof similarity difference is below a threshold a function of extracting a second shadow candidate, a shadow candidate in one of the extraction process of the second shadow candidates and the first shadow candidate removing the target pixels from the reference image Features and that features a shadow removal processing program for actualizing.

Further, the present invention provides a computer that realizes shadow removal processing ability for sequentially inputting color images taken by one camera and removing a shadow portion from an input current image and a past image including a changed pixel. A function of inputting a current input image and an input image before N frames from an image buffer that stores a predetermined number of color images taken in units of frames taken in step 1, and a pixel unit of the current input image and the input image before N frames an image extracting difference region, which is the differential processing of the and function of removing unnecessary noise extraction of a moving object region from the difference processed image difference area, the noise has been removed the difference region by comparison with, checking a function of the area of the rectangular area on the screen, whether the region of has been noted picture element of the differential region is a change pixel displaying the color image If the change pixel, the N frame before the input image a reference image, said calculated brightness reduction ratio of the current input image as a difference target image, when the brightness reduction ratio is within the shadow removal determination range, the N A function of obtaining a similarity difference between the input image before the frame and the color information of the current input image and extracting the target pixel whose similarity difference is equal to or less than a threshold value as the first shadow candidate; and the current input image a reference image, said calculated brightness reduction ratio input image before N frame as a difference target image, when the brightness reduction ratio is within the shadow removal determination range, the current of the input image of the N frame before the input image calculated similarity difference between the color information, the a function of the similarity difference extracting the pixel of interest is less than the threshold value as a second shadow candidate, extracts one of the second shadow candidates and the first shadow candidate process Before determining as a shadow candidate in A function of removing the pixel of interest from the reference image, the shadow removal processing program for implementing the features.

  According to the present invention, in an image processing apparatus having an area processing function for extracting an area including a change pixel from a plurality of time-series images taken by a camera, a background image is not required, and versatility and high reliability are achieved. Shadow removal processing can be expected.

1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. The block diagram which shows the structure of the shadow removal candidate extraction and shadow removal process part provided in the image processing apparatus which concerns on the said embodiment. 5 is a flowchart showing a processing procedure of an image processing unit in the image processing apparatus according to the embodiment. 6 is a flowchart showing a main processing routine of the image processing apparatus according to the embodiment. 1 is a diagram illustrating an operation concept of an image processing apparatus according to a first embodiment of the present invention. The flowchart which shows the procedure of the shadow removal process which concerns on the said 1st Embodiment. The flowchart which shows the procedure of the shadow removal process which concerns on the said 1st Embodiment. The flowchart which shows the procedure of the shadow removal process which concerns on the said 1st Embodiment (it uses by 1st Embodiment). The flowchart which shows the re-regionalization process procedure after the shadow removal process which concerns on the said 1st Embodiment. The flowchart which shows the re-regionalization process procedure after the shadow removal process which concerns on the said 1st Embodiment. The figure which shows the characteristic of the threshold applied to the shadow removal process which concerns on the said 1st Embodiment. The figure which shows the operation | movement concept of the image processing apparatus which concerns on 2nd Embodiment of this invention. The flowchart which shows the procedure of the shadow removal process which concerns on the said 2nd Embodiment. The flowchart which shows the procedure of the shadow removal process which concerns on the said 2nd Embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

  The present invention utilizes the feature that “the shadow includes the color information of the projected destination and leaves the saturation and hue of the background color, and only the lightness is reduced”. This is realized by constructing an algorithm that uses the feature information of color information (R, G, B) and luminance (Y), and shadows can be applied without applying a difference method using a background image (texture). Eliminate shadow images efficiently and accurately from included images. As a result, an image processing apparatus that does not require a background image and has a versatile and reliable shadow removal processing function can be provided.

  FIG. 1 shows the configuration of the entire image processing apparatus according to the embodiment of the present invention, and FIG. 2 shows functional blocks of the image processing unit 13 that constitutes the main components of this embodiment. In this embodiment, image data for one frame (one screen) captured from a camera (monocular camera) is simply referred to as an image, an image on the screen, or an image for one screen.

  As shown in FIG. 1, the image processing apparatus according to the embodiment of the present invention includes a camera 11, a capture unit 12, an image processing unit 13, and a display unit 18.

  The camera 11 includes a lens unit and a color image pickup device (for example, a CCD color image sensor or a CMOS color image sensor) provided at an image forming position of the lens unit, and a subject (animal body) that moves outdoors or indoors. ) Is output in a predetermined pixel unit (for example, one frame 320 × 240 pixels = QVGA).

  The capture unit 12 has a processing function (capture function) that captures an image in frame units captured by the camera 11 as a processing target image (input image) of the image processing unit 13 and stores it in a buffer (not shown) of the image processing unit 13.

  The image processing unit 13 includes a detection processing unit 14, a shadow removal candidate extraction / shadow removal processing unit 15, and a re-regionalization processing unit 16. Further, although not shown here, the image processing unit 13 stores an image buffer memory for storing a predetermined number (for example, 30 frames) of color images input from the capture unit 12 at regular time intervals, difference processing, noise removal, region A working buffer, a working image memory, a parameter setting unit for storing external parameters, a program storage unit for storing an image processing program including a shadow removal processing program, and the like are provided. The image buffer is denoted by reference numeral 21 in FIG. 2, and as a work image memory, a processing result memory 22 is shown in FIG. 2. As the work buffer, the reference image buffer 23 and the shadow removal are shown in FIG. A candidate buffer 24 is shown.

  The detection processing unit 14 includes a difference processing unit that inputs a past image and a current image from the image buffer of the image processing unit 13, compares them in pixel units, and extracts a difference region, and a diffusion filter and a contraction filter. A noise removing unit that removes noise unnecessary for extracting a moving object region from a processed image, and a regionizing processing unit that performs processing for dividing the noise-reduced difference image into a rectangular region on the screen. Is done. The functional structures of the difference processing unit, the noise removal unit, and the regionalization processing unit are known (see, for example, Japanese Patent Application Laid-Open No. 2008-250892), and detailed description of each of these components is omitted here. The noise removal unit of the detection processing unit 14 is denoted by reference numeral 25 in FIG. 2, and the shadow removal candidate extraction unit and the shadow removal processing unit of the shadow removal candidate extraction / shadow removal processing unit 15 are respectively illustrated in FIG. Reference numerals 15a and 15b indicate. In addition, the detection processing unit 14, the shadow removal candidate extraction / shadow removal processing unit 15, and the re-regional processing unit 16 are denoted by the same reference numerals in FIG.

  The shadow removal candidate extraction / shadow removal processing unit 15 sets one frame as a reference frame and the other one or two frames as reference frames (difference target frames) for the above-described plurality of framed images. A luminance reduction rate calculation function unit that calculates the luminance reduction rate per pixel of the reference frame with reference to the reference frame, and the luminance reduction rate of the pixel for which the luminance reduction rate is calculated is set to the first attenuation For a first determination function unit that determines whether or not the pixel is within the range and a pixel that is determined to be within the first attenuation range, the luminance reduction rate of each color information of the pixel is calculated, and the luminance reduction of each color information When the rate is uniformly within the set second attenuation range, a second determination function unit that determines the pixel as a shadow candidate and a shadow with respect to the image of the reference frame based on the shadow candidate pixel Shadow removal processor to remove parts And a part. The brightness reduction rate calculation function unit, the first determination function unit, and the second determination function unit constitute the shadow removal candidate extraction unit 15a shown in FIG. 2, and the shadow removal processing function unit shows the shadow removal process shown in FIG. Part 15b is configured. The configuration and processing operations of the shadow removal candidate extraction / shadow removal processing unit 15 will be described later with reference to the drawings.

  The reregion processing unit 16 performs the region processing again on the image subjected to the shadow removal processing by the shadow removal processing unit of the shadow removal candidate extraction / shadow removal processing unit 15. This reregion processing will be described later with reference to the drawings.

  A processing procedure of the basic operation of the image processing unit 13 shown in FIG. 1 is shown in a flowchart in FIG.

  The detection processing unit 14 of the image processing unit 13 performs reference processing (a color image having red, green, and blue color information (R, G, B)) and a reference image (pre-processing after capture processing) and a reference image ( A luminance image that is not subjected to luminance expansion is prepared (step S1 in FIG. 3).

  The shadow removal candidate extraction / shadow removal processing unit 15 calculates the luminance reduction rate per pixel of the standard image with reference to the reference image, and the color information (R, G, B) is the same as the Y luminance reduction rate. Pixels that have been reduced to shadows are determined as shadow pixels (step S2 in FIG. 3), and pixels (shadow pixels) determined to be shadows (considered as shadows) are excluded from the reference image according to the determination result (steps in FIG. 3). S3).

  After removing the noise from the area where the shadow pixel is excluded (step S3 in FIG. 3), the remaining difference image is re-regioned (step S4 in FIG. 3).

  A processing procedure of the image processing unit 13 shown in FIG. 2 is shown in a flowchart in FIG.

  Since the difference processing, noise removal, and regionizing processing (steps S11 to S13 in FIG. 4) by the detection processing unit 14 have already been described, these processing are omitted here.

  In the shadow removal process, a difference interval with respect to the reference image and the target number (1 or 2) are designated from the outside for the shadow removal candidate extraction / shadow removal processing unit 15. The difference interval and the target number are registered in the reference image buffer 23. The shadow removal candidate extraction 15a reads (R) the base image from the image buffer memory 21, reads the reference image from the reference image buffer 23, and reads the processing target area data from the processing result memory 22, and stores each processing target area. Extraction processing of a shadow removal candidate is started for a pixel. In this process, a luminance reduction rate and a color similarity check are performed on the reference image, and shadow removal candidates are extracted in units of pixels (step S14 in FIG. 4). The extraction area is limited to the difference area in order to speed up the processing. The shadow removal processing unit 15b creates a shadow removal difference binarized image from the shadow removal candidates, rewrites the binarized image subjected to noise removal by the noise removal unit 25, and re-regions the region again by the re-regionalization processing unit 16. The finalization result is executed (step S15 in FIG. 4), and the image subjected to the shadow removal process is displayed on the display unit 18.

  FIG. 5 shows an operation concept of the image processing apparatus according to the first embodiment of the present invention. The image processing apparatus according to the first embodiment realizes a shadow removal processing system suitable for a detection processing system or a tracking processing system for a moving object.

  In the first embodiment, pixels whose luminance is decreased and whose color ratio does not change are handled as shadow pixels, and a difference target image of two frames is used for a reference image of one frame.

  In the shadow removal processing, after the capture processing, as the preprocessing, the latest input image having input images (luminance images that are not subjected to luminance expansion) and color images (R, G, B) 2N frames before and N frames before are prepared. . An input image N frames before is set as a reference image. The input image 2N frames before and the latest input image are set as difference target images.

  Using the input image, it is checked whether or not the luminance of the reference image is smaller than the luminance of the difference target image. If the luminance is small, a color difference is obtained and checked.

  A pixel that is determined to be “shadow” based on a difference result between the latest input image, an input image that is N frames before, and an input image that is 2N frames ago (the luminance is reduced and the color is approximated) is “ It is excluded from the reference image as a “shadow pixel”. Here, the pixel determined to be a shadow based on the difference between the input image N frames before the reference image and the input image 2N frames before the difference target image is the shadow candidate 1, and the input image N frames before the reference image The pixel determined to be a shadow based on the difference result between the difference input image and the latest input image as the difference target image is set as a shadow candidate 2, and the pixel determined to be a shadow candidate by any one of the shadow candidates is excluded from the reference image.

The procedure of the shadow removal process according to the first embodiment is shown in the flowcharts of FIGS. In addition, equations (1) to (5) that express the processing steps S27 to S29 in the shadow removal processing procedure are shown. The shadow removal sensitivity parameter (Param, determination threshold) is an external parameter. Moreover, each processing step of the flowchart shown in FIG. 8 is used in a second embodiment to be described later.

  In this shadow removal processing, first, an image pointer (the reference image is an input image N frames before, the difference target image is an input image 2N frames before and the latest input image this time) is set to secure an image necessary for the processing. (Step S21 in FIG. 6), the number of regions to be processed is acquired (Step S22 in FIG. 6).

  A loop is performed for the number of areas (steps S23 and S24 in FIG. 6), the area data pointer is set, and processing for the area size is executed for the number of areas (step S23 to step S38 in FIG. 6).

  In this process, it is checked whether or not the target pixel is a change pixel. If the target pixel is a change pixel, the brightness of the corresponding pixel of the difference target image (here, the input image 2N frames before) is lower than the minimum brightness value. On the condition that it is large, the shadow candidate pixel extraction process (steps S26 to S30 in FIG. 6) shown in the processing range (a) is executed (step S25 in FIG. 6). If the target pixel is not a change pixel, the shadow candidate pixel extraction process shown in the processing range (a) and the shadow candidate pixel extraction process (steps S31 to S35 in FIG. 7) shown in the processing range (b) are not executed.

  In this shadow candidate pixel extraction process, the luminance reduction rate (dounRatio) between the reference image and the difference target image is calculated, and it is checked whether or not the luminance reduction rate is within the shadow removal determination range. Here, the luminance reduction rate (dounRatio) is calculated for the corresponding pixel of the input image 2N frames before the difference target image, and whether or not the luminance reduction rate is within the shadow removal determination range (parameter range) is determined. This is checked (steps S26 and S27 in FIG. 6). An equation for calculating the luminance reduction rate (dounRatio) is shown in [Expression (1)], and an example of setting parameters is shown in [Expression (2)].

  When the luminance reduction rate is within the shadow removal determination range, the similarity between the color information (R, G, B) of the reference image (input image before N frames) and the difference target image (input image before 2N frames) The difference is calculated, the color difference threshold is calculated (steps S28a and S28b in FIG. 6), and when the similarity difference is equal to or smaller than the threshold, the target pixel is registered as the shadow candidate pixel 1 (steps S29 and S30 in FIG. 6). If the luminance reduction rate is outside the shadow removal determination range, the above-described extraction process of the shadow candidate pixel 1 (steps S28a to S30 in FIG. 6) is not executed. Even when the similarity difference is not less than or equal to the threshold value, the above-described extraction process of the shadow candidate pixel 1 (steps S28a to S30 in FIG. 6) is not executed.

  The equation for calculating the similarity difference in step S28a in FIG. 6 is shown in [Expression (3)], and an example of setting the threshold value in step S28b in FIG. 6 is shown in [Expression (4)] and FIG. Here, when the luminance attenuation rate is large (close to 0% shown in FIG. 11) (when the luminance value is extremely low), the color information (R, G, B) tends to be lost. As shown, the threshold value is inclined so that the shadow removal determination range is expanded as the luminance attenuation rate increases (closer to 0%). Also, the determination formula for the shadow candidate in step S29 in FIG. 6 is shown in [Expression (5)] (I = 0; shadow).

  Subsequently, on the condition that the luminance of the corresponding pixel of the difference target image (here, the input image of the current (latest) frame) is larger than the minimum luminance value, the luminance reduction rate of the reference image and the difference target image (current input image) Is calculated to check whether the luminance reduction rate is within the shadow removal determination range (within the parameter range) (steps S31 and S32 in FIG. 7).

  If it is within the shadow removal determination range, the similarity difference between the color information (R, G, B) of the reference image (input image before N frames) and the difference target image (current input image) is calculated. A threshold is calculated (steps S33a and S33b in FIG. 7), and when the similarity difference is equal to or smaller than the threshold, the target pixel is registered as the shadow candidate pixel 2 (steps S34 and S35 in FIG. 7). If the luminance reduction rate is outside the shadow removal determination range, the shadow candidate pixel 2 extraction process (steps S33a to S35 in FIG. 7) is not executed. Even when the similarity difference is not less than or equal to the threshold value, the above-described extraction process of the shadow candidate pixel 2 (step S35 in FIG. 7) is not executed.

  In the extraction process of the candidate pixels 1 and 2, when a shadow candidate pixel is determined in any of the processes, the determined shadow candidate pixel is determined as a shadow pixel (shadow pixel) (step S36 in FIG. 7). S37).

  After performing the above-described processing for the region size, noise removal processing is performed on the changed pixels created by the above processing (steps S38 and S39 in FIG. 7), and after performing these processing repeatedly for the number of regions, The difference image completed by the processing is copied, and unnecessary areas are released (steps S41 and S42 in FIG. 8).

  FIG. 9 and FIG. 10 are flowcharts showing the region reconstruction processing procedure after the shadow pixel removal processing described above.

  In this area reconstruction processing, the current image element number to be subjected to area reconstruction processing is acquired, a label image is set, and the number of areas is acquired (steps S51 and S52 in FIG. 9).

  A loop is performed for the number of areas, and the following area reconstruction processing is performed (step S3 in FIG. 9 to step S63 in FIG. 10). In the area reconstruction process, a labeling process is performed on the processing target area (step S54 in FIG. 9). Here, the area size (width, height) of the processing target area is calculated, the label image pointer is set, and then the labeling process (area limitation) is executed.

  If the labeling process is successful, the following area reconstruction process (step S56 in FIG. 9 to step S62 in FIG. 10) is executed. In this process, a regionizing process is performed, and a neighboring area combining process is performed on the area reconstructed by this process (steps S56 and S57 in FIG. 9). Subsequently, an extremely small unnecessary area (small area) and an extremely large unnecessary area (large area) are sequentially deleted (steps S58 and S59 in FIG. 9). The number of labels after re-regioning by the above-mentioned re-regioning process is checked. If one or more labels exist, storage processing of the region (region data) is performed. If no label exists, the region (region) Data) is deleted (steps S60 to S61 / S62 in FIG. 10).

  This re-region processing is executed for the number of regions.

  According to the first embodiment described above, a shadow using the color information (R, G, B) and luminance (Y) feature amount of an image having a time difference while suppressing the processing load as much as possible without using a background texture. By constructing the removal algorithm, it is possible to simplify the configuration and perform shadow removal processing with high versatility and high reliability.

  FIG. 12 shows an operation concept of the image processing apparatus according to the second embodiment of the present invention. In the image processing apparatus according to the second embodiment, the first embodiment described above is suitable for a detection processing system or a tracking processing system for a moving object, whereas there is no movement (present / not present). ) It is suitable for use in a detection processing system or a tracking processing system for a stationary object.

  In the second embodiment, pixels whose luminance is reduced and whose color ratio is not changed are handled as shadow pixels, and one frame of a reference image and one frame of a difference target image are used.

  In the second embodiment, at the time of shadow removal processing, an input image before N frames and a current input image (latest input image) are prepared as preprocessing after the capture processing. After performing the shadow pixel processing using the input image N frames before as a reference image and the current input image as a difference target image, the current input image is used as a reference image, and the shadow pixel processing is performed using the input image N frames before as a difference target image. carry out.

  Either the difference result with the input image N frames before as the reference image and the current input image as the difference target image, or the difference result with the current input image as the reference image and the input image N frames before as the difference target image. The pixel determined as “shadow” is set as “shadow pixel”, and the shadow pixel is excluded from the input image N frames before which is the original reference image.

  The shadow removal processing procedure according to the second embodiment is shown in the flowcharts of FIGS. 13, 14, and 8. In addition, FIG. 8 uses the above-described first embodiment. In the second embodiment, the input image N frames before is used as a reference image, and after performing shadow pixel processing using the current input image as a difference target image, the image for taking the difference is replaced, as described above. This is the same as the processing operation of one embodiment.

  In the shadow removal processing according to the second embodiment, first, an image pointer (the reference image is the input image N frames before and the difference target image is the current latest input image) is set in order to secure an image necessary for the processing ( Step S71 in FIG. 13), the number of regions to be processed is acquired (step S72 in FIG. 13).

  A loop is performed for the number of areas (steps S73 and S74 in FIG. 13), the area data pointer is set, and processing for the area size is executed for the number of areas (steps S73 to S88 in FIG. 13).

  In this process, it is checked whether or not the target pixel is a change pixel. If the target pixel is a change pixel, the brightness of the corresponding pixel of the difference target image (here, the current input image) is greater than the minimum brightness value. As a condition, the shadow candidate pixel extraction process (steps S76 to S80 in FIG. 13) shown in the processing range (c) is executed (step S75 in FIG. 13). If the target pixel is not a change pixel, the shadow candidate pixel extraction process shown in the processing range (c) and the shadow candidate pixel extraction process (steps S81 to S85 in FIG. 14) shown in the processing range (d) are not executed.

In this shadow candidate pixel extraction process, the luminance reduction rate (dounRatio) between the reference image and the difference target image is calculated, and it is checked whether or not the luminance reduction rate is within the shadow removal determination range. Here, the luminance reduction rate is calculated for the corresponding pixel of the current input image that is the difference target image, and it is checked whether the luminance reduction rate is within the shadow removal determination range (parameter range) (step in FIG. 13). S76, S77).

  When the luminance reduction rate is within the shadow removal determination range, the similarity difference between the color information (R, G, B) of the reference image (input image before N frames) and the difference target image (current input image) is calculated. Then, the color difference threshold is calculated (steps S78a and S78b in FIG. 13), and when the similarity difference is equal to or smaller than the threshold, the target pixel is registered as the shadow candidate pixel 1 (steps S79 and S80 in FIG. 13). If the luminance reduction rate is outside the shadow removal determination range, the above-described extraction process of the shadow candidate pixel 1 (steps S78a to S80 in FIG. 13) is not executed. Even when the similarity difference is not less than or equal to the threshold value, the above-described extraction process of the shadow candidate pixel 1 (step S80 in FIG. 13) is not executed.

  Next, the same difference process (steps S81 to S85 in FIG. 14) is executed with the current input image as the reference image and the input image N frames before as the difference target image. In this process, the luminance reduction rate of the reference image and the difference target image is calculated on the condition that the luminance of the corresponding pixel of the difference target image (here, the input image N frames before) is greater than the minimum luminance value, and the luminance reduction rate Is within the shadow removal determination range (parameter range) (steps S81 and S82 in FIG. 14).

  If it is within the shadow removal determination range, the similarity difference between the color information (R, G, B) of the reference image (current input image) and the difference target image (input image before N frames) is calculated, and the color difference threshold value is calculated. Is calculated (steps S83a and S83b in FIG. 14), and when the similarity difference is equal to or smaller than the threshold value, the target pixel is registered as the shadow candidate pixel 2 (steps S84 and S85 in FIG. 14). If the luminance reduction rate is outside the shadow removal determination range, the shadow candidate pixel 2 extraction process (steps S83a to S85 in FIG. 14) is not executed. Even when the similarity difference is not less than or equal to the threshold value, the extraction process of the shadow candidate pixel 2 (step S85 in FIG. 14) is not executed.

  In the extraction process of the candidate pixels 1 and 2, when a shadow candidate pixel is determined in any of the processes, the determined shadow candidate pixel is determined as a shadow pixel (shadow pixel) (step S86 in FIG. 14). S87).

  After performing the above-described processing for the region size, noise removal processing is performed on the change pixels created by the above processing (steps S88 and S89 in FIG. 14), and after repeating these processing for the number of regions, The difference image completed by the processing is copied, and unnecessary areas are released (steps S91 and S92 in FIG. 8).

  According to the second embodiment described above, a shadow that uses the color information (R, G, B) and luminance (Y) feature amounts of an image having a time difference while suppressing the processing load as much as possible without using a background texture. By constructing the removal algorithm, it is possible to simplify the configuration and perform shadow removal processing with high versatility and high reliability. The image processing apparatus according to the second embodiment can realize a shadow removal function suitable for use in a detection processing system or a tracking processing system for a stationary object that does not move (exists / does not exist). The functions of the shadow removal candidate extraction process and the shadow removal processing unit described above can be provided as a single program.

  DESCRIPTION OF SYMBOLS 11 ... Camera, 12 ... Capture part, 13 ... Image processing part, 14 ... Detection processing part, 15 ... Shadow removal candidate extraction and shadow removal processing part, 16 ... Re-region process part, 21 ... Image buffer memory, 22 ... Process Result memory, 23 ... reference image buffer, 24 ... shadow removal candidate buffer, 25 ... noise removal unit.

Claims (7)

An image buffer capable of storing a predetermined number of frame-by-frame color images taken by one camera, and an input image N frames before from the image buffer as a reference image and an input image 2N frames before and the current input image Is extracted as a difference target image, a difference region is extracted by comparing the reference image and the difference target image in pixel units, and noise unnecessary for extraction of the moving object region is removed from the difference processed image of the difference region. Image processing means for performing a process of converting the difference- processed image of the noise-removed difference area into a rectangular area on a screen for displaying the color image ;
Wherein said region of has been noted picture element of the difference region is checked whether the change pixel, when the change pixel, obtains a luminance reduction rate of the 2N previous frame of the input image and the reference image, the When the luminance reduction rate is within the shadow removal determination range, a similarity difference between the color information of the input image 2N frames before and the reference image is obtained, and the pixel of interest whose similarity difference is equal to or less than a threshold extracted as 1 shadow candidate, the current of the input image calculated luminance reduction ratio of the reference image, when the brightness reduction ratio is within the shadow removal determination range, the color of the current input image and the reference image A shadow removal method extracting means for obtaining a similarity difference with the information and extracting the target pixel whose similarity difference is not more than a threshold as a second shadow candidate;
A shadow removal processing means for the pixel of interest is determined that shadow candidate is removed from the reference image by any one of the extraction process of the second shadow candidates and the first shadow candidate,
An image processing apparatus comprising: An image buffer capable of storing a predetermined number of frame-by-frame color images photographed by one camera is input, the current input image and the input image N frames before are input from the image buffer, and the current input image and the A difference area is extracted by comparison with an input image of N frames before in a pixel unit, noise unnecessary for extraction of a moving object area is removed from the difference-processed image of the difference area, and the noise-removed difference area image processing means for the said differential processing images, performs processing for regions of the rectangular area on the screen for displaying the color image,
Check whether the region of has been noted picture element of the differential region is a change pixel, when the change pixel, the N frame before the input image a reference image, the current input image difference target image When the luminance reduction rate is within the shadow removal determination range, the similarity difference between the input image before the N frames and the color information of the current input image is obtained, and the similarity difference is The pixel of interest that is less than or equal to a threshold is extracted as a first shadow candidate, a luminance reduction rate is obtained by using the current input image as a reference image , and the input image before N frames as a difference target image, and the luminance reduction rate is a shadow removal If it is within the determination range, a similarity difference between the current input image and the color information of the input image N frames before is obtained, and the target pixel whose similarity difference is equal to or less than a threshold is extracted as a second shadow candidate To remove shadows And,
A shadow removal processing means for the pixel of interest is determined that shadow candidate is removed from the reference image by any one of the extraction process of the second shadow candidates and the first shadow candidate,
An image processing apparatus comprising:   The image processing apparatus according to claim 1 or 3, wherein a difference interval between the input image of 2N frames before and / or the input image of N frames before is designated from the outside.   3. The image processing apparatus according to claim 1, wherein the shadow removal method extraction unit performs the processing for extracting the first or second shadow candidate for the number of regions.   The said shadow removal method extraction means performs the said extraction, when the brightness | luminance of the pixel corresponding to the said difference object image has reached the set minimum threshold value about the pixel which calculated the said brightness | luminance reduction rate. 2. The image processing apparatus according to 2. To a computer that realizes shadow removal processing ability to sequentially input a color image taken by one camera and remove a shadow part from an input current image and a past image including a change pixel,
A function of inputting an input image of N frames before as a reference image from an image buffer storing a predetermined number of color images taken in units of frames photographed by the camera as a reference image, and a difference input image as an input image of 2N frames before;
A function to extract the difference region, to remove unwanted noise extraction of a moving object region from the image difference processing of the difference region by comparison in pixel units of the difference target image and the reference image,
A function of converting the difference- processed image of the difference area from which the noise has been removed into a rectangular area on a screen for displaying the color image ;
Wherein said region of has been noted picture element of the difference region is checked whether the change pixel, when the change pixel, obtains a luminance reduction rate of the 2N previous frame of the input image and the reference image, the When the luminance reduction rate is within the shadow removal determination range, a similarity difference between the color information of the input image 2N frames before and the reference image is obtained, and the pixel of interest whose similarity difference is equal to or less than a threshold A function to extract as one shadow candidate;
When a luminance reduction rate between the corresponding pixel of the current input image and the reference image is obtained, and the luminance reduction rate is within a shadow removal determination range, similarity between color information of the current input image and the reference image A function of obtaining a degree difference and extracting the target pixel whose similarity degree difference is equal to or less than a threshold value as a second shadow candidate;
A function of removing the target pixel determined as shadow candidate in one of the extraction process of the second shadow candidates and the first shadow candidate from the reference image,
A shadow removal processing program characterized by realizing the above. To a computer that realizes shadow removal processing ability to sequentially input a color image taken by one camera and remove a shadow part from an input current image and a past image including a change pixel,
A function of inputting a current input image and an input image N frames before from an image buffer that stores a predetermined number of color images taken by the camera in units of frames;
A function of extracting a difference area by pixel-by-pixel comparison between the current input image and the input image before N frames, and removing noise unnecessary for extracting a moving object area from the difference-processed image of the difference area ; ,
A function of converting the difference- processed image of the difference area from which the noise has been removed into a rectangular area on a screen for displaying the color image ;
Check whether the region of has been noted picture element of the differential region is a change pixel, when the change pixel, the N frame before the input image a reference image, the current input image difference target image When the luminance reduction rate is within the shadow removal determination range, the similarity difference between the input image before the N frames and the color information of the current input image is obtained, and the similarity difference is A function of extracting the target pixel that is not more than a threshold value as a first shadow candidate;
When the current input image is a reference image and the input image N frames before is used as a difference target image to obtain a luminance reduction rate, and the luminance reduction rate is within a shadow removal determination range, the current input image and the N frame A function of calculating a similarity difference with the color information of the previous input image and extracting the target pixel whose similarity difference is equal to or less than a threshold as a second shadow candidate;
A function of removing the target pixel determined as shadow candidate in one of the extraction process of the second shadow candidates and the first shadow candidate from the reference image,
A shadow removal processing program characterized by realizing the above.

Images