JP2015194829A - Image processor and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect an object in an image by separating the object from a background even in the case that the brightness of the image in a video image changes.SOLUTION: An image processor includes a storage part for storing a plurality of images photographed by a photographing device, a correction part for correcting the color value of an image of a processing target so as to highlight light and darkness on the basis of a representative value of color values of the image of the processing target and a representative value of color values of one or more past images photographed at a time before a time when the image of the processing target is photographed, and a detection part for detecting an object of a detection target from the image of the processing target by comparing the corrected color value of the image of the processing target with a threshold, and outputting the result.

Description

  The present invention relates to an image processing apparatus and an image processing method.

  In recent years, there is a growing need to reduce inspection costs by simply inspecting roads and road accessories using in-vehicle video. In the past, when at least two people boarded a vehicle, one driver operated the vehicle, and the other found an inspection item on the road, the image of the inspection item was acquired by pressing the shooting button on the drive recorder. . However, since the speed of the traveling vehicle is high, there are problems such as missing inspection items or delays in pressing the shooting button, and inspection items are not reflected in the image, so efficient inspection is not possible and images of all inspection items are acquired. Therefore, the inspection vehicle has to travel the same route several times.

  As a conventional technique, the vehicle environment recognition apparatus described in Patent Document 1 detects an edge in an image and performs color conversion of the image, and refers to an average value of the color in the image and a shape such as a speed sign, and the vehicle-mounted image A speed indicator is detected from within.

JP 2010-044445 A

  However, since the traveling direction of the vehicle changes according to the route of the road, the brightness of the image in the video changes gradually by changing the state of the light beam to forward light, backlight, direct sunlight, etc. Even if the average value of the colors in the image is used, the background in the image and the object (inspection object) in the background cannot be separated, and as a result, the object in the background cannot be detected.

  FIG. 2 is an explanatory diagram of a problem to be solved by the present invention.

  FIG. 2A shows an image 210 as an example of an image when the image is bright due to direct sunlight or the like. The image 210 includes a sky 211, a wall 212, a number plate 213, and a road 214. In this example, the wall 212 is a side wall of the road 214, and the number plate 213 is an object on which a number (for example, “123”) is displayed and installed on the side wall. In the example of FIG. 2A, the sky 211, the wall 212, the number plate 213, and the road 214 are all bright because of direct sunlight. For this reason, the gradation values of the pixels of the image 210 are unevenly distributed in a high value range as shown in the histogram H1.

  FIG. 2B shows an image 220 as an example of an image when the image is clear due to direct light or the like. The image 220 includes a sky 221, a wall 222, a number plate 223, and a road 224. These may be similar to sky 211, wall 212, number plate 213, and road 214, respectively, but at least wall 222, number plate 223, and road 224 are darker than wall 212, number plate 213, and road 214, respectively. It is reflected. For this reason, the gradation values of the pixels of the image 220 are distributed in a medium value range as shown in the histogram H2.

  FIG. 2C shows an image 230 as an example of an image when the image is dark due to backlight or the like. The image 230 includes a sky 231, a wall 232, a number plate 233, and a road 234. These may be similar to sky 221, wall 222, number plate 223 and road 224, respectively, but at least wall 232, number plate 233 and road 234 are further than wall 222, number plate 223 and road 224, respectively. It looks dark. For this reason, the gradation values of the pixels of the image 230 are distributed unevenly in a low value range as shown in the histogram H3.

  In view of the above problems, an object of the present invention is to provide an image processing apparatus that separates a background in an image from an object in the background and realizes object detection in the background even when the brightness of the image in the video changes. It is to provide that method.

  In order to solve the above problems, the present invention provides a storage unit that stores a plurality of images photographed by a photographing device, a representative value of color values of a processing target image, and the processing target image. A correction unit that corrects the color value of the image to be processed so as to emphasize light and dark based on the representative value of the color value of one or more past images taken at a time before the time; A detection unit that detects a detection target object from the processing target image by comparing the corrected color value of the processing target image with a threshold value, and outputs the result. .

  According to an aspect of the present invention, in an image processing apparatus that processes a moving image, even when the brightness of an image in a video changes, the background in the image is separated from the object in the background, and the object in the background is detected. can do. Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

1 is a block diagram illustrating an image processing apparatus according to Embodiment 1 of the present invention. It is explanatory drawing of the subject which this invention tends to solve. It is a figure explaining an example of operation | movement of the brightness correction part of the image processing apparatus which concerns on Example 1 of this invention. It is a figure explaining an example of operation | movement of the object detection part of the image processing apparatus which concerns on Example 1 of this invention. It is a figure explaining an example of operation | movement of the drawing part of the image processing apparatus which concerns on Example 1 of this invention. 3 is a flowchart illustrating an operation of an image processing method of the image processing apparatus according to the first embodiment of the present invention. It is a block diagram which shows the image processing apparatus which concerns on Example 2 of this invention. It is a figure explaining an example of operation | movement of the object detection part of the image processing apparatus which concerns on Example 2 of this invention. It is a flowchart which shows operation | movement of the image processing method of the image processing apparatus which concerns on Example 2 of this invention.

  Embodiments of the present invention will be described below with reference to the drawings. In each drawing, the same code | symbol was provided to the common component.

  FIG. 1 is a block diagram illustrating an image processing apparatus 1 according to the first embodiment of the present invention. The image processing apparatus 1 includes an input unit 10, an image correction unit 11, a color space conversion unit 12, a light / dark correction unit 13, an object detection unit 14, a drawing unit 15, a recording unit 16, a control unit 91, and a memory 90.

  The operation of each part will be described. Moving image data is input to the input unit 10. For example, the input unit 10 obtains encoded still image data such as JPG, Jpeg2000, PNG, BMP, etc. captured by imaging means (not shown) such as a drive recorder at predetermined time intervals, and the image May be output to the image correction unit 11 as an input image. In addition, the input unit 10 includes Motion JPEG, MPEG, H.264. It is also possible to extract still image data of frames at a predetermined interval from moving image data such as H.264, HD / SDI format, and output the image as an input image to the image correction unit 11. The input unit 10 may output an image acquired by the imaging unit via a bus or a network as an input image to the image correction unit 11.

  Further, as will be described later, the input unit 10 may acquire an image that has already been stored in a removable recording medium, and output the acquired image to the image correction unit 11 as an input image.

  The image correction unit 11 performs image smoothing, contour enhancement, and contrast enhancement on the input image to create a corrected input image. This correction can be performed by any method including a known method.

  The color space conversion unit 12 creates an image obtained by converting the color space of the input image after correction.

  The brightness correction unit 13 uses the brightness information of the current image (that is, the corrected input image) and the brightness information of the past image (that is, the corrected input image older than the current image) stored in the memory 90. Thus, the brightness change amount of the current image is obtained, and an image in which the brightness is corrected using the brightness change amount is created.

  The object detection unit 14 obtains a threshold value from the image after light and dark correction, uses the threshold value to separate the background and the object in the background, and detects the object in the background.

  The drawing unit 15 draws a detection frame on the image so as to surround the object detected by the object detection unit 14.

  The recording unit 16 stores an image in which the detection frame is drawn on the original image by the drawing unit 15 in the memory 90.

  The control unit 91 is connected to each element in the image processing apparatus 1 (for example, the input unit 10 to the recording unit 16 and the memory 90 described above). Each element of the image processing apparatus 1 may operate autonomously or may operate according to an instruction from the control unit 91.

  Each of the input unit 10 to the recording unit 16 may be realized by dedicated hardware, or may be realized by a general-purpose processor executing a program stored in the memory 90. In the latter case, the controller 91 may be a processor that implements the input unit 10 to the recording unit 16.

  As described above, the feature of the image processing apparatus 1 according to the present embodiment is that the image after color conversion obtained by the color space conversion unit 12 and the brightness change amount of the image calculated by the light / dark correction unit 13 are used for each image. And the object detection unit 14 obtains a threshold value from the image after the brightness correction, separates the background in the image from the object in the background using the threshold value, and determines the object in the background. Is to detect.

  Hereinafter, the configuration and operation of each unit will be described in detail.

  The color space conversion unit 12 creates an image obtained by converting the RGB color space of the input image into the Lab color space using, for example, a conventional technique.

  FIG. 3 is a diagram illustrating an example of the operation of the light / dark correction unit 13 of the image processing apparatus 1 according to the first embodiment of the present invention.

  Assuming that an image converted from the input and corrected image N into the Lab color space obtained by the color space conversion unit 12 is an image NA, the light and dark correction unit 13 uses the a value that is the color value of the image NA. A representative value of the a value in the region R2 of the image N is calculated. In the following example, the light / dark correction unit 13 calculates the average value aveR2 and the minimum value minR2 of the a value as representative values of the a value. Further, the light / dark correction unit 13 reads the average value aveR1 and the minimum value minR1 from the memory 90 as representative values of the a value in the region R1 of the image N-1.

  Here, the image N-1 is an image taken before the image N. For example, an image captured at a time N by an imaging unit mounted on a traveling vehicle is an image N, and an image captured by the imaging unit of the vehicle at a time N−1 before the time N is an image N−. 1. Hereinafter, a case where the detection target object is a number plate similar to the example of FIG. 2 will be described as an example. That is, the image N includes a sky 321, a wall 322, an object 323 installed on the wall 322, and a road 324, and the object 323 is a number plate to be detected by the image processing apparatus 1. In this embodiment, the object 323 is an object to be inspected by the user.

  Similarly, the image N-1 includes a sky 311, a wall 312, an object 313, and a road 314. These are the same kind as the sky 321, the wall 322, the object 323 installed on the wall 322, and the road 324, but were taken at different times at different locations. For example, the brightness changes as the angle of sunlight changes depending on the traveling direction of the vehicle, and the object 313 is a number plate that is different in brightness from the object 323. Therefore, the brightness of the image N may be different from the brightness of the image N-1.

  The region R2 is a region corresponding to a predetermined range in the image N. For example, the entire image N or a predetermined part of the image N may be the region R2. For example, when the range in which the number plate may be reflected in the image N is limited from the relationship between the installation position and orientation of the image pickup means and the position of the number plate to be photographed, the range is set as the region R2. May be. The region R1 in the image N-1 is the same as the region R2 in the image N.

  Next, the light / darkness correction unit 13 further uses the equation 1 to aveR1 obtained by blending the average values of the brightness of the image N-1 and the past image (a value in the above example), and further to the image N. An average value aveRN obtained by blending the brightness average values aveR2 is calculated. However, in Formula 1, C1 = C2 + C3.

  Further, the brightness correction unit 13 blends the minimum value minR2 of the brightness of the image N with the minR1 obtained by blending the minimum value of the brightness of the image N-1 and the past image further using the formula 2. A minimum value minRN is calculated. However, in Formula 2, D1 = D2 + D3.

  When calculating aveRN and minRN as described above for the image N to be processed and images past it, and then calculating new aveRN and minRN with the image taken after the image N as a new processing target The brightness correction unit 13 calculates Equations 1 and 2 with the new image to be processed as a new image N, the previous calculated aveRN as a new aveR1, and the previously calculated minRN as a new minR1. . Such a calculation method includes a representative value of pixel values of an image to be processed (average value and minimum value in the above example) and a representative value of pixel values of one or more images taken at a time earlier than that. And a representative value of pixel values of an image to be processed is an example of a calculation method for synthesizing the representative value of pixel values of one or more images taken at a time earlier than that.

  Note that the values of C1 to C3 and D1 to D3 can be appropriately set in advance. For example, the smaller the ratio of C3 to C2, the smaller the composite ratio of the average value of pixel values of an image with a new shooting time. Similarly, the smaller the ratio of D3 to D2, the smaller the composition ratio of the minimum pixel value of an image with a new shooting time. The values of C1 to C3 may be the same as the values of D1 to D3, respectively, but may be different. Generally, since there is a difference in the difficulty of object detection between when the image is bright and when it is dark, the object detection accuracy may be improved by setting C1 to C3 and D1 to D3 to different values. .

  In addition, the light / dark correction unit 13 calculates the magnification value v using Equation 3. However, in Equation 3, when aveR1> = aveR2, E1 is aveR2, E2 is aveR1, and when aveR1 <aveR2, E1 is aveR1 and E2 is aveR2.

  Furthermore, the brightness correction unit 13 uses Expression 4 to emphasize the brightness of the image N, so that the pixels that are darker than the average value in the region R2 of the image N are darker, and in the region R2 of the image N For the pixels brighter than the average value, the image N is corrected brighter. However, “an” in Expression 4 indicates the a value of each pixel of the image NA.

  The light / dark correction unit 13 obtains the anCor value, which is a value after correction, for each pixel, thereby creating an image NB with light and dark corrected. In other words, the brightness correction unit 13 corrects the a value of each pixel in the region R2 of the image N (the a value of the image NA) so that the difference from the average value in the region R2 increases, and the enlargement ratio Is the magnification value v of Equation 3. As shown in Equation 3, the magnification value v is a magnification value between aveR1 and aveR2.

  The object detection unit 14 obtains a threshold value Th for each image using Expression 5. For example, α = 0.5 and β = 0.5 may be used.

  The object detection unit 14 compares each pixel value in the image (an Cor value in the above example) with the threshold Th using the threshold Th obtained for each image. If each pixel value ≦ Th, an s value (for example, , S = 255), and when each pixel value> Th, a t value (for example, t = 0) is set, and an image in which the background and the object are separated is created based on the setting.

  FIG. 4 is a diagram illustrating an example of the operation of the object detection unit 14 of the image processing apparatus 1 according to the first embodiment of the invention.

  In the example of FIG. 4, the gradation values of the image N (a value in the above example) are distributed in a biased range of medium values as shown in the histogram H4. The gradation values (an Cor value in the above example) of the image NB generated from the image N using the equations 3 and 4 are distributed and distributed over a wider range than the histogram H4 as shown in the histogram H5. This facilitates separation of the background and the object by the threshold Th.

  An image ND obtained by changing the anCor value less than or equal to the threshold Th to the S value and the anCor value greater than the threshold Th to the t value was detected as an object 402 in the region R2, for example, as shown in FIG. The part is white and the other background 401 is black. In this example, the area of the detected object 402 matches the area of the actual object 323. This makes it possible to separate the detection target object from the background.

  As illustrated in FIG. 5, the drawing unit 15 draws a detection frame on the image so as to surround the object detected by the object detection unit 14.

  FIG. 5 is a diagram illustrating an example of the operation of the drawing unit 15 of the image processing apparatus 1 according to the first embodiment of the invention.

  The drawing unit 15 generates the image NE by drawing the image N shown in FIG. 3 or 4 by superimposing a detection frame 501 corresponding to the boundary line between the s value and the t value of the image ND. This detection frame 501 corresponds to the boundary between the object 402 (number plate in the above example) detected by the object detection unit 14 and the background 401 thereof. For example, by displaying the detection frame 501 in a manner that is easy for the user to visually recognize (for example, in a conspicuous color), it is possible to assist the user in checking the object.

  The recording unit 16 stores in the memory 90 the image NE in which the drawing unit 15 has drawn the detection frame on the original image.

  FIG. 6 is a flowchart showing the operation of the image processing method of the image processing apparatus 1 according to the first embodiment of the present invention. The procedure will be described.

  In step (S) 601, the input unit 10 outputs an input image to the image correction unit 11.

  In step 602, the image correction unit 11 performs image smoothing, contour enhancement, and contrast enhancement, which are conventional techniques, and creates a corrected image N.

  In step 603, the color space conversion unit 12 obtains a corrected image output from the image correction unit 11, that is, an image NA obtained by converting an RGB color space image into a Lab color space image as an example.

  In step 604, the brightness correction unit 13 calculates an average value aveR2 and a minimum value minR2 in the region R2 of the image N from the image NA obtained by the color space conversion unit 12. Further, the light / dark correction unit 13 reads the average value aveR1 and the minimum value minR1 in the region R1 of the image N-1 from the memory 90. Next, the light / dark correction unit 13 creates an image NB with light and dark corrected using Expression 3 and Expression 4.

  In step 605, the object detection unit 14 obtains a threshold value Th for each image using Equation 1, Equation 2, and Equation 5. Next, in step 606, the object detection unit 14 compares the pixel value anCor of the image NB with the corrected brightness with the threshold Th. That is, if anCor ≦ threshold Th, the process proceeds to step 607, and if not anCor ≦ threshold Th, the process proceeds to step 608. In step 607, the object detection unit 14 sets an s value (for example, 255) as the correction value, and in step 608, sets the t value (for example, 0) as the correction value.

  In step 609, the object detection unit 14 determines whether or not correction values have been obtained for all pixels in the target image, and the above steps are determined until it is determined that correction values have been obtained for all pixels in the target image. Repeat 606 to 608. By performing the above steps 605 to 609, for example, the region R2 of the image N in FIG. 4 can be separated into the background 401 and the object 402 as shown in the image ND after the object detection.

  In step 610, the drawing unit 15 draws the detection frame 501 on the image so as to surround the object 402 detected by the object detection unit 14.

  In step 611, the recording unit 16 stores the image NE in which the object detection frame is drawn in the memory 90.

  When the processing from step 601 to step 611 in FIG. 6 is completed for the image N and then the processing in FIG. 6 is performed for an image captured after the image N, the image N in the previous processing is the current one. The image N-1 in the processing is obtained, and the image taken after the previous image N becomes the image N in the current processing, and the above formulas 1 to 5 are calculated. At this time, the aveRN calculated by the previous equation 1 becomes the current aveR1, and the minRN calculated by the previous equation 2 becomes the current minR1. Thereby, a value indicating the brightness of the past image is blended.

  As described above, the image processing apparatus 1 according to the present exemplary embodiment includes aveR1 obtained by blending the average values of the brightness of the past images up to the image N-1, minR1 obtained by blending the minimum values of the image brightness, and the brightness of the image N. An image NB in which the brightness is corrected is created using the average value aveR2 and the minimum brightness value minR2 of the image N, and the threshold Th is obtained for each image NB. Therefore, it is possible to separate the background in the image N and the object in the background using the threshold Th.

  In the first embodiment, the example in which the average value and the minimum value are used as the representative values of the brightness of each image (specific example: a value) is shown, but other representative values may be used. The representative value indicates the state of unevenness of the brightness of the pixels included in the image (for example, whether it is biased toward the brighter side or the darker side, concentrated in a narrow range, or distributed over a wide range). Examples of usable representative values other than those described above are the maximum value and the median value. In the present embodiment, it is assumed that the number plate to be detected is always darker than the background wall, so that the average value and the minimum value are used as representative values so that the threshold Th is darker than the average value. However, for example, when it is assumed that the number plate is always brighter than the background, the maximum value may be used instead of the minimum value as the representative value. Moreover, you may use a median instead of an average value. In any case, the above calculation method can be applied. However, since it is considered that the brightness of the entire image is more strongly reflected by the average value than the median value, higher detection accuracy can be expected by using the average value.

  According to the present embodiment, the brightness information of the target image is created using the brightness information of the past image and the target image, and the threshold value for separating the region in the image for each image is obtained, whereby the moving vehicle Even when the brightness of the image in the video changes depending on the traveling direction, the object in the image can be detected separately from the background. For example, an inspection operation is performed by extracting only an image in which an object is detected by the above method from a vast number of images continuously captured by an imaging unit from a running vehicle and presenting it to a user together with a detection frame. It is possible to reduce the labor required for the user.

  Next, a second embodiment of the present invention will be described. Except for the differences described below, each part of the image processing apparatus 2 according to the second embodiment has the same function as each part denoted by the same reference numeral in the first embodiment shown in FIGS. Those explanations are omitted.

  FIG. 7 is a block diagram illustrating the image processing apparatus 2 according to the second embodiment of the present invention. The image processing apparatus 2 includes the same configuration as that in FIG. 1 of the first embodiment, but the operations of the object detection unit 24 and the control unit 92 are different from those in FIG.

  FIG. 8 is a diagram illustrating an example of the operation of the object detection unit 24 of the image processing apparatus 2 according to the second embodiment of the present invention.

  As shown in FIG. 8 (a), a shadow 801 of a moving vehicle (for example, a vehicle equipped with an imaging means for capturing an image N) is a background in the region R2 (that is, at least a part of the sky 321, the wall 322, and the road 324). When the image is overlaid on the object 323, the image becomes dark, and the difference in brightness and color between the background and the object 323 is reduced. Therefore, even if the average value of the colors in the image is used for each image, the object in the background and the background There is a problem that cannot be separated properly. Therefore, the object detection unit 24 further divides the region in the image into a plurality of small regions, and calculates a threshold value for each of these regions.

  For example, as shown in FIG. 8B, the object detection unit 24 divides the region R2 of the image N into three regions R21, R22, and R23. Next, the object detection unit 24 calculates a minimum value and an average value, that is, minR21, aveR21, minR22, aveR22, minR23, and aveR23 for each of the regions R21 to R23. Next, the object detection unit 24 obtains threshold values Th21, Th22, and Th23 for each region using Expression 6. That is, in order to obtain the threshold Th21, 21 is set to γ in Equation 6. For example, α = 0.5 and β = 0.5 may be used. In the example of FIG. 8B, the minimum value and the average value of the brightness of the region R21 (a value in the above example) are higher than the values of the other regions, and the minimum value and the average value of the brightness of the region R23 are the other values. Therefore, Th21 ≧ Th22 ≧ Th23.

  Next, the object detection unit 24 uses the threshold value Th21 to set an s value (for example, s = 255) when each pixel value ≦ Th21, and a t value (for example, t = 0) when each pixel value> Th21. Is set to create an image N21 shown in FIG. Similarly, an image N22 and an image N23 are created using the threshold values Th22 and Th23 as shown in FIGS. 8D and 8E. In the example of FIG. 8, the entire region R2 is detected as the object 802 in the image N21, and the entire region R2 is detected as the background 805 in the image N23, and the background and the object cannot be separated. On the other hand, since the brightness of the number plate becomes dark based on the value of the threshold value Th2 of the region R22, the background 803 and the object 804 corresponding to the object 323 are displayed in the image N22 as shown in FIG. Separation is successful. Therefore, by using the result of the image N22 that has succeeded in separating the background 803 and the object 804, the object detection unit 24 can create an image NF in which a detection frame is drawn as shown in FIG. It becomes. The object detection unit 24 determines that the background and the object have been successfully separated if there is at least one s value and t value in the region R2 of each image. Therefore, the object detection unit 24 can suppress the shadow of the moving vehicle and can separate the background in the image from the object in the background.

  The control unit 92 is connected to each element (for example, the input unit 10 to the recording unit 16, the object detection unit 24, and the memory 90) in the image processing apparatus 2. Each element of the image processing apparatus 2 may operate autonomously or may operate according to an instruction from the control unit 92.

  FIG. 9 is a flowchart showing the operation of the image processing method of the image processing apparatus 2 according to the second embodiment of the present invention. The procedure will be described.

  In step 901, the input unit 10 outputs the input image to the image correction unit 11.

  In step 902, the image correction unit 11 performs image smoothing, contour enhancement, and contrast enhancement, which are conventional techniques, and creates a corrected image.

  In step 903, the color space conversion unit 12 obtains a corrected image output from the image correction unit 11, that is, an image NA obtained by converting an RGB color space image into a Lab color space image as an example.

  In step 904, the light / dark correction unit 13 calculates an average value aveR2 and a minimum value minR2 in the region R2 of the image N from the image NA obtained by the color space conversion unit 12. Further, the light / dark correction unit 13 reads the average value aveR1 and the minimum value minR1 in the region R1 of the image N-1 from the memory 90. Next, the light / dark correction unit 13 creates an image NB with light and dark corrected using Expression 3 and Expression 4.

  In step 905, the object detection unit 24 obtains n threshold values Thn for each region using Equation 1, Equation 2, and Equation 6. That is, as an example, when there are three regions (n = 3), three threshold values Th1, Th2, and Th3 are obtained. Next, in step 906, the object detection unit 24 sets 1 to i. Next, in step 907, the object detection unit 24 compares each pixel value anCor of the image NB with the corrected brightness with a threshold value Thn (eg, when there are three regions, n is 1 to 3). That is, if anCor ≦ threshold Thn, the process proceeds to step 908, and if not anCor ≦ threshold Thn, the process proceeds to step 909. In step 908, the object detection unit 24 sets an s value (eg, 255) as the correction value, and in step 909, sets a t value (eg, 0) as the correction value.

  In step 910, the object detection unit 24 determines whether correction values have been obtained for all pixels in the target image area, and is determined to have obtained correction values for all pixels in the target image area. Steps 907 to 909 are repeated until the above. If it is determined in step 910 that correction values have been obtained for all pixels, the process proceeds to step 911.

  In step 911, the object detection unit 24 determines whether i> n. In step 911, when i ≦ n, correction values are not obtained for all threshold values Thn obtained from all regions (eg, when there are three regions, all of Th1 to Th3). Repeat steps 907 to 909 for the next threshold value Thn (ie, not yet used for object detection). In step 911, when i> n, correction values are obtained for all threshold values Thn. For example, when the region R2 in FIG. 8B is divided into three regions R21 to R23, the object detection unit 24 uses the image N21 in FIG. 8C and the image N21 in FIG. The image N22 in d) and the image N23 in FIG.

  By repeating the above steps 907 to 911 for the number of regions and using information obtained by separating the background and the object in the background, for example, the region R2 of the image N in FIG. The background 803 and the object 804 shown in FIG.

  In step 912, the drawing unit 15 draws the detection frame 501 on the image so as to surround the object 804 detected by the object detection unit 24, as shown in the image NF in FIG.

  In step 913, the recording unit 16 stores the image NF in which the object detection frame is drawn in the memory 90.

  As described above, the image processing apparatus 2 according to the present exemplary embodiment has the aveR1 blended with the average brightness of the past images up to the image N-1, the minR1 blended with the minimum image brightness, and the brightness of the image N. An image NB corrected for light and dark is created using the average value aveR2 and the minimum brightness value minR2 of the image N, the image NB is divided into a plurality of regions, and a threshold value Thn corresponding to the number of regions is obtained for each region. ing. Therefore, by obtaining the correction value for each threshold value Thn using the threshold value Thn for the number of regions, even when the shadow of the moving vehicle is applied to the background and the object in the background, the background in the image NB and the object in the background Can be separated.

  According to this embodiment, the brightness information of the target image is created using the brightness information of the past image and the target image, and the threshold for separating the background and the object in the background for each region in the image is set. As a result, even when the shadow of the moving vehicle is applied to the background or an object in the background, the object in the image can be detected separately from the background.

  According to each embodiment described above, in any case, an object in the background can be detected more suitably. The above-described embodiments have the following specific effects.

  According to the first embodiment, an image in which the brightness of the target image is emphasized using the brightness information of the past image and the target image is created, and an area in the image is set for each image according to the brightness of the image that changes in time series. A threshold value for separating is obtained. Thereby, even when the brightness of the image in the video changes according to the traveling direction of the moving vehicle, the background in the image and the object in the background can be separated, so that the object in the background can be detected.

  According to the second embodiment, an image in which the brightness of the target image is emphasized using the brightness information of the past image and the target image is created, and the background is set for each region in the image according to the brightness of the image that changes in time series. And threshold values for separating objects in the background are obtained, and images are separated using these threshold values. As a result, even if the brightness of a part of the image changes because the shadow of the moving vehicle is applied to the background or an object in the background, the object in the background is separated from the object in the background. Can be detected.

  Each embodiment described above can be modified as follows.

  The color space conversion unit 12 creates an image obtained by converting the input image into the Lab color space. However, the image may be converted into an image in another color space such as the HSV color space, and in this case, the same effect as described above is obtained. . According to the relationship between the color and brightness between the object to be detected and the background, a conversion method that can obtain color values that can be easily separated can be selected.

  In the object detection unit 24, the region R2 in FIG. 8B is divided into three regions, but may be divided into two regions or four or more regions, and in either case, the same effect as described above is obtained. For example, the region R2 may be divided into the number of lines of the image of the region R2 to obtain threshold values for the number of lines, and the background and the object in the background may be separated using those threshold values. It has the effect of.

  In addition, this invention is not limited to the Example mentioned above, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Further, each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files for realizing each function is a memory, a hard disk drive, a storage device such as an SSD (Solid State Drive), or a computer-readable non-transitory data such as an IC card, an SD card, or a DVD. It can be stored in a storage medium.

  Further, the drawings show control lines and information lines that are considered necessary for explaining the embodiments, and not necessarily all control lines and information lines included in an actual product to which the present invention is applied. Not necessarily. Actually, it may be considered that almost all the components are connected to each other.

DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 10 ... Input part, 11 ... Image correction part, 12 ... Color space conversion part, 13 ... Light / darkness correction part, 14 ... Object detection part, 15 ... Drawing part, 16 ... Recording part, 91 ... Control part DESCRIPTION OF SYMBOLS 2 ... Image processing apparatus, 24 ... Object detection part, 92 ... Control part

Claims (14)

A storage unit for storing a plurality of images photographed by the photographing device;
Based on the representative value of the color value of the image to be processed and the representative value of the color value of one or more past images taken at a time before the time at which the image of the process target was taken, A correction unit that corrects the color value of the image to be processed so as to emphasize light and dark,
A detection unit that detects a detection target object from the processing target image by comparing the corrected color value of the processing target image with a threshold value, and outputs the result. Image processing device. The image processing apparatus according to claim 1,
The representative value of the color value of each image is an average value of the color value of each image,
The correction unit is
Calculating a magnification value between an average color value of the image to be processed and an average color value of the one or more past images;
An image processing apparatus that corrects a color value of each pixel of an image to be processed so that a difference from an average value of color values of the image to be processed is enlarged according to the magnification value. The image processing apparatus according to claim 1,
The detection unit calculates the threshold based on a representative value of color values of the image to be processed and a representative value of color values of the one or more past images. . The image processing apparatus according to claim 3,
The representative value of the color value of each image is a minimum value and an average value of the color value of each image,
The detection unit uses the minimum value of the color value of the one or more past images and the minimum value of the color value of the image to be processed, and sets the minimum value of the color value of the image to be processed before that. A value synthesized with a minimum value of color values of one or more images photographed at a time, an average value of color values of the one or more past images, and an average value of color values of the image to be processed And a value obtained by combining the average value of the color values of the image to be processed with the average value of the color values of one or more images taken at a previous time, and multiplying each by a predetermined coefficient. An image processing apparatus that calculates an added value as the threshold value. The image processing apparatus according to claim 3,
The detector is
Each image is divided into a plurality of regions, and based on the representative value of the color value of each region of the image to be processed and the representative value of the color value of each region of the one or more past images. Calculating the threshold for each region,
An image processing apparatus that outputs the detection result when an object to be detected is successfully detected using the threshold value calculated for any one of the regions. The image processing apparatus according to claim 1,
The representative value of the color value of each image includes at least one of a minimum value, a maximum value, an average value, and a median value of the color values of each image. The image processing apparatus according to claim 1,
The image processing apparatus further includes a color space conversion unit that converts a color space of each image into a Lab color space,
The image processing apparatus according to claim 1, wherein the color value of each image is an a value in the Lab color space. An image processing method executed by an image processing apparatus having a storage unit that stores a plurality of images captured by an imaging device, and a control unit connected to the storage unit,
Based on the representative value of the color value of the image to be processed and the representative value of the color value of one or more past images taken at a time before the time at which the image of the process target was taken, A correction procedure for correcting the color value of the image to be processed so as to emphasize light and dark,
A detection procedure for detecting a detection target object from the processing target image by comparing the corrected color value of the processing target image with a threshold value, and outputting the result. Image processing method. The image processing method according to claim 8, comprising:
The representative value of the color value of each image is an average value of the color value of each image,
The correction procedure is as follows:
A procedure for calculating a magnification value between an average value of color values of the image to be processed and an average value of color values of the one or more past images;
And a procedure for correcting the color value of each pixel of the image to be processed so that the difference from the average value of the color values of the image to be processed is enlarged according to the magnification value. Method. The image processing method according to claim 8, comprising:
The detection procedure includes a procedure of calculating the threshold based on a representative value of color values of the image to be processed and a representative value of color values of the one or more past images. Image processing method. The image processing method according to claim 10, comprising:
The representative value of the color value of each image is a minimum value and an average value of the color value of each image,
The detection procedure uses the minimum value of the color value of the one or more past images and the minimum value of the color value of the image to be processed, and sets the minimum value of the color value of the image to be processed before that. A value synthesized with a minimum value of color values of one or more images photographed at a time, an average value of color values of the one or more past images, and an average value of color values of the image to be processed And a value obtained by combining the average value of the color values of the image to be processed with the average value of the color values of one or more images taken at a previous time, and multiplying each by a predetermined coefficient. An image processing method comprising a step of calculating an added value as the threshold value. The image processing method according to claim 10, comprising:
The detection procedure includes:
Each image is divided into a plurality of regions, and based on the representative value of the color value of each region of the image to be processed and the representative value of the color value of each region of the one or more past images. And calculating the threshold value for each region,
And a procedure of outputting the detection result when the detection target object is successfully detected using the threshold value calculated for any one of the regions. The image processing method according to claim 8, comprising:
The image processing method characterized in that the representative value of the color value of each image includes at least one of a minimum value, a maximum value, an average value, and a median value of the color values of each image. The image processing method according to claim 8, comprising:
A color space conversion procedure for converting the color space of each image into a Lab color space;
The image processing method according to claim 1, wherein the color value of each image is an a value in the Lab color space.