JP4483067B2 - Target object extraction image processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a target object extraction image processing apparatus.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a target object extraction image processing apparatus for extracting a target object from an image generally uses a characteristic that an object close to an illumination unit becomes brighter than an object far from the illumination unit (Japanese Patent Laid-Open No. Hei 9). -259278).
[0003]
FIG. 2 is a diagram showing the positional relationship between the background and the target object in the conventional target object extraction image processing apparatus.
[0004]
As shown in the figure, the target object 11 is disposed at a position closer to the imaging unit 13 than the background 12, and the illumination unit 14 is turned on to illuminate the target object 11 and the background 12, and the imaging near the illumination unit 14. The target object 11 and the background 12 are photographed by the unit 13, and the target object 11 is extracted by comparing the brightness of the target object 11 illuminated by the illumination unit 14 with the brightness of the background 12 illuminated by the illumination unit 14. Like to do.
[0005]
Therefore, for example, the illumination image obtained by lighting the illumination unit 14 and photographing the target object 11 and the background 12 and the illumination image obtained by extinguishing the illumination unit 14 and photographing the target object 11 and the background 12 are obtained. On the image in which the brightness changes greatly between the state where the illumination unit 14 is turned off and the state where it is turned on by performing difference / threshold processing including difference processing and threshold value processing based on the unilluminated image. Are extracted as the target object 11.
[0006]
[Problems to be solved by the invention]
However, in the conventional target object extraction image processing device, the control method of the target object extraction image processing device, the threshold set in the difference / threshold processing, and the method of extracting the target object 11 after the difference / threshold processing is performed Thus, the target object 11 cannot always be extracted stably and at high speed in any environment.
[0007]
When the target object 11 is photographed with a certain illumination intensity, when the target object 11 is far from the imaging unit 13, the target object 11 appears dark, and when the target object 11 is close to the imaging unit 13, the target object 11 is It will appear bright. Therefore, when the difference / threshold processing is performed, for example, when the target object 11 appears dark, it becomes impossible to extract a part or all of the target object 11.
[0008]
Therefore, the intensity of illumination is controlled based on the result obtained by performing the difference / threshold processing between the illumination image and the non-illumination image.
[0009]
However, when such control is performed, if the intensity of illumination is inappropriate, it is necessary to perform illumination image and non-illumination image capture and difference / threshold processing at least twice before extracting the target object 11. , Processing time will be long.
[0010]
Further, when the target object 11 is close to the background 12, the background 12 is also brightened to some extent when the illumination unit 14 is turned on. Therefore, if an appropriate threshold is not set in the difference / threshold processing, the target object 11 can be correctly extracted. Can not. For example, when the threshold value is low, a brightly changed background portion is also extracted as the target object 11, while when the threshold value is high, a part or all of the target object 11 cannot be extracted.
[0011]
An object of the present invention is to solve the problems of the conventional target object extraction image processing device and to provide a target object extraction image processing device which can always extract a target object stably and at high speed. And
[0012]
[Means for Solving the Problems]
Therefore, in the target object extraction image processing apparatus of the present invention, an illumination unit that illuminates the target object and the background, an imaging unit that captures the target object and the background, and a distance from the imaging unit to the target object are measured. The distance measurement unit, the optimum illumination parameter recording means for recording the optimum illumination intensity when the target object and the background are illuminated for each distance from the imaging unit to the target object, and the distance measurement unit The optimum illumination intensity is read from the optimum illumination parameter recording means based on the measured distance, and the illumination unit illuminates the target object and the background with the optimum illumination intensity, or the illumination intensity lower than the optimum illumination intensity. Illuminate the target object and background with the illumination control unit that illuminates the target object and the background or turns off the illumination unit, and images of the target object and the background illuminated with the optimal illumination intensity Shooting that records a certain illumination image and an unilluminated image, which is an image obtained by photographing the target object and the background with the illumination unit turned off with an illumination intensity lower than the optimum illumination intensity, or with the illumination unit turned off. A control unit, an optimum threshold value calculation processing unit that calculates an optimum threshold value based on a difference between a background luminance value in the illumination image and a background luminance value in the non-illuminated image, and records the optimum threshold value recording unit; and the illumination image A difference process for generating a difference image between the unilluminated image and the difference / threshold processing unit for performing a threshold process for generating an image of the target object candidate region by comparing the difference image with the optimum threshold; A target object extraction unit that extracts a target object based on an image of the target object candidate region and a model of the target object.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0014]
FIG. 1 is a block diagram of a target object extraction image processing apparatus according to the first embodiment of the present invention.
[0015]
In the figure, 10 is a target object extraction image processing device, 11 is a target object, 12 is a background, 201 is an illumination unit that illuminates the target object 11 and the background 12, 202 is disposed near the illumination unit 201, and An imaging unit that captures the target object 11 and the background 12, 203 a distance measuring unit that measures the distance from the imaging unit 202 to the target object 11, and 21 controls the intensity (brightness) of illumination of the illumination unit 201. An illumination control unit 22 is an imaging control unit that controls the imaging unit 202, and 23 is a ranging control unit that controls the ranging unit 203. The illumination control unit 21, the imaging control unit 22, and the distance measurement control unit 23 constitute a device control unit 204.
[0016]
Reference numeral 205 denotes a distance recording unit that records the distance from the imaging unit 202 measured by the distance measuring unit 203 to the target object 11, and 206 denotes an illumination that records the intensity of illumination controlled by the illumination control unit 21. A parameter recording unit 207 is an image recording unit that records at least two images captured by the imaging unit 202.
[0017]
208 is based on the image recorded in the image recording unit 207, the intensity of illumination recorded in the illumination parameter recording unit 206, and the optimum threshold value read from the optimum threshold value database 25 as the optimum threshold value recording means. A difference / threshold processing unit 209 for performing difference / threshold processing, and a target object 209 based on the result of the difference / threshold processing by the difference / threshold processing unit 208 and the model of the target object 11 read from the target object model database 210 11 is a target object extraction unit that extracts the extraction result and outputs the extraction result. The target object model database 210 records model data of the target object 11 to be extracted.
[0018]
Reference numeral 211 denotes an optimum parameter database. The optimum parameter database 211 serves as optimum illumination parameter recording means for recording optimum illumination intensity as a first optimum parameter with respect to the distance from the imaging unit 202 to the target object 11. And the optimum threshold value database 25 in which an optimum threshold value as a second optimum parameter for each illumination parameter used in the difference / threshold processing unit 208 is recorded. The optimum threshold value is sent to the control unit 21 to the difference / threshold processing unit 208. Reference numeral 212 denotes an optimum parameter calculation unit that calculates the optimum illumination intensity and optimum threshold value, and reference numeral 213 denotes a warning output unit that outputs a warning when an abnormal value is detected in the optimum parameter calculation unit 212.
[0019]
Next, the operation of the target object extraction image processing apparatus 10 having the above configuration will be described.
[0020]
The operation of the target object extraction image processing apparatus 10 according to the present embodiment includes two phases (processing flow): a parameter data setting phase and a target object extraction phase.
[0021]
The parameter data setting phase is for setting two databases that record optimum illumination intensity and optimum threshold value, and needs to be performed before the target object extraction phase.
[0022]
The target object extraction phase is for extracting the target object 11 from the image, and is a main phase in the target object extraction image processing apparatus 10. The parameter data setting phase is not necessarily performed when the optimum parameter database 211 that has already been set for the imaging environment and the target object 11 can be used.
[0023]
In the target object extraction phase, first, the distance from the imaging unit 202 to the target object 11 is measured by the distance measuring unit 203, and the measured distance is recorded in the distance recording unit 205. Next, the illumination control unit 21 reads the optimum illumination intensity from the optimum illumination parameter database 24 based on the distance recorded in the distance recording unit 205, and turns on the illumination unit 210 with the optimum illumination intensity. Increase the intensity of lighting. The illumination control unit 21 records the optimal illumination intensity at that time in the illumination parameter recording unit 206. Note that the target object 11 can be prevented from being erroneously detected by the optimal illumination intensity, and the target object 11 can be extracted stably.
[0024]
Next, the illumination unit 201 is turned on with the optimum illumination intensity, the target object 11 and the background 12 are photographed by the imaging unit 202, and the photographed illumination images of the target object 11 and the background 12 are recorded in the image recording unit 207. . Immediately after shooting the illumination image, the illumination unit 201 is turned off to reduce the intensity of illumination, the imaging unit 202 captures the target object 11 and the background 12, and the captured target object 11 and background 12 are captured. An unilluminated image is recorded in the image recording unit 207. Subsequently, the illumination image and the non-illumination image recorded in the image recording unit 207 are sent to the difference / threshold processing unit 208, and a candidate area for extracting the target object 11 (hereinafter referred to as “target object candidate area”). Is set. Therefore, the difference / threshold processing unit 208 performs difference / threshold processing based on the illumination image, the non-illumination image, the intensity of illumination, and the optimum threshold.
[0025]
In the present embodiment, the illumination intensity is reduced by turning off the illumination unit 201 when taking a non-illuminated image. However, the illumination image is taken without turning off the illumination unit 201. It is also possible to lower the intensity of illumination than when doing so. However, the larger the difference between the intensity of illumination when capturing an illumination image and the intensity of illumination when capturing an unilluminated image, the better.
[0026]
Next, the basic principle for extracting the target object 11 will be described.
[0027]
FIG. 3 is a diagram illustrating an example of an unilluminated image according to the first embodiment of the present invention, FIG. 4 is a diagram illustrating an example of an illuminated image according to the first embodiment of the present invention, and FIG. FIG. 6 is a diagram showing an example of a difference image in the first embodiment, FIG. 6 is a diagram showing a luminance histogram of the difference image in the first embodiment of the present invention, and FIG. 7 is an object in the first embodiment of the present invention. The figure which shows the 1st example of an object candidate area | region, FIG. 8 is a figure which shows the 2nd example of the target object candidate area | region in the 1st Embodiment of this invention, FIG. 9 is 1st Embodiment of this invention FIG. 10 is a diagram showing a third example of a target object candidate region in FIG. 10, FIG. 10 is a diagram showing a background image when an object with high reflectance exists in the background in the first embodiment of the present invention, and FIG. The figure which shows the illumination image in case a target object exists in front of the background in 1st Embodiment of this, 12 is a diagram showing a fourth example of the target object candidate area in the first embodiment of the present invention. In FIG. 6, the horizontal axis represents the luminance difference and the vertical axis represents the frequency.
[0028]
If the illumination unit 201 is turned on when the target object 11 (FIG. 1) is located away from the background 12 and close to the imaging unit 202, the object near the illumination unit 201 is generally brighter than the background 12. The non-illuminated image is as shown in FIG. 3, and the illuminated image is as shown in FIG.
[0029]
Then, by performing the difference processing of the difference / threshold processing, and subtracting the brightness of the non-illuminated image from the brightness of the illumination image, a difference image as shown in FIG. 5 is generated. It is represented by a luminance histogram as shown in FIG. In FIG. 6, the part A mainly represents the area of the background 12, and the part B mainly represents the area of the target object 11. In this case, the luminance change between the lighting portion 201 and the background portion 12 that is far from the lighting unit 201 is small, and the luminance change between the lighting unit 201 and the target object 11 that is close to the lighting unit 201 is large. Become.
Therefore, when threshold processing of difference / threshold processing is performed with respect to the difference image with a predetermined threshold, images as shown in FIGS. 7 to 9 are obtained, and regions that have changed to white are the target object candidate regions. It becomes. Then, when threshold processing is performed with an appropriate threshold T1, the target object candidate region is formed as a silhouette of the target object 11 as shown in FIG. 7, so that the target object 11 can be easily extracted. it can. On the other hand, when threshold processing is performed with the threshold T2 (<T1), a target object candidate region including the background 12 is formed as shown in FIG. 8, and the threshold processing is performed with the threshold T3 (> T1). If this is done, as shown in FIG. 9, a target object candidate region lacking the target object 11 is formed, which makes it difficult to extract the target object 11.
[0030]
Therefore, in the present embodiment, an optimum threshold value such as the threshold value T1 is recorded in advance in the optimum threshold value database 25 in association with the intensity of each illumination, and when the threshold value processing is performed, the optimum threshold value database 25 An optimum threshold value corresponding to the illumination intensity at that time is determined. Therefore, an optimal target object candidate region can be formed in any environment. Note that the optimum threshold value database 25 used in threshold value processing is set in the parameter data setting phase.
[0031]
By the way, by performing the processing as described above, the target object candidate region as shown in FIG. 7 may be formed as a silhouette of the target object 11 as it is. When there is an object having a higher reflectance than the object 11, the target object candidate region is not necessarily formed as a silhouette of the target object 11. For example, as shown in FIG. 10, there is a window glass as a highly reflective object in the background 12, and when the target object 11 is a person standing in front of the window glass as shown in FIG. When the part 201 is turned on, the background 12 becomes brighter than the target object 11. As a result, when the difference / threshold processing is performed on the illumination image and the non-illumination image as described above, a target object candidate region as shown in FIG. 12 is formed, and the window glass of the background 12 is erroneously set as the target object 11. It will be extracted.
[0032]
Therefore, the target object extraction unit 209 extracts the target object 11 from the target object candidate area.
[0033]
The method of extracting the target object 11 from the target object candidate area in the target object extraction unit 209 differs depending on the type of the target object 11 to be extracted, but the basic processing flow is the same. That is, the target object extraction unit 209 is preliminarily assumed for the target object candidate region formed by the difference / threshold processing unit 208 and is based on the model information of the target object 11 recorded in the target object model database 210. Further, the target object 11 is extracted based on the distance from the imaging unit 202 to the target object 11 as necessary. For example, when a target object candidate region as shown in FIG. 12 is formed, the target object 11 is extracted by fitting a model of a human shape. Further, a face model and a body model are prepared in advance as models of the target object 11, and the face model is extracted from the target object candidate area as shown in FIG. 12, and the extracted face is extracted. By fitting a body model from the size and position of the body, the body size and position can be estimated to extract the body. And a body can also be extracted using the target object candidate area | region shown by FIG. 12, and the illumination image of FIG. 11 (refer Japanese Patent Application No. 11-149281 specification).
[0034]
As described above, in the present embodiment, the optimal illumination unit 201 is controlled based on the distance from the imaging unit 202 to the target object 11, and the optimum threshold value in the difference / threshold processing is determined based on the illumination intensity. Finally, since the target object 11 is extracted using the model of the target object 11, the target object 11 can be extracted stably and at high speed in any environment.
[0035]
Next, the parameter data setting phase will be described.
[0036]
The parameter data setting phase is for setting the optimum illumination intensity and the optimum threshold used in the target object extraction phase.
[0037]
FIG. 13 is a diagram showing a luminance histogram of a difference image when the target object is close to the background in the first embodiment of the present invention, and FIG. 14 is a case where the illumination intensity is low in the first embodiment of the present invention. FIG. 15 is a diagram showing a luminance histogram of the difference image, and FIG. 15 is a diagram showing a luminance histogram of the difference image when the illumination intensity is high in the first embodiment of the present invention. 13 to 15, the horizontal axis represents the luminance difference and the vertical axis represents the frequency. 13 to 15, the part A mainly represents the area of the background 12 (FIG. 1), and the part B mainly represents the area of the target object 11.
[0038]
First, the optimum illumination control method and the creation method of the optimum illumination parameter database 24 will be described.
[0039]
In the target object extraction image processing apparatus 10, when the target object 11 is far from the illumination unit 201 and close to the background 12, when the target object 11 and the background 12 are illuminated with a constant illumination intensity, an illumination image and an unilluminated image are displayed. In the luminance histogram of the difference image, the part A and the part B overlap as shown in FIG. In this case, if threshold processing is performed, a part of the target object 11 is missing even if the optimum threshold T1 is set, and a target object candidate area including a part of the background 12 is formed. If threshold values T2 and T3 are set, a target object candidate area in which the target object 11 is further missing or the background 12 is included is formed. Therefore, the processing load of the target object extraction unit 209 increases and the processing time becomes long.
[0040]
Therefore, an optimal illumination control method for optimally controlling the illumination unit 201 will be described.
[0041]
In general, in the case of an object whose surface is diffusely reflected, when the illumination unit 201 is turned on and the object is photographed, the brightness of the object becomes brighter in proportion to the intensity of illumination, and is the square of the distance from the illumination unit 201. Darkens inversely. Therefore, unless the reflectance of the background 12 is much higher than the reflectance of the target object 11 (for example, a mirror or the like), if the target object 11 and the background 12 are photographed with varying illumination intensity, the illumination image is not displayed. The luminance histogram of the difference image from the illumination image is as shown in FIG. 14 when the illumination intensity is low, and as shown in FIG. 15 when the illumination intensity is high.
[0042]
That is, when the intensity of illumination is increased, the illumination image is generally brightened, but the object closer to the illumination unit 201 than the amount of the background 12 that is far from the illumination unit 201 becomes brighter due to the diffuse reflection characteristics as described above. The part where the part of the object 11 becomes brighter becomes larger. Therefore, since the luminance histogram of the difference image between the illumination image and the non-illumination image is such that the portion A and the portion B are separated as shown in FIG. 15, the target object 11 can be easily extracted by threshold processing. Can do. Thus, it becomes easier to extract the target object 11 as the intensity of illumination is higher.
[0043]
However, if the intensity of the illumination is excessively high, the luminance of the target object 11 to be extracted is deteriorated due to saturation of the luminance. Therefore, the intensity of illumination of the illumination unit 201 is controlled so that the luminance is not saturated. It is preferable to do. That is, the illumination unit 201 is optimally controlled so that the target object 11 is illuminated most brightly so that the luminance is not saturated at any distance from the imaging unit 202.
[0044]
Therefore, in the present embodiment, the optimal illumination parameter database 24 is set at each distance so that the target object 11 can be illuminated with an appropriate illumination intensity according to the distance of the target object 11 from the imaging unit 202. The corresponding optimum illumination intensity is recorded.
[0045]
Next, a process for creating the optimum illumination parameter database 24 will be described.
[0046]
FIG. 16 is a flowchart showing the operation of the optimum illumination parameter database creation processing in the first embodiment of the present invention, and FIG. 17 is an example of the measured values of the distance and the optimum illumination intensity in the first embodiment of the present invention. FIG. 18 is a diagram showing an example of the optimum illumination parameter database in the first embodiment of the present invention.
[0047]
First, a test chart having properties equivalent to those of the target object 11 is arranged at a predetermined position in front of the imaging unit 202 (FIG. 1). When the target object 11 and the test chart are photographed, the luminances of the photographed target object 11 and the test chart are substantially equal. In the present embodiment, the target object 11 is configured by a test chart in the process of creating the optimum illumination parameter database 24.
[0048]
Next, the distance measurement control unit 23 performs distance recording processing, measures the distance from the imaging unit 202 to the test chart by the distance measurement unit 203, and records the measured distance in the distance recording unit 205. Subsequently, the illumination control unit 21 performs an illumination control process, illuminates the test chart with the illumination unit 201 at a predetermined illumination intensity, records the illumination intensity at that time in the illumination parameter recording unit 206 as an illumination intensity, The imaging control unit 22 performs image capturing / recording processing, images a test chart by the imaging unit 202 for each illumination intensity, and records an image of the captured test chart in the image recording unit 207.
[0049]
Then, when the illumination control unit 21 performs the illumination control process for all the illumination intensities set in advance, and the imaging control unit 22 performs the image capturing / recording process for each illumination intensity. Optimal illumination parameter calculation processing means (not shown) of the optimal parameter calculation unit 212 performs optimal illumination parameter determination processing, and calculates image feature amounts (average luminance, maximum luminance, etc.) for all images recorded in the image recording unit 207. The calculated image feature value takes the most appropriate value, and the test chart selects an image that is brightest and bright enough not to saturate the luminance.
[0050]
Further, the optimum illumination parameter calculation processing means performs optimum illumination parameter recording processing, calculates the optimum illumination intensity by reading the illumination intensity corresponding to the selected image from the illumination parameter recording unit 206, and The distance when the image is taken is read from the distance recording unit 205, and the optimum illumination intensity and distance are recorded in the optimum illumination parameter database 24.
[0051]
These processes are repeated by changing the position where the test chart is arranged, and the actual measured values of the distance and the optimal illumination intensity at each position as shown in FIG. 17 can be obtained.
[0052]
Subsequently, the optimum parameter calculation unit 212 performs an interpolation process, interpolates the measured values of the distance and the optimum illumination intensity, and finally optimizes corresponding to the distance as shown in FIG. An optimum illumination parameter database 24 in which the intensity of various illuminations is recorded is created.
[0053]
Next, a flowchart will be described.
Step S1 A test chart is arranged.
Step S2 A distance recording process is performed.
Step S3: Perform illumination control processing.
Step S4 Image photographing / recording processing is performed.
Step S5: It is determined whether or not the illumination control process and the image capturing / recording process have been completed for all illumination intensities. If the illumination control process and the image capturing / recording process have been completed for all illumination intensities, the process proceeds to step S6. If not completed, the process returns to step S3.
Step S6: Optimal illumination parameter determination processing is performed.
Step S7: Perform the optimum illumination parameter recording process and return to Step S1.
[0054]
Next, a method for creating the optimum threshold value and optimum threshold value database 25 will be described.
[0055]
FIG. 19 is a diagram showing a luminance histogram of a difference image of threshold processing in the first embodiment of the present invention, FIG. 20 is a first diagram showing results of threshold processing in the first embodiment of the present invention, and FIG. FIG. 22 is a second diagram showing the threshold processing result in the first embodiment of the present invention, and FIG. 22 is a third diagram showing the threshold processing result in the first embodiment of the present invention. In FIG. 19, the horizontal axis represents the luminance difference and the vertical axis represents the frequency. In FIG. 19, the part A mainly represents the area of the background 12 (FIG. 1), and the part B mainly represents the area of the target object 11.
[0056]
First, the optimum threshold for performing the difference / threshold processing will be described. For example, when the luminance histogram of the difference image between the illumination image and the non-illumination image shown in FIG. 19 is obtained, the threshold values T11, T22, and T33 are optimum, depending on the target object extraction method. Then, when threshold processing is performed with the threshold T11, the target object candidate region shown in FIG. 20 performs threshold processing with the threshold T22, and when the target object candidate region illustrated in FIG. 21 performs threshold processing with the threshold T33. The target object candidate region shown in FIG. 22 is formed. The target object candidate area shown in FIG. 21 is substantially equal to the target object candidate area when the luminance histogram shown in FIG. 6 is obtained.
[0057]
In this case, when threshold processing is performed with the threshold T22, as shown in FIG. 21, the portion of the background 12 is hardly included, and the portion of the target object 11 is substantially included. On the other hand, when threshold processing is performed with the threshold T11, the portion of the target object 11 is not missing as shown in FIG. 20, but the portion of the background 12 is relatively large. Further, when threshold processing is performed with the threshold T33, the background 12 portion is hardly included as shown in FIG. 22, but the target object 11 portion is slightly chipped.
[0058]
Here, in the extraction process of the target object extraction unit 209, even when the background 12 portion is slightly included, if it is preferable that the target object 11 is less missing, the threshold T11 is the optimum threshold. On the contrary, even if the target object 11 is slightly missing, if it is better not to include the portion of the background 12, the threshold T33 is an optimal threshold, the target object 11 is less missing, and the background If it is better not to include the 12 portion, the threshold T22 is the optimum threshold.
[0059]
In this way, the optimum threshold value can extract most of the target object 11 portion as the threshold value T11, can remove most of the background 12 portion as the threshold value T33, In addition, there is a case where the background 12 is small and most of the target object 11 can be extracted as in the threshold T22.
[0060]
By the way, the luminance histogram of the difference image as shown in FIG. 19 changes as shown in FIGS. 14 and 15 when the illumination intensity is changed corresponding to the position of the target object 11 as described above. . Therefore, it is necessary to determine an optimum threshold value based on the difference image between the illumination image and the non-illumination image for the background 12 and the target object 11 when the illumination intensity is changed.
[0061]
Next, the operation of the optimum threshold database creation process will be described.
[0062]
FIG. 23 is a flowchart showing the operation of the first optimum threshold database creation processing in the first embodiment of the present invention, and FIG. 24 is the operation of the second optimum threshold database creation processing in the first embodiment of the present invention. FIG. 25 is a flowchart showing the operation of the third optimum threshold value database creating process in the first embodiment of the present invention, and FIG. 26 is the optimum illumination intensity and the first embodiment of the present invention. FIG. 27 is a diagram illustrating an example of an actual measurement value of the optimum threshold value, and FIG. 27 is a diagram illustrating an example of an optimum threshold value database according to the first embodiment of the present invention.
[0063]
In this case, FIG. 23 shows a processing procedure when the optimum threshold value database 25 (FIG. 1) is created with the threshold value T11 shown in FIG. 19 as the optimum threshold value, and FIG. 24 shows the optimum threshold value database 25 with the threshold value T33 as the optimum threshold value. FIG. 25 shows a processing procedure when the optimum threshold value database 25 is created with the threshold value T22 as an optimum threshold value.
[0064]
First, when the optimum threshold value database 25 is created with the threshold value T11 as an optimum threshold value, the illumination control unit 21 performs the first illumination control process, turns off the illumination unit 201, and the imaging control unit 22 uses the unilluminated background. Image capturing / recording processing is performed, the background 12 without the target object 11 and the like is captured by the imaging unit 202, and the captured image of the background 12 is recorded in the image recording unit 207 as an unilluminated background image.
[0065]
Next, the illumination control unit 21 performs a second illumination control process, reads a predetermined optimum illumination intensity recorded in the optimum illumination parameter database 24, and lights the illumination unit 201 with the optimum illumination intensity. Then, the imaging unit 202 performs illumination background image photographing / recording processing, photographs the background 12, and records the photographed image of the background 12 in the image recording unit 207 as the illumination background image. Subsequently, an optimum threshold value calculation processing unit (not shown) of the optimum parameter calculation unit 212 performs an optimum threshold value calculation process, performs a difference process between the non-illuminated background image and the illuminated background image recorded in the image recording unit 207, and luminance Create a histogram. Since the luminance histogram generally has a distribution as shown in the portion A in FIG. 19, the threshold value T11 is determined based on characteristics in the luminance histogram distribution, for example, an average value.
[0066]
Subsequently, the optimum threshold value calculation processing means performs an optimum threshold value recording process, and records the threshold value T11 as an optimum threshold value in the optimum threshold value database 25 together with the predetermined optimum illumination intensity.
[0067]
In this way, the respective processes are performed for all optimum illumination intensities recorded in the optimum illumination parameter database 24, the optimum threshold value is calculated for each optimum illumination intensity, and the optimum illumination intensity and By recording the optimum threshold value in the optimum threshold value database 25, it is possible to obtain the optimum illumination intensity and the measured value of the optimum threshold value as shown in FIG.
[0068]
Subsequently, the optimum parameter calculation unit 212 performs an interpolation process, interpolates the measured values of the optimum illumination intensity and the optimum threshold value, and finally each optimum illumination as shown in FIG. An optimum threshold value database 25 in which optimum threshold values are recorded corresponding to the intensity is created.
[0069]
Next, a flowchart will be described.
Step S11 A first illumination control process is performed.
Step S12 A non-illuminated background image photographing / recording process is performed.
Step S13 A second illumination control process is performed.
Step S14 The illumination background image photographing / recording process is performed.
Step S15: An optimum threshold value calculation process is performed.
Step S16: Perform the optimum threshold value recording process, and return to Step S11.
[0070]
Next, when creating the optimum threshold value database 25 using the threshold value T33 as an optimum threshold value, the device control unit 204 first reads one of the distances recorded in the optimum illumination parameter database 24. Then, the operator places a test chart at the position of the distance in front of the imaging unit 202.
[0071]
Subsequently, the illumination control unit 21 performs a first illumination control process, turns off the illumination unit 201, the imaging control unit 22 performs a non-illuminated image capturing / recording process, and the test chart is captured by the imaging unit 202. Then, the photographed image of the test chart is recorded in the image recording unit 207 as a non-illuminated image.
[0072]
Next, the illumination control unit 21 performs a second illumination control process, reads the optimum illumination intensity corresponding to the distance recorded in the optimum illumination parameter database 24, and uses the optimum illumination intensity to illuminate the illumination unit. 201 is turned on, and the imaging unit 202 performs illumination image capturing / recording processing, captures a test chart, and records an image of the captured test chart in the image recording unit 207 as an illumination image. Subsequently, the optimum threshold value calculation processing means performs an optimum threshold value calculation process, performs a difference process between the non-illuminated image recorded in the image recording unit 207 and the illumination image, and creates a luminance histogram only for the test chart portion. To do. As described above, since the luminance histogram has a distribution as shown in the portion B in FIG. 19, the threshold value T33 is determined based on characteristics in the distribution of the luminance histogram, such as an average value.
[0073]
Subsequently, the optimum threshold value calculation processing means performs optimum threshold value recording processing and records the threshold value T33 in the optimum threshold value database 25 as the optimum threshold value together with the predetermined optimum illumination intensity.
[0074]
In this way, test charts are placed at all the distances recorded in the optimum illumination parameter database 24, and optimum threshold values are calculated for each distance and each optimum illumination intensity. By recording the threshold value in the optimum threshold value database 25, it is possible to obtain the optimum illumination intensity and the actual measured value of the optimum threshold value similar to those shown in FIG.
[0075]
Subsequently, the optimum parameter calculation unit 212 performs an interpolation process, interpolates each optimum illumination intensity and optimum measured value of the optimum threshold value, and finally, an optimum threshold value database similar to that shown in FIG. 25 is created.
[0076]
Next, a flowchart will be described.
Step S21 A test chart is arranged.
Step S22 A first illumination control process is performed.
Step S23 An unilluminated image photographing / recording process is performed.
Step S24: A second illumination control process is performed.
Step S25 The illumination image photographing / recording process is performed.
Step S26: An optimal threshold value calculation process is performed.
Step S27: Perform the optimum threshold value recording process and return to Step S21.
[0077]
When creating the optimum threshold value database 25 using the threshold value T22 as the optimum threshold value, the illumination control unit 21 first performs the first illumination control process and turns off the illumination unit 201. In addition, the imaging control unit 22 performs non-illuminated background image capturing / recording processing, the background 12 without the target object 11 or the like is captured by the imaging unit 202, and the captured image of the background 12 is recorded as an unilluminated background image. Recorded in the unit 207.
[0078]
Next, the illumination control unit 21 performs a second illumination control process, reads a predetermined optimum illumination intensity recorded in the optimum illumination parameter database 24, and lights the illumination unit 201 with the optimum illumination intensity. Then, the imaging unit 202 performs illumination background image photographing / recording processing, photographs the background 12, and records the photographed image of the background 12 in the image recording unit 207 as the illumination background image.
[0079]
Subsequently, the device control unit 204 reads a distance corresponding to the optimum illumination intensity recorded in the optimum illumination parameter database 24. Then, the operator places a test chart at the position of the distance in front of the imaging unit 202.
[0080]
Next, the illumination control unit 21 performs a third illumination control process, turns off the illumination unit 201, the imaging control unit 22 performs a non-illuminated image capturing / recording process, and the test chart is captured by the imaging unit 202. Then, the photographed image of the test chart is recorded in the image recording unit 207 as a non-illuminated image.
[0081]
Subsequently, the illumination control unit 21 performs a fourth illumination control process, turns on the illumination unit 201 with the optimal illumination intensity read in the second illumination control process, and the imaging unit 202 The photographing / recording process is performed, the test chart is photographed, and an image of the photographed test chart is recorded in the image recording unit 207 as an illumination image.
[0082]
The optimum threshold value calculation processing means performs an optimum threshold value calculation process, and includes an unilluminated background image recorded in the unilluminated background image capturing / recording process, an illuminated background image recorded in the illuminated background image capturing / recording process, An optimum threshold value is calculated based on the non-illuminated image recorded in the illumination image capturing / recording process and the illumination image recorded in the illumination image capturing / recording process. For example, the distribution of the luminance histogram of the difference image between the unilluminated background image and the illumination background image indicated by the portion A in FIG. 19 and the luminance histogram of the difference image between the unilluminated image and the illumination image indicated by the portion B in FIG. Threshold value T22 is determined based on each feature of the distribution (for example, the Mahalanobis distance of the distribution of the two luminance histograms).
[0083]
Subsequently, the optimum threshold value calculation processing means performs optimum threshold value recording processing, and records the threshold value T22 in the optimum threshold value database 25 as the optimum threshold value together with the predetermined optimum illumination intensity.
[0084]
In this way, test charts are placed at all the distances recorded in the optimum illumination parameter database 24, and optimum threshold values are calculated for each distance and each optimum illumination intensity. By recording the threshold value in the optimum threshold value database 25, it is possible to obtain the optimum illumination intensity and the actual measured value of the optimum threshold value similar to those shown in FIG.
[0085]
Subsequently, the optimum parameter calculation unit 212 performs an interpolation process, interpolates each optimum illumination intensity and optimum measured value of the optimum threshold value, and finally, an optimum threshold value database similar to that shown in FIG. 25 is created.
[0086]
As described above, the optimum threshold value calculation processing unit is configured to obtain an optimum threshold value based on an image obtained by illuminating at least one of the unilluminated background image and the unilluminated image, and at least one of the illuminated background image and the illuminated image. Is calculated.
[0087]
By the way, the optimum threshold value calculated in the first to third optimum threshold value database creation processes is an abnormal value and may be inappropriate for extracting the target object 11. For example, if the optimum threshold is too small (very close to 0 in 256 gradations), the shooting environment is extremely bright, and there is no difference in luminance between when the illumination unit 201 is turned on and when it is turned off. If the optimum threshold is too large (very close to 255 in 256 gradations), the distance from the imaging unit 202 to the background 12 is extremely short.
[0088]
In such a case, since the target object candidate area cannot be formed, it is assumed that the optimum threshold value is an abnormal value, and the warning output unit 213 outputs a warning that the optimum threshold value is inappropriate.
[0089]
In the present embodiment, the optimum threshold value is obtained for all optimum illumination intensities recorded in the optimum illumination parameter database 24. However, a predetermined optimum of the optimum illumination intensities is obtained. It is also possible to create an optimum threshold value database 25 by obtaining an optimum threshold value for the intensity of illumination and performing an interpolation process for the obtained optimum threshold value.
[0090]
As described above, in the present embodiment, since the optimum illumination intensity can be obtained in correspondence with the distance from the imaging unit 202 to the target object 11, the illumination intensity can be made appropriate. .
[0091]
Then, since the optimum threshold value is calculated and set corresponding to each optimum illumination intensity, it is necessary to repeat the photographing of the illumination image and the non-illuminated image and the difference / threshold processing until the target object 11 is extracted. Disappear. Therefore, since the processing time can be shortened, the target object 11 can be extracted at high speed.
[0092]
In addition, since the optimum threshold value is calculated and set corresponding to each optimum illumination intensity, the target object 11 can always be extracted stably and correctly.
[0093]
Since the target object 11 is finally extracted using the model of the target object 11, for example, in any environment, for example, there is an object having a higher reflectance than the target object 11 in the background 12 Even in this case, the target object 11 can be extracted stably and at high speed.
[0094]
Furthermore, when the optimum threshold value is inappropriate, a warning is output, so that the target object 11 can be correctly extracted.
[0095]
Next, a flowchart will be described.
Step S31 A first illumination control process is performed.
Step S32 A non-illuminated background image photographing / recording process is performed.
Step S33 A second illumination control process is performed.
Step S34 An illumination background image is captured and recorded.
Step S35 A test chart is arranged.
Step S36: A third illumination control process is performed.
Step S37 An unilluminated image photographing / recording process is performed.
Step S38: A fourth illumination control process is performed.
Step S39 The illumination image photographing / recording process is performed.
Step S40: Optimal threshold calculation processing is performed.
Step S41 An optimum threshold value recording process is performed, and the process returns to Step S31.
[0096]
Next, a second embodiment of the present invention will be described. In addition, about the thing which has the same structure as 1st Embodiment, the description is abbreviate | omitted by providing the same code | symbol.
[0097]
FIG. 28 is a block diagram of a target object extraction image processing apparatus according to the second embodiment of the present invention.
[0098]
In this case, for example, the target object 11 is at a substantially constant distance from the imaging unit 202 as in the case of extracting a person in a car. Since the position of the target object 11 is constant, the target object extraction image processing apparatus 100 does not include a distance measurement unit, a distance measurement control unit, and a distance recording unit. Therefore, the cost of the target object extraction image processing apparatus 100 can be reduced.
[0099]
In addition, since the position of the target object 11 is constant, the optimum illumination parameter database 24 records only one optimum illumination intensity, not a change in distance. Along with this, one optimum threshold value is recorded in the optimum threshold value database 25 in correspondence with one optimum illumination intensity.
[0100]
By the way, for example, in the first and second embodiments, if the shooting environment changes greatly due to sunlight or the like in the target object extraction phase, the illumination image becomes too bright or too dark. 11 is too bright or too dark, and the image quality of the target object 11 is degraded.
[0101]
Therefore, a third embodiment in which the image quality of the image of the target object 11 can be checked will be described. In addition, about the thing which has the same structure as 1st Embodiment, the description is abbreviate | omitted by providing the same code | symbol.
[0102]
FIG. 29 is a block diagram of the target object extraction image processing apparatus according to the third embodiment of the present invention, FIG. 30 is a diagram showing an example of the optimum illumination parameter database according to the third embodiment of the present invention, and FIG. It is a figure which shows the example of the optimal threshold value database in the 3rd Embodiment of invention.
[0103]
In the figure, reference numeral 200 denotes a target object extracted image processing apparatus, and 414 denotes an extracted object image quality check unit. The image quality of the extracted image of the target object 11 is checked so that a good image can be obtained.
[0104]
In the present embodiment, the extracted object image quality check unit 414 checks the image quality of the image of the target object 11 with the brightness. Therefore, the extracted object image quality check unit 414 calculates the average luminance in a predetermined area of the image of the target object 11, and determines whether the image quality is appropriate depending on whether the average luminance is within a preset range. Judge whether. If the average luminance does not fall within the range and the illumination image is too bright or too dark and the image quality is not appropriate, the extracted object image quality check unit 414 sends an instruction to the imaging control unit 42, and the imaging control unit 42 is controlled. The imaging control unit 42 changes the imaging unit parameters (camera gain, etc.), captures the non-illuminated image and the illuminated image again, and then performs the target object extraction phase to extract the target object 11. At this time, the extracted object image quality check unit 414 also sends an instruction to the illumination control unit 21 to control the illumination control unit 21. The illumination control unit 21 changes the intensity of illumination.
[0105]
Note that since the image quality changes when the imaging unit parameters are changed, the optimal illumination intensity and the optimal threshold value also change. Therefore, the optimum illumination parameter database 24 is represented by the distance, the image pickup unit parameter, and the optimum illumination intensity as shown in FIG. 30, and the optimum threshold beta base 25 is the optimum illumination as shown in FIG. Represented by the intensity, imaging unit parameters, and optimum threshold.
[0106]
Thus, in the present embodiment, even if the imaging environment changes greatly, the imaging unit parameters are changed, so that the target object 11 can be correctly extracted.
[0107]
In the present embodiment, as in the second embodiment, the distance measurement unit, the distance measurement control unit, and the distance recording unit may be omitted.
[0108]
In each of the embodiments, visible light is used in the illumination unit 201, but light in a specific wavelength region such as infrared light other than visible light can also be used. In that case, if the imaging unit 202 can perform imaging only with light in the wavelength range used by the illumination unit 201, the imaging unit 202 is less susceptible to environmental light in the imaging environment and can stably extract the target object 11. can do.
[0109]
In each of the above embodiments, the illumination image is captured after the non-illuminated image is captured. However, the non-illuminated image can also be captured after the illumination image is captured. In any case, if the target object 11 changes its posture, position, etc., when taking an unilluminated image and when taking an illuminated image, a difference image between the unilluminated image and the illuminated image can be accurately obtained. Therefore, it is preferable that the interval between the timing for capturing the non-illuminated image and the timing for capturing the illuminated image is shorter.
[0110]
When there are two or more target objects 11 to be extracted, in the luminance histogram distribution, one background 12 region as shown by the portion A in FIG. 19 is formed, whereas FIG. Two or more regions of the target object 11 as shown by the part B in FIG. Therefore, when setting the optimum threshold value in the parameter data setting phase, the threshold value is set in the area of the background 12 as in the case where the threshold value T11 is set in the first embodiment, It is preferable to set a threshold value in a region where the luminance difference is the smallest.
[0111]
In addition, this invention is not limited to the said embodiment, It can change variously based on the meaning of this invention, and does not exclude them from the scope of the present invention.
[0112]
【The invention's effect】
As described above in detail, according to the present invention, in the target object extraction image processing apparatus, the illumination unit that illuminates the target object and the background, the imaging unit that captures the target object and the background, and the imaging unit A distance measuring unit that measures the distance to the target object, and an optimum illumination parameter recording unit that records the optimal illumination intensity when the target object and the background are illuminated for each distance from the imaging unit to the target object; Reading the optimum illumination intensity from the optimum illumination parameter recording means based on the distance measured by the distance measuring unit, and illuminating the target object and the background with the optimum illumination intensity by the illumination unit, or Illuminate the target object and the background with an illumination intensity lower than the intensity of the illumination, or turn off the illumination unit, and the target object illuminated with the optimal illumination intensity by the illumination unit And an illuminated image that is an image of the background, and an unilluminated image that is an image obtained by photographing the target object and the background with the illumination unit turned off with an illumination intensity lower than the optimum illumination intensity. An optimum threshold value is calculated based on the difference between the background luminance in the illumination image and the background luminance in the non-illuminated image, and recorded in the optimum threshold value recording means. The difference which performs the threshold value process which produces | generates the image of a target object candidate area | region by comparing a difference means and the said optimal threshold value with the difference process which produces | generates the difference image of the said process means, and the said illumination image and a non-illuminated image A threshold processing unit, and a target object extraction unit that extracts a target object based on an image of the target object candidate region and a model of the target object.
[0113]
In this case, the optimum threshold value is calculated based on the difference between the luminance of the illumination image obtained by photographing the background and the luminance of the non-illuminated image.
[0114]
Accordingly, since the optimum threshold value is calculated in correspondence with the optimum illumination intensity, it is not necessary to repeat the shooting of the illumination image and the non-illuminated image, the difference process, and the threshold process until the target object is extracted. Can be shortened. As a result, the target object can be extracted at high speed.
[0115]
In addition, since the optimum threshold value is calculated corresponding to each optimum illumination intensity, the target object can always be stably and correctly extracted.
[Brief description of the drawings]
FIG. 1 is a block diagram of a target object extraction image processing apparatus according to a first embodiment of the present invention.
FIG. 2 is a diagram illustrating a positional relationship between a background and a target object in a conventional target object extraction image processing apparatus.
FIG. 3 is a diagram illustrating an example of an unilluminated image according to the first embodiment of the present invention.
FIG. 4 is a diagram showing an example of an illumination image in the first embodiment of the present invention.
FIG. 5 is a diagram showing an example of a difference image in the first embodiment of the present invention.
FIG. 6 is a diagram showing a luminance histogram of a difference image in the first embodiment of the present invention.
FIG. 7 is a diagram showing a first example of a target object candidate region in the first embodiment of the present invention.
FIG. 8 is a diagram illustrating a second example of the target object candidate region in the first embodiment of the present invention.
FIG. 9 is a diagram showing a third example of the target object candidate region in the first embodiment of the present invention.
FIG. 10 is a diagram showing a background image when an object having a high reflectance exists in the background according to the first embodiment of the present invention.
FIG. 11 is a diagram showing an illumination image when a target object is present in front of a background in the first embodiment of the present invention.
FIG. 12 is a diagram showing a fourth example of a target object candidate area in the first embodiment of the present invention.
FIG. 13 is a diagram showing a luminance histogram of a difference image when the target object is close to the background in the first embodiment of the present invention.
FIG. 14 is a diagram showing a luminance histogram of a difference image when the illumination intensity is low according to the first embodiment of the present invention.
FIG. 15 is a diagram showing a luminance histogram of a difference image when the intensity of illumination is high according to the first embodiment of the present invention.
FIG. 16 is a flowchart showing an operation of optimum illumination parameter database creation processing in the first embodiment of the present invention.
FIG. 17 is a diagram showing an example of measured values of distance and optimum illumination intensity according to the first embodiment of the present invention.
FIG. 18 is a diagram showing an example of an optimum illumination parameter database according to the first embodiment of the present invention.
FIG. 19 is a diagram showing a luminance histogram of a difference image in threshold processing according to the first embodiment of the present invention.
FIG. 20 is a first diagram illustrating a threshold processing result according to the first embodiment of this invention.
FIG. 21 is a second diagram showing a threshold processing result according to the first embodiment of the present invention.
FIG. 22 is a third diagram showing a threshold processing result according to the first embodiment of the present invention.
FIG. 23 is a flowchart showing an operation of first optimal threshold value database creation processing in the first exemplary embodiment of the present invention.
FIG. 24 is a flowchart showing the operation of second optimal threshold value database creation processing in the first exemplary embodiment of the present invention.
FIG. 25 is a flowchart showing the operation of a third optimum threshold database creation process in the first embodiment of the present invention.
FIG. 26 is a diagram showing an example of the optimum illumination intensity and the measured value of the optimum threshold value in the first embodiment of the present invention.
FIG. 27 is a diagram showing an example of an optimum threshold database according to the first embodiment of the present invention.
FIG. 28 is a block diagram of a target object extraction image processing apparatus according to a second embodiment of the present invention.
FIG. 29 is a block diagram of a target object extraction image processing apparatus according to a third embodiment of the present invention.
FIG. 30 is a diagram showing an example of an optimum illumination parameter database according to the third embodiment of the present invention.
FIG. 31 is a diagram showing an example of an optimum threshold database according to the third embodiment of the present invention.
[Explanation of symbols]
10, 100, 200 Target object extraction image processing apparatus
11 Target object
12 Background
24 Optimal lighting parameter database
25 Optimal threshold database
201 Lighting section
202 Imaging unit
203 Ranging unit
205 Distance recording unit
207 Image recording unit
208 Difference / threshold processing unit
209 Target object extraction unit
212 Optimal parameter calculator
213 Warning output section
414 Extracted object image quality check section

Claims (3)

(A) an illumination portion that illuminates the Target object and the background,
(B) an imaging unit that captures the target object and the background;
(C) a distance measuring unit that measures a distance from the imaging unit to the target object;
(D) Optimal illumination parameter recording means in which the optimal illumination intensity when illuminating the target object and the background is recorded for each distance from the imaging unit to the target object ;
(E) reading the optimum illumination intensity from the optimum illumination parameter recording means based on the distance measured by the distance measuring unit, and illuminating the target object and the background with the optimum illumination intensity by the illumination unit, or An illumination control unit that illuminates the target object and the background with an illumination intensity lower than the optimal illumination intensity, or turns off the illumination unit;
(F) An illumination image that is an image of the target object and background that is illuminated by the illumination unit with the optimum illumination intensity, and the illumination unit illuminates with an illumination intensity lower than the optimum illumination intensity, or the illumination unit is turned off. A shooting control unit that records an unilluminated image that is an image of the target object and the background in the image recording unit;
(G) an optimum threshold value calculation processing unit that calculates an optimum threshold value based on a difference between a background luminance in the illumination image and a background luminance in the non-illuminated image, and records the optimum threshold value in the optimum threshold value recording unit ;
(H) A difference process for generating a difference image between the illumination image and the non-illuminated image , and a threshold process for generating an image of the target object candidate region by comparing the difference image with the optimum threshold value. A threshold processing unit;
(I) A target object extraction image processing apparatus comprising: a target object extraction unit that extracts a target object based on an image of the target object candidate region and a model of the target object . The front Symbol optimum threshold recording means, the object extracting apparatus according to claim 1, optimum threshold to correspond to the intensity of the optimal illumination is recorded. 3. The target object extraction according to claim 1, further comprising an extracted object image quality check unit that checks whether or not the image quality of the extracted target object is appropriate and controls the imaging unit and the illumination unit when the image quality is not appropriate. Image processing device .

Images