JP2007129277A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor for more surely recognizing a photographing environment thereby carrying out image processing adapted thereto. <P>SOLUTION: A method adopted by the image processor recognizes a position of the sun directly affecting a shadow, recognizes a three-dimensional structure of buildings around a photographing area, obtains the shadow to be formed, specifies an area of an imaged image receiving the effect of the shadow on the basis of the obtained shadow and the photographing area, and can apply shadow correction image processing to the area, and can more properly execute the shadow correction image processing in comparison with a method wherein a shadow correction object area is specified from a photographed image and shadow correction image processing is carried out, or a method wherein shadow correction image processing is applied to the whole photographing image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that obtains environmental conditions when shooting is performed with an imaging apparatus installed outdoors and performs image correction on each captured image according to the shooting environment.

  The background art of the image processing apparatus of the present invention is described in Japanese Patent No. 3359729 and Japanese Patent No. 3550874, and will be described in order.

  First, the background art described in the specification of Japanese Patent No. 3359729 performs vehicle recognition using image processing and image recognition processing on an image input from an imaging device, and the number of vehicles and vehicle speed based on the results. In a traffic flow automatic measurement method for performing at least one vehicle measurement process among vehicle time occupancy, vehicle space occupancy, and various amounts calculated using these values, radiation of an image input from an imaging device The histogram based on the intensity is statistically processed, the histogram based on the radiation intensity after the statistical processing is input to the neural network, and the parameters necessary for automatic traffic flow measurement are output from the neural network, and the vehicle measurement processing is performed using this output value. This is an automatic traffic flow measurement method.

  According to this automatic traffic flow measurement method, which is the background technology, the histogram of the radiation intensity obtained from road images in fine weather, cloudy weather, and rainy weather is statistically processed and given as an input to the neural network. Automatic selection and parameter selection can be realized, and traffic flow measurement can be automated.

  Next, the background art described in Japanese Patent No. 3550874 includes an image input unit that inputs an image captured by an imaging device, a luminance detection unit that calculates a luminance average value of a specific designated area in the image, Using the exposure information of the imaging device as auxiliary information, the day / night determination means for performing day / night determination from the luminance average value of the designated area, and the daytime and nighttime are different based on the determination result by the day / night determination means And a recognizing unit that recognizes an object from the image using an image processing technique.

According to the monitoring device as the background art, determination of day and night is performed from the luminance average value of a specific designated area instead of the entire area of the image, and an object is detected using different image processing techniques day and night based on the determination result. recognize. Therefore, if the designated area is set appropriately, the area other than the designated area will not be subject to day / night judgment, and the influence of the surrounding environment on the object can be reduced, thereby improving the day / night judgment accuracy. The recognition accuracy and monitoring accuracy of an object can be improved. In addition, the aperture of the image pickup device is released near the switching between day and night (early morning and twilight), and the shutter speed becomes the lowest. Using this information reduces erroneous determination when the luminance dispersion value increases during the daytime. be able to.
Japanese Patent No. 3359729 Japanese Patent No. 3550874

  In the automatic traffic flow measurement method described in the specification of Japanese Patent No. 3359729, an image recognition processing algorithm can be selected and parameters can be selected by judging sunny, cloudy, and rainy weather. Is divided into rough classifications such as fine weather, cloudy weather, and rainy weather, and is therefore not sufficient when higher accuracy is required. For example, even in fine weather, a photographed image is received when the photographing environment in which the imaging device is disposed is under backlight conditions, when the photographing environment is in a twilight condition, or when the photographing region is in shadow. The influence is great, and more appropriate image processing is required for such a captured image.

  Further, in the monitoring device described in the specification of Japanese Patent No. 3550874, although it is possible to correctly determine the luminance dispersion value in the daytime based on the aperture and shutter speed of the imaging device, these determinations are performed on the captured image. Depending on the aperture and shutter speed of the image pickup apparatus at the time of shooting, there is a problem that erroneous image processing is performed when these conditions are the same and actual environmental conditions are different. In other words, it is difficult to reliably grasp environmental conditions under these conditions.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus that more reliably grasps a shooting environment and performs image processing that matches the shooting environment.

  (1) An image processing apparatus according to the present invention includes a solar altitude acquisition unit that acquires the altitude of the sun based on the shooting date and time and the latitude and longitude of a shooting location of a target shooting image shot by an imaging device that shots a predetermined shooting area. , A twilight determination means for determining the twilight at the shooting location at the shooting date and time from the altitude of the acquired sun, and image processing for correcting twilight on the photographed image when it is determined that the twilight is determined as twilight And twilight correction image processing means.

  (2) An image processing apparatus according to the present invention includes a sun azimuth obtaining unit that obtains the azimuth of the sun based on the shooting date and time and the latitude and longitude of the shooting location of the target shooting image shot by the imaging device that shots the predetermined shooting area. Back light determination means for performing back light determination of the imaging device from the acquired orientation of the sun and the orientation of the imaging device, and image processing for correcting the back light with respect to the photographed image when it is determined that the back light is determined to be back light. And a backlight correction image processing means to be applied.

  (3) The image processing apparatus according to the present invention acquires the altitude and direction of the sun based on the shooting date and time and the latitude and longitude of the shooting location of the target shooting image shot by the imaging device that shots the predetermined shooting area. An acquisition unit; a three-dimensional information storage unit for surrounding buildings that records, for each imaging device, three-dimensional information about a building around the location where the imaging device is likely to cast a shadow on at least the imaging region of the imaging device; Shadow determination means for determining a shadow to be formed based on the three-dimensional information of the surrounding building of the imaging device and the acquired sun altitude and direction, and determining whether a shadow is formed in the imaging region based on the determined shadow And shadow correction image processing means for performing image processing for correcting the shadow on the area affected by the shadow in the captured image when it is determined that there is an influence of the shadow as a result of the shadow determination.

  (4) The image processing apparatus according to the present invention does not perform determination and / or correction image processing when the illuminance at the shooting location at the shooting date and time is equal to or less than a predetermined threshold, as necessary.

(5) The image processing apparatus according to the present invention is an image of an object photographed by an image pickup apparatus for photographing a vehicle in which a target photographed image passes through a predetermined photographing region as necessary and subjected to correction image processing. Vehicle color recognition means for recognizing the vehicle color from the image is newly provided.
These outlines of the invention do not enumerate the features essential to the present invention, and a sub-combination of these features can also be an invention.

Here, the present invention can be implemented in many different forms. Therefore, it should not be interpreted only by the description of the following embodiment. Also, the same reference numerals are given to the same elements throughout the embodiment.
In the embodiment, the apparatus will be mainly described. However, as is apparent to those skilled in the art, the present invention can also be implemented as a program, system, and method usable on a computer. In addition, the present invention can be implemented in hardware, software, or software and hardware embodiments. The program can be recorded on any computer-readable medium such as a hard disk, CD-ROM, DVD-ROM, optical storage device, or magnetic storage device. Furthermore, the program can be recorded on another computer via a network.

(First embodiment of the present invention)
An image processing apparatus according to a first embodiment of the present invention will be described with reference to the drawings.
[1. System configuration]
FIG. 1 is an explanatory diagram of a shooting environment according to the present embodiment. In the present embodiment, the imaging device 10 is disposed on the side of a road laid in a commercial facility, and the image processing device 200 is disposed in the commercial facility management room. Captured images taken by the imaging devices 10 respectively arranged at a plurality of entrances and exits in a commercial facility are transmitted to the image processing device 200 via a communication path connecting the imaging device 10 and the image processing device 200, and the image processing device 200 Information processing is performed on the received captured image to recognize and output the vehicle color reflected in the captured image. The vehicle colors output from the management computer 30 operated by the administrator are totaled. Thus, the vehicle color totaled by the management computer 30 is used for marketing or the like. In addition, the imaging device 10 is arrange | positioned in the multiple entrance / exit in a commercial facility, and is image | photographing the vehicle which enters in a commercial facility.

FIG. 2 is a system configuration diagram and a hardware configuration diagram of the image processing apparatus according to the present embodiment. The hardware configuration of the image processing apparatus 200 is a configuration of a general computer, a main memory such as a CPU (Central Processing Unit) 210 and a DRAM (Dynamic Random Access Memory) 220, and an HD (hard disk) that is an external storage device. 230, a display 240 as a display device, a keyboard 250 and a mouse 260 as input devices, a LAN card 270 as an expansion card for connecting to a network, a CD-ROM drive 280, and the like. That is, an image processing program stored in a storage medium such as a CD-ROM or an image processing program transmitted from another computer via a network can be copied to the HD 230 of the computer and appropriately read out to the main memory 220. The installation to be performed is executed, and a general computer constitutes the image processing apparatus. Note that the image processing apparatus can be configured using hardware specialized for image processing instead of a general computer. Specifically, the logic on the vehicle discriminating apparatus can be implemented by an ASIC (Application Specific IC), and the image processing apparatus can be configured by a system LSI together with a plurality of LSIs such as a memory.
Similarly to the image processing apparatus 200, the management computer 30 has a hardware configuration similar to that of a general computer. Here, the image processing apparatus 200 and the management computer 30 can be constructed on the same computer.

  FIG. 3 is a block diagram of the image processing computer according to the present embodiment. The image processing computer 200 includes an input unit 211 that captures a photographed image, photographing date / time, illuminance, and photographing location identification information from the imaging device 10, and the solar altitude and the shooting date / time of the photographing location from the longitude / latitude and the photographing date / time of the imaging device. A solar altitude azimuth calculating means 212 for obtaining an azimuth angle, a cloudiness determining means 213 for determining whether or not the shooting location is cloudy from the illuminance at the shooting date and time, and a shooting area at the shooting location from the calculated sun altitude and azimuth at the shooting date and time. , A shadow determination unit 214 that determines whether or not the image is included in the shadow, a backlight determination unit 215 that determines whether or not the imaging device 10 is in the backlight condition at the shooting date and time from the azimuth angle of the sun, and a shooting location at the shooting date and time Based on the determination results of the twilight determination means 216 for determining whether or not the sky is twilight, the cloudiness determination means 213, the shadow determination means 214, the backlight determination means 215, and the twilight determination means 216. Image processing means 217 for performing image processing on the photographed image, vehicle color recognition means 218 for recognizing the vehicle color from the photographed image subjected to image processing as necessary, recognized vehicle color, photographed image after processing, and processing The output unit 219 outputs the previous captured image, the captured date and time, and the captured location identification information to the management computer 30.

[2. Solar altitude bearing calculation]
The altitude and direction of the sun can be obtained from the following equations (1) to (9). Here, λ is the longitude of the location of the imaging device, φ is the latitude of the location, T is a time variable, and K is the number of days that have elapsed since J2000.0 (noon at 1st of January 2000) The ecliptic inclination is ε, the stellar time is Θ, the elapsed time from 0 o'clock is d (a small number of days), the solar red longitude is α, the declination is δ, the solar hour angle is t, Let A be the azimuth angle and h be the altitude of the sun.

  Formulas for obtaining red longitude α and red latitude δ from the longitude longitude λ and longitude latitude β are shown below. Since the sun's longitude is β = 0, the equations (6) and (7) can be derived by using the equations (11) to (13) to obtain the red longitude α and the declination δ.

Further, the solar visual meridian λ _S and the distance r can be obtained by the following equations (14) to (16).

For more information on calculating the solar altitude, see the following book. (Nagasaki Works, “Sunrise / Sunset Calculation”, Jinshokan)
However, the azimuth angle and altitude of the sun can be obtained by a method other than obtaining the azimuth angle and altitude of the sun according to the formula shown here, which is an example. As another example, there is a method of obtaining the azimuth angle and altitude of the sun using a scientific chronology.

[3. Cloudiness judgment]
When the illuminance from the illuminometer 11 is equal to or less than a threshold value (which varies depending on the time) and is daytime, the cloudiness determination unit 213 determines “cloudiness”. There is no need to make a determination by the shadow determination unit 214, the backlight determination unit 215, and the twilight determination unit 216, and no environmental influences on the light (backlight, twilight, building shadow) occur, so that special image correction is performed by the image processing unit 217. The color recognition unit 218 performs the color recognition process without any problem.
On the other hand, when the illuminance is larger than the threshold value, the environmental influence on light is large, and therefore image correction according to the environmental influence is necessary.

  In general, the shadowing condition is when there is one strong light source, and there is a light bias. Therefore, it is preferable to install a plurality of illuminance meters and acquire each illuminance. When all the illuminance differences are within a certain range, there is no illuminance difference, that is, there is no light bias and no shadow is produced. In addition, not only the illuminance at the photographing date and time but also the illuminance before or after the photographing time is measured, and if there is no change in the illuminance, it can be determined that there is no light bias.

“Unit of illuminance ([lx (lux)])” is “amount of luminous flux per unit area of the surface that is exposed to light”. Therefore, it shows how much light reaches the place where the illuminometer is placed. 1 [lx] indicates the illuminance when a light beam of 1 [lm (lumen)] is incident on an area of 1 [m ² ]. Here, “lm” is a unit indicating the amount of light. In addition, there is “cd (candela)” indicating the intensity of light, and “nt (nit) cd / m ² ” indicating the brightness of light reduction / exemption when the light source has a spread. The candela indicates how much light is emitted in a certain direction from the light source. 1 [Nito] is a luminance of 1 [cd] per 1 [m ² ].

[4. Shadow judgment]
When shadows of surrounding buildings appear in the shooting area, the subject (vehicle) is shaded, and the image is taken darker than the original color. Therefore, in order to remove the influence of the shadow of the building, the shadow of the building is calculated.
4 and 5 are explanatory diagrams of shadow determination according to the present embodiment. First, in a three-dimensional coordinate having an xyz axis, the origin is set at the center of the imaging region. It has a table in which each vertex of the building around the camera installation location is converted into a value of (x, y, z) (see FIGS. 4B and 5A). Here, when the structure of the building is complicated, it can be the vertex of a cube that includes the structure.

  In the equation in FIG. Input the altitude of the sun at the shooting date and time obtained in [Calculation of solar altitude direction] and the height of each vertex of the top surface of the building, and obtain the length of the shadow. The opposite direction of the sun's direction is the direction in which the shadow falls, and it relates to each vertex on the top surface of the building corresponding to each vertex in the direction (sun direction +180 [°]) starting from each vertex on the bottom surface of the building The point separated by the length of the shadow becomes the vertex of the shadow. A region surrounded by the vertexes of the shadow becomes a shadow region (see FIGS. 5B to 5D).

The shadow determination means 214 determines whether or not the camera shooting area is included in the shadow area from the shadow area and the camera shooting area thus obtained. When the camera shooting area is included in the shadow area, an area included in the shadow area in the camera shooting area is further specified.
For details on shadow calculation, see the following book. (Nissho Planning Study Group, “Shadow Map Creation Practice and Practice”, Shokokusha)

[5. Twilight judgment]
The twilight determination means 216 is the same as the previous [2. When the solar altitude at the photographing date and time obtained in the calculation of solar altitude direction is -50 [minutes] to -6 [degrees], it is determined that the sky is dusk.

[6. Backlight judgment]
FIG. 6 is an explanatory diagram of the retrograde determination according to the present embodiment. When the backlight determination unit 215 sets 0 [°] with respect to the shooting (camera) direction and the subject (vehicle) as a reference, the description [2. The relative angle can be obtained from the azimuth angle of the sun at the shooting date and time obtained in [Solar altitude azimuth calculation]. When this relative angle is within a predetermined angle range, the backlight condition is met. As shown in FIG. 6, it can also be classified into backlight, forward light, and irradiation from the lateral direction. The backlight determining means 215 determines whether or not the backlight is backlit.
Since the positional relationship between the sun and the camera is known, it is not only simple to determine whether the backlight is backlit, but not only to perform image processing in the case of backlighting conditions, but also to determine how much backlighting is in accordance with it By performing image processing, it is possible to recover the actual vehicle color with higher accuracy.

[7. Image processing]
[7.1 Image processing during shadow determination]
When the calculated building shadow portion overlaps the center coordinate periphery (camera shooting region), the density correction of the overlapping region is performed. Here, when density correction is performed, the correction intensity is changed according to the illuminance. When the illuminance is high, shadows tend to appear dark, and it is necessary to increase the intensity of correction.

[7.2 Image Processing for Backlight Determination]
In order to make the dark part easy to see, correction is performed to increase the brightness of the dark part. For the correction, a process for uniformly increasing the brightness of the entire image may be used, or a process for increasing only the dark part (for example, a histogram process) may be used.
The image processing means 217 applies a filter that increases the luminance of the vehicle body portion according to the intensity of the backlight. When the backlight intensity is high, the correction intensity is also increased. When the backlight intensity is weak, the correction intensity is set as a standard.
There are various well-known techniques for the brightening correction, and those skilled in the art can adopt various well-known techniques, and detailed description thereof is omitted here. The histogram processing is detailed in the following book. (Mikio Takagi, “Image Analysis Handbook”, The University of Tokyo Press)

[7.3 Image processing during twilight determination]
As image processing in twilight, noise removal processing and brightness correction processing are performed.
At twilight time, the illuminance of the object to be photographed is insufficient, and noise may be mixed into the image due to insufficient illuminance. Therefore, noise removal processing is performed to remove the noise. For noise removal, an image processing filter such as a median filter or blur can be used.

In addition, brightness correction processing is performed to compensate for the lack of illuminance during twilight. As a process for making it easy to see while maintaining the overall balance of the image, for example, a histogram averaging process or a process of simply increasing the brightness can be used.
Here, in the case of backlight and twilight, it is difficult to be backlit because the amount of light is small, and the influence of twilight is large, so it is possible to cope with only the image processing for twilight without performing image processing for backlight. You can also.

  There are various well-known techniques for noise removal processing and brightness correction, and those skilled in the art can adopt various well-known techniques, and detailed description thereof is omitted here. The blur filter is detailed in the following book. (Mikio Takagi, “Image Analysis Handbook”, The University of Tokyo Press) The median filter is well-known in the following publications. (TS Huang, GY Yang, and GY Tang. “IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL. ASP-27, NO. 1”, FEBRUARY 1979)

FIG. 7 is an explanatory diagram of city coefficients and twilight definitions according to this embodiment. As shown in FIG. 7A, a city coefficient is determined for each rectangular area. By performing image processing for twilight correction based on the city coefficient, a captured image with higher accuracy can be obtained. In the present embodiment, the image processing unit 217 determines that the twilight is determined to be twilight by the twilight determination unit 215, performs redness correction when the shooting time is morning and the city coefficient is large, and when the shooting time is morning and the city coefficient is small. Blueness correction is performed. When the shooting time is evening and the city coefficient is large, redness correction is performed strongly, and when the shooting time is evening and the city coefficient is small, redness correction of the initial value is performed. FIG. 7B shows the types of twilight.

[7.4 Auto Gain Control]
FIG. 8 is an explanatory diagram of correction processing when the automatic gain control is OFF according to the present embodiment. When auto gain control (AGC) is set to OFF on the imaging apparatus 10 side, the image is biased toward high luminance and low luminance. Therefore, correction is possible by stretching as follows.
Usually, in the imaging apparatus 10, an auto gain control function and a white balance function are mounted as a circuit.

  The auto gain control function is a function for performing correction so that the height of the envelope of the analog signal photographed by the CCD camera is within a certain range. In the case of a high brightness image, the brightness of the whole image is lowered, and in the case of a low brightness image, the brightness of the whole image is raised. The power of the output signal is controlled on the mounting of the CCD camera.

White balance (AWB) is a function that automatically corrects the difference in appearance depending on the color temperature of the light source. Reproduces the appropriate white color under any light. That is, the RGB values are adjusted so that the color of the central portion of the screen at the time of shooting is mixed to become white.
Through this auto gain control function and white balance function, an analog signal (composite signal: a signal obtained by combining luminance and color signals) is output to the A / D converter, and finally output from the imaging device 10 as JPEG. Is done.

[8. Operation]
FIG. 9 is an overall flowchart of the image processing apparatus according to this embodiment. FIG. 10 is a detailed flowchart of the image processing apparatus according to this embodiment. FIG. 11 is a flowchart of auto gain of the image processing apparatus according to this embodiment.
The latitude and longitude of each imaging device are recorded in the latitude / longitude table together with the camera spot number serving as shooting location identification information (see FIG. 1B).
The processor 210 (input unit 211) captures a set of captured images, corresponding shooting dates and times, corresponding illuminance, and corresponding shooting location identification information.

The processor 210 determines whether the captured image is a gray scale image (step 100). The imaging device 10 captures a photographed image in color during the daytime and a grayscale image using infrared light at night. In the case of a gray scale image, it is difficult to recognize an accurate color by color recognition described later, and processing for specifying a candidate color is performed.
In order to speed up the subsequent processing, the processor 210 reduces the image size of the captured image (step 200). Thus, the subsequent processing is speeded up by reducing the image size of the captured image.

The processor 210 grasps the shooting environment and performs image processing (step 300). Since the processing of the processor 210 here includes an essential part of the present invention, details will be described later.
The processor 210 performs HSI conversion on the RGB captured image (step 400). Conversion is performed because it is easier for the user to grasp the color when the hue value is displayed than the RGB value as an output when the color is recognized.
The processor 210 designates the vehicle body shooting position (step 500). The processor 210 performs color recognition (step 600).

Hereinafter, the image processing in step 300 will be described in detail with reference to FIG.
The shooting location identification information indicating the shooting date and the shooting location is a part of the file name of the shot image file, and the processor 210 acquires the shooting date and shooting location identification information from the file name (step 301).
The processor 210 (solar altitude direction calculation means 212) acquires the longitude and latitude of the target imaging device 10 from the latitude / longitude table based on the shooting location identification information. [2. The altitude and azimuth angle of the sun are obtained by the calculation method described in [Solar altitude azimuth calculation] (step 311).
The processor 210 (cloudiness determination means 213) is the same as that described in [3. Cloudiness determination] is performed (step 321). If it is not cloudy, that is, if it is determined that there is no shadow, the process ends and proceeds to step 400.

  If it is determined in step 321 that there is a shadow, the processor 210 (shadow determination means 214) determines that the previous [4. As described in [Shadow determination], the shadow of the building is calculated, and it is determined whether or not the camera shooting area is included in the shadow area (step 331). If it is determined that the image is included in the shadow area, the processor 210 (image processing means 217) executes the image processing described in the previous section [7.1 Image processing during shadow determination] (step 332).

  After it is determined in step 331 that it is not included in the shadow area, or after step 332, the processor 210 (the twilight determination means 216) determines whether or not it is within the range of the twilight threshold (step 341). ). Before this step 341, a step of adjusting a threshold value of altitude at which twilight falls from terrain information can be inserted.

  If it is determined that it is within the range of the twilight threshold, the correction coefficient used for image processing for twilight correction is adjusted using the city coefficient of the corresponding region (step 351). The processor 210 determines whether or not the shooting time is in the early morning time zone (step 352). If it is determined that the shooting time is in the early morning, the processor performs the dawn correction processing in the previous section [7.3 Image processing at twilight determination]. 210 (image processing means 217) executes (step 353). When it is determined that it is not early morning but dusk, the processor 210 (image processing means 217) executes the dusk correction process described in the previous section [7.3 Image processing during twilight determination] (step 354).

If it is determined in step 341 that it is not within the twilight threshold range, the processor 210 (backlight determination means 215) determines the difference from the camera orientation and the sun orientation (step 361), and the backlight orientation, It is determined whether or not the altitude is high (step 362). If it is determined that the azimuth and the altitude of the backlight are not, the process ends and proceeds to step 400. If it is determined that the heading is in the backward direction and the altitude, the processor 210 (image processing means 217) executes the image processing in the previous [7.2 Image processing during backlight determination] (step 363). Here, in step 362, up to the altitude of the sun is used as an element for determining the backlight, but it is also possible to perform the backward determination using only the direction of the sun.
After step 353, step 354, step 362, and step 363 are completed, the process proceeds to step 400.

The vehicle color recognized by the processor 210 (output means 219), the captured image after image processing, the shooting date and time, and the shooting location identification information are output to the management computer 30. In the management computer 30, these vehicle colors are totaled in the respective colors, and data including the vehicle color and the number of vehicles is updated. Here, it is also possible to configure a database having as attributes the photographed image before image processing, the photographed image after image processing, the photographing date and time, photographing location identification information, illuminance, and vehicle color. In particular, the captured image before image processing and the captured image after image processing can be compared and displayed on the display of the management computer 30.

[10. Effects of this embodiment]
As described above, according to the image processing apparatus according to the present embodiment, the altitude and azimuth angle of the sun are obtained, the environmental conditions (backlight, shadow, dusk) are grasped more accurately than in the past, and the respective environments are determined. Since the vehicle color is recognized after image processing that matches the conditions, the vehicle color can be recognized more accurately.

(Other embodiments)
[Auto gain and white balance]
In the first embodiment, when auto gain and white balance are not performed by the imaging apparatus 10, auto gain control correction processing and white balance correction processing can be executed, respectively.

  For example, processing can be incorporated into the first embodiment as follows. The processing of the next steps 701 to 704 can be inserted before, for example, step 301 in FIG. The processor 210 receives a flag indicating whether or not auto gain control is being performed from the imaging apparatus 10 and determines whether or not the setting is ON (see FIG. 11; step 701). If the setting is OFF, auto gain control is performed. Correction processing is executed (step 702), and if the setting is on, nothing is done.

  Similarly, the processor 210 receives a flag indicating whether or not the white balance is being executed from the imaging apparatus 10 and determines whether or not the setting is ON (step 703). If the setting is OFF, the white balance correction process is performed. (Step 704). If the setting is on, nothing is done.

[Shadow correction image processing]
Shadow correction is applied to the captured image for the part that overlaps the shadow in the shooting area, but more specifically, since the vehicle is three-dimensional, if the shadow border is on the vehicle, the shadow also affects that effect. Since the boundary between the shadows is not linear but has a slight bend and is substantially straight, an image with higher accuracy can be obtained by considering the bend in the image processing for shadow correction. Specifically, in the case of a vehicle, a method of calculating the slight bend by detecting the edge of the vehicle, and also from a specified shadow region and at least one of RGB value, hue, brightness, and saturation. A method can be adopted in which the shadow region is specified again more accurately and image processing for shadow correction is performed thereafter.
Further, regarding the vehicle color recognition, it is possible to recognize the vehicle color more accurately by recognizing the part of the vehicle color that avoids the shadow area. In this case, it is not necessary to perform image processing for shadow correction.

[Cloudiness judgment]
In the first embodiment, whether or not it is “cloudy” is determined by whether or not it is equal to or less than a threshold value, and if it is determined to be “cloudy” by the cloudiness determination, subsequent shadow determination, twilight determination, and backlight determination are performed. In addition, the vehicle color is recognized without performing correction image processing, but it is determined whether the illuminance is equal to or higher than the first threshold value. Is determined to be greater than or equal to a second threshold, and if it is greater than or equal to the second threshold, twilight determination is performed. If the illuminance is greater than or equal to the third threshold, determination is made to perform retrograde determination. It can also be configured. That is, in detail, shadow, twilight, and backlight that affect a captured image have different illuminance threshold values, and more accurate processing can be performed by providing an illuminance threshold value for each.

  Although the present invention has been described with the above embodiments, the technical scope of the present invention is not limited to the scope described in the embodiments, and various modifications or improvements can be added to these embodiments. . And embodiment which added such a change or improvement is also contained in the technical scope of the present invention. This is apparent from the claims and the means for solving the problems.

[Appendix] The following appendices are further disclosed with respect to the embodiment.
(Appendix 1) Solar altitude acquisition means for acquiring the altitude of the sun based on the shooting date and time and the latitude and longitude of the shooting location of the target shooting image shot by the imaging device that shoots the predetermined shooting area, and shooting from the acquired sun altitude Twilight judgment means for carrying out twilight judgment at the shooting place during the day,
An image processing apparatus comprising twilight correction image processing means for performing image processing for correcting twilight on a captured image when it is determined that twilight is twilight as a result of twilight determination.

  Therefore, do not perform twilight correction image processing when it is estimated that the twilight is about from the image, but only perform twilight correction image processing when it is determined that twilight is after determining the position of the sun that directly affects twilight. Therefore, it is possible to execute twilight correction image processing in the case of twilight accurately without misunderstanding of twilight.

In particular, the illuminance meter can measure the illuminance at the shooting time, and the twilight determination can be performed only when the illuminance result causes twilight.
Further, by combining the method for estimating twilight from a conventional photographed image and the method for determining twilight from the position of the sun according to the present invention, processing can be performed more accurately without using an illuminometer.
Furthermore, it is possible to adopt a configuration in which only the date and time at which twilight may occur is first extracted using the weather information, and twilight determination is performed after the extraction. On the contrary, it is possible to adopt a configuration in which the image processing for twilight correction is canceled when no twilight occurs using the weather information after twilight determination.

“Dusk” means “dusk with twilight”, but in the present invention, it is used in a state of faint brightness before sunrise or after sunset. In other words, it is used not only at dusk. In other cases, it may be preferable to use the word “twilight” indicating “subtle brightness of the sky seen before sunrise or after sunset”. In any case, the sun's light below the horizon is scattered by the influence of the upper atmosphere or airborne fines and affects the captured image. Yes.
In the present invention, shooting is used uniformly, and the term “imaging” is used only for the imaging device, but the term “imaging” may be used instead of “shooting”.

    (Supplementary Note 2) Solar azimuth acquisition means for acquiring the azimuth of the sun based on the shooting date and time and the latitude and longitude of the shooting location of the target photographic image shot by the imaging device that shoots the predetermined shooting area, and the acquired sun azimuth and imaging A backlight determination unit that performs backlight determination of the imaging device from the orientation of the device, and a backlight correction image processing unit that performs image processing for correcting backlight on the captured image when it is determined as backlight as a result of the backlight determination. An image processing apparatus.

Therefore, the position of the sun that directly affects the backlight is grasped, it is accurately grasped whether it is the backlight from the azimuth angle of the camera, and when it is determined that it is the backlight, the image processing of the backlight correction is performed. Backlight correction image processing can be executed in the case of accurate backlighting without mistaking the backlighting.

    (Supplementary Note 3) Solar altitude direction acquisition means for acquiring the altitude and direction of the sun based on the shooting date and time and the latitude and longitude of the shooting location of the target shooting image shot by the imaging device that shots the predetermined shooting area, and at least the imaging device A peripheral building 3D information storage means for recording, for each imaging device, 3D information of a building around the location where the imaging device is likely to cast a shadow in the imaging area, and 3 of the peripheral building of the target imaging device Shadow determination means for determining a shadow to be formed based on the dimensional information and the acquired altitude and direction of the sun, and determining whether a shadow is to be formed in the shooting area based on the calculated shadow; An image processing apparatus comprising: a shadow correction image processing unit that performs image processing for correcting a shadow on a region affected by a shadow in a captured image when it is determined that there is an influence.

  Therefore, the position of the sun that directly affects the shadow is grasped, the three-dimensional structure of the building around the photographing region is grasped, the shadow formed is obtained, and the photographed image is affected by the shadow from the obtained shadow and the photographing region. A region that is shadowed and image processing for shadow correction can be performed on the region, and a method for performing shadow correction image processing by identifying a target region for shadow correction from the captured image or a shadow on the entire captured image. Compared with the method of performing image processing for correction, it is possible to execute image processing for shadow correction more appropriately.

(Supplementary Note 4) The image processing apparatus according to any one of Supplementary Notes 1 to 3, wherein the image processing for determination and / or correction is not performed when the illuminance at the photographing location on the photographing date and time is equal to or less than a predetermined threshold.
Therefore, if the illuminance at the shooting location at the shooting date and time is less than or equal to a predetermined value, no twilight, backlight or shadow will occur, and in such a case, unnecessary processing is performed by not performing the image processing for each determination and each correction. Can be prevented.

(Supplementary Note 5) Vehicle color recognition means for recognizing a vehicle color from a target captured image that is an image captured by an imaging device that captures a vehicle in which the target captured image passes through a predetermined capturing area. The image processing apparatus according to any one of appendices 1 to 4, which is newly provided.
Therefore, by applying the present invention, it is possible to obtain an appropriate vehicle color using a captured image showing an accurate vehicle color.

(Additional remark 6) The solar altitude direction acquisition means which acquires the altitude and direction of the sun based on the shooting date and time and the latitude and longitude of the shooting location of the target shooting image shot by the imaging device for shooting the predetermined shooting area, Peripheral building 3D information storage means for recording for each imaging device 3D information of the building around the location where the imaging device is likely to cast a shadow on the imaging area of the imaging device, and peripheral construction of the target imaging device A shadow determination means for determining a shadow to be formed based on the three-dimensional information of the object and the acquired altitude and direction of the sun, and performing a shadow determination as to whether a shadow is formed in the photographing region based on the calculated shadow; An image processing program for functioning as a shadow correction image processing unit that performs image processing for correcting a shadow on a region affected by a shadow in a captured image when it is determined that the result has a shadow influence.
The image processing apparatus can also be understood as an image processing program.

(Supplementary Note 7) An imaging apparatus that captures a predetermined imaging area, and an image processing apparatus that performs image processing on a target captured image captured by the imaging apparatus, wherein the image processing apparatus captures a shooting date and a shooting location of the target captured image. The solar altitude direction acquisition means for acquiring the altitude and direction of the sun based on the latitude and longitude of the image, and at least three-dimensional information of the buildings around the location of the imaging device that may cast a shadow on the imaging area of the imaging device. Peripheral building 3D information storage means to record for each imaging device, 3D information of the surrounding building of the target imaging device and the shadow formed based on the acquired altitude and direction of the sun are obtained, and shooting is performed based on the obtained shadow Shadow determination means for performing a shadow determination as to whether a shadow is formed in the area, and correcting the shadow for the area affected by the shadow in the captured image when it is determined that there is an influence of the shadow as a result of the shadow determination Image processing A vehicle color recognition system comprising: a shadow-corrected image processing unit that performs the above-described processing; and a vehicle color recognition unit that recognizes the vehicle color from the target captured image subjected to the image processing to be corrected.
The image processing apparatus can also be regarded as a system configured with other components.

It is explanatory drawing of the imaging environment which concerns on the 1st Embodiment of this invention. 1 is a system configuration diagram and a hardware configuration diagram of an image processing apparatus according to a first embodiment of the present invention; It is a block block diagram of the image processing computer which concerns on the 1st Embodiment of this invention. It is explanatory drawing of the shadow determination which concerns on embodiment which concerns on the 1st Embodiment of this invention. It is explanatory drawing of the shadow determination which concerns on embodiment which concerns on the 1st Embodiment of this invention. It is explanatory drawing of the retrograde determination which concerns on the 1st Embodiment of this invention. It is explanatory drawing of the city coefficient and twilight definition which concern on the 1st Embodiment of this invention. It is explanatory drawing of the correction process at the time of the auto gain control OFF concerning the 1st Embodiment of this invention. 1 is an overall flowchart of an image processing apparatus according to a first embodiment of the present invention. It is a detailed flowchart of the image processing apparatus which concerns on the 1st Embodiment of this invention. 3 is a flowchart of auto gain of the image processing apparatus according to the first embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Imaging device 11 Illuminance meter 30 Management computer 200 Image processing device 210 CPU (processor)
211 Input means 212 Solar altitude direction calculation means 213 Cloudiness determination means 214 Shadow determination means 215 Backlight determination means 216 Dusk determination means 217 Image processing means 218 Color recognition means 219 Output means 220 DRAM
230 HD
240 display 250 keyboard 260 mouse 270 LAN card 280 CD-ROM drive

Claims (5)

Solar altitude acquisition means for acquiring the altitude of the sun based on the shooting date and time and the latitude and longitude of the shooting location of the target shooting image shot by the imaging device shooting the predetermined shooting area;
Twilight judgment means for carrying out twilight judgment at the shooting location from the acquired sun altitude at the shooting date and time,
An image processing apparatus comprising twilight correction image processing means for performing image processing for correcting twilight on a captured image when it is determined that twilight is twilight as a result of twilight determination. Solar azimuth obtaining means for obtaining the azimuth of the sun based on the photographing date and time and the latitude and longitude of the photographing location of the target photographing image photographed by the imaging device photographing the predetermined photographing region;
Back light determination means for performing back light determination of the imaging device from the acquired orientation of the sun and the orientation of the imaging device;
An image processing apparatus comprising: a backlight correction image processing unit that performs image processing for correcting backlight on a captured image when it is determined as backlight as a result of backlight determination. Solar altitude azimuth acquisition means for acquiring the altitude and azimuth of the sun based on the shooting date and time and the latitude and longitude of the shooting location of the target photographic image shot by the imaging device that shoots the predetermined shooting area;
Peripheral building 3D information storage means for recording for each imaging device 3D information of the building around the location where the imaging device is likely to cast shadows at least in the imaging area of the imaging device;
Shadow determination for obtaining a shadow formed based on the three-dimensional information of the surrounding building of the target imaging device and the acquired altitude and direction of the sun, and performing a shadow determination as to whether a shadow is formed in the imaging region based on the obtained shadow Means,
An image processing apparatus comprising: a shadow correction image processing unit that performs image processing for correcting a shadow on a region affected by a shadow in a captured image when it is determined that there is a shadow influence as a result of the shadow determination. The image processing apparatus according to any one of claims 1 to 3, wherein the image processing for determination and / or correction is not performed when the illuminance at the shooting location on the shooting date and time is equal to or less than a predetermined threshold. The target captured image is an image captured by an imaging device that captures a vehicle passing through a predetermined capturing area,
The image processing apparatus according to claim 1, further comprising vehicle color recognition means for recognizing a vehicle color from a target captured image that has undergone image processing to be corrected.

Images