JP2010226652A - Image processing apparatus, image processing method, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology which accurately determines by image processing whether a picked-up image is an image taken against the sun or not. <P>SOLUTION: An image processing apparatus 200 obtains a picked-up image, presumes the picked-up direction of the pickup image, obtains the information on the sun direction to specify the direction of the sun, presumes the direction of the sun from a picked-up position of the picked-up image based on the information of the sun direction, presumes whether the picked-up image is an image taken against the sun or not based on the presumed pickup direction and direction of the sun, decides an area of the picked-up image to be used correspondent to the presumed result of the presumption processing, and determines whether the picked-up image is the image taken against the sun or not based on a pixel value of the pixel of the decided area. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and a computer program.

  It is possible to display information on facilities and roads on the ground surface on the display of electronic devices, including technology that provides route guidance to cars and pedestrians using computerized map data that can be used on computers. Widely done. In order to improve the quality of map data, it is desired to accurately and sufficiently describe road attribute information such as traffic regulations in the map data. In order to enhance road attribute information quickly and accurately, a technique for performing image processing on an image taken on a road with a computer is known. A technique for determining whether or not a captured image is a backlight image captured under backlight conditions in such image processing is known (for example, Patent Document 1).

JP 2007-129277 A JP 2007-315799 A JP 2007-11430 A JP 2008-51621 A

  However, there is a demand for improvement in determination accuracy for determining whether or not a captured image is a backlight image by image processing.

  An object of the present invention is to provide a technique for accurately determining whether or not a captured image is a backlight image by image processing.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms or application examples.

Application Example 1 An image processing apparatus,
A captured image acquisition unit for acquiring captured images;
A backlight estimation unit that estimates a relative angle between the shooting direction of the captured image and the direction of the sun at the shooting position of the captured image, and estimates whether the captured image is a backlight image based on the relative angle. When,
Backlight determination that determines a region of the captured image to be used according to an estimation result of the backlight estimation unit, and determines whether the captured image is a backlight image based on a pixel value of a pixel in the determined region And
An image processing apparatus comprising:

  In this way, based on the shooting direction and the direction of the sun, it is estimated whether or not the captured image is a backlight image, and further, it is determined whether or not it is a backlight image using the pixels of the captured image. In addition, it is possible to accurately determine whether the captured image is a backlight image. Furthermore, since the region where the pixel value is to be used is determined by the backlight determination unit according to the estimation result of the backlight estimation unit, it can be accurately determined whether or not the captured image is a backlight image.

[Application Example 2] The image processing apparatus according to Application Example 1,
A captured image dividing unit that divides the captured image into a plurality of regions;
The said backlight determination part is an image processing apparatus determined based on the pixel value of the pixel of at least 1 of the said area | region.
In this way, it is possible to accurately determine whether or not the captured image is a backlight image by specializing in the captured image captured while traveling on the road.

[Application Example 3] The image processing apparatus according to Application Example 2,
The photographed image dividing unit divides the photographed image into a sky region where the sky is a subject and other regions,
The said backlight determination part is an image processing apparatus determined based on the pixel value of the pixel of the said other area | region.
In this way, since it is determined whether or not it is a backlight image based on pixels in a region other than the sky, which is likely to appear dark in the backlight image, the determination accuracy can be increased.

[Application Example 4] The image processing apparatus according to Application Example 2,
The photographed image is an image of the front taken while traveling on the road,
The captured image dividing unit divides the captured image into a left region, a middle region, and a right region,
The said backlight determination part is an image processing apparatus determined based on the pixel value of the pixel of the said left area | region or a right area | region.
In this way, since it is determined whether or not it is a backlight image based on the pixels in the area other than the sky and the road, which is likely to appear dark in the backlight image, the determination accuracy can be increased.

  The present invention can be realized in various forms, such as an image processing method, a computer program for causing a computer to perform image processing, and a recording medium on which the computer program is recorded. be able to.

It is explanatory drawing which shows schematic structure of the vehicle which comprises some image processing systems as an Example. It is explanatory drawing which shows schematic structure of the image processing apparatus which comprises some image processing systems as an Example. It is a flowchart which shows the process step of a backlight image selection process. It is a flowchart which shows the process step of a backlight estimation process. It is a figure which shows the relationship between a relative angle and the flag output. It is a flowchart which shows the process step of a backlight determination process. It is a figure explaining a backlight determination process. It is a 1st figure explaining the division | segmentation of the frame image in a modification. It is a 2nd figure explaining the division | segmentation of the frame image in a modification. It is a figure which shows the imaging device in a modification. It is a figure which shows the aspect of the division | segmentation of the image used in the backlight determination process in the image image | photographed with each imaging device shown in FIG. 10, and the image image | photographed with each imaging device. It is a figure which shows the relationship between the imaging | photography direction and flag of the imaging device 110B.

  Hereinafter, embodiments of the present invention will be described based on examples with reference to the drawings.

A. Example:
・ Configuration of image processing system:
FIG. 1 is an explanatory diagram showing a schematic configuration of a vehicle 100 constituting a part of an image processing system as an embodiment. As illustrated, the vehicle 100 includes an imaging device 110, a GPS receiver 120, and a calculator 130.

  The imaging device 110 is a camera device that can generate digital or analog image data at a predetermined time interval, such as a digital video camera or a digital still camera capable of continuous shooting. The imaging device 110 is operable in accordance with the control of the computer 130 and is communicably connected to the computer 130 so that the generated image data can be transmitted to the computer 130. For example, the imaging device 110 is installed so as to photograph the front of the vehicle 100 through the front window of the vehicle 100. Since the imaging device 110 has an automatic exposure adjustment function, the subject color may be black in the backlight image.

  The GPS receiver 120 is a device that receives radio waves transmitted from artificial satellites that constitute a GPS (Global Positioning System / Global Positioning System). The GPS receiver 120 converts radio waves received from the artificial satellite into digital data (GPS information) and transmits the digital data to the computer 130.

  The computer 130 is a known personal computer such as a notebook computer. The computer 130 includes a central processing unit (CPU) (not shown), an internal storage device such as a RAM and a ROM, and an external storage device such as a hard disk. Based on the GPS information acquired from the GPS receiver 120, the computer 130 generates current position information (latitude, longitude) that represents the current position of the vehicle 100 and current time information that represents the current time at regular intervals. Is always aware of the current location and time. The vehicle 100 may further include an odometer or a gyro sensor to improve the recognition accuracy of the current position by the computer 130. The computer 130 stores the image data received from the imaging device 110 in an external storage device in association with the shooting position and shooting time of the shot image represented by the image data. The shooting position is the current position of the vehicle 100 at the shooting time.

  Shooting with the imaging device 110 while the vehicle 100 is traveling on a target route for which a traffic sign is to be maintained can be stored as data of the captured image, the shooting position, and the shooting time of the route.

  FIG. 2 is an explanatory diagram illustrating a schematic configuration of an image processing apparatus that constitutes a part of the image processing system according to the embodiment. The image processing apparatus 200 is a known computer, and includes a CPU 201, an input / output unit 202, an internal storage device 203 such as a ROM or a RAM, and an external storage device 204 such as a hard disk. The input / output unit 202 is an interface for exchanging data with an external device, and is connected to the computer 130 described above, for example. As a result, the image processing apparatus 200 can acquire the image data of the captured image stored in the computer 130, the capturing position information indicating the capturing position of the captured image, and the capturing time information indicating the capturing time.

  The image processing program is stored in the internal storage device 203, and the functions of the functional blocks illustrated in the internal storage device 203 in FIG. 2 are realized by the CPU 201 executing the program. The realized functional blocks are a captured image acquisition unit M11, a travel locus acquisition unit M12, a shooting direction estimation unit M13, a solar direction acquisition unit M14, a solar direction estimation unit M15, a captured image division unit M16, and backlight estimation. Part M17 and backlight determination part M18.

  The captured image acquisition unit M11 acquires, via the input / output unit 202, image data captured by the imaging device 110 described above, imaging position information associated with the image data, and imaging time information. In the present embodiment, the imaging device 110 is a video camera, and a moving image composed of a plurality of frame images, shooting position information associated with each frame image, and shooting time information are acquired. The traveling locus acquisition unit M12 identifies the traveling locus of the vehicle 100 on which the imaging device 110 is mounted based on the shooting position information and map data (not shown), and acquires the identified traveling locus as traveling locus information. The shooting direction estimation unit M13 estimates the shooting direction of the imaging device 110 based on the acquired travel locus information. In the present embodiment, as described above, since the imaging device 110 is installed in the vehicle 100 so as to photograph the front of the vehicle 100, the traveling direction that can be easily obtained from the traveling locus information is estimated as the photographing direction. When the imaging device 110 is installed not at the front of the vehicle 100 but at a predetermined angle with the traveling direction, the shooting direction is estimated in consideration of the mounting angle. The solar direction acquisition unit M14 acquires solar position data as solar direction information for specifying the direction of the sun. The acquired solar position data is, for example, sunrise data indicating the sunrise, sunset time, and azimuth by region and date. Such sunrise data can be obtained with high reliability from, for example, the National Astronomical Observatory of Japan / Astronomy Information Center. The sun direction estimation unit M15 estimates the direction of the sun direction as viewed from the shooting position at the shooting time when each frame image of the moving image as the shot image was shot based on the sun position data. For example, if sunrise data of an area to which a shooting position belongs or is close is acquired as sun position data, the direction of the sun at the shooting time can be easily estimated using a known method. The captured image dividing unit M16 divides the frame image when performing backlight determination (described later) by image processing. In the present embodiment, as a first stage, the frame image is divided into an empty area where the sky is a subject and another area where the sky is a subject (road, building, etc.). Next, as a second step, the frame image is divided into three regions, a right region, a middle region, and a left region, and a vertical straight line is divided as a dividing line. The division is executed by a known image processing technique such as edge extraction or hue extraction. The backlight estimation unit M17 estimates whether the frame image is a backlight image based on the shooting direction and the direction of the sun. The backlight determination unit M18 determines whether the frame image is a backlight image by image processing on the frame image while using the backlight estimation result.

  The external storage device 204 stores a shooting moving image storage area D11 for storing a frame image of a shooting movie, and shooting position information and shooting time information (shooting attribute information) associated with each frame image of the shooting movie. It has a photographing attribute information storage area D12 and a solar position data storage area D13 for storing the above-described solar position data. The photographing position information is, for example, information representing coordinates on the ground surface, and is represented by (latitude, longitude), for example.

The operation of the image processing apparatus 200:
FIG. 3 is a flowchart showing processing steps of the backlight image sorting process. The backlight image sorting process is a process executed by the CPU 201 of the image processing apparatus 200. In the backlight image sorting process, the CPU 201 acquires the data of the shooting moving image, the shooting position information, and the shooting time information (shooting attribute information) from the computer 130 via the input / output unit 202, and the shooting movie storage area D11 and the shooting attribute are acquired. The information is stored in the information storage unit D12 and is executed in a state where the sunrise data is stored in the solar position data storage area D13 as the solar position data.

  When the backlight image sorting process is started, the CPU 201 reads the shooting attribute information and the sun position data of the shooting moving image to be a backlight determination target into the internal storage device 203 (step S110). Next, the CPU 201 executes a backlight estimation process (step S200).

  FIG. 4 is a flowchart showing the processing steps of the backlight estimation processing. In the backlight estimation process, the travel route in which the captured video is captured is divided into sections (capturing sections) in units of straight roads (units in which the traveling direction can be regarded as the same), and the frame images constituting the captured videos in units of the captured sections are backlight images. This is a process of primarily determining whether or not there is.

  When the backlight estimation process is started, the CPU 201 acquires shooting position information and shooting time information for the shooting section to be processed (step S210). Next, the CPU 201 reads the sunrise data (sun position data) that is close to the position (latitude, longitude) of the shooting section specified by the shooting position information or belongs to the position of the shooting section, and is specified by the shooting time information. The sunrise time and direction of the day to which the shooting time belongs are acquired (step S220).

  When the sunrise time and azimuth are acquired, the CPU 201 calculates the direction of the sun at the shooting time in the target shooting section (step S230). The direction of the sun is a direction vector in a coordinate system having the ground surface as a plane, for example, a vector on a plane with the north-south direction and the east-west direction as axes. Next, the CPU 201 calculates the shooting direction of the imaging device 110 in the target shooting section (step S240). As described above, in the present embodiment, the imaging device 110 is installed so as to photograph the front of the vehicle 100, so the photographing direction is the traveling direction of the vehicle 100 in the photographing section. The shooting direction is calculated by a direction vector in a coordinate system having the ground surface as a plane, like the direction of the sun, for example, a vector on a plane having the north-south direction and the east-west direction as axes. Next, the CPU 201 calculates a relative angle between the direction of the sun and the shooting direction (step S250). On the computer, the relative angle is calculated as an inner product of a vector representing the sun direction and a vector representing the shooting direction. However, for the sake of easy understanding, the relative angle is represented by a frequency method in which one round is divided into 360 equal parts.

  When the relative angle is calculated, the CPU 201 outputs a flag relating to backlight and forward light for the target shooting section according to the relative angle. That is, the flag is associated with the target shooting section. FIG. 5 is a diagram illustrating the relationship between the relative angle and the output flag. In FIG. 5, the shooting direction of the imaging device 110 is a vertically upward direction. In FIG. 5, a 2-digit numerical value attached to the center of the sun mark indicates a flag. The relative angle is 0 degree when the shooting direction is coincident with the direction in which the sun is located when viewed from the shooting position, and the relative direction is when the shooting direction is opposite to the direction in which the sun is located when viewed from the shooting position. It will be described as 180 degrees. When the relative angle is in the range of 60 degrees from 0 degrees to 30 degrees clockwise and 30 degrees counterclockwise, the output flag is “10”. When the relative angle is in the range of 30 to 60 degrees clockwise, the output flag is “11”. When the relative angle is in the range from 60 degrees to 120 degrees clockwise, the output flag is “21”. When the relative angle is in the range from 120 degrees to 150 degrees clockwise, the output flag is “01”. When the relative angle is in the range from 150 degrees to 210 degrees clockwise, the output flag is “00”. When the relative angle is in the range of 30 to 60 degrees counterclockwise, the output flag is “12”. When the relative angle is in the range from 60 degrees to 120 degrees counterclockwise, the output flag is “22”. When the relative angle is in the range from 120 degrees to 150 degrees counterclockwise, the output flag is “02”. In this embodiment, the flags “10”, “11”, “12”, “22”, and “21” are flags indicating backlight and are also referred to as backlight flags. In the present embodiment, the flags “02”, “00”, and “01” are flags indicating a follow light and are also referred to as a follow light flag. When the flag is output, the backlight estimation process ends.

  Returning to FIG. 3, the description will be continued. When the backlight estimation process is completed, the CPU 201 reads a frame image shot in the target shooting section (step S300). When the frame images are read, the CPU 201 performs backlight determination processing for these frame images (step S400). Hereinafter, for easy understanding, a backlight determination process for one frame image will be described.

  FIG. 6 is a flowchart showing the processing steps of the backlight determination process. When the backlight determination process is started, the CPU 201 determines whether or not the target shooting section is a backlight section with reference to a flag associated with the shooting section (step S410). Here, the backlight section refers to a shooting section associated with a backlight flag in the backlight estimation process described above. If the CPU 201 determines that the target shooting section is not a backlight section, that is, a shooting section associated with a follow light flag (step S410: NO), the backlight estimation process is terminated. That is, the frame image shot in the shooting section associated with the follow light flag is determined to be a follow light image.

  When the CPU 201 determines that the target shooting section is a backlight section (step S410: YES), the CPU 201 converts the pixel value of the frame image into the HSV color space (step S420). In this embodiment, since the photographed image is expressed in the RGB color space, in this step, processing for converting from the RGB color space to the HSV color space is performed. If the captured image is expressed in the HSV color space from the beginning, this step is not necessary. The HSV color space is a color space composed of three components of hue (Hue), saturation (Saturation), and lightness (Value). Also called HSB color space (Hue, Saturation, Brightness). In this embodiment, an HSV color space in which each component is represented by a value from 0 to 255 is used.

Next, the CPU 201 extracts an empty area from the processing target frame image, and divides it into an empty area and another area (another area) (step S430). The extraction conditions for the empty area are as follows.
1. 1. A pixel (sky blue pixel) in a predetermined pixel value range. It is continuous via the upper end of the frame image and other sky blue pixels or directly. Note that the range of pixel values that are the sky blue pixels is, for example, the following 1 or 2.
1. 1. Brightness V is in the range of 220 ≦ V ≦ 255. Lightness V is in the range of 150 ≦ V ≦ 255, saturation S is in the range of 20 ≦ S ≦ 255, and hue H is in the range of 100 ≦ H ≦ 255.

  FIG. 7 is a diagram for explaining the backlight determination process. FIG. 7 shows a frame image 500 to be processed. In the frame image 500, a portion surrounded by a thick line indicates a region extracted as an empty region, and a portion excluding a portion surrounded by a thick line indicates another region.

  Next, the CPU 201 divides the other area into three areas. In the present embodiment, the frame image 500 is divided into a left region 510, a middle region 520, and a right region 530 by vertical dividing lines (FIG. 7: bold broken lines). In this embodiment, the frame image 500 has a size of 480 pixels vertically and 780 pixels horizontally. In this embodiment, the left area 510 is an area in the range of 250 pixels from the left (about 32% with respect to the horizontal width), and the right area 530 is an area of 250 pixels from the right. The ratio of the left region 510 and the right region 530 to the lateral width is preferably 25% to 40%, and more preferably 30% to 35%. In the backlight determination below, the pixel values of the pixels in the left region 510 and the right region 530 are used. The reason is that the backlight determination in the present embodiment is a pre-process for recognizing a traffic sign by image processing, and the captured moving image uses a moving image captured in front of a running car. In this case, there is a high possibility that a traffic sign exists in the left region 510 or the right region 530, and in the middle region 520, the hood, sky, and road of the vehicle 100 are likely to indicate most of the subject. For this reason, it is preferable to determine whether or not the image is a backlight image in which the subject becomes dark using the pixels of the left region 510 and the right region 530 that are highly likely to have traffic signs.

  When the other area is divided into three, the CPU 201 calculates the average value of the brightness of the pixels in the left area 510 and the right area 530, respectively (step S450). That is, the CPU 201 calculates a left average brightness VL that is an average value of the brightness V of all the pixels constituting the left area 510 and a right average brightness VR of the brightness V of all the pixels constituting the right area 530.

  When the left average lightness VL and the right average lightness VR are calculated, the CPU 201 performs backlight determination (step S460). The backlight determination in this step is performed based on whether the frame image 500 is a backlight image or a quasi-backlight image based on the backlight flag associated with the shooting section of the frame image 500, the left average brightness VL, and the right average brightness VR. It is to determine whether there is. The quasi-backlight image is, for example, an image that is estimated that the subject is not so dark although there is a high possibility of being backlit in the direction of the sun.

When the following conditions are satisfied, the frame image 500 is determined to be a backlight image, and when the conditions are not satisfied, the frame image 500 is determined to be a quasi-backlight image.
1. Backlight flag “10”, and average value of right average brightness VR and left average brightness VL <85
2. Backlight flag “11” and right average brightness VR <98
3. Backlight flag “12” and left average brightness VL <98
4). Backlight flag “21”, right average brightness VR <98, and left average brightness VL> 98
5). Backlight flag “22”, left average brightness VL <98, and right average brightness VR> 98

  As a result of the backlight determination, when it is determined that the image is a backlight image (step S460: YES), the processing is ended as it is. That is, the frame image 500 remains associated with the backlight flag. As a result of the backlight determination, if it is determined that the image is a quasi-backlight image (step S460: NO), the quasi-backlight flag is output instead of the backlight flag, that is, the quasi-backlight flag is output to the frame image 500 instead of the backlight flag. (Step S470), and the process ends.

  According to the present embodiment described above, it is estimated whether or not the frame image 500 is backlit using the relative angle between the direction of the sun and the shooting direction, and further backlit using the pixel values of the frame image 500. In combination with the determination process, it is finally determined whether the frame image 500 is a backlight image, a follow light image, or a quasi-backlight image. As a result, it can be accurately determined whether or not the frame image 500 is a backlight image. In this embodiment, since the pixel values of the left region 510 and the right region 530 are used to determine whether the image is a backlight image or a quasi-backlight image, the frame image 500 taken while traveling on a road is specialized. It can be accurately determined whether or not the frame image 500 is a backlight image. Further, since it is determined whether the image is a backlight image or a quasi-backlight image using the pixel values of other regions excluding the sky region, the frame image 500 is based on the pixels of the region other than the sky, which is likely to appear dark in the backlight image. It is possible to accurately determine whether the image is a backlight image. In addition, since the conditions applied to the backlight determination in the backlight determination process are changed according to the estimation result based on the direction of the sun and the shooting direction, it can be accurately determined whether or not the frame image 500 is a backlight image.

B. Modifications-First modification:
In the above embodiment, the frame image 500 is divided into the left region 510, the middle region 520, and the right region 530, but various other methods for dividing the frame image 500 are conceivable. 8 and 9 are diagrams for explaining division of the frame image in the modification. For example, the frame image 500a may be divided into three parts into a left area 510a, a middle area 520a, and a right area 530a in a shape as shown in FIG. This is because the lower part of the frame image 500a is highly likely to show the hood and roadway of the vehicle 100, and therefore, it is preferable to include that part in the middle region 520a that is not considered in the backlight determination. In the example shown in FIG. 9, the lower half of the frame image 500b is a lower region 540b, and the upper half of the frame image 500b is divided into a left region 510b, a middle region 520b, and a right region 530b. The left region 510b and the right region 530b are considered in the backlight determination. This is because the lower part of the frame image 500a is highly likely to show the hood and roadway of the vehicle 100, and therefore, it is preferable to include that part in the middle region 520a that is not considered in the backlight determination. In addition, when the image processing performed after the backlight determination is a traffic sign recognition process, the traffic sign generally has a high possibility of being reflected in the upper part of the screen. It is preferable that the determination can be made well. Therefore, it may be more preferable to divide as shown in FIG.

・ Second modification:
In the above embodiment, the installation direction (imaging direction) of the imaging device 110 is the same as the traveling direction of the vehicle 100, but the installation direction of the imaging device 110 may be at an angle with respect to the traveling direction of the vehicle 100. . In this case, for example, when the imaging device 110 is installed slightly to the right rather than in front of the installation direction, in the frame image 500, a region where roads and sky are reflected and features along the road are reflected. Since the area changes, it is preferable to change the division ratio, the division shape, and the like as appropriate. An example thereof will be described with reference to FIGS.

  FIG. 10 is a diagram illustrating an imaging apparatus according to a modification. FIG. 10 shows five imaging devices 110A to 110E. In the imaging apparatus 110A, the traveling direction and the shooting direction coincide with each other as in the embodiment. The imaging direction of the imaging device 110B is 72 degrees clockwise with respect to the traveling direction. The imaging direction of the imaging device 110C is at an angle of 144 degrees clockwise relative to the traveling direction. The imaging direction of the imaging device 110D forms an angle of 144 degrees counterclockwise with respect to the traveling direction. The imaging direction of the imaging device 110E is 72 degrees counterclockwise with respect to the traveling direction.

  FIG. 11 is a diagram illustrating an image captured by each imaging device illustrated in FIG. 10 and an image division mode used in the backlight determination process in the image captured by each imaging device. FIGS. 11A to 11E show images taken by the imaging devices 110A to 110E, respectively. The image photographed by the imaging device 110A is divided into three in the backlight determination process, as in the embodiment. On the other hand, in the images taken by the imaging devices 110B and 110E, there is a possibility that a feature having a height such as a building or a traffic sign is the subject as a whole (FIGS. 11B and 11E). Therefore, it is not divided in the backlight determination process. As a result, the pixels of the entire image are taken into account in the backlight determination process for the images captured by the imaging devices 110B and 110E.

  On the other hand, in the image captured by the imaging device 110C, there is a high possibility that roads, sky, and the like are subjects in the right half of the image, and there are high-level features such as buildings and traffic signs in the left half of the image. There is a high possibility of becoming a subject. For this reason, in the backlight determination process for the image captured by the imaging device 110C, the captured image is equally divided into left and right (FIG. 11C). In the backlight determination process for the image captured by the imaging device 110C, only the pixels in the left area are considered, and the pixels in the right area are not considered. In addition, in the image captured by the imaging device 110D, there is a high possibility that roads, sky, and the like are subjects in the left half of the image, and in the right half of the image, there are tall features such as buildings and traffic signs. There is a high possibility of becoming a subject. For this reason, in the backlight determination process for the image captured by the imaging device 110D, the captured image is equally divided into left and right (FIG. 11D). In the backlight determination process for the image captured by the imaging device 110C, only the pixels in the right region are considered, and the pixels in the left region are not considered.

  FIG. 12 is a diagram illustrating the relationship between the shooting direction of the imaging apparatus 110B and the flags. As shown in FIG. 12, in the backlight estimation process in the present modification, a follow light flag or an output flag is output corresponding to the relative angle between the shooting direction of the imaging device 110B and the sun direction, as in the embodiment. . The relative angle between the traveling direction and the sun direction is irrelevant, and a flag is output corresponding to the relative angle between the shooting direction and the sun direction.

・ Third modification:
In the above embodiment, the frame image 500 is divided into the left region 510, the middle region 520, and the right region 530, and the brightness of the left region 510 and the right region 530 excluding the sky region is used for backlight determination. In the areas of the left area 510 and the right area 530, the brightness of the area excluding the road area by edge extraction or the like may be used for the backlight determination. This is because it is a preferable subject for determining whether or not a three-dimensional feature portion is a backlight image, and a plane subject such as a road is not a preferable subject for determining whether or not it is a backlight image. .

-Fourth modification:
In the above embodiment, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced by hardware.

  As mentioned above, although the Example and modification of this invention were demonstrated, this invention is not limited to these Example and modification at all, and implementation in a various aspect is possible within the range which does not deviate from the summary. It is.

DESCRIPTION OF SYMBOLS 100 ... Vehicle 110 ... Imaging device 120 ... GPS receiver 130 ... Calculator 200 ... Image processing apparatus 201 ... CPU
202 ... Input / output unit 203 ... Internal storage device 204 ... External storage device 500 ... Frame image

Claims (6)

An image processing apparatus,
A captured image acquisition unit for acquiring captured images;
A backlight estimation unit that estimates a relative angle between the shooting direction of the captured image and the direction of the sun at the shooting position of the captured image, and estimates whether the captured image is a backlight image based on the relative angle. When,
Backlight determination that determines a region of the captured image to be used according to an estimation result of the backlight estimation unit, and determines whether the captured image is a backlight image based on a pixel value of a pixel in the determined region And
An image processing apparatus comprising: The image processing apparatus according to claim 1,
A captured image dividing unit that divides the captured image into a plurality of regions;
The said backlight determination part is an image processing apparatus determined based on the pixel value of the pixel of at least 1 of the said area | region. The image processing apparatus according to claim 2,
The photographed image dividing unit divides the photographed image into a sky region where the sky is a subject and other regions,
The said backlight determination part is an image processing apparatus determined based on the pixel value of the pixel of the said other area | region. The image processing apparatus according to claim 2,
The photographed image is an image of the front taken while traveling on the road,
The captured image dividing unit divides the captured image into a left region, a middle region, and a right region,
The said backlight determination part is an image processing apparatus determined based on the pixel value of the pixel of the said left area | region or a right area | region. An image processing method comprising:
(A) a captured image acquisition step in which the computer acquires a captured image;
(B) The computer estimates a relative angle between the photographing direction of the photographed image and the direction of the sun at the photographing position of the photographed image, and whether the photographed image is a backlight image based on the relative angle. A backlight estimation process for estimating
(C) A computer determines an area of the captured image to be used according to an estimation result of the backlight estimation step, and whether the captured image is a backlight image based on a pixel value of a pixel of the determined area A backlight determination step for determining whether or not
An image processing method comprising: A computer program for causing a computer to execute image processing,
A captured image acquisition function for acquiring captured images;
A backlight estimation function that estimates a relative angle between the shooting direction of the captured image and the direction of the sun at the shooting position of the captured image and estimates whether the captured image is a backlight image based on the relative angle. When,
Backlight determination that determines a region of the captured image to be used according to an estimation result of the backlight estimation function and determines whether or not the captured image is a backlight image based on a pixel value of a pixel of the determined region Function and
A computer program for causing the computer to realize the above.

Images