JP5383246B2 - Vehicle periphery monitoring device - Google Patents

Description

  The present invention relates to an apparatus for monitoring the periphery of a vehicle based on a visible image, and more specifically to an apparatus capable of monitoring the periphery of a vehicle based on a visible image even at night.

  2. Description of the Related Art Conventionally, an image pickup apparatus is mounted on a vehicle, an object such as a pedestrian existing around the vehicle is detected, and driving is supported by notifying the driver of the information.

  In the following Patent Document 1, an infrared camera that captures a warm body image in front of a vehicle and a visible camera that captures a visible image in front of the vehicle are mounted as imaging devices, and images captured from both cameras are captured. It uses to detect a pedestrian.

Japanese Patent No. 3716623

  Since the infrared camera outputs an image having a luminance value corresponding to the temperature of the object, a high-temperature object such as a pedestrian can be detected by using the infrared camera even during night driving.

  However, the infrared camera is relatively expensive, and when mounted on a vehicle, the cost of the entire vehicle may be increased.

  Therefore, there is a demand for an apparatus that can detect a predetermined object such as a pedestrian even when traveling at night using a visible camera that captures a visible image.

  According to one aspect of the present invention, a vehicle periphery monitoring device that includes an imaging unit that captures a visible image around a vehicle and monitors the periphery of the vehicle estimates ambient light around the vehicle, and Judgment means for judging whether the periphery of the vehicle is brightly illuminated, and if it is determined that the periphery of the vehicle is not brightly illuminated, at least the light of the vehicle in the visible image is illuminated When an object around the vehicle is detected based on an image area corresponding to the range and it is determined that the periphery of the vehicle is brightly illuminated, at least light from the vehicle is illuminated in the visible image. Means for switching an object detection method so as to detect an object in the vicinity of the vehicle based on an image area above the image area corresponding to the range being set.

  Normally, when an image is captured using a visible camera in a dark environment such as at night, the luminance value of the captured image is uniformly reduced, and thus it is difficult to detect a predetermined object. However, when ambient light such as a streetlight is present, the surroundings are brightly illuminated by the ambient light, and therefore, a predetermined object can be detected from the captured visible image as in the case of daytime. On the other hand, even when the ambient light is low and the surroundings are dark, it is possible to detect a predetermined object from the visible image within a range illuminated by a vehicle light (for example, a headlight). The present invention is based on this knowledge, and determines whether or not the surroundings of the vehicle are brightly illuminated by ambient light. If it is determined that the vehicle is not illuminated, the present invention is based on the image area within the range in which the light of the vehicle is illuminated. If the object is detected and it is determined that the object is not illuminated, the object is detected based on the image area above the image area. By doing so, a predetermined object can be detected even at night.

  In one embodiment of the present invention, when the determination unit determines that the periphery of the vehicle is not brightly illuminated, the object around the vehicle is detected by determining the movement of the object. If the determination means determines that the periphery of the vehicle is brightly illuminated, the object around the vehicle is detected by determining the shape of the object. In one embodiment, the object is a pedestrian, and if it is determined that the periphery of the vehicle is not brightly illuminated, the pedestrian is detected by determining the movement of the pedestrian's legs, When it is determined that the surrounding area is brightly illuminated, the pedestrian is detected by determining the shape of the pedestrian's head.

  In many cases, an object has characteristics that can be distinguished from other objects at the upper part (upper body). In particular, in the case of a pedestrian, the shape of the head (including the face) is distinguishable from other objects such as an artificial structure. Therefore, when the periphery of the vehicle is bright, the object can be detected with better accuracy by using the shape determination method. On the other hand, if the periphery of the vehicle is not bright, the object can be detected from the range illuminated by the vehicle light. In this range, the upper part of the object is not imaged and the lower part of the object (lower body) is detected. Only) may be detectable. Only the shape of the lower part of the object may be difficult to distinguish from other objects. Therefore, attention is paid to the movement of the lower part of the object. When the target object is a living body such as a pedestrian, there is a feature in the movement of the foot. Therefore, by detecting the movement, it can be detected separately from other target objects such as an artificial structure.

  In one embodiment of the present invention, the determination means includes: 1) an image region having a luminance value greater than or equal to a predetermined value and an area greater than or equal to a predetermined value in a visible image captured by the imaging means 2) In the visible image, the brightness in the image area corresponding to the range far from the vehicle and the image area corresponding to the range in the vicinity of the vehicle in the visible image When at least one of the difference between the brightness and the brightness is less than a predetermined value, and 3) when the detected value of the illuminance sensor mounted on the vehicle is greater than or equal to the predetermined value, Judged to be brightly lit.

  In this way, it is possible to determine whether or not the surroundings of the vehicle are brightly illuminated by the ambient light, so that the object detection method can be automatically switched according to the determination result.

  In one embodiment of the present invention, if the object is detected when it is determined that the periphery of the vehicle is not brightly illuminated, an indicator display and a warning sound are generated to alert the driver. If the object is detected when it is determined that the periphery of the vehicle is brightly illuminated, at least one of the indicator display and the warning sound is displayed. In addition, a notification means for notifying the driver by displaying the position of the object superimposed on the captured image displayed on the display device is provided.

  When the periphery of the vehicle is not brightly illuminated, an object at a short distance from the vehicle is detected, so it is preferable to inform the driver of the presence of the object at an early stage. Therefore, the notification is performed by a relatively simple method such as indicator display or alert sound. On the other hand, when the periphery of the vehicle is brightly illuminated, it is possible to detect an object at a long distance from the vehicle, so there is a time margin to inform the driver compared to the above case. For this reason, not only a relatively simple notification such as an indicator display or a warning sound, but also the position of the object is superimposed on the captured image of the display device to notify the driver of the specific position of the object.

  Other features and advantages of the present invention will be apparent from the detailed description that follows.

The block diagram which shows the structure of the periphery monitoring apparatus according to one Example of this invention. The figure for demonstrating the attachment position of the camera according to one Example of this invention. The figure for demonstrating the principle of this invention. 3 is a flowchart illustrating a process in an image processing unit according to an embodiment of the present invention. The figure which shows an example of the distant area | region and vicinity area | region used for estimation of environmental light according to one Example of this invention. The figure for demonstrating the shape determination of a pedestrian's head and the motion determination of a pedestrian's foot | leg according to one Example of this invention. The figure which shows the alerting | reporting form according to one Example of this invention.

  Next, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention. The apparatus is mounted on a vehicle, and includes visible cameras 1R and 1L, an image processing unit 2 for detecting an object around the vehicle based on image data obtained by the cameras 1R and 1L, and based on the detection result. A head-up display (hereinafter referred to as HUD) 4 that displays an image obtained by a speaker 3 that generates an alarm by sound and an image obtained by the camera 1R or 1L and that allows a driver to recognize an object around the vehicle. And. The visible cameras 1R and 1L can be configured by a CCD camera, a CMOS camera, or the like that can capture an image in the visible light region, and the higher the illuminance of the imaged object, the higher the level of the output video signal (that is, , The brightness value in the captured image is increased).

  In this embodiment, as shown in FIG. 2, the cameras 1 </ b> R and 1 </ b> L are arranged in a symmetric position with respect to the central axis passing through the center of the vehicle width at the front portion of the vehicle 10 and image the front of the vehicle. To do. The two cameras 1R and 1L are fixed to the vehicle so that their optical axes are parallel to each other and their height from the road surface is equal.

  The image processing unit 2 includes an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, a central processing unit (CPU) that performs various arithmetic processing, and a data RAM (Random Access Memory) used to store data, ROM (Read Only Memory) that stores programs executed by the CPU and data used (including tables and maps), driving signals for the speaker 3, display signals for the HUD 4, etc. Is provided. The output signals of the cameras 1R and 1L are converted into digital signals and input to the CPU. As shown in FIG. 2, the HUD 4 is provided so that a screen 4 a is displayed at a front position of the driver on the front window of the vehicle 10. Thus, the driver can visually recognize the screen displayed on the HUD 4.

  Here, the principle of the present invention will be described with reference to FIG. FIG. 3 (a) is an example of a visible image when the front of the vehicle is imaged by a visible camera mounted on the vehicle when the ambient light around the vehicle is low at night, and (b) is It is an example of a visible image when the front of a vehicle is imaged by a visible camera mounted on the vehicle when there is a lot of ambient light around the vehicle. In both (a) and (b), the vehicle headlight is lit in a low beam state. Ambient light indicates light existing in the surrounding environment of the vehicle, such as streetlight light, headlight light of an oncoming vehicle, store light, signboard light, and the like. In (b), there are more ambient lights of street light as indicated by arrow 121 and headlight of oncoming vehicle as indicated by arrow 123 than in (a). An image area below the horizontal line 101 (also referred to as a lower area) is an image area corresponding to a range in the vicinity of the vehicle illuminated by the headlight of the vehicle.

  As shown in (a), when there is almost no ambient light, the image area (also referred to as the upper area) above the horizontal line 101 has a lower luminance than the lower area. The pedestrian 105 exists in the area surrounded by the reference numeral 103, but the foot part of the pedestrian 105 can be recognized because it belongs to the lower area, but the upper body including the head is in the upper area. It is difficult to recognize because it belongs. As described above, the object or the object part existing in the lower region can be detected, but the object or the object part existing in the upper region is difficult to detect.

  However, even in such a nighttime, as shown in (b), if there is ambient light such as streetlight light 121 or headlight light 123 of the oncoming vehicle, the object can be recognized even in the upper region. it can. Although the pedestrian 115 exists in the area surrounded by the reference numeral 113, the upper body including the head of the pedestrian 115 is recognizable regardless of belonging to the upper area. Can be detected.

  The present invention has been made on the basis of the above-described knowledge, and when there is little or no ambient light as shown in (a), at least the area irradiated with the light of the vehicle, that is, the lower part of the captured image An object is detected based on the region. On the other hand, when there is a lot of ambient light as in (b), the object is detected based on at least the area illuminated by the ambient light, that is, the upper area of the captured image. In the following embodiments, the object to be detected is a pedestrian, and therefore, in the case of (a), the foot part of the pedestrian is detected based on at least the lower region, and in the case of (b) Detects the pedestrian's head based at least on the upper region.

  Note that the upper part (upper body) of an object often has a characteristic that distinguishes it from other objects compared to the lower part (lower body). Therefore, it is normal to determine the object by detecting the upper part. Done. When the object is a pedestrian, the head (including the face) is imaged to have a circle or ellipse shape that is common to all pedestrians. It can be distinguished from other objects. In order to increase the accuracy, it is also possible to detect the shape from the head to the shoulder or to detect a face pattern (a pattern of parts such as eyes and mouth). In the case of (b), since the upper body of the pedestrian can be detected, the pedestrian is detected using such shape determination.

  However, in the case of (a), it is difficult to detect the upper body of the pedestrian. Further, when shape determination is performed on the captured image as shown in (a), an elliptical shape corresponding to a pedestrian head that cannot be recognized originally is searched, which may lead to erroneous detection. . Therefore, in the case of (a), the foot part of the pedestrian is detected. Since the foot is imaged so as to have two squares that are long in the vertical direction, there is a possibility that it cannot be distinguished from an artificial structure or the like only by determining the shape of the foot. On the other hand, the movement of a foot by a pedestrian has a feature that two feet move alternately, which is a feature that does not exist in an artificial structure or the like. Therefore, a pedestrian is detected by determining the movement of the foot. In addition, about the pedestrian who exists in a long distance, since the walk distance (distance of one step) is very small on a captured image, it is difficult to detect the pedestrian by movement, In this case, since the pedestrian existing in the vicinity of the vehicle irradiated with the headlight of the vehicle is detected, the pedestrian can be detected by the movement.

  As described above, the object can be detected from the visible image even at night by switching the object detection method between the cases (a) and (b).

  Next, specific processing contents will be described with reference to FIG. FIG. 4 is a flowchart illustrating a process performed by the image processing unit 2 according to one embodiment of the present invention. The process is performed at predetermined time intervals.

  In steps S11 to S13, the output signals of the cameras 1R and 1L (that is, captured image data) are received as input, A / D converted, and stored in the image memory. The image data obtained here is a grayscale image including luminance information. In the following processing, processing is performed based on a gray scale image of the right image (which may be an alternative or left image) captured by the camera 1R.

  In step S14, ambient light is estimated, and it is determined whether the periphery of the vehicle (in this embodiment, the front side) is brightly illuminated.

  There are several methods for estimating ambient light. According to the first method, in a visible image (grayscale image), ambient light is estimated by pixels having a height from a road surface that is equal to or greater than a predetermined value and having a luminance value that is equal to or greater than a predetermined value. When the value is greater than or equal to the value, it is determined that there is a lot of ambient light and therefore the periphery of the vehicle is brightly illuminated. Otherwise, it is determined that the periphery of the vehicle is not brightly illuminated.

  Here, the predetermined value, which is a threshold value for the luminance value, is obtained by examining the luminance value of ambient light such as a street lamp or a signboard light in a visible image through simulation or the like, and an image area having a luminance value equal to or greater than the predetermined value is obtained. It is preset to represent these ambient lights. The predetermined value, which is a threshold value for the height from the road surface, is set in advance to a value that includes street lights, sign lights, and the like, and is about 1.8 meters, for example. As is well known, the actual height of the object being imaged can be calculated based on the distance to the object and the focal length of the camera. The distance to the object can be detected by a known triangulation method based on the distance between the pair of cameras 1R and 1L and the parallax obtained from the pair of images obtained by these cameras. .

  Further, the predetermined value, which is a threshold for the number of pixels, is set in advance so that the upper body of the pedestrian can be recognized in the visible image as shown in FIG.

  In the second method, when the ambient light is estimated based on the luminance value of the road surface far from the vehicle, and the difference between the luminance value of the remote road surface and the luminance value of the road surface near the vehicle is equal to or less than a predetermined value, It is judged that there is a lot of ambient light that illuminates the road surface, and therefore the periphery of the vehicle is brightly illuminated. Otherwise, it is determined that the periphery of the vehicle is not brightly illuminated.

  As is clear from comparison between FIGS. 3A and 3B, when a road surface extending in front of the vehicle is illuminated by a streetlight or a headlight of an oncoming vehicle, However, if the road surface extending in front of the vehicle is not illuminated by streetlights or headlights of oncoming vehicles, the vicinity of the vehicle is illuminated by the headlights of the vehicle. At a long distance, the headlight illumination does not reach, so it remains dark.

  The positions of the image areas corresponding to the distance and the vicinity of the vehicle can be determined in advance on the visible image. For example, FIG. 5 shows the same visible image as FIG. 3B, but the region surrounded by the reference numeral 131 located near the center of the bottom of the image is a far region, and the pixels included in this region are The average luminance value is calculated. The area surrounded by the reference numeral 133 located near the center of the upper side of the image is a neighboring area, and similarly, the average of the luminance values of the pixels included in the area is calculated. If the difference (absolute value) between the average values of the two is not more than a predetermined value, it is determined that the periphery of the vehicle is brightly illuminated. Instead of the average of the luminance values, the sum of the luminance values of the pixels included in the region may be used.

  In the third method, a known illuminance sensor that detects the brightness around the vehicle may be provided in the vehicle, and the ambient light may be estimated from the detection value of the illuminance sensor. If the detection value of the illuminance sensor is equal to or greater than a predetermined value, it is determined that there is a lot of ambient light and the surroundings of the vehicle are brightly illuminated. As shown in FIG. 3B, the predetermined value here is set in advance through simulation or the like so that the object can be recognized in the upper area of the image.

  Furthermore, in another fourth method, the ambient light may be estimated based on an average or sum of luminance values of the entire image. If the average value or the sum is equal to or greater than a predetermined value, it is determined that there is a lot of ambient light and the surroundings of the vehicle are brightly illuminated. Further, FIGS. 3A and 3B are the same in that the lower area of the image is illuminated by the headlight of the vehicle, so that instead of the entire image, the average luminance value of the upper area of the image or Ambient light may be estimated by the sum. If the average value or the sum is equal to or greater than a predetermined value, it is determined that there is a lot of ambient light and the surroundings of the vehicle are brightly illuminated. Here, the horizontal line 101 can be determined in advance by examining an image region corresponding to a range irradiated by the headlight of the vehicle by simulation or the like.

  Furthermore, in the fourth method, instead of using the average or sum of the luminance values of the entire image, the variance of the luminance values of the entire image (alternatively, a standard deviation may be used). When there is a lot of ambient light as shown in FIG. 3B, the entire image is illuminated with light and is uniformly bright. Therefore, if the variance value is less than or equal to a predetermined value, it is determined that the vehicle periphery is brightly illuminated, and if the variance value is greater than the predetermined value, it can be determined that the vehicle periphery is not brightly illuminated. it can.

  The determination may be made by combining any of the first to fourth methods. For example, the first and second methods may be combined, and when both conditions are satisfied, or when one of the conditions is satisfied, it may be determined to be bright.

  Returning to FIG. 4, in step S15, the determination result in step S14 is examined to determine whether it is determined bright. If it is determined that it is bright, it is determined that the upper body of the pedestrian can be recognized because the periphery of the vehicle is bright when illuminated by ambient light, as described with reference to FIG. Therefore, in step S21, a pedestrian is detected using shape determination. As described above, since the pedestrian's head (including the face) has a characteristic shape, the pedestrian is detected by determining the shape of the head. Since the pedestrian's head is imaged in the upper region above the horizontal line 101 of the image, it is detected from the upper region. Therefore, instead of performing the pedestrian detection process on the entire image, the pedestrian detection process may be performed only on the upper region of the image.

  On the other hand, if it is not determined to be bright in step S15, it is determined that it is difficult to recognize the upper body of the pedestrian as described with reference to FIG. Therefore, since it is difficult to detect pedestrians by determining the shape of the upper body, in step S23, pedestrians are detected by movement. As described above, since the movement of the pedestrian's foot is characteristic compared to other objects, the pedestrian is detected by determining the movement of the foot. Since the pedestrian's foot is imaged in the lower area below the horizontal line 101 of the image, that is, the area irradiated by the headlight of the vehicle, it is detected from the lower area. Therefore, instead of performing the pedestrian detection process on the entire image, the pedestrian detection process may be performed only on the lower region of the image.

  An arbitrary method can be used for the pedestrian detection using the shape performed in step S21. As described above, since the pedestrian's head (including the face) is generally elliptical (including a circle), the pedestrian's head (including the face) is easy to detect and easily distinguishable from other objects. It is preferable to perform pedestrian detection by detecting an elliptical head 141 as shown in FIG. Various techniques for detecting the shape of the head have been proposed, and any appropriate technique can be used.

  Typically, an edge image is generated by applying, for example, a differential filter to the visible image to extract an edge. Thereby, the outline of a pedestrian's head is extracted. On the other hand, a pattern (template) in which the head is modeled is stored in advance in a memory. A well-known pattern matching (template matching) is used for the edge image, and an edge having a high similarity (correlation) with the pattern stored in the memory is extracted to extract the head. A method for determining the shape of the head using such a template is described in, for example, Japanese Patent Application Laid-Open Nos. 2001-222719, 2005-149144, and 2005-25568. In order to further improve the detection accuracy, the shape from the head to both shoulders may be determined using a template. Such a method is disclosed in, for example, Japanese Patent Application Laid-Open No. 2005-149145.

  Alternatively, the head may be detected using a neural network. For example, Japanese Patent Application Laid-Open Nos. 2005-190400 and 2005-122351 show a method for searching a face image area using a neural network.

  Furthermore, pedestrian detection may be performed by detecting a face pattern. Various face pattern detection methods have been proposed. For example, grayscale template matching is performed on the extracted face (head) region, and the positional relationship between both eyebrows and mouth is appropriate. By determining whether or not there is a face, it is possible to determine whether or not it is a face (for example, JP-A-8-287216). In addition, a method for recognizing a part such as an eye or a mouth of a face based on an edge image obtained by extracting an edge component of the image has been proposed (for example, Japanese Patent Application Laid-Open No. 2004-78637).

  Arbitrary methods can be used for the pedestrian detection using a motion implemented at Step S23. As mentioned above, pedestrian foot movement is characterized by the fact that the left and right feet are moved alternately, and can easily be distinguished from other objects. Preferably it is done. Various techniques for detecting the movement of the foot have been proposed, and any appropriate technique can be used.

  For example, as shown in FIG. 6B, the movement of the foot can be detected by detecting the part 143 of both feet from the visible image and tracking the opening degree W of both feet with time. More specifically, edge extraction processing is performed on the visible image using a known filter or the like to generate an edge image including the contour of the pedestrian's foot. On the other hand, a pattern (template) that models a human foot is stored in advance in a memory. A well-known pattern matching (template matching) is performed on the edge image, edges (contours) representing the left and right feet are extracted from the edge image, and the opening W of the left and right feet is detected. By performing this process on a temporally continuous visible image, the opening degree is temporally tracked. If the opening changes with time over a predetermined value, it can be determined that the pedestrian is walking while moving his left and right feet. Such a method is described in, for example, Japanese Patent Application Laid-Open No. 2007-264778. In addition, the state in which both legs are opened is detected using a pattern that models both legs that have been opened, and the state in which both legs are closed is detected by using a pattern that models both legs that have been closed. A pedestrian can be determined by alternately detecting a closed leg state and a closed leg state, and such a method is described in, for example, Japanese Patent Application Laid-Open No. 2008-288684.

  Thus, when it is determined that the periphery of the vehicle is bright, the pedestrian is determined by shape determination. When it is determined that the periphery of the vehicle is not bright, the pedestrian is determined by motion determination. Even at night, a pedestrian can be detected from a visible image.

  Returning to FIG. 4, if a pedestrian is detected by shape determination, the driver is notified of the information in step S25. In this embodiment, the notification is realized by displaying an indicator “BRAKE” and a warning sound. The display content of the indicator may be arbitrary, and the alert sound may be any kind of sound, and human voice may be used.

  Further, when the periphery of the vehicle is bright, as shown in FIG. 3B, an image is captured in a state where an object existing at a long distance from the vehicle can be recognized. Moreover, when shape determination is used, a pedestrian existing at a relatively long distance from the vehicle can be detected. Therefore, the driver can have a time margin for recognizing the object. Therefore, when a pedestrian is detected by shape determination, it is preferable to calculate the position of the pedestrian and notify the driver of the pedestrian in addition to the indicator display and the alert sound as described above.

  Specifically, in response to the detection of the pedestrian's head in step S21, the height of the pedestrian on the image is estimated based on the distance to the head and the average height of the adult, Set a rectangular area having a length corresponding to the estimated height in the vertical direction and having a predetermined width in the horizontal direction (this can be set based on the general torso width of an adult) The sign indicating the region is superimposed on the captured image. For example, the average height of an adult is Ha, the pixel interval of the image in the height direction, that is, the length (cm) per pixel is pch, the focal length of the lens mounted on the camera 1R is h, and the detected head If the distance to the part is Z, the height h of the pedestrian on the image can be calculated by (Ha / pch) × f / Z.

  Here, referring to FIG. 7A, the notification form in step S25 is shown, and an example of a captured image displayed on the display screen 4a of the HUD 4 is shown in the upper part, and a frame 201 is superimposed. It is displayed. A frame 201 indicates a square region set as described above for the detected pedestrian.

  In addition, a display 203 “BRAKE” indicated by reference numeral 203 is an example of an indicator display, and this is superimposed on the captured image displayed on the HUD 4 if the display is visible to the driver. Or may be displayed on a predetermined display or the like on the instrument panel. As described above, the display content of the indicator may be arbitrary. Reference numeral 205 denotes a speaker, which indicates that a warning sound is emitted.

  Thus, the driver is not only informed of the presence of the object to be noted by the indicator or the alert sound, but can also recognize the specific position of the object on the display screen. In order not to confuse the driver, it is preferable that the output of the indicator 203 and the alert sound 205 is performed at the same time as the position of the object is calculated and the highlight 201 is displayed on the captured image as described above.

  On the other hand, returning to FIG. 4, if a pedestrian is detected by the motion determination, the driver is notified of the information in step S27. In this embodiment, the notification is realized by the display of the indicator and the alert sound as described above. The display content of the indicator may be arbitrary, and the alert sound may be any kind of sound, and human voice may be used.

  When the periphery of the vehicle is not bright, an object existing at a short distance from the vehicle is detected as shown in FIG. Therefore, it is desirable for the driver to recognize the presence of the object at an earlier stage because the driver has little time to recognize the object. On the other hand, when the position of the object as described with reference to step S25 is calculated and highlighted, it takes time for the driver to visually recognize the display screen. May be delayed. Therefore, in step S27, in order to promote the driver's early recognition, notification of only the indicator display and the warning sound is promptly performed without calculating the position of the object.

  Here, referring to FIG. 7 (b), the notification form of step S27 is shown. Although a captured visible image is displayed on the screen 4a of the HUD 4, it is indicated by a frame 201 in (a). Such object highlighting is not performed. As indicated by reference numerals 203 and 205, an indicator display and a warning sound are output. Thus, the driver can know the presence of the object to be noted visually and audibly. In addition, since the target object is not highlighted on the displayed captured image, it can be recognized that the target object exists at a short distance.

  In steps S25 and S27, notification may be performed using only one of the indicator display and the alert sound.

  In the case of step S25, the shape of a pedestrian (a pedestrian located across the horizontal line 101, with the upper body in the upper region and the lower body in the lower region) is also shaped. Since it may be detected by the determination, the distance value of the detected pedestrian is obtained (as described above, obtained by a known triangulation method or the like), and the notification form is changed according to the distance value. It may be. For example, if the distance value is farther than a predetermined value, the object is highlighted as shown in FIG. 7A. If the distance value is less than the predetermined value, an indicator display and ( Or) Notification based on the alert sound.

  In the above embodiment, any one of the object detection methods is selected according to the determination result of the brightness, but there may be various variations as this modification.

  For example, since the image area below the horizontal line 101 is an irradiation area of the headlight, the object can always be detected. Therefore, motion detection may always be performed regardless of whether it is determined bright or not bright. The shape determination is performed only when it is determined to be bright.

  Furthermore, the object detection method may be switched according to the vehicle speed. It is desirable to detect an object that exists farther as the vehicle speed is higher, and it is desirable to detect an object that is present closer as the vehicle speed is lower. Therefore, when the vehicle speed is equal to or higher than the predetermined value, the shape is determined if the vehicle periphery is determined to be bright, and if the vehicle periphery is not determined to be bright, motion detection is performed and the vehicle speed is lower than the predetermined value. If it is determined that the periphery of the vehicle is bright, not only the shape determination but also the movement determination is performed. If it is determined that the periphery of the vehicle is dark, the movement determination is performed. In this way, when the vehicle speed is low, the motion determination can always be performed to notify early.

  Furthermore, either or both of the shape determination and the motion determination may be executed in accordance with the brightness level determined by the environmental light estimation. For example, as described above, according to the number of pixels in the first method, according to the magnitude of the difference between the average values (or sums) of the luminance values in the far and neighboring regions in the second method, In the method, the brightness around the vehicle is determined according to the magnitude of the detected value of the illuminance sensor, and in the fourth technique, the brightness around the vehicle is determined according to the average value (or sum) of the luminance values. Are determined in three stages. If it is determined that it is the brightest stage, shape determination is performed, if it is determined that it is the darkest stage, motion determination is performed, and if it is determined that it is an intermediate stage, both shape determination and motion determination are performed. Run. When both are executed, if an object is detected in any of the determinations, the notification in step S25 or S27 can be performed accordingly.

  Note that if shape determination is performed when it is determined that the periphery of the vehicle is not bright, a shape (ellipse) corresponding to a pedestrian's head that cannot be visually recognized is searched for, which leads to erroneous detection. There is a fear. Therefore, when it is determined that the periphery of the vehicle is not bright, it is preferable to perform only motion detection.

  In the above embodiment, the object to be detected is a pedestrian. However, the present invention is also applicable to animals, for example. In the case of a large animal such as a deer or a bear, if there is little ambient light, as described with reference to FIG. 3A, there is a possibility that an image is captured so that only the foot portion can be recognized. Moreover, it can be determined whether it is an animal by determining whether it is moving by moving a quadruped. When there is a lot of ambient light, the head and torso of the animal can be detected by pattern matching based on the shape. Therefore, the process as shown in FIG. 4 can be applied to animals.

  Furthermore, in the above-described embodiment, an example in which a camera that images the front of the vehicle is attached has been described as an example. However, the present invention can also be applied to an embodiment in which a camera that images other parts of the vehicle, for example, the rear is attached. It is. In this case, a range irradiated by a tail lamp (reverse lamp) is used instead of the range irradiated by the headlight.

  In the present invention, the brightness around the vehicle is determined by estimating the ambient light, and the processing is switched according to the determination result. However, this technique is applicable to any other system. For example, systems that detect a lane (white line or the like) on which a vehicle travels have been proposed in various forms. A detected lane is displayed on a display device, or a preceding vehicle is determined using the detected lane. In such a detection system, if there is little ambient light, it is difficult to detect a lane at a long distance from the vehicle. Therefore, when the ambient light is low, it is possible to notify the driver that lane detection is difficult unless the vehicle speed is limited to a predetermined low speed, or to urge the driver to limit the vehicle speed to a low speed. Also, if the ambient light is small, it is difficult to detect the lane on a strong curve, so the environment light is small, and the curvature of the travel path (this can be obtained from the road information of a known navigation unit). ) Is greater than or equal to a predetermined value, the driver can be notified that lane detection is difficult. In this way, the driving assistance method can be switched according to whether the surroundings of the vehicle are bright by estimating the ambient light.

  In the above embodiment, a pair of visible cameras are used to determine the distance to the object. However, the present invention can also be applied to a single visible camera. In this case, the distance to the object can be detected by attaching a device capable of detecting the distance, such as a radar, to the vehicle, or at each position on the captured image via simulation or the like. The approximate distance value of the object may be obtained from the position on the captured image of the object by associating the approximate distance value.

  As described above, specific embodiments of the present invention have been described. However, the present invention is not limited to these embodiments.

1R, 1L visible camera (imaging means)
2 Image processing unit 3 Speaker 4 Head-up display

Claims (5)

A vehicle periphery monitoring device equipped with an imaging means for capturing a visible image around the vehicle and monitoring the periphery of the vehicle,
Determining means for estimating ambient light around the vehicle and determining whether the surrounding of the vehicle is brightly illuminated;
If it is determined that the periphery of the vehicle is not brightly illuminated, an object in the vicinity of the vehicle is selected for an image region corresponding to at least a range illuminated by the vehicle light in the visible image. If it is determined that the periphery of the vehicle is brightly illuminated, at least a portion of the visible image that is above the image region corresponding to the range illuminated by the vehicle light is detected. Means for switching the detection method of the object by changing the image processing area so as to perform a detection process for detecting the object around the vehicle on the image area;
A vehicle periphery monitoring device. When it is determined by the determination means that the periphery of the vehicle is not brightly illuminated, an object around the vehicle is detected by determining the movement of the object, and the vehicle is detected by the determination means. If it is determined that the periphery of the vehicle is brightly illuminated, the object around the vehicle is detected by determining the shape of the object.
The vehicle periphery monitoring apparatus according to claim 1. When the object is a pedestrian and the determination means determines that the periphery of the vehicle is not brightly illuminated, the pedestrian in the vicinity of the vehicle is determined based on the movement of the pedestrian's legs. And when the judgment means judges that the periphery of the vehicle is brightly illuminated, the pedestrian around the vehicle is detected by determining the shape of the head of the pedestrian. To
The vehicle periphery monitoring apparatus according to claim 1 or 2. The determination means is: 1) In a visible image captured by the imaging means, an image region having a luminance value greater than or equal to a predetermined value and having an area greater than or equal to a predetermined value is located where the height from the road surface is greater than or equal to a predetermined value. 2) In the visible image, the difference between the luminance in an image area corresponding to a range far from the vehicle and the luminance in an image area corresponding to a range near the vehicle is less than a predetermined value in the visible image. 3) When the detected value of the illuminance sensor mounted on the vehicle is equal to or greater than a predetermined value, if at least one of the detected values is satisfied, it is determined that the periphery of the vehicle is illuminated brightly ,
The vehicle periphery monitoring apparatus according to any one of claims 1 to 3. If the object is detected when it is determined that the periphery of the vehicle is not brightly illuminated, the driver uses at least one of an indicator display and a warning sound to alert the driver. If the object is detected when it is determined that the periphery of the vehicle is brightly illuminated, the position of the object is added to at least one of the indicator display and the alert sound. And a notification means for notifying the driver by displaying the image superimposed on the captured image displayed on the display device,
The vehicle periphery monitoring apparatus according to any one of claims 1 to 4.