JP4972116B2 - Vehicle periphery monitoring device - Google Patents

Description

  The present invention relates to an apparatus for monitoring the periphery of a vehicle, and more specifically to an apparatus for changing a threshold value for extracting an object in accordance with a situation around the vehicle.

  Conventionally, as a method of separating a desired object from a background in an image, there is a method of binarizing with a predetermined threshold. In the following Patent Document 1, in an image captured using an image sensor such as a CCD, a threshold used for binarization so as to separate an object and a background even if the object to be recognized itself has uneven brightness. It is described to change.

  On the other hand, it has been proposed to monitor the periphery of a vehicle using an infrared camera. Since living bodies such as pedestrians and animals are usually at a higher temperature than the background, these living bodies can be detected separately from the background using an infrared camera. In an image captured by an infrared camera, Patent Document 2 below describes a threshold setting method for binarization using a so-called mode method. According to the mode method, in the luminance value histogram, a threshold value that bisects the luminance value in the image is determined between the peak corresponding to the background and the peak corresponding to the high temperature object.

JP-A-8-315153 Japanese Patent No. 3756452

  If an infrared camera is used to capture the surroundings of a vehicle, the mode method as described above is used if two peaks, a peak corresponding to the background and a peak corresponding to the high-temperature object, appear in the brightness value histogram of the captured image. Thus, it is effective to set a threshold value.

  However, in the image captured by the infrared camera, various objects may be captured depending on the situation around the vehicle, and two peaks may not appear clearly. In such a case, it is difficult to use the mode method. On the other hand, if the luminance values are classified based on a fixed threshold value, there is a possibility that the desired object to be detected cannot be extracted well separated from the background.

  Therefore, even if the above two peaks do not appear clearly in a captured image using an infrared camera, the luminance value can be divided into two so that a desired object can be satisfactorily extracted from the background. Is desired.

  According to one aspect of the present invention, a vehicle periphery monitoring device creates an infrared camera mounted on a vehicle and a brightness value histogram of a captured image captured by the infrared camera, and the entire frequency in the brightness value histogram Using a predetermined ratio S (%) with respect to a low luminance range in which the cumulative frequency is (100−S)% and a high luminance range in which the cumulative frequency is S%, the two are included in the high luminance range Means for extracting an object having a luminance value as a candidate for an object to be detected; determination means for determining whether the extracted candidate for an object is the object to be detected; and the object to be detected Means for notifying the driver when the object is determined. Furthermore, the apparatus is configured to estimate whether or not there are many high-temperature objects around the vehicle, and according to an estimation result whether or not there are many high-temperature objects. Means for changing the predetermined ratio.

  In the present invention, a so-called P tile method is used. According to the P tile method, in the luminance value histogram, a predetermined ratio S with respect to the entire frequency is used, and a low luminance range in which the cumulative frequency is (100-S)% and a high luminance range in which the cumulative frequency is S%. Therefore, the present invention can be applied to an image in which two peaks do not appear clearly. The situation around the vehicle changes as it travels, and there are places where there are many high-temperature objects such as urban areas, while there are places where there are few high-temperature objects such as highways and suburban areas. Regardless of the situation around the vehicle, if the above-mentioned predetermined ratio S is fixed, there is a possibility that the object to be detected is not included in the candidate for the extracted object in a place where there are many high-temperature objects. On the other hand, in a place where there are few high-temperature objects, the candidate for the object to be extracted may include a background. According to the present invention, it is estimated whether there are many high-temperature objects around the vehicle, and the predetermined ratio S is changed according to the estimation result. Phenomena such as leakage of an object or including a background can be prevented more reliably, and the extraction accuracy of the object can be improved.

  According to an embodiment of the present invention, the estimating means detects a predetermined number or more of objects when a predetermined number or more are detected from the captured image, and when a predetermined number or more of objects are detected. When the target object is detected within a predetermined distance from the vehicle and when at least one of the predetermined type of target object is detected within a predetermined distance from the vehicle is satisfied Is estimated to be a situation where there are many high-temperature objects, otherwise it is assumed that there are not many high-temperature objects, and is not a situation where many high-temperature objects exist. If it is estimated, the predetermined ratio is made lower than in the case where it is estimated that there are many high-temperature objects.

  According to this invention, the situation around the vehicle can be estimated using the object of the captured image. That is, when a predetermined number or more of objects are detected, it can be estimated that there are many high-temperature objects such as pedestrians. Further, if a predetermined number of objects of a predetermined type, for example, predetermined artificial structures (such as street lamps) are detected, it can be estimated that there are many high-temperature objects such as pedestrians. By considering the distance from the vehicle, such a situation can be estimated more accurately. In addition, when it is estimated that there are not many situations where there are many high-temperature objects, the predetermined ratio S is made lower than when it is assumed that there are many situations where there are many high-temperature objects. The value can be divided into two, and the object to be detected can be extracted with good accuracy.

  According to an embodiment of the present invention, the estimation means estimates that the high-temperature object is present when a variance or average of luminance values in a predetermined region of the captured image is equal to or greater than a predetermined value. If not, it is estimated that the high temperature object does not exist in a large amount. If it is estimated that the high temperature object does not exist in a large amount, the high temperature object exists in a large amount. Compared to the case where it is estimated that the predetermined ratio is reduced.

  According to the present invention, the situation around the vehicle can be estimated from the luminance value of the captured image. In other words, if the variance of the luminance value in a predetermined area of the captured image is high, the luminance value varies greatly due to the presence of pedestrians, street lamps, etc. around the vehicle, and thus there are many high-temperature objects. It can be estimated that. In addition, if the average value of the luminance value of a predetermined area of the captured image is high, the luminance value is generally high due to the presence of pedestrians, street lamps, etc. around the vehicle, and thus there are many high-temperature objects. It can be estimated that this is the situation. Furthermore, when it is estimated that there are not many high-temperature objects, the predetermined ratio S is lowered compared to the case where it is estimated that there are many high-temperature objects. The value can be divided into two, and the object to be detected can be extracted with good accuracy.

  According to an embodiment of the present invention, the vehicle includes a detection unit that detects a driving state of the vehicle, and the estimation unit includes a large amount of the high-temperature object according to the driving state of the vehicle detected by the detection unit. Estimate whether the situation.

  According to this invention, the situation around the vehicle can be estimated from the driving state of the vehicle. For example, according to the vehicle speed, it can be estimated whether or not the vehicle is traveling on a highway where there is almost no high-temperature object.

  According to an embodiment of the present invention, the vehicle is provided with detection means for detecting the current position of the vehicle, and the estimation means is acquired for at least one of a road and a region where the current position of the vehicle detected by the detection means exists. It is estimated whether or not there is a large amount of the high-temperature object according to the information.

  According to this invention, the situation around the vehicle can be estimated using the navigation function of the vehicle. For example, whether or not the vehicle is traveling on a highway where there is almost no high-temperature object can be estimated from information about the road at the current position of the vehicle. In addition, whether or not the vehicle is traveling in a relatively populated place where high-temperature objects are likely to exist can be estimated from information on the current location of the vehicle.

  Other features and advantages of the present invention will be apparent from the detailed description that follows.

The block diagram which shows the structure of the periphery monitoring apparatus according to one Example of this invention. The figure for demonstrating the attachment position of the camera according to one Example of this invention. The figure for demonstrating the difference of the mode method and P tile method. The figure for demonstrating the setting of the predetermined ratio in the condition where many high temperature objects exist around the vehicle according to one Example of this invention, and the setting of the predetermined ratio in the condition where there are not many high temperature objects . The flowchart which shows the target object determination process according to one Example of this invention. The figure which shows an example of the area | region of the captured image used for estimating the condition around the vehicle according to one Example of this invention. The block diagram which shows the structure of the periphery monitoring apparatus according to 2nd Example of this invention. The block diagram which shows the structure of the periphery monitoring apparatus according to 3rd Example of this invention. The flowchart of the process for estimating the condition of the periphery of a vehicle according to 3rd Example of this invention.

  Next, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention. The apparatus is mounted on a vehicle and has two infrared cameras 1R and 1L capable of detecting far infrared rays, and an image processing unit 2 for detecting an object around the vehicle based on image data obtained by the cameras 1R and 1L. And a speaker 3 that generates an alarm by voice based on the detection result, and an image obtained by the camera 1R or 1L, and a head-up display that displays a display for allowing the driver to recognize an object around the vehicle (Hereinafter referred to as HUD) 4.

  In this embodiment, as shown in FIG. 2, the cameras 1 </ b> R and 1 </ b> L are arranged at positions symmetric with respect to the central axis passing through the center of the vehicle width at the front of the vehicle 10 so as to capture the front of the vehicle 10. Has been. The two cameras 1R and 1L are fixed to the vehicle so that their optical axes are parallel to each other and their height from the road surface is equal. The infrared cameras 1R and 1L have a characteristic that the level of the output signal becomes higher (that is, the luminance in the captured image becomes higher) as the temperature of the object is higher.

  The image processing unit 2 includes an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, a central processing unit (CPU) that performs various arithmetic processing, and a data RAM (Random Access Memory) used to store data, ROM (Read Only Memory) that stores programs executed by the CPU and data used (including tables and maps), driving signals for the speaker 3, display signals for the HUD 4, etc. Is provided. The output signals of the cameras 1R and 1L are converted into digital signals and input to the CPU. As shown in FIG. 2, the HUD 4 is provided so that a screen 4 a is displayed at a front position of the driver on the front window of the vehicle 10. Thus, the driver can visually recognize the screen displayed on the HUD 4.

  Here, before describing specific embodiments, the basic concept of the present invention will be described with reference to FIGS.

  Referring to FIG. 3, (a) shows a luminance value histogram of one captured image, and (b) shows a luminance value histogram of another captured image. The reason why the brightness value of the captured image is divided is to distinguish the high-temperature object region from the other background regions. As described above, there is a so-called mode method as a method for dividing the luminance value of a captured image into two. As in (a), when the peak 111 corresponding to the background and the peak 113 corresponding to the high temperature object clearly appear, by setting a threshold value TH between the two peaks, It can be extracted well separated from the background.

  However, depending on the situation around the vehicle, there may be a luminance value histogram in which the above two peaks do not appear clearly as shown in (b). In order to cope with such a case, the present invention uses the P tile method.

  As is well known, in the P tile method, a predetermined ratio S (%) is set for the entire frequency of the histogram (the sum of the frequencies from the minimum luminance value L to the maximum luminance value H), and the cumulative frequency is S%. In this method, the luminance value is divided into a high luminance range and a low luminance range in which the cumulative frequency is (100-S)%. In the figure, the high luminance range is indicated by a hatched area A1, and the low luminance range is indicated by an area A2. The area ratio between the region A1 and the region A2 is S: (100-S).

  For example, if the total frequency of the histogram is 1000 (corresponding to the number of pixels of the captured image) and the predetermined ratio S is 30%, the frequency is added from the maximum value H of the luminance value, and the cumulative frequency is , 1000 × S (%), the luminance value at which 300 pieces are reached is defined as a threshold value TH. The luminance value of the captured image is divided into a high luminance range A1 that is equal to or higher than the threshold TH and a low luminance range A2 that is lower than TH. An object having a luminance value in the high luminance range A1 equal to or higher than the threshold value TH is extracted as a high temperature object.

  Thus, if the P tile method is used, the luminance value can be divided into two even for a captured image in which two peaks do not appear. However, if the predetermined ratio (ratio) S is set to a fixed value, it is difficult to extract the desired object by separating it well from the background.

  This point will be described. FIG. 4A shows an example of an image (grayscale image) captured by the camera 1R mounted on the vehicle when traveling on a main road in an urban area. (B) shows an example of an image (grayscale image) captured by the camera 1R mounted on the vehicle when traveling on a general road in the suburbs. Since the infrared camera is used, the higher the temperature of the object, the more white the image is captured.

  As is clear from comparison between the two, when driving in a place such as (a), not only high-temperature objects such as pedestrians 101 but also relatively high temperatures such as other vehicles 103 and street lamps 105 are generated. There are many artificial structures to be presented, and for this reason, many image regions that are white or close to white exist on the captured image. On the other hand, when traveling in a place such as (b), there are almost no high-temperature objects around the vehicle, and in this example, there is only an animal 107. Therefore, compared with (a), there are few image areas close to white or white.

  (C) shows a luminance value histogram of an image taken in a situation where there are many high-temperature objects around the vehicle as shown in (a), and (d) shows the vehicle as shown in (b). The brightness | luminance value histogram of the image imaged in the condition where a high temperature target object hardly exists in the periphery is shown. As is clear by comparing the two, in (c), the frequency is high in the high luminance region as shown by the arrow 121, but in (d), the luminance is high as shown by the arrow 123. The frequency in the area is low.

  In order to satisfactorily separate the high-temperature object from the background, it is desirable to separate only the portion (mountain portion) where the frequency is high in the high luminance region from other portions. Therefore, in (a), the predetermined ratio S according to the P tile method is set to a higher value S2, and in (b), the predetermined ratio S is set to a lower value S1 (<S2). Is preferred. In the figure, the threshold when the predetermined ratio is S2 is indicated by TH2, and the threshold when the predetermined ratio is S1 is indicated by TH1. As described above, the threshold value TH2 is obtained by adding the frequencies from the maximum value H of the luminance values in the luminance value histogram of (c), and the total of the frequencies becomes S2 (%) with respect to the entire frequency. In the luminance value histogram of (d), the threshold value TH1 is obtained by adding the frequencies from the maximum value H of the luminance values, and the total of the frequencies becomes S1 (%) with respect to the overall frequency. Brightness value.

  In (c), an object having a luminance value in the high luminance range A1 (shown by hatching) equal to or higher than the threshold TH2 is extracted as a high temperature object, and in (d), a high luminance range equal to or higher than the threshold TH1. An object having a luminance value within A1 (indicated by diagonal lines) is extracted as a high temperature object.

  If a lower value S1 is set as the predetermined ratio S in the luminance value histogram as shown in (c), the luminance value histogram is bisected by the dotted line 125 in the figure. In this case, a part of the region to be extracted as the high temperature object is out of the high luminance range A1, and thus it is difficult to separate only the high temperature object from the background. Further, when a higher value S2 is set as the predetermined ratio S in the luminance value histogram as shown in (d), it is divided into two by the dotted line 127 in the figure. In this case, the background portion is included in the high luminance range extracted as the high temperature object, and thus it is difficult to separate only the high temperature object from the background.

  Therefore, in the present invention, if the situation around the vehicle is estimated and it is estimated that there are many high-temperature objects as shown in (a), the predetermined ratio S is increased as shown in (c). Set to the value S2. On the other hand, if the situation around the vehicle is estimated and it is estimated that there are not many high-temperature objects as in (b), the predetermined ratio S is set to a lower value S1 as shown in (d). Set to. Since the predetermined ratio S for dividing the luminance value into two according to the situation around the vehicle is changed, the luminance value of the captured image can be divided into two so as to be suitable for the situation around the vehicle. A high temperature object can be well separated from the background and extracted.

  In the following embodiments, as shown in FIGS. 4A and 4C, a situation where a high value S2 should be set to the predetermined ratio S, that is, a situation where there are many high-temperature objects around the vehicle. Is the first situation, as shown in FIGS. 4B and 4D, a situation where a lower value S1 should be set to the predetermined ratio S, that is, there are not many high-temperature objects around the vehicle. The situation is called the second situation.

  FIG. 5 is a flowchart illustrating a process performed by the image processing unit 2 according to one embodiment of the present invention. The process is performed at predetermined time intervals.

  In steps S11 to S13, the output signals of the cameras 1R and 1L (that is, captured image data) are received as input, A / D converted, and stored in the image memory. The stored image data is a gray scale image including luminance information.

  In step S14, a luminance value histogram of a gray scale image (which may be a right image or alternatively a left image) is created. In step S15, the situation around the vehicle is estimated. Details of this estimation method will be described later.

  If the first situation is estimated as a result of estimation in step S16, a predetermined ratio S used in the P tile method is set to a predetermined value S2 in step S17. If the above-described second situation is estimated, in step S18, the predetermined ratio S is set to a predetermined value S1. Here, S1 <S2. Thus, when it is estimated that the first situation, that is, a situation where there are many high-temperature objects around the vehicle, the predetermined ratio S is set to a higher value S2, and in the second situation that is not the case, the predetermined ratio S is set. S is set to a lower value S1. Therefore, in the first situation, a living body such as a pedestrian or an animal is prevented from being classified as a background outside the extracted high luminance range, and in the second situation, the background is extracted. It can prevent more reliably that it will be included in the high-intensity range. As a result, the luminance value can be divided into two so as to be suitable for the situation around the vehicle.

  In step S21, the right image is used as a reference image (alternatively, the left image may be used as a reference image), and the binarization of the image signal is performed by the P tile method using a predetermined ratio S. When the predetermined value S1 is set to the predetermined ratio S (%), N × S1 (%) luminances from the maximum luminance value H on the histogram with respect to the total frequency N of the luminance value histogram. The value pixel is “1” (white), and the other luminance value pixels are “0” (black). In other words, as described above, the threshold value TH1 corresponding to the predetermined ratio S1 is obtained, and a pixel having a luminance value equal to or higher than the threshold value TH1 is set to “1” (white), and a pixel having a luminance value lower than the threshold value TH1 is set. Is “black” (0). By this binarization processing, an object having a temperature equal to or higher than the threshold value TH1 is extracted as a white region. Even when the predetermined value S2 is set to the predetermined ratio S, binarization is performed by the same method.

  In step S22, the binarized image data is converted into run-length data. Specifically, with respect to an area that has become white due to binarization, the coordinates of the start point (the leftmost pixel of each line) of the white area (referred to as a line) of each pixel row and the end point (each The run length data is represented by the length (expressed by the number of pixels) up to the pixel on the right end of the line. For example, if the white region in the pixel row whose y coordinate is y1 is a line from (x1, y1) to (x3, y1), this line consists of three pixels, so (x1, y1, 3) This is represented by run-length data.

  In steps S23 and S24, the object is labeled and the object is extracted. That is, of the lines converted into run length data, a line having a portion overlapping in the y direction is regarded as one object, and a label is given thereto. In this way, one or a plurality of object candidates are extracted.

  In step S25, an object determination process for determining whether or not the extracted object candidate is an object to be detected is executed. Specifically, it is determined whether the extracted object is a pedestrian, an animal, or an artificial structure. These determinations can be made using any known technique. For example, pedestrian determination processing is described in Japanese Patent Application Laid-Open No. 2007-241740, Japanese Patent Application Laid-Open No. 2007-334751, and the like. For example, the animal determination processing is described in JP2007-310705A, JP2007-310706A, and the like. For example, the artificial structure determination processing is described in Japanese Unexamined Patent Application Publication No. 2007-310705, Japanese Unexamined Patent Application Publication No. 2008-276787, and the like.

  In step S26, if the object determined in step S25 is an object to be detected, a notification (alarm) is made. In this embodiment, the object to be detected is a living body such as a pedestrian or an animal. Therefore, when it is determined that the object is a living body such as a pedestrian or an animal, a report is output to the driver. The notification can be output in an arbitrary form. For example, the captured image is displayed on the display screen 4a of the HUD 4, and the object captured on the captured image is highlighted (for example, the object is framed). (Displayed by enclosing). Further, in addition to such highlighting or instead of highlighting, the driver may be informed through the speaker 3 that an object is present. On the other hand, if the object determined in step S25 is an artificial structure, it is not a warning object, and the process is exited without notification.

  In addition to the above processing, for a plurality of past captured images, the same object is tracked (tracked), and the distance to the object is calculated based on the two captured images of the cameras 1R and 1L. You may make it determine whether a target object exists in the predetermined range around a vehicle, and whether there exists a possibility of entering the predetermined range. Details of these processes are described in, for example, JP-A-2001-006096.

  Hereinafter, several embodiments of the process for estimating the situation around the vehicle, which is performed in step S15, will be described.

In the first embodiment, this estimation is performed using the captured image (grayscale image) acquired in step S13. When a captured image is used, the following two forms can be considered.
1) Form using brightness value of captured image 2) Form using target object captured in captured image

  The above 1) will be described. As is clear from comparison between FIGS. 4A and 4B, in the captured image of FIG. 4A, there are various objects at higher temperatures than the background around the left and right sides of the vehicle. On the other hand, in the captured image of (b), such a dot is not seen around the left and right sides of the vehicle and is almost a background portion.

  Therefore, as shown in FIGS. 6A and 6B, a predetermined area of each captured image in FIGS. 4A and 4B, preferably a predetermined area 131L set at the left and right ends of the captured image, and The variance of the luminance value of 131R is calculated. Under the situation as shown in (a), since various objects are interspersed as described above, the variation in the luminance values of the regions 131L and 131R, that is, the variance becomes high. On the other hand, in the situation as shown in (b), since the object is hardly present around the vehicle, the luminance values of the regions 131L and 131R, that is, the dispersion, becomes low.

  Therefore, in one example, the variance (which may be a standard deviation) of the luminance values of the predetermined regions 131L and 131R of the captured image is calculated, and if the variance value is higher than the predetermined value, the first situation is estimated, and less than the predetermined value If so, the second situation is estimated.

  Instead of the variance, an average value may be used. The areas 131L and 131R in FIG. 6A include a lot of high luminance depending on the objects present around the vehicle, whereas the areas 131L and 131R in FIG. This is because there is little inclusion of high luminance.

  Therefore, in another example, the average value of the luminance values of the predetermined regions 131L and 131R at the left and right ends of the captured image is calculated, and if the average value is higher than the predetermined value, the first situation is estimated, and below the predetermined value If there is, the second situation is estimated.

  Here, the predetermined areas 131L and 131R are preferably set at the left and right ends of the captured image as shown in FIG. When the vehicle is traveling on a road, in a city area such as (a), there are artificial structures such as pedestrians and buildings on the left and right of the road. On the other hand, since the center area of the captured image is an area in front of the road, these objects may not be captured. Therefore, the predetermined areas 131L and 131R are preferably set at the left and right ends of the captured image so that these objects existing on the left and right of the road are imaged. Furthermore, the horizontal widths of the predetermined areas 131L and 131R are preferably set so that the objects present on the left and right sides of the road are imaged.

  Further, if the predetermined area is extended downward, the predetermined area may include a road. Furthermore, when there is no tall building, if the predetermined area is extended upward, there is a possibility that the predetermined area includes a sky. If roads and sky parts are included in the predetermined area, the variance and average may be calculated low even in urban areas such as (a). To prevent this, the vertical length of the predetermined area extends from a distance from the bottom of the image to a distance from the top of the image so as not to include roads and sky. It should be set to do.

  Next, the above 2) will be described. As shown in FIG. 4 (a), an image taken in a situation where many high-temperature objects exist in the vicinity is compared with a taken image as shown in FIG. 4 (b). The number of objects to be extracted is large. Focusing on this point, in one example, the situation around the vehicle is estimated from the number of extracted objects.

  Specifically, at the time when step S15 is executed, as described above, the object extraction process has not yet been performed on the image captured this time. On the other hand, since the process of FIG. 5 reports an object in real time, its execution cycle (period) is set to be relatively short (for example, 100 milliseconds), so the current captured image and the previous captured image The difference is small. Therefore, the previous captured image can be used.

  That is, when the number of objects extracted in step S24 is greater than or equal to a predetermined value at the time when the process of FIG. 5 is executed in the previous cycle, the first situation is estimated. Estimate the situation. Alternatively, the object extraction results for a predetermined number of captured images that are continuously captured until the previous cycle may be used. For example, an average value of the number of object extractions per captured image is calculated. If the average value is equal to or greater than a predetermined value, the first situation is estimated. If the average value is smaller than the predetermined value, the second situation is estimated. To do.

  In another example, the situation around the vehicle is estimated from the number of objects of a predetermined type. In one example, an artificial structure is used as the predetermined type of object. As shown in FIG. 4 (a), a situation where many high-temperature objects exist around the vehicle occurs in a place where a certain population or more exists, and therefore, occurs in a place where the number of artificial structures is large. Can do. Therefore, the situation around the vehicle is estimated according to the number of artificial structures. Specifically, since the determination process of the artificial structure is performed in step S25 of FIG. 5, the artificial structure has not yet been determined for the current captured image. Therefore, the previous captured image is used. If the number of the artificial structures determined in step S25 at the time of executing the process of FIG. 5 in the previous cycle is greater than or equal to a predetermined value, the first situation is estimated, and if it is less than the predetermined value, the second situation is estimated. Estimated. Alternatively, an artificial structure determination result for a predetermined number of captured images that are continuously captured until the previous cycle may be used. For example, an average value of the number of artificial structures per captured image is calculated. If the average value is equal to or greater than a predetermined value, the first situation is estimated. If the average value is smaller than the predetermined value, the second situation is calculated. presume.

  Here, you may further specify the kind of artificial structure used as the object which counts. The first or second situation may be estimated depending on whether or not there are a predetermined number or more of the specific type of artificial structure. The specific type is selected as a type that exists where there are many high-temperature objects. For example, in an urban area, there are many illuminating objects (including street lamps, signboard lights, and building lights) having a predetermined height or more, and this can be a specific type of artificial structure. The number of types to be specified may be one or plural. In the case of a plurality of cases, the first or second situation may be estimated according to whether the number of any type of artificial structures is equal to or greater than a predetermined number, and whether the number of all types of artificial structures is equal to or greater than the predetermined number. Depending on, the first or second situation may be estimated.

  The determination of the type of artificial structure can be included in the object determination process in step S25. Typically, the type of artificial structure is specified from the viewpoint of shape and size. For example, well-known shape matching is performed using a template that models the shape of the specific type of artificial structure, and the size of the extracted target in real space is the specific type of artificial structure. It is judged whether it is within the range corresponding to. If the degree of similarity (degree of correlation) as a result of shape matching is higher than a predetermined value and the calculated size is within a range corresponding to the specific type of artificial structure, the object is It is determined that it is a specific type of artificial structure. Alternatively, the particular type of artificial structure may be determined based on either shape and size.

  In another example, a pedestrian is used as the predetermined type of object. This is because a large number of pedestrians indicates a situation in which many high-temperature objects exist in the vicinity. In this case, if the number of pedestrians determined in step S25 is greater than or equal to a predetermined value when the previous captured image is used and the process of FIG. 5 is executed in the previous cycle, the first situation is estimated and the predetermined If it is less than the value, the second situation is estimated. As in the case of an artificial structure, pedestrian determination results for a predetermined number of captured images that are continuously captured until the previous cycle may be used.

  Note that the number of objects or the number of objects of a predetermined type may be counted for objects existing within a predetermined distance value from the vehicle. By doing so, the situation in the vicinity of the vehicle can be more reliably determined. Here, in this embodiment, since the pair of cameras 1R and 1L are used in this embodiment as shown in FIG. 1, the distance from the vehicle can be calculated using a known triangulation method.

  In the first embodiment described above, the situation around the vehicle is estimated using various conditions, but one of these conditions may be used, or a plurality of arbitrary conditions may be used. May be. In the latter case, if any one of the conditions is satisfied, or if any of a plurality of conditions are satisfied, the first situation is estimated. Otherwise, the second situation may be determined. Good.

  Next, a second embodiment will be described. FIG. 7 shows a block diagram of a vehicle periphery monitoring device based on the second embodiment. The difference from FIG. 1 is that driving state detection means 21 for detecting the driving state of the vehicle is provided, and the driving state detected by the driving state detection means 21 is input to the image processing unit 2. The process of FIG. 5 is similarly applied to the second embodiment. In this case, in step S15, the situation around the vehicle is estimated using the detected driving state.

  In one example, the driving state detection means 21 is a means for detecting the vehicle speed, and can be realized by a vehicle speed sensor, for example. For example, since a speed limit is normally provided on a highway, the vehicle speed is detected at predetermined time intervals. If the vehicle speed is equal to or higher than the speed limit and continues for a predetermined time, the vehicle travels on the highway. Can be estimated. On highways, there are almost no hot objects around the vehicle. Therefore, when it is estimated that it is a highway, it is estimated as the second situation, and when it is estimated that it is not a highway, it is estimated as the first situation.

  On the other hand, when traveling in an urban area as shown in FIG. 4 (a), the vehicle stops more frequently due to the presence of traffic lights or pedestrians than when traveling in the suburbs as shown in FIG. 4 (b). Or drive at low speed. The urban area is a place where many high-temperature objects exist. Therefore, the vehicle speed is detected at predetermined time intervals, the average vehicle speed for a predetermined time is calculated based on the detected vehicle speed, and when the average vehicle speed is equal to or less than the predetermined value, the first situation is estimated, If it is larger than the value, the second situation may be estimated.

  In another example, the driving state detection means 21 is a means for detecting the steering angle of the vehicle, and can be realized by a steering angle sensor, for example. When traveling in an urban area as shown in FIG. 4 (a), steering of the steering wheel is performed more frequently than when traveling in a suburb as shown in FIG. 4 (b). Therefore, the rudder angle is detected at predetermined time intervals, the frequency at which the rudder angle changes with a magnitude greater than or equal to a predetermined value over a predetermined time is counted, and if the frequency is equal to or greater than the predetermined value, the first situation is estimated. If it is smaller than the predetermined value, the second situation can be estimated.

  In still another example, the driving state detection means 21 is a means for detecting the yaw angle of the vehicle (the rotation angle of the vehicle center of gravity about the vertical axis), and can be realized by a yaw rate sensor, for example. The urban area as shown in FIG. 4 (a) has more corners than the case where the vehicle is traveling in the suburbs as shown in FIG. 4 (b). Therefore, the yaw angle is detected at predetermined time intervals, and the frequency at which the yaw angle changes over a predetermined value over a predetermined time is counted. If the frequency exceeds the predetermined value, the first situation is estimated. If it is smaller than the predetermined value, the second situation can be estimated.

  Next, a third embodiment will be described. FIG. 8 shows a block diagram of a vehicle periphery monitoring device based on the third embodiment. The difference from FIG. 1 is that the navigation unit 25 is connected to the image processing unit 2.

  The navigation unit 25 receives a GPS signal for measuring the position of the vehicle 10 using, for example, an artificial satellite, and detects the current position of the vehicle 10 based on the GPS signal. The navigation unit 25 includes a map information storage unit 27 that stores map information. The map information storage unit 27 can be realized in any storage device (such as a recording medium or a hard disk drive). The navigation unit 25 can read the map information stored in the map information storage unit 27 and display it on the display screen of a predetermined display device (or HUD 4).

  The process of FIG. 5 is similarly applied to the third embodiment. In this case, in step S15, the current position of the vehicle detected via the navigation unit 25 is obtained and the current position exists. Information on at least one of a road and a region is acquired, and the situation around the vehicle is estimated using the information.

  First, a case where the situation around the vehicle is estimated using information on the road will be described. In this embodiment, road type information is used as information about roads. The type of road is preset as attribute information of each road in the map information of the navigation unit 25. For example, the road type information of the map information indicates an expressway, a city expressway, a toll road, a general road (national road, prefectural road, etc.). Therefore, it is possible to determine whether the road is an expressway by acquiring the road type information of the road where the vehicle is currently traveling from the map information.

  Alternatively, when road type information is not set in the map information, information indicating whether each road is a highway is set in a table separate from the map information (for example, whether the road is a highway or not). The flag shown can be set in association with the name of each road), and the table can be stored in a storage device such as a memory. By acquiring the name of the road where the vehicle is currently traveling from the map information and referring to the table based on the acquired name, it is possible to determine whether the road is an expressway.

  As described above, the highway is a situation in which there are almost no high-temperature objects in the vicinity. Therefore, if the highway is determined to be a highway, it is estimated as the second situation; Estimate the situation.

  Next, the case where the surrounding situation of a vehicle is estimated using the information regarding a region is described. As information regarding the area, information indicating whether or not it is “town” is used. Here, the “town” is a place where a certain number of populations exist. A “city” can be defined as an area that meets any suitable condition based on any information source. For example, in the case of Japan, it is determined whether the city is a “city” in units of municipalities. If the population density is equal to or higher than a predetermined value, it is classified as “city”. If the population density is lower than the predetermined value, “city” is determined. It can be classified. In addition to the population density, you may consider the inflowing population for working or going to school in that area. For example, even if the artificial density is smaller than the predetermined value, if the inflowing population is equal to or higher than the predetermined value, there are relatively many people such as companies in the area, and therefore it can be classified as “town”. . In general, the greater the number of buildings such as buildings and houses, the more people there are, so if the number of buildings is greater than or equal to the predetermined value, classify it as “City”, otherwise classify it as other than “City”. You may make it do.

  Alternatively, for certain animals (for example, certain large animals such as deer), information that they have appeared in the past, habitat information, and signs indicating the likelihood of these animals appearing Based on location information, etc., areas where animals may appear may be identified, and these areas may be classified as “towns”, and other areas may be classified as “towns”. .

  In this way, information indicating whether or not “city” is associated with each city, and can be stored in advance in map information (for example, a flag indicating whether or not “city” is set for the place name). Set). Alternatively, information (may be a flag as described above) indicating whether each city is a city may be set in a table separate from the map information and stored in a memory or the like. By referring to the map information or the table, it is possible to determine whether or not the place where the vehicle is currently traveling is classified as “town”.

  A “town” is a place where a certain amount of population exists, and thus a place where many high-temperature objects exist. Therefore, if it is determined that the area of the current position of the vehicle is “town”, the first situation is estimated, and if not, the second situation is estimated.

  Note that the user may be able to register information indicating the type of road and information indicating whether it is “town” as described above. For example, on the map information displayed on the display device, the user designates a municipality to be classified as “town” via, for example, a touch panel, and the image processing unit 2 displays the designated area. On the other hand, a flag indicating “town” can be set and stored in the map information.

  Moreover, you may make it acquire these information in real time using the communication function of the navigation unit 25. FIG. For example, the vehicle communicates with a predetermined computer (server or the like), and the server stores road type information and information indicating whether it is “town” for each road and each municipality. The vehicle transmits the current position of the vehicle to the server, and the server can inform the vehicle of the information set for the current position. Alternatively, a system that provides traffic information to the vehicle such as an existing VICS system may be used, and the information may be transmitted to the vehicle as part of the traffic information.

  Here, with reference to FIG. 9, the flow of the process for estimating the situation around the vehicle using the navigation function executed in step S15 will be described.

  In step S31, the current position of the vehicle is acquired via the navigation unit 25. In step S32, the type of the road at the current position is acquired from the map information of the navigation unit 25. As described above, in the map information of the navigation unit 25, information indicating the type of road is set as attribute information for each road.

  In step S33, it is determined whether the road at the current position is an expressway based on the acquired road type information. If it is an expressway (Yes in step S33), a value 1 is set in the flag Load_flag in step S34, and if it is not a highway (No in step S33), a value 2 is set in the flag Load_flag in step S35.

  In step S36, information indicating whether or not the “town” is set for the area to which the current position belongs is acquired from the map information of the navigation unit 25. As described above, in the map information, information indicating whether it is “town” is set in units such as municipalities.

  In step S <b> 37, it is determined whether the area at the current position is classified as “city” based on the acquired information indicating whether it is “city”. If it is determined that it is “town” (Yes in step S37), the flag Town_flag is set to 1 in step S38. If it is determined that it is not “city” (step S37 is No), step S39 is performed. The value 2 is set in the flag Town_flag.

  In step S <b> 40, it is determined whether Load_flag has a value of 1 or Town_flag has a value of 2. If Load_flag has a value of 1 or Town_flag has a value of 2, it indicates that the current position of the vehicle is on a highway or in an area that is not a “city”. Since this indicates a situation where there are not many high-temperature objects, it is estimated as a second situation in step S41. On the other hand, when the determination in step S40 is No, the load_flag is the value 2 or the Town_flag is the value 1, which indicates that the current position of the vehicle is not a highway or “town” It indicates that it exists in a certain area. This indicates a situation where there are many high-temperature objects, so that it is estimated as the first situation in step S42.

  In the first to third embodiments described above, the two values S1 and S2 are used as the predetermined ratio S. However, the present invention is not limited to such a form, and depends on the situation around the vehicle. The predetermined ratio S may be gradually changed between the values S1 and S2.

  For example, in the first embodiment, the predetermined ratio S is gradually changed from the lower value S1 to the higher value S2 as the luminance value variance or average value of the predetermined region of the captured image increases. And vice versa. In the first embodiment, the predetermined ratio S is gradually changed from the lower value S1 to the higher value S2 as the number of extracted objects or the number of objects of a predetermined type increases. And vice versa.

  Further, in the first to third embodiments described above, the predetermined ratio S is set between two values S1 and S2 depending on whether one of the first and second situations is estimated. Although the switching is performed, the predetermined ratio S may be changed gradually instead of changing between the values S1 and S2 at the time of the switching.

  For example, when the second situation is estimated in the previous cycle and the first situation is estimated in the current cycle, in the current cycle, the predetermined value α (however, A value obtained by adding | α | <| S2−S1 |) is set as a current value of a predetermined ratio S. Next, over the period until the second situation is estimated, the increment value α is added to the previous value of the predetermined ratio S for each cycle with the higher value S2 as the upper limit value. Here, the increment value α may be constant for each cycle or may not be constant.

  Similarly, when the first situation is estimated in the previous cycle and the second situation is estimated in the current cycle, in the current cycle, a predetermined decrement value α ( A value having a negative value and adding | α | <| S2−S1 |) is set as a current value of a predetermined ratio S. Next, over the period until the first situation is estimated, the decrement value α is added to the previous value of the predetermined ratio S for each cycle with the lower value S1 as the lower limit value. Here, the decrement value α may be constant for each cycle or may not be constant.

  In this way, it is possible to suppress a rapid change in the threshold value that bisects the luminance value. In particular, when switching from the value S1 to S2 at a time, the ratio of the area extracted as a high-temperature object increases at a time, so depending on the situation around the vehicle, the extracted high luminance range includes the background. There is a risk of it. Such a situation can be avoided by switching as described above.

In the embodiment described above, the steps S14 to S18 for updating the predetermined ratio S are performed as part of the process of FIG. 5, but alternatively, the steps S14 to S18 are different from the other steps. It may be a separate process. In this case, the update cycle of the predetermined ratio S may be made variable by changing the execution cycle of the processes in steps S14 to S18.
For example, in a place such as an urban area where many high temperature objects exist, sudden appearance and disappearance of high temperature objects may frequently occur. In order to deal with such a high-temperature object, it is preferable to shorten the update cycle of the predetermined ratio S while the first situation is estimated.

  On the other hand, depending on the size of the high-temperature object existing in the vicinity, the area where the high-temperature object is imaged increases as the image is captured so as to approach the vehicle. Therefore, for example, when the first or second situation is estimated based on the luminance value of the first embodiment, if the predetermined ratio S is updated in a short period, the predetermined ratio S may be changed from a lower value S1 to a higher value S2, or may increase toward the higher value S2. If the higher value S2 is used in a situation where the lower value S1 should be set, as described with reference to FIG. 4, the extracted high luminance range may include the background. In order to prevent this, it is preferable to lengthen the update cycle of the predetermined ratio S while the second situation is estimated.

  Therefore, in one example, when a higher value S2 is set to the predetermined ratio S, the update cycle is shortened, and when a lower value S1 is set, the update cycle is made longer than that. Can do. Alternatively, the update period may be gradually increased as the predetermined ratio S moves from the higher value S2 to the lower value S1. In either case, the shortest period can be set to be the same as the process period of FIG.

  In the above embodiment, as shown in FIG. 1, the configuration uses two infrared cameras. However, the present invention is also applicable to an image captured using one infrared camera. . In the case of one camera, the distance value to the object may be detected using, for example, a radar, or the distance from the position where the object is imaged in the captured image of the camera to the object. You may make it estimate.

  Further, in the process of FIG. 5, a black and white image is generated by binarizing the image in step S21 using the predetermined ratio S set as described above. The generation itself is not necessarily performed. That is, only pixels belonging to the high luminance range defined by the predetermined ratio S are directly extracted from the grayscale image, and based on these pixels. Step S22 and subsequent steps may be performed.

  As described above, specific embodiments of the present invention have been described. However, the present invention is not limited to these embodiments.

1R, 1L infrared camera (imaging means)
2 Image processing unit 3 Speaker 4 Head-up display

Claims (4)

An infrared camera mounted on the vehicle,
A luminance value histogram of a captured image captured by the infrared camera is created, and the frequency ratio, which is the ratio of the cumulative frequency calculated from the high luminance side in the luminance value histogram to the total frequency, is a predetermined value or less. A value range is a high luminance range, a luminance value range in which the frequency ratio exceeds the predetermined value is a low luminance range, and an object having a luminance value included in the high luminance range is extracted as a candidate for an object to be detected. Means to
Determination means for determining whether the extracted object candidate is the object to be detected;
A vehicle periphery monitoring device comprising: means for notifying a driver when the object to be detected is determined;
Estimating means for estimating whether there are many high-temperature objects around the vehicle;
Means for changing the predetermined ratio according to an estimation result as to whether or not there is a large amount of the high-temperature object;
With
The estimating means detects a predetermined distance from the vehicle when a predetermined number of objects are detected from the captured image, when a predetermined number of objects are detected. A large amount of the high-temperature object is present when at least one of a predetermined number of objects is detected within a predetermined distance from the vehicle. If not, it is assumed that there are not many high-temperature objects,
If it is estimated that the high-temperature object is not present in a large amount, the predetermined ratio is reduced as compared with the case where it is estimated that the high-temperature object is present in a large amount.
Car both periphery monitoring device. An infrared camera mounted on the vehicle,
A luminance value histogram of a captured image captured by the infrared camera is created, and the frequency ratio, which is the ratio of the cumulative frequency calculated from the high luminance side in the luminance value histogram to the total frequency, is a predetermined value or less. A value range is a high luminance range, a luminance value range in which the frequency ratio exceeds the predetermined value is a low luminance range, and an object having a luminance value included in the high luminance range is extracted as a candidate for an object to be detected. Means to
Determination means for determining whether the extracted object candidate is the object to be detected;
A vehicle periphery monitoring device comprising: means for notifying a driver when the object to be detected is determined;
Estimating means for estimating whether there are many high-temperature objects around the vehicle;
Means for changing the predetermined ratio according to an estimation result as to whether or not there is a large amount of the high-temperature object;
With
When the variance or average of luminance values in a predetermined area of the captured image is equal to or greater than a predetermined value, the estimation means estimates that the high-temperature object is present, and otherwise, Estimating that there are not many high-temperature objects,
If it is estimated that the high-temperature object is not present in a large amount, the predetermined ratio is reduced as compared with the case where it is estimated that the high-temperature object is present in a large amount.
Car both periphery monitoring device. An infrared camera mounted on the vehicle,
A luminance value histogram of a captured image captured by the infrared camera is created, and the frequency ratio, which is the ratio of the cumulative frequency calculated from the high luminance side in the luminance value histogram to the total frequency, is a predetermined value or less. A value range is a high luminance range, a luminance value range in which the frequency ratio exceeds the predetermined value is a low luminance range, and an object having a luminance value included in the high luminance range is extracted as a candidate for an object to be detected. Means to
Determination means for determining whether the extracted object candidate is the object to be detected;
A vehicle periphery monitoring device comprising: means for notifying a driver when the object to be detected is determined;
Estimating means for estimating whether there are many high-temperature objects around the vehicle;
Means for changing the predetermined ratio according to an estimation result as to whether or not there is a large amount of the high-temperature object;
Detecting means for detecting a driving state of the vehicle ;
With
The estimation means estimates whether or not the high-temperature object is present depending on the driving state of the vehicle detected by the detection means.
Car both periphery monitoring device. An infrared camera mounted on the vehicle,
A luminance value histogram of a captured image captured by the infrared camera is created, and the frequency ratio, which is the ratio of the cumulative frequency calculated from the high luminance side in the luminance value histogram to the total frequency, is a predetermined value or less. A value range is a high luminance range, a luminance value range in which the frequency ratio exceeds the predetermined value is a low luminance range, and an object having a luminance value included in the high luminance range is extracted as a candidate for an object to be detected. Means to
Determination means for determining whether the extracted object candidate is the object to be detected;
A vehicle periphery monitoring device comprising: means for notifying a driver when the object to be detected is determined;
Estimating means for estimating whether there are many high-temperature objects around the vehicle;
Means for changing the predetermined ratio according to an estimation result as to whether or not there is a large amount of the high-temperature object;
Detecting means for detecting a current position of the vehicle ;
With
The estimation means estimates whether or not the high-temperature object is present according to information acquired for at least one of a road and a region where the current position of the vehicle detected by the detection means exists. To
Car both periphery monitoring device.