JP3912358B2 - Threshold setting device and threshold setting method - Google Patents

Description

  The present invention relates to a threshold setting device and a threshold setting method mounted on a vehicle.

  Conventionally, a threshold setting device mounted on a vehicle is known (for example, Patent Document 1).

  In the technique described in Patent Document 1, a front image of a vehicle is captured to obtain a thermal image (an image having a thermal data value in front of the vehicle). Based on the thermal image, the thermal data value and the frequency of the thermal data value are calculated. Create a histogram showing the relationship. And the threshold which shows the range of the thermal data value which a person has is set with reference to the maximum point which the said histogram has. Specifically, the threshold is set with reference to the maximum point having the highest corresponding thermal data value among the maximum points of the histogram. Note that the threshold can be set in this way because the histogram usually has two local maximum points, and the local maximum point corresponding to the higher thermal data value corresponds to a normal person. Because it does.

Then, a person area having a thermal data value within the range indicated by the threshold is extracted from the thermal image, and a person is detected from the extracted person area.
Japanese Patent Laid-Open No. 2003-16458

  However, in the technique, among the local maximum points of the histogram, the local maximum point referred to corresponds to a person (in other words, the thermal data value corresponding to the local maximum point matches the thermal data value of the person). Since the threshold value can be accurately set, the person region may not be accurately extracted depending on the number of local maximum points and the thermal data value corresponding to the local maximum point.

  Specifically, there are cases where a person cannot be accurately extracted when the histogram has only one local maximum point or has three or more local points. This is because the referenced local maximum point may not correspond to a person.

  Further, even if the histogram has two local maximum points, the local maximum point referred to may not correspond to a person. Therefore, even if the histogram has two local maximum points, it may not be possible to accurately extract a person.

  In addition, in order to specify the range of the thermal data value possessed by a person, it is necessary to set an upper limit and a lower limit threshold. However, the threshold width (difference between the lower limit threshold and the upper limit threshold) is set only at the maximum point. Since it cannot be specified, it was necessary to set the width in advance. For this reason, depending on the shape of the histogram, the range indicated by the threshold is too narrow to extract the person area sufficiently, or the range indicated by the threshold is too wide to extract extra person areas. was there. Therefore, even in this respect, the person region may not be extracted accurately.

  In the technique, a threshold is set based on the image every time one image is acquired. For this reason, if the portion of the histogram that does not correspond to a person has a maximum point, the threshold value is set outside the range of the thermal data value that the person has even if the threshold value has been set appropriately. . Therefore, even in this respect, the person region may not be extracted accurately.

  The present invention has been made to solve such a conventional problem, and the main object of the present invention is to provide a threshold setting device and a threshold setting method capable of extracting a person region more accurately. That is.

  In order to achieve the above object, the invention described in the claims of the present application includes, in a threshold setting device mounted on a vehicle, an image acquisition unit that acquires an image having a state quantity data value around the vehicle, and an image acquisition unit. Based on the acquired image, based on the frequency distribution graph created by the graph creating means for creating the frequency distribution graph indicating the relationship between the state quantity data value and the frequency of the state quantity data value, and the graph creating means, Based on the feature amount calculating means for calculating the feature amount including the maximum point and the inflection point of the frequency distribution graph, the feature amount calculated by the feature amount calculating means, and the surrounding environment of the vehicle, a predetermined detection object is detected. Threshold value setting means for setting a threshold value indicating the range of the state quantity data value, and state quantity data within the range indicated by the threshold value set by the threshold setting means from the image acquired by the image acquisition means A region extracting means for extracting a detection subject region having a value, in that it comprises the main features.

  In the invention described in the claims of the present application, the following effects can be mainly obtained. In other words, in the present invention, the maximum point and the inflection point are calculated as the feature amount of the histogram. Then, a threshold is set based on the feature amount. Accordingly, since the feature amount used for setting the threshold value is increased as compared with the prior art, it is possible to set a threshold value that is more appropriate and accurate than the prior art.

  Specifically, among the local maximum points of the histogram, even if the local maximum point corresponding to the highest thermal data value does not correspond to the person, other feature amounts, that is, feature amounts corresponding to the person from the inflection points And a threshold value can be set using the selected feature amount. Therefore, it is possible to set a threshold value that is more appropriate and accurate than in the past. Therefore, the person area can be extracted more accurately.

(First embodiment)
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, a first embodiment of the invention will be described with reference to the drawings. First, based on FIG. 1 to FIG. 9, components of the threshold setting device 1 according to an embodiment of the present invention and main functions of each component will be described.

  FIG. 1 is a block diagram showing the configuration of the threshold setting device 1, and FIG. 2 is a side view and a plan view showing the installation position of the camera 10. As shown in FIGS. 1 to 2, the threshold setting device 1 is mounted on a vehicle P, and includes a camera (image acquisition unit) 10, an image memory 11, a histogram creation unit (graph creation unit) 12, and a histogram feature amount. A calculation unit (hereinafter referred to as “feature amount calculation unit”) (feature amount calculation means) 13, an initial setting unit using environment information (hereinafter referred to as “initial setting unit”) 14, and a threshold setting unit (threshold value) A setting unit) 15, a histogram / threshold storage memory (hereinafter referred to as “storage memory”) 16, and a region extraction unit (region extraction unit) 17.

  As shown in FIG. 2, the camera 10 is installed at the front of the vehicle P so that its optical axis is parallel to the traveling direction of the vehicle P. Then, the front of the vehicle P is photographed, and a thermal image A having a thermal data value in front of the vehicle P is acquired in the form of thermal image data. The installation position and shooting direction of the camera 10 are not limited to those shown in FIG. 2, but may be other installation positions and shooting directions.

  An example of the thermal image A is shown in FIG. FIG. 3 shows a thermal image A acquired by the camera 10 when there are two persons, a vehicle, and a power pole in front of the vehicle P. The thermal image A includes person images A1 to A2, vehicle image A3, and utility pole images A4 to A6. This thermal image data is composed of a plurality of pixel data, and each pixel data includes coordinate data and thermal data.

  In addition, the camera 10 outputs the thermal image data to the image memory 11. The image memory 11 stores thermal image data given from the camera 10.

  The histogram creation unit 12 obtains thermal image data from the image memory 11, and creates a histogram indicating the relationship between the thermal data value and the frequency (frequency) of the thermal data value based on the acquired thermal image data. . FIG. 4 shows a histogram 120 as an example of the histogram. In FIG. 4, the horizontal axis represents the thermal data value.

  Then, the histogram creation unit 12 generates histogram data related to the created histogram and outputs it to the feature amount calculation unit 13.

  The feature amount calculation unit 13 calculates the maximum point (maximum point), minimum point, and inflection point of the histogram as the feature amount based on the histogram data given from the histogram creation unit 12. Specifically, the maximum point and the minimum point are calculated by first-order differentiation of the histogram, and the inflection point is calculated by second-order differentiation. Note that a point at which the rate of change in frequency is greater than or equal to a reference value (or less than the reference value) may be used as the feature amount. For example, when the histogram data indicates the histogram 120 shown in FIG. 4, the feature amount calculation unit 13 calculates the maximum point 121, the minimum point 122, and the inflection point 123 of the histogram 120 as feature amounts as shown in FIG. To do.

  Then, the feature amount calculation unit 13 generates feature amount data related to the calculated feature amount and outputs the feature amount data to the threshold setting unit 15.

  The initial setting unit 14 includes a sensor (not shown) and stores a threshold map indicating the relationship between the initial threshold and the surrounding environment. Then, the surrounding environment of the vehicle P (for example, the current time and the temperature around the vehicle P) is detected using the sensor, and an initial threshold is set based on the detected surrounding environment and the threshold map. The initial threshold value includes a first initial threshold value indicating an upper limit value of the thermal data value possessed by the person's face, a lower limit value of the thermal data value possessed by the person's face portion, and an upper limit value of the thermal data value possessed by the person's body. And a third initial threshold value indicating the lower limit value of the thermal data value of the human body. The initial setting unit 14 may set the initial threshold based on the surrounding environment and a relational expression between the initial threshold and the surrounding environment instead of the surrounding environment and the threshold map.

  Then, the initial setting unit 14 generates initial setting data regarding the set initial threshold and outputs the initial setting data to the threshold setting unit 15.

  The threshold value setting unit 15 sets a threshold value indicating the range of the thermal data value possessed by the person based on the feature value data given from the feature value calculation unit 13 and the initial setting data given from the initial setting unit 14. Specifically, among the feature amounts calculated by the feature amount calculation unit 13, the thermal data value indicated by the feature amount closest to the first initial threshold is set as the threshold Th1, and the feature closest to the second initial threshold is set. The thermal data value indicated by the quantity is set as the threshold Th2. In addition, among the feature amounts calculated by the feature amount calculation unit 13, the thermal data value indicated by the feature amount closest to the third initial threshold is set as Th3. Here, the thermal data value between the threshold Th2 and the threshold Th1 indicates the thermal data value that the person's face has, and the thermal data value that exceeds the threshold Th3 and the threshold Th2 indicates the thermal data value that the person's body has.

  For example, when the feature amounts calculated by the feature amount calculation unit 13 are the maximum point 121, the minimum point 122, and the inflection point 123 shown in FIG. 5, the threshold value setting unit 15 is configured as shown in FIGS. The thermal data value indicated by the minimum point 122a is set as the threshold value Th1, and the thermal data value indicated by the inflection point 123a is set as the threshold value Th2. Further, the thermal data value indicated by the maximum point 121a is set as the threshold value Th3.

  Then, the threshold setting unit 15 generates threshold setting data regarding the set thresholds Th <b> 1 to Th <b> 3 and outputs the threshold setting data to the storage memory 16. The storage memory 16 stores the threshold setting data. Note that a person-equivalent temperature zone is formed by setting the threshold values Th1 to Th3. Here, the person-equivalent temperature zone is a region formed by a range between the threshold Th3 and the threshold Th1, and a portion corresponding to the range in the histogram. This person-equivalent temperature zone is further composed of a face-equivalent temperature zone and a body-equivalent temperature zone. Here, the face-corresponding temperature zone is a region formed by a range between the threshold Th2 and the threshold Th1 and a portion corresponding to the range in the histogram. The body-equivalent temperature zone is a region formed by a range between the threshold Th3 and the threshold Th2 and a portion corresponding to the range in the histogram. For example, in the case illustrated in FIG. 6, the region 130 is a person-equivalent temperature zone. Further, the region 131 is a face equivalent temperature zone, and the region 132 is a body equivalent temperature zone. These temperature zones will be described later.

  The area extraction unit 17 acquires threshold setting data from the storage memory 16 and thermal image data from the image memory 11, and based on the acquired threshold setting data and thermal image data, the region extraction unit 17 calculates a threshold Th2 or more from the thermal image A. A face candidate area having a heat data value equal to or less than the threshold Th1 is extracted. Also, a body candidate region having a heat data value not less than the threshold Th3 and not more than the threshold Th2 is extracted.

  Then, based on these extracted areas and the technique disclosed in Japanese Patent Laid-Open No. 11-328364, a person area in which a person is drawn is extracted from the thermal image A. Thereafter, a detection unit (not shown) detects a person from the extracted person area. Thereby, a person can be detected.

  For example, when the thermal image A is as shown in FIG. 3, the region extraction unit 17 extracts face candidate regions a1 to a11 from the thermal image A as shown in FIG. Further, as shown in FIG. 8, body candidate regions b1 to b8 are extracted from the thermal image A. And as shown in FIG. 9, based on the technique shown in face candidate area | region a1-a11, body candidate area | region b1-b8, and Unexamined-Japanese-Patent No. 11-328364 etc., face candidate area | region a1 and A person area c1 including the body candidate area b1 and a person area c2 including the face candidate area a2 and the body candidate area b2 are extracted.

  Next, a processing procedure by the threshold setting device 1 will be described with reference to a flowchart shown in FIG.

  In step S10 shown in FIG. 10, the camera 10 captures the front of the vehicle P and acquires a thermal image A having thermal data in front of the vehicle P in the form of thermal image data. Next, the acquired thermal image data is output to the image memory 11. The image memory 11 stores thermal image data given from the camera 10.

  Next, in step S11, the histogram creation unit 12 obtains thermal image data from the image memory 11, and creates a histogram based on the obtained thermal image data. Next, histogram data relating to the created histogram is generated and output to the feature amount calculation unit 13.

  Next, in step S12, the feature amount calculation unit 13 calculates the maximum point, the minimum point, and the inflection point of the histogram as the feature amount based on the histogram data given from the histogram creation unit 12. Next, feature amount data relating to the calculated feature amount is generated and output to the threshold setting unit 15.

  In step S13, the initial setting unit 14 detects the surrounding environment of the vehicle P, and sets first to third initial threshold values based on the detected surrounding environment and a threshold map. Next, initial setting data regarding the set first to third initial threshold values is generated and output to the threshold setting unit 15.

  Next, in step S14, the threshold value setting unit 15 performs the above-described processing based on the feature value data provided from the feature value calculating unit 13 and the initial setting data provided from the initial setting unit 14, by the above-described processing. Set Th3. Next, threshold setting data regarding the set threshold values Th <b> 1 to Th <b> 3 is generated and output to the storage memory 16. The storage memory 16 stores the threshold setting data.

  Next, in step S15, the region extraction unit 17 acquires threshold setting data from the storage memory 16 and thermal image data from the image memory 11, and based on these acquired threshold setting data and thermal image data, Through the processing, the face candidate region and the body candidate region are extracted from the thermal image A.

  Next, in step S <b> 16, the region extraction unit 17 extracts a person region from the thermal image A based on the extracted face candidate region and body candidate region, and the technique disclosed in JP-A-11-328364. . Thereafter, a detection unit (not shown) detects a person from the extracted person area. Thereby, a person can be detected.

  As described above, in the first embodiment, as illustrated in FIGS. 5 to 6, the threshold setting device 1 calculates the maximum point, the minimum point, and the inflection point as the feature amount of the histogram. Then, based on the feature amount, threshold values Th1 to Th3 are set. Therefore, since the feature amount used for setting the threshold values Th1 to Th3 is increased as compared with the conventional case, it is possible to set the threshold values Th1 to Th3 that are more appropriate and more accurate than the conventional case.

  Specifically, among the local maximum points of the histogram, even if the local maximum point corresponding to the highest thermal data value does not correspond to the person, the feature corresponding to the person from another feature amount, for example, an inflection point An amount can be selected, and threshold values Th1 to Th3 can be set using the selected feature amount. Therefore, it is possible to set thresholds Th1 to Th3 that are more appropriate and more accurate than those in the past.

  In addition to the local maximum point, the inflection point or the like is used as the feature amount, so that the width of the threshold values Th1 to Th3 can be changed by appropriately changing the feature amount corresponding to the threshold values Th1 to Th3. Accordingly, there is no need to predetermine the widths of the threshold values Th1 to Th3 (for example, the width from the threshold value Th1 to the threshold value Th2). Can do. Therefore, also in this respect, the threshold values Th1 to Th3 can be set accurately.

  In addition to the threshold map indicating the relationship between the environment around the vehicle P and the initial threshold, the threshold setting device 1 sets the thresholds Th1 to Th3 with reference to the histogram. Threshold values Th1 to Th3 can be set. Moreover, even if the environment around the vehicle P fluctuates (for example, even if the vehicle is in shadow), the threshold values Th1 to Th3 can be set so that the threshold values Th1 to Th3 coincide with the thermal data value of the person. Therefore, also in this respect, the threshold values Th1 to Th3 can be set accurately.

  Therefore, the threshold value setting device 1 can accurately set the threshold values Th1 to Th3, so that the person region can be accurately extracted.

(Second Embodiment)
Hereinafter, a second embodiment of the present invention will be described with reference to the drawings. First, based on FIG.1 and FIG.11, the main function of the component of the threshold value setting apparatus 2 which concerns on one embodiment of this invention, and each component is demonstrated. FIG. 11 is an explanatory diagram showing a histogram created at each time.

  As shown in FIG. 1, the threshold setting device 2 includes a histogram creating unit 12, a threshold setting unit 15, and a storage memory 16 of the threshold setting device 1, a histogram creating unit (graph creating unit) 22, a threshold setting unit (threshold setting unit). ) 25 and storage memory (storage means) 26, and the feature amount calculation unit 13 is omitted.

  The histogram creation unit 22 obtains thermal image data from the image memory 11 and creates a histogram indicating the relationship between the thermal data value and the frequency (frequency) of the thermal data value based on the acquired thermal image data. . Then, the histogram creation unit 22 generates histogram data related to the created histogram and outputs it to the storage memory 26. These processes are repeated every predetermined time. FIG. 11 shows histograms 120a to 120c as examples of histograms created every predetermined time.

  The storage memory 26 sequentially stores the histogram data given from the histogram creation unit 22. Therefore, the histogram data is stored in time series. Further, the threshold setting data given from the threshold setting unit 25 is stored.

  When the number of histogram data stored in the storage memory 26, that is, the number of histograms exceeds a specified number, the threshold setting unit 25 sets the histogram data equal to or greater than the specified number and the initial setting data provided from the initial setting unit 14. Based on the above, threshold values Th1 to Th3 are set. Here, the prescribed number is determined according to a target process (for example, a person detection process or a vehicle detection process). The threshold values Th1 to Th3 are the same as those in the first embodiment.

  Specifically, the threshold setting unit 25 acquires a predetermined number or more of histogram data from the storage memory 26. Then, histogram averaging processing is performed based on the acquired histogram data and the initial setting data given from the initial setting unit 14, and threshold values Th1 to Th3 are set.

  Note that the threshold values Th1 to Th3 may be set by the following processing. That is, the threshold setting unit 25 calculates the frequency change rate of the thermal data value at each point on the histogram, and extracts a point where the calculated frequency change rate is larger than a predetermined reference change rate. And among the extracted points, you may set any thermal data value as threshold value Th1-Th3. Further, the shape of the histogram may be predicted, and the threshold values Th1 to Th3 may be set based on the predicted shape. Further, threshold values Th1 to Th3 may be set based on the predicted shape histogram and the processing described in the first embodiment.

  Then, the threshold setting unit 25 generates threshold setting data related to the set thresholds Th <b> 1 to Th <b> 3 and outputs the threshold setting data to the storage memory 26.

  Next, the procedure of processing by the threshold setting device 2 will be described with reference to the flowchart shown in FIG.

  In step S <b> 20 shown in FIG. 12, the camera 10 captures the front of the vehicle P, acquires the thermal image A in the form of thermal image data, and outputs the acquired thermal image data to the image memory 11. The image memory 11 stores thermal image data given from the camera 10.

  Next, in step S21, the histogram creation unit 22 acquires thermal image data from the image memory 11, and generates a histogram based on the acquired thermal image data. Next, histogram data relating to the created histogram is generated and output to the storage memory 26.

  Next, in step S22, the storage memory 26 stores the histogram data given from the histogram creation unit 22.

  Next, in step S23, the threshold setting unit 25 determines whether the number of histogram data stored in the storage memory 26, that is, whether the number of histograms is equal to or greater than a specified number.

  As a result, if the number of histograms is equal to or greater than the prescribed number, the process proceeds to step S24, and if less than the prescribed number (NO in step S23), the process returns to step S20.

  In step S <b> 24, the initial setting unit 14 generates initial setting data by the same processing as in the first embodiment, and outputs the initial setting data to the threshold setting unit 25. Next, the threshold setting unit 25 acquires a predetermined number or more of the histogram data stored in the storage memory 26, and based on the acquired histogram data and the initial setting data given from the initial setting unit 14, the above-described method. Threshold values Th1 to Th3 are set. Next, threshold setting data relating to the set threshold values Th <b> 1 to Th <b> 3 is generated and output to the storage memory 26. The storage memory 26 stores the threshold setting data given from the threshold setting unit 25.

  Next, in step S25, the region extraction unit 17 acquires threshold setting data from the storage memory 26 and thermal image data from the image memory 11, and based on the acquired threshold setting data and thermal image data, The face candidate region and the body candidate region are extracted by the same processing as in the first embodiment.

  Next, in step S26, the region extraction unit 17 extracts a person region in which a person is drawn from the thermal image A by the same processing as in the first embodiment. Thereafter, a detection unit (not shown) detects a person from the extracted person area.

  As described above, in the second embodiment, the threshold value setting device 2 stores the histogram data sequentially, that is, in time series as time passes, as shown in FIG. Then, after the stored histogram data becomes equal to or greater than the specified number, thresholds Th1 to Th3 are set based on the histogram data equal to or greater than the specified number.

  Therefore, even if some of the histograms stored in time series have a feature value (for example, a maximum point) with a thermal data value that does not correspond to a person, it corresponds to another histogram, that is, a person. Threshold values Th1 to Th3 can be set based on a histogram having a feature value with thermal data values. Therefore, even in this case, the threshold values Th1 to Th3 can be set within the range of the thermal data values corresponding to the person, so that the threshold values Th1 to Th3 can be set accurately.

(Third embodiment)
Hereinafter, a third embodiment of the present invention will be described with reference to the drawings. First, based on FIG. 1, components of a threshold setting device 3 according to an embodiment of the present invention and main functions of each component will be described.

  As illustrated in FIG. 1, the threshold setting device 3 includes a threshold setting unit 15, a storage memory 16, and a region extraction unit 17 of the threshold setting device 1, a threshold setting unit (threshold setting unit) 35, and a storage memory (storage unit) 36. , And a region extraction unit (region extraction means) 37, and the threshold value setting device 1 is provided with a counter (not shown). The initial value of this counter is 1.

  The threshold setting unit 35 confirms the value of the counter when the feature amount data is given from the feature amount calculation unit 13. As a result, when the value of the counter is 1, the threshold values Th1 to Th3 are set by the same processing as the threshold value setting unit 15. On the other hand, when the value of the counter is 2 or more, the threshold setting data stored last time is acquired from the storage memory 36, and the following processing is performed based on the acquired threshold setting data and feature amount data. That is, the thermal data value closest to the previously set threshold value Th1 among the thermal data values indicated by the feature amounts is set as the current threshold value Th1. Similarly, the thermal data value closest to the previously set threshold Th2 is set as the current threshold Th2, and the thermal data value closest to the previously set threshold Th3 is set as the current threshold Th3.

  Then, threshold setting data regarding the thresholds Th <b> 1 to Th <b> 3 is generated and output to the storage memory 36. The storage memory 36 sequentially stores the threshold setting data given from the threshold setting unit 35. In other words, it is stored in time series.

  The region extraction unit 37 acquires thermal image data from the image memory 11 and acquires threshold setting data stored this time from the storage memory 36. Then, based on the acquired thermal image data and threshold setting data, a person region is extracted from the thermal image A by the same process as the region extraction unit 17. Thereafter, the counter value is incremented by one.

  Next, a processing procedure by the threshold setting device 3 will be described with reference to a flowchart shown in FIG.

  In step S30 to step S32 shown in FIG. 13, the threshold value setting device 3 performs the same processing as the processing of step S10 to step S12 shown in FIG.

  Next, in step S33, the threshold setting unit 35 checks the value of the counter when the feature amount data is given from the feature amount calculation unit 13. As a result, if the counter value is 1, the process proceeds to step S35, and if the counter value is 2 or more (NO in step S33), the process proceeds to step S34.

  In step S <b> 34, the threshold setting unit 35 acquires the threshold setting data stored last time from the storage memory 36. Based on the acquired threshold setting data and feature amount data, each feature amount is obtained by the above-described processing. Among the thermal data values shown, the thermal data values closest to the previously set threshold values Th1 to Th3 are set as the current threshold values Th1 to Th3. Next, threshold setting data relating to the set threshold values Th <b> 1 to Th <b> 3 is generated and output to the storage memory 36. The storage memory 36 stores threshold setting data given from the threshold setting unit 35. Thereafter, the process proceeds to step S37.

  In steps S35 to S36, the threshold value setting device 3 performs the same processing as steps S13 to S14 shown in FIG. Thereafter, the process proceeds to step S37.

  In step S <b> 37, the region extraction unit 37 acquires thermal image data from the image memory 11 and acquires threshold setting data stored this time from the storage memory 36. Next, based on the acquired thermal image data and threshold setting data, a face candidate region and a body candidate region are extracted from the thermal image A by the same process as the region extraction unit 17.

  Next, in step S38, the region extracting unit 37 extracts a person region from the thermal image A by the same processing as the region extracting unit 17 based on the extracted face candidate region and body candidate region. Thereafter, the counter value is incremented by 1, and the process returns to step S30.

  As described above, in the third embodiment, the threshold setting device 3 stores the threshold set by the threshold setting unit 35 in time series. Then, the current threshold value is set based on the stored threshold value. Specifically, the thermal data values closest to the previously set threshold values Th1 to Th3 are set as the current threshold values Th1 to Th3 among the thermal data values indicated by the feature values of the histogram. Therefore, even if the shape of the histogram fluctuates due to a sudden change in the surrounding environment or a turn of the vehicle P (particularly, even if the maximum point of the histogram does not correspond to a person), the threshold values Th1 to Th3. Fluctuations can be suppressed. In other words, even in such a case, the threshold values Th1 to Th3 can be set in a range of thermal data values corresponding to a person.

(Fourth embodiment)
Hereinafter, a fourth embodiment of the present invention will be described with reference to the drawings. First, based on FIG. 1, FIG. 6, and FIG. 14, the components of the threshold value setting device 4 according to an embodiment of the present invention and the main functions of each component will be described. Here, FIG. 14 is an explanatory diagram showing an example of a histogram and a person-equivalent temperature zone.

  As illustrated in FIG. 1, the threshold setting device 4 includes a histogram creating unit 12, a threshold setting unit 35, and a region extracting unit 37 of the threshold setting device 3, a histogram creating unit (graph creating unit) 42, and a threshold setting unit (threshold setting unit). , Determination means) 45 and area extraction unit (area extraction means) 47.

  The histogram creation unit 42 generates histogram data by the same process as the histogram creation unit 12, and outputs the generated histogram data to the feature amount calculation unit 13 and the threshold setting unit 45.

  The threshold setting unit 45 performs the following processing in addition to performing the same processing as the threshold setting unit 35. That is, after setting the threshold values Th <b> 1 to Th <b> 3, the area of the person equivalent temperature zone is calculated based on the set threshold values Th <b> 1 to Th <b> 3 and the histogram data given from the histogram creation unit 42. Here, as described above, the person-equivalent temperature zone is a region formed by a range between the threshold Th3 and the threshold Th1 and a portion corresponding to the range in the histogram. The threshold setting unit 45 further calculates the area of the thermal image temperature zone. Here, the thermal image equivalent temperature zone is an area formed by the entire range of the thermal data values indicated by the histogram (the range from the lower limit value Th0 to the upper limit value Th4 of the thermal data values indicated by the histogram) and the entire histogram. . FIG. 6 shows a person equivalent temperature zone 130 and a thermal image temperature zone 133 as an example of the person equivalent temperature zone and the thermal image temperature zone.

  Then, the threshold setting unit 45 divides the area of the person-equivalent temperature zone by the area of the thermal image temperature zone to calculate the ratio α of the person region in the thermal image A. Then, the calculated ratio α is compared with a preset ratio β. As a result, when the ratio α is equal to or higher than the ratio β, it is determined that a person cannot be detected from the person area, and when the ratio α is smaller than the ratio β, the person can be detected from the person area. Is determined. An example when the ratio α is smaller than the ratio β is shown in FIG. 14A, and an example when the ratio α is equal to or higher than the ratio β is shown in FIG. FIG. 14 shows a histogram 120, person-equivalent temperature zones 130a to 130b, and a thermal image temperature zone 133 as examples of the histogram, the person-equivalent temperature zone, and the thermal image temperature zone.

  Note that this determination can be made because it becomes difficult to detect a person because the person area increases as the ratio α increases. Therefore, the ratio β coincides with the ratio α when the person cannot be detected from the person area.

  In addition, when the ratio α is equal to or higher than the ratio β, the threshold setting unit 45 presents that a person cannot be detected by a presentation device (not shown) (for example, a display or a speaker) and sets the counter value to 1. Reset.

  In addition, when the ratio α is smaller than the ratio β, the threshold setting unit 45 generates threshold setting data regarding the thresholds Th1 to Th3 and outputs the threshold setting data to the storage memory 36.

  When it is determined that a person can be detected, the region extraction unit 47 extracts a person region from the thermal image A by the same processing as the region extraction unit 37.

  Next, the procedure of processing by the threshold setting device 4 will be described with reference to the flowchart shown in FIG.

  In step S40 to step S46 shown in FIG. 15, the threshold value setting device 4 performs the same processing as in step S30 to step S36 shown in FIG. 13, and sets the threshold values Th1 to Th3. In step S 41, the histogram creation unit 42 outputs the generated histogram data to the threshold value setting unit 45 in addition to the feature amount calculation unit 13.

  Next, in step S47, the threshold value setting unit 45, based on the set threshold values Th1 to Th3 and the histogram data given from the histogram creation unit 42, the area of the person equivalent temperature zone and the area of the thermal image temperature zone. Is calculated. Next, the ratio α of the person region in the thermal image A is calculated by dividing the area of the person equivalent temperature zone by the area of the thermal image temperature zone.

  Next, in step S48, the threshold setting unit 45 compares the calculated ratio α with a preset ratio β. If the ratio α is smaller than the ratio β, the process proceeds to step S49. If the ratio α is equal to or greater than the ratio β (NO in step S48), the process proceeds to step S51.

  In step S49, the threshold setting unit 45 determines that a person can be detected from the person area. Next, threshold setting data regarding the thresholds Th <b> 1 to Th <b> 3 is generated and output to the storage memory 36. The storage memory 36 stores threshold setting data given from the threshold setting unit 45.

  Next, the region extraction unit 47 acquires thermal image data from the image memory 11 and acquires threshold setting data stored this time from the storage memory 36. Next, a face candidate region and a body candidate region are extracted from the thermal image A by the same processing as the region extracting unit 37.

  Next, in step S50, the region extraction unit 47 extracts a person region from the thermal image A by the same processing as the region extraction unit 37, and increments the counter value by one. Then, it returns to step S40.

  On the other hand, if the ratio α is greater than or equal to the ratio β in the process of step S48 described above, in step S51, the threshold setting unit 45 presents that a person cannot be detected by a not-shown presentation device. The counter value is reset to 1. Then, it returns to step S40.

  As described above, in the fourth embodiment, the threshold setting device 4 stores the thresholds Th1 to Th3 set by the threshold setting unit 35 in time series only when a person can be detected from the person area. . Then, the current threshold values Th1 to Th3 are set based on the stored threshold values Th1 to Th3.

  Therefore, the threshold value setting device 4 can set the threshold values Th1 to Th3 more reliably than the third embodiment within the range of the thermal data value possessed by the person. That is, the reliability of the threshold values Th1 to Th3 can be improved as compared with the third embodiment.

  Further, the threshold setting device 4 extracts a person area from the thermal image A only when a person can be detected from the person area. Therefore, the area extraction unit 47 does not need to determine whether or not a person can be detected from the person area. Specifically, there is no need to perform a labeling process normally performed in the person region extraction stage, calculation of the area of the person region, grasping the shape of the person region, and the like. Therefore, the processing speed by the threshold setting device 4 is improved. Further, it is possible to determine whether or not a person can be detected and the necessity of resetting a threshold without performing these processes.

  Needless to say, the techniques described in the first to fourth embodiments can be combined with each other.

  In the first to fourth embodiments, a person is detected as a detection target. However, another detection object, for example, a road structure may be detected.

It is a block diagram which shows the structure of the threshold value setting apparatus which concerns on one embodiment of this invention. (A) It is the side view which showed the installation position of the camera. (B) It is the top view which showed the installation position of the camera. It is explanatory drawing which shows an example of the thermal image acquired with a camera. It is explanatory drawing which shows an example of a histogram. It is explanatory drawing which shows an example of the feature-value. It is explanatory drawing which shows an example of a person equivalent temperature range. It is explanatory drawing which shows an example of a face candidate area | region. It is explanatory drawing which shows an example of a body candidate area | region. It is explanatory drawing which shows an example of a person area. It is the flowchart which showed the procedure of the process by a threshold value setting apparatus. It is explanatory drawing which shows an example of the histogram produced for every predetermined time. It is the flowchart which showed the procedure of the process by a threshold value setting apparatus. It is the flowchart which showed the procedure of the process by a threshold value setting apparatus. (A) It is explanatory drawing which shows an example of a person equivalent temperature range. (B) It is explanatory drawing which shows an example of a person equivalent temperature range. It is the flowchart which showed the procedure of the process by a threshold value setting apparatus.

Explanation of symbols

1 to 4 Threshold setting device 10 Camera (image acquisition means)
DESCRIPTION OF SYMBOLS 11 ... Image memory 12, 22, 42 ... Histogram creation part (graph creation means)
13 ... feature amount calculation unit (feature amount calculation means)
14 ... Initial setting unit 15, 25, 35, 45 ... Threshold setting unit (threshold setting means)
16, 26, 36... Storage memory (storage means)
17, 37, 47... Region extraction unit (region extraction means)
120, 120a to 120c ... Histogram (frequency distribution graph)
121, 121a ... Maximum point 122, 122a ... Minimum point 123, 123a ... Inflection point

Claims (5)

In a threshold setting device mounted on a vehicle,
Image acquisition means for acquiring an image having a state quantity data value around the vehicle;
Based on the image acquired by the image acquisition means, a graph creation means for creating a frequency distribution graph indicating a relationship between the state quantity data value and the frequency of the state quantity data value;
Based on the frequency distribution graph created by the graph creation means, feature quantity calculation means for calculating a feature quantity including the maximum point and the inflection point of the frequency distribution graph;
Threshold setting means for setting a threshold value indicating a range of state quantity data values of a predetermined detection object based on the feature quantity calculated by the feature quantity calculation means and the surrounding environment of the vehicle;
A region extracting unit that extracts a detection target region having a state quantity data value within a range indicated by the threshold set by the threshold setting unit from the image acquired by the image acquiring unit; Threshold setting device. In a threshold setting device mounted on a vehicle,
Image acquisition means for acquiring an image having a state quantity data value around the vehicle;
Based on the image acquired by the image acquisition means, a graph creation means for creating a frequency distribution graph indicating a relationship between the state quantity data value and the frequency of the state quantity data value;
Storage means for storing the frequency distribution graph created by the graph creation means in time series;
Based on the frequency distribution graph stored by the storage means , predict the shape of the frequency distribution graph, calculate the feature amount including the maximum point and the inflection point of the frequency distribution graph of the predicted shape, and calculate Threshold setting means for setting a threshold value indicating a range of state quantity data values of a predetermined detection object based on the feature amount and the surrounding environment of the vehicle ;
A region extracting unit that extracts a detection target region having a state quantity data value within a range indicated by the threshold set by the threshold setting unit from the image acquired by the image acquiring unit; Threshold setting device. The threshold value setting device according to claim 1,
Storage means for storing the threshold value set by the threshold value setting means in time series,
The threshold value setting unit sets the current threshold value based on the threshold value stored by the storage unit. In the threshold value setting device according to any one of claims 1 to 3,
Based on the frequency distribution graph created by the graph creating means and the threshold set by the threshold setting means, the proportion of the detection target area in the image is calculated, and the calculated proportion is calculated. Based on determination means for determining the detectability of the detection object,
The threshold value setting device, wherein the area extraction unit extracts the detection object area from the image when it is determined that the detection object is detectable. A first step of acquiring an image having state quantity data values around the vehicle;
A second step of creating a frequency distribution graph showing a relationship between the state quantity data value and the frequency of the state quantity data value based on the image acquired by the first step;
A third step of calculating a feature amount including a local maximum point and an inflection point of the frequency distribution graph based on the frequency distribution graph created by the second step;
A fourth step of setting a threshold value indicating a range of state quantity data values of a predetermined detection object based on the feature amount calculated in the third step and the surrounding environment of the vehicle;
A fifth step of extracting, from the image acquired by the first step, a detection target region having a state quantity data value within a range indicated by the threshold set by the fourth step. A characteristic threshold setting method.

Images