WO2014203658A1

WO2014203658A1 - Distance measurement device and distance measurement method

Info

Publication number: WO2014203658A1
Application number: PCT/JP2014/062897
Authority: WO
Inventors: 自広山谷; 基広浅野
Original assignee: コニカミノルタ株式会社
Priority date: 2013-06-21
Filing date: 2014-05-14
Publication date: 2014-12-24
Also published as: JPWO2014203658A1

Abstract

In the distance measurement device and distance measurement method according to the present invention, the distance to a prescribed object included in a latest photographed image that is the most recent photographed image from among chronological photographed images photographed by a photography unit is determined on the basis of the latest photographed image and a reference image stored in a reference information storage unit. The photography state of the photography unit is obtained, a determination is made about whether the obtained photography state meets a prescribed condition, and on the basis of the determination result, a determination is made about whether to discard the reference image stored in the reference information storage unit.

Description

Ranging device and ranging method

The present invention relates to a distance measuring device and a distance measuring method for measuring a distance from an image to a predetermined object.

In recent years, as an effort to improve safety, for example, a camera mounted on a moving body such as a vehicle or a robot is used to measure the distance to a predetermined object existing in the moving direction of the moving body, and according to the distance to the object A system for controlling the moving body has been researched and developed. As a distance measuring device used in such a system, for example, there is a device disclosed in Patent Document 1.

In the forward vehicle tracking system disclosed in Patent Document 1, the distance to another vehicle is required as follows. First, another vehicle is detected from an image captured by a camera mounted on the host vehicle, and the detected image of the other vehicle is defined and stored as a reference template. Next, the correlation between the input image newly picked up by the camera and the reference template is obtained while enlarging or reducing the input image or the reference template, whereby a new vehicle of the other vehicle in the input image is obtained. The position and size are detected. Then, the distance between the host vehicle and the other vehicle is obtained based on the detected position and size of the other vehicle.

By the way, in the distance measuring method disclosed in Patent Document 1, when obtaining the distance, the input image newly captured by the camera and the image acquired by the camera before the input image are obtained. A reference template is used. For this reason, when the imaging state such as the imaging direction of the camera or the exposure condition changes relatively greatly, the input image and the reference template will be inconsistent. As a result, the new position of the other vehicle in the input image and There is a possibility that the size is erroneously detected or not detected.

JP 2004-112144 A

The present invention has been made in view of the above circumstances, and an object thereof is to provide a distance measuring device and a distance measuring method capable of improving detection accuracy even when the imaging state changes.

In the distance measuring apparatus and the distance measuring method according to the present invention, the latest imaging based on the latest latest captured image of the time-series captured images captured by the imaging unit and the reference image stored in the reference information storage unit. A distance to a predetermined object included in the image is obtained. Here, the imaging state of the imaging unit is acquired, it is determined whether or not the acquired imaging state satisfies a predetermined condition, and the reference stored in the reference information storage unit based on the determination result It is determined whether or not to discard the image. For this reason, such a distance measuring apparatus and distance measuring method can improve detection accuracy even when the imaging state changes.

The above and other objects, features and advantages of the present invention will become apparent from the following detailed description and the accompanying drawings.

It is a block diagram which shows the structure of the distance measuring device in embodiment. It is a figure for demonstrating the distance calculation method in the distance measuring device of embodiment. It is a flowchart which shows operation | movement of the ranging apparatus in embodiment. When a vehicle equipped with an imaging unit enters a tunnel, a method for determining the imaging state when the median luminance value of the captured image is used as the imaging state of the imaging unit in the distance measuring apparatus of the embodiment is described. FIG. When a vehicle equipped with an image pickup unit exits a tunnel, the method for determining the image pickup state when the median luminance value of the picked-up image is used as the image pickup state of the image pickup unit in the distance measuring device of the embodiment FIG. It is a figure which shows typically the picked-up image imaged by the said imaging part before and after a water droplet adheres to the imaging part in the distance measuring device of embodiment. It is a figure for demonstrating the determination method of the said imaging state in case the frequency characteristic of a captured image is used as an imaging state of the imaging part in the distance measuring device of embodiment. It is a figure for demonstrating the image area | region used by the state acquisition part in the distance measuring device of embodiment. It is a figure for demonstrating the determination method of the said imaging state at the time of using the steering angle of the vehicle carrying the said imaging part as an imaging state of the imaging part in the distance measuring device of embodiment.

Hereinafter, an embodiment according to the present invention will be described with reference to the drawings. In addition, the structure which attached | subjected the same code | symbol in each figure shows that it is the same structure, The description is abbreviate | omitted suitably. Further, in this specification, when referring generically, it is indicated by a reference symbol without a suffix, and when referring to an individual configuration, it is indicated by a reference symbol with a suffix.

FIG. 1 is a block diagram showing a configuration of a distance measuring device in the embodiment. FIG. 2 is a diagram for explaining a distance calculation method in the distance measuring apparatus according to the embodiment. FIG. 2A is a diagram for describing a reference image extracted from a captured image, and FIG. 2B illustrates a region that matches the reference image in the latest latest captured image by pattern matching processing (correlation processing). It is a figure for doing.

The distance measuring device according to the embodiment is a device that measures a distance from the moving body to a predetermined object included in the captured image based on a captured image captured by an imaging unit mounted on the moving body. The moving body is, for example, a vehicle such as an automobile or a train, a robot having a moving function, a movement support device for a visually impaired person, or the like. The mobile body may be capable of self-propelled.

Such a distance measuring device DM of the present embodiment includes, for example, an imaging unit 1, a processing unit 2, a storage unit 3, and a notification unit 4 as shown in FIG.

The imaging unit 1 is a device that is mounted on the mobile body (not shown) and continuously images a subject. Preferably, the imaging unit 1 is disposed on the moving body such that the imaging direction (optical axis direction) coincides with the moving direction of the moving body. For example, when the imaging unit 1 is mounted on a vehicle, the imaging unit 1 is disposed on a dashboard, for example, with the imaging direction (optical axis direction) facing forward. The imaging unit 1 includes, for example, an imaging optical system, a color or monochrome image sensor, and an image processing unit, and an optical image of a subject is formed on the imaging surface (light receiving surface) of the image sensor by the imaging optical system. The optical image of the subject is photoelectrically converted by the image sensor, and the captured image of the subject is generated from the signal obtained by the photoelectric conversion by the image processing of the image processing unit. The imaging unit 1 continuously images the subject at predetermined time intervals. For example, the imaging unit 1 continuously images the subject in time so as to obtain a predetermined frame rate such as 15 frames / second, 24 frames / second, and 30 frames / second. The imaging unit 1 is connected to the processing unit 2 and sequentially outputs the generated captured images to the processing unit 2.

The storage unit 3 is connected to the processing unit 2, and stores various programs executed by the processing unit 2 and data necessary for the execution in advance (ROM (Read Only Memory)) or EEPROM (Electrically Erasable Programmable Read Only Memory). And the like, a volatile memory element such as a RAM (Random Access Memory) serving as a so-called working memory of the processing unit 2, and a peripheral circuit thereof. The storage unit 3 is functionally configured with a reference information storage unit 31 that stores a later-described reference image.

The processing unit 2 controls each unit of the distance measuring device DM according to the function of each unit in order to measure the distance from the moving body to a predetermined object included in the captured image. The processing unit 2 includes, for example, a CPU (Central Processing Unit) and its peripheral circuits. The processing unit 2 functionally executes a ranging program for measuring the distance from the moving object to a predetermined object included in the captured image, thereby functionally controlling the control unit 21, the distance calculation unit 22, and the state. An acquisition unit 23, a discard determination unit 24, and a notification unit 25 are configured. The control unit 21 controls the entire distance measuring device DM.

The distance calculation unit 22 stores one of time-series captured images captured continuously in time by the imaging unit 1 in the reference information storage unit 31 as a reference image, and when captured by the imaging unit 1 A distance to a predetermined object included in the latest captured image is obtained based on the latest latest captured image of the series of captured images and the reference image stored in the reference information storage unit 31.

The reference image may be the entire captured image so that a predetermined image of a predetermined object is extracted from the captured image when the distance is obtained. In this embodiment, the reference image is captured. This is an image of a predetermined object extracted from the image and is a part of the captured image. For this reason, the distance calculation unit 22 of the present embodiment has a predetermined timing set in advance at a predetermined timing (for example, when the distance measurement is started or the reference image is not stored in the reference information storage unit 31). The object image (object image) is extracted from the captured image, and the extracted object image is stored in the reference information storage unit 31 as a reference image. More specifically, as shown in FIG. 2A, the distance calculation unit 22 detects a predetermined object Ob set in advance from a captured image Pp captured temporally before the latest captured image, and this detection. The rectangular image (object image) surrounding the predetermined object Ob is extracted from the captured image Pp as the reference image TP, and the extracted object image is stored in the reference information storage unit 31 as the reference image TP. If the predetermined object is not present in the image, the process returns to the beginning without extracting the reference image TP. The predetermined object Ob may be an obstacle to movement of a moving object such as a vehicle (automobile and train), a motorcycle (bicycle and motorcycle), a person, an animal, and an artificial object. It is. When the predetermined object Ob is an automobile, as disclosed in Patent Document 1, for vehicle detection by image processing, for example, (1) It is sandwiched between white lines drawn on a road A first horizontal edge (lateral edge) is detected from the bottom to the top of the image, (2) a second horizontal edge above the first horizontal edge is detected, and (3) the first Well-known conventional means (conventional method) such as detecting a pair of first and second vertical edges in a region sandwiched between the horizontal edge and the second horizontal edge is used. That is, for vehicle detection by image processing, first and second vertical edges that form a pair of top and bottom or left and right pairs from a histogram (columnar diagram) of an edge sandwiched between white lines. Known conventional means (conventional method) such as edge detection is used.

The distance calculation unit 22 detects the predetermined object Ob from the latest captured image Pt by template matching using the reference image TP as a template while enlarging or reducing the reference image TP or the latest captured image Pt. Thereby, the distance calculation unit 22 detects a new position and size of the predetermined object Ob on the latest captured image Pt. Then, the distance calculation unit 22 obtains a distance between the moving body and the predetermined object Ob based on the detected new position and size. The distance z is given by the following equations (1) and (2), as disclosed in Patent Document 1.
θ = atan (f / xc) (1)
z = f · w / wc (2)
Here, θ is the direction in which the object Ob at the position detected at the center xc in the width direction of the object Ob on the image exists, f is the focal length of the imaging unit 1, and w is the actual object The width of Ob, and wc is the width of the object Ob on the image.

The state acquisition unit 23 acquires the imaging state of the imaging unit 1. More specifically, the state acquisition unit 23 acquires the imaging state of the imaging unit 1 based on a predetermined feature amount in the latest captured image Pt captured by the imaging unit 1. For example, preferably, the state acquisition unit 23 acquires a statistical value (for example, an average value or a median value) of a pixel value or a luminance value as the predetermined feature amount. Further, for example, preferably, the state acquisition unit 23 acquires a frequency characteristic in the latest captured image as the predetermined feature amount. When calculating the predetermined feature amount, the state acquisition unit 23 may use a part of the latest captured image.

The state acquisition unit 23 may acquire the imaging direction of the imaging unit 1 as the imaging state of the imaging unit 1.

The discard determination unit 24 determines whether or not the imaging state acquired by the state acquisition unit 23 satisfies a predetermined condition, and uses the reference image TP stored in the reference information storage unit 31 based on the determination result. It is determined whether or not to discard. The discard determination unit 24 discards the reference image TP stored in the reference information storage unit 31 when it is determined that the imaging state acquired by the state acquisition unit 23 satisfies a predetermined condition. That is, the discard determination unit 24 discards the reference image TP in the reference information storage unit 31.

More specifically, the discard determination unit 24 functionally includes a state determination unit 241, a holdability determination unit 242, and a discard control unit 243. The state determination unit 241 determines whether the imaging state acquired by the state acquisition unit 23 satisfies a predetermined condition. The holdability determination unit 242 determines whether or not to discard the reference image TP stored in the reference information storage unit 31 based on the determination result determined by the state determination unit 241. The discard control unit 243 discards the reference image TP in the reference information storage unit 31 when it is determined that the imaging state acquired by the state acquisition unit 23 satisfies a predetermined condition.

When the imaging state acquired by the state acquisition unit 23 satisfies a predetermined condition, the notification unit 25 determines that the state acquisition unit 25 determines that the continuous determination count n continuously determined by the discard determination unit 24 is equal to or greater than a threshold value Thn. The notification unit 4 notifies the outside that the imaging state acquired by the video camera 23 satisfies a predetermined condition.

The notification unit 4 is a device that is connected to the processing unit 2 and notifies the outside that the imaging state acquired by the state acquisition unit 23 in accordance with the control of the notification unit 25 satisfies a predetermined condition. The notification unit 4 includes, for example, a light emitting diode that emits light, and turns on (including the case of blinking) in accordance with the control of the notification unit 25 to notify the outside. Further, for example, the notification unit 4 is configured to include a buzzer or a speaker that emits a sound, and notifies the outside by issuing a warning sound according to the control of the notification unit 25. Further, for example, the notification unit 4 is configured to include a display device such as a CRT display or a liquid crystal display, for example, and notifies the outside by displaying a message indicating the above in accordance with the control of the notification unit 25.

Next, the operation of the distance measuring apparatus DM in the present embodiment will be described below with a focus on discarding the reference image according to the imaging conditions. FIG. 3 is a flowchart showing the operation of the distance measuring apparatus in the embodiment.

In the distance measuring device DM according to the present embodiment, when the distance measurement is started by an input of an unillustrated start switch, an input of a distance measurement start instruction switch, or the like, the imaging unit 1 follows the control of the control unit 21 of the processing unit 2. The captured image of the subject is captured at a predetermined frame rate, and the time-series captured images are sequentially output to the processing unit 2.

When the captured image (input image) Pt is input, the distance calculation unit 22 of the processing unit 2 first determines whether or not the reference image TP is stored in the reference information storage unit 31 of the storage unit 3 in FIG. (S1). As a result of this determination, when the reference image TP is not stored (is not stored) in the reference information storage unit 31 (N), the distance calculation unit 22 executes the next process S3. On the other hand, when the reference image TP is stored (exists) in the reference information storage unit 31 as a result of the determination in step S1 (Y), template matching is performed on the captured image (input image) Pt. Then, the reference image TP is acquired (S2), and the process S3 is executed.

In this process S3, the distance calculation unit 22 detects a predetermined object Ob. More specifically, when it is determined in step S1 that there is no reference image TP, the distance calculation unit 22 newly adds a predetermined object Ob to the captured image (input image) Pt by, for example, the above-described known technique. Is detected. On the other hand, when it is determined in step S1 that the reference image TP is present, the distance calculation unit 22 performs template matching on the captured image (input image) using the reference image TP acquired in step S2 as a template. The predetermined object Ob is detected, and the position and size of the new predetermined object Ob on the captured image (input image) Pt are detected.

Next, the state acquisition unit 23 of the processing unit 2 acquires the imaging state of the imaging unit 1, and notifies the state determination unit 241 of the discard determination unit 24 of the acquired imaging state of the imaging unit 1 (S4). Next, when receiving the notification of the imaging state of the imaging unit 1, the state determination unit 241 determines whether the imaging state of the imaging unit 1 satisfies a predetermined condition and notifies the determination result to the holdability determination unit 242. (S5). The acquisition of the imaging state in the process S4 and the determination of the imaging state in the process S5 will be described later.

As a result of the determination in step S5, when the imaging state of the imaging unit 1 does not satisfy the predetermined condition (N), the state determination unit 241 notifies the distance calculation unit 22 accordingly, thereby the distance calculation unit. 22 stores the image including the predetermined object Ob detected in the process S3 in the reference information storage unit 31 as a new reference image TP and stores it (S6). When it is determined that the reference image TP is stored in the reference information storage unit 31 in the process S1, the distance calculation unit 22 adds a new image on the image image (input image) Pt obtained in the process S3. Based on the position and size of the predetermined object Ob, a distance between the moving object and the predetermined object Ob is obtained. Then, the distance calculation unit 22 returns the process to the process S1 in order to process the next captured image (next input image) sequentially input in time series.

On the other hand, as a result of the determination in step S5, when the imaging state of the imaging unit 1 satisfies a predetermined condition (Y), the state determination unit 241 notifies the hold determination unit 242 of the discard determination unit 24 to that effect. Accordingly, the holdability determination unit 242 determines whether to discard the reference image TP stored in the reference information storage unit 31 based on the determination result determined by the state determination unit 241. That is, the holdability determination unit 242 determines whether or not the reference image TP is stored in the reference information storage unit 31 (S7).

When the reference image TP is stored (present) in the reference information storage unit 31 as a result of the determination in step S7 (Y), the holdability determination unit 242 notifies the discard control unit of the discard determination unit 24 to that effect. The discard control unit 243 causes the reference information storage unit 31 to discard the reference image TP stored in the reference information storage unit 31 (S9), and processes the next captured image (next input image). In order to do so, the process returns to process S1. In this process S9, for example, the reference image TP may be discarded by deleting the data of the reference image TP from the reference information storage unit 31. For example, the reference image TP stores the data of the reference image TP. It may be abandoned by releasing the storage area.

On the other hand, when the reference image TP is not stored (is not stored) in the reference information storage unit 31 (N) as a result of the determination in the process S7, the holdability determination unit 242 determines that the imaging state of the imaging unit 1 is predetermined. The notification unit 25 notifies the notification unit 25 that the condition is satisfied, and when the notification unit 25 is continuously notified that the imaging state of the imaging unit 1 satisfies the predetermined condition, the notification unit 25 counts the number of notifications as the continuous determination number n. Then, it is determined whether or not the counted number n of consecutive determinations is equal to or greater than a threshold value Thn (S8).

As a result of the determination in step S8, when the number of consecutive determinations n is less than the threshold value Thn (N), the notification unit 25 processes the next captured image (next input image) sequentially input in time series. In order to do so, the process returns to process S1.

On the other hand, if the result of determination in step S8 is that the number of consecutive determinations n is equal to or greater than the threshold value Thn (N), the notification unit 25 indicates that the imaging state acquired by the state acquisition unit 23 satisfies a predetermined condition. Is notified to the outside by the notification unit 4 (S10). The notification unit 25 returns the process to step S1 in order to process the next captured image (next input image) sequentially input in time series.

Note that the processes S1 to S10 shown in FIG. 3 are executed for each captured image (input image) Pt sequentially input in time series as described above. Therefore, when it is continuously notified in the process S8 that the imaging state of the imaging unit 1 satisfies the predetermined condition, the captured images (input images) Pt sequentially input in time series are continuously displayed. This is a case where images are continuously captured in an imaging state that satisfies the predetermined condition.

Here, the acquisition of the imaging state in the above-described process S4 and the determination of the imaging state in the process S5 will be described below. FIG. 4 illustrates the determination method of the imaging state when the vehicle luminance value of the image is used as the imaging state of the imaging unit in the distance measuring apparatus according to the embodiment when the vehicle equipped with the imaging unit enters the tunnel. It is a figure for doing. FIG. 5 illustrates the determination method of the imaging state when the vehicle luminance value of the image is used as the imaging state of the imaging unit in the distance measuring device of the embodiment when the vehicle equipped with the imaging unit exits the tunnel. It is a figure for doing. 4A and 5A are diagrams showing temporal changes in the median luminance value in a time-series captured image (each frame), the horizontal axis is a frame (time), and the vertical axis is the luminance value. Is the median of FIG. 4B and FIG. 5B are diagrams showing the time change of the change amount of the median value, the horizontal axis is the frame (time), and the vertical axis is the change amount of the luminance value. FIG. 6 is a diagram schematically illustrating captured images captured by the imaging unit before and after water droplets adhere to the imaging unit in the distance measuring apparatus according to the embodiment. 6A schematically illustrates a captured image captured by the imaging unit before water droplets adhere to the imaging unit, and FIG. 6B illustrates an image captured by the imaging unit after water droplets adhere to the imaging unit. An image is shown typically. FIG. 7 is a diagram for describing a method for determining the imaging state when the frequency characteristics of the image are used as the imaging state of the imaging unit in the distance measuring apparatus according to the embodiment. FIG. 7A shows each frequency characteristic of each captured image before and after water droplets adhere to the imaging unit, and FIG. 7B illustrates a determination method according to the first aspect for determining whether or not water droplets have adhered to the imaging unit. FIG. 7C is a diagram for explaining a determination method of the second mode for determining whether or not water droplets have adhered to the imaging unit. In FIGS. 7A, 7B, and 7C, the horizontal axis represents frequency, and the vertical axis represents intensity. FIG. 8 is a diagram for explaining an image region used by the state acquisition unit in the distance measuring apparatus according to the embodiment. FIG. 8A is a diagram for explaining an image area of the first mode used by the state acquisition unit, and FIG. 8B is a diagram for explaining an image region of the second mode used by the state acquisition unit. FIG. 9 is a diagram for explaining a method for determining the imaging state when the steering angle of a vehicle equipped with the imaging unit is used as the imaging state of the imaging unit in the distance measuring apparatus according to the embodiment. FIG. 9A is a diagram showing the time change of the steering angle, the horizontal axis is time, and the vertical axis is the steering angle. FIG. 9B is a diagram showing the change over time in the change amount of the steering angle, the horizontal axis is time, and the vertical axis is the change amount of the steering angle.

The state acquisition unit 23 described above acquires the imaging state of the imaging unit 1 based on a predetermined feature amount in the latest latest captured image Pt among the time-series captured images captured by the imaging unit 1. It may be. Such a distance measuring device DM uses the latest captured image (input image) Pt imaged by the imaging unit 1, and therefore does not require a separate device for acquiring the imaging state of the imaging unit 1.

As the predetermined feature amount, various attribute values in the latest captured image Pt can be used. For example, the predetermined feature amount includes an average value of each pixel value of each pixel in the latest captured image Pt, a median value of each pixel value of each pixel in the latest captured image Pt, and each luminance value of each pixel in the latest captured image Pt. And a predetermined statistic of each pixel in the latest captured image Pt, such as a median value of each luminance value of each pixel in the latest captured image Pt. In order to reduce the influence of disturbance, the predetermined feature amount is preferably a median value. More specifically, the median value of each pixel value of each pixel in the latest captured image Pt is shown in FIG. 4A when the moving body enters the tunnel with respect to each time-series captured image P (each frame, time elapsed). As shown in Fig. 2, it changes at a relatively large value before entering the tunnel (outside the tunnel), gradually decreases when it reaches the tunnel, and changes at a relatively small value after entering the tunnel (inside the tunnel). To change. Further, the median value of each pixel value of each pixel in the latest captured image Pt is as shown in FIG. 5A when the moving body exits the tunnel with respect to each time-series captured image P (each frame, time elapsed). Change. The median value is relatively small before exiting the tunnel (inside the tunnel), gradually increasing when reaching the tunnel, and relatively large after exiting the tunnel (outside the tunnel). To change. Thus, since the median value of each pixel value of each pixel in the latest captured image Pt changes, the amount of change (the difference between the median value of the latest captured image Pt and the median value of the previous captured image P). As shown in FIG. 4B in the case of FIG. 4A and as shown in FIG. 5B in the case of FIG. 5A, the value is relatively large only when the tunnel is approaching. Since the change amount of the median value changes in this way, the discard determination unit 24 determines whether or not the change amount in the median value of the latest captured image Pt acquired by the state acquisition unit 23a is equal to or greater than a predetermined threshold Th1. It is possible to determine whether or not the imaging state acquired by the state acquisition unit 23a satisfies a predetermined condition. That is, when the discard determination unit 24 determines that the amount of change in the median value of the latest captured image Pt is equal to or greater than the predetermined threshold Th1, the imaging state acquired by the state acquisition unit 23a satisfies a predetermined condition. Is determined. If the discard determination unit 24 determines that the amount of change in the median value of the latest captured image Pt is less than the predetermined threshold Th1, the imaging state acquired by the state acquisition unit 23a satisfies a predetermined condition. Judge that there is no. The predetermined threshold Th1 is statistically appropriately set by measuring a plurality of samples, for example.

Note that the change in the median value and the change in the median value shown in FIGS. 4 and 5 are not only in the case of entering / exiting the tunnel, but also when entering / exiting a relatively long guard, or from normal light to backlight. This may occur when the brightness changes, such as when the image is changed, when the backlight changes to the forward light, or when the imaging unit 1 is irradiated on the light of the oncoming vehicle.

Further, when such an imaging state changes, as shown in FIGS. 4 and 5, for example, until the AE (Automatic Exposure, automatic exposure) of the imaging unit 1 functions, the imaging state after the change Usually, a predetermined time is required until the imaging unit 1 copes with and the captured image is stabilized (AE unstable period Tu). For this reason, in the distance measuring device DM described above, as indicated by a broken line in FIG. 1, when the reference image TP stored in the reference information storage unit 31 is discarded, the processing of processing S1 to processing S10 that determines this discarding After a predetermined number of images have been captured after the captured image (input image, the latest captured image) Pt used in the above is captured (after a predetermined time has elapsed), the captured image P captured by the imaging unit 1 is newly referred to. The distance measuring apparatus DM may be configured to further include a reference information updating unit that stores the image TP in the reference information storage unit 31. By including such a reference information update unit, storing the captured image captured in this unstable state in the reference information storage unit 31 is reduced, and the distance measuring device DM can improve the detection accuracy. it can.

Further, for example, the predetermined feature amount is a frequency characteristic in the latest captured image Pt. For example, when a foreign matter such as an insect or a water droplet adheres to the imaging unit 1, a part of the front is unclear, and the frequency characteristics of the latest captured image Pt change before and after the foreign matter. The frequency characteristic of the captured image P is a frequency spectrum that is a distribution of frequency components (intensities) included in the captured image P, and is an intensity with respect to the frequency. For example, when the water droplet Dp adheres to the imaging unit 1, before the water droplet Dp adheres, the captured image P1 is relatively clear as a whole as schematically shown in FIG. 6A, but the water droplet Dp adheres. Later, as shown schematically in FIG. 6B, a part of the captured image P2 becomes relatively unclear. For this reason, the high frequency component of the captured image P2 after the water drop Dp is attached is reduced compared to the captured image P1 before the water drop Dp is attached. Therefore, as shown in FIG. 7A, the frequency characteristic α of the captured image P1 before the water droplet Dp is attached is different from the frequency characteristic β of the captured image P2 after the water droplet Dp is attached, and the captured image after the water droplet Dp is attached. The frequency characteristic β of P2 is higher in intensity on the low frequency side (for example, the frequency range of frequency fL or lower) than the frequency characteristic α of the captured image P1 before the water drop Dp is attached, while on the high frequency side (for example, frequency of frequency fH or higher) (Range) is a small profile.

Thus, since the frequency characteristics of the captured image change before and after the attachment of the foreign matter, the discard determination unit 24, for example, as shown in FIG. 7B, the frequency of the latest captured image Pt acquired by the state acquisition unit 23a. By determining whether or not the amount of change in the low frequency fL in the characteristic is greater than or equal to a predetermined threshold Th21, it is possible to determine whether or not the imaging state acquired by the state acquisition unit 23a satisfies a predetermined condition. That is, when the discard determination unit 24 determines that the amount of change in the low frequency fL in the frequency characteristic of the latest captured image Pt is equal to or greater than the predetermined threshold Th21, the imaging state acquired by the state acquisition unit 23a is predetermined. It is determined that the above condition is satisfied. Then, when determining that the amount of change is less than a predetermined threshold Th21, the discard determination unit 24 determines that the imaging state does not satisfy a predetermined condition. Further, for example, as illustrated in FIG. 7B, the discard determination unit 24 determines whether or not the amount of change in the high frequency fH in the frequency characteristics of the latest captured image Pt acquired by the state acquisition unit 23a is equal to or greater than a predetermined threshold Th22. It is possible to determine whether or not the imaging state acquired by the state acquisition unit 23a satisfies a predetermined condition. That is, when the discard determination unit 24 determines that the amount of change in the high frequency fH in the frequency characteristic of the latest captured image Pt is equal to or greater than the predetermined threshold Th22, the imaging state acquired by the state acquisition unit 23a is predetermined. It is determined that the above condition is satisfied. When determining that the amount of change is less than a predetermined threshold Th22, the discard determination unit 24 determines that the imaging state does not satisfy a predetermined condition. Further, for example, the discard determination unit 24 has the change amount of the low frequency fL in the frequency characteristic of the latest captured image Pt acquired by the state acquisition unit 23 equal to or greater than a predetermined threshold Th21 and the change amount of the high frequency fH is predetermined. By determining whether or not the threshold Th22 is greater than or equal to the threshold value Th22, it is possible to determine whether or not the imaging state acquired by the state acquisition unit 23a satisfies a predetermined condition. That is, when the discard determination unit 24 determines that the amount of change in the low frequency fL in the frequency characteristics of the latest captured image Pt is equal to or greater than the predetermined threshold Th21 and the amount of change in the high frequency fH is equal to or greater than the predetermined threshold Th22. Is determined that the imaging state acquired by the state acquisition unit 23a satisfies a predetermined condition, and otherwise, it is determined that the imaging state does not satisfy the predetermined condition.

Alternatively, for example, as illustrated in FIG. 7C, the discard determination unit 24 determines whether the intensity of the low frequency fL in the frequency characteristics of the latest captured image Pt acquired by the state acquisition unit 23a is equal to or greater than a predetermined threshold Th31. It is possible to determine whether or not the imaging state acquired by the state acquisition unit 23a satisfies a predetermined condition. That is, when the discard determination unit 24 determines that the intensity of the low frequency fL in the frequency characteristics of the latest captured image Pt is equal to or greater than the predetermined threshold Th31, the imaging state acquired by the state acquisition unit 23a is a predetermined value. It is determined that the condition is satisfied. If the discard determination unit 24 determines that the intensity is less than the predetermined threshold Th31, the discard determination unit 24 determines that the imaging state does not satisfy the predetermined condition. Further, for example, as illustrated in FIG. 7C, the discard determination unit 24 determines whether the intensity amount of the high frequency fH in the frequency characteristics of the latest captured image Pt acquired by the state acquisition unit 23a is equal to or greater than a predetermined threshold Th32. It is possible to determine whether or not the imaging state acquired by the state acquisition unit 23a satisfies a predetermined condition. That is, when the discard determination unit 24 determines that the intensity amount of the high frequency fH in the frequency characteristic of the latest captured image Pt is equal to or greater than the predetermined threshold Th32, the imaging state acquired by the state acquisition unit 23a is predetermined. It is determined that the above condition is satisfied. If the discard determination unit 24 determines that the intensity is less than the predetermined threshold Th32, the discard determination unit 24 determines that the imaging state does not satisfy the predetermined condition. Further, for example, the discard determination unit 24 has the change amount of the low frequency fL in the frequency characteristic of the latest captured image Pt acquired by the state acquisition unit 23 equal to or greater than a predetermined threshold Th21 and the change amount of the high frequency fH is predetermined. By determining whether or not the threshold Th22 is greater than or equal to the threshold value Th22, it is possible to determine whether or not the imaging state acquired by the state acquisition unit 23a satisfies a predetermined condition. That is, when the discard determination unit 24 determines that the amount of change in the low frequency fL in the frequency characteristics of the latest captured image Pt is equal to or greater than the predetermined threshold Th21 and the amount of change in the high frequency fH is equal to or greater than the predetermined threshold Th22. Is determined that the imaging state acquired by the state acquisition unit 23a satisfies a predetermined condition, and otherwise, it is determined that the imaging state does not satisfy the predetermined condition.

The predetermined threshold values Th21, Th22, Th31, and Th32 are statistically appropriately set, for example, by measuring a plurality of samples.

In the above description, the state acquisition unit 23 acquires one feature amount, and the discard determination unit 24 determines the imaging state based on the one feature amount acquired by the state acquisition unit 23. The unit 23 may acquire a plurality of feature amounts, and the discard determination unit 24 may determine the imaging state based on the plurality of feature amounts acquired by the state acquisition unit 23.

In addition, when detecting a feature amount from such a captured image P, it is preferable that the state acquisition unit 23a uses a part of the latest captured image Pt. Thus, since the imaging state of the imaging unit 1 is acquired using a part of the latest captured image (input image) Pt, the load of information processing is reduced. A part of the latest captured image Pt may be, for example, a rectangular region Pt11 from the lower end closest to the moving body to the vanishing point in the latest captured image Pt1, as shown in FIG. 9A. In general, in the captured image P, for example, an area above the vanishing point of the road surface is often drawn in the sky, and can be excluded from the area for determining an obstacle to the movement of the moving body. For this reason, from the viewpoint of determining an obstacle to the movement of the moving body by setting a part of the latest captured image Pt in a region below the vanishing point, the distance measuring device DM determines the feature amount from the captured image P. It is possible to appropriately set the image area for detecting the image. Further, since it is important that an obstacle to the movement of the moving body is present in the traveling area of the moving body, for example, a part of the latest captured image Pt is traveled by the moving body as shown in FIG. 9B. It may be a region Pt21. The traveling area Pt21 of the moving object may be, for example, a triangular area (virtual traveling area) surrounded by both ends of the lower end closest to the moving object and the vanishing point in the latest captured image Pt2. The travel area Pt21 may be a triangular area (actual travel area) surrounded by a lower end closest to the moving body in the latest captured image Pt2 and a pair of white lines drawn on both ends of the road surface. In the example shown in FIG. 9B, a pair of white lines drawn at both ends of the road surface intersect at a vanishing point. The vanishing point is a point where parallel lines that are actually parallel intersect in the image, and occurs because the captured image is formed using the perspective method. Thus, by setting a part of the latest captured image Pt in the travel region Pt21 of the moving body, the area becomes narrower than the area of the rectangular region Pt11 shown in FIG. 9A, so that the information processing load is further reduced. .

Further, the state acquisition unit 23 described above may be a state acquisition unit 23b that acquires the imaging direction of the imaging unit 1 as the imaging state of the imaging unit 1. Since such a distance measuring device DM sets the imaging direction of the imaging unit 1 to the imaging state of the imaging unit 1, the moving direction of the moving body can be diverted to the imaging state of the imaging unit 1. It is possible to cope with a change in the moving direction of the moving body such as a right turn.

As for the imaging direction of the imaging unit 1, since the imaging unit 1 mounted on the moving body is normally arranged to match the moving direction of the moving body as described above, the moving direction of the moving body is acquired. Can be obtained. More specifically, for example, the distance measuring device DM described above is mounted on a moving body and further includes a gyro sensor that detects the moving direction of the moving body, and the state acquisition unit 23b is based on the detection result of the gyro sensor. The moving direction of the moving body, that is, the imaging direction of the imaging unit 1 is acquired. For example, when the moving body is a vehicle, the state acquisition unit 23b acquires the steering angle information from vehicle information that transmits a so-called CAN (Controller Area Network), and the moving body is based on the acquired steering angle information. Movement direction, that is, the imaging direction of the imaging unit 1 is acquired. In addition to the steering angle information, the vehicle information includes lateral acceleration, yaw rate, operation information of the skid prevention device, and the like, and the state acquisition unit 23b considers these information based on these information. Thus, the moving direction of the moving body, that is, the imaging direction of the imaging unit 1 may be acquired. Thus, when the state acquisition unit 23b acquires the imaging direction of the imaging unit 1, the discard determination unit 24 determines whether the imaging direction of the imaging unit 1 acquired by the state acquisition unit 23b satisfies a predetermined condition. It is possible to determine whether or not the imaging state satisfies a predetermined condition. For example, when the state acquisition unit 23b uses the steering angle as the imaging direction of the imaging unit 1, as the driver steers, the steering angle starts to start as shown in FIG. The steering starts to increase as the steering wheel starts rotating, and the steering is maintained during the change of direction of the moving body (because the steering wheel is fixed), so that the direction of the moving body becomes the desired direction of the driver. Since the steering is returned (as the steering wheel is returned), the profile starts to decrease. For this reason, for example, as shown in FIG. 9A, the discard determination unit 24 determines whether or not the steering angle acquired by the state acquisition unit 23b is equal to or greater than a predetermined threshold Th41 by the state acquisition unit 23b. It can be determined whether or not the acquired imaging state satisfies a predetermined condition. That is, when the discard determination unit 24 determines that the steering angle acquired by the state acquisition unit 23b is equal to or greater than the predetermined threshold Th41, the imaging state acquired by the state acquisition unit 23b satisfies a predetermined condition. Is determined. If the discard determination unit 24 determines that the steering angle is less than the predetermined threshold Th41, the discard determination unit 24 determines that the imaging state does not satisfy the predetermined condition. In addition, since the steering angle changes as described above, the amount of change (the difference between the steering angle when the latest captured image Pt is captured and the steering angle when the previous captured image P is captured) is As shown in FIG. 9B, a relatively large value is obtained only when steering is being performed (when the steering wheel is rotating). Since the change amount of the steering angle changes in this way, for example, as shown in FIG. 9B, the discard determination unit 24 determines that the change amount of the steering angle acquired by the state acquisition unit 23b is greater than or equal to a predetermined threshold Th42. By determining whether or not there is, it is possible to determine whether or not the imaging state acquired by the state acquisition unit 23b satisfies a predetermined condition. In other words, when the discard determination unit 24 determines that the change amount (absolute value) of the steering angle acquired by the state acquisition unit 23b is equal to or greater than the predetermined threshold Th42, the imaging acquired by the state acquisition unit 23b. It is determined that the state satisfies a predetermined condition. If the discard determination unit 24 determines that the change amount (absolute value) of the steering angle is less than the predetermined threshold Th42, the discard determination unit 24 determines that the imaging state does not satisfy the predetermined condition. These predetermined threshold values Th41 and Th42 are statistically appropriately set by measuring, for example, a plurality of samples.

As described above, in the distance measuring device DM and the distance measuring method implemented in the present embodiment, it is determined whether or not the imaging state satisfies a predetermined condition, and the reference information storage unit 31 is based on the determination result. It is determined whether or not the reference image TP stored in is to be discarded. For this reason, since the distance measuring device DM and the distance measuring method in this embodiment can determine whether or not the reference image TP stored in the reference information storage unit 31 should be discarded based on the determination result of the discarding, the imaging state Even when the change occurs, it is possible to reduce the above-described erroneous detection and non-detection and improve the detection accuracy.

In addition, the distance measuring device DM and the distance measuring method in the present embodiment discard the reference image TP stored in the reference information storage unit 31 when it is determined that the imaging state satisfies a predetermined condition. Detection and non-detection can be reliably prevented.

Further, in the distance measuring device DM and the distance measuring method according to the present embodiment, the user continuously determines that the imaging state satisfies a predetermined condition by the notification of the notification unit 25 using the notification unit 4. Can be recognized. When it is continuously determined that the imaging state satisfies a predetermined condition, it is presumed that a non-temporary abnormality such as sticking of a foreign object in the imaging unit 1 or a failure of the distance measuring device DM has occurred. Therefore, since the distance measuring device DM and the distance measuring method in the present embodiment include the notification unit 25, it is possible to prompt the user to take appropriate measures such as inspecting the imaging unit 1, for example. Therefore, the threshold value Thn is set to an appropriate number of times according to the frame rate, for example, 15 times or 30 times, from this viewpoint.

This specification discloses various modes of technology as described above, and the main technologies are summarized below.

A distance measuring apparatus according to an aspect is mounted on a moving body, and an imaging unit that continuously images a subject in time and one of time-series captured images captured by the imaging unit as a reference image Based on the reference information storage unit to be stored, the latest latest captured image of the time-series captured images captured by the imaging unit, and the reference image stored in the reference information storage unit, A distance calculation unit that obtains a distance to a predetermined object included, a state acquisition unit that acquires an imaging state of the imaging unit, and whether or not the imaging state acquired by the state acquisition unit satisfies a predetermined condition And a discard determination unit that determines whether to discard the reference image stored in the reference information storage unit based on the determination result. The distance measuring method according to another aspect is mounted on a moving body and is one of an imaging process for imaging a subject continuously in time, and a time-series captured image captured by the imaging process. A reference image stored in the reference information storage unit, a latest latest captured image of the time-series captured images captured in the imaging step, and the reference image stored in the reference information storage unit A distance calculation step for obtaining a distance to a predetermined object included in the latest captured image, a state acquisition step for acquiring an imaging state of the imaging unit, and the imaging state acquired by the state acquisition step are predetermined. A discard determination step of determining whether or not a condition is satisfied and determining whether or not to discard the reference image stored in the reference information storage unit based on the determination result.

In such a distance measuring apparatus and distance measuring method, it is determined whether or not the imaging state satisfies a predetermined condition, and the reference image stored in the reference information storage unit is discarded based on the determination result. It is determined whether or not. For this reason, since the distance measuring apparatus and the distance measuring method can determine whether or not the reference image stored in the reference information storage unit should be discarded based on the determination result of the discard, the imaging state changes. Even in this case, the detection accuracy can be improved.

In another aspect, in the distance measuring device described above, when the discard determination unit determines that the imaging state acquired by the state acquisition unit satisfies a predetermined condition, the discard determination unit stores the reference information storage unit. The stored reference image is discarded. Preferably, when the reference image stored in the reference information storage unit is discarded, the distance measuring device is captured by the imaging unit after capturing a predetermined number of images after capturing the latest captured image. And a reference information update unit that stores the captured image as a new reference image in the reference information storage unit.

Since such a distance measuring device discards the reference image stored in the reference information storage unit when it is determined that the imaging state satisfies a predetermined condition, the above-described erroneous detection and non-detection are ensured. Can be prevented.

In another aspect, in the above-described distance measuring device, the state acquisition unit acquires the imaging state of the imaging unit based on a predetermined feature amount in the latest captured image captured by the imaging unit. . Preferably, in the distance measuring apparatus, the state acquisition unit acquires a statistic (for example, an average value or a median value) of a pixel value or a luminance value as the predetermined feature amount. Preferably, in the distance measuring apparatus, the state acquisition unit acquires a frequency characteristic in the latest captured image as the predetermined feature amount.

Since such a distance measuring device acquires the imaging state of the imaging unit based on a predetermined feature amount in the latest captured image captured by the imaging unit, an apparatus for acquiring the imaging state of the imaging unit Is not required separately.

In another aspect, in the distance measuring device described above, the state acquisition unit uses a part of the latest captured image.

Such a distance measuring device acquires the imaging state of the imaging unit by using a part of the latest captured image, so that the load of information processing can be reduced.

In another aspect, in the above-described distance measuring device, the state acquisition unit acquires an imaging direction of the imaging unit as an imaging state of the imaging unit.

Usually, the imaging unit is arranged on the moving body so that the imaging direction coincides with the moving direction of the moving body. Therefore, since such a distance measuring device sets the imaging direction of the imaging unit to the imaging state of the imaging unit, the moving direction of the moving body can be diverted to the imaging state of the imaging unit. It is possible to cope with changes in the moving direction of the moving body, such as turning right or turning.

In another aspect, in the above-described distance measuring device, the number of continuous determinations continuously determined by the discard determination unit when the imaging state acquired by the state acquisition unit satisfies a predetermined condition is a threshold value. In the case described above, the information processing apparatus further includes a notification unit that notifies the outside that the imaging state acquired by the state acquisition unit satisfies a predetermined condition.

In such a distance measuring device, the user can recognize that the imaging state is continuously determined to satisfy a predetermined condition by the notification of the notification unit. Therefore, since such a distance measuring device includes the notification unit, it is possible to prompt the user to take appropriate measures such as checking the imaging unit.

This application is based on Japanese Patent Application No. 2013-130709 filed on June 21, 2013, the contents of which are included in this application.

In order to express the present invention, the present invention has been properly and fully described through the embodiments with reference to the drawings. However, those skilled in the art can easily change and / or improve the above-described embodiments. It should be recognized that this is possible. Therefore, unless the modifications or improvements implemented by those skilled in the art are at a level that departs from the scope of the claims recited in the claims, the modifications or improvements are not covered by the claims. To be construed as inclusive.

According to the present invention, a distance measuring device and a distance measuring method can be provided.

Claims

An imaging unit that is mounted on a moving body and continuously images a subject; and
A reference information storage unit that stores one of time-series captured images captured by the imaging unit as a reference image;
The distance to a predetermined object included in the latest captured image based on the latest latest captured image of the time-series captured images captured by the imaging unit and the reference image stored in the reference information storage unit A distance calculation unit for obtaining
A state acquisition unit for acquiring an imaging state of the imaging unit;
It is determined whether the imaging state acquired by the state acquisition unit satisfies a predetermined condition, and whether to discard the reference image stored in the reference information storage unit based on the determination result A ranging device comprising: a discard determination unit for determining.
The discard determination unit discards the reference image stored in the reference information storage unit when it is determined that the imaging state acquired by the state acquisition unit satisfies a predetermined condition. Item 3. The distance measuring device according to Item 1.
The measurement state according to claim 1 or 2, wherein the state acquisition unit acquires an imaging state of the imaging unit based on a predetermined feature amount in the latest captured image captured by the imaging unit. Distance device.
The distance measuring device according to claim 3, wherein the state acquisition unit uses a part of the latest captured image.
The distance measuring device according to claim 1, wherein the state acquisition unit acquires an imaging direction of the imaging unit as an imaging state of the imaging unit.
When the imaging state acquired by the state acquisition unit satisfies a predetermined condition, the imaging acquired by the state acquisition unit when the number of continuous determinations continuously determined by the discard determination unit is greater than or equal to a threshold value The distance measuring device according to any one of claims 1 to 5, further comprising a notification unit that notifies the outside that the state satisfies a predetermined condition.
An imaging process that is mounted on a moving body and images a subject continuously in time,
A storage step of storing one of the time-series captured images captured by the imaging step as a reference image in a reference information storage unit;
The distance to a predetermined object included in the latest captured image based on the latest latest captured image of the time-series captured images captured in the imaging step and the reference image stored in the reference information storage unit A distance calculation step for obtaining
A state acquisition step of acquiring an imaging state of the imaging unit;
It is determined whether or not the imaging state acquired in the state acquisition step satisfies a predetermined condition, and whether or not to discard the reference image stored in the reference information storage unit based on the determination result And a discard determination step for determining.