JP2018074411A - Object detection device and object detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology that improves detection accuracy when an object is detected by comparing images.SOLUTION: In an object detection system 10, an image display device 2 includes a movement amount detection unit 23 that detects movement of a vehicle, an image obtaining unit 21 that is mounted on the vehicle ans obtains a first image and a second image that have been captured at different times through a camera 1 that captures the surroundings of the vehicle on the basis of movement of the vehicle, a resolution correction unit 24 that reduces a resolution of the second image and corrects the second image in accordance with a resolution of the first image, and an object detection unit 25 that detects an object that has been captured in the first image and the second image on the basis of a difference between the corrected first image and second image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an object detection device and an object detection method.

  2. Description of the Related Art Conventionally, techniques for detecting an object that can be an obstacle to vehicle travel and avoiding a collision with the object have been developed. Therefore, in order to detect such an object that is a candidate for an obstacle around the vehicle, there is a technique for capturing the periphery of the vehicle using a camera or the like and processing the captured image. There is a background difference method (also referred to as an inter-frame difference method) as one method for detecting an object displayed in the image.

  In the background subtraction method, for two images (current frame and past frame) taken at different times, the distance moved from the acquisition of the past frame to the acquisition of the current frame is obtained, and based on the moved distance Then, the difference between the shooting positions of the past frame and the current frame is corrected (position correction), and an object is detected from the difference between the corrected images. In addition, as a technique relevant to this invention, there exist patent documents 1-2, for example.

JP 2011-198241 A JP 2009-76030 A

  When an object is detected by the background difference method as described above, the resolution differs between the past frame and the current frame, so even if the accuracy of position correction is high, if an object is detected from these differences, the object is accurately detected. It is difficult to detect.

  For example, if an object (road surface, obstacle, etc.) that is far away in the past frame appears near in the current frame, even if it is the same object, the resolution of the object is low in the past frame, so it is blurred The image is clear in the current frame because the resolution of the object is high. For this reason, when these are compared, even if an object such as a road surface where a difference should not originally appear, the difference remains, and the road surface or the like may be erroneously detected as an obstacle.

  Therefore, an object of the present invention is to provide a technique for improving detection accuracy when an object is detected by comparing images.

The object detection apparatus of the present invention solves the above-described problem,
A movement detector for detecting movement of the vehicle;
An image acquisition unit that is mounted on the vehicle and acquires a first image and a second image captured at different times based on movement of the vehicle from an imaging device that captures the surroundings of the vehicle;
A resolution correction unit that corrects the resolution of the second image by reducing the resolution according to the resolution of the first image;
An object detection unit that detects an object photographed in the first image and the second image based on a difference between the corrected first image and the second image;
Is provided.
In addition, the object detection device of the present invention solves the above-described problem,
An image acquisition unit that is mounted on a vehicle and acquires a first image and a second image that are captured at different times from a photographing device that photographs the surroundings of the vehicle;
A resolution correction unit that corrects the resolution of the second image by reducing the resolution according to the resolution of the first image;
An object detection unit that detects an object photographed in the first image and the second image based on a difference between the corrected first image and the second image;
Is provided.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which improves the detection accuracy at the time of detecting an object by comparing an image can be provided.

FIG. 1 is a block diagram illustrating a configuration of an object detection system according to the first embodiment. FIG. 2 is a diagram illustrating the arrangement of the cameras and the direction in which the camera captures images. FIG. 3 is an explanatory diagram of position correction by the position correction unit. FIG. 4 is a diagram illustrating an example of a hardware configuration of the object detection apparatus. FIG. 5 is an explanatory diagram of an object detection method executed by the object detection apparatus. FIG. 6 is a diagram for explaining the effect of resolution correction in the object detection method according to the present embodiment. FIG. 7 is an explanatory diagram of an object detection method when a camera is provided at the front of the vehicle. FIG. 8 is an explanatory diagram of an object detection method executed by the object detection apparatus according to the second embodiment. FIG. 9 is an explanatory diagram of an object detection method when a camera is provided at the front of the vehicle.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The configuration of the following embodiment is an exemplification, and the present invention is not limited to the configuration of the embodiment.
<Embodiment 1>
<System configuration>
First, the configuration of the object detection system 10 of the present embodiment will be described. FIG. 1 is a block diagram illustrating a configuration of an object detection system 10 according to the first embodiment. The object detection system 10 includes a camera 1, an object detection device 2, and a display device 3. The object detection system 10 is mounted on a vehicle such as an automobile, and has a function of performing driving assistance during reverse travel by displaying an image behind the vehicle. The object detection system 10 may be, for example, an audio / visual / navigation integrated electronic device (hereinafter also referred to as an AVN machine) mounted on a vehicle. In addition, the object detection system 10 may have a function of, for example, displaying a guide line as parking assistance superimposed on an image behind the vehicle.

  The camera 1 is a photographing device that obtains a photographed image by photographing the rear of the vehicle. The camera 1 includes a lens and an image sensor and electronically acquires a captured image. The camera 1 acquires captured images at a predetermined cycle (for example, 1/30 second cycle), and inputs a video signal having each captured image as one frame of a moving image to the object detection device 2.

FIG. 2 is a diagram illustrating the arrangement of the camera 1 and the direction in which the camera 1 captures images. As shown in FIG. 2, the camera 1 is provided on a rear door 92 at the rear end of the vehicle, and its optical axis 15 is directed rearward along the front-rear direction of the vehicle and is directed horizontally or below the horizontal. Provided. That is, the camera 1 of this example is a back camera that captures the state of the rear of the vehicle, and is installed so as to acquire a parking frame and an image around the parking frame, for example, when moving backward and parking. . Further, a fish-eye lens is adopted as the lens of the camera 1 and has an angle of view θ of 180 to 150 degrees, for example. The lens of the camera 1 is not limited to a fisheye lens, but may be any lens having an angle of view (for example, a wide-angle lens of 150 to 100 degrees) that can capture a required region.

  Returning to FIG. 1, when the object detection device 2 detects an object around the vehicle based on a captured image acquired by the camera 1 and determines that the obstacle is an obstacle to driving, the object detection device 2 (mainly driving) Display to call attention. The object detection device 2 includes an image acquisition unit 21, a position correction unit 22, a movement amount detection unit 23, a resolution correction unit 24, an object detection unit 25, a shift position acquisition unit 26, an obstacle determination unit 27, and an output control unit 28. I have. The movement amount detection unit 23 and the shift position acquisition unit (movement direction detection unit) 26 constitute a movement detection unit of the present invention.

  The image acquisition unit 21 acquires a captured image from the camera 1, and acquires a first image and a second image captured at different times based on the movement of the vehicle from the captured image. For example, when the moving direction of the vehicle is backward, the image acquisition unit 21 acquires an image taken at a time before the second image as the first image, and when the moving direction of the vehicle is forward, An image taken at a time later than the second image is acquired as the first image.

  The position correction unit 22 corrects the viewpoint position of the first image acquired by the image acquisition unit 21 so as to match the second image. FIG. 3 is an explanatory diagram of position correction by the position correction unit 22. Since the first image and the second image have different shooting times, when the vehicle moves during this time, as shown in FIG. 3A, the position of the camera 1 when the first image is shot ( A position 81 (also referred to as a viewpoint position) is different from a position (viewpoint position) 82 of the camera 1 when the second image is taken. For this reason, even when a common subject is photographed in the first image and the second image, the subject is photographed differently in the first image and the second image.

  FIG. 3B shows an example of a subject 83 that is photographed in common for the first image and the second image. The subject 83 is a pattern drawn on the road surface 84. In this example, the subject 83 has a shape obtained by dividing a square into three vertically and horizontally. When the subject 83 is photographed from the viewpoint position 81, the optical axis 15 is inclined with respect to the road surface 84 as shown in FIG. The far sides a <b> 1 to a <b> 4 are deformed shorter than the near sides d <b> 1 to d <b> 4 and photographed. When the subject 83 is photographed from the viewpoint position 82, the subject 83B shown in the second image 72 is closer to the camera 1 than the subject 83A of the first image 71, as shown in FIG. It appears in the position, that is, the lower end side of the second image 72, and the degree of deformation of the sides A1 to A4 and the like is reduced. Note that the distance between the viewpoint position 81 and the viewpoint position 82 is, for example, 75 mm, and the difference between the subject 83A on the first image 71 and the subject 83B on the second image 72 is not large. 3 is exaggerated for the sake of explanation.

  Then, the position correction unit 22 is configured so that the points a1 to a4, b1 to b4, and c1 to c4 of the subject 83A of the first image 71 correspond to the second image 72 as illustrated in FIG. The subject 83A of the first image 71 has the same shape as the subject 83B of the second image 72 by converting the coordinates so that the positions of the points A1 to A4, B1 to B4, and C1 to C4 are obtained. . As described above, by converting the coordinates of each pixel in the first image 71 based on the difference between the first image 71 and the second image 72, the first image 71 can be viewed from the viewpoint of the second image 72. The image is converted like an image photographed at the position 82. Hereinafter, this process is referred to as viewpoint conversion or position correction.

  In addition, when the first image 71 and the second image 72 are photographed with a fish-eye lens or a wide-angle lens and a linear subject is curved and photographed due to distortion, the position correction unit 22 corrects this distortion. You may correct | amend distortion so that it may return to linear form.

The movement amount detection unit 23 has a function of detecting the movement amount of the vehicle using a captured image taken by the camera 1. For example, the movement amount detection unit 23 extracts a portion displaying the same subject in the captured images that change in time, and derives the movement amount of the vehicle based on the relationship between the display positions.

  As a method for extracting pixels displaying the same subject, for example, a template matching method can be used. The template matching method is a method for searching a position similar to a template image designated in advance from images. When used in the present embodiment, for example, a process of comparing a captured image of a previous frame using a partial image of a captured image of the previous frame as a template may be executed over the entire image. As such a template matching method, for example, a method such as SAD (Sum of Absolute Difference) or SSD (Sum of Squared Difference) can be used.

  In addition to the template matching method, a feature point method can also be used. The feature point method is a method using so-called optical flow. That is, feature points are extracted from each of a plurality of photographed images acquired at different points in time, and an optical flow indicating the movement of the feature points between the plurality of overhead images is derived. A feature point is a point that can be detected prominently from an image, for example, a point such as a corner of an object (intersection of edges). In the feature point method, the feature points extracted from the latest bird's-eye view image are associated with the feature points extracted from the past bird's-eye view image, and a movement vector is derived based on the positions of the associated feature points. .

  In many cases, a captured image includes both a portion where a road surface is captured and a portion where a three-dimensional object such as an automobile is displayed. Since the road surface is a plane and not a moving object, all the movement vectors of the road surface are movement vectors having substantially the same direction and the same size. On the other hand, the movement vector of the three-dimensional object is a movement vector having a different direction and size depending on the movement direction and height of the three-dimensional object.

  For this reason, the movement vector of the three-dimensional object does not accurately reflect the movement amount of the vehicle, and the movement vector of the road surface accurately reflects the movement amount of the vehicle. Therefore, the movement vector derived by the movement amount detection unit 23 includes movement vectors of various directions and sizes. In order to accurately derive the movement amount of the vehicle, the movement vector is determined from among them. For example, it is desirable to extract a road movement vector.

  In general, an image obtained by photographing the periphery of a vehicle includes the most part for displaying the road surface, and thus usually has the largest number of movement vectors related to the road surface. Therefore, in the present embodiment, a size-related histogram is generated for all the movement vectors, and the movement vector having the largest number is extracted as a road surface movement vector.

  Then, the movement amount detection unit 23 has the feature points of the upper part of the photographed image, that is, the part where the distant road surface is photographed in FIG. 3A, and the lower part of the photographed image, that is, the nearby road surface in FIG. When a vector that moves to a portion to be photographed is detected, it is detected that the movement vector of the vehicle is a backward vector, that is, that the vehicle has moved backward and its amount (movement amount). On the contrary, when the feature point photographed at the lower part of the photographed image moves to the upper part of the photographed image, it is detected that the vehicle has moved forward and its amount (movement amount).

  Although the movement amount detection unit 23 detects the movement vector of the vehicle using the captured image, the present invention is not limited to this. For example, the movement amount detection unit 23 may detect the movement amount of the vehicle using an acceleration sensor provided in the AVN machine. Alternatively, the movement amount may be obtained by obtaining a vehicle speed pulse or a shift position from the ECU of the vehicle.

In the resolution correction unit 24, the distance between the subject photographed in common with the first image and the second image and the vehicle is shorter at the time when the second image is photographed than when the first image is photographed. In this case, correction is performed by reducing the resolution of the first image in accordance with the resolution of the second image. For example, the resolution is lowered by processing the first image with a spatial filter such as an averaging filter, a Gaussian filter, or a median filter.

  The object detection unit 25 compares the first image after the resolution correction with the second image, and the object (for example, parking) taken in the first image and the second image from the difference of each pixel. A three-dimensional object that becomes an obstacle to the above. The object detection unit 25 may detect the object as an object when the difference exceeds a predetermined threshold. In the example of FIG. 3 described above, the position correction is performed so that the subject 83A of the first image is the same as the subject 83B of the second image. Since the subjects 83A and 83B are a pattern drawn on the road surface 84, signs and parking frames drawn on the road surface, which are almost at the same height as the road surface, match between the first image and the second image. However, what exists at a height different from the road surface, that is, a three-dimensional object, does not match between the first image and the second image. For this reason, the object detection unit 25 detects a portion where a difference has occurred between the first image and the second image as an object image. That is, it is detected that an object (a three-dimensional object) exists at a position on the object plane corresponding to the portion where the difference occurs on the image. Note that the object detection unit 25 of the present embodiment specifies the position of the detected object on the second image that is captured later, out of the first image and the second image.

  The shift position acquisition unit 26 acquires a signal indicating the shift position of the host vehicle from an ECU (not shown) or the like. The shift position indicates the position of a shift lever (also referred to as a shift knob, shifter, select lever, change lever, etc.), that is, the state of the transmission operated by the shift lever. That is, the moving direction of the vehicle is indicated by the shift position. In the present embodiment, the shift position acquisition unit 26 corresponds to a movement direction detection unit.

  The obstacle determination unit 27 uses the object detection unit 25 to detect an object that becomes a driving obstacle among the detected objects as an obstacle. For example, the rudder angle of the vehicle is acquired from the ECU, the traveling path of the vehicle is predicted, and an object on and near the traveling path is detected as an obstacle. Note that the obstacle determination unit 27 may be omitted in the case where all the obstacles are not narrowed down from the object (three-dimensional object) detected by the object detection unit 25.

  The output control unit 28 acquires the first image or the second image (in the present embodiment, the first image), outputs it to the display device 3 as a video signal, and displays it. At this time, the output control unit 28 acquires the information of the obstacle detected from the image from the obstacle determination unit 27, and in the image, the mark indicating the obstacle is displayed on the part of the object extracted as the obstacle. (Hereinafter also referred to as a warning pattern). The warning pattern is, for example, a line, shading, a figure, a color, or a luminance change surrounding the object. Moreover, the output control part 28 may synthesize | combine a parking guide line with a picked-up image or a bird's-eye view image.

  Further, the output control unit 28 of the present embodiment displays the second image on the display device 3 when the shift position indicates the R range, that is, the reverse of the vehicle, and when the shift position is other than the R range. Causes the display device 3 to display an image other than the second image, for example, a navigation screen of an AVN machine, an operation screen of an audio function, or the like.

  The display device 3 is arranged at a position where a user (mainly a driver) can visually recognize the vehicle interior of the vehicle, and displays various types of information to the user. The display device 3 may be an AVN display device having a navigation function for guiding a route to a destination and an audio function for reproducing music.

FIG. 4 is a diagram illustrating an example of a hardware configuration of the object detection device 2. As illustrated in FIG. 4, the object detection apparatus 2 includes a CPU (Central Processing Unit) 201, a memory 202, and an input / output IF 203 that are connected to each other via a connection bus 210. The CPU 201 is a central processing unit that controls the entire object detection apparatus 2. The CPU 201 is also called a processor. However, the CPU 201 is not limited to a single processor and may have a multiprocessor configuration. A single CPU 201 connected by a single socket may have a multi-core configuration.

The memory 202 includes a main storage device and an auxiliary storage device. The main storage device is used as a work area for the CPU 201, a storage area for programs and data, and a buffer area for communication data. The main storage device is formed by, for example, a random access memory (RAM) or a combination of a RAM and a read only memory (ROM). The main storage device is a storage medium in which the CPU 201 caches programs and data and expands a work area. The main storage device includes, for example, a flash memory, a RAM (Random Access Memory), and a ROM (Read Only Memory). The auxiliary storage unit is a storage medium that stores programs executed by the CPU 201, operation setting information, and the like. The auxiliary storage device is, for example, an HDD (Hard-disk Drive), an SSD (Solid State Drive), an EPROM (Erasable Programmable ROM), a flash memory, a USB memory, a memory card, or the like.

  The input / output IF 203 is an interface for inputting / outputting data to / from devices such as sensors, operation units, communication modules, and ECUs connected to the object detection device 2. In the object detection system 10, a selection operation by the user is input from the operation unit to the object detection device 2 via the input / output IF 203. In addition, a plurality of the above-described components may be provided, or some of the components may not be provided. Further, the above-described components may be included in the components of the AVN machine.

In the object detection device 2, the CPU 201 executes the program, whereby the image acquisition unit 21, the position correction unit 22, the movement amount detection unit 23, the resolution correction unit 24, the object detection unit 25, and the shift position acquisition unit 26 illustrated in FIG. 1. It functions as each processing unit of the obstacle determination unit 27 and the output control unit 28. However, at least a part of the processing of each processing unit may be provided by a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or the like. Further, at least a part of each of the processing units may be a dedicated LSI (large scale integration) such as an FPGA (Field-Programmable Gate Array) or other digital circuit. Moreover, it is good also as a structure which includes an analog circuit in at least one part of each said process part.

<Display control method>
FIG. 5 is an explanatory diagram of an object detection method executed by the object detection device 2. The object detection device 2 periodically executes the process of FIG. 5 when a start instruction is input or when the ACC power supply of the vehicle is turned on.

  First, the object detection device 2 acquires a shift position and determines whether the moving direction of the vehicle is reverse. If the moving direction of the vehicle is reverse (step S10, Yes), the object detection device 2 acquires the first image from the captured image acquired from the camera 1 (step S20).

  Further, the object detection device 2 obtains the amount of movement of the vehicle based on the optical flow of the photographed image acquired from the camera 1, and obtains the amount of movement of the vehicle calculated from the time when the first image was photographed (step S30). ). Then, the object detection device 2 determines whether or not the movement amount of the vehicle has reached a predetermined threshold (for example, 75 mm) (step S40). The threshold value is not limited to 75 mm, and can be arbitrarily set according to required accuracy, camera specifications, vehicle speed, and the like, such as 30 to 100 mm.

  If the amount of movement of the vehicle has reached a predetermined threshold value, the captured image captured at this time is acquired as the second image (step S50).

  The object detection device 2 processes the second image acquired in step S50 with a spatial filter to reduce the resolution, and corrects the resolution of the second image to match the first image (step S60). That is, in step S60, since the vehicle is moving backward, the second image acquired in step S50 is photographed closer to the common subject than the first image acquired in step S20 first. Since the resolution of the second image is higher, the resolution is lowered according to the first image.

  Next, the object detection device 2 compares the first image with the second image, and detects a three-dimensional object (object) photographed in the first image and the second image by the background subtraction method. (Step S70). In addition, the object detection device 2 predicts the travel path of the vehicle based on the moving direction and the steering angle of the vehicle, and among the objects detected in step 70, the object existing on the travel path and the object close to the travel path. Is detected as an obstacle (step S80).

  And the object detection apparatus 2 synthesize | combines the warning pattern which shows that it is an obstruction in the part of the object extracted as an obstruction in a 2nd image based on the information of the obstruction detected by step S80 ( Step S90). Moreover, the object detection apparatus 2 outputs the 2nd image which synthesize | combined the warning pattern to the display apparatus 3 as a video signal, and displays it on the display apparatus 3 (step S100).

  On the other hand, if the moving direction of the vehicle is not reverse in step S10 (No in step S10), the object detection device 2 acquires a second image from the captured image acquired from the camera 1 (step S120).

  Further, the object detection device 2 obtains the amount of movement of the vehicle based on the optical flow of the photographed image acquired from the camera 1, and obtains the amount of movement of the vehicle calculated from the time when the second image was photographed (step S130). ). Then, the object detection device 2 determines whether or not the amount of movement of the vehicle has reached a predetermined threshold (for example, 75 mm) (step S140).

  If the amount of movement of the vehicle has reached a predetermined threshold value, the captured image captured at this time is acquired as the first image (step S150).

  The object detection device 2 processes the second image acquired in step S120 with a spatial filter to reduce the resolution, and corrects the resolution of the second image to match the first image (step S160). That is, in step S160, since the vehicle is moving forward, the second image acquired in step S120 is taken closer to the common subject than the first image acquired in step S150. Since the image has a higher resolution, the resolution is lowered in accordance with the first image.

  Next, the object detection device 2 compares the first image with the second image, and detects a three-dimensional object (object) photographed in the first image and the second image by the background subtraction method. (Step S170). In addition, the object detection device 2 predicts the road of the vehicle based on the moving direction and the steering angle of the vehicle, and among the objects detected in step 170, the object existing on the road and the object close to the road Is detected as an obstacle (step S180).

In the present embodiment, when the moving direction of the vehicle is forward, the image captured by the camera 1 is not displayed on the display device 3, so the processing until the obstacle is detected (steps S 120 to S 180) is performed. Processing for pattern synthesis and output to the display device 3 is not performed. As a result, for example, even when the shift position is switched from reverse to forward because of switching back when parked, and back again, the captured image immediately after switching to reverse using the information on obstacles obtained during forward is warned. Patterns can be synthesized and output to the display device 3.

  Further, the object detection system 10 of the present embodiment includes a dedicated display device that displays an obstacle, and similarly uses the information on the obstacle detected in step S180 during advance, and is similar to steps S90 and S100 during update. Alternatively, a warning pattern may be combined with the second image and output to a dedicated display device for display.

<Effect of resolution correction>
FIG. 6 is a diagram for explaining the effect of resolution correction in the object detection method according to the present embodiment. In FIG. 6, the same positions on the road surface of the first image 41 and the second image 43 are enlarged and shown as enlarged images 42 and 44. That is, the enlarged images 42 and 44 are portions where a common subject is photographed.

  When the enlarged images 42 and 44 are compared, the outline of the enlarged image 42 is blurred, whereas the enlarged image 44 displays a fine outline shape. For this reason, when the enlarged images 42 and 44 are compared with each other and an object is detected, a difference in the contour portion is erroneously detected as an object as in the enlarged image 45.

  Therefore, the second image 43 is processed by the spatial filter by the process of FIG. 5, and the resolution is lowered to match the resolution of the first image as in the enlarged image 46. Thereby, in the enlarged image 46, the outline portion is blurred as much as the enlarged image 42, and the difference is reduced. For this reason, when the enlarged images 42 and 46 are compared to detect an object, erroneous detection does not occur in the contour portion as in the enlarged image 47.

<Effects of Embodiment>
As described above, according to the first embodiment, the correction is performed by reducing the resolution of the second image in accordance with the resolution of the first image, and the corrected first image and the second image are compared. Based on the difference, an object photographed in the first image and the second image is detected.

  Thereby, the object detection apparatus 2 of the present embodiment suppresses the difference in resolution due to the shooting positions of the first image and the second image being different, and compares the first image with the second image to detect the object. The detection accuracy at the time of detection can be improved.

  Further, the object detection device 2 of the first embodiment includes a shift position acquisition unit 26 (movement direction detection unit) that detects the moving direction of the vehicle, and is installed so that the camera 1 captures the rear of the vehicle and acquires an image. Based on the shift position (movement direction of the vehicle) detected by the shift position acquisition unit 26, the unit 21 uses the first image captured at a time before the second image when the vehicle movement direction is reverse. When the moving direction of the vehicle is forward, an image taken at a time later than the second image is acquired as the first image.

  Thereby, the object detection device 2 of the present embodiment appropriately acquires the first image and the second image based on the moving direction of the vehicle, and positions the common subject closer to the first image than the first image. It is possible to reduce the resolution of the second image photographed in step 1 and improve the detection accuracy when detecting the object by comparing the first image and the second image.

  In the object detection device 2 according to the first embodiment, the shift position acquisition unit (movement direction detection unit) 26 detects the movement direction of the vehicle based on the shift position of the vehicle.

  Thereby, the object detection apparatus 2 of this embodiment can detect the moving direction of a vehicle reliably at the time of a turnback.

  In addition, the object detection device 2 of the first embodiment includes a movement amount detection unit 23 that detects the movement amount of the vehicle, and the image acquisition unit 21 has one of the first image and the second image. An image captured at the time when the moving amount of the vehicle calculated from the captured time reaches the threshold is acquired as the other image.

  Thereby, the object detection apparatus 2 according to the present embodiment acquires the first image and the second image at appropriate timing, and detects the object by comparing the first image and the second image. Accuracy can be improved. For example, if there is no movement on the vehicle side, there is little difference even when comparing images at different times, so that when the amount of movement of the vehicle reaches a threshold value, this difference will have an appropriate magnitude. By performing object detection, an object that is stationary properly can be detected.

<Modification>
In the first embodiment, the camera 1 is provided at the rear of the vehicle to detect an obstacle behind the vehicle. However, the present invention is not limited to this, and the camera (imaging device) is provided at the front of the vehicle. It may be configured to detect a front obstacle. FIG. 7 is an explanatory diagram of an object detection method when a camera is provided at the front of the vehicle. Compared with the above-described first embodiment, the present modified example has a configuration in which the camera 1 is provided at the front of the vehicle and the image captured by the camera 1 is displayed on the display device 3 even when moving forward, and a part of the processing shown in FIG. Different, other configurations are the same. For this reason, the description of the same element is omitted.

  In this modified example, since the camera is provided in the front part of the vehicle, the process in FIG. 7 is opposite to the process in the forward direction and the process in the reverse direction as compared to FIG. 5 according to the first embodiment. . For example, in FIG. 7, when it is determined in step S10A that the moving direction of the vehicle is forward, the process proceeds to step S20. In step S10A, when it is determined that the moving direction of the vehicle is forward, the process proceeds to step S120. The processes in steps S20 to S100 and steps S120 to S180 are the same as those in FIG. In the process of FIG. 7, at the time of reverse travel, after detecting an obstacle at Step 180, the object detection device 2 extracts the part of the object extracted as an obstacle from the second image based on the obstacle information. Then, a warning pattern indicating an obstacle is synthesized (step S190). Moreover, the object detection apparatus 2 outputs the 2nd image which synthesize | combined the warning pattern to the display apparatus 3 as a video signal, and displays it on the display apparatus 3 (step S100).

  As described above, according to this modification, the camera 1 is installed so as to capture the front of the vehicle, and the image acquisition unit 21 is based on the vehicle movement direction detected by the shift position acquisition unit 26. When the vehicle is moving forward, an image taken at a time earlier than the second image is acquired as the first image, and when the vehicle is moving backward, the image is taken at a time later than the second image. The acquired image is acquired as the first image.

  As a result, the object detection device 2 of the present modified example generates the first image and the second image based on the moving direction of the vehicle even when the camera 1 is provided in the vehicle unit to detect an object in front of the vehicle. When properly detecting and reducing the resolution of the second image taken at a close position of a common subject compared to the first image, and comparing the first image with the second image to detect the object Detection accuracy can be improved.

<Embodiment 2>
In the first embodiment described above, a process for detecting an object when the amount of movement of the vehicle reaches a predetermined threshold is shown, and mainly a stationary object. The second embodiment is configured to detect a moving object in addition to the configuration of the first embodiment described above. In the second embodiment, the same elements as those of the first embodiment are denoted by the same reference numerals and the description thereof is omitted.

  FIG. 8 is an explanatory diagram of an object detection method executed by the object detection device 2 according to the second embodiment. First, the object detection device 2 acquires a shift position and determines whether the moving direction of the vehicle is reverse. If the moving direction of the vehicle is reverse (step S210, Yes), the object detection device 2 acquires a second image from the captured image acquired from the camera 1 (step S220).

  Further, the object detection device 2 acquires a captured image that is a frame immediately before the second image acquired in step S220 among the captured images acquired from the camera 1 as a first image (step S230). In the present embodiment, the frame immediately before the second image is the first image. However, the present invention is not limited to this, and a predetermined number of frames before the second image may be the first image. That is, the first image may be acquired with a predetermined time interval from the second image.

  The object detection device 2 obtains the amount of movement of the vehicle from the time when the first image was captured to the time when the second image was captured based on the optical flow of the captured image acquired from the camera 1 (step S240). ).

  The object detection apparatus 2 processes the second image acquired in step S230 with a spatial filter to reduce the resolution. Here, the object detection device 2 determines a filter coefficient based on the movement amount of the vehicle obtained in step S240, and sets the resolution of the second image to the first image with a spatial filter having a coefficient corresponding to the movement amount of the vehicle. (Step S250). For example, when the amount of movement of the vehicle is large and the difference between the resolutions of the first image and the second image is large, the degree to which the resolution is reduced is increased, the amount of movement of the vehicle is large, and the first image and the first image When the difference between the resolutions of the two images is small, the filter coefficient is set so as to reduce the degree of lowering the resolution.

  Next, the object detection device 2 compares the first image with the second image, and detects a three-dimensional object (object) photographed in the first image and the second image by the background subtraction method. And stored in the memory (step S260). Moreover, the object detection apparatus 2 detects the moving object compared with the past detection result memorize | stored in memory among the objects detected by step 260 as a moving object (step S270).

  The object detection device 2 predicts the traveling path of the vehicle based on the moving direction and the steering angle of the vehicle, and moves toward the traveling object detected on the traveling path among the moving objects detected in step S270. The detected object is detected as an obstacle (step S280).

  Further, the object detection device 2 determines whether or not the moving amount of the vehicle calculated from the time when the stationary object was detected last time is greater than or equal to a predetermined threshold (for example, 75 mm) (step S290).

  If the amount of movement of the vehicle is not equal to or greater than the predetermined threshold value (step S290, No), the object detection device 2 proceeds to step S340, and if the amount of movement of the vehicle is equal to or greater than the predetermined threshold value (step S290, Yes). Then, the photographed image taken at the time when the stationary object was detected last time is acquired as the first image (step S300).

  The object detection device 2 processes the second image acquired in step S220 with a spatial filter to reduce the resolution, and corrects the resolution of the second image to match the first image (step S310). The filter coefficient of the spatial filter in step S310 may be set to be larger than the filter coefficient of the spatial filter in step S250.

  Next, the object detection device 2 compares the first image with the second image, and the object (mainly a stationary object) photographed in the first image and the second image by the background difference method. It detects (step S320). Further, the object detection device 2 predicts the road of the vehicle based on the moving direction and the steering angle of the vehicle, and among the objects detected in step 320, the object existing on the road and the object close to the road Is detected as an obstacle (step S330).

  Then, the object detection device 2 displays a warning pattern indicating an obstacle on the part of the object extracted as an obstacle in the second image based on the obstacle information detected in steps S280 and S330. These are synthesized (step S340). Further, the object detection device 2 may change the warning pattern between the obstacle that is the moving object detected in step S280 and the obstacle that is the stationary object detected in step S330. If it is determined in step S290 that the amount of movement of the vehicle is not equal to or greater than the predetermined threshold (No in step S290) and the process proceeds from step S290 to step S340, based on the obstacle information detected in step S280. Synthesize a warning pattern.

  The object detection device 2 outputs the second image obtained by synthesizing the warning pattern to the display device 3 as a video signal and displays the second image on the display device 3 (step S350).

  On the other hand, if the moving direction of the vehicle is not reverse in step S210 (No in step S210), the object detection device 2 acquires a first image from the captured image acquired from the camera 1 (step S420).

  Further, the object detection device 2 acquires a captured image that is a frame immediately before the first image acquired in step S420 among the captured images acquired from the camera 1 as a second image (step S430). In the present embodiment, the frame immediately before the first image is the second image. However, the present invention is not limited to this, and a predetermined number of frames before the first image may be the second image. That is, the second image may be acquired with a predetermined interval from the first image.

  The object detection device 2 obtains the amount of movement of the vehicle from the time when the first image was captured to the time when the second image was captured based on the optical flow of the captured image acquired from the camera 1 (step S440). ).

  The object detection apparatus 2 processes the second image acquired in step S420 with a spatial filter to reduce the resolution. Here, the object detection device 2 determines a filter coefficient based on the movement amount of the vehicle obtained in step S440, and sets the resolution of the second image to the first image with a spatial filter having a coefficient corresponding to the movement amount of the vehicle. (Step S450).

  Next, the object detection device 2 compares the first image with the second image, and detects a three-dimensional object (object) photographed in the first image and the second image by the background subtraction method. And stored in the memory (step S460). Further, the object detection device 2 detects, as a moving object, an object that is moving in comparison with the past detection result stored in the memory among the objects detected in step 460 (step S470).

  The object detection device 2 predicts the traveling path of the vehicle based on the moving direction and the rudder angle of the vehicle, and moves toward the traveling path and the object present on the traveling path among the moving objects detected in step S470. The detected object is detected as an obstacle (step S480).

  Further, the object detection device 2 determines whether or not the moving amount of the vehicle calculated from the time when the stationary object was detected last time is greater than or equal to a predetermined threshold (for example, 75 mm) (step S490).

  If the amount of movement of the vehicle is not greater than or equal to the predetermined threshold (No at step S490), the object detection device 2 ends the process of FIG. 8, and if the amount of movement of the vehicle is greater than or equal to the predetermined threshold (step S490, Yes), a captured image captured at the time when the stationary object was detected last time is acquired as a second image (step S500).

  The object detection apparatus 2 processes the second image acquired in step S500 with a spatial filter to reduce the resolution, and corrects the second image so that the resolution matches the first image (step S510). The filter coefficient of the spatial filter in step S510 may be set to be larger than the filter coefficient of the spatial filter in step S450.

  Next, the object detection device 2 compares the first image with the second image, and detects a stationary object photographed in the first image and the second image by the background subtraction method (step S520). ). In addition, the object detection device 2 predicts the road of the vehicle based on the moving direction and the steering angle of the vehicle, and among the objects detected in step 520, the object existing on the road and the object close to the road Is detected as an obstacle (step S530).

  Thus, according to the second embodiment, it is possible to detect a moving obstacle and a stationary obstacle. In particular, by performing obstacle detection for each frame of a photographed image taken by the camera 1, it is possible to accurately detect a moving obstacle, and to perform obstacle detection every time the amount of movement of the vehicle reaches a threshold value. Thus, the stationary obstacle can be detected at a timing when the difference between the stationary objects becomes an appropriate size.

  Further, in the object detection device 2 according to the second embodiment, the resolution correction unit 24 performs the first operation according to the distance (shooting distance) between the subject and the vehicle shot in common with the first image and the second image. The degree of lowering the resolution of the second image is varied. For example, the filter coefficient of the spatial filter for processing the second image is varied according to the shooting distance.

  As a result, the object detection device 2 of the second embodiment increases the degree to which the resolution is reduced when the amount of movement of the vehicle is large and the difference in resolution between the first image and the second image is large. If the amount of movement is large and the difference in resolution between the first image and the second image is small, the filter coefficient is set so as to reduce the degree of resolution reduction, and the resolution of the second image is appropriately set. And the detection accuracy when detecting an object can be improved.

<Modification>
In Embodiment 2 described above, the camera 1 is provided at the rear of the vehicle to detect obstacles behind the vehicle. However, the present invention is not limited to this, and the camera (imaging device) is provided at the front of the vehicle. It may be configured to detect a front obstacle. FIG. 9 is an explanatory diagram of an object detection method when a camera is provided at the front of the vehicle. Compared with the above-described second embodiment, the present modified example has a configuration in which the camera 1 is provided at the front of the vehicle and the image captured by the camera 1 is displayed on the display device 3 even when moving forward, and a part of the processing shown in FIG. Different, other configurations are the same. For this reason, the description of the same element is omitted.

In this modified example, since a camera is provided in the front part of the vehicle, the process in FIG. 9 is opposite to the process in the forward direction and the process in the reverse direction as compared to FIG. 8 according to the second embodiment. . For example, in FIG. 9, when it is determined in step S210A that the moving direction of the vehicle is forward, the process proceeds to step S220. In step S210A, when it is determined that the moving direction of the vehicle is reverse, the process proceeds to step S420. The processes in steps S220 to S350 and steps S420 to S530 are the same as those in FIG. In the process of FIG. 7, step 530 is performed during reverse travel.
Then, after detecting the obstacle, the object detection device 2 synthesizes a warning pattern indicating the obstacle in the portion of the object extracted as the obstacle in the second image based on the obstacle information. (Step S540). Further, the object detection device 2 displays a warning pattern indicating an obstacle on the part of the object extracted as an obstacle in the second image based on the obstacle information detected in steps S480 and S530. These are synthesized (step S540). If it is determined in step S490 that the amount of movement of the vehicle is not equal to or greater than the predetermined threshold (No in step S490) and the process proceeds from step S490 to step S540, based on the obstacle information detected in step S480. Synthesize a warning pattern.

  Then, the object detection device 2 proceeds to step S350, outputs the second image obtained by synthesizing the warning pattern to the display device 3 as a video signal, and causes the display device 3 to display the second image.

  As a result, the object detection device 2 of the present modified example generates the first image and the second image based on the moving direction of the vehicle even when the camera 1 is provided in the vehicle unit to detect an object in front of the vehicle. When properly detecting and reducing the resolution of the second image taken at a close position of a common subject compared to the first image, and comparing the first image with the second image to detect the object Detection accuracy can be improved.

  In the said Embodiment 2 and a modification, the process which mainly detects a moving object (step S220-S280, S420-S480) and the process which mainly detects a stationary object (step S290-S330, S490-S530) are performed. Although it was set as the structure, both a moving object and a stationary object may be detected by step S220-S280, S420-S480, and step S290-S330, S490-S530 may be abbreviate | omitted. In this case, among the three-dimensional objects detected in steps S260 and S460, the moving objects detected in steps S270 and S470 are detected as stationary objects, and obstacles are detected in steps S280 and S480 for these moving objects and stationary objects. Also good. Alternatively, steps S270 and S470 may be omitted, and obstacles may be detected in steps S280 and S480 for all three-dimensional objects detected in steps S260 and S460 without distinguishing between moving objects and stationary objects.

  The above-described embodiments and modifications of the present invention are merely examples, and the present invention is not limited thereto. The characteristic configurations shown in the above-described embodiments and modifications can naturally be combined without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Camera 2 Object detection apparatus 3 Display apparatus 10 Object detection system 15 Optical axis 21 Image acquisition part 22 Position correction part 23 Movement amount detection part 24 Resolution correction part 25 Object detection part 26 Shift position acquisition part 27 Obstacle determination part 28 Output control Part

Claims (9)

A movement detector for detecting movement of the vehicle;
An image acquisition unit that is mounted on the vehicle and acquires a first image and a second image captured at different times based on movement of the vehicle from an imaging device that captures the surroundings of the vehicle;
A resolution correction unit that corrects the resolution of the second image by reducing the resolution according to the resolution of the first image;
An object detection unit that detects an object photographed in the first image and the second image based on a difference between the corrected first image and the second image;
An object detection apparatus comprising: The photographing device is installed so as to photograph the rear of the vehicle;
The movement detecting unit detects a moving direction of the vehicle;
When the moving direction of the vehicle is reverse based on the moving direction of the vehicle detected by the movement detecting unit, the image acquiring unit captures an image taken at a time before the second image. The object detection device according to claim 1, wherein the object detection device acquires an image captured as a first image and an image captured at a time later than the second image when the moving direction of the vehicle is forward. . The photographing device is installed so as to photograph the front of the vehicle;
The movement detecting unit detects a moving direction of the vehicle;
When the moving direction of the vehicle is forward based on the moving direction of the vehicle detected by the movement detecting unit, the image acquiring unit captures an image taken at a time before the second image. The object detection device according to claim 1, wherein the first image is acquired as a first image and an image taken at a time later than the second image is acquired when the moving direction of the vehicle is backward. .   The object detection device according to claim 2, wherein the movement detection unit detects a moving direction of the vehicle based on a shift position of the vehicle. The movement detection unit detects a movement amount of the vehicle;
The image acquisition unit obtains an image captured at a time when a movement amount of the vehicle calculated from a time when one of the first image and the second image is captured reaches a threshold value. The object detection apparatus of any one of Claims 1-4 acquired as an image. An image acquisition unit that is mounted on a vehicle and acquires a first image and a second image that are captured at different times from a photographing device that photographs the surroundings of the vehicle;
A resolution correction unit that corrects the resolution of the second image by reducing the resolution according to the resolution of the first image;
An object detection unit that detects an object photographed in the first image and the second image based on a difference between the corrected first image and the second image;
An object detection apparatus comprising:   The resolution correction unit varies a degree of reducing the resolution of the second image according to a distance between a subject photographed in common with the first image and the second image and the vehicle. Item 7. The object detection device according to any one of Items 1 to 6. Detecting the movement of the vehicle;
Acquiring a first image and a second image captured at different times based on movement of the vehicle from an imaging device mounted on the vehicle and imaging the surroundings of the vehicle;
Correcting by reducing the resolution of the second image in accordance with the resolution of the first image; and
Detecting the object imaged in the first image and the second image based on a difference between the corrected first image and the second image;
An object detection method in which a computer executes. Obtaining a first image and a second image taken at different times from a photographing device mounted on a vehicle and photographing the surroundings of the vehicle; and
Correcting by reducing the resolution of the second image in accordance with the resolution of the first image; and
Detecting an object photographed in the first image and the second image based on a difference between the corrected first image and the second image;
An object detection method in which a computer executes.

Images