JP4946903B2 - Vehicle background image collection device - Google Patents

Description

  The present invention relates to an image recognition apparatus for recognizing an abnormal region in which an object or the like that is different from a normal image is captured in an input image.

  Conventionally, PCS (Pre-crush Safety System) is a control system that performs various vehicle controls so as to avoid a collision with an obstacle recognized by an in-vehicle camera, a radar device, or the like, or to reduce a shock at the time of the collision. It is known by names such as.

  Obstacles that exist in a predetermined area of the captured image naturally exist in the area that is difficult to recognize when the vehicle should avoid an obstacle such as an obstacle or a road depression using an in-vehicle camera. It is discriminated as a thing (a white line or the like drawn on the road) and recognized. For such determination, it is preferable to prepare a background image in which nothing other than a white line or the like is prepared in advance and compare it with a captured image.

In this connection, a monitoring camera that captures an image of the detection area and an image processing unit that detects the presence or absence of a vehicle in the detection area based on image data captured by the monitoring camera are sequentially provided by the monitoring camera. The image data to be captured is image-processed by grayscale projection processing, and the maximum difference value between the current projection processing data and the previous projection processing data is obtained, and the maximum projection difference when the vehicle is not detected An invention relating to a traveling vehicle detection system that determines the presence or absence of a vehicle based on the maximum difference between the projection processing data and the background data has been disclosed (for example, Patent Document 1). reference).
JP 2004-185537 A

  Patent Document 1 describes that background data is obtained by obtaining a moving average with respect to a projection difference maximum value when no vehicle is detected in the monitoring lane (paragraph “0028”). However, this system assumes that a vehicle is detected by a fixed camera, and is not necessarily a technology that can be applied to an in-vehicle device as it is. An image that is unsuitable for use as a background image at each point on a path because the vehicle trajectory is basically shaken by an action that avoids an obstacle even when traveling in the same lane. This is because there is a case where the image is captured.

  The present invention is for solving such problems, and provides a vehicle background image collection device capable of appropriately collecting a background image serving as a reference when performing obstacle detection in a vehicle. The main purpose.

In order to achieve the above object, one embodiment of the present invention provides:
Imaging means for periodically imaging the periphery of the vehicle;
For each imaging timing imaged by the imaging means or each sampled imaging timing, it is determined whether or not the moving path of the vehicle deviates from the reference path stored in the reference path storage means, and is determined not to deviate. A determination storage control unit that stores an image captured by the imaging unit at the imaging timing in the background image storage unit as a background image when
A vehicle background image collecting apparatus.

  According to this aspect of the present invention, the image captured by the imaging unit in a state where the moving path of the vehicle is not deviated from the reference path stored in the reference path storage unit is stored in the background image storage unit as a background image. For this reason, only the image captured while the vehicle is traveling according to the reference path is stored in the background image storage device. Accordingly, an image captured when the vehicle travels on a special path, such as when the vehicle travels around an obstacle, is excluded. As a result, it is possible to appropriately collect background images that serve as a reference when performing obstacle detection in the vehicle.

In one embodiment of the present invention,
The determination control means is means for sequentially executing the determination process and the storage control process in accordance with, for example, traveling of the vehicle.

In one embodiment of the present invention,
Each captured image of the imaging means is sequentially stored in the captured image storage means,
The determination control means may be means for collectively executing the determination process and the storage control process on the stored captured image at a predetermined execution timing.

In one embodiment of the present invention,
A position specifying means for specifying the position of the vehicle;
The determination storage control means may be means for determining whether or not a moving path of the vehicle deviates from the reference path based on the position of the vehicle specified by the position specifying means.

In one embodiment of the present invention,
Vehicle behavior detection means for detecting the vehicle behavior,
The determination storage control unit may be a unit that determines whether or not a moving path of the vehicle deviates from the reference path based on the vehicle behavior detected by the vehicle behavior detecting unit.

In one embodiment of the present invention,
A driving operation amount detection means for detecting the driving operation amount of the driver is provided,
The determination storage control unit may be a unit that determines whether or not the moving path of the vehicle deviates from the reference path based on the driving operation amount detected by the driving operation amount detecting unit.

In one embodiment of the present invention,
A position specifying means for specifying the position of the vehicle;
The reference path may be generated based on the position of the vehicle specified by the position specifying unit in the past.

In one embodiment of the present invention,
The reference path may be generated with reference to a road shape stored in a road shape storage unit.

  Note that the reference path storage unit, the background image storage unit, the captured image storage unit, and the road shape storage unit may be realized by the same storage device or may be realized by different storage devices.

  ADVANTAGE OF THE INVENTION According to this invention, the background image collection apparatus for vehicles which can collect appropriately the background image used as the reference | standard at the time of performing the obstacle detection in a vehicle can be provided.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the accompanying drawings.

<First embodiment>
Hereinafter, a vehicle background image collecting apparatus 1 according to a first embodiment of the present invention will be described. A vehicle background image collection device 1 according to the present embodiment is a device mounted on a vehicle, and is a device for collecting a background image serving as a reference image for detecting an obstacle position and the like with respect to each imaging position. is there. Note that the present invention is not limited thereto, and the present invention may be applied to a mobile robot or the like that moves around a predetermined moving path.

[Constitution]
FIG. 1 is a diagram illustrating an example of the overall configuration of a vehicular background image collection apparatus 1 according to the first embodiment. The vehicle background image collection device 1 includes an in-vehicle camera 10, a storage device 20, a vehicle speed sensor 30, a navigation device 40, and an ECU (Electronic Control Unit) 50.

  The in-vehicle camera 10 is, for example, a camera that uses an image sensor such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) disposed in the upper center of the windshield. Imaging is repeated every [ms]. FIG. 2 is a diagram illustrating an example of a hardware configuration of the in-vehicle camera 10. The camera 10 includes, for example, an imaging lens 12, an image sensor 14, and an electronic control unit 16. In the following description, it is assumed that the camera 20 is a CCD camera.

  The image sensor 14 is, for example, an interline type image sensor, in which a photodiode that is a light receiving element that performs photoelectric conversion and a corresponding CCD are arranged in a two-dimensional plane, and an analog is provided between the photodiode and the CCD. A transfer gate functioning as a switch is included. A microlens for condensing light is attached to the front side of each photodiode (on the imaging lens 12 side). Note that such a structure is merely schematically illustrated in order to simply explain the function of the image sensor 14, and any design change for improving functionality or the like is allowed. In addition to the interline image sensor, a full frame transfer type or frame transfer type image sensor in which the CCD itself functions as a light receiving element may be used.

  The electronic control unit 16 uses, for example, a microcomputer or an electronic circuit, and controls the shutter speed and imaging cycle (for example, several tens of times per second) of the camera 10 by controlling the opening / closing timing of the transfer gate of the image sensor 14. Degree). Then, the electronic control unit 16 amplifies the data output from the output circuit of the image sensor 14 with a predetermined gain and outputs the data to the ECU 50 as input image data.

  The storage device 20 is a storage device such as an HDD (Hard Disc Drive), a DVD (Digital Versatile Disk) drive, a CD drive, or a flash memory, or a combination thereof. Reference path information 22 and background image information 24 are stored in the storage medium of the storage device 20.

  The vehicle speed sensor 30 includes, for example, a wheel speed sensor and a skid control computer attached to each wheel of the vehicle, and the skid control computer converts the wheel speed pulse signal output from the wheel speed sensor into a vehicle speed rectangular wave pulse signal (vehicle speed signal). It converts and outputs to ECU50. Note that such information communication may be performed using a dedicated communication line, but information may be output to a multiplex communication line where a plurality of control devices can refer to the information.

  The navigation device 40 includes, for example, a GPS (Global Positioning System) receiver 42, a storage device 44 storing map data 45, an input / output device 46 including a display device and a voice input / output device, and a navigation control computer 48. . The map data 45 expresses the road shape by, for example, nodes indicating intersections and links connecting the nodes, and information such as the number of lanes, road curvature, road width, etc. is stored with each link. Yes. The navigation control computer 48 specifies the current position (latitude, longitude, altitude) of the vehicle by a calculation based on the time difference of the radio signal from the GPS satellite received by the GPS receiver 42. Then, a recommended route to the destination input to the input / output device 46 is generated with reference to the map data 45, and the input / output device 46 is controlled to perform navigation display and voice guidance. The current position of the vehicle specified by the navigation control computer 48 is output to the ECU 50.

  The ECU 50 is, for example, a computer unit in which a ROM (Read Only Memory), a RAM (Random Access Memory), and the like are connected to each other via a bus with a central processing unit (CPU) as the center, and an HDD (Hard Disc Drive). ), A DVD (Digital Versatile Disk) drive, a CD (Compact Disc) drive, a storage device such as a flash memory, an I / O port, a timer, a counter, and the like. The ROM stores programs and data executed by the CPU.

[Background image collection processing]
The ECU 50 stores the reference path information in which the moving path of the vehicle is stored in the storage device 20 based on the position of the vehicle specified by the navigation control computer 48 for each imaging timing captured by the in-vehicle camera 10 or each sampled imaging timing. It is determined whether or not there is a deviation from the reference path included in the image 22, and when it is determined that there is no deviation, the image captured by the in-vehicle camera 10 at the imaging timing is stored as background image information 24 in the storage device 20. . The ECU 50 according to the present embodiment sequentially executes the determination process and the storage control process in accordance with the traveling of the vehicle.

  The “sampled imaging timings” refer to imaging timings sampled at every predetermined travel distance or every predetermined time from each imaging timing at which the in-vehicle camera 10 images about several tens of times per second. In the following description, it is assumed that each time the vehicle travels 1 [m], an image captured by the in-vehicle camera 10 is extracted, and the above determination is performed with respect to the timing when the image is captured. The travel distance of the vehicle can be grasped by integrating the output of the vehicle speed sensor 30 or counting the number of output pulses.

  The reference path in the present invention refers to a standard or average path that has passed when traveling on the road, and means a path that has been traveled without performing any particular obstacle avoidance action. The reference path is generated in advance by a learning process or the like described later, and is stored in the storage device 20 as reference path information 22.

  For example, when the deviation in the lane width direction from the reference path is a predetermined value or less at a certain imaging timing, the ECU 50 determines that the moving path of the vehicle does not deviate from the reference path at the imaging timing (see FIG. 3). ). Then, the image captured at the imaging timing is stored in the storage device 20 as a background image together with the current position and traveling direction of the vehicle at the imaging timing.

  Further, at a certain imaging timing, the degree of similarity of the moving path before and after that (for example, the current position of the vehicle extracted several [m] before and after several [m]) with the reference path at the position is predetermined evaluation. It may be evaluated by the quantity to determine whether or not the moving path of the vehicle deviates from the reference path (see FIG. 4). In this case, the predetermined evaluation amount is defined as a function of the total sum or the maximum value of deviations in the lane width direction.

  By these processes, only an image captured while the vehicle is traveling on a standard or average route on the road is stored in the storage device 20 as the background image information 24. Accordingly, an image captured when the vehicle travels on a special path, such as when the vehicle travels around an obstacle, is excluded. As a result, it is possible to appropriately collect background images that serve as a reference when performing obstacle detection in the vehicle. The usage mode of the background image will be described later.

[Reference path generation processing]
The ECU 50 generates a reference path based on the current position of the vehicle specified by the navigation control computer 48. Specifically, for example, every time each link in the map data 45 described above is passed, a series of the current position of the vehicle specified by the navigation control computer 48 within that link is sampled and extracted at a predetermined cycle. A curve or straight line (which may be a set of points on the computer processing) obtained by connecting the current position and performing linear interpolation or the like is stored in the storage device 20 as one of moving paths in the link. Then, when a certain number of samples are obtained for each link, a reference path is generated by a technique such as obtaining an average of a plurality of moving paths. Further, when the number of samples is not sufficient, or when a unique path appears, the reference path is generated by referring to the shape for each link (curvature link length, etc.) included in the map data 45. Good.

  According to the vehicle background image collection device 1 of the present embodiment, it is possible to appropriately collect background images that serve as a reference when performing obstacle detection in a vehicle.

<Second embodiment>
Hereinafter, the vehicle background image collecting apparatus 2 according to the second embodiment of the present invention will be described. The vehicle background image collection device 2 according to the present embodiment is a device mounted on a vehicle, and is a device for collecting a background image serving as a reference image for detecting the position of an obstacle with respect to each imaging position. is there. Note that the present invention is not limited thereto, and the present invention may be applied to a mobile robot or the like that moves around a predetermined moving path.

[Constitution]
Since the vehicular background image collection device 2 according to the second embodiment is common to the vehicular background image collection device 1 according to the first embodiment in the hardware configuration, reference is made to FIG. Description is omitted.

[Background image collection processing]
In the present embodiment, each captured image of the in-vehicle camera 10 is sequentially (temporarily) stored in the storage device 20. The ECU 50 according to the present embodiment performs determination processing and batch processing for each captured image stored in the storage device 20 at a predetermined execution timing (for example, when one run is completed by stopping the engine). A storage control process is executed. Note that the determination method and the like at each imaging timing are the same as in the first embodiment. [Reference path generation processing] is the same as that in the first embodiment, and a description thereof will be omitted.

  According to the vehicle background image collection device 2 of the present embodiment, it is possible to appropriately collect background images that serve as a reference when performing obstacle detection in a vehicle.

<Other examples>
In the first and second embodiments, it is determined whether or not the moving path of the vehicle deviates from the reference path based on the transition of the current position of the vehicle specified by GPS or the like. The determination may be made by comparing the moving path of the vehicle derived from the output of the yaw rate sensor and the vehicle speed with the reference path. In other words, when traveling on a curved path on a road with no large curvature, it is determined that an obstacle has been avoided and roadside equipment has been used in the vicinity of the road, and the captured image captured in the vicinity is excluded from the background image. .

  For the same reason, whether or not the moving path of the vehicle deviates from the reference path based on the output of sensors that detect the amount of driving operation of the driver, such as a steering angle sensor and a brake pedal amount sensor. It may be determined.

<Use of background image>
Hereinafter, an example of how the background image is used will be described. A background image is converted into a background edge image in which edge points having a large luminance difference or pixel value difference from an adjacent pixel are extracted by applying a SOBEL filter or the like, and a series of contours such as road lane markings are shown therein. Edge points are extracted. And the picked-up image which the vehicle-mounted camera 10 images according to driving | running | working of a vehicle is similarly converted into an edge image, and the pedestrian is removed by removing the edge point which shows outlines, such as a road lane marking, from the converted edge image. Edge points indicating the contours of obstacles such as vehicles and vehicles are extracted.

  When the position area on the image defined by the extracted edge points is converted into a position on the two-dimensional plane, the distance to the obstacle and the relative speed can be calculated. Various collision safety controls are performed using the calculated distance to the obstacle and relative speed.

  Collision safety control is, for example, determining the possibility of collision by applying a predetermined threshold to the distance to the obstacle, relative speed, etc., and depending on the determination result, alarm control by voice and display, brake pedal Control to increase the braking force boost rate with respect to the amount of operation, control to increase the steering ratio, control to automatically wind up the seat belt and restrain the occupant to the seat with appropriate force, control to automatically output the braking force, etc. To do. Note that the ECU 50 according to the embodiment may be integrated with a control device that performs the collision safety control.

  The best mode for carrying out the present invention has been described above with reference to the embodiments. However, the present invention is not limited to these embodiments, and various modifications can be made without departing from the scope of the present invention. And substitutions can be added.

  The present invention can be used in the automobile manufacturing industry, the automobile parts manufacturing industry, and the like.

It is a figure which shows an example of the whole structure of the vehicle background image collection device 1 which concerns on 1st Example, and the vehicle background image collection device 2 which concerns on 2nd Example. It is a figure which shows an example of the hardware constitutions of the vehicle-mounted camera. It is a figure which shows a mode that it is determined whether the moving path of a vehicle has deviated from the reference path. It is a figure which shows the other mode which determines whether the moving path of a vehicle has deviated from the reference path.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle background image collection device 10 Car-mounted camera 12 Imaging lens 14 Image sensor 16 Electronic control part 20, 44 Memory | storage device 22 Reference path information 24 Background image information 30 Vehicle speed sensor 40 Navigation apparatus 42 GPS receiver 44 Memory | storage device 45 Map data 46 Input / output device 48 Computer for navigation control 50 ECU

Claims (8)

Imaging means for periodically imaging the periphery of the vehicle;
For each imaging timing imaged by the imaging means or each sampled imaging timing, it is determined whether or not the moving path of the vehicle deviates from the reference path stored in the reference path storage means, and is determined not to deviate. A determination storage control unit that stores an image captured by the imaging unit at the imaging timing in the background image storage unit as a background image when
A vehicle background image collecting apparatus. The determination control means is means for sequentially executing the determination process and the storage control process in accordance with the traveling of the vehicle.
The vehicle background image collection device according to claim 1. Each captured image of the imaging means is sequentially stored in the captured image storage means,
2. The vehicle background image collection device according to claim 1, wherein the determination control unit is a unit that collectively executes the determination process and the storage control process on the stored captured image at a predetermined execution timing. . A position specifying means for specifying the position of the vehicle;
The determination storage control means is means for determining whether or not a moving path of the vehicle is deviated from the reference path based on the position of the vehicle specified by the position specifying means.
The vehicle background image collection device according to any one of claims 1 to 3. Vehicle behavior detection means for detecting the vehicle behavior,
The determination storage control means is means for determining whether or not a moving path of the vehicle deviates from the reference path based on the vehicle behavior detected by the vehicle behavior detecting means.
The background image collection device for vehicles according to any one of claims 1 to 4. A driving operation amount detection means for detecting the driving operation amount of the driver is provided,
The determination storage control means is means for determining whether or not a moving path of a vehicle deviates from the reference path based on the driving operation amount detected by the driving operation amount detection means.
The vehicle background image collection device according to any one of claims 1 to 5. A position specifying means for specifying the position of the vehicle;
The reference path is generated based on the position of the vehicle specified by the position specifying means in the past.
The vehicle background image collection device according to any one of claims 1 to 6. The reference path is generated with reference to the road shape stored in the road shape storage means.
The vehicle background image collection device according to any one of claims 1 to 7.

Images