JP5278108B2 - Moving object detection system and moving object detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce burden of process for detecting a moving object in a system and a method for detecting moving object which detect a moving object in the vicinity of a vehicle. <P>SOLUTION: The moving object detecting system A includes an external camera 20, a position detecting sensor 21, database 11-13, a moving object extracting nit 25d, and a moving object distance calculation unit 25e. The moving object extracting nit 25d extracts a moving object, by generating an image of geographic feature of the current position of the vehicle by performing a viewpoint transformation to the image of geographic feature into an image which is viewed from the current position of the vehicle, based on the current position of the vehicle detected by the position detecting sensor 21, and an image of geographic feature, a distance image, and an imaging position stored in the database 11-13; and by performing a differential process to the image captured at the current position of the vehicle by using the external camera 20 and the image of the geographic feature at the current position of the vehicle. By generating a distance image of the current position of the vehicle, the moving object distance calculation unit calculates a distance to the contact position of the moving object extracted by the moving object extracting nit to the ground, based on the distance image. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a moving body detection system and a moving body detection method for detecting a moving body around a vehicle.

  2. Description of the Related Art Conventionally, it is known to calculate a distance to a moving body based on a captured image of an in-vehicle image capturing unit that captures the periphery of a vehicle. For example, when the distance to the moving body is short, an alarm device is activated to notify the passenger to that effect.

In Patent Document 1, first, an image at a different time point is acquired from a camera and stored in a storage device, and a motion vector in each region of this image is obtained based on the image at a different time point stored in the storage device. calculate. Next, a model of a space in which the vehicle travels is generated from the traveling state of the vehicle, and virtual motion vectors corresponding to each region of the image are calculated based on the generated model. Then, a difference between a motion vector for each region of the image and a virtual motion vector corresponding to each region is detected, an obstacle is detected based on the difference, and a distance to the obstacle is calculated.
JP 2008-140155 A

  However, since the thing of patent document 1 detects the difference of a motion vector and a virtual motion vector, and detects an obstacle based on this difference, the processing burden of the detection is large.

  The present invention has been made in view of the above points, and an object of the present invention is to reduce the processing load of moving object detection in a moving object detection system and a moving object detection method for detecting a moving object around a vehicle. There is to do.

  According to a first aspect of the present invention, there is provided an image pickup means mounted on a vehicle for picking up an image around the vehicle, a position detection means for detecting a current position of the vehicle, a feature image obtained by deleting a moving body from the picked-up image, and the feature Image storage means for storing the distance image corresponding to the image and information on the imaging position of the feature image, information on the current position of the vehicle detected by the position detection means, and the feature stored by the image storage means Based on the image, the distance image, and the imaging position information, the feature image at the current position of the vehicle is created by converting the viewpoint of the feature image into an image that can be seen from the current position of the vehicle. A moving body extracting means for extracting a difference area as a moving body by performing a difference process between a captured image captured at the current position of the vehicle by the imaging means and a feature image at the current position of the vehicle; and at least the current position of the vehicle A distance image at the current position of the vehicle including a distance image of the ground contact portion of the mobile body extracted by the mobile body extraction means in the vehicle, and a distance to the ground contact portion of the mobile body based on the distance image And a moving object distance calculating means for calculating the distance to the moving object.

  Thereby, the moving body extracting means captures the current position information of the vehicle detected by the position detecting means, the feature image stored by the image storage means, the distance image corresponding to the feature image, and the imaging of the feature image. The feature image at the current position of the vehicle is created by converting the viewpoint of the feature image into an image that can be seen from the current position of the vehicle based on the position information. The difference between the captured image and the feature image can be reduced, and the burden of the difference processing, and hence the processing load of the mobile object detection and the distance calculation to the mobile object can be reduced.

  In a second aspect based on the first aspect, the moving body extraction means converts the viewpoint of a plurality of feature images at different positions into an image that can be seen from the current position of the vehicle, and the feature image that has undergone the viewpoint conversion. By complementing each other missing region, one feature image at the current position of the vehicle is created.

  Thus, the moving object extracting means converts the plurality of feature images at different positions into images that can be seen from the current position of the vehicle, and complements the missing regions of the feature images that have undergone the viewpoint conversion. Since one feature image at the current position is created, a feature image at the current position of the vehicle without a missing region can be created, and a moving object can be extracted with high accuracy.

  According to a third invention, in the first invention, the moving body distance calculating means is configured to create only a distance image of a ground contact portion of the moving body at a current position of the vehicle. To do.

  Thereby, only the distance image of the ground contact portion of the moving body at the current position of the vehicle is created by the moving body distance calculating means, so that the processing load of the moving body detection can be reduced.

  According to a fourth invention, in the first invention, the feature image is created by creating the feature image and storing the feature image in the image storage means, and the distance image is created, A distance image creating means for storing a distance image in the image storage means, wherein the image storage means, the feature image creating means, and the distance image creating means are installed outside the vehicle and It is provided in an information center capable of mutual communication with a vehicle.

  As a result, the information storage means, the feature image creating means for creating the feature image, and the distance image creating means for creating the distance image are provided in the information center, thereby reducing the processing load of the moving object detection on the vehicle side. be able to.

  In a fifth aspect based on the fourth aspect, the moving body extracting means and the moving body distance calculating means are mounted on the vehicle, and the information center is a predetermined position where the vehicle is set in advance. The feature image, the distance image, and the information on the imaging position stored by the image storage means are transmitted to the vehicle.

  As a result, the information center transmits the feature image, the distance image, and the imaging position information stored by the image storage means to the vehicle when the vehicle is at a predetermined position set in advance. The vehicle can receive the provision without requesting the transmission.

  In a sixth aspect based on the first aspect, the image corresponding to the feature image stored by the image storage means and the captured image captured after capturing the captured image corresponding to the feature image. The feature image stored in the storage means and the feature image at the same imaging position are subjected to differential processing, and the difference area is extracted as a newly created or removed feature, and the feature image stored in the image storage means is stored. And a new / removed feature extracting means for updating the system.

  Thus, the image storage unit corresponding to the captured image captured after the feature image stored by the image storage unit and the captured image corresponding to the feature image are captured by the new / removed feature extraction unit. The feature image stored at the same position as the feature image stored at the same imaging position is subjected to differential processing, and the difference area is extracted as a newly added or removed feature, and the feature image stored by the image storage means is extracted. Since it updates, it can suppress extracting the newly installed or removed feature as a moving body, and can suppress the misdetection of a moving body.

  In a seventh aspect based on the first aspect, a plurality of feature images at different positions are subjected to matching processing to calculate a parallax, and one of the plurality of feature images is selected based on the parallax. It further comprises distance image creating means for creating a distance image corresponding to the object image and storing the distance image in the image storage means.

  Thereby, a distance image can be created by the distance image creating means.

  In an eighth aspect based on the seventh aspect, the distance image creating means is configured to perform a matching process on the plurality of feature images in a predetermined region set in advance. It is.

  As a result, the distance image creating means performs a matching process on a plurality of feature images in a predetermined region set in advance, so that a precise distance image can be created and the distance to the moving object can be calculated with high accuracy. Can do.

  In a ninth aspect based on the seventh aspect, the distance image creating means creates a plurality of distance images corresponding to any one of the plurality of feature images, and the plurality of distances. It is configured to create one distance image by statistically processing the distance information of the image.

  Thus, the distance image creating means creates a plurality of distance images corresponding to any one of the plurality of feature images, and statistically processes the distance information of the plurality of distance images, thereby obtaining 1 Since two distance images are created, a precise distance image can be created, and the distance to the moving body can be calculated with high accuracy.

  In a tenth aspect based on the first aspect, a parallax is calculated by performing a matching process on the captured image captured at the current position of the vehicle by the imaging means and the feature image at the current position of the vehicle, The apparatus further comprises position correction / pitch angle calculating means for calculating at least one of the position correction amount of the vehicle and the pitch angle of the vehicle based on the parallax.

  Accordingly, the position correction / pitch angle calculation means calculates the parallax by matching the captured image captured at the current position of the vehicle by the imaging means with the feature image at the current position of the vehicle, and based on the parallax. Then, at least one of the vehicle position correction amount and the vehicle pitch angle is calculated. That is, it is possible to calculate at least one of the vehicle position correction amount and the vehicle pitch angle by using the captured image of the imaging means.

  An eleventh aspect of the present invention is an image pickup unit mounted on a vehicle for picking up an image of the surroundings of the vehicle, a feature image obtained by deleting a moving body from the picked-up image, a distance image corresponding to the feature image, and the pickup of the feature image. Image storage means for storing position information is provided, a position detection step for detecting the current position of the vehicle, information on the current position of the vehicle detected by the position detection step, and storage by the image storage means. The feature image at the current position of the vehicle is created by converting the viewpoint of the feature image into an image that can be seen from the current position of the vehicle based on the obtained feature image, the distance image, and the imaging position information. A moving body extraction step of performing differential processing between the captured image captured at the current position of the vehicle by the imaging means and the feature image at the current position of the vehicle and extracting the difference area as a moving body; A distance image at the current position of the vehicle including a distance image of the ground contact portion of the moving object extracted by the moving object extracting step at both current positions is created, and the grounding of the moving object is performed based on the distance image. And a moving body distance calculating step of calculating a distance to the part as a distance to the moving body.

  Thereby, the same effect as that of the first invention can be obtained.

  According to the present invention, the moving body extracting means detects the current position information of the vehicle detected by the position detecting means, the feature image stored by the image storage means, the distance image corresponding to the feature image, and the feature. The feature image at the current position of the vehicle is created by converting the viewpoint of the feature image into an image that can be seen from the current position of the vehicle based on the information on the imaging position of the image. The difference between the picked-up image picked up by the image and the feature image can be reduced, and the burden of the difference processing, and hence the load of detecting the moving object and calculating the distance to the moving object can be reduced.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a block diagram of a mobile body detection system A according to an embodiment of the present invention, and FIG. 2 shows a vehicle V on which an in-vehicle unit 2 of the mobile body detection system A is mounted. An information center (base station) 1 of the mobile body detection system A is installed outside the vehicle V, and a transceiver 10 for communicating data with the in-vehicle unit 2 of the mobile body detection system A. The acquired image database 11 for storing the received image data received by the transceiver 10, the feature luminance image database 12 for storing the following feature luminance image data, the following feature distance image data, and the like. A feature distance image database 13 to be stored and a server 14 that provides a feature luminance image, a feature distance image, and the like to the in-vehicle unit 2 are provided, and these are connected to be able to communicate with each other. The acquired image database 11, the feature luminance image database 12, and the feature distance image database 13 constitute the image storage means of the present invention. The feature luminance image corresponds to the feature image of the present invention, and the feature distance image corresponds to the distance image of the present invention.

  The transceiver 10 includes a transmission unit 10a that transmits transmission data created by the transmission data creation processing unit 14e of the server 14 to the in-vehicle unit 2, and a reception unit 10b that receives transmission data from the in-vehicle unit 2. ing.

  The acquired image database 11 stores the captured image (luminance image) of the external camera 20 of the in-vehicle unit 2 and the data of the imaging position and the imaging date and time received by the receiving unit 10b in association with each other. Specifically, the acquired image database 11 includes a captured image that is a predetermined imaging position in which the imaging position is preset among the captured images of the external camera 20 of the host vehicle V and other vehicles (not shown), and The imaging position and imaging date / time data are stored in association with each other. The predetermined imaging position is an imaging point set at predetermined intervals (for example, every several m to several tens of meters) on the road, for example, as point A and point B shown in FIG. .

  The feature brightness image database 12 stores the feature brightness image obtained by deleting the moving object from the captured image, and the imaging position and date / time data of the captured image corresponding to the feature brightness image in association with each other. ing. The feature distance image database 13 is a feature distance image obtained by processing a plurality of feature luminance images by a known triangulation method to obtain a distance for each pixel, and a captured image corresponding to the feature distance image. The data of the imaging position and the imaging date and time are stored.

  The server 14 is created by a feature brightness image creation processing unit 14a (feature image creation means) that performs a feature brightness image creation process for creating a feature brightness image, and the feature brightness image creation processing unit 14a. New / removed feature extraction processing unit 14b (new / removed feature extraction means) for performing a new / removed feature extraction process for extracting new or removed features from the feature luminance image, and a feature luminance image creation process A region attribute data creation processing unit 14c that performs region attribute data creation processing for creating region attribute data of the feature brightness image created by the unit 14a, and a feature brightness image created by the feature brightness image creation processing unit 14a. A feature distance image creation processing unit 14d (distance image creation means) that performs feature distance image creation processing for creating a corresponding feature distance image, and transmission data for creating transmission data to be transmitted to the in-vehicle unit 2 Has a transmission data generation processing unit 14e for performing conversion treatment, it is mutually communicably connected.

  The feature brightness image creation processing unit 14a creates a feature brightness image in which the moving body is deleted from each captured image of the external camera 20 stored in the acquired image database 11, and uses the data of the feature brightness image. The feature luminance image database 12 is stored in the feature brightness image database 12 via the new / removed feature extraction processing unit 14b and the region attribute data creation processing unit 14c in association with the data of the imaging position and the imaging date / time of the captured image corresponding to this. It has become.

  Hereinafter, an example of the feature luminance image creation processing of the feature luminance image creation processing unit 14a will be described with reference to FIG. First, a plurality of (two in the illustrated example) captured images (in-vehicle camera images (luminance images)) continuously captured in the vicinity of a certain imaging point (point B in the illustrated example) by the external camera 20 are compared and the difference is obtained. It processes and extracts the difference area (difference extraction image) as a mobile body (pedestrian P in the example of a figure). Next, the pedestrian P is deleted from any one of the two captured images, and a feature luminance image at the point B is created. Then, the feature luminance image data is stored in the feature luminance image database 12 in association with the imaging position (point B in the example in the figure) and imaging date / time data corresponding to the feature luminance image data. In addition, as described above, a feature luminance image is also created at a predetermined imaging point (point A,...) Other than point B, which includes only stationary objects such as buildings and roads from which the moving object is deleted. The feature brightness image database 12 is stored. Note that the feature luminance image may be created and stored in advance for each predetermined photographing point based on a photographed image photographed by a probe car or the like.

  Hereinafter, another example of the feature luminance image creation processing of the feature luminance image creation processing unit 14a will be described with reference to FIG. First, two captured images (in-vehicle camera image (luminance image) captured by the external camera 20 at a certain two predetermined imaging points (A point in the example and B point which is the current position of the vehicle V). ), The viewpoint of the captured image at the point A is converted into an image that can be seen from the external camera 20 of the vehicle V at the point B based on the captured image at the point A and the position data of the points A and B. Next, the picked-up image of the point B and the picked-up image obtained by converting the viewpoint are compared, and a difference process (difference extracted image) is extracted as a moving object (pedestrian P in the example in the figure). Then, the pedestrian P is deleted from the captured image of the B point that is the current position of the vehicle V, and a feature luminance image of the B point is created. Then, the feature luminance image data is stored in the feature luminance image database 12 in association with the imaging point (point B in the example in the figure) and imaging date / time data of the captured image corresponding to the feature luminance image data. In addition, as described above, a feature luminance image at a predetermined imaging point other than the point B is also created.

  The viewpoint conversion process is known. For example, first, an overhead view of point A is created based on a captured image of point A, and then, based on the overhead view and position data of point B. The overhead view of the point A is converted into an image obtained by looking down from the point B to create an overhead view of the point B. Based on the overhead view, the captured image of the point A is obtained from the vehicle V at the point B. This is performed by creating an image whose viewpoint is converted to an image that can be seen from the external camera 20. Note that the viewpoint conversion process described below is also performed by substantially the same process as the viewpoint conversion process.

  The new / removed feature extraction processing unit 14b captures a feature brightness image stored in the feature brightness image database 12 and a captured image corresponding to the feature brightness image (the feature brightness image is The feature brightness image stored in the feature brightness image database 12 corresponding to the captured image captured after being stored in the object brightness image database 12 is compared with the feature brightness image at substantially the same imaging position. Then, the difference area is extracted as a feature that has been newly created or removed, and the feature brightness image stored in the feature brightness image database 12 is added to the feature brightness image from the feature brightness image. Or it updates to the deleted feature luminance image.

  Hereinafter, an example of the new / removed feature extraction process of the new / removed feature extraction processing unit 14b will be described with reference to FIG. First, the past feature luminance image of the point A (for example, several days ago) stored in the feature luminance image database 12 and the current feature luminance image of the point A (that is, the feature luminance image creation processing unit 14a). The difference brightness (difference image) is newly created or removed (in the example in the figure, the feature newly established on the shoulder of the road). It is extracted as a certain road sign). Then, the past feature luminance image at the point A is updated to the feature luminance image to which the feature is added from the feature luminance image (that is, the current feature luminance image at the point A). In addition, when a new / removed feature extraction process is performed on a feature luminance image at a predetermined imaging point (such as a B point) other than the point A, and a new or removed feature is extracted, Update feature luminance image. It should be noted that the past feature luminance image at the point A and the current feature luminance image at the point A are compared / differed a plurality of times (that is, a plurality of feature luminances at different dates and times from the past feature luminance image). In the case where substantially the same difference area is extracted a plurality of times by comparing / differing with the image), the difference area may be determined as a newly-installed or removed feature.

  The region attribute data creation processing unit 14c is configured to estimate each region (for example, a building, a power pole, a road sign, a tree, a feature in the feature brightness image created by the feature brightness image creation processing unit 14a). Area attributes such as guardrails, ground, sidewalks, pedestrian crossings, road lane markings, road surface indications, road edge steps, etc. (see the bottom two images in FIG. 10) Data is created, and the region attribute data is stored in the feature luminance image database 12 in association with the feature luminance image. This area attribute data includes data indicating the type of the area and data indicating the use of the area. The road sign area is used for matching processing when creating a feature distance image, matching processing when calculating the position correction amount of the vehicle V, and the like. The area of the road marking line is used for a matching process when calculating the pitch angle of the vehicle V. Note that the region used for the matching process is not limited to this, and may be a feature that hardly changes over time, such as a building or a stepped region on a road edge.

  The feature distance image creation processing unit 14d uses a plurality of feature brightness images (feature brightness images stored by the feature brightness image database 12) created by the feature brightness image creation processing unit 14a to perform well-known triangulation. The feature distance image is created by calculating the distance for each pixel by processing in accordance with the method, and the data of the feature distance image is associated with the data of the imaging position and the imaging date and time of the corresponding captured image. The feature distance image database 13 is stored. Note that the feature distance image has distance data for each pixel.

  Hereinafter, an example of the feature distance image creation process of the feature distance image creation processing unit 14d will be described with reference to a portion surrounded by a two-dot chain line in FIG. First, based on a plurality of feature luminance images (feature luminance images at point A and point B in the example in the figure) stored in the feature luminance image database 12, the distance for each pixel is known. Obtained by triangulation. Further, a parallax is calculated (calculated) by performing a matching process on a predetermined area (the same object, which is a road sign area in the example in the figure) set in advance, and the feature brightness at points A and B is calculated based on the parallax. A distance error for each pixel of any feature luminance image in the image is calculated, and a distance error of all pixels of the feature luminance image is corrected to create a feature distance image. Then, the feature distance image data is stored in the feature distance image database 13 in association with the data of the imaging point (point A or B in the example in the figure) and the imaging date and time of the captured image corresponding thereto. In addition, as described above, a feature distance image at a predetermined imaging point other than the point A or the point B is also created. As described above, instead of creating the feature distance image based on the feature luminance images at the points A and B, it is based on a plurality of feature luminance images at each of the points A or B. Thus, a feature distance image of each point may be created.

  Hereinafter, another example of the feature distance image creation processing of the feature distance image creation processing unit 14d will be described with reference to FIG. First, based on the feature luminance images of points A and B of a plurality of sets (two sets in the example in the figure) stored in the feature luminance image database 12 at different imaging dates and times (for example, imaging dates and times that differ by several minutes). By the above method, a plurality of feature distance images with different shooting dates and times at each point are created. Next, a feature distance image is created by performing statistical processing (for example, an average or a weighted average) on the distance data for each pixel of a plurality of feature distance images having different shooting dates and times at the same point (for example, point A). . Then, the feature distance image data is stored in the feature distance image database 13 in association with the data of the imaging point (point A or B in the example in the figure) and the imaging date and time of the captured image corresponding thereto. In addition, as described above, a feature distance image at a predetermined imaging point other than the point A or the point B is also created. Note that, in the above, statistical processing is performed on time-sequentially captured images at each predetermined shooting point, but instead of this, a plurality of captured images are acquired at each predetermined shooting point (for example, point A). Based on the plurality of captured images, create a plurality of feature luminance images from which the moving object is deleted, and further create a plurality of feature distance images based on the plurality of feature luminance images. The final feature distance image may be created by statistically processing the distance data for each corresponding pixel of the feature distance image.

  The transmission data creation processing unit 14e is a feature luminance image stored by the databases 11 to 13, a feature distance image corresponding to the feature luminance image, and an imaging corresponding to the feature luminance image and the feature distance image. Data of the image capturing position and image capturing date and time is created as transmission data.

  When the vehicle V is in a predetermined transmission position set in advance, the transmission unit 10a is configured to detect a feature luminance image at a certain imaging point, a feature distance image corresponding to the feature luminance image, and the feature luminance. Data of the imaging position and imaging date / time of the captured image corresponding to the image and the feature distance image are automatically transmitted to the in-vehicle unit 2. The predetermined transmission position is, for example, a transmission point set on a road at predetermined intervals (for example, every several m to several tens of m). The feature luminance image and the feature distance image transmitted at a certain transmission point are, for example, a feature luminance image and a feature distance image at a point behind the vehicle V at the transmission point (for example, point A in FIG. 3) FIG. 4 is a feature luminance image and a feature distance image at two points (for example, point A and point B in FIG. 3) in front of the vehicle V and behind the vehicle V of the transmission point. The transmission position may be the same as the predetermined imaging point such as point A or point B described above. That is, when the vehicle V is at each of the predetermined points A and B, the captured image of each predetermined point is transmitted from the vehicle V to the information center 1, and at the same time, the ground of each predetermined point is transmitted from the information center 1 to the vehicle V. An object luminance image or a feature distance image may be transmitted.

  The vehicle V includes a single external camera (vehicle camera) 20 (imaging means) composed of a monocular camera that images the front of the vehicle V, and a position detection sensor 21 (position detection means) composed of a GPS that detects the current position of the vehicle V. ), A transmitter / receiver 22 for mutual communication of data with the information center 1, an alarm device 23 for alarming the driver's vision and hearing, a brake control unit 24 for brake control, an alarm The control unit 25 which controls the apparatus 23 and the brake control unit 24 is provided, and these are connected so that mutual communication is possible. The vehicle V operates the alarm device 23 in order to notify the driver when the degree of danger of the moving body in front of the vehicle V is relatively high.

  The external camera 20 is attached to the front part of the vehicle V in the vehicle width direction, and features in front of the vehicle V (for example, buildings, utility poles, road signs, trees, guardrails, ground, sidewalks, crosswalks). , Road markings, road surface indications, road edge steps, etc.) and moving objects in front of the vehicle V (for example, pedestrians P, bicycles, etc.) are imaged. The captured image data of the external camera 20 is input to the image acquisition unit 25a of the control unit 25. The position data of the position detection sensor 21 is input to the transmission data creation processing unit 25b, the moving body extraction processing unit 25d, and the position correction / pitch angle calculation processing unit 25g of the control unit 25.

  The transmitter / receiver 22 is attached to the rear part of the vehicle V, and transmits from the information center 1 a transmission unit 22a that transmits transmission data created by the transmission data creation processing unit 25b of the control unit 25 to the information center 1. And a receiving unit 22b for receiving data. The alarm device 23 is attached to the dashboard 26 and warns the danger of collision with a moving body due to sound or light emission. The brake control unit 24 performs brake control based on the position correction amount and the pitch angle of the vehicle V calculated by the position correction / pitch angle calculation processing unit 25g of the control unit 25, and the risk level of the control unit 25. When the determination processing unit 25f determines that a collision with the moving body is unavoidable, brake control is performed.

  The control unit 25 includes an image acquisition unit 25 a that acquires a captured image of the external camera 20, a transmission data generation processing unit 25 b that performs transmission data generation processing for generating transmission data to be transmitted to the information center 1, and the information center 1. A received data processing unit 25c that performs received data processing to process the received data, and a moving object extraction processing unit 25d (moving object extraction means) that performs a moving object extraction process for extracting a moving object from the acquired image of the image acquiring unit 25a; A moving body distance calculation processing unit 25e (moving body distance calculation means) for performing a moving body distance calculation process for calculating a distance from the current position of the vehicle V to the moving body extracted by the moving body extraction processing unit 25d; A risk determination processing unit 25f for performing a risk determination process for determining the risk of the body, and a position correction / pitch angle calculation process for calculating the position correction amount and pitch angle of the vehicle V; Has a a performing position correction / pitch angle calculation unit 25 g (position correction / pitch angle calculating means), all of which are communicably interconnected.

  The acquired image of the image acquisition unit 25a is input to the transmission data creation processing unit 25b, the moving body extraction processing unit 25d, and the moving body distance calculation processing unit 25e. The transmission data creation processing unit 25b creates the acquired image of the image acquisition unit 25a and the data of the imaging position and imaging date and time as transmission data. The reception data processing unit 25c processes the reception data of the reception unit 22b.

  The moving body extraction processing unit 25d is configured to store the current position data of the vehicle V detected by the position detection sensor 21 and the processing data of the reception data processing unit 25c (that is, the ground data stored in the databases 11 to 13 of the information center 1). On the basis of the object brightness image, the feature distance image corresponding to the feature brightness image, and the data of the imaging position of the feature brightness image), the feature brightness image can be seen from the external camera 20 at the current position of the vehicle V. By converting the viewpoint to an image, a feature luminance image at the current position of the vehicle V is created, and a captured image captured at the current position of the vehicle V by the external camera 20 and a feature luminance image at the current position of the vehicle V Are subjected to differential processing, and the differential area is extracted as a moving object.

  Hereinafter, an example of the moving body extraction process of the moving body extraction processing unit 25d will be described with reference to FIG. First, the feature brightness image at a predetermined imaging point (A point behind the vehicle V in the example in the figure) received last from the information center 1 at the current position of the vehicle V is used as an external camera 20 at the current position of the vehicle V. The feature luminance image at the current position of the vehicle V is created by converting the viewpoint to an image that can be seen from the vehicle. Then, the captured image (in-vehicle camera image) captured by the external camera 20 at the current position of the vehicle V and the feature luminance image at the current position of the vehicle V are compared and subjected to difference processing, and the difference area (difference extraction image) ) As a moving body (pedestrian P in the example in the figure).

  Hereinafter, another example of the moving object extraction process of the moving object extraction processing unit 25d will be described with reference to FIG. First, a plurality of feature luminance images at different predetermined imaging points close to the current position of the vehicle V received from the information center 1 (in the example shown in FIG. The viewpoint is converted into an image that can be seen from the external camera 20 at the current position of the vehicle V. Next, the missing region of the feature luminance image at point A that has undergone the viewpoint conversion (the region that is the shadow of the road sign in the example in the figure) is complemented with the feature luminance image at point B that has undergone viewpoint conversion, so that One feature luminance image at the current position is created. Then, the captured image (in-vehicle camera image) captured by the external camera 20 at the current position of the vehicle V and the feature luminance image at the current position of the vehicle V are compared and subjected to difference processing, and the difference area (difference extraction image) ) As a moving body (pedestrian P in the example in the figure).

  The moving body distance calculation processing unit 25e includes the current position of the vehicle V including the distance image of the grounding portion (boundary part) of the moving body and the ground extracted by the moving body extraction processing unit 25d at the current position of the vehicle V. A distance image is created, and a distance from the vehicle V to the ground contact portion of the moving body is calculated as a distance from the vehicle V to the moving body based on the distance image.

  Hereinafter, an example of the moving object distance calculation process of the moving object distance calculation processing unit 25e will be described with reference to FIG. First, a moving body (pedestrian P in the example in the figure) is detected by the above method, and the ground contact portion is extracted. Next, a feature distance image at a predetermined imaging point (A point behind the vehicle V in the example shown in the figure, see FIGS. 7 and 8) received from the information center 1 lastly, Based on the position data of the point A and the data of the current position of the vehicle V, the feature distance image of the point A is converted into an image that can be seen from the external camera 20 at the current position of the vehicle V, thereby A feature distance image for each pixel at the current position of the vehicle V corresponding to the entire feature luminance image is created. In other words, the feature distance image at the current position of the vehicle V is created by converting the viewpoint of the feature distance image at the point A into an image seen from the external camera 20 corresponding to the ground contact portion of the pedestrian P. And based on the distance image, the distance to the grounding part of the pedestrian P is calculated.

  The risk determination processing unit 25f determines the risk of the moving object based on the distance to the moving object calculated by the moving object distance calculation processing unit 25e, and the risk of the moving object is relatively When it is high, the alarm device 23 is activated. Specifically, for example, when the distance to the moving object is less than a predetermined distance, the risk determination processing unit 25f determines that the risk of the moving object is relatively high.

  The position correction / pitch angle calculation processing unit 25g stores data of the current position of the vehicle V detected by the position detection sensor 21 and processing data of the reception data processing unit 25c (that is, the databases 11 to 13 of the information center 1). The feature brightness image, the feature distance image corresponding to the feature brightness image, and the image pickup position data of the feature brightness image). The feature luminance image at the current position of the vehicle V is created by converting the viewpoint to an image that can be seen from the image 20, and the captured image captured at the current position of the vehicle V by the external camera 20 and the feature at the current position of the vehicle V are displayed. The parallax is calculated by performing a matching process with the luminance image, and the position correction amount (position error amount of the vehicle V) and the pitch angle of the vehicle V are calculated based on the parallax. It has become to so that.

  Hereinafter, an example of the position correction / pitch angle calculation processing of the position correction / pitch angle calculation processing unit 25g will be described with reference to FIG. First, the feature brightness image at a predetermined imaging point (A point behind the vehicle V in the example in the figure) received last from the information center 1 at the current position of the vehicle V is used as an external camera 20 at the current position of the vehicle V. The feature luminance image at the current position of the vehicle V is created by converting the viewpoint to an image that can be seen from the vehicle. Then, a predetermined image (in the example of the figure, a road sign region) in which a captured image (in-vehicle camera image) captured by the external camera 20 at the current position of the vehicle V and a feature luminance image at the current position of the vehicle V are set. ) To calculate (calculate) the parallax by performing the matching process, and calculate the position correction amount of the vehicle V based on the parallax. In addition, a matching process is performed between a captured image captured at the current position of the vehicle V by the external camera 20 and a feature luminance image at the current position of the vehicle V in a predetermined area (road lane line area in the example in the figure). Then, the parallax is calculated (calculated), and the pitch angle of the vehicle V is calculated based on the parallax.

  Hereinafter, the moving body detection control of the moving body detection system A will be described with reference to the flowcharts of FIGS. 11 and 12.

  First, the processing flow on the in-vehicle unit 2 side will be described. First, in step SA1, the camera image of the external camera 20 and the vehicle position data (own vehicle position data) Pos0 of the position detection sensor 21 are acquired. In subsequent step SA2, the vehicle position data Pos is corrected. Specifically, when the vehicle position correction amount data is ΔPos and the vehicle V pitch angle data is γ, the vehicle position data Pos is calculated by the following equation. The correction amount ΔPos and the pitch angle γ are calculated based on the parallax of the image, and the initial values are zero.

Pos = Pos0 + f (ΔPos, γ)
In the subsequent step SA3, the acquired image acquired in step SA1 and the vehicle position data Pos corrected in step SA2 are transmitted to the information center 1. In the subsequent step SA4, when the vehicle is at a predetermined point (A point, B point, etc.), the feature luminance image and the feature distance image are received from the information center 1. In subsequent step SA5, the received feature luminance image is subjected to viewpoint conversion into an image seen from the external camera 20 at the current position of the vehicle V. In subsequent step SA6, the moving object is detected from the acquired image and the viewpoint conversion image subjected to the viewpoint conversion in step SA5. Specifically, the moving object is detected by comparing the acquired image and the viewpoint conversion image.

  In a subsequent step SA7, it is determined whether or not there is a moving object in the acquired image. When the determination result of step SA7 is YES, the process proceeds to step SA8, and when the determination result is NO, the process proceeds to step SA13.

  In step SA8, a ground contact area with the ground of the moving body is extracted. In subsequent step SA9, a feature distance image at the current position of the vehicle V is created for the ground contact area. In subsequent step SA10, the distance to the ground contact area of the moving body is calculated as the distance to the mobile body from the feature distance image of the ground contact area.

  In subsequent step SA11, it is determined whether or not the distance to the moving body is less than a predetermined distance. If the determination result in step SA11 is YES and less than the predetermined distance, the process proceeds to step SA12. If the determination result is NO and the distance is greater than or equal to the predetermined distance, the process proceeds to step SA13.

  In step SA12, an alarm is output. In step SA13, the position correction amount of the vehicle V is compared by comparing the acquired image acquired in step SA1 with the viewpoint conversion image obtained by converting the feature luminance image into an image seen from the external camera 20 at the current position of the vehicle V. The pitch angle is calculated. In the subsequent step SA14, the position correction amount data ΔPos and the pitch angle data γ are stored, and the process returns to the start. Note that the position correction amount data ΔPos and the pitch angle data γ are used for correction of the vehicle position data Pos of the position detection sensor 21 and brake control of the brake control unit 24 as described above.

  Next, the processing flow on the information center 1 side will be described. First, in step SB1, an acquired image and vehicle position data (that is, shooting position data of the acquired image) are received from the in-vehicle unit 2. In subsequent step SB2, it is determined whether or not the vehicle V is at a predetermined point based on the vehicle position data. If the determination result in step SB2 is YES and the vehicle is at a predetermined point, the process proceeds to step SB3. If the determination result is NO and the battery is not in the predetermined point, the process proceeds to step SB14.

  In step SB3, the acquired image is stored in the acquired image database 11 for each position of the vehicle V and the date and time when the acquired image is captured. In subsequent step SB4, a plurality of acquired images are read from the acquired image database 11 based on the vehicle position data. Specifically, when the vehicle V is at a predetermined point, two acquired images near the predetermined point are read out, or an acquired image at the predetermined point and an acquired image at a point behind the vehicle V at the predetermined point are read out. . In subsequent step SB5, a feature luminance image of the predetermined point is created based on the plurality of acquired images. Specifically, a moving object is extracted by comparing a plurality of acquired images, and the moving object is deleted from the acquired image, thereby generating a feature luminance image at the predetermined point.

  In subsequent step SB6, region attribute data of the feature luminance image of the predetermined point created in step SB5 is created. In subsequent step SB7, the newly established or removed feature is extracted from the feature luminance image of the predetermined point created in step SB5. Specifically, by comparing the current feature luminance image of the predetermined point created in step SB5 with the past feature luminance image of the predetermined point, the newly established or removed feature is extracted. In subsequent step SB8, it is determined whether or not there is a feature newly added or removed in the feature luminance image of the predetermined point created in step SB5. If the determination result in step SB8 is NO, the process proceeds to step SB9, whereas if the determination result is YES, the process proceeds to step SB10.

  In step SB9, it is determined whether or not there are a plurality of feature brightness images at a predetermined point in the feature brightness image database 12. If the determination result in step SB9 is NO, the process proceeds to step SB10. If the determination result is YES and there are a plurality of determination results, the process proceeds to step SB11.

  In step SB10, the feature luminance image of the predetermined point created in step SB5 and its area attribute data are stored in the feature luminance image database 12.

  In step SB11, the feature luminance image and its area attribute data are read from the feature luminance image database 12 based on the vehicle position data. Specifically, when the vehicle V is at a predetermined point, a feature luminance image at the predetermined point and a feature luminance image at a point behind the vehicle V at the predetermined point are read.

  In the subsequent step SB12, a feature distance image of the predetermined point is created based on the feature luminance image read in SB11 and its region attribute data. In subsequent step SB13, the feature distance image of the predetermined point created in step SB12 is stored in the feature distance image data.

  In subsequent step SB14, the feature luminance image (including the region attribute data) is read from the feature luminance image database 12 and the feature distance image is read from the feature distance image database 13 based on the vehicle position data. Specifically, when the vehicle V is at a predetermined point, a feature luminance image and a feature distance image at a point behind the vehicle V at the predetermined point (for example, point A) are read out, or the vehicle V behind the predetermined point And feature luminance images and feature distance images at two points ahead of the vehicle V (for example, point A and point B). In subsequent step SB15, the feature luminance image and the feature distance image read in step SB14 are transmitted to the in-vehicle unit 2, and the process returns to the start.

-Effect-
As described above, according to the present embodiment, the moving object extraction processing unit 25d uses the current position information of the vehicle V detected by the position detection sensor 21 and the feature luminance image stored in the databases 11 to 13, and this feature. Based on the feature distance image corresponding to the luminance image and the information on the imaging position of the feature luminance image, the feature luminance image is subjected to viewpoint conversion into an image that can be seen from the current position of the vehicle V. Since the feature luminance image at the position is created, the deviation between the captured image captured at the current position of the vehicle V by the external camera 20 and the feature luminance image can be reduced, and the burden of the difference processing, and hence the movement The processing load of body detection and distance calculation to the moving body can be reduced.

  Further, the moving body distance calculation processing unit 25e calculates the distance to the grounded portion of the moving body as the distance to the moving body based on the feature distance image at the current position of the vehicle V. Even a single device can accurately calculate the distance to the moving body without being affected by the inclination of the vehicle V, the height position of the ground, or the like.

  In addition, the moving object extraction processing unit 25d performs viewpoint conversion of a plurality of feature luminance images at different positions into images that can be seen from the current position of the vehicle V, and complements the missing areas of the feature luminance images that have undergone the viewpoint conversion. Thus, since one feature luminance image at the current position of the vehicle V is created, it is possible to create a feature luminance image at the current position of the vehicle V without a missing region, and to accurately extract the moving object. it can.

  Since the information center 1 is provided with the database 11 to 13, the feature brightness image creation processing unit 14a for creating the feature brightness image, and the feature distance image creation processing unit 14d for creating the feature distance image, the vehicle It is possible to reduce the processing load of the V-side moving object detection.

  Further, when the vehicle V is at a predetermined transmission position set in advance, the information center 1 captures the feature luminance image, the feature distance image, and the feature luminance image stored by the databases 11 to 13. Therefore, the vehicle V can receive the information without requesting the transmission from the vehicle V side.

  Further, the new / removed feature extraction processing unit 14b corresponds to the feature luminance image stored in the feature luminance image database 12 and the captured image captured after the captured image corresponding to the feature luminance image is captured. The feature luminance image stored in the feature luminance image database 12 and the feature luminance image at the same imaging position are subjected to differential processing, and the difference area is extracted as a new or removed feature, Since the feature brightness image stored by the feature brightness image database 12 is updated, it is possible to suppress the extraction of the newly installed or removed feature as a moving object, and to suppress erroneous detection of the moving object.

  In addition, since the feature distance image creation processing unit 14d performs a matching process on a plurality of feature brightness images in a predetermined region set in advance, a precise feature distance image can be created, and the distance to the moving object Can be calculated with high accuracy.

  Further, the feature distance image creation processing unit 14d creates a plurality of feature distance images corresponding to any one of the feature brightness images among the plurality of feature brightness images, and the distance between the plurality of feature distance images. By statistically processing the information, one feature distance image is created, so that a precise feature distance image can be created, and the distance to the moving object can be calculated with high accuracy.

  Further, the position correction / pitch angle calculation processing unit 25g calculates a parallax by performing a matching process on the captured image captured at the current position of the vehicle V by the external camera 20 and the feature luminance image at the current position of the vehicle V. The position correction amount of the vehicle V and the pitch angle of the vehicle V are calculated based on the parallax. That is, at least one of the position correction amount of the vehicle V and the pitch angle of the vehicle V can be calculated using the captured image of the external camera 20.

(Other embodiments)
In the above embodiment, the external camera 20 images the moving body in front of the vehicle V. However, the present invention is not limited to this. For example, a moving object on the side of the vehicle V may be imaged.

  Moreover, in the said embodiment, although the feature distance image for every pixel in the present position of the vehicle V corresponding to the whole feature luminance image in the present position of the vehicle V is produced, it is not restricted to this, For example, Only the distance image of the ground contact portion of the pedestrian P at the current position of the vehicle V corresponding to the ground contact portion of the pedestrian P in the feature luminance image at the current position of the vehicle V may be created. Thereby, the processing load of a mobile body detection and the distance calculation to a mobile body can be reduced.

  Furthermore, in the said embodiment, although each database 11-13, the feature brightness image creation process part 14a, the feature distance image creation process part 14d, etc. are provided in the information center 1, it is not restricted to this, For example, vehicle V May be provided. When the vehicle V is provided in this way, the moving object can be detected by the vehicle V alone.

  Furthermore, in the above-described embodiment, the acquired image database 11 stores a captured image whose captured position is a predetermined captured position among captured images of the external camera 20. The captured image whose imaging position is a predetermined imaging position among the captured images of the external camera 20 may be transmitted to the information center 1 by 25b.

  Further, in the present embodiment, the feature luminance image at the current position of the vehicle V is created by the position correction / pitch angle calculation processing unit 25g, but at the current position of the vehicle V created by the moving object extraction processing unit 25d. The feature luminance image may be input to the position correction / pitch angle calculation processing unit 25g.

  The present invention is not limited to the embodiments, and can be implemented in various other forms without departing from the spirit or main features thereof.

  As described above, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The scope of the present invention is defined by the claims, and is not limited by the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

  As described above, the mobile object detection system and the mobile object detection method according to the present invention can be applied to applications that require reducing the processing load of mobile object detection.

It is a block diagram of the mobile body detection system which concerns on embodiment of this invention. It is a schematic top view which shows the vehicle carrying the vehicle-mounted unit of a moving body detection system. It is explanatory drawing explaining an example of a feature luminance image creation process. It is explanatory drawing explaining the other example of a feature luminance image creation process. It is explanatory drawing explaining an example of a new establishment / removal feature extraction process. It is explanatory drawing explaining an example of a feature distance image creation process. It is explanatory drawing explaining an example of a mobile body extraction process. It is explanatory drawing explaining the other example of a mobile body extraction process. It is explanatory drawing explaining an example of a mobile body distance calculation process. It is explanatory drawing explaining an example of a position correction / pitch angle calculation process. It is a flowchart figure which shows a part of mobile body detection control of a mobile body detection system. It is a flowchart figure which shows a part of mobile body detection control of a mobile body detection system.

A Moving object detection system P Pedestrian (moving object)
V Vehicle 1 Information Center 11 Acquired image database (image storage means)
12 Feature brightness image database (image storage means)
13 Feature distance image database (image storage means)
14 Server 14a Feature luminance image creation processing unit (feature image creation means)
14b New / removal feature extraction processing section (new / removal feature extraction means)
14d Feature distance image creation processing unit (distance image creation means)
2 In-vehicle unit 20 External camera (imaging means)
21 Position detection sensor (position detection means)
25 Control unit 25d Moving object extraction processing unit (moving object extraction means)
25e Moving object distance calculation processing unit (moving object distance calculating means)
25g Position correction / pitch angle calculation processing section (position correction / pitch angle calculation processing means)

Claims (11)

An imaging means mounted on the vehicle for imaging the surroundings of the vehicle;
Position detecting means for detecting the current position of the vehicle;
Image storage means for storing a feature image obtained by deleting a moving object from a captured image, a distance image corresponding to the feature image, and information on an imaging position of the feature image;
Based on the information on the current position of the vehicle detected by the position detection means and the information on the feature image, the distance image, and the imaging position stored by the image storage means, the feature image is converted into the current position of the vehicle. The feature image at the current position of the vehicle is created by converting the viewpoint to an image that can be seen from the image, and the captured image captured at the current position of the vehicle by the imaging means and the feature image at the current position of the vehicle are Mobile object extraction means for performing differential processing and extracting a difference area as a mobile object;
Creating a distance image at the current position of the vehicle including at least a distance image of the ground contact portion of the moving object extracted by the moving object extracting means at the current position of the vehicle, and moving the vehicle based on the distance image A moving body detection system comprising: a moving body distance calculating means for calculating a distance to a grounded portion of the body as a distance to the moving body. The moving object detection system according to claim 1,
The moving body extraction means converts a plurality of feature images at different positions into images that can be seen from the current position of the vehicle, and complements the missing regions of the feature images that have undergone the viewpoint conversion. A moving object detection system configured to create one feature image at a current position. The moving object detection system according to claim 1,
The moving body detection system, wherein the moving body distance calculation means is configured to create only a distance image of a ground contact portion of the moving body at a current position of the vehicle. The moving object detection system according to claim 1,
A feature image creating means for creating the feature image and storing the feature image in the image storage means;
A distance image creating means for creating the distance image and storing the distance image in the image storage means;
The image storage means, the feature image creation means, and the distance image creation means are provided in an information center installed outside the vehicle and capable of mutual communication with the vehicle. Mobile object detection system. The moving body detection system according to claim 4,
The moving body extracting means and the moving body distance calculating means are mounted on the vehicle,
The information center is configured to transmit the feature image, the distance image, and the imaging position information stored by the image storage unit to the vehicle when the vehicle is at a predetermined position set in advance. A moving object detection system. The moving object detection system according to claim 1,
The same imaging position as the feature image stored by the image storage unit corresponding to the feature image stored by the image storage unit and the captured image captured after capturing the captured image corresponding to the feature image And a new / removed feature extracting means for updating the feature image stored in the image storage means by extracting a difference area from the feature image and extracting the difference area as a new or removed feature. A moving object detection system. The moving object detection system according to claim 1,
A parallax is calculated by matching a plurality of feature images at different positions, a distance image corresponding to any one of the plurality of feature images is created based on the parallax, and A moving object detection system further comprising distance image creation means for storing a distance image in the image storage means. The mobile object detection system according to claim 7, wherein
The distance image creating means is configured to perform a matching process on the plurality of feature images in a predetermined area set in advance. The mobile object detection system according to claim 7, wherein
The distance image creating means creates a plurality of distance images corresponding to any one of the plurality of feature images, and statistically processes the distance information of the plurality of distance images, thereby obtaining one A moving object detection system configured to create a distance image. The moving object detection system according to claim 1,
A parallax is calculated by performing a matching process on a captured image captured at the current position of the vehicle by the imaging unit and a feature image at the current position of the vehicle, and based on the parallax, the position correction amount of the vehicle and the vehicle A moving body detection system further comprising position correction / pitch angle calculation means for calculating at least one of the pitch angles of the vehicle. An imaging means mounted on the vehicle for imaging the surroundings of the vehicle;
An image storage means for storing a feature image obtained by deleting the moving object from the captured image, a distance image corresponding to the feature image, and information on an imaging position of the feature image;
A position detecting step for detecting the current position of the vehicle;
Based on the information on the current position of the vehicle detected by the position detection step and the information on the feature image, the distance image, and the imaging position stored in the image storage means, the feature image is converted into the current position of the vehicle. The feature image at the current position of the vehicle is created by converting the viewpoint to an image that can be seen from the image, and the captured image captured at the current position of the vehicle by the imaging means and the feature image at the current position of the vehicle are A moving body extraction step of performing difference processing and extracting a difference area as a moving body;
Creating a distance image at the current position of the vehicle including at least a distance image of a ground contact portion of the moving object extracted by the moving object extraction step at the current position of the vehicle, and moving the vehicle based on the distance image A moving body detection method comprising: a moving body distance calculating step of calculating a distance to a grounded portion of the body as a distance to the moving body.