JP2006236094A - Obstacle recognition system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a simple and low-cost obstacle recognition system having improved recognition accuracy and speed. <P>SOLUTION: This obstacle recognition system recognizing an obstacle on a road by use of road-vehicle communication has: a roadside imaging means installed on a roadside, imaging the road; a roadside transmission means installed on the roadside, transmitting roadside image data imaged by the roadside imaging means to a vehicle together with characteristic information; a vehicle side imaging means loaded in the vehicle, imaging the road; a storage means storing and holding vehicle side image data imaged by the vehicle side imaging means associatively with imaged time; a correction means loaded in the vehicle, correcting the roadside image data transmitted by the roadside transmission means on the basis of the characteristic information and a travel state of its own vehicle; and a recognition means extracting the corrected roadside image data obtained by the correction means and the vehicle side image data imaged at the same time from the storage means, and detecting and recognizing the obstacle from the extracted vehicle side image data and corrected roadside image data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention generally relates to an obstacle recognition system that recognizes (detects) an obstacle on a road in cooperation between a vehicle and an infrastructure, and in particular, is easy and inexpensive, and recognizes an obstacle with improved recognition accuracy and speed. About the system.

  Conventionally, there has been proposed an obstacle recognition system / device that detects and recognizes obstacles on the road in cooperation between the vehicle and the infrastructure, such as using road-to-vehicle communication (see, for example, Patent Document 1).

  In the conventional system, when an obstacle is recognized on the infrastructure (road) side, usually, such a fact is transmitted to the vehicle using road-to-vehicle communication, and a measure is issued such that an alarm is issued in the vehicle.

For example, the device (system) disclosed in Patent Document 1 detects a pedestrian on a road from an image captured by a camera on the infrastructure side, transmits the result to the vehicle, and is identical to the detection result on the vehicle side. By confirming the above, the aim is to reliably and rapidly detect pedestrians in the vehicle.
JP 2001-195698 A

  However, in the conventional system represented by the device (system) disclosed in the above-mentioned Patent Document 1, not only the imaging of the image by the camera but also the recognition processing of the obstacle or the like is performed on the infrastructure side, and the result is It has been sent to the vehicle.

  On the other hand, in recent years, vehicles are also provided with a function of recognizing an obstacle or the like from an image captured by the camera and the camera. The above Patent Document 1 also clearly states that the vehicle has a function of detecting a pedestrian from an image captured by an in-vehicle camera.

  As described above, while the vehicle has a function of recognizing an obstacle or the like from the image data, the recognition result transmitted from the infrastructure side as in the device (system) disclosed in Patent Document 1 is further displayed on the vehicle side. However, if the recognition process is to be performed again, it will be time-consuming and rather the recognition speed will be slow.

  Further, if such a recognition process twice on the infrastructure side and the vehicle side is an indispensable configuration, each of the facilities on the infrastructure side is not only a camera and a road-to-vehicle communication device, but also an obstacle from image data. It is necessary to have a function for recognizing the system, and the structure of the equipment becomes complicated, and the infrastructure development takes enormous cost and time.

  Furthermore, since both the infrastructure side camera and the vehicle side camera see the obstacles to be recognized only from a substantially fixed direction, the recognition results may not match well.

  The present invention has been made to solve such problems, and it is a main object of the present invention to provide an obstacle recognition system that is simple and inexpensive and has improved recognition accuracy and speed.

  One aspect of the present invention for achieving the above object is an obstacle recognition system for recognizing an obstacle on the road using road-to-vehicle communication, and is installed on the road side and roadside imaging means for imaging a road (for example, Camera), roadside transmission means installed on the roadside and transmitting roadside image data imaged by the roadside imaging means together with characteristic information to the vehicle, and vehicle side imaging means (for example, camera) mounted on the vehicle and imaging the road And a storage unit mounted on the vehicle and stored in association with the time when the vehicle-side image data captured by the vehicle-side imaging unit is captured, and the road-side image mounted on the vehicle and transmitted by the road-side transmission unit Based on the above characteristic information and the traveling state of the host vehicle (for example, by converting the same road plane area into an image captured by the vehicle side camera with the road plane as a reference) (Correction means for correcting an object not on the road plane) and vehicle-side image data captured at the same time as the corrected road-side image data obtained by the correction means are extracted from the storage means and extracted. An obstacle recognition system having a recognition means for detecting and recognizing an obstacle from the vehicle-side image data and the corrected road-side image data.

  In this aspect, the characteristic information is information related to installation and specifications, such as the absolute position, the imaging (optical axis) direction, the focal length, the pixel size, and the lens distortion parameter of the roadside imaging means in addition to the time when the imaging was performed. Point to. The traveling state of the host vehicle refers to any arbitrary vehicle state parameter that can affect the position and angle of the optical axis of the vehicle-side imaging means, such as the vehicle speed, the roll angle, the yaw angle, and the pitch angle.

  According to this aspect, since only the imaging is performed on the infrastructure side and the recognition process is performed only on the vehicle side, the infrastructure side equipment may be configured simply and inexpensively. it can. Along with this, the cost and time for infrastructure development are also greatly reduced.

  Further, according to this aspect, since the obstacle is detected and recognized from the image picked up by the imaging means on the infrastructure side and the image picked up from a different angle by the imaging means on the vehicle side, the detection and recognition of the obstacle is performed. Accuracy can be improved. In addition, since the recognition process only needs to be performed once, the recognition speed is also improved.

  In this embodiment, 1) the obstacle recognition system is installed on the road side, alert means for alerting a pedestrian, and a vehicle mounted on the vehicle, the recognition result by the recognition means and the vehicle for the recognition result Vehicle-side transmission means for transmitting to the warning means, the warning means alerts the pedestrian based on the recognition result by the recognition means and the vehicle's response to the recognition result. And / or 2) the obstacle recognition system is installed on the roadside and is equipped with signal control means for controlling the switching of the traffic lights, and the recognition Vehicle-side transmission means for transmitting the recognition result by the means and the correspondence of the vehicle to the recognition result to the signal control means, and the signal control means includes the recognition result by the recognition means and the recognition result. When switching control of the traffic light is performed based on the response of the vehicle to be operated, what kind of response the vehicle has taken based on the highly accurate obstacle recognition information obtained in the vehicle and the obstacle recognition information (stop , Deceleration, steering, inevitable, etc.) can be used on the infrastructure side, so that safer driving and improved traffic flow can be expected.

  According to the present invention, it is possible to provide an obstacle recognition system that is simple and inexpensive and has improved recognition accuracy and speed.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the accompanying drawings. Note that the basic concept of object detection / recognition and road-to-vehicle communication based on image data is known to those skilled in the art, and a detailed description thereof will be omitted.

  An obstacle recognition system according to an embodiment of the present invention will be described with reference to FIGS. Here, a pedestrian that is a moving body will be described as an example of an obstacle on the road.

  FIG. 1 is a schematic diagram illustrating a configuration of the entire obstacle recognition system according to the present embodiment. Image data captured by the roadside camera 101 provided on the infrastructure side is transmitted to the vehicle 102 using the road-to-vehicle communication T as raw data without being recognized on the infrastructure side. At this time, characteristic information for correcting and converting image data captured by the roadside camera is also transmitted to the vehicle 102 at the same time. The characteristic information will be described in detail later.

  The vehicle 102 detects and recognizes an obstacle (in this example, a pedestrian 103) by integrating an image captured by the roadside camera and an image captured by the in-vehicle camera of the host vehicle and performing a recognition process. .

  Thus, in the obstacle recognition system according to the present embodiment, the recognition process is executed only on the vehicle side. In addition, as illustrated in FIG. 1, since the roadside camera and the vehicle-mounted camera normally capture the obstacle 103 from different directions, the recognition accuracy is improved by using image data captured by both cameras. To do.

  FIG. 2 is a block diagram illustrating a schematic configuration of the roadside facility 200. For example, a roadside facility 200 installed at an intersection includes a transmission / reception unit 201 that performs road-to-vehicle communication with a vehicle, a roadside camera 202, and a storage unit 203 that stores and holds characteristic information to be transmitted to the vehicle together with image data.

  The roadside facility 200 according to the present embodiment further has, as an optional component, an alarm intended to alert a pedestrian or the like not to enter an intersection when an image recognition result is received from a vehicle. An alarm unit 204 that outputs sound and a traffic signal control unit 205 that controls switching of traffic signals based on the recognition result when an image recognition result is received from the vehicle.

  FIG. 3 is a block diagram showing a schematic configuration of the in-vehicle device 300 mounted on the vehicle. The in-vehicle device 300 is a transmission / reception unit 301 that performs road-to-vehicle communication with the roadside equipment 200, and traveling of the host vehicle that can affect the position and angle of the optical axis of the in-vehicle camera such as vehicle speed, roll angle, yaw angle, and pitch angle A vehicle running state detection unit 302 that detects the state.

  The in-vehicle device 300 further uses the in-vehicle camera to receive the image data received from the roadside equipment 200 based on the characteristic information received from the roadside equipment 200 and the traveling state of the host vehicle detected by the vehicle traveling state detection unit 302. And a correction unit 303 that corrects and converts the data into the same coordinate system equivalent to that captured from the same position as the captured image data.

  The in-vehicle device 300 further includes a vehicle-side camera 304 that captures an image of the road ahead of the host vehicle, and a storage unit 305 that stores and holds the image data captured by the vehicle-side camera 304 in association with the time when the image data was captured. From the viewpoint of limiting the capacity of the storage unit 305, for example, imaging by the vehicle-side camera 304 is executed only when entering the intersection (before the intersection), and when passing the intersection, the storage unit 305 stores the image stored and held. It may be designed and set to be discarded.

  The in-vehicle device 300 further includes a recognition processing unit 306 that integrates the road-side image data corrected by the correction unit 303 and the vehicle-side image data stored and held in the storage unit 305 to execute an obstacle recognition process. Have. The recognition processing unit 306 is based on the imaging time information included in the characteristic information received from the roadside facility 200, and the vehicle side captured at the same time (or substantially the same) as the image data received from the roadside facility 200. Image data captured by the camera 304 is extracted from the storage unit 305.

  The in-vehicle device 300 according to the present embodiment arbitrarily transmits the recognition result by the recognition processing unit 306 to the roadside facility 200 through the transmission / reception unit 301 in addition to using the recognition result of the vehicle in order to alert the driver. And

  With respect to the obstacle recognition system according to the present embodiment having such a configuration, the flow of recognition processing will be described with reference to FIG. FIG. 4 is a flowchart showing the flow of obstacle recognition processing by the obstacle recognition system according to the present embodiment.

  First, on the vehicle side, the road in front of the host vehicle is specifically imaged by the vehicle-side camera 304, and the captured image data is stored and held in the storage unit 305 in association with the imaging time (S401).

  Next, on the road side, a road within a predetermined range is imaged by the road camera 202 (S402). The captured image data is directly transmitted to the vehicle side by the transmission / reception unit 201 without performing any recognition processing (S403). At this time, the characteristic information stored and held in the storage unit 203 is also transmitted to the vehicle side simultaneously (or substantially simultaneously).

  Here, the characteristic information stored and held in the storage unit 203 always includes the time when the image data to be transmitted to the vehicle is captured, and the correction of the road-side image data on the vehicle side (the correction unit 303) as described above. Information necessary for conversion, for example, information on installation and specifications such as absolute position of the roadside camera 202, imaging (optical axis) direction, focal length, pixel size, lens distortion parameters, and the like.

  On the vehicle side, when image data and characteristic information from the road side are received, the correction unit 303 corrects and converts the received road side image data so as to be integrated with the vehicle side image data as described above (S404). .

  Next, on the vehicle side, the recognition processing unit 306 extracts, from the storage unit 305, vehicle-side image data captured by the vehicle-side camera 304 at the same (or substantially the same) time as the received road-side image data ( S405).

  Next, an obstacle detection / recognition process is executed by the recognition processing unit 306 using the road-side image data corrected by the correction unit 303 and the vehicle-side image data extracted from the recording unit 305 (S406). .

  The reason why images at the same time are used and a specific example of integrated recognition processing will be described with reference to FIG. FIG. 5 shows a case where the integrated recognition process is executed by a technique for obtaining a stereo image.

  At time t1 in FIG. 5, the epipolar plane Σ is a plane composed of a straight line connecting the roadside camera C, the vehicle side camera C ′, and the gazing point X that is one point in the obstacle (here, a pedestrian). .

  The plane π is an imaging plane imaged by the roadside camera C, and the plane π ′ is an imaging plane imaged by the vehicle-side camera C ′. The imaging plane π intersects the epipolar plane Σ at points e and x, and the imaging plane π ′ intersects the epipolar plane Σ at points e ′ and x ′.

  A straight line connecting the point e and the point x is an intersection line between the imaging plane π and the epipolar plane Σ, and is called an epipolar line l. Similarly, a straight line connecting the point e ′ and the point x ′ is an intersection line between the imaging plane π ′ and the epipolar plane Σ, and is called an epipolar line l ′. The epipolar lines l and l 'limit the search range when detecting the parallax between the images of the target point X reflected by the two cameras C and C'.

  When the search range is limited in this way, as known to those skilled in the art, the target point X can be mathematically searched from the absolute positions of the cameras C and C ′ and the optical axis direction of each camera (for example, Satoshi Sato, “Computer Vision-Geometry of Vision”, Corona, pages 80-117, etc.).

  Therefore, by using the characteristic information received together with the roadside image data and the vehicle traveling state information detected by the vehicle traveling state detection unit 302, the positions of the cameras C and C ′ and the optical axis direction Therefore, stereo image recognition can be executed. Thereby, the distance to a pedestrian, the height of a pedestrian, and the position of a pedestrian are specified.

  FIG. 5 also shows the vehicle-side camera C ′ at time t0. As is apparent from the figure, when the obstacle is a moving body such as a pedestrian, if the imaging times of the two cameras are different, the target point X does not match and accurate stereo image recognition is not executed. Therefore, it is important to use an image captured at the same time in the recognition process.

  Returning to the description of FIG. When the integrated recognition process based on the road-side image data and the vehicle-side image data is completed in this way, in this embodiment, the recognition result is arbitrarily transmitted to the road-side equipment 200 and information is shared (S407). At this time, what kind of response (stopping, deceleration, steering, inability to avoid, etc.) the vehicle has taken is also transmitted to the roadside based on the obstacle recognition result. The transmission of the recognition result to the road side in S407 is not essential, but is preferably performed.

  The roadside facility 200 that has received the recognition result and the vehicle response, for example, the alarm unit 204, for example, alerts a pedestrian or the like who is about to enter an intersection based on such information, for example, by sounding an alarm. Alternatively, the traffic signal control unit 205 that performs traffic signal switching control performs traffic signal switching control such as immediately switching a signal for a pedestrian who is about to enter an intersection to red based on such information. The alarm unit 204 and the traffic light control unit 205 are not essential components of the roadside facility 200, but if the vehicle side is informed of a highly accurate recognition result in this way, it is preferable for beneficial use on the infrastructure side. It is a component. Even when the traffic signal is switched, the device for controlling the traffic signal does not necessarily have to be provided in the roadside equipment 200, and the control signal may be transmitted to a separate traffic signal control device. Good.

  On the other hand, on the vehicle side, of course, alerting such as notifying the driver with an alarm sound should be performed based on the obstacle recognition result (S409).

  As described above, according to the present embodiment, since only the image is picked up on the road side and the recognition process is performed only on the vehicle side, the infrastructure side equipment has a simple structure and is inexpensive. Can be. Along with this, the cost and time for infrastructure development are also greatly reduced.

  Further, according to the present embodiment, since the obstacle is detected and recognized from the image taken on the road side and the image taken from a different angle on the vehicle side, the obstacle detection and recognition accuracy can be improved. it can. Further, since the recognition process needs to be executed only once on the vehicle side, the recognition speed is also improved.

  Although one embodiment of the present invention has been described above, as will be apparent to those skilled in the art, embodiments of the present invention are not limited to the above one embodiment.

  For example, as known to those skilled in the art, the number of cameras required for performing stereo image recognition is at least two, and may be three or more. Rather, more accurate recognition processing is realized by using many cameras.

  Therefore, in the above embodiment, one camera is provided on the road side and one camera is provided on the vehicle side, but a total of two cameras are provided. The invention is valid.

  That is, when the present invention is applied to a vehicle equipped with two cameras so that stereo image recognition can be performed only by the in-vehicle camera, only one of the two in-vehicle cameras may be used, or both The recognition process may be performed by a total of three cameras including one on the road side using cameras.

  In the description of the above embodiment, a pedestrian is taken as an example of an obstacle on the road. However, as will be apparent to those skilled in the art, an obstacle that can be detected and recognized by the present invention is not limited to a pedestrian. It may be a stationary object such as a fallen object or a parked vehicle on the road, or may be a moving body other than a pedestrian such as a preceding vehicle or a bicycle.

  The present invention can be used in an obstacle recognition system using road-to-vehicle communication. The appearance, weight, size, running performance, etc. of the vehicle are not limited.

It is the schematic which shows the structure of the whole obstruction recognition system which concerns on one Example of this invention. It is a block diagram which shows schematic structure of the roadside installation which concerns on one Example of this invention. It is a block diagram which shows schematic structure of the vehicle-mounted apparatus which concerns on one Example of this invention. It is a flowchart which shows the flow of the obstacle recognition process by the obstacle recognition system which concerns on one Example of this invention. It is a figure for demonstrating the method of stereo image recognition.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Roadside camera 102 Vehicle 103 Pedestrian 200 Roadside equipment 201 Transmission / reception unit 202 Roadside camera 203 Storage unit 204 Alarm unit 205 Traffic signal control unit 300 In-vehicle device 301 Transmission / reception unit 302 Vehicle running state detection unit 303 Correction unit 304 Vehicle side camera 305 Storage unit 306 Recognition processing section

Claims (3)

An obstacle recognition system that recognizes obstacles on the road using road-to-vehicle communication,
Roadside imaging means installed on the roadside for imaging the road;
Roadside transmission means installed on the roadside and transmitting roadside image data captured by the roadside imaging means to the vehicle together with characteristic information;
Vehicle-side imaging means mounted on the vehicle and imaging the road;
A storage unit mounted on a vehicle and stored in association with a time when the vehicle-side image data captured by the vehicle-side imaging unit is captured;
Correction means mounted on a vehicle and correcting the road-side image data transmitted by the road-side transmission means based on the characteristic information and the traveling state of the host vehicle;
Vehicle-side image data captured at the same time as the corrected road-side image data obtained by the correction means is extracted from the storage means, and obstacles are detected and recognized from the extracted vehicle-side image data and the corrected road-side image data. An obstacle recognizing system comprising: The obstacle recognition system according to claim 1,
Alarm means installed on the roadside to alert pedestrians;
A vehicle-side transmission unit that is mounted on a vehicle and transmits a recognition result by the recognition unit and a vehicle response to the recognition result to the alarm unit;
The obstacle recognition system characterized in that the warning means alerts a pedestrian based on a recognition result by the recognition means and a vehicle response to the recognition result. The obstacle recognition system according to claim 1,
A signal control means that is installed on the roadside and performs switching control of the traffic light; and
Vehicle-side transmission means mounted on the vehicle, for transmitting the recognition result by the recognition means and the correspondence of the vehicle to the recognition result to the signal control means;
The obstacle control system according to claim 1, wherein the signal control means performs switching control of traffic lights based on a recognition result by the recognition means and a vehicle response to the recognition result.

Images