JP5097681B2 - Feature position recognition device - Google Patents

Description

  The present invention relates to a feature position recognition device for recognizing the position of a target feature existing around a vehicle such as a road marking, a traffic light, a sign, and the like based on an image captured by a vehicle-mounted camera.

  2. Description of the Related Art Conventionally, an apparatus for recognizing a position of a feature from an image captured by a vehicle-mounted camera has been proposed in order to correct the position of the vehicle (see, for example, Patent Document 1).

JP 2008-164384 A

  In the case of the apparatus disclosed in Patent Document 1, a method of measuring the relative distance between the own vehicle and the road marking, using a predetermined part of the road marking as a measurement point is adopted. The part that is the measurement point of the road marking has a shape that can be distinguished from other parts, and when the shape is detected by image processing, the part having such a shape is determined to be the measurement point is doing.

  Therefore, when the shape of the road marking changes due to, for example, aging or drowning, and a part having a shape similar to the measurement point is formed in a part different from the measurement point, the erroneous part is used as the measurement point. There is a risk of false detection.

  The present invention has been made in view of the above points, and it is an object of the present invention to provide a feature position recognition device that can recognize the exact position of a target feature with respect to the host vehicle.

  The feature position recognizing device of the present invention estimates which part of the target feature is imaged from a plurality of images taken along with the movement of the vehicle, and the estimated imaging part is the target feature. It is determined whether or not it is a part including a preset measurement point.

  According to the feature position recognizing device of the present invention, it is estimated which part of the target feature is imaged from a plurality of images imaged along with the movement of the host vehicle, and the estimated imaging part is Since it is determined whether or not it is a part including a measurement point set in advance with respect to the target feature, for example, when a measurement point is detected at an imaging part where the measurement point of the target feature does not exist, It can be determined that the detection of such a measurement point is a false detection. Therefore, it is possible to prevent the vehicle position from being recognized using an incorrect point as a measurement point.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram showing an example of a scene to which the host vehicle position detection system is applied, and FIG. 2 is a diagram conceptually showing the overall configuration of the host vehicle position detection system. In the present embodiment, as shown in FIG. 1, the road surface 1 is imaged while the host vehicle C is traveling, the position of the road marking 2 such as a pedestrian crossing is recognized from the image, and the current position of the host vehicle C is obtained. The case of the own vehicle position detection system 100 will be described as an example.

  As shown in FIG. 2, the vehicle position detection system 100 includes a car navigation device 101 and a road marking position recognition device (feature position recognition device) 103. The road marking position recognizing device 103 detects the road marking 2 by performing image processing on an image obtained by imaging the road surface 1, and obtains a recognition result including information on the type of the road marking 2 and the distance L from the vehicle C to the road marking 2. It has a function of outputting to the car navigation device 101.

  For example, as shown in FIG. 1, the imaging of the road surface 1 is performed by a rear camera 108 (on-vehicle camera) attached to the rear portion of the vehicle. The rear camera 108 images the road surface 1 behind the host vehicle over the viewing angle range 3. The road marking position recognition device 103 stores in advance information on the measurement point P in the road marking 2 and information on the distance L from the vehicle C to the measurement point P. The distance L is set in advance according to the position of the measurement point P in the image.

  The car navigation apparatus 101 has a function of calculating the vehicle position, providing the information to the driver and the occupant, and performing route guidance from the vehicle position to the destination. The car navigation device 101 can obtain the exact position of the vehicle C based on the position of the road marking 2 by comparing the map data 102 and the recognition result of the road marking position recognition device 103.

  The road marking 2 is paint drawn on the road surface 1. For example, in addition to the pedestrian crossing shown in FIG. 1, there are primary stop lines, maximum speed markings, traffic direction markings according to the direction of travel, turning prohibition markings, lane markings, etc. included. In the map data 102, in addition to general map information, information such as the type, shape, position, and direction of the road marking 2 is stored.

  FIG. 3 is a block diagram schematically showing the configuration of the vehicle position detection system. As shown in FIG. 3, the road marking position recognition device 103 includes functional units such as an image processing unit 111, a feature amount extraction unit 112, a state estimation unit 113, and a recognition result output determination unit 114. Each of these functional units is implemented and configured by hardware and / or software.

  The image processing unit 111 acquires a plurality of images captured by the rear camera 108 as the host vehicle moves from the rear camera 108. Then, image processing for recognizing the road marking 2 is performed on each image, the type of the road marking 2 as a recognition object is determined based on the result of the image processing, and the measurement that the road marking 2 has A process for recognizing the point P is performed. These processes are determined by, for example, known template matching.

  The road marking 2 that is the recognition target can be the road marking 2 that exists in front of the direction in which the host vehicle C travels, and its presence is calculated by the host vehicle position calculation unit 121 of the car navigation device 101. Based on the vehicle position and the map data 102. Information on the measurement point P of the road marking 2 is stored in the road marking position recognition device 103 or the map data 102 so as to be readable according to the type of the road marking 2.

  The measurement point P is set according to the contour shape of the road marking 2, and, for example, on the contour line orthogonal to the traveling direction of the host vehicle C, or on other corners such as the corner portion when the corner has a corner portion. It is set to an identifiable part. For example, in the case of a pedestrian crossing, the measurement point P is a straight line (for example, FIG. 6 (FIG. 6)) connecting a plurality of parallel strip-shaped contour lines extending along the traveling direction of the host vehicle C. c)).

  When the measurement point P is recognized in the image, a process for deriving the positional relationship between the vehicle C and the measurement point P is performed. In the road marking position recognition device 103, the positional relationship between the measurement point P and the own vehicle C is obtained and stored in advance based on the mounting position of the rear camera 108 on the own vehicle C and the imaging direction. Therefore, based on the arrangement of the measurement points P in the image, the positional relationship between the own vehicle C and the measurement points P at the time of imaging can be derived.

  The feature amount extraction unit 112 extracts an amount in which the feature of the road marking 2 is included in the image (hereinafter referred to as a feature amount). The feature amount is used when the state estimation unit 113 estimates the state, and is extracted based on the feature of the road marking 2. The features of the road marking 2 are stored in the road marking position recognition device 103. For example, when the road marking 2 is a pedestrian crossing, the number of white lines arranged at a predetermined interval and the length of white lines in the image are characterized, and the more white lines or the longer the white line length, the more characteristic The amount is said to be large.

  Based on the feature amount extracted by the feature amount extraction unit 112, the state estimation unit (imaging part estimation unit) 113 indicates which part of the road marking 2 is visible in the image, in other words, the road marking 2 in the image. The process which estimates the state of which part of is contained is performed. For example, in the case of a pedestrian crossing, which part of the image includes the start part of the pedestrian crossing (part located on the rear side in the vehicle traveling direction), the central part, or the end part (part located on the front side in the vehicle traveling direction) Is estimated globally (see, for example, FIG. 6).

  The recognition result output determination unit 114 determines whether or not a preset output condition is satisfied. If the output condition is satisfied, the recognition result output determination unit 114 outputs the recognition result to the car navigation device 101 and satisfies the output condition. If not, the recognition result is not output. The recognition result output from the recognition result output determination unit 114 includes information on the type of the road marking 2 determined by the image processing unit 111 and the distance L from the vehicle C to the road marking 2.

  The car navigation device 101 uses the recognition result of the vehicle position calculation unit 121 that calculates the vehicle position based on the detection signal from the position measurement means such as the GPS 1202 and the road surface position recognition device 103 to determine the vehicle position. The vehicle position correction unit 122 for correction is provided.

  The own vehicle position correcting unit 122 corrects the own vehicle position calculated by the own vehicle position calculating unit 121 according to the position of the road marking 2. The correction of the vehicle position is performed by comparing the recognition result received from the road marking position recognition device 103 with the map data 102.

  Next, processing contents in the road marking position recognition apparatus 103 will be described with reference to FIGS. 4 is a flowchart for explaining the contents of processing executed in the road marking position recognizing device 103, FIG. 5 is a flowchart for explaining the contents of the processing in step S205 in FIG. 4, and FIG. It is a figure which shows the some image imaged with the movement of C. FIG.

  First, in the image input process of step S201, a plurality of images captured as the host vehicle C moves are input. In the recognition object determination process in step S202, image processing is performed on each image to determine the recognition object. In the present embodiment, the road marking 2 is a recognition target, and after image processing such as binarization processing and noise removal processing is performed, template matching is performed to determine the type of the road marking 2, and then the road surface Processing for recognizing the preset measurement point P of the sign 2 is performed.

  When the presence of the measurement point P is recognized in the image, the distance L from the measurement point P to the host vehicle C is obtained. The image processing unit 111 executes the image input process in step S201 and the road marking determination process in step S202.

  When a plurality of road markings are determined as recognition objects in the recognition object determination process in step S202, the following processes from step S203 to step S206 are repeatedly executed for the number of road markings.

  Next, in the road marking feature value extraction process in step S203, a process for extracting the feature value of the road marking 2 from the image acquired in step S201 is performed. This road marking feature quantity extraction processing is executed by the feature quantity extraction unit 112.

  Here, for example, the contour of the road marking 2 is extracted by detecting a change in luminance value between the road surface 1 and the road marking 2 in the image. Then, the contour of the road marking 2 is analyzed based on the extraction condition set in advance for each type of the road marking 2, and the feature amount of the road marking 2 is extracted.

  For example, in the case where the road marking 2 is a pedestrian crossing, an image including the start end portion of the pedestrian crossing (see FIG. 6A) and an image including the end portion of the pedestrian crossing (see FIG. 6C). In the image including the central part of the pedestrian crossing (see FIG. 6B), the length of the white line of the pedestrian crossing is included in the image longer. Therefore, more feature amounts are extracted in the case of the image shown in FIG.

  In the state estimation process in step S204, a process for estimating the state (imaging site) of the road marking 2 in the image is performed based on the feature amount extracted in step S203. The state of the road marking 2 in the image at time t is S (t). The state estimation process is executed by the state estimation unit 113. The state S (t) of the road marking 2 in the image refers to a state indicating where the road marking 2 is visible. For example, in the case of a pedestrian crossing, a state where the start end portion of the pedestrian crossing is visible in the image (see FIG. 6A), a state where the central portion of the pedestrian crossing is visible (see FIG. 6B), This refers to the state where the end part of the sidewalk is visible (see FIG. 6C).

The feature amount extracted at time t is defined as an observed value Y (t). The feature amount is extracted by the road marking feature amount extraction process in step S203. By using the data of the state S (t) and the observed value Y (t) and the dynamic Bayesian network technique, the current state (for example, the part of the pedestrian crossing can be seen from the time change of the feature value of the road marking 2 observed. Can be obtained probabilistically using the following equation (1).

  The left side of this equation is the probability of the current state S (t) when a time change of the observed value Y (t) is observed. In addition, the values of the first item and the second item on the right side are obtained in advance by learning a lot of data.

  The first item on the right side is the probability of what “observation value” may be obtained when a certain “state” is observed. For example, when the central part of the pedestrian crossing is visible (the state shown in FIG. 6B), it means that there is a high possibility that many features of the pedestrian crossing (the contour of the road marking) are observed.

  The second item is the probability of what kind of “state” is likely to be changed next when a predetermined “state” is observed in the previous imaging. For example, there is a high possibility that after the start end part of the pedestrian crossing (the state shown in FIG. 6A) is seen, the state where the central part of the pedestrian crossing is visible (the state shown in FIG. 6B) is high. Means. Based on the stochastic relationship between these “state” and “observed value”, the current state S (t) (where the pedestrian crossing is visible) is stochastically determined by the state estimation processing flow shown in FIG. calculate.

  First, the state transition probability is read in the state transition probability reading process in step S301 shown in FIG. The value of the state S (t) predefines the states of FIGS. 6A to 6C as a state value 1, a state value 2, and a state value 3, respectively.

  The state transition probability is a probability obtained by probabilistically determining the relationship between the previous state S (t−1) and the current state S (t) as the probability that the state of the image transitions between the images. That is, when the previous state S (t−1) is in a certain state, what state the current state S (t) is in is obtained probabilistically. The state transition probability is learned in advance and stored as a database in the road marking position recognition device 103. In this process, the learned data is read from the database.

  Next, the observation probability is read in the observation probability reading process in step S302. The observation probability is obtained by probabilistically obtaining the relationship between the observed value Y (t) and the state S (t) as the probability that the state of the image becomes a predetermined state when a predetermined feature amount is extracted from the image. Yes, that is, the state S (t) is stochastically obtained when the observed value Y (t) (the appearance of the contour of the road marking) is observed. The observation probability is learned in advance and stored in the road marking position recognition device 103 as a database. In this process, the learned data is read.

  Then, in the state estimation process in step S303, the current state S (t) is estimated by the above equation (1) using the state transition probability read in step S301 and the observation probability read in step S302.

  Returning to the description of the flow shown in FIG. 4 again, in the recognition result output process of step S205, a process of outputting the recognition result to the car navigation device 101 is performed when a preset output condition is satisfied. The recognition result output process is executed by the recognition result output determination unit 114.

  Here, the output condition is set when the contour of the road marking 2 extracted by the road marking feature amount extraction processing in step S204 matches the contour of the road marking 2 to be recognized and the recognition result may be output. It is judged that

  Whether or not the contours match is determined by the recognition object determination process in step S202. Whether or not the recognition result may be output is determined based on the current state S (t) estimated by the state estimation processing in step S205.

  For example, as shown in FIG. 6C, when the state estimated by the state estimation unit 113 is a state including the measurement point P in the image, the recognition result may be output (output enabled state). To be judged.

  On the other hand, if the state estimated by the state estimation unit 113 is a state that does not include the measurement point P in the image as shown in FIG. 6A or 6B, the recognition result should not be output. It is determined that the state (output disabled state).

  For example, in a situation where a white line is missing in the center part of the pedestrian crossing in the width direction of the road and it is erroneously detected that the measurement point P is present in the image processing, the conventional method is erroneously detected as it is. There is a possibility that the vehicle position is corrected based on the detected position.

  In contrast, the road marking position recognition device 103 according to the present embodiment globally estimates where the road marking 2 is visible in the image, and therefore there is a measurement point P by image processing. Is detected at the central part of the pedestrian crossing, the measurement point P does not exist, and it can be determined that it is a false detection.

  Accordingly, the output of the recognition result to the car navigation device 101 is prohibited (output disabled state), and the vehicle position can be prevented from being corrected based on the erroneously detected position, and the measurement point P of the road marking can be accurately set. Can be detected.

  Finally, in step S208, it is determined whether there is a new image input signal. If no new image input signal is received (NO), the process waits. If a new image input signal is received (YES), Return to the process 201.

Next, a hardware configuration for realizing the present invention will be described with reference to FIGS.
FIG. 7 shows a first embodiment of the hardware configuration, and FIG. 8 shows a second embodiment of the hardware configuration.

  First, the first embodiment will be described. The rear camera 108 has a lens 1001 and a CCD 1002 as shown in FIG. The road marking position recognition device 103 and the car navigation device 101 are mounted on the CPU 1004, and the map data 102 is mounted on the hard disk 1005. The CPU 1004 refers to the map data 102 on the hard disk 1005.

  The rear camera 108 and the CPU 1004 are connected by a video signal line 1003, and the video of the rear camera 108 is transmitted to the road marking position recognition device 103 of the CPU 1004 through the video signal line 1003. Further, the CPU 1004 is connected to a CAN (Control Area Network) 1006 so as to exchange data with other CPUs and control devices of the own vehicle C.

  When the configuration of the first embodiment is applied when the data transmission / reception amount between the road marking position recognition device 103 and the car navigation device 101 is large, the data transmission / reception speed and capacity increase, and the system performance is good.

  Next, a second embodiment will be described. As shown in FIG. 8, the rear camera 108 includes a lens 1101, a CCD 1102, and a CPU 1107. A road marking position recognition device 103 is mounted on the CPU 1107.

  On the other hand, a car navigation apparatus 101 is mounted on a CPU 1104 provided separately from the rear camera 108, and map data 102 is mounted on a hard disk 1105. The CPU 1104 refers to the map data 102 on the hard disk 1105.

  The CPU 1107 and the CPU 1104 are connected to the CAN 1106 and transmit / receive data between the road marking position recognition device 103 and the car navigation device 101, and also transmit / receive data to / from other CPUs and control devices of the host vehicle C. To do.

  Data transmission / reception between the road marking position recognition device 103 and the car navigation device 101 can also be performed by a dedicated signal line. When the configuration of the second embodiment is applied when the processing load of the road marking position recognition device 103 and the car navigation device 101 is large, road marking recognition and high-load processing of the car navigation device 101 are possible, and the system performance is good. .

  According to the road marking position recognizing device 103 described above, it is estimated where a part of the road marking 2 is imaged from a plurality of images captured as the host vehicle C moves, and the estimated imaging part is It is determined whether or not the site includes a measurement point P set in advance for the road marking 2.

  Therefore, when the image processing unit 111 determines that the measurement point P is detected at a part where the measurement point of the target feature does not exist, the recognition result output determination unit 114 determines that the detection result is a false detection, It is not output to the car navigation device 101. Accordingly, it is possible to prevent the vehicle position from being recognized with the erroneous point as the measurement point P.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, in the above-described embodiment, the case where the rear camera 108 is used has been described, but it can also be realized by using a vehicle-mounted camera installed in another part of the vehicle.

  For example, instead of the rear camera 108, a rear side camera installed in a door mirror portion where the rear side of the host vehicle C is set as a shooting range, and a front camera where the front side of the host vehicle C is set as a shooting range can be used. Moreover, the reliability of the road marking position recognition device 103 can be improved by using a plurality of cameras and collating the recognition results of the images of the cameras with each other. In that case, a combination of the rear camera 108 and the rear side camera, a combination of the rear camera 108 and the front camera, a combination of two rear side cameras installed on the left and right door mirrors, and a combination of the rear side camera and the front camera are possible. It is.

  Further, the road marking position recognition device 103 of the present invention can also recognize surrounding recognition objects other than road markings such as traffic lights and road signs in the own vehicle C equipped with the in-vehicle camera and the car navigation device 101. It can also be applied to a preventive safety system for a vehicle that performs collision prevention and driving support.

The figure which shows an example of the scene where the own vehicle position detection system is applied. The figure which shows notionally the whole structure of the own vehicle position detection system. The block diagram which shows typically the structure of the own vehicle position detection system. The flowchart explaining the processing content of a road marking position recognition apparatus. The flowchart explaining the content of the process of step S205 of FIG. It is a figure which shows the some image imaged with the movement of the own vehicle. 1 shows a first embodiment of a hardware configuration. 2 shows a second embodiment of the hardware configuration.

Explanation of symbols

1 road surface 2 road marking (crosswalk)
DESCRIPTION OF SYMBOLS 100 Own vehicle position detection system 101 Car navigation apparatus 102 Map data 103 Road marking position recognition apparatus (feature position recognition apparatus)
108 Rear camera 111 Image processing unit 112 Feature amount extraction unit 113 State estimation unit (imaging site estimation unit)
114 Recognition result output determination unit (determination unit)
121 Own vehicle position calculating means 122 Own vehicle position correcting means C Own vehicle L Distance P Measuring point

Claims (2)

A feature position recognition device that processes an image captured by an in-vehicle camera and recognizes the position of a target feature existing around the vehicle,
An imaging part estimation unit that estimates which part of the target feature is being imaged from a plurality of images that are imaged along with the movement of the vehicle;
A determination unit for determining whether the imaging part estimated by the imaging part estimation unit is a part including a measurement point set in advance for the target feature;
Have a, a feature extraction unit for extracting a quantity characteristic of the target feature is included in the image,
The imaging part estimation unit estimates the imaging part from a temporal change in the feature amount of each image, and also includes a state transition probability that is a probability that the state of the image transitions between the images, and a predetermined value from the image A feature position recognizing apparatus that estimates the imaging region using an observation probability that is a probability that the state of the image becomes a predetermined state when the feature amount is extracted . The feature position recognition apparatus according to claim 1 , wherein the state transition probability and the observation probability are learned in advance and stored as a database.

Images