JP2010060299A - Object detection apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object detection apparatus which detects a line pattern with high precision, while suppressing a processing load. <P>SOLUTION: A processing section 12, when a white line is detected by a processing section 22, calculates image characteristic quantities (saturation and hue) of the white line. The processing section 12 then detects a position having image characteristic quantities which agree with the calculated characteristic quantities of the white line in the entire image. An image at the position of the white line detected by the processing section 22 of a laser radar apparatus 2 and an image of the remote white line extracted by a camera are combined with each other. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an object detection device that detects the position of an obstacle or the like around a vehicle, and more particularly to an object detection device that detects a line pattern on a road surface.

  Conventionally, there are devices that detect obstacles around a vehicle and white lines on a road surface. An obstacle or a white line is detected using this device, and an alarm is issued when an obstacle is approached or when it is about to deviate from the white line.

  For example, Patent Documents 1 to 4 describe techniques for performing white line recognition using a monochrome camera. The white line recognition methods described in Patent Documents 1 to 4 detect white lines using complex image processing such as edge detection and Hough transform.

  Patent Documents 5 to 7 propose a method for detecting a white line using a laser radar device. The laser radar devices described in Patent Documents 5 to 7 scan a laser beam toward a road surface and detect a white line based on the intensity of the reflected wave. However, compared with the vehicle ahead, the intensity of the reflected wave from the white line is low, and the limit distance at which the white line can be detected is shorter than the limit distance at which the vehicle can be detected.

Further, Patent Documents 8 and 9 and Non-Patent Document 1 describe techniques for performing white line recognition from a color image. The white line recognition described in Patent Documents 8 and 9 and Non-Patent Document 1 is to register the color of the white line in advance and extract the white line portion compared with the color of the actually acquired color image. However, in order to register the color of the white line in advance, it is necessary to analyze the tendency of the color of the white line in advance. In addition, the hue may be different between the new white line and the old white line due to dirt or the like.
JP-A-10-188198 JP-A-11-66488 JP-A-11-85999 Japanese Patent Laid-Open No. 11-219435 JP 2000-147124 A JP 2003-121546 A JP-A-8-248133 Japanese Patent No. 3120648 Japanese Patent No. 3413745 "Lane recognition technology using laser radar", Takashi Ogawa, Seiwa Takagi, p5-p8, Automobile Engineering Society 2006 Spring Conference Academic Lecture Preprint No. 42-06

  An object of the present invention is to provide an object detection device that detects a line pattern with high accuracy while suppressing a processing load.

  The present invention relates to an electromagnetic pulse transmitting means for transmitting an electromagnetic pulse such as a laser beam, a scanning means for scanning the electromagnetic pulse, a scanning angle detecting means for detecting a scanning angle, a reflected wave receiving means for receiving a reflected wave of the electromagnetic pulse, an electromagnetic wave First object detection means for detecting an object based on an elapsed time from pulse transmission to reception and a scanning angle at the time of electromagnetic wave pulse transmission, imaging means for photographing the front, and feature amount calculation for calculating an image feature amount of the photographed image And a second object detecting means for detecting an object based on the calculated feature amount.

  Furthermore, the present invention provides a line pattern determination unit that determines whether or not the first object detection unit has detected a line pattern, a first line pattern position detection unit that detects a position of the detected line pattern, A line pattern feature amount calculating unit that calculates the image feature amount at a position detected by the first line pattern position detecting unit; and the image having an image feature amount that matches the image feature amount calculated by the line pattern feature amount calculating unit. Second line pattern position detecting means for extracting the position of the line pattern as a line pattern, a line pattern at the position of the line pattern detected by the first line pattern position detecting means, and the line pattern extracted by the second line pattern position detecting means And a line pattern synthesizing unit that synthesizes a line pattern far from the position detected by the first line pattern position detecting unit.

  Thus, according to the present invention, when a line pattern such as a white line is detected by the first object detection unit (for example, a laser radar device), the line pattern feature amount calculation unit (for example, a camera) detects the white line image feature amount at the position. (Saturation, hue, etc.) are calculated. Further, the second line pattern position detecting means extracts a position in the image where the white line is present by detecting a position having an image feature amount that matches the calculated image feature amount of the white line in the entire image. . Then, the image at the position of the white line detected by the laser radar device and the image of the far white line extracted by the camera are synthesized. First, the image feature amount is calculated at the position of the white line detected by the laser radar device, and then the white line image is extracted from the entire image. Therefore, it is possible to detect the line pattern with high accuracy while suppressing the processing load. it can.

  In the above invention, the line pattern synthesizing unit may be arranged around a position separated from the position of the line pattern detected by the first line pattern position detecting unit by a distance corresponding to a preset lane width in the image. The position of the image extracted as a line pattern by the second line pattern position detecting means can be handled as the position of the line pattern detected by the first line pattern position detecting means.

  In this case, even if the position of the white line detected by the laser radar apparatus is one, the positions of the two white lines sandwiching the own vehicle can be detected.

  According to the present invention, it is possible to detect a line pattern with high accuracy while suppressing a processing load.

  Hereinafter, an object detection apparatus including a camera and a laser radar apparatus will be described as an embodiment of the present invention.

  FIG. 1 is a diagram illustrating an example in which an object detection device is attached to an automobile. The object detection device includes a camera 1 and a laser radar device 2. The camera 1 and the laser radar device 2 are connected via a communication cable 3 such as CAN. The camera 1 is attached to, for example, the rear side of a rearview mirror in a vehicle interior, and photographs the front of the vehicle. The laser radar device 2 is attached to a bumper or the like in front of the vehicle, and irradiates laser light as an electromagnetic wave pulse in front of the vehicle.

  The photographing range of the camera 1 is set to be the same as the scanning range of the laser radar device 2 (or wider than the scanning range). In addition, the axes on which the camera 1 is installed (horizontal direction XC, vertical direction YC, front-rear direction ZC) and the axes on which the laser radar device 2 is installed (horizontal direction XL, vertical direction YL, front-rear direction ZL) are the same. It is attached to become. By making the axis where the camera 1 and the laser radar device 2 are installed the same, the scanning direction of the laser light and the photographing direction of the camera 1 are the same, and the scanning direction of the laser light is the photographing field of view (image). It is possible to convert it to (coordinates).

  FIG. 2 is a block diagram illustrating a configuration of the object detection apparatus. The camera 1 includes a communication unit 11, a processing unit 12, an imaging unit 13, and a lens 14. The laser radar device 2 includes a communication unit 21, a processing unit 22, an LD (Laser Diode) 23, a light projecting lens 24, a light receiving lens 25, a PD (Photo Diode) 26, a scanning control unit 27, a lens driving unit 28, and a scanning position. A detection unit 29 is provided.

  The imaging unit 13 of the camera 1 is composed of a CCD, a CMOS, or the like, and images the front of the vehicle via the lens 14 under the control of the processing unit 12 and outputs the captured image to the processing unit 12.

  The processing unit 12 performs analysis (image processing) of the image input from the imaging unit 13. Further, the processing unit 12 outputs the result of the image processing to the communication unit 11. The communication unit 11 outputs the result of image processing to the laser radar device 2.

  The processing unit 22 of the laser radar device 2 controls the light emission of the LD 23. The laser beam generated by the LD 23 is irradiated to the front of the vehicle via the light projection lens 24. The reflected light that is reflected by the irradiated laser light and reflected back to the front target (vehicle or road surface) is collected by the light receiving lens 25 and received (received) by the PD 26.

  Further, the processing unit 22 controls the scanning control unit 27. The scanning control unit 27 scans the laser beam in a predetermined scanning range according to the control of the processing unit 22. That is, the scanning control unit 27 drives the lens driving unit 28 to scan the laser light by integrally moving the light projecting lens 24 and the light receiving lens 25 to a predetermined position. In the example of FIG. 2, scanning in the horizontal direction is shown, but laser light can be scanned in the same manner in the vertical direction.

  FIG. 3 is a diagram illustrating a state in which laser light is scanned. In the figure, an example in which the irradiation angle of the laser beam is changed in three stages in the vertical direction is shown. The broken line in the figure indicates the cross-sectional shape of the laser light projected on the road surface or the vehicle, and the one-dot broken line indicates the locus of the laser light. As shown in FIG. 3, the scanning direction of the laser beam and the shooting direction of the camera 1 are the same direction, and the scanning direction of the laser beam can be converted into the shooting field of view (image coordinates) of the camera 1. It has become.

  In FIG. 2, the scanning position detection unit 29 detects the scanning direction (scanning angle) in the horizontal direction and the vertical direction of the laser light in the scanning control unit 27, and outputs it to the processing unit 22. The conversion from the scanning direction to the image coordinates is performed by the processing unit 22.

  A signal corresponding to the reflection intensity of the reflected light received by the PD 26 is input to the processing unit 22. The processing unit 22 digitizes the reflection intensity and stores it in a memory (not shown) corresponding to the scanning direction input from the scanning position detection unit 29.

  Further, the processing unit 22 measures the distance between the target (target object) and the own vehicle based on the elapsed time from when the LD 23 emits light and irradiates the laser beam to when the reflected light is received. The processing unit 22 identifies the type of object from the characteristic of the reflection intensity. In the present embodiment, it is determined whether or not the object is a white line. When it is determined that the line is a white line, the position (distance and scanning direction) of the white line is converted into image coordinates of the camera 1 and transmitted to the camera 1 via the communication unit 21.

  The object detection apparatus of the present embodiment calculates image feature amounts (saturation, hue, etc.) at the position of the white line detected by the laser radar apparatus 2. Further, the processing unit 12 extracts a position having an image feature amount that matches the calculated image feature amount of the white line in the entire image. Then, the white line image near the own vehicle detected by the laser radar device 2 and the far white line image are synthesized, and the position of the white line in the image is detected. The position of the white line detected in this way is output from the processing unit 22 of the laser radar apparatus 2 to the vehicle control unit or the like, and is used for issuing an alarm.

  Hereinafter, a specific method of white line recognition will be described with reference to the flowchart shown in FIG. FIG. 12 is a flowchart showing the operation of the object detection apparatus. First, the processing unit 22 of the laser radar device 2 detects a white line from the characteristics of the received light intensity (s11).

  A method for detecting a white line from the characteristic of the reflection intensity will be described with reference to FIG. FIG. 4A is a diagram showing the relationship between the emission intensity and the reflection intensity when the preceding vehicle and the road surface are scanned with laser light. FIG. 4B is a diagram illustrating characteristics of received light intensity.

  When irradiating the laser beam toward the preceding vehicle, the laser beam is irradiated in the horizontal direction in front of the vehicle as viewed from the side of the vehicle. In this case, the laser beam is reflected on the reflecting plate of the preceding vehicle. Since the reflector is vertically attached to the rear of the preceding vehicle, the distance from the laser radar device 2 to the reflector is almost the same regardless of which surface of the reflector is irradiated with the laser beam. Therefore, the time from light projection to light reception is almost the same no matter which surface of the reflecting plate reflects the laser light. As a result, the reflection intensity shows a steep peak in a short time.

  On the other hand, when irradiating the laser beam toward the road surface, the laser beam is irradiated obliquely downward in front of the vehicle. In this case, since the time from light projection to light reception differs depending on the position where the road surface reflects, the reflection intensity shows a gradual peak. Further, since the white line on the road surface contains a material having high reflectivity such as glass, the laser light applied to the white line shows a relatively high reflection intensity. Therefore, the processing unit 22 of the laser radar apparatus 2 can detect a white line from the values of the peak received light intensity Ir and the pulse width (half width) Wr shown in FIG. That is, the processing unit 22 determines that a white line is present when the received light intensity Ir is equal to or greater than a predetermined threshold and the pulse width Wr is equal to or greater than the predetermined threshold. The distance Dp between the vehicle and the white line is represented by the elapsed time from light projection to peak detection.

  Next, in the flowchart of FIG. 12, after detecting the white line, the processing unit 22 converts the distance and the scanning direction of the white line into the coordinate information of the image of the camera 1 and transmits it to the camera 1 (s12). When the position of the white line detected by the processing unit 22 of the laser radar device 2 is projected onto the image, the result is as shown in FIG. A broken line in FIG. 3A indicates the position of the white line detected by the processing unit 22 of the laser radar device 2. In this example, a solid line is present on the left side of the road surface ahead of the vehicle, an arrow line is present in the vicinity of the front center, and a broken line is present on the front right side, and a part of each white line is detected by the laser radar device 2. . The processing unit 12 of the camera 1 sets a rectangular area as shown in FIG. 5B at the position of the white line detected by the processing unit 22 of the laser radar device 2 (s13). The size of the rectangular area may be appropriately set according to the processing capability of the processing unit 12 and the required processing time. The processing unit 12 calculates the image feature amount within the set rectangular area.

  First, the processing unit 12 obtains an average value of RGB values of each pixel in the set rectangular area (s14). Then, a process of converting the RGB value average value into the HSV color coordinate system is performed (s15). The HSV color coordinate system will be described with reference to FIG. The HSV color coordinate system is a color space composed of three components of hue (H), saturation (S), and brightness (V). The processing unit 12 calculates the hue and saturation as an image feature amount. As shown in FIG. 6, the processing unit 12 obtains the hue and saturation distribution of each rectangular area, and clusters (classifies) those having similar hue and saturation as the main cluster (set) representing the color of the white line. ) (S16). Note that the processing unit 12 does not consider the brightness, and eliminates the influence of the change in the brightness of the white line due to dirt or the like.

  Thereafter, the processing unit 12 calculates the hue and saturation in all the pixels of the image, and extracts a position having the hue and saturation included in the main cluster (s17). FIG. 7 is a diagram showing that only the pixels included in the main cluster are extracted from all the pixels. The hatched portion in the figure shows the pixels included in the main cluster.

  Thereafter, the processing unit 12 sets N straight lines connecting to the horizontal line for any one of the plurality of white line positions detected by the laser radar device 2 (s18). The straight line setting will be described with reference to FIG. As illustrated in FIG. 8, the processing unit 12 sets any one position as the position A among the positions of the white lines detected by the laser radar device 2. From this position A, a plurality of straight lines 1 to N are set with respect to a preset horizontal line (for example, a horizontal line passing through the vanishing point). The number of straight lines is appropriately set according to the image processing capability of the processing unit 12 and the required processing time. The processing unit 12 calculates the number of pixels included in the main cluster of white lines in each set straight line, and creates a frequency distribution (histogram) (s19).

  FIG. 9A shows the frequency distribution of each straight line. The horizontal axis of the graph shown in FIG. 6A indicates the respective straight lines 1 to N, and the vertical axis indicates the number of pixels (frequency) included in the main cluster. The processing unit 12 identifies the straight line K having the highest frequency (having a frequency peak) from these straight lines (s20). FIG. 9B is a diagram showing the identified straight line K on the image.

  The processing unit 12 repeats the above-described straight line setting, frequency distribution calculation, and straight line identification process having a frequency peak a plurality of times (for example, the number of set rectangular areas) (s21) to identify a plurality of straight lines. (S22).

  FIG. 10A shows an example in which three straight lines are specified. FIG. 4A shows an example in which straight lines A, B, and C are specified for a solid line on the front left side, an arrow line on the front side of the host vehicle, and a broken line on the front right side. The processing unit 12 compares the frequencies in the three identified straight lines. FIG. 10B is a diagram comparing the frequency of each straight line. The processing unit 12 selects two straight lines with the highest frequency from these straight lines (s23).

  FIG. 11 is a diagram showing the two selected straight lines in the image. As shown in FIG. 6A, since solid lines and broken lines existing on the road surface have more positions including image features of white lines than arrow lines, if two straight lines with the highest frequency are selected, the vehicle Can be approximated as two lanes. Then, the processing unit 12 extracts pixels included in the main cluster in the vicinity of the two selected straight lines, and detects the position of the extracted pixels as the position of the white line (s24). FIG. 11B shows the detected white line in the image. The hatched portion in the figure is the position detected as a white line. In this example, two lanes sandwiching the host vehicle are approximated by a straight line, and pixels included in the main cluster are extracted in the vicinity thereof, so that the position of the lane can be accurately detected even if the lane is slightly bent. it can.

  In this way, by synthesizing the white line image around the position of the white line detected by the laser radar device 2 and the image of the white line extracted far from the position, two lanes (white line) sandwiching the vehicle ) Position can be accurately detected.

  Finally, the processing unit 12 stores the extracted white line pixel position in a memory (not shown) or outputs it to the processing unit 22 of the laser radar device 2 (s25). When output to the processing unit 22 of the laser radar device 2, the information indicating the position of the white line is output to the vehicle control unit or the like and used for issuing an alarm.

  As described above, the object detection apparatus according to the present embodiment first detects a white line with a laser radar device, calculates an image feature amount at the position, and then extracts a position included in the image feature amount at all pixels. Thus, it is possible to detect the line pattern with high accuracy while suppressing the processing load.

  In the flowchart shown in FIG. 12, the processing performed in s21 to s23 can be replaced with the processing in the following example. FIG. 13 is a diagram illustrating a white line detection method in another example. FIG. 14 is a flowchart showing an operation in another example of the object detection apparatus. In the flowchart shown in FIG. 14, processes that are the same as those in FIG. 12 are given the same reference numerals, and descriptions thereof are omitted.

  In this example, the processing unit 12 specifies the straight line K having the largest number of pixels (frequency) included in the main cluster from the position A of the white line detected by the laser radar device 2, and then sets in advance from the position A of the white line. A search range for detecting a white line is set around a position separated by a distance corresponding to the lane width (minimum lane width) (s31). The processing unit 12 determines whether or not there are pixels included in the main cluster in the set search range (s32). When it is determined that there are no pixels included in the main cluster and no white line is present, the operation ends (waits for the next measurement). When a white line is detected by the laser radar device 2 on the right side of the image, a search range is set toward the left side of the image, and when a white line is detected by the laser radar device 2 on the left side of the image, the right side of the image is directed. Set the search range. If a white line is detected near the center, the operation ends.

  If it is determined that a white line is present, the processing unit 12 sets N straight lines with respect to the horizontal line as shown in FIG. 8 from the position including the pixel, and the number of pixels (frequency) included in the main cluster. The straight line K having the largest number is identified (s33). Then, pixels included in the main cluster are extracted in the vicinity of the two specified straight lines, and the positions of the extracted pixels are detected as white lines (s24).

  As described above, in the example shown in FIGS. 13 and 14, the image characteristic amount of the white line around the position separated by the distance corresponding to the preset lane width is set with respect to the position of the white line detected by the laser radar device 2. Extract. The position where the image feature amount can be extracted is treated as the position of the white line detected by the laser radar device 2. In this case, even if the position where the white line is detected in the laser radar device 2 is one place, the positions of the two white lines sandwiching the own vehicle can be detected.

  Note that the line pattern detected by the present invention is not limited to a white line as in the present embodiment, but may be any line pattern such as a yellow line, a median strip, a road and roadside boundary line, and the like. Good.

It is a figure which shows the example which attached the object detection apparatus to the motor vehicle. It is a block diagram which shows the structure of a camera and a laser radar apparatus. It is a figure which shows a mode that the laser beam is scanned. It is a figure which shows the relationship between the light emission intensity | strength at the time of scanning a laser beam to a preceding vehicle and a road surface. It is a figure which shows a mode that the position of the white line detected by the process part 22 of the laser radar apparatus 2 was projected on the image. It is a figure which shows having converted RGB into HSV and having assembled it as a main cluster. It is a figure which shows having extracted only the pixel contained in the main cluster in all the pixels. It is a figure which shows the example which sets the N straight line connected with a horizontal line with respect to one (position A) among several white line positions detected with the laser radar apparatus. It is the figure which shows the frequency distribution of each straight line, and the figure which represented the specified straight line K on the image. It is the figure which showed the specified several straight line and each frequency. It is the figure which represented two straight lines with the most frequent frequency in the image. It is a flowchart which shows operation | movement of an object detection apparatus. It is a figure which shows the method of the white line detection in another example. It is a flowchart which shows the operation | movement in the other example of an object detection apparatus.

Claims (2)

An electromagnetic pulse transmitting means for transmitting an electromagnetic pulse forward;
Scanning means for scanning the electromagnetic wave pulse transmitted by the electromagnetic wave pulse transmitting means;
Scanning angle detection means for detecting a scanning angle of the scanning means;
About the electromagnetic wave pulse transmitted by the electromagnetic wave pulse transmitting means, the reflected wave receiving means for receiving the electromagnetic wave pulse reflected by the target;
The intensity of the received electromagnetic wave pulse, the elapsed time from when the electromagnetic wave pulse transmitting unit transmits the electromagnetic wave pulse until the reflected wave receiving unit receives the reflected electromagnetic wave pulse, and the electromagnetic wave pulse transmitting unit transmits the electromagnetic wave pulse First object detection means for detecting an object reflecting the electromagnetic wave pulse based on the scanning angle detected by the scanning angle detection means when
Imaging means for imaging the front and outputting an image;
Feature quantity calculating means for calculating an image feature quantity based on the image output by the imaging means;
An object detection device comprising: second object detection means for detecting a type of an object in the image based on a calculation result of the feature quantity calculation means;
Line pattern determination means for determining whether or not the first object detection means has detected a line pattern;
First line pattern position detecting means for detecting the position of the detected line pattern;
A line pattern feature quantity calculating means for calculating the image feature quantity at the position detected by the first line pattern position detecting means;
Second line pattern position detecting means for extracting, as a line pattern, the position of the image having an image feature quantity that matches the image feature quantity calculated by the line pattern feature quantity calculating means;
Of the line pattern at the position of the line pattern detected by the first line pattern position detection unit and the line pattern extracted by the second line pattern position detection unit, farther than the position detected by the first line pattern position detection unit A line pattern combining means for combining the line pattern of
An object detection apparatus comprising:   The line pattern synthesizing unit is configured to output the second line pattern around a position separated from the position of the line pattern detected by the first line pattern position detecting unit by a distance corresponding to a preset lane width in the image. The object detection apparatus according to claim 1, wherein the position of the image extracted as a line pattern by the position detection unit is handled as the position of the line pattern detected by the first line pattern position detection unit.

Images