JP6307037B2 - Obstacle detection method and apparatus - Google Patents

Description

  The present invention relates to an obstacle detection method and apparatus, and more particularly to a method and apparatus for detecting an obstacle present on a track by stereo image processing.

  As a conventional technique of an obstacle detection method based on image processing, there is a method of detecting HOG (Histogram of Oriented Gradients) features and edges. A method for calculating the distance from an image with parallax to an obstacle by stereo image processing is also known.

  Stereo image processing is a method of three-dimensionally detecting the position of an object existing ahead and measuring the distance to the object. The parallax is calculated by performing a matching process on the images (stereo images) of the target object obtained by the left and right cameras placed at horizontal intervals with an interval between them, and the target object is obtained from the principle of triangulation using the parallax. Is calculated (for example, refer to Patent Documents 1 and 2).

FIG. 16 shows the principle of distance measurement by the stereo image processing technique. FIG. 16 shows images I and J when the object m is imaged by two left and right cameras whose y-axis (v-axis) in the image is aligned by parallelization. The u axis and u ′ axis indicate the horizontal positions of the images I and J, and the v axis indicates the vertical position of the images. On the image of each camera, the object m is shown at the intersections p 1 and q between the lines l and r connecting the camera viewpoint and the object m and the screen, so that the three-dimensional coordinates (X, Y) of the object m are displayed. , Z) is obtained from the coordinates (u _s1 , v ₁ ) and (u _s2 , v ₂ ) of p and q. The difference d between the coordinate positions of the points p and q, that is, the number of pixels is parallax, and the parallax d can be obtained from the following equation 1.
d = u _s1 −u _s2 (Formula 1)

When the focal length of the camera is f and the distance between the center positions of the two images I and J, that is, the base line length is B, the relationship between the world coordinates of m and the position on the image is as follows.
_{X = Bu s1 / d, Y} = Bv 1 / d, Z = Bf / d
Z is the distance from the camera installation position to the object m.

  Therefore, in order to measure the distance Z to the object m in the stereo image processing, it is necessary to calculate the parallax d. For this purpose, a point having a correlation with a reference point p on the left camera image I (hereinafter referred to as a reference image), that is, a corresponding point q is present on the right camera image J (hereinafter referred to as a corresponding image). Image matching (association) for searching whether to do is necessary. The reference point is a predetermined point in the reference image, and the corresponding point is a point on the corresponding image whose correlation is obtained with the reference point.

  Conventionally, a block matching method and a phase only correlation method are used for image matching.

A conventional image matching (image search) method in the block matching method will be described with reference to FIG. The reference image I and the corresponding image J are each divided into the same number of small area regions (hereinafter referred to as blocks). 17A and 17B, the reference image I is blocks I _{11 to} I ₁₆ , I _{21 to} I ₂₆ ,... I _{41 to} I ₄₆ , and the corresponding image J is blocks J _{11 to} J ₁₆ , J _{21 to} J ₂₆ ,... J _{41 to} J ₄₆ are shown in a simplified example.

Each block includes n pixels in the horizontal direction and m pixels in the vertical direction as illustrated in FIGS. 17C and 17D for the blocks I ₁₁ and J ₁₁ of the reference image I and the corresponding image J. Each pixel has a pixel value (pixel data). In block matching, the corresponding image J is searched for a block having the same pixel value array as the pixel value array of each block of the basic image I.

To perform the search, the first pixel _{JP 11} in the first row and first column of the block _{J 11} pixel values of the first pixel _{IP 11} of the first row and first column of the block _{I 11} of the reference image I and the corresponding image J of comparing the pixel value (subtraction), the difference as shown in (e) of FIG. 17, recorded in the first area DP ₁₁ of pixel value difference matrix DT.

Then, such pixel value comparison is performed for all the pixels in the block I ₁₁ and the block J ₁₁ . That is, other pixels _IP 12 _{~IP 1n} the bank of the block _{I 11} of the reference image I, pixel _IP 21 _~IP 2n of the other rows, the difference in the respective similarly pixel values of _{··· IP} m1 ~IP _mn The difference is recorded in other areas DP _{12 to} DP _mn of the pixel value difference matrix DT. Then, the sum of the difference between the pixel values of all the pixels, the total value t ₁₁ as shown in (f) of FIG. 17, and records the pixel value difference sum matrix TT.

Then, also for the first row and second column of the block J ₁₂ of the corresponding image J, taking the difference between the pixel values of all pixels in the first row and first column of the block I ₁₁ of the reference image I, the difference was recorded in the pixel value difference matrix DT, sums the difference, and records the total value t ₁₂ to the pixel value difference sum matrix TT.

It performs the same process until the last block J ₁₆ of the first row of the corresponding image J. Then, in the total value _t 11 _{~t 16} pixel value difference corresponding to all the block _I 11 _{~I 16} of the first row of the corresponding image J of the pixel value difference sum matrix TT, the total value is based on the smallest block It is determined that it corresponds to the block I ₁₁ in the first row and first column of the image I, that is, matching has been achieved.

Subsequently, the first row and second column of the block I ₁₂ of the reference image I, by scanning all blocks of the first row of the corresponding image J performs the same matching processing as described above. Further, all blocks in the last row of the reference image I are scanned and matching processing is performed in the same manner as described above.

  In the block matching method, since the block on the corresponding image J having the smallest sum of the pixel value differences of the pixel value difference sum matrix TT corresponds to the block of the reference image I, it corresponds from the block of the reference image I at that time. The number of pixels up to the block of image J represents parallax.

  As described above, in the conventional search method, in order to search for the corresponding point q on the corresponding image J corresponding to the reference point p of the reference image I, on the same scanning line (line) as the reference point p of the corresponding image J. Was the search range (see, for example, Patent Document 3).

JP 2002-250605 A JP 2008-039491 A JP 2014-235615 A

IEEE Transactions on Industry Applications Vol.134 No.10 pp921-929,2014

  Therefore, in the conventional stereo image processing method, since the entire length of the corresponding image on the scanning line is set as the search range, the matching process takes a very long time. Therefore, when the conventional stereo image processing method is applied to, for example, detecting an obstacle on a track from a vehicle traveling on a track, there is a problem that an obstacle cannot be detected in real time.

  In addition, the phase-only correlation method has an advantage that high-precision matching is possible. However, since the complex Fourier transform is performed on each block of each line of the reference image and the corresponding image, the calculation amount becomes enormous. In addition to the large computational load of the computer, the calculation cost is high, and there is a problem in the obstacle detection speed as in other conventional stereo image processing methods.

  Therefore, an object of the present invention is to provide an obstacle detection method and apparatus capable of improving the obstacle detection speed when an obstacle present on a track is detected by stereo image processing.

  One aspect of the present invention is an obstacle detection method, the step of imaging the front of the train traveling direction with the left and right cameras, the step of extracting the rail from the acquired image, and the rail from which the left and right images are extracted The step of calculating the parallax of the left and right rails by performing a matching process using “parallax by rail extraction”, the step of comparing the calculated parallax with a predetermined parallax model when there is no obstacle, and the result of the comparison When it does, it determines with the obstruction not existing on a track | line, When it does not correspond, the process of determining with an obstruction existing on a track | line is included.

Another aspect of the present invention is an obstacle detection device for using the above-described obstacle detection method, and includes a stereo image processing unit, an obstacle presence / absence determination unit, and an obstacle detection signal output unit. A distance measuring unit is often added to the obstacle detection device.
The stereo image processing unit includes left and right camera devices installed at horizontal equidistant positions, a rail extraction unit that extracts a rail image from an image signal acquired by each camera device, and an extracted rail image. A parallax calculation unit that calculates a parallax by performing a matching process on a stereo image having a parallax using “rail extraction parallax”.

The parallax calculation unit detects a primary corresponding point on the corresponding image having the same coordinates as the reference point of the rail image of the reference image, and outputs a corresponding point on the same scanning line from the primary corresponding point. A second corresponding point generating unit having a point shifted left or right by a distance equal to “parallax by rail extraction” as a second corresponding point, and a matching process from the second corresponding point to the left end or right end of the corresponding image The matching processing means for obtaining the correlation value and the correlation value calculated by the matching processing means are compared with a predetermined threshold value. When the correlation value does not satisfy the predetermined condition with respect to the predetermined threshold value, a first determination signal is output. A correlation value comparison unit that outputs a second determination signal when the correlation value satisfies a predetermined condition with respect to a predetermined threshold, and a well-known matching method when the correlation value comparison unit outputs a first determination signal, For example When the parallax calculation means that executes the matching process by the dense search method and the correlation value comparison means outputs the second determination signal, the “parallax by rail extraction” stored in the memory unit is read, Disparity reading means for determining the “parallax by rail extraction” as the reference point disparity.
The first determination signal and the second determination signal output from the correlation value comparison unit are also provided to the obstacle existence determination unit. The obstacle presence / absence determination unit determines that there is an obstacle when the first determination signal is given, and determines that there is no obstacle when the second determination signal is given.
When the distance measuring unit is added, the parallax calculated by the parallax calculating unit and the parallax read by the parallax reading unit are given to the distance measuring unit, and the distance to the obstacle is measured based on the triangulation method.

“Parallax by rail extraction” is calculated by one of the following formulas using coordinates previously searched using a well-known matching technique.
d _h = L _h (I) −L _h (J)
d _h = R _h (I) −R _h (J)
d _h = {(L _h (I) −L _h (J)) + (R _h (I) −R _h (J))} / 2
Where d _h is the rail parallax at height _h in the image,
L _h (I) is a function representing the u coordinate of the left rail at the height h in the left image,
L _h (J) is a function representing the u ′ coordinate of the left rail at the height h in the right image,
R _h (I) is a function representing the u coordinate of the right rail at the height h in the left image,
R _h (J) is a function representing the u ′ coordinate of the right rail at the height h in the right image.

  According to the present invention, it is possible to speed up the detection of the presence of an obstacle.

It is a block diagram which shows the structure of the obstruction detection apparatus which enforces the obstruction detection method which concerns on embodiment of this invention. It is a block diagram which shows the structure of the camera apparatus of FIG. It is a block diagram which shows the structure of the rail extraction part of FIG. It is a flowchart explaining the effect | action of the rail image | video estimation part of FIG. It is a figure which shows the typical rail shape pattern registered into the memory | storage part of FIG. It is a flowchart explaining the effect | action of the rail image | video comparison part of FIG. It is a figure which shows an example of the stereo image which has displayed the extracted rail. It is a figure which shows the means to implement | achieve the function of the parallax calculation part of FIG. It is a figure explaining "parallax by rail extraction". It is a figure explaining how to obtain "parallax by rail extraction". It is a schematic diagram explaining the specific method of a matching process. It is a side view explaining the principle of the method of detecting the obstruction which exists on a track | line by the stereo image processing of this invention. It is also a plan view. It is a figure which shows the image of the camera on either side in FIG. It is a figure which shows the relationship between the image of a camera on either side, and parallax when an obstruction exists on a track. It is a figure explaining the principle of the distance measurement by a stereo image processing technique. It is a figure explaining the data processing in the conventional block matching method.

  Next, embodiments of the present invention will be described with reference to the drawings.

  The stereo image processing method according to the embodiment of the present invention is applied to a case where an object to be imaged by an imaging means that is two left and right cameras is a railroad track. When the front of the train is imaged by the left and right cameras installed on the train cab, the range in which the image of the vicinity of the image including the track or the track is captured is limited.

  Further, when the imaging target is a railroad track, a rail can be extracted from the train front image by a rail extraction algorithm, and coordinates on the extracted rail image can be detected. As the rail extraction algorithm, there is a method that focuses on the edge of the rail and the luminance gradient (see Non-Patent Document 1). In this method, the processing is performed by dividing the rail into a rail in the vicinity region from the train cab to about 2 to 30 m and a rail in the far region at a distance longer than that. The near area is matched with a representative rail template prepared in advance, and the far area is dynamically generated based on the information obtained in the near area while connecting short straight lines and curved segments. Track the rails far away.

  The rail extraction algorithm is a method based on matching with the above-described representative rail template and matching with a rail shape pattern, a method based on feature amount extraction in an image, a method based on features based on color and intensity, a method using a luminance gradient, There are others. In the present invention, any of these known rail extraction algorithms may be used.

  When the front of the train is photographed with two cameras on the left and right, if the rail is extracted from the left camera image (reference image) and the right camera image (corresponding image) by the rail extraction algorithm, the coordinates on the extracted rail image I understand. The number of pixels between coordinates indicating the same point on the rail in the reference image and the corresponding image is “parallax by rail extraction”.

Since two rails are laid in parallel, the “parallax by rail extraction” can be calculated from the coordinates of one of the left and right rails. The average value of the difference between the left and right rail coordinates (parallax due to rail extraction of each rail) may be set as “parallax due to rail extraction”. Therefore, “parallax by rail extraction” can be obtained by calculating one of the following expressions.
d _h = L _h (I) −L _h (J)
d _h = R _h (I) −R _h (J)
d _h = {(L _h (I) −L _h (J)) + (R _h (I) −R _h (J))} / 2
Where d _h is the rail parallax at height _h in the image,
L _h (I) is a function representing the u coordinate of the left rail at the height h in the left image,
L _h (J) is a function representing the u ′ coordinate of the left rail at the height h in the right image,
R _h (I) is a function representing the u coordinate of the right rail at the height h in the left image,
R _h (J) is a function representing the u ′ coordinate of the right rail at the height h in the right image.

  The present invention is limited to a range where a video of a neighborhood including a track or a track is limited among videos captured in the left and right images, a rail linear portion extracted by a rail extraction algorithm, The basic technology is to use the fact that the rate of change of “parallax due to rail extraction” corresponding to the height position on the screen is almost constant in any of the curved parts to speed up the parallax calculation of stereo images. It is thought.

  If “parallax by rail extraction” is known, it is possible to estimate where the corresponding point of the corresponding image exists. In other words, a true corresponding point in the vicinity of a point (secondary corresponding point) shifted to the left or right by the amount of “parallax by rail extraction” from the same coordinate (primary corresponding point) on the corresponding image corresponding to the reference point (coordinate). Can be considered to exist. Therefore, in the present invention, matching is started from the secondary corresponding point, the point where the same pixel value is found is determined as the true corresponding point, and the number of moving pixels from the reference point to the true corresponding point is set as the parallax. It is. Of the left and right images, the direction of shifting differs by “parallax by rail extraction” depending on which is the basic image and which is the corresponding image. When the left image is used as a basic image, it is shifted to the left. When the right image is used as a basic image, it is shifted to the right.

[First embodiment]
<Schematic configuration>
FIG. 1 shows an example of an obstacle detection apparatus according to an embodiment of the present invention. The obstacle detection apparatus A includes a stereo image processing unit 1, an obstacle presence / absence determination unit 2, an obstacle detection signal output unit 3, and a distance measurement unit 4.

<Stereo image processing unit>
The stereo image processing unit 1 includes two camera devices 10L and 10R, rail extraction units 20L and 20R to which video signals are given from the camera devices 10L and 10R, and one parallax connected to the rail extraction units 20L and 20R. And a calculation unit 30.

  As the camera devices 10L and 10R and the rail extraction units 20L and 20R, those disclosed in Patent Document 3 can be used. Hereinafter, the description in Patent Document 3 will be described.

  The camera devices 10L and 10R have the same configuration. The camera device 10L will be described as an example. As illustrated in FIG. 2, the camera device 10L includes an optical lens 11, an image sensor 12 that converts an optical image of a subject incident from the optical lens 11 into an electrical signal, and an image sensor. 12 includes a video signal generation unit 13 that generates a video signal based on the 12 outputs, and an imaging control unit 14.

  The optical lens 11 optically forms an image of the subject on the image sensor 12. The optical lens 11 has a focus adjustment mechanism for adjusting the focus of the lens and an aperture adjustment mechanism for adjusting the amount of light incident on the image sensor 12. Furthermore, the optical lens 11 may have a zoom mechanism that changes the focal length.

  For the imaging device 12, for example, an aggregate of about several million charge coupled devices such as CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) is used.

  The imaging control unit 14 controls imaging of a subject by the optical lens 11, the imaging element 12, and the video signal generation unit 13. That is, the imaging control unit 14 controls the focus adjustment mechanism, the diaphragm adjustment mechanism, the zoom mechanism, and the like of the optical lens 11.

  The video signal generation unit 13 generates a video signal based on the output (that is, pixel value) of the image sensor 12. For example, the video signal generation unit 13 periodically captures the output of each charge coupled device of the image sensor 12 and generates a video signal under the control of the imaging control unit 14. The cycle in which the video signal generator 13 generates the video signal is, for example, about several tenths of a second. When a plurality of video signals periodically generated in this way are displayed as still images in chronological order, they are recognized as moving images by human eyes.

  The rail extraction units 20L and 20R in FIG. 1 have the same configuration. Below, the rail extraction part 20L is demonstrated to an example. In summary, the function of the rail extraction unit 20L is to take in the video signal generated by the video signal generation unit 13 of the camera device 10L and extract the rail included in the video signal.

  More specifically, the rail extraction unit 20L compares a plurality of rail shape patterns (image signals) registered in advance as images of a predetermined rail with the actually captured rails (image signals). Then, “match” or “mismatch” is output as the extraction result. That is, when the rail extraction unit 20L outputs “match”, the video signal generated by the video signal generation unit 13 includes a predetermined rail (the video signal thereof), and the rail extraction unit 20L includes the predetermined rail ( Video). When the rail extraction unit 20L outputs “mismatch”, the video signal generated by the video signal generation unit 13 does not include the predetermined rail (the video signal thereof), and the rail extraction unit 20L does not include the predetermined rail ( Video) could not be extracted.

  As illustrated in FIG. 3, the rail extraction unit 20 </ b> L receives the video signal output from the video signal generation unit 13 and extracts a video signal estimated to include a rail from the video signal. A storage unit 22 that is a memory that stores a plurality of rail shape patterns, a rail image comparison unit 23 that compares a rail (image) estimated by the rail image estimation unit 21 and a plurality of rail shape patterns, and And a data storage unit 24.

  The rail video estimation unit 21 extracts a video signal that is estimated to be a rail (video) from the video signal output from the video signal generation unit 13. For example, since the camera device 10 is mounted at the head of a railway vehicle (hereinafter simply referred to as a vehicle), a rail image is usually reflected in the lower part of the field of view of the camera device 10. Therefore, it is estimated that the two lines (images) extending in the vertical direction from the lower part of the field of view are the rails (images). Therefore, the rail video estimation unit 21 generates a video signal in which only the video signal portion that is estimated to be a rail is left and other video signals are deleted, and sends the video signal to the rail video comparison unit 22. This video signal is represented as coordinate information in which vertical and horizontal pixel values corresponding to the arrangement positions of the vertical and horizontal charge coupled devices of the image sensor 12 are recorded.

  The operation of the rail image estimation unit 21 will be described in more detail with reference to the flowchart of FIG. When the video signal output from the video signal generation unit 13 is input to the rail video estimation unit 21, the condition of “START” in FIG. 4 is satisfied, and the flow proceeds to step S1. Note that the processing from “START” to “END” in FIG. 4 is processing for one cycle. After the flow becomes “END”, the flow is executed again when the condition of “START” is satisfied. Is done.

  In step S1, the rail image estimation unit 21 searches the position of the rail (image) in the lower part of the visual field in the image signal input from the image signal generation unit 13, and the flow proceeds to step S2.

  In step S2, the rail image estimation unit 21 determines whether two vertical lines are found. If it is determined in step S2 that it has been found, the flow proceeds to step S3. On the other hand, if it is determined in step S2 that no sheet is found, the flow returns to step S1.

  In step S3, the rail video estimation unit 21 deletes video signals other than video signals estimated as rails, and the flow proceeds to step S4.

  In step S4, the rail image estimation unit 21 outputs a video signal estimated to be rail (image) to the rail image comparison unit 23, and ends the processing (END).

  The rail video comparison unit 23 compares the video signal estimated from the rail (video) output from the rail video estimation unit 21 with the rail shape pattern (video signal) stored in the storage unit 22.

  The storage unit 22 stores rails that can be seen from the position of the camera device 10 as a plurality of types of rail shape patterns. FIG. 5 illustrates seven types (# 1 to # 7) of rail shape patterns stored in the storage unit 22. This rail shape pattern represents the type of rail appearance assumed when the vehicle is in a straight line or traveling on a curve in the line section of the vehicle on which the camera device 10 is mounted. . Such a rail shape pattern is set based on a video signal of a rail imaged by running a vehicle on which the camera device 10 is mounted.

  The rail shape patterns # 1 to # 7 stored in the storage unit 22 also have the vertical and horizontal directions corresponding to the arrangement positions of the vertical and horizontal charge-coupled elements of the image sensor 12, similarly to the video signal output from the rail video estimation unit 21. The pixel value is expressed as recorded coordinate information.

  The rail image comparison unit 23 compares the coordinates corresponding to the image signal estimated from the rail image output from the rail image estimation unit 21 and the image signal of the rail shape pattern stored in the storage unit 22. As a result of the comparison, the rail image comparison unit 23 determines “match” if the coordinate matching ratio is equal to or higher than a predetermined ratio (for example, about 80%), and determines “mismatch” if it is less than the predetermined ratio. .

  Next, the operation of the rail image comparison unit 23 will be described in more detail with reference to the flowchart of FIG. When the video signal output from the rail video estimation unit 21 is input to the rail video comparison unit 23, the condition of “START” in FIG. 6 is satisfied, and the flow proceeds to step S10. Note that the processing from “START” to “END” in FIG. 6 is processing for one cycle, and after the flow becomes “END”, when the condition of “START” is satisfied, the flow is resumed. Executed.

  In step S10, the rail image comparison unit 23 sets the pattern #n to be compared to the rail shape pattern # 1, and the flow proceeds to step S11.

  In step S11, the rail image comparison unit 23 compares the image signal received from the rail image estimation unit 21 with the pattern #n (here, n = 1) of the rail shape patterns stored in the storage unit 22. Then, the flow proceeds to step S12.

  In step S12, the rail image comparison unit 23 determines whether or not the comparison result in step S11 is “match”. If it is determined in step S12 that the comparison result is “match”, the flow proceeds to step S13. On the other hand, if it is determined in step S12 that the comparison result is not “match” (ie, “mismatch”), the flow proceeds to step S14.

  In step S13, the rail image comparison unit 23 outputs the comparison result “match” to the normality determination unit (not shown), and the rail shape pattern stored in the storage unit 22 when “match” is obtained. Are stored in the data storage unit 24, and the processing is terminated (END).

  In step S14, the rail image comparison unit 23 determines whether or not the comparison with all the rail shape patterns has been completed. That is, it is determined whether pattern #n is pattern # 7. If it is determined in step S14 that the comparison with all patterns has been completed, the flow proceeds to step S15. On the other hand, if it is determined in step S14 that the comparison with all patterns has not yet been completed, the flow proceeds to step S16.

  In step S15, the rail image comparison unit 23 outputs the comparison result indicating “mismatch” to the normality determination unit (not shown) and ends the process (END).

  In step S16, the rail image comparison unit 23 changes the pattern #n to the pattern # (n + 1) (here, the pattern # 1 is changed to the pattern # 2), and the flow returns to step S11.

  In this way, when the comparison result does not become “match” in step S12, the flow of step S11 → step S12 → step S14 → step S16 → step S11 is repeatedly executed for all patterns # 1 to # 7.

  The rail image extracted by the rail extraction unit 20L as described above is stored in the data storage unit 24.

  The camera devices 10L and 10R and the rail extraction units 20L and 20R are provided with two sets of left and right as shown in FIG. 1 in order to acquire a reference image and a corresponding image. FIG. 7 illustrates the reference image I (n) and the corresponding image J (n) on which the rails extracted by the rail extraction units 20L and 20R are displayed.

  The rail video data in the data storage unit 24 of each of the rail extraction units 20L and 20R is given to the parallax calculation unit 30 in order to calculate the rail parallax from the reference image and the corresponding image.

  As illustrated in FIG. 8, the parallax calculation unit 30 includes a primary corresponding point detection unit 31, a secondary corresponding point generation unit 32, a matching processing unit 33, a correlation value comparison unit 34, and a parallax reading unit 35. , Having a parallax calculation means 36 and a memory (not shown).

  The primary corresponding point detecting means 31 detects a point (primary corresponding point) on the same scanning line on the corresponding image having the same coordinates as the reference point of the rail image of the reference image I (n), and outputs the coordinates. The secondary corresponding point generating means 32 sets a point shifted leftward from the primary corresponding point on the same scanning line by a distance equal to “parallax by rail extraction” described later as a secondary corresponding point. The matching processing unit 33 performs a matching process from the secondary corresponding point to the left end of the corresponding image to obtain a correlation value.

“Parallax by rail extraction” is acquired using a known matching method during train travel in advance and is stored as described later. As shown in FIG. 9 (a), any coordinates of the reference point r rails RI of the reference image _{I (n) r (u r1} , v 1), the rail R of Ebipora line of the corresponding image J (n) _Assuming that the coordinates of the corresponding point s are s ( _{ur 2} , v ₁ ), the parallax between the reference point u and the corresponding point s, that is, the “parallax due to rail extraction” d _h ′ at the height h in the image is ,
d _h ′ = u _{r1 −ur 2}
Can be obtained from

The parallax decreases as the distance from the camera increases and increases as the distance from the camera increases. Therefore, as shown in a metaphorical manner in FIG. 9B, “parallax by rail extraction” d _h ′ (number of pixels) at each point in the longitudinal direction of the rails RI and RJ of the images I (n) and J (n). Indicates the distance from the camera installation position to each point (height h in the screen) 2,5,10,15 from the higher side to the lower side (from the far side). , 25, 30... “Parallax by rail extraction” having such a fixed pattern is stored in the memory of the parallax calculation unit 30 as a parallax model.

The parallax varies depending on the elevation angle of the camera. Therefore, the “parallax value by rail extraction” d _h ′ corresponding to the elevation angle of the camera is stored in the memory.

  The actions of the primary corresponding point detecting unit 31, the secondary corresponding point generating unit 32, and the matching processing unit 33 will be specifically described with reference to FIGS.

As shown in FIG. 10, the primary corresponding point calculation means 31 is a point on the corresponding image J (n) that has the same coordinates as the coordinates r ( _{ur 1} , v ₁ ) of the reference point r on the reference image I (n). r is obtained, and this point r is set as the primary corresponding point. Point second response point generation unit 32, a distance to the left by a distance equal to the primary "parallax by rail extraction" of the corresponding image J (n) on the corresponding point same Ebipora line as _{r d h '= u r1 -u} r2 Is a secondary corresponding point s corresponding to the reference point r. Subsequently, the matching processing unit 33 obtains a correlation value α between the coordinates (u _r1 , v ₁ ) of the reference point r and the coordinates (u _r2 , v ₁ ) of the secondary corresponding point s by matching processing.

FIG. 11 shows one block I _i1 and J _i1 on the same scanning line of the reference image I (n) and the corresponding image J (n). When searching for the corresponding point s (u _r2 , v ₁ ) corresponding to the reference point r (u _r1 , v ₁ ) of the block I _i1 of the reference image I (n), the block J _{i1 of the} corresponding image J (n) is searched. The same coordinate point r ′ (u _r1 , v ₁ ) as the reference point r (u _r1 , v ₁ ) on the same scanning line, that is, on the same scanning line from the primary corresponding point to the left by “parallax by rail extraction” d _h ′ The shifted point s ′ (u _r2 , v ₁ ) is set as a secondary corresponding point. Then, the correlation value α between the pixel value from the secondary corresponding point s ′ (u _r2 , v ₁ ) to the left end on the same scanning line of the block J _i1 and the pixel value on the same scanning line of the block I _i1 is obtained. A pixel value difference matrix similar to the pixel value difference matrix DT in FIG. 17E and a pixel value difference total matrix similar to the pixel value difference total matrix TT in FIG.

  The direction in which the “parallax by rail extraction” is shifted when searching for secondary corresponding points differs depending on which of the left and right images is used as a reference image and which is used as a corresponding image. FIGS. 10 and 11 are examples in which the left image is a reference image and the right image is a corresponding image. Therefore, “parallax by rail extraction” is shifted to the left, but the right image is a reference image, If the image is a corresponding image, it is shifted to the right.

  The correlation value α obtained by the matching processing by the matching processing means 33 is given to the correlation value comparison means 34. The correlation value comparison means 34 compares the correlation value α with a predetermined threshold value t. When the correlation value α satisfies a predetermined condition with respect to a predetermined threshold t, the first determination signal a is output and input to the parallax reading means 35. When the correlation value α does not satisfy the predetermined condition with respect to the predetermined threshold t, the second determination signal b is output and output to the parallax calculation unit 36.

  The calculation method of the correlation value differs depending on the matching processing method. When the matching processing method is block matching, the correlation value calculation method includes SSD (sum of squared difference) and SAD (sum of absolute difference). In these cases, the higher the matching is, the lower the correlation value (the higher the similarity). Therefore, in this case, the predetermined condition is satisfied when the calculated correlation value α is equal to or less than the threshold value t (α ≦ t), and when the calculated correlation value α exceeds the threshold value t (α> t), the predetermined condition is satisfied. Will not be satisfied. On the other hand, when the matching processing method is the phase only correlation method, the correlation value increases as the matching is achieved. Therefore, when the calculated correlation value α exceeds the threshold t (α> t), the predetermined condition is satisfied, and when the calculated correlation value α is equal to or less than the threshold t (α ≦ t), the predetermined condition is not satisfied. It becomes.

  When the first determination signal a is input, the parallax reading unit 35 reads the “parallax value by rail extraction” stored in the memory corresponding to the scanning line on which the matching processing has been performed. Then, the “parallax by rail extraction” is determined as the parallax between the reference point and the corresponding point in the scanning line, and is given to the distance measuring unit 4. That is, when the correlation value between the reference point and the corresponding point satisfies a predetermined condition, the coarse / fine search method that takes time for the calculation process is not executed. Therefore, the time for calculating the parallax is shortened by the amount that the coarse / fine search method is not executed.

  When the correlation value comparison unit 34 outputs the second determination signal b to the parallax calculation unit 36, the parallax calculation unit 36 applies the well-known coarse / fine search method to the scanning line on which the matching processing has been performed, and the parallax value And the obtained parallax value is given to the distance measuring unit 4.

  The distance measurement unit 4 uses a parallax output from the parallax reading unit 35 or a parallax output from the parallax calculation unit 36 to calculate the distance from the camera installation position to the gaze point (imaging target) of the rail ahead of the train. Measure by

<Obstacle detection>
In order to detect an obstacle on the track, the two cameras installed on the cab are attached so as to gaze at a rail surface 200 m ahead from the camera. When there are obstacles on the track, the obstacles are closer to the camera side than the camera's gazing point as the train progresses. FIG. 12 is a view of the photographing range of the camera as viewed from the side. (A) of the figure shows a state when there is no obstacle inside the gazing point Vp, and (b) shows a state when the obstacle m exists inside the gazing point Vp.

  FIG. 13 is a plan view of the imaging range of the camera of FIG. 12 as viewed from above. Assuming that there are two obstacles m1 and m2 inside the gazing point Vp, the image (basic image I (n)) acquired by the left camera 10L includes the gazing point Vp and the two obstacles m1 and m2. Since it exists on the common line of sight e1, as shown in FIG. 14A, it exists on the substantially common horizontal axis u on the right side of the image I (n), and on the vertical axis v, the gazing point Vp and the obstacle m1 , M2 is extracted at a position slightly separated by coordinates corresponding to the distance between m2 (the coordinates are close to each other).

  On the other hand, in the image (corresponding image J (n)) acquired by the right camera 10R in FIG. 13, the gazing point Vp and the two obstacles m1 and m2 exist on the lines of sight e2, e3, and e4, respectively. Therefore, as shown in FIG. 14 (b), the position on the horizontal axis u ′ and the position on the vertical axis v are different according to the distance between the gazing point Vp and the obstacles m1 and m2 (the coordinates are different). ing).

  Therefore, the reference image I (n) is overlaid on the corresponding image J (n) so that the centers of the screens coincide with each other, and the gazing point Vpl of the reference image I (n) and the obstacles m1l and m2l are overlapped with the corresponding image J (n ) When indicated by dotted lines above, the distance D1 between the left and right gazing points Vpl and Vpr, the distance D2 between the left and right obstacles m1l and m1r, and the distance D3 between the left and right obstacles m2l and m2r are parallaxes. D1 <D2 <D3. That is, the higher the height on the screen, the farther the distance from the camera, and the lower the height, the closer the distance from the camera. And it turns out that parallax is so large that distance is near.

  FIG. 15A shows left and right images I (n) and J (n) when an obstacle m exists on the track, and FIG. 15B shows an example of parallax calculated in that case. Show. When there is no obstacle on the near side of the camera's gazing point on the track, parallax with a pattern that matches the parallax model shown in FIG. 9B is obtained. However, as shown in FIG. 15 (a), when the obstacle m exists on the near side of the point of sight, the image corresponding to the obstacle m is a secondary corresponding point as shown in FIG. 15 (b). Since it extends to the left side of the screen, the parallax of the portion corresponding to the height of the obstacle on the screen is larger than the parallax before and after that. Therefore, it does not coincide with the parallax model shown in FIG.

  This mismatch with the parallax model is a second determination signal output when the correlation value comparison unit 34 outputs a correlation value α obtained by the matching processing by the matching processing unit 33 when the correlation value α does not satisfy a predetermined condition with respect to the predetermined threshold t. It is possible to detect based on b. Therefore, the obstacle presence / absence determination unit 2 can determine that an obstacle exists by being given the second determination signal b output from the correlation value comparison means 34.

  Further, the output signal c when the obstacle presence / absence determination unit 2 determines that an obstacle is present is given to an obstacle detection display unit 3 provided on a cab or the like. Thereby, the obstacle detection display unit 3 displays that an obstacle exists within a predetermined distance. The output signal c is also provided to the train automatic operation control device, and can be used for braking to avoid a collision with the obstacle.

A Obstacle detection device 1 Stereo image processing unit 2 Obstacle presence / absence determination unit 3 Obstacle detection display unit

Claims (2)

The process of imaging the front of the train with the left and right cameras,
A step of extracting rails from the acquired image;
A process of calculating the parallax of the left and right rails by performing a matching process on the extracted rails of the left and right images using “parallax by rail extraction”;
Comparing the calculated parallax with a predetermined parallax model when no obstacle is present;
As a result of the comparison, an obstacle detection method comprising: determining that there is no obstacle on the track if they match, and determining that there is an obstacle on the track if they do not match. A stereo image processing unit, an obstacle presence / absence determination unit, and an obstacle detection signal output unit;
The stereo image processing unit includes left and right camera devices installed at horizontal equidistant positions, a rail extraction unit that extracts rail images from image signals acquired by the camera devices, and an extracted rail. A parallax calculation unit that calculates a parallax by performing a matching process on a stereo image having a video of the above using “parallax by rail extraction”, and
The parallax calculating unit detects a primary corresponding point on the corresponding image having the same coordinates as the reference point of the rail image of the reference image, and outputs the coordinates, and the same scanning from the primary corresponding point A second corresponding point generating unit having a point corresponding to the line shifted to the left or right by a distance equal to “parallax by rail extraction” as a second corresponding point, and matching from the second corresponding point to the left end or right end of the corresponding image A matching processing means for performing a process to obtain a correlation value and a correlation value calculated by the matching processing means are compared with a predetermined threshold value, and when the correlation value does not satisfy a predetermined condition with respect to the predetermined threshold value, the first A correlation value comparing means for outputting a determination signal and outputting a second determination signal when the correlation value satisfies a predetermined condition with respect to a predetermined threshold; and the correlation value comparing means outputs the first determination signal. When When the parallax calculation means for calculating the parallax by executing a matching process by a known matching method and the correlation value comparison means output the second determination signal, the “parallax by rail extraction” stored in the memory unit , And parallax reading means for determining the “parallax by rail extraction” as the parallax of the reference point,
The obstacle presence / absence determination unit determines that an obstacle is present when the first determination signal is given, and determines that no obstacle exists when the second determination signal is given;
Obstacle detection device characterized by.

Images