TWI253998B - Method and apparatus for obstacle avoidance with camera vision - Google Patents

Abstract

The present invention relates to a method and an apparatus of operating an obstacle avoidance system with camera vision. The invention is used in day and night, and provides a strategy of obstacle avoidance without complicated fuzzy inference for safe driving. The method includes the following steps: capturing, analyzing plural images of an obstacle, positioning an image sensor, running an obstacle recognizing flow, obtaining an absolute velocity of a system carrier, obtaining a relative velocity and a relative distance of the system carrier with respect to the obstacle, and performing a strategy of obstacle avoidance.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an obstacle collision avoidance device and an implementation method thereof, and more particularly to an anti-collision device and method based on video perception, which is particularly suitable for use in a vehicle. . [Prior Art] Many academic research units in China are engaged in research on vehicle anti-collision, and the car collision warning system in the Intelligent Transportation Systems (ITS) integration plan of the National Chiao Tung University is For example, the principle is to measure the distance between vehicles using an ultrasonic sensor. In foreign countries, research on automotive-related safety systems has been in progress for a long time and has been integrated into the intelligent transportation system ITS in conjunction with other related information systems. At present, automatic anti-collision devices have been completed (Aut〇m〇tive Collision). Avoidance System (ACAS), the principle is to use infrared to measure the distance between the driver's own car and the car in front, and then calculate the relative speed between the two cars. Finally, the driver's interface is used to remind the driver to do the same. Safety measures. The establishment of the AC AS can be based on the sensor's access to environmental information, the use of captured images for vehicle identification and the establishment of anti-collision strategies to illustrate its system architecture. The function of the sensor is to capture the information of the external environment. The sensors used in relevant experiments at home and abroad, such as the ultrasonic wave proposed by Kawabata (the latest ultrasonic engineering), the radio waves and thunder proposed by Health Physics (Infrared) (International Commission 〇n Non-Ionizing Radiation Protection: Guidelines for limiting exposure to 96682 catching corrections. doc. 5 . 1253998 time-varying electric, magnetic and electromagnetic fields), Wann's proposed GPS three Position tracking and velocity estimation for mobile positioning systems, Camera calibration using geometric constraints, and the like. The characteristics of each sensor are shown in Table 1. Table 1 sensor characteristics sensor ultrasonic wave laser (infrared) satellite positioning CCD camera principle Doppler effect speed Doppler effect speed infrared effect speed GPS positioning image plane coordinates converted to three-dimensional space coordinates, smart The advantages of image recognition have no adverse effects on people, and are cheap and easy to implement. It can measure medium and long distances of about 100~200M. The measuring distance is long, up to 500~600M, and the measured value is accurate. It can provide the function of car navigation. With a measuring distance of up to 100M, it provides the most complete information on road conditions, including roadside line detection, front distance, and speed. Disadvantages The measurement distance is short, only about 0~10m. 完整 No complete road information is provided. 0 Electric S wave hazard. Miscellaneous> It is impossible to provide more complete road information than the image. It is more harmful to the human body (especially the human eye) than electromagnetic waves. Unable to provide complete road information. The price is too high, and the positioning error is about 10m. It is not effective to provide the car collision avoidance function. Other obstacles need to be installed at the same time to be positioned. It is affected by the weather light, but it can be handled properly by intelligent signal processing. Sensors common use occasions, reverse collision anti-collision system, self-propelled vehicle, vehicle anti-collision police speed detector, vehicle anti-collision police speed detector, vehicle anti-collision, star navigation process image detection, robot arm vision establishment, self-propelled The collision avoidance of vehicles and vehicles is shown in Table 1. Although the image captured by the CCD camera can provide the most complete road condition information, the disadvantage is that it is easily interfered by light and cannot be used for obstacle recognition at night. %682 added corrections.doc 1253998 At present, there are many ways to use images for vehicle identification at home and abroad, including A Method for Identifying Specific Vehicles Using Template Matching, and the three fronts proposed by Marmoiton. Position and relative speed estimation of vehicles by monocular vision, Preceding Vehicle Recognition Based on Learning From Sample Images, Kruger's optical flow (Real-time estimation and tracking) The optical flow vectors for obstacle detection (EMS-vision: recognition of intersections on unmarked road networks). The comparison of various image recognition vehicle methods is shown in Table 2. Table 2 Image Recognition Vehicle Method Using License Plate Recognition Three easy-to-identify orientations with known directions in front. Graphic recognition Vehicle image boundary combination Theoretical basis High-pass rate filter can identify license plate number; license plate size is consistent. Use the license plate pixel to determine the distance to the car. Ft uses the front three easy-to-recognize marks; and the three relative orientations are known to 0 to find the characteristic vector of the vehicle and perform neural network training. Use the image to distribute the boundaries of the vehicle. Application ➢ Parking lot management system Active safety driving assistance system steel plate detection; face recognition... Active safety driving assistance system algorithm High-pass rate wave two-point precise perspective method Neural network training using HCDFCPvI to make strong boundary search In the CPU computing resource usage degree, only a single CCD camera is used. In the image, only a single CCD camera is used for image high. When the neural network training is relatively low, only the color gradation data 96682 of the upper line segment of the image is added and corrected. D〇c 1253998 Input, processing a single image at a time, but the image processing of the entire image consumes a lot of CPU resources. Input, process a single image at a time, but image processing the entire image, which consumes a lot of CPU resources. π, the quality of the five training materials determines the quality of identification. (Up to 720 pixels). Parameters or information that must be determined in advance. The coefficient of the High-pass Filter. The relative coordinate information of the three preceding signs. The establishment of the template database; the establishment of a neural network. The distribution of vehicle boundaries in the image. The method is difficult and difficult, the background can not be too complicated, and the method is only suitable for establishing the totem database in 10 meters. It is difficult to find the car and the road totem with the quality and quantity to be effective. Although the vehicle boundary The distribution is not fixed, but it is quite different from other boundary combinations on the road. The measurable distance is less than 10 meters and can reach 100 meters or so, up to 100 meters, up to 100 meters. Accuracy is not high, high and high, high computational efficiency, medium and fast, low cost, medium cost, and government The road sign engineering of road vehicles is compatible with the high development of commercialization, and the time and labor costs are extremely high. Low-cost computing and hardware resources are less anti-collision reaction strategies are mainly to simulate the human response before the collision accident. Generally, humans can rely on experience and intuition by observing the distance and relative speed from the preceding vehicle. Make an appropriate response to avoid a collision accident. There are quite a few studies on the anti-collision response strategies proposed for domestic and international active driving safety systems. Among them, Mar J. proposed car-following collision prevention system (CFCPS) and An ANFIS controller for the car-following collision prevention system have obtained the anti-collision performance after comparing several related anti-collision reaction strategies. An excellent 96,826 supplemental amendment said the effect of .doc 1253998. CFCPS is based on the relative speed of the front and rear vehicles, the distance between the front and rear vehicles minus the safety distance, and the fuzzy inference engine based on 25 fuzzy rules is the core of calculation, and finally the basis of vehicle acceleration and deceleration is obtained. In addition, it is discussed that in order to make the vehicle safe and stable, the time required for the system is mentioned, CFCPS | takes 7~8 seconds, experiments of the same nature such as GM (General Motors) modelf 1 sec, Kikuchi Fu ^ _ _ 〇 The kiss model takes 12 to 14 seconds. SUMMARY OF THE INVENTION The main object of the present invention is to provide an anti-collision method and apparatus for all weather obstacles, which can perform obstacle recognition during day and night, and can be obtained without complicated fuzzy rule inference calculation. A set of anti-collision strategies for driving a system carrier as the basis for driving. Another object of the present invention is to provide an anti-collision method and device for all-weather obstacles, which can directly restore the positioning of the image sensor without being subjected to field measurement when the position of the image sensor is changed due to the impact of the system carrier. . In order to achieve the above object, the present invention discloses a video-aware obstacle collision avoidance method, which is applied to an obstacle and a moving system carrier, and an image sensor is mounted on the system carrier. . The obstacle tampering method comprises the following steps (4)~(1), wherein λ step (a) takes a plurality of images of the obstacle at the first and second moments and analyzes; step 0) locates the image. a sensor; step (4) is to perform an obstacle recognition process; step (4) is to obtain an absolute speed of the system carrier; step (4) is to obtain a relative distance between the system carrier and the obstacle and a relative speed; and f) Perform an anti-collision strategy. 96682 Supplementary Amendment states that .doc 1253998 is based on a video-aware obstacle anti-collision device

The warning device will emit a sound or a vibration to warn. The above-mentioned anti-collision method can be implemented in one system, and is installed in a system of cymbals and cymbals. [Embodiment] FIG. 1 is a view showing a collision avoidance device for visually impaired by the present invention. 2G' is mounted on the _system carrier 24. The anti-collision device includes an image sensor 22, an arithmetic unit 26, and a warning device 25. The image sensor 22 is operative to scan and capture a plurality of images of the obstacle at the first and second moments. The arithmetic unit 26 analyzes the plurality of images. If the result of the analysis is that an obstacle 21 is present, the alarm 25 will sound or emit a vibration to warn.圊 2 shows the flow of the video-aware obstacle collision avoidance method 1 Q of the present invention. The method includes the following steps 11 to 16, wherein the step is to capture a plurality of images and perform analysis, the step 12 is to locate the image sensor, the step 13 is to perform the Luyi obstacle recognition process, and the step 14 is to obtain the system carrier. Absolute speed, step 15 is to obtain a relative distance between the system carrier and the obstacle and a relative speed 'and step 16 is to implement a collision avoidance strategy. The details of the above steps are explained below: Step 11 captures and analyzes the plurality of images, which includes the following steps (refer to FIG. 3): (a) Measuring the depth distance 111 (ie, the system carrier 24 and the obstacle 21) The 96,826 supplementary correction says that the mountain-10- 1253998 relative distance): The imaging geometry of the depth distance measurement is shown in Figure 4. This figure includes two coordinate systems, two-dimensional image plane coordinates and three-dimensional real space coordinates ( Cry). The coordinate origin of the former is the image plane 5〇 center 〇, and the latter coordinate origin y is the physical geometric center of the image sensor 2 2 lens. The height of image sensor indicates "the vertical height to the ground, that is, f is the focal length of the image sensor 22. The optical axis 52 of the image sensor 22 is such that the intersection of the ray and the ground is C; the point Α is on a ray that is parallel to the ground and passes through the 〇w. If there is a target point D located at the distance of the front of the point F, and the corresponding point of the point D on the image plane is E. If / = z _ "fc , 0'=ZAOWC , Θ , B C0WD 2 ZEC^cy and 巧 = 〇 can get the following relationship A = tan -1 A (1) = fine - 丨 (△巧*(ci ))) (2) =tan-1 (+) (3) L == Hc (4) tan( θχ^θ2)

Pi^JL^x tan((tan -1 令一&)) (6) Δρ; L w 96682 supplementary correction says d〇c • 11 · 1 = Pi 御 (5) here image sensor 2 2 The focal length f is known, c is taken as half of the image vertical position (the c value of the image of 240*320 is 12〇), and the meaning of 义 and 々 can be obtained by actual measurement, which represents the position of the end of a straight road in the image. It is easy to be judged by the human eye through the rapid judgment of the image; 3 is also called the shadow degree of the 1253998 image sensor 22 (Depression Angle; DA), which is an important parameter affecting the coordinate mapping, equations (1) and (2) For the two simple image calibration methods, it can be deduced by 3 without additional measurement by the angle measuring instrument. The equation (3) can be obtained by image processing and equations (5) and (6), wherein the outer pixel length (pixe) [length) represents the amount of pixels occupied by FIG. 4, and is finely the pitch of pixels on the image plane. . The L obtained by the equation (4) is the true distance between the image sensor 22 and the front obstacle 2丨. The measurement involves the appeal of the hardware structure of the image sensor 22. Taking the CCD of the CCD camera as an example, the hardware structure is shown in FIG. 5. The resolution of the pixel is 64〇*48〇 (eight* of the photosensitive plate is negatively received to receive the external light color signal, and the length of the diagonal of the image sensor 22 is 〇5) is 1/3 inch, so the formula can be 7) Convert the pitch of the pixels to △ (cm). △A is clever, χ丄

Wy Py 1 2 1 3 (7) = Γϋ9·77χ10 · In addition, △8 can also be obtained from the image, and the formula (8) can be obtained according to the formula. 'A f ′ When the focal length f of the image sensor 22 is known, eight can be observed from Fig. 4, and A and Z can be obtained through actual measurement. Then it can be found. The different 可, different external can correspond to the no. 96682 supplementary correction said .doc •12- 1253998 the same kurt, so it is possible to take multiple points A to obtain multiple δα, and find the average of multiple △λ; or can use more The fine and / or simultaneous equations are solved. The experimental result is 8.3ixi〇-3m (cm), the accuracy is 85% 〇(b) The lateral distance is 1丨2: If the circle 4 should be The tile is extracted from the figure, and its internal geometric relationship is unchanged. See Figure 6 for a clearer explanation. Figure 6 shows the geometric relationship of μ lateral distance measurement, and the point D in the figure is negative. The K point is obtained by moving the W distance in the f direction, and the actual space coordinate _ position is (-%坼, the image of the K point on the image plane is G point, and the plane coordinate position is (-W, /).;; ; Vector; 5 represents the vector (2) and (10). λ 4 n. a (9) (10) Θι = cos —~

MM ^ =i/ccsc(^ +<92)tan^

(c) Measuring the height of the obstacle 113: Fig. 7 illustrates the height measurement method of the obstacle 21 with the vehicle as an embodiment. In the image range that can be formed by the vehicle, as shown in the square frame, the pixel length ^length of detecti〇, n window) can be obtained by the following formula (1)). In the equation (1)), [is half the value of the horizontal coordinate of the image], and for the image with the horizontal and vertical coordinates of the binary 320, the value of c is 240/2 = 120. The ordinate value of the tail of the vehicle in the image plane, this value is given by And the coordinates are incremented in order. The formula (1)) can be obtained by adding the correction of the formula 96682. doc -13- 1253998 (12), where the small height of the vehicle, the width of the vehicle, the _/^ is / the depth of the actual space point . Referring to the figure ~ (# does not, for different Lj, the same car will present a different L on the image. The image sensor 22 at this time is in a fixed state. L-p can be obtained by the formula (13), The depression angle of the image sensor 22 of the formula 0, A = ZCO-D = ZjE: 〇V (refer to Fig. 4). (11) (12) (13)

Idw = C + p丨’-i , combined with tandtan'^p^i))

Ρι 二< —P Ancient xtan(4) ΜβιΓ1^二浩 Hv>Hc i Hc τ ^2), if L· _ ρ = person tandA), if ze[c+, 239] Table 3 is four embodiments, wherein // K = n4cm, 4 = 18360111, i/c = l29cm, to prove that equations (11) ~ (13) are feasible. It can also be observed that the average error rate is about 7.21 ° /. That is, the accuracy rate is over 90%, and the explicit formulas (11) to (13) are practical. Table 3 Verification formula (11) ~ (13) Feasibility statistics table data experimental image tail image position i tail and image sensor depth I" (10) Equation (11) ~ (13) calculated L actual amount Measured L error % Idw 1 ών Figure 8(a) 38 6.8 135 140 3.57~ Figure 8 (b) 96 12.4 75 79 5.06 Figure 8(c) 130 23.4 40 44 9·09~ 圊8(d) 157 78.5 12 13.5 11.1Γ" 96682 Supplementary Amendment doc.doc -14- 1253998 Step 12 is to locate the image sensor, which includes the following steps (see Figure (a). First, the scanning line linei is from bottom to top. 3 to 5 meters for horizontal ^ scanning, assuming that the line P is found to find the point P of the road edge feature (located on the road center divider line 32), p, (on the road edge 3 1); (b) by p Point along the road center dividing line segment 32 on the left side of the figure to look up and down the two ends of the road center dividing line segment 32 (usually a white line segment) _, such as p pp p2, and accordingly form 丨ine3 and Une2, The other Pi', p2' are the intersections of line3 and line2 with the roadside line 31 on the right side of the figure; (4) find the intersection of two rays; (d) is substituted (2) Therefore, the depression angle θ of the image sensor 22 can be obtained; (e) According to FIG. 9 and (4), the equation (14) can be derived, wherein “the sum is the depth of the line 3 and the line 2 and the image sensor 22 respectively. Distance; another SM n, which is the basis of the definition, Ζα> defined by the same Lu ZCOw£> 〇

La La

Hc (14) tan(^ ^θ2) Η" tan(^ -f θ2) The formula (15) is obtained from the formula (14), where q is the length of a road segment Η __g 96682 supplementary correction. -15- 1253998 Θ (the depression angle of the image sensor 22) and 圪 (the height of the image sensor 22 to the ground) are determined, that is, the image sensor 22 has been positioned. The collision method and device can directly determine the depression angle and height of the image sensor by image analysis, so even if the position of the image sensor is changed due to the impact of the system carrier, it can be automatically repositioned without field measurement. Step 13 performs an obstacle recognition process, which includes the following steps (refer to FIG. 10): (a) setting a scan line pattern 13 1, the scan line pattern is selected from any of the following aspects, as shown in FIG. The image shown is shown in the box. Aspect 1: Single-line scan line, as shown in Figure 11 (a). Aspect 2: Zigzag scan line, as shown in Figure ,, its scanning method As follows: the range of the two side lines 33 is about a few meters in front of the position of the image sensor 22 The scanning width is determined according to requirements. The scanning line 40 scans the pixel points one by one from the bottom of the image in a zigzag manner, and scans the direction of each depth when the depth is about several meters. The distance of the advancement is also determined according to the demand. The second line scan line, as shown in the sub-picture (c) of FIG. 11, is a scan pattern of three linear scan lines 4〇, which includes a range approximately in front of the system carrier 24 where the image sensor 22 is located. The width is 1.5 times wider than the width of the system carrier 24. The distance from the scan line is also determined according to the requirements. Aspect 4: Five line scan lines, as shown in the sub-graph (d) of Figure n, .doc •16· 1253998 The range of the image is Η11 (4) The extension of the three line scan lines 40 and the two scan lines 4〇. Aspect 5: Turning scan line, as shown in the figure (4), and Figure U The biggest difference of the scan line 40 of the sub-picture (c) is that the scan line pattern adjustment increases the range of the left and right side scan lines 4 ,, and can be used as a scan line state when the vehicle turns. Aspect 6: Lateral scan Line, as shown in the figure (7). With the fourth aspect, it can detect obstacles in the opposite direction, the intersection of the intersection and the insertion after the overtaking, or the emergency stop.

The opposite obstacle is detected, so the night can be used as the basis for automatically adjusting the near-high beam and adjusting the speed of the vehicle, that is, when the distance between the obstacle and the system is measured, the distance is less than a set distance & In the case of a meter, f is adjusted to the near-light illumination, and vice versa to the high-light illumination. (b) An edge point identification 132 is provided, as detailed below: calculating the Euclidean distance of the adjacent pixels on the scan line in the gradation. If the image is a color image, and (8) indicates the Euclidean distance between the moxibustion and the moxibustion, then i?(8) is defined as 2+((+(,一...3. If this is greater than >c2, then the A: pixel is regarded as an edge point, where <, qA respectively represent the gradation value of the red, green and blue color of the moxibustion pixel, q is a critical constant 'can be set by the empirical value If the image is a black and white grayscale image, then W is defined as (mox) = 〇吼 + 1 - Gn%, and if the sum (8) is greater than > (: 3, then the A: pixel is considered An edge point, which indicates that the third image, 96682, is added to the correction. The .doc -17- 1253998 C3 is a critical constant. The scan mode can be selected from the gray scale value of any of the following square points, (C) setting A scanning method 133 type · (c.iM scanning interval type scanning method · from bottom to top scanning when finding the edge point 'fake false 兮 T1k ° and this point is the position of the rear of the car in the image In the setup, measurement interval, and further analysis of the pixel data of the scan line in the measurement interval, the detection interval length L will be when the object 21 is at the same depth distance from the image sensor. Figure 8 (4) ~ (4) is a car - for example, when the car at different depths of the distance will be different _ interval length L. The scanning line end point of this scanning method, for example, to identify the car, for the image of Figure 8 (4) The length of the detection interval formed when the image of the current car tail is at the time of i=0 (ie, the bottom of the image) is at the L (ie /~) of the 0. .2) Step-by-step scanning method··Scanning method scans the image pixel points step by step from bottom to top, and does not set the detection interval. The scanning end point is generally the image position of the end point of the road. Lu (d) provides two The true and false value of the Boolean variable is 134. The method is as follows: (cLl) The characteristic of the shadow at the bottom of the obstacle 21 is used because the three-dimensional object will produce a shadow, and the non-stereoscopic object such as the marking of the road surface cannot produce a shadow, so the shadow can be used as the resolution. The basis of the obstacle 21. Provide a Boolean variable a, then the true and false of & is determined by the formulas (16) and (17), if &kC4 is established, then a is true (16) 96682 supplementary amendment said .doc -18- 1253998 shadow pixel ~一^ <c4 established' then a is false (17) where L In order to detect the length of the interval, it refers to the amount of pixels conforming to the shadow feature. Usually, the pixel data of the bottom of the QAP interval is about q < long. C4 and C5 are a constant value. The shadow of the bottom of the vehicle (shadow_pixel) should be consistent with The relationship of equation (18): [R<CexRr shadow_pixel = < [Gray ^C7x Gray r · The symbols in equation (18) are explained as follows. When analyzing color images, the scales represent the red gradation values of the pixel data. 'Especially represents the red, green and blue gradation values of the gray road; when analyzing the black and white grayscale image, the gentry represents the gradation value of the pixel data, and W represents the gradation value of the road. In the color gradation value of the gray road, it is usually taken to take a pixel group that is more gray-like in the image, and the average color value of the pixel group is obtained. In addition, the color average value of the pixel group can be used to determine the brightness of the location of the system, and as the basis for automatically adjusting the brightness of the light, that is, the brighter the brightness of the day, the brightness of the light can be dimmed, and vice versa. The darker the brightness of the day, the brightness of the lights can be brightened. C6 and c7 are a constant value. (d·2) The light projected or reflected by the obstacle 21 has a characteristic of a luminance decreasing effect. Generally, when the weather is dark, most of them are image recognition at night, and the brightness can be used to determine the position of the obstacle in the image. Due to the multi-gradation distribution of the light, if the distribution of t»U〇c •19· 1253998 degrees is added as the basis of the identified obstacle by calculating the light 96682 supplementary correction, the calculation effect is consumed, and the found position is found. Nor is it a precise obstacle location. Here, a Boolean variable b is provided as a basis for judging whether it is the obstacle gate. Then the true and false of b is determined by the formula (9), and if it is true, b is true or false (19) On behalf of the analysis of color images, the red, green and blue primary colors of the pixel data are red as the main color gradation value, and the green or blue gradation value can also be referred to as appropriate; ~ less representative analysis of black and white grayscale images, pixel data Grayscale level value. Observed by multiple color or black-and-white gray-scale images, when the sum of the analyzed pixel groups or the gradation value of σ(6) is increased or decreased to q(10) (critical constant), it is generally the position of the obstacle in the image. . (4) The obstacle type 135 is judged, wherein the two Boolean variables relating to the obstacle shading characteristic, the obstacle projection or the dimming characteristic of the reflected light are denoted by a and b, respectively. The identification method of daytime identification and nighttime identification is different, and the time separation point between daytime and nighttime can be determined within the system time, including the following discriminating steps: φ (1) If a is true during daytime identification, the obstacle is identified as a car, a locomotive On the road, such as a bicycle, there are obstacles on the bottom of the pixel with dark colors; (11) If a is false during daytime identification, the obstacle is identified as a road marking '-tree shadow, a guardrail, a mountain wall , a house, a separate island, a person and other obstacles with no dark color at the bottom; 96682 supplementary correction said. doc -20· 1253998 (Hi) If b is true during night identification, the obstacle is identified as FAW Locomotives, a barrier, a mountain wall, a house, a separate island, a person and other steric obstacles; and (iv) if the nighttime identification is false, the obstacle is identified as a line or obstacle. 13(a), 13(b) and 13(c) include 17 sub-pictures (a) to (q), which exemplify the identification data by the identification rule described in the determination of the obstacle type 135. Here, the scanning line of the single-line type is used for scanning and identification, and the obstacles on the road are the main obstacle objects to be identified, the feasibility of the obstacle identification rule is verified, and the obtained experimental data is compiled in Table 4. The subgraphs (a) to (k) in 囷13(a), 13(b) and 13(c) are the obstacle identification data applied during the daytime, mainly using the Boolean variable a as the identification rule. The sub-囷(1)~(q) is the obstacle identification data applied to the nighttime, mainly based on the Boolean variable b as the identification rule. In the subgraph (aMq) in Figures 13(a), 13(b) and 13(c), the range scanned by the representative single-line scan line; L2 represents the default boundary threshold given by experience (here 25), where the coordinates greater than L2 are regarded as true boundaries, in the figure is the boundary line distributed to the left of L2. When identifying during the day, it is mainly based on the Boolean variables & It is the position of an obstacle that is judged to belong to a pixel with a dark color at the bottom of a steam locomotive, and is classified as a cockroach obstacle. L4 is the position of obstacles at the bottom of the pixel without dark colors, such as road markings, tree shadows, barriers, mountain walls, houses, islands, people and other obstacles, which are classified as 〇2 obstacles. . At night identification, it is mainly based on the Bulin change 96682 supplemental correction. doc • 21 · 1253998 number b for judgment, L5 is the position of locomotives such as steam locomotives, guardrails, mountain walls, houses, islands, people, etc. The steric obstacle has the function or characteristic of emitting or reflecting a light source, which is classified herein as a 〇3 type of obstacle. Table 4 Identification rules and figures for daytime and nighttime ------- Figure 13(a), 13(b), ^^- 13(c) subgraph (a) ~ Daytime identification rules and diagrams — Nshadow pixel Idw of formula (16), (17) (C4 is set to 〇·ι) Boolean variable a Identification result 0.416 True L3 is marked as 〇ι (8) (car; tree 〇 588 (car); True; L3 is marked as 〇ι; 影) ——_____ 〇 (tree shadow) -——_____ False L4 is marked as 〇 2 (C) (car; road 〇 · 612 (car); true; L3 is marked as 〇 1; Marking line) 〇 (tree shadow) False L4 is marked as 〇 2 (d) (locomotive; road 〇 · 313 (locomotive); true; L3 is marked as 〇 1; marking line) 〇 (road marking) false L4 Marked as 〇2 (e) (bicycle; road 0·24 (bicycle); true; ^ — L3 is marked as 〇1; surface marking) —---一丨丨.丨.一〇 (road marking ) a...丨丨_丨--L__ False L4 is marked as 〇2 (1) (barrier) 0 False L4 is marked as 〇2 (g) (mountain wall) 0 False L4 is marked as 〇2 (h) (house ) 0 False L4 is marked as (1) (separated island) 0 False L4 is marked as 〇 2 (1) (人) 0 False _------: L4 is marked as 〇2 — ——---—^ 96682 Supplementary amendment said.doc 1253998 (k) (car in grayscale image) ... ........—---___ 0.416 True L3 is marked as 〇1 Figure 13U), 13(b), 13(c) sub-graphs (1)~(q) Nighttime identification rules and identification Rule \Data experiment diagram \ R (R) of the formula (19) (C8, C9 are set to 200) Boolean variable b Identification result (1) (front car) 一丨丨丨丨···1_丨|丨丨•...,丨丨- 212 True L5 is marked as 〇3 (m) (front reverse car) 219 True L5 is marked as 〇3 (η) (person and locomotive) 207 True L5 is marked as ( 〇) (house) 205 True L5 is marked as 〇3 (Ρ) (car in grayscale image) 234 True L5 is marked as 〇3 (q) (front car; road marking) 209 (front car); 158 (Pavement marking) true; False L5 is marked as 〇3, not affected by the pavement markings by the above Table 4 and Figures 13(a), 13(b) and 13(c) subgraphs (a)~(q ) It is shown that using the Boolean variables a and b can accurately and stably identify all kinds of obstacles that may affect traffic safety. Thereof. Referring to Figure 9, step 14 is to obtain the absolute speed of the system carrier, as detailed below. (a) After finding out from the pi point in Fig. 9, the pi point is the end point of the central dividing line segment 32 of the road, and then looking for What about the next image? 1 point location. In the 96682 supplementary amendment, it is stated that .doc -23- 1253998 assumes that the road central dividing line segment 32 is a white line segment. (b) The pi point of the next image is usually closer, so the linel scan line in Figure 9 can be scanned laterally 3 to 5 meters downward, or the white line segment can be found down according to the slope in Figure 9. End point. (c) Comparing the two images before and after, the position change of the end point p丨 of the white line segment can be used to derive the actual moving distance, and the distance is the distance that the system carrier 24 of the image sensor 22 moves, and if The absolute velocity of the system carrier 24 can be derived from the difference in the time taken between the front and back images. In addition, the absolute speed of the system carrier 24 can also be obtained directly from the speedometer on the system carrier 24 via an analog-to-digital converter. Step 15 is to obtain a relative distance and a relative speed of the system carrier from the obstacle. The details are as follows: After the position of the obstacle 21 in the image is recognized, the relative distance L between the system carrier 24 and the obstacle 21 can be obtained according to the formulas (丨) to (6), as shown in the following formula (20). .

L tan (9) (20) wherein the height, the depression angle q, the focal length /, and the spacing between the pixels 4?/ of the image sensor 22 are known, and can be obtained from the image position of the vehicle. The relative velocity of the system carrier 24 to the obstacle 21 (Reiative Vel〇city; rV) can be obtained according to the following equation (21). (twenty one)

At Δί, ΔΖ (〇 each represents the time difference between the image before and after the image and the distance difference recognized by the vehicle. 96682 Supplementary amendment says doc -24-1253998 Step 16 is to implement a collision avoidance strategy, which includes the following steps ( Referring to Figure i2): (a) provides an equivalent speed 161. The equivalent speed is defined as the greater of the absolute speed of the system carrier 24 and the relative speed at which the system carrier 24 and the obstacle approach each other; (b) providing a safe distance 162. The safety distance is approximately between one thousandth and one thousandth plus 10 meters of the equivalent speed. A preferred embodiment The safety distance is defined as one-half of the equivalent speed value in kilometers per hour plus five, and the safety distance is in meters; (c) providing a safety factor 163 ^ The safety factor is defined as the ratio of the relative distance to the safety distance, and the magnitude of the safety factor is between 〇 and 1; (d) providing an alarm level 164. The alarm level is defined as 1 minus the safety factor. (e) issued Sound and light or vibrate 165 ^ The size of the alarm according to the degree of,

Using the warning device 25 to emit an acousto-optic or vibrating warning to the driver of the system carrier 24, and to audibly and visually alert the person around the system carrier 24; and (f) to provide an absolute speed 166, which is defined as The current load of the system carrier 24 is the product of the absolute speed and the safety factor. The anti-collision strategy further includes an instant recording function step. When the safety factor is less than a certain constant value, the function will be turned on, and the scene before the occurrence of the hazard is recorded. The embodiment described above is a car, but has an edge feature. The obstacle 96682 supplemental amendment says that d〇c •25· 1253998 can be identified by the method disclosed by the present invention, so the obstacles claimed in the present invention can include automobiles, locomotives, trucks, trucks, trains, people, dogs. , barriers, islands and houses. The above system carrier 24 is described by taking an automobile as an example, but the actual application is not limited to an automobile, that is, the system carrier 24 may be any type of vehicle such as a bicycle, a truck, or a freight car. In the above embodiments, any image capturing device can be used as the image sensor 22, so the image sensor 22 can be a charge coupled device (CCD) or a complementary metal oxide. Any device such as a semiconductor CMOS (CMOS) component camera, a digital camera, a single strip camera, or a digital camera on a handheld device. The technical contents and technical features of the present invention have been disclosed as above, and those skilled in the art can still make various substitutions and modifications without departing from the spirit and scope of the invention. Therefore, the scope of the present invention should be construed as being limited by the scope of the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic diagram of a video-aware obstacle collision avoidance device according to the present invention; FIG. 2 is a flow chart of a video-aware obstacle collision prevention method according to the present invention; Figure 4 is a flow chart for analyzing a plurality of image steps of the obstacle; Figure 4 is an imaging geometry of the depth distance measurement; Figure 5 is a schematic diagram of the hardware structure of the photosensitive plate; Figure 6 is an imaging geometry for measuring the lateral distance; Figure 7 shows the height measurement of the vehicle as an example. (The picture of the square frame is like the 96682 catching correction. The doc -26- 1253998 is the length D image; Figure 8 (4) ~ (4) is the vehicle in four different ways. Figure 9 is a schematic diagram of the image geometry of the image sensor; Figure 1 is a flow chart showing the steps of the obstacle recognition process; 圊11 Example 6 Figure 12 is a flow chart of the steps of performing a collision avoidance strategy of Figure 2; and Figures 13(a), 13(b) and 13(c) illustrate an experiment for obstacle recognition using Boolean variables Figure. [Key component symbol description] 10 Video-sensing Obstacle collision avoidance method 11~16 Step 20 Obstacle anti-collision device 21 with video perception Obstacle 22 Image sensor 24 System carrier 25 Warning device 26 Operation unit 31 Road edge 32 Road center separation line 33 Road line 40 Line drawing line 50 Image plane 52 Optical axis 111 Measuring depth distance 112 Measuring lateral distance 113 Measuring the height of the obstacle 131 Setting a scan line pattern 132 Providing an edge point identification 133 Setting a scan mode 96682 supplementary correction. Doc -27- 1253998 134 provides the true and false values of two Boolean variables 135. Determine the obstacle category 161 to provide an equivalent speed 162. Provide a safety distance 163. Provide a safety factor 164. Provide an alarm level 165 to emit sound or light or generate vibration. 166 Provides an absolute speed of 96,826 supplementary corrections. % doc -28-

Claims (1)

1253998 X. Patent application scope: 1. A video-aware obstacle collision prevention method, which is applied to a system carrier, and an image sensor is erected on the system carrier, and the collision avoidance method comprises the following steps: · capturing and analyzing a plurality of images; locating the image sensor; performing an obstacle recognition process; acquiring an absolute speed of the system carrier; obtaining a relative distance between the system carrier and the obstacle and a relative speed; and executing A collision avoidance strategy. 2. The video-aware obstacle collision avoidance method according to claim 1, wherein the step of positioning the image sensor is to obtain a depression angle of the image sensor and a distance between the image sensor and the ground. 3. The method according to claim 2, wherein the image sensor's depression angle and the distance between the image sensor and the ground comprise the following steps: Performing a horizontal scan on each interval; identifying a feature point having a feature of the roadside edge; identifying two first end points of a feature line segment where the feature point is located; and horizontally scanning the two first end points to obtain two horizontal lines, the second The horizontal lines are respectively assigned to the other characteristic line segments at the two second end points; the intersection of the two first end point lines and the two second end point lines is identified; the depression angle of the image sensor is obtained; and the 96682 supplementary correction is obtained It is said that .doc 39c finds the distance of the image sensor to the ground. According to the method 3 of the video-aware obstacle collision avoidance method, wherein the image sensor has a depression angle using a distance between pixels on the image, a value of half of a longitudinal length of the image, a focal length of the image sensor, and Get it at the intersection. 6. The video-aware obstacle collision avoidance method according to claim 3, wherein the distance from the image sensor to the ground is obtained by using a depression angle of the image sensor and a depth distance between the two horizontal lines and the image sensor. According to claim 3, the image-aware obstacle collision avoidance method, wherein the image sensor's depression angle is obtained according to the following formula: - wherein Θ is the depression angle of the image sensor; Μ is the pixel on the image plane The spacing; e is the value of half the longitudinal length of the image; the position of the intersection; and / is the focal length of the image sensor. 'According to the obstacle-obstacle-based obstacle collision avoidance method, the distance between the current sensor and the ground is obtained according to the following formula: Hc:——;—~^~~ a tan (4)+θ2 The tai is the distance from the image sensor to the ground, q is the length of a road segment; 0 is the depression angle of the image sensor; and &amp; Level 96682 supplemental correction says doc 1253998 line to the depth distance of the image sensor. 8. The video-aware obstacle collision avoidance method according to claim 1, wherein the obstacle recognition process comprises the following steps: a scan line state of the scan line selected from: a single-line scan line zigzag scan Line, two line scan lines, five line scan lines, hip type scan lines and horizontal type scan lines; provide an edge point identification; set a scan mode, the scan mode is to detect interval or stepwise; provide at least two One of the Boolean variables, the two Boolean variables are divided into the shadow characteristic of the obstacle, the projection of the obstacle or the brightness of the reflected light; determining the true and false value of the Boolean variable; and determining the type of the obstacle . 9. The video-aware obstacle collision avoidance method according to claim 8, wherein the edge point clock comprises the following steps: calculating one of the pixels on the horizontal scan line and its neighboring pixels in the color gradation De distance; and if the Euclidean distance is greater than a critical constant, the pixel is considered to be an edge point. 10. The method for preventing obstacle collision by video perception according to claim 8, wherein the true and false values of the Boolean variables of the shadow characteristic of the obstacle are judged by the following formula: If ^^L Μ* is established, the Bollinger The variable is true; if ^^ <A is established, the Boolean variable is false; 96682 supplementary correction says d〇c 1253998 where is a constant value; 匕 is the length of the detection interval; and Nshadow a pixel is the shadow The amount of pixels of the feature. 11. The method according to claim 8, wherein the true value of the Boolean variable of the diminishing characteristic of the obstacle projection or reflected light is determined by the following formula: 'If or not, the cloth The forest variable is true, otherwise it is false; 〇8 and C9 are critical constants; Λ represents the red, green and blue primary colors of the pixel data in the analysis of color images _ with red as the main color gradation value, Gr noisy represents the analysis of black and white images When the grayscale level value of the pixel data. 12. According to the item 8 of the obstacle-observing obstacles of video perception, which additionally includes the conversion procedure of the daytime and nighttime identification rules, wherein the daytime identification rule is the Boolean variable using the shadow characteristic of the obstacles. The law is a Boolean variable that uses a brightness reduction characteristic of an obstacle projection or reflected light, and the conversion time of the conversion step is determined by a video unit that is set on the system carrier unit 13' according to the clearing item 8 An obstacle collision avoidance method, wherein if the true or false value of the Boolean variable of the obstacle shadow characteristic is true, the obstacle is an object having a dark color pixel at the bottom, otherwise the obstacle is discriminated &quot;· is a bottom without dark Color pixel object. 14. Obstacle-sensing obstacle according to the clearing item 8: Bolling change of the brightness decreasing characteristic of the projected or reflected light: Right one, the obstacle is recognized as a steric obstacle, otherwise the obstacle 96682 In the supplementary amendment, d〇c 1253998 was identified as an obstacle-free object. 15 16. 17. 18. • Obstacle-sensing obstacle collision avoidance method according to Item 8 _ - The automatic switching procedure of the near-far light is calculated by continuation. Whether the distance of the straight spoon is less than - a specific distance is used as a basis for switching. According to the request 8 of the video-aware obstacle collision avoidance, / /, with the additional - the lamp brightness automatic adjustment step, by taking the road pixels and their color gradation average, can determine the location of the system carrier The brightness of the weather' is used as the basis for automatically adjusting the brightness of the lights. According to the method of claim i, the method for obscuring the obstacle collision avoidance, wherein the obtaining of the absolute speed of the system carrier comprises the following steps: identifying a position of one of the characteristic line segments in a first image; The endpoint is located at a second image; and the distance between the two endpoints is divided by the time difference between the first and second images; wherein the first and second images are included in the plurality of images, and the second The capture of the image is later than the capture of the first image. The video-aware obstacle collision avoidance method according to claim 1, wherein the anti-collision strategy comprises the steps of: providing an equivalent speed selected from the absolute speed and the greater of the relative speed; providing a safe distance; A safety factor is provided, the size of which is defined as the ratio of the relative distance to the safety distance 'and the magnitude of the safety factor is between 〇 and 1; an alarm level is provided, the size of which is defined as 1 minus the safety factor; 96682 supplement According to the amendment, doc 1253998 warns the driver of the system carrier by sound and light or vibration to warn the person around the carrier of the system according to the magnitude of the alarm; provide an absolute speed, the absolute speed Defined as the product of the current absolute speed of the system and the safety factor; and provides a video function. 19. The video-aware obstacle collision avoidance method of claim 18, wherein the recording function is turned on when the safety factor is less than a certain constant value. 20. A method of video-aware obstacle collision avoidance according to claim 1, wherein the absolute speed of the system carrier is obtained directly from a speedometer of the system carrier. According to claim 1, the image sensing device is selected from one of the following: a charge coupled device as a camera, a complementary metal oxide semiconductor device as a camera, and a camera. A single strip camera and a camera on a handheld communication device. 22. The method of claim 1 for video-aware obstacle collision avoidance, wherein the system carrier is a vehicle. A video-aware obstacle collision avoidance device for use in a system carrier, comprising: Λ an image sensor for capturing a plurality of images of an obstacle; and an arithmetic unit, It includes the following functions: (a) analyzing the plurality of images; (b) performing an obstacle identification process based on the analysis results of the plurality of images to determine whether the obstacle exists; and (c) performing an anti-collision strategy. </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> </ RTI> <RTIgt; Sound and light or vibration. 2 5 · A video-aware obstacle anti-collision device according to claim 2, wherein the image sensor is selected from one of the following: a charge coupled device camera, a complementary metal oxide semiconductor device camera, A single strip camera and a camera on a handheld communication device. 26. A video-aware obstacle avoidance device according to claim 23, wherein the system carrier is a vehicle. 96682 Supplementary amendment said. d〇c 1253998 VII. Designated representative map: (1) The representative representative of the case is: (2). (2) Simple description of the symbol of the representative figure: 10 Obstacle avoidance method by video perception 11~16 Step 8. If there is a chemical formula in this case, please disclose the chemical formula that best shows the characteristics of the invention: (none) 96682 supplementary correction Said in the .doc