CN1987357B - Intelligent parking auxiliary device based on omnibearing computer sight - Google Patents

Abstract

The auxiliary device includes vision sensor, microprocessor in use for carrying out aided decision making for parking, and display unit in use for displaying situation around vehicle. Being connected to microprocessor, the vision sensor is all-directional vision sensor. The microprocessor includes image display unit, scaling module for sensor, image expansion module, module for detecting edges of obstructers around vehicle, module for calculating minimum distance to edges of obstructers around vehicle, module for alarming obstructers, and back running knowledge base etc. The invention provides visual images around vehicle, gives hint of turning angle of steering wheel to driver so as to assistant procedure for driver to run back vehicle.

Description

Intelligent parking auxiliary device based on omnidirectional computer vision

(1) technical field

The present invention relates to omnibearing vision sensor technology, image recognition technology, the application of Computer Control Technology aspect vehicle parking, especially a kind of intelligent parking auxiliary device.

(2) background technology

Along with the development of China's automobile industry and improving constantly of people's living standard, the quantity of automobile increases year by year on the one hand, and make that with 7,000,000 speed increments highway, street and the parking lot of China are crowded to capacity every year, and rotating space is fewer and feweri; On the other hand, the growth of vehicle absolute number has caused fresh driver and non-full-time driver to get more and more, and the dispute that causes because of reversing gets more and more.In the generation ratio of motor-vehicle accident in 2002, the accident that causes of reversing accounts for 21%, reversing become make us the task of having a headache, even veteran driver is the thing that part is taken great energy in the complaint reversing also.The awkward situation that improving moves backward runs into is paid close attention to by increasing people, people have higher requirement to the convenience of automobile operation, wishing has kind of device can solve the inconvenience that reversing automobile brings to people, eliminate the unsafe factor in driving, car can be docked to appointed positions rapidly and accurately.

The difficulty that exists in the reversing automobile, the first come from the driver and are sitting in and can't understand around the automobile the particularly environment at rear on the driver's seat fully.The driver can only rely on rearview mirror and observe rear obstacle, and rearview mirror is subjected to the restriction of its position, has tunnel vision, poor definition, can't reach the purpose of backsight in the reversing at all.In its 2 when reversing,, the driver will take into account the situation of vehicle front, both sides simultaneously, must twisting later observes the situation of vehicle back, and muscle power and mental consumption are excessive, easily produce unsafe factor.Its three reversing is a complex engineering, and it depends on driver's driving experience, the driving skills and the degree that is quick on the draw, and arbitrary link goes wrong and all causes the driver can't finish the reversing task rapidly and accurately.

Chinese invention patent 00100982.6 discloses a kind of astern radar device with multichannel display function.This scheme adopts and sets up ultrasonic sensing device and video camera separately in tailstock appropriate location, the two is connected to a radar for backing car main frame and Flame Image Process module respectively, the radar for backing car main frame also with Flame Image Process module line, Flame Image Process module output terminal then is connected with the display that is positioned at the driver's seat place.During reversing, the driver at first will take the photograph people's image judgement car and obstruction distance by video camera, and during near barrier, the ultrasonic sensing device also receives reflected signal simultaneously, convert communications codes to through radar host computer and deliver to the Flame Image Process module, the Azimuth ﹠ Range of barrier also is shown on the display.Chinese invention patent number 00101620.2 discloses a kind of radar for backing car detecting device with multiple Presentation Function, it comprises at least one group of above radar for backing car, position range display and microprocessor, and this position range display includes a decoding circuit, two blinkers and a numeric type display circuit; This microprocessor is that the signal that set each radar for backing car of the tailstock is sent into is encoded, and deliver to described position range display, after the decoding circuit decoding, except that but the direction of passage pilot lamp shows the orientation of car and obstruction, this numeric type display circuit also can show the concrete distance of the tailstock at a distance of barrier, grasps the distance of the tailstock and barrier when being beneficial to drive reversing really.Chinese invention patent 02208981.0 discloses a kind of reversing automobile monitor and alarm unit, constitute by display screen, alarm, digital display tube, camera and range finder, it is characterized in that described range finder is electrically connected with alarm and digital display tube, and be electrically connected with the power supply of reversing automobile lamp, described digital display tube is arranged on the display screen, described camera is electrically connected with display screen, has solved automobile can't be understood the tailstock and obstacle distance and which kind of barrier when reversing a difficult problem.Above-mentioned several solution, by the ultrasonic sensing device, radar detector comes the inspection vehicle obstruction, confirm distance between car and obstruction and which kind of barrier with camera, the generation of these solutions minimizing accident to a certain extent, but, adopt ultrasonic listening or radar detector, the circuit structure complexity, while intuitive and poor reliability, the inconvenience that can cause the driver to use, and adopt camera to confirm that distance between car and obstruction and which kind of barrier still need the driver to confirm with vision, just helped the driver can understand the environment at automobile rear, and because this camera is to make a video recording at rear view of vehicle, therefore taken is the angular field of view of rear view of vehicle only, can't confirm the barrier of the right and left and the stop line of the back lower place by video image.Chinese invention patent 200580000399.2 discloses a kind of parking aid, is used to catch the camera of the image of vehicle back; Arrange near the display of the pilot set of vehicle; Be used to detect the steering angle sensor of steering angle; Be used to detect the crab angle pick-up unit of the crab angle of vehicle; Be used for issuing the guiding device of driver behavior guidance information to the driver; And controller, its be used for when vehicle when initial stop position travels forward, the crab angle of the vehicle that detects based on the steering angle that detects by steering angle sensor and crab angle pick-up unit is obtained the instantaneous position of vehicle and the instantaneous crab angle of vehicle, in case be used for vehicle entered by stop to allow the position and the zone that allows the starting position of moving backward that crab angle limits of stopping in or entered this zone after a period of time at vehicle, stop vehicle by guiding device issue guidance information, be used for when vehicle moves backward, on display, showing the image of catching by camera, and be used at vehicle under the situation that the current steering angle that maintenance detects by steering angle sensor moves backward, the vehicle space that on display, shows estimation in the mode of stack, the vehicle space of estimating is according to current steering angle motion, to inform driver's suitable steering angle of moving vehicle backward thus.

(3) summary of the invention

For the visual pattern of existing parking aid around can not providing being provided, the deficiency of reversing aid decision making can not effectively being provided, the invention provides a kind of visual pattern that vehicle's surroundings can be provided, can point out the corner of driver's operation bearing circle, the effective intelligent parking auxiliary device based on omnidirectional computer vision of assisting car backing process.

The technical solution adopted for the present invention to solve the technical problems is:

A kind of intelligent parking auxiliary device based on omnidirectional computer vision, comprise vision sensor, the microprocessor of aid decision making and the display unit that is used to show the vehicle periphery situation are used to stop, described vision sensor connects microprocessor, described microprocessor comprises: image-display units, be used for the signal of acquisition vision sensor, and output to display unit; Described vision sensor is an omnibearing vision sensor, this omnibearing vision sensor is installed in the top in the middle of the vehicle, described omnibearing vision sensor comprises the evagination catadioptric minute surface that is used for reflecting vehicle periphery field object, evagination catadioptric minute surface down, be used to prevent anaclasis and the saturated dark circles cone of light, the dark circles cone is fixed on the center of catadioptric minute surface male part, be used to support the transparent cylinder of evagination catadioptric minute surface, be used to take the camera of imaging body on the evagination mirror surface, camera facing to the evagination mirror surface up; Described microprocessor also comprises: the transducer calibration module is used to set up the pavement image in space and the corresponding relation of the video image that is obtained; The image stretching module is used for the image of omnibearing vision sensor collection is launched, and the circular video image that reads is expanded into the panorama histogram; Vehicle periphery barrier rim detection module, be used to adopt the Suo Beier operator to carry out rim detection, adopt the template of setting size, compute gradient between each template, the tonsure value of selection calculating as marginal point, connects into marginal position with each marginal point greater than the point of gradient magnitude in image; Vehicle periphery barrier edge minimum distance computing module, be used at left and right directions, with the length of vehicle as detection line, add alarm distance detection line to start with from width from vehicle, outwards detect from left and right directions respectively, when the edge line of running into detection of straight lines and barrier when detection line has the pixel of predetermined number to intersect, judge that this place is barrier and the minimum distance of vehicle on left and right directions; On fore-and-aft direction, with the width of vehicle as detection line, add alarm distance detection line to start with from length from vehicle, outwards detect from fore-and-aft direction respectively, when the edge line of running into detection of straight lines and barrier when detection line has the pixel of predetermined number to intersect, judge that this place is barrier and the minimum distance of vehicle on fore-and-aft direction;

The barrier alarm module is used to judge that minimum distance whether less than default lower limit, as less than this lower limit, sends alarm command to alarm device; The reversing knowledge base is used to deposit driver's reversing skill, technical ability, and the backing track line between various current location and the target location, calls during for generation backing track line; The parking spot designated module is used for the user and wishes the position that parks cars by the input of user interface, utilize omnibearing vision sensor resulting around image information, user select target position on touch-screen on the image information of this demonstration; Backing track line generation module, be used for detecting the current vehicles position by omnibearing vision sensor, the user is by the input target location of user interface, automatically generate a backing track line, make corresponding reversing envelope according to this backing track line, described reversing envelope is greater than the vehicle minimum path circular arc curvature of can travelling, and judges whether the reversing envelope and the image border of barrier intersect, as just definite this backing track line does not take place to intersect; Otherwise, to the intersection backing track line of making amendment takes place; Steering wheel angle action computing module is used for according to the backing track line that is generated, and obtains destination path on the key point of this backing track line angle ψ that travels _P, the destination path angle ψ that travels _PWith steering wheel angle δ _hRelation with formula (22) expression, ψ _P=tan ^-1(Y _P/ X _P) ≈ Y _P/ X _P=K _s* δ _h* t _p/ 2 (22)

In the following formula, Be a constant, estimate the time t when under this vehicle speed condition, arriving next one reversing key point _pAnd with this steering wheel angle δ _hOutput to display unit;

Backing track line correction module, being used for recomputating when vehicle departs from the backing track line backing track line rectifies a deviation, whether the key point of detection on the backing track line skew has taken place, if the off-set value that detects surpasses default scope, regenerate the backing track line according to the position of current vehicle.

Further, described catadioptric minute surface is in order to access the corresponding point with the space object coordinate, and the catadioptric minute surface adopts hyperbolic mirror to design: shown in the optical system that constitutes of hyperbolic mirror can represent by following 5 equatioies;

((X ²+Y ²)/a ²)-(Z ²/b ²)＝-1(Z＞0) (1)

$c=\sqrt{a^2+b^2}---\left(2\right)$

β＝tan ^-1(Y/X) (3)

α＝tan ^-1[(b ²+c ²)sinγ-2bc]/(b ²+c ²)cosγ (4)

$\mathrm{\&gamma;}={\tan }^{-1}[f/\sqrt{({\mathrm{<figure-callout\; id="2"\; label="X"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">X</figure-callout>}}^2+Y^2)}]---\left(5\right)$

X in the formula, Y, Z representation space coordinate, c represents the focus of hyperbolic mirror, and 2c represents two distances between the focus, a, b is respectively the real axis of hyperbolic mirror and the length of the imaginary axis, β represents the angle-position angle of incident ray on the XY plane, and α represents the angle-angle of depression of incident ray on the XZ plane, and f represents the distance of imaging plane to the virtual focus of hyperbolic mirror;

The setting height(from bottom) of omnibearing vision sensor is that the situation according to the angle of depression of omnibearing vision sensor and automobile body decides, omnibearing vision sensor can be monitored object beyond 0.35 meter on vehicle, and with the position of its omnibearing vision sensor index as setting height(from bottom); The XY plane of described omnibearing vision sensor is parallel with ground level;

According to the characteristics of the hyperboloid in omnibearing vision sensor folding transmitting mirror, by subpoint P (x, y) calculate with respect to spatially P of the barrier of vehicle center point (X, Y, Z) point, computing formula is provided by formula (11):

$x=f((b^2-c^2)X/((b^2+c^2)Z-2\mathrm{bc}\sqrt{X^2+{\mathrm{<figure-callout\; id="2"\; label="Y"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">Y</figure-callout>}}^2+Z^2})---\left(11\right)$

$y=f((b^2-c^2)Y/((b^2+c^2)Z-2\mathrm{bc}\sqrt{X^2+{\mathrm{<figure-callout\; id="2"\; label="Y"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">Y</figure-callout>}}^2+Z^2})$

Formula (11) has represented that (Z) ((x, value y) can obtain this point at P (X, Y, Z) point to the subpoint P on imaging plane to the P on the space therefore to obtain P from imaging plane for x, relation y) for X, Y.

Further again, visual angle, described the place ahead, visual angle, rear, left visual angle and right-hand visual angle are evenly cut apart, and show the video information of level 160 degree on each visual direction.

Further, described image stretching module comprises: read the coordinate information unit, be used to read the centre coordinate of circular omnidirectional images and the inside and outside circle radius of image; The approximate expansion computing unit is used for the initial point O with the centre coordinate setting plane coordinate system of circular omnidirectional images ^*(0,0), X ^*Axle, Y ^*Axle, the internal diameter of image is r, external diameter is R, radius of a circle: r in the middle of setting ₁=(r+R)/2, the position angle is: β=tan ^-1(y ^*/ x ^*); The rectangle cylinder panoramic image is with true origin O ^*(0,0), X ^*Axle, Y ^*Axle is a plane coordinate system, is r and X with the internal diameter in the circular omnidirectional images ^*The intersection point (r, 0) of axle is as true origin O ^*Counterclockwise launch with azimuthal angle beta (0,0); Set up any some pixel coordinate P in the rectangle cylinder panoramic image ^*(x ^*, y ^*) with circular omnidirectional images in pixel coordinate Q ^*(x ^*, y ^*) corresponding relation, its calculating formula is:

x ^*＝y ^*/(tan(360x ^**/π(R+r))) (8)

y ^*＝(y ^**+r)cosβ (9)

In the following formula, x ^*, y ^*Be the pixel coordinate value of rectangle cylinder panoramic image, x ^*, y ^*Be the pixel coordinate value of circular omnidirectional images, R is the external diameter of circular omnidirectional images, and r is the internal diameter of circular omnidirectional images, and β is the position angle of circular omnidirectional images coordinate.

The image stretching module comprises: read the coordinate information unit, be used to read the centre coordinate of circular omnidirectional images and the inside and outside circle radius of image; Mapping matrix launches the unit, is used for the centre coordinate of circular omnidirectional images is set the initial point O of plane coordinate system ^*(0,0), X ^*Axle, Y ^*Axle, the internal diameter of image is r, and external diameter is R, and the position angle is: β=tan ^-1(y ^*/ x ^*); The rectangle cylinder panoramic image is with true origin O ^*(0,0), X ^*Axle, Y ^*Axle is a plane coordinate system, is r and X with the internal diameter in the circular omnidirectional images ^*The intersection point (r, 0) of axle is as true origin O ^*Counterclockwise launch with azimuthal angle beta (0,0); According to any some pixel coordinate Q in the circular omnidirectional images ^*(x ^*, y ^*) with the rectangle cylinder panoramic image in pixel coordinate P ^*(x ^*, y ^*) corresponding relation, set up from Q ^*(x ^*, y ^*) to P ^*(x ^*, y ^*) the mapping matrix corresponding relation, its calculating formula is:

P ^**(x ^**，y ^**)←M×Q ^*(x ^*，y ^*) (10)

In the following formula, Q ^*(x ^*, y ^*) be the matrix of each pixel coordinate on the omnidirectional images, M is the corresponding relation matrix from the omnidirectional images coordinate to rectangle cylinder panoramic image coordinate, P ^*Matrix for each pixel coordinate on the rectangle cylinder panoramic image.

But the no doubt important understanding of the environment automobile of in reversing process, understanding the rear aspect other (front, the left and right sides) also be necessary; The situation of vehicle front and back, both sides be to take into account simultaneously, greatly muscle power and mental paying needed; More satisfactory angular field of view is to be environment around all of center by looking down with the vehicle above vehicle; Simultaneously driver's driving experience, driving skills and the degree that is quick on the draw also depend on the key that reversing is whether successful.We consider these three aspects as the reversing three elements.

Thereby need a kind of make easily the driver can be on display screen the intended target parking spot, and can put automatic generation backing track line according to the current position and the target parking position of vehicle, the intelligent parking auxiliary device of the corner of prompting pilot control bearing circle in reversing process.

The present invention can provide the aid decision making of reversing prompting, distance alarm and reversing for the driver, and comprise in the device: omnibearing vision sensor is used to detect the barrier of vehicle periphery and obtains parking spot information on video; Target parking position installs order unit, is used to set the initial position that the target parking position that parks cars is put; Vehicle periphery situation display unit, be used to show vehicle visual angle, the place ahead, visual angle, rear, left visual angle and right-hand visual angle video image and with the nearest obstacle distance at these visual angles; Backing track line generation unit is used for according to the current position of reversing knowledge base, vehicle and parks the target location generating the backing track line automatically; Steering wheel angle action computing unit is used for determining corner at certain bearing circle on a bit on this trajectory according to the backing track line computation that is generated; Backing track line correction unit is used for recomputating the backing track line and rectifies a deviation when vehicle departs from the backing track line; The nearest detection of obstacles of vehicle unit is used for detecting and the nearest obstacle distance of vehicle according to the present situation of vehicle, as minimum distance employing alarm driver during less than some values; The reversing knowledge base is used to deposit outstanding driver's some reversing skills, technical ability, so that call when generating the backing track line.

Car backing operation for the driver, after car is hung the reverse gear position, device provides: (1) all is presented at the image information of vehicle periphery in the car on the display screen, the shown video information of this display screen is divided into four the small screen such as visual angle, the place ahead, visual angle, rear, left visual angle and right-hand visual angle, each view directions all detects the minimum distance with barrier, realize anticollision warning function-at vehicle apart from obstacle distance during less than 0.60 meter, the flicker of minimum distance value, font shown on the corresponding view directions become redness; When distance is used auditory tone cues during less than 0.35 meter; (2) minimum spacing shows-shows that minimum spacing is to remind the driver on display screen; (3) operation indicating-according to the corner that calculates resulting backing track line computation front-wheel and the corner of bearing circle, and provide the operation indication to the driver corner of bearing circle, the driver finishes the reversing task according to the corner of suggested bearing circle.

Beneficial effect of the present invention mainly shows: 1, omnibearing vision sensor can provide the video information of vehicle's surroundings for the driver; 2, can point out the corner of driver's operation bearing circle, effective assisting car backing process.

(4) description of drawings

Fig. 1 is based on the hardware configuration schematic diagram of the intelligent car-backing device of omnidirectional computer vision;

Intelligent car-backing device the place ahead screen display that Fig. 2 is based on omnidirectional computer vision is the synoptic diagram of video vision all around;

The intelligent car-backing device that Fig. 3 is based on omnidirectional computer vision install and all around the video vision cut apart synoptic diagram;

Fig. 4 be in the video Visual Display of the place ahead with about the area schematic that repeats mutually;

Fig. 5 is the schematic diagram of omnibearing vision device catadioptric imaging;

Fig. 6 is vehicle and the minimum distance of barrier and the visual range synoptic diagram that omni-directional visual is blocked by vehicle itself in the reversing process;

Fig. 7 is definite synoptic diagram of backing track and target parking position;

Fig. 8 is vehicle backing aid decision making control principle figure;

Fig. 9 is based on the intelligent car-backing device detection of obstacles and the reversing aid decision making control flow chart of omnidirectional computer vision;

Figure 10 is the synoptic diagram that a plurality of circular arcs come the match backing track;

The illustraton of model of vehicle when Figure 11 is reversing.

(5) embodiment

Below in conjunction with accompanying drawing the present invention is further described.

Embodiment 1

With reference to Fig. 1～Fig. 9, a kind of intelligent parking auxiliary device based on omnidirectional computer vision, comprise vision sensor 1, the microprocessor 7 of aid decision making and the display unit 8 that is used to show the vehicle periphery situation are used to stop, described vision sensor connects microprocessor 7, described microprocessor 7 comprises: image-display units, be used for the signal of acquisition vision sensor, and output to display unit 8; Described vision sensor is an omnibearing vision sensor, this omnibearing vision sensor is installed in the top in the middle of the vehicle, described omnibearing vision sensor comprises the evagination catadioptric minute surface 2 that is used for reflecting vehicle periphery field object, evagination catadioptric minute surface down, be used to prevent anaclasis and the saturated dark circles cone 4 of light, dark circles cone 4 is fixed on the center of catadioptric minute surface male part 2, be used to support transparent column 3 bodies of evagination catadioptric minute surface, be used to take the camera 5 of imaging body on the evagination mirror surface, camera 5 facing to the evagination mirror surface up; Described microprocessor 7 also comprises: the transducer calibration module is used to set up the pavement image in space and the corresponding relation of the video image that is obtained; The image stretching module is used for the image of omnibearing vision sensor collection is launched, and the circular video image that reads is expanded into the panorama histogram; Vehicle periphery barrier rim detection module, be used to adopt the Suo Beier operator to carry out rim detection, adopt the template of setting size, compute gradient between each template, the tonsure value of selection calculating as marginal point, connects into marginal position with each marginal point greater than the point of gradient magnitude in image; Vehicle periphery barrier edge minimum distance computing module, be used at left and right directions, with the length of vehicle as detection line, add alarm distance detection line to start with from width from vehicle, outwards detect from left and right directions respectively, when the edge line of running into detection of straight lines and barrier when detection line has the pixel of predetermined number to intersect, judge that this place is barrier and the minimum distance of vehicle on left and right directions; On fore-and-aft direction, with the width of vehicle as detection line, add alarm distance detection line to start with from length from vehicle, outwards detect from fore-and-aft direction respectively, when the edge line of running into detection of straight lines and barrier when detection line has the pixel of predetermined number to intersect, judge that this place is barrier and the minimum distance of vehicle on fore-and-aft direction; The barrier alarm module is used to judge that minimum distance whether less than default lower limit, as less than this lower limit, sends alarm command to alarm device; The parking spot designated module is used for the user and wishes the position that parks cars by the input of user interface, utilize omnibearing vision sensor resulting around image information, user select target position on touch-screen on the image information of this demonstration; The reversing knowledge base is used to deposit driver's reversing skill, technical ability, and the backing track line between various current location and the target location, calls during for generation backing track line; Backing track line generation module, be used for detecting the current vehicles position by omnibearing vision sensor, the user is by the input target location of user interface, automatically generate a backing track line, make corresponding reversing envelope according to this backing track line, described reversing envelope is greater than the vehicle minimum path circular arc curvature of can travelling, and judges whether the reversing envelope and the image border of barrier intersect, as just definite this backing track line does not take place to intersect; Otherwise, to the intersection backing track line of making amendment takes place; Steering wheel angle action computing module is used for according to the backing track line that is generated, and obtains destination path on the key point of this backing track line angle ψ that travels _P, the destination path angle ψ that travels _PWith steering wheel angle δ _hRelation with formula (22) expression,

ψ _P＝tan ^-1(Y _P/X _P)≈Y _P/X _P＝K _s*δ _h*t _p/2 (22)

In the following formula, Be a constant, estimate the time t when under this vehicle speed condition, arriving next one reversing key point _pAnd with this steering wheel angle δ _hOutput to display unit; Backing track line correction module, being used for recomputating when vehicle departs from the backing track line backing track line rectifies a deviation, whether the key point of detection on the backing track line skew has taken place, if the off-set value that detects surpasses default scope, regenerate the backing track line according to the position of current vehicle.

The invention provides a kind of intelligent parking auxiliary device based on omnidirectional computer vision, the aid decision making of reversing prompting, distance alarm and reversing can be provided for the driver, comprise in the device: omnibearing vision sensor is used to detect the barrier of vehicle periphery and obtains parking spot information on video; Target parking position installs order unit, is used to set the initial position that the target parking position that parks cars is put; Vehicle periphery situation display unit, be used to show vehicle visual angle, the place ahead, visual angle, rear, left visual angle and right-hand visual angle video image and with the nearest obstacle distance at these visual angles; Backing track line generation unit is used for according to the current position of reversing knowledge base, vehicle and parks the target location generating the backing track line automatically; Steering wheel angle action computing unit is used for determining corner at certain bearing circle on a bit on this trajectory according to the backing track line computation that is generated; Backing track line correction unit is used for recomputating the backing track line and rectifies a deviation when vehicle departs from the backing track line; The nearest detection of obstacles of vehicle unit is used for detecting and the nearest obstacle distance of vehicle according to the present situation of vehicle, as minimum distance employing alarm driver during less than some values; The reversing knowledge base is used to deposit outstanding driver's some reversing skills, technical ability, so that call when generating the backing track line.

Present embodiment will provide a reversing prompting for the driver, the aid decision making of distance alarm and reversing, comprise in the device: after car is hung the reverse gear position, provide (1) that the image information of vehicle periphery all is presented on the display screen in driver the place ahead in the car, the shown video information of this display screen is divided into visual angle, the place ahead, the visual angle, rear, four the small screen such as left visual angle and right-hand visual angle, each view directions all detects the minimum distance with barrier, realize anticollision warning function-apart from obstacle distance during, shown minimum distance value flicker on the corresponding view directions less than 0.60 meter at vehicle, font becomes redness; When distance is used auditory tone cues during less than 0.35 meter; (2) minimum spacing shows-shows that minimum spacing is to remind the driver on display screen; (3) operation indicating-according to the corner that calculates resulting backing track line computation front-wheel and the corner of bearing circle, and provide the operation indication to the driver corner of bearing circle, the driver finishes the reversing task according to the corner of suggested bearing circle.

Designed omnidirectional computer vision device can show omnidirectional image information all around simultaneously on a display screen, make the driver can take into account the situation of vehicle front and back, both sides easily; By understanding, confirm situation, and estimate and its minimum distance at the barrier of vehicle periphery to the omni-directional visual image; Outstanding driver's reversing driving experience, reversing driving skills are made a knowledge base, device can be according to information such as direction that omni-directional visual obtained, distances when the driver moves backward, utilize stored corresponding reversing driving skills knowledge in the knowledge base, adopt software algorithm to realize the aid decision making of moveing backward, with muscle power, the brainwork intensity that alleviates the driver, reduce the reversing difficulty, avoid the driver because of the accident that sense of direction is strong, judgement and misoperation cause.

At first be to obtain vehicle to show omnidirectional image information all around on omnidirectional image information and the display screen in vehicle all around, adopt omnibearing vision sensor ODVS (OmniDirectionalVision Sensors) and corresponding software to obtain vehicle omnidirectional image information all around among the present invention, utilize the software and hardware of embedded system that the full spectrum information that is obtained is shown to the driver in the mode at four visual angles by display screen respectively;

Described omnibearing vision sensor (ODVS), the manufacturing technology scheme of its opticator are mainly constituted by vertically downward catadioptric mirror with towards last camera.It is concrete that to constitute be to be fixed on bottom by the cylinder of transparent resin or glass by the image unit that collector lens and CCD constitute, the top of cylinder is fixed with the catadioptric mirror of a downward deep camber, the dark circles cone that between catadioptric mirror and collector lens, has a diameter to diminish gradually, this coniform body is fixed on the middle part of catadioptric mirror, and the pyramidal purpose of dark circles is to cause light in cylinder inside light reflex saturated and that produce by the cylinder body wall in order to prevent superfluous light from injecting.Fig. 5 is the schematic diagram of the optical system of expression omnibearing imaging device of the present invention.

The principle of work of omnibearing vision sensor is: enter the light at the center of hyperbolic mirror, reflect towards its virtual focus according to bi-curved minute surface characteristic.Material picture reflexes to imaging in the collector lens through hyperbolic mirror, a some P1 on this imaging plane (x, y) corresponding the coordinate A of a point spatially in kind (x1, y1, z1).

11-hyperbolic curve face mirror among Fig. 5,12-incident ray, the focus Om (0 of 13-hyperbolic mirror, 0, c), the virtual focus of 14-hyperbolic mirror is camera center O c (0,0 ,-c), the 15-reflection ray, the 16-imaging plane, the volume coordinate A of 17-material picture (x1, y1, z1), 18-incides the volume coordinate of the image on the hyperboloid minute surface, 19-be reflected in some P1 on the imaging plane (x, y).

The optical system that hyperbolic mirror shown in Fig. 5 constitutes can be represented by following 5 equatioies;

((X ²+Y ²)/a ²)-(Z ²/b ²)＝-1(Z＞0) (1)

$c=\sqrt{a^2+b^2}---\left(2\right)$

β＝tan ^-1(Y/X) (3)

α＝tan ^-1[(b ²+c ²)sinγ-2bc]/(b ²+c ²)cosγ (4)

$\mathrm{\&gamma;}={\tan }^{-1}[f/\sqrt{({\mathrm{<figure-callout\; id="2"\; label="X"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">X</figure-callout>}}^2+Y^2)}]---\left(5\right)$

X in the formula, Y, Z representation space coordinate, c represents the focus of hyperbolic mirror, and 2c represents two distances between the focus, a, b is respectively the real axis of hyperbolic mirror and the length of the imaginary axis, β represents the angle-position angle of incident ray on the XY plane, and α represents the angle-angle of depression of incident ray on the XZ plane, and f represents the distance of imaging plane to the virtual focus of hyperbolic mirror.

Described omnibearing vision sensor is installed in the top in the middle of the vehicle, as shown in Figure 1, the setting height(from bottom) of omnibearing vision sensor (with respect to the height on ground) is to decide according to the angle of depression of omnibearing vision sensor and the situation of automobile body, among the present invention omnibearing vision sensor can be monitored from 0.35 meter object in addition of vehicle, and with the position of its omnibearing vision sensor index as setting height(from bottom), that is to say, the visual range of the omnibearing vision sensor of being installed (from the angle of vehicle center point) all is visible from 0.35 meter barrier in addition of vehicle at least, can not blocked by vehicle body; The XY plane of described omnibearing vision sensor is parallel with ground level.

With Fig. 1 360 ° of comprehensive principles of making a video recording are described, a point on the space reflexes on the lens 6 a subpoint P1 (x should be arranged through catadioptric minute surface 2, y), the light of scioptics 6 becomes directional light and projects CMOS image unit 5, microprocessor 7 reads in this ring-type image by video interface, adopt software that this ring-type image is launched to obtain the omnibearing video image evenly cut apart according to visual angle, the place ahead, visual angle, rear, left visual angle and right-hand visual angle, and be presented in the car on the display unit 8.

Visual angle, described the place ahead, visual angle, rear, left visual angle and right-hand visual angle are evenly cut apart by shown in the accompanying drawing 3, show the video information of level 160 degree on each visual direction.

On method of deploying, adopted a kind of algorithm of approximate expansion fast in this patent, can drop to minimum, kept Useful Information simultaneously as much as possible with time loss with to the requirement of various parameters.Launching rule has three,

(1) X ^*Axle is a reference position, launches by counterclockwise mode;

(2) X among the left figure ^*Axle and the intersection point O of internal diameter r correspond to the initial point O (0,0) in the lower left corner among the right figure;

(3) width of the right figure after the expansion equals the girth of the circle shown in the dotted line among the left figure.Wherein broken circle is the concentric circles of external diameter in the left figure, and its radius r 1=(r+R)/2.

If the center of circle O of circular diagram ^*Coordinate (x ^*0, y ^*0), the histogram lower left corner origin O of expansion ^*(0,0), any 1 P in the histogram ^*=(x ^*, y ^*) pairing coordinate in circular diagram is (x ^*, y ^*). it is following that we need ask is (x ^*, y ^*) and (x ^*, y ^*) corresponding relation.Can obtain following formula according to geometric relationship:

β＝tan ^-1(y ^*/x ^*) (6)

r1＝(r+R)/2 (7)

Make the radius r 1=(r+R)/2 of broken circle, purpose is in order to allow the figure after launching seem that deformation is even.

x ^*＝y ^*/(tan(2x ^**/(R+r))) (8)

y ^*＝(y ^**+r)cosβ (9)

Can obtain a point (x on the circular omnidirectional images from formula (8), (9) ^*, y ^*) and the rectangle panorama sketch on a point (x ^*, y ^*) corresponding relation.This method has come down to do the process of an image interpolation.After the expansion, the image of dotted line top is that horizontal compression is crossed, and the image of dotted line below is that cross directional stretch is crossed, dotted line originally on one's body point then remain unchanged.

The calculating needs equally can be according to a point (x on the circular omnidirectional images in real time in order to satisfy ^*, y ^*) and the rectangle panorama sketch on a point (x ^*, y ^*) corresponding relation, set up (x ^*, y ^*) and (x ^*, y ^*) mapping matrix.Because this one-to-one relationship can be being transformed into indeformable panoramic picture by the mapping matrix method.Can set up formula (10) relation by the M mapping matrix.

P ^**(x ^**，y ^**)←M×P ^*(x ^*，y ^*) (10)

According to formula (10), for each the pixel P on the imaging plane ^*(x ^*, y ^*) a some P arranged on omnidirectional images ^*(x ^*, y ^*) correspondence, set up the M mapping matrix after, the task that realtime graphic is handled can obtain simplifying.

What above-mentioned expansion mode obtained is the comprehensive column stretch-out view of one 360 degree, we are divided into four equal portions with the comprehensive column stretch-out view of these 360 degree, each equal portions is the horizontal view angle (the horizontal visual custom that meets the people) of 160 degree, when the device initialization, with above-mentioned four equal portions and visual angle, the place ahead, the visual angle, rear, left visual angle and right-hand visual angle are mapped, there is the repetition of 35 degree on the right at the left side at visual angle, the place ahead and left visual angle like this, there is the repetition of 35 degree on the right at visual angle, the place ahead and the left side at right-hand visual angle, there is the repetition of 35 degree on the right at same visual angle, rear and the left side, left visual angle, there is the repetition of 35 degree on the left side at visual angle, rear and the right, right-hand visual angle, the driver can see the video image of different visual angles simultaneously on display screen like this, and shown image has continuity, as shown in Figure 4.

After having obtained visual angle, the place ahead, visual angle, rear, left visual angle and right-hand visual angle video image, for help the driver obtain barrier on each view directions with the minimum distance of vehicle, roll over the characteristics of transmitting mirror among the present invention according to the hyperboloid in the omnibearing vision sensor, by subpoint P (x, y) calculate barrier P (X spatially with respect to vehicle center point, Y, Z) point, computing formula is provided by formula (11):

$x=f((b^2-c^2)X/(({\mathrm{<figure-callout\; id="2"\; label="b"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">b</figure-callout>}}^2+c^2)Z-2\mathrm{<figure-callout\; id="2"\; label="bc\; X"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">bc</figure-callout>}\sqrt{X^{}}\sqrt{{}^2+{\mathrm{<figure-callout\; id="2"\; label="Y"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">Y</figure-callout>}}^2+Z^2})---\left(11\right)$

$y=f((b^2-c^2)Y/(({\mathrm{<figure-callout\; id="2"\; label="b"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">b</figure-callout>}}^2+c^2)Z-2\mathrm{<figure-callout\; id="2"\; label="bc\; X"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">bc</figure-callout>}\sqrt{X^{}}\sqrt{{}^2+{\mathrm{<figure-callout\; id="2"\; label="Y"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">Y</figure-callout>}}^2+Z^2})$

Formula (11) represented the P on the space (X, Y, Z) the subpoint P on imaging plane (x, relation y), therefore from imaging plane obtain P (x, value y) can obtain this point at P (X, Y, Z) point;

In order to simplify calculating, in this patent barrier is given prominence to the distance of border as barrier and vehicle on the ground, frontier point P (X spatially so, Y, Z) Z wherein is exactly the value of the true origin of ground and omnibearing vision sensor, this value is a definite value, by P (x, y) value just can obtain the distance of vehicle and barrier easily, because the length and the width of type of vehicle different vehicle are also inconsistent, therefore needed in the past the length L and the width W data of vehicle are input in the embedded system at operative installations, the direction of in this patent vehicle being advanced is as X-axis, the left side of vehicle is as Y-axis, and the distance of vehicle and barrier can be calculated by following formula so:

Wherein, D _yWith D _xBe illustrated respectively in the distance of certain barrier and vehicle on certain view directions, X _BarBe the distance of certain barrier and vehicle center point and X-axis, Y _BarBe the distance of certain barrier and vehicle center point and Y-axis, L is the length of vehicle, and W is the width of vehicle;

The physical boundary of barrier object is the very important descriptor of a class of describing object features, and these borders may produce marginal information in imaging process.The edge is meant the combination that those pixels of significant change are arranged in its surrounding pixel gray scale.The edge is the vector with amplitude and direction, and it shows as the sudden change of gray scale in image.Rim detection is exactly the noncontinuity that will detect this gray scale in the image, determines their exact positions in image simultaneously, calculates distance with vehicle according to formula (12), (13) then.

There is several method to select to rim detection at present, because what expectation obtained in this patent is edge from the nearest barrier of vehicle, and it is less demanding to the integrality and the slickness of edge wheel hub, therefore it is simple that we adopt calculating wherein, classical edge detection method-the method for differential operator of fast operation, this method relies on image differentiated and tries to achieve gradient and carry out rim detection, main from marginal point often corresponding to the big point of single order differential amplitude, while also sets out corresponding to the zero cross point of second-order differential, design some single orders or second-order differential operator, try to achieve its gradient or second derivative zero crossing, select certain threshold value to extract the border again.

Described edge detection method can be divided into following four steps haply:

1. filtering: edge detection algorithm mainly is based on the first order derivative and the second derivative of image intensity, but the calculating of derivative is very sensitive to noise, therefore must use wave filter to improve the performance of the edge detection method relevant with noise.It may be noted that most of wave filters have also caused the loss of edge strength when reducing noise.Therefore the edge strengthens and reduces between the picture noise needs to obtain a kind of balance.

2. strengthen: the basis that strengthens the edge is a changing value of determining each vertex neighborhood intensity in the image.Enhancement algorithms can be given prominence to the point that the neighborhood intensity level has significant change.The edge strengthens generally to be finished by the compute gradient amplitude.

3. detect: in image, have the gradient magnitude of many points bigger, and these might not all be the edges under specific situation, so should be with coming someway to determine that those points are marginal points.The simplest rim detection criterion is a gradient magnitude A value criterion.

4. locate: determine the pixel at place, edge, if more accurate definite marginal position also can come the estimated edge position on subpixel resolution, the direction at edge also can be estimated.

Adopt Suo Beier (Sobel) operator as edge detection algorithm in the present invention, the Sobel operator adopts the template of 3*3 size, has so just avoided compute gradient on the interpolated point between the pixel.The Sobel operator calculates partial derivative with following formula:

S _x＝(a ₂+ca ₃+a ₄)-(a ₀+ca ₇+a ₆)

(14)

S _y＝(a ₀+ca ₁+a ₂)-(a ₆+ca ₅+a ₄)

Constant c is 2 in the formula.The Sobel operator can be realized with following convolution template:

$S_x=\left[\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ -1& 0& 1\end{array}\right]$ $S_y=\left[\begin{array}{ccc}1& 2& 1\\ 0& 0& 0\\ -1& -2& -1\end{array}\right]---\left(15\right)$

We analyze the universal law of reversing then: under the situation of no any prompting, and driver's warehouse-in of moveing backward, at first the width of syllabus measuring car position checks whether the back has barrier, and whether both sides have car to park, and whether length meets self vehicle.Need get off when night, sight line was unclear to fall again behind the environment of observation position.

After the edge of barrier has been arranged, next step work is the minimum distance that will obtain vehicle and these edge lines, the way of this patent is: on left and right directions, with the length of vehicle as detection line, add 0.35 meter detection line to start with from width from vehicle, outwards detect every a pixel ground from left and right directions respectively, when the edge line of running into detection of straight lines and barrier when detection line has K above pixel crossing, think that just this place is barrier and the minimum distance of vehicle on left and right directions; On fore-and-aft direction, also be to adopt to use the same method, with the width of vehicle as detection line, add 0.35 meter detection line to start with from length from vehicle, outwards detect every a pixel ground from fore-and-aft direction respectively, when the edge line of running into detection of straight lines and barrier when detection line has K above pixel crossing, think that just this place is barrier and the minimum distance of vehicle on fore-and-aft direction;

Above-mentioned detection is all carried out on video image, the process that a physical size and the corresponding conversion of display size on the video image are therefore arranged, be also referred to as the demarcation of omnibearing vision sensor, the method of demarcating is, use the height (being installed in the top of car) of omnibearing vision sensor that omnibearing vision sensor is fixed with reality, place a gage from the place at vehicle center position on the ground then, if the length of this gage is D, the projected length that obtains on video image is d (representing with pixel), so just can obtain following relational expression:

$n=\frac{d\left(\mathrm{pixel}\right)}{D\left(\mathrm{cm}\right)}---\left(16\right)$

In the formula (16): n is a calibration coefficient, just can obtain actual size by through type (16) by obtain pixel value on video image.The distance of the length L of vehicle and width W data and barrier and vehicle all is to be that unit calculates with the pixel value, then through type (16) calculate the in fact distance of barrier and vehicle, and this distance value is presented on the display of vehicle front.

Adopt software algorithm to realize the aid decision making of moveing backward in this patent, realize the aid decision making of computing machine reversing, at first outstanding driver's reversing driving experience, reversing driving skills will be made the knowledge base of a relevant reversing;

Universal law from above-mentioned driver's reversing, during reversing, crucial information is the relative orientation and the distance of online assurance car and barrier, the operation of relative orientation and distance decision reversing, the result of correct car backing operation must produce one backing track line (relatively and the center of vehicle), because vehicle is a rigid body, vehicle center is put all around, and the distance of side is determined by the width and the length of vehicle, therefore can make the reversing envelope of vehicle external form in the horizontal direction according to the backing track line of vehicle center, the accidents such as scratch and collision that take place in reversing process are all because reversing envelope and barrier intersect and cause.This patent utilizes the reversing envelope of above-mentioned vehicle external form of making according to the backing track line and whether the edge of the barrier that obtains according to the understanding of video image intersects, so just do not think that selected reversing strategy is feasible if do not take place to intersect, vehicle is stopped to the position of expectation otherwise will finish by twice above reverse action;

The backing track line must have two main information, be the position of present vehicle and the target location that will park, the position of vehicle can obtain by the detection of omnibearing vision sensor at present, and the input of user by user interface need be leaned in the target location, utilize the resulting image information on every side of omnibearing vision sensor among the present invention, the user needs only the approximate location of select target position on touch-screen on the image information of this demonstration, after device receives user's input information, automatically go to accurate vehicle positioning stop target location, then according to current position that is estimated to and target location, automatically generate a backing track line (first scheme), then make corresponding reversing envelope according to this backing track line, whether intersect the image border of further judging reversing envelope and barrier, does not just determine that this reversing scheme is feasible if do not take place to intersect; Otherwise to the intersection backing track line of making amendment takes place, repeat said process then, produce the second reversing scheme, the 3rd ..., till moveing backward successfully.

The reversing scheme is will be to the steering wheel angle of driver during where, and steer angle directly is associated with the backing track line, that is to say and to determine needed steer angle by some points and this tangential direction of backing track line, and being direction of passage dish corner, the driver comes the Control of Automobile wheel turning angle, therefore need set up the relation of steering wheel angle u and wheel turning angle θ, its relational expression of general vehicle can be expressed from the next:

$\frac{\mathrm{\&theta;}}{u}=\frac{40}{1.5}---\left(17\right)$

In the formula (17), the steering locking angle of 40 expression deflecting rollers is that the greatest limit number of turns of 40,1.5 expression bearing circle is 1.5 circles, so can obtain formula (18)

u≈0.0375θ (18)

In the formula (18), u unit is " circle ", and θ unit is " degree ".

After in car, having specified the trajectory of wishing the position of stopping or parking on the display screen the driver, as shown in Figure 7, system generates the reversing scheme automatically, after system successfully generates the backing track scheme, the system prompt driver can move backward and point out the corner of bearing circle, and the driver carries out car backing operation according to shown content, and flow process as shown in Figure 9, vehicle is in reversing process, and omnibearing vision sensor constantly obtains the video information of vehicle-surroundings; And the barrier of vehicle-surroundings is carried out rim detection according to this video information; Calculate minimum distance with the barrier of vehicle-surroundings according to the length of vehicle own and width then; Then its video image that is launched into visual angle, the place ahead, visual angle, rear, left visual angle and right-hand visual angle can be taken into account simultaneously the situation of vehicle front and back, both sides for the driver according to the omni-directional visual image; When the minimum distance of the barrier that detects vehicle and vehicle-surroundings during, on this orientation, visual angle display unit, have red font and show that alerting drivers is noted less than 0.6 meter; When the minimum distance of the barrier that detects vehicle and vehicle-surroundings during, with audible alarm and implement to stop in emergency less than 0.35 meter; Then carry out backing track line offset detection, in case generate good backing track line, because the driver can produce deviation with planning backing track line well in the reversing driving procedure, just need adjust during when deviation, upgrade the reversing scheme according to the final stand of residing position of present vehicle and vehicle greater than some values.The automatic generation method of described renewal reversing scheme is same as the above-mentioned method, just replaces old reversing scheme with new reversing scheme; If detect in the vehicle backing process is when carrying out according to backing track basically, adopt the corner method that changes bearing circle on the key point among the present invention, in case vehicle has reached this key point, the corner of front-wheel on this key point and the corner of bearing circle will calculate in system, the bearing circle that will point out the driver will change vehicle continues according to the good track reversing of plan vehicle simultaneously;

On the run, no matter how the speed of a motor vehicle changes, driver's the time of taking aim in advance constant substantially (or changing very little).If can the appropriate design vehicle or know the steering characteristic of vehicle, make and necessarily taking aim in advance under the time, the steering wheel angle that vehicle reaches the rear lateral position does not change with the speed of a motor vehicle, the compensation that the driver changes vehicle feature will be reduced to a great extent, alleviate driver's burden, alleviate the difficulty of reversing.

The driver in the process of steering vehicle at first according to the current location of vehicle and the specific precalculated position of taking aim at time (being generally 1 second to 1.2 seconds) back vehicle in advance of driver, determine the desirable vehicle route angle of travelling, then according to the travel current state (speed of a motor vehicle, side acceleration, yaw velocity) of angle and vehicle of this path, and driver's the experience of driving determines steering wheel angle, wishes by this steering wheel angle vehicle to be made it a little.

In conventional steering system, because the elasticity of steering mechanism and the mechanism components nonlinear characteristic of tire particularly, the steering gain of vehicle changes with the difference of the speed of a motor vehicle.The driver must make prediction and compensates the variation of vehicle response characteristic, if the speed of a motor vehicle in the time of can or being defined in reversing by change steering ratio of gear, travelled in driver's steering wheel angle and path and present the fixed proportion relation irrelevant between the angle, driver's steering operation is simplified greatly with the speed of a motor vehicle.

Adopted the mode of fixed vehicle speed scope to guarantee travel corresponding relation between the angle of steering wheel angle and vehicle route among the present invention.

For the convenience of discussing, we suppose that driver's straight-line travelling of driving on straight road surface imports then and arrive the required steering wheel angle of predetermined target point, and ignore vehicle transient response in this course, be after the driver imports given steering wheel angle, vehicle enters stable state circumference steering state at once.

As shown in figure 10 at t=t ₀Constantly, central point of omnibearing vision sensor (vehicle's center of gravity) and inertial coordinate initial point (X ₀, Y _o) overlap, and the attitude of supposition vehicle on X-axis is zero.After the driver imported given steering wheel angle, vehicle was made the stable state circumference with yaw velocity y around the O point and is turned to, and establishing the speed of a motor vehicle is U, supposes the side drift angle infinitesimal of Vehicular system, and ignores high order component, and vehicle is at t=t ₀The time be engraved in position (X in the inertial coordinates system _P, Y _P) and path angle (vehicle attitude angle) ψ that travels can be represented by the formula:

X _P＝U*t _P

ψ＝γ*t _P/2 (19)

$Y_P=X_P*\tan \left(\mathrm{\&psi;}\right)\&ap;X_P*\mathrm{\&psi;}=U*\mathrm{\&gamma;}*{\mathrm{<figure-callout\; id="2"\; label="t\; P"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">t</figure-callout>}}_P^{}/2$

The dynamic yaw velocity γ of vehicle can represent the yaw velocity gain G of vehicle to the front wheel angle input _δ ^γ, to turn to be gain G _s, and steering wheel angle δ _hFunction:

$\mathrm{\&gamma;}=G_{\mathrm{\&delta;}}^{\mathrm{\&gamma;}}*G_s*{\mathrm{\&delta;}}_h---\left(20\right)$

Wherein, G _sBe the inverse of steering system ratio i, that is: G _s=1/i=δ _f/ δ _h, δ _fBe front wheel angle.Suppose

Relation is set up,

Then given steering wheel for vehicle corner δ _h, t _PVehicle can be obtained by following formula in the position of inertial coordinates system after second:

$Y_P=U*K_s*{\mathrm{\&delta;}}_h*{\mathrm{<figure-callout\; id="2"\; label="t\; p"\; filenames="H2006101554908E00011.png"\; state="\{\{state\}\}">t</figure-callout>}}_p^{}/2---\left(21\right)$

Can obtain the destination path angle ψ that travels by formula (21) _P, represent with formula (22):

ψ _P＝tan ^-1(Y _P/X _P)≈Y _P/X _P＝K _s*δ _h*t _p/2 (22)

Suppose the backsight time t of reversing _p=1 second, just can calculate vehicle to target location point (X by following formula _P, Y _P) the angle of travelling, path:

ψ _P＝K _s*δ _h/2 (23)

As long as it is gain G that setting reasonably turns to _s, making has in a certain vehicle speed range Be a constant, then at this vehicle speed range, vehicle reaches the required steering wheel angle of impact point and travels with the path and have linear corresponding relation between the angle after one second.K _sRepresent the Vehicular system gain, its occurrence can require to obtain according to the parameter and the general reversing driving of a certain type of vehicle.

In order to simplify the technical difficulty of reversing, the burden when reducing driver's reversing and from now on for realizing that full-automatic reversing prepare, it would be desirable setting up the path angle ψ that travels _PWith steering wheel angle δ _hCorresponding relation, like this in case determine just to calculate behind the good backing track curve path angle ψ that travels _P, then according to the path angle ψ that travels _PWith steering wheel angle δ _hCorresponding relation obtain steering wheel angle δ _h

In general Fu Za backing track can be put together by a plurality of circular arcs, as shown in figure 11, as long as hold several common-denominator target points in reversing process, as first impact point among Figure 11, second impact point and the 3rd impact point, just can finish reversing smoothly; Among the present invention when generating backing track, the impact point that on backing track, is divided into several stages, remove these impact points of match with several track circular arcs then, described backing track circular arc must satisfy two conditions: the track circular arc that 1) is generated is greater than the path of the vehicle minimum circular arc curvature of travelling; 2) the vehicle envelope according to the backing track line that is generated can not overlap with peripheral obstacle; Change steering wheel angle δ when therefore vehicle arrives this phase targets point in reversing process _h, just can guarantee that vehicle finishes reversing smoothly, otherwise will regenerate new backing track line.

Described microprocessor 15 is embedded systems, and the implementation algorithm among the present invention is realized by Java language.

Embodiment 2

With reference to Fig. 1～Figure 11, identical with the above embodiments 1, in the driving procedure (non-reversing) of vehicle, also can use this device to observe the situation of vehicle periphery, open up wide driver's the visual field, the generation of minimizing accident.

Embodiment 3

With reference to Fig. 1～Figure 11, identical with the above embodiments 1, this enforcement is the application on some engineering trucks, because driver's vision of engineering truck is littler than general vehicle, the engineering truck volume ratio is huger, adopt this device and it is installed in the middle part of engineering truck, can open up wide engineering truck driver's the visual field.

Embodiment 4

With reference to Fig. 1～Figure 11, identical with the above embodiments 1, only looking down with the vehicle by omnibearing vision sensor is all environment on every side at center, and the video information of vehicle front and back, both sides is simultaneously displayed on the display in the car, make the driver can take into account the situation of vehicle front and back, both sides simultaneously, be used for alleviating the driver in the muscle power of reversing process and mental paying.

The invention effect that the above embodiments 1,2,3,4 are produced is: can in time find the object of vehicle periphery and the minimum distance of vehicle by omnibearing video image rim detection, avoid the accident in reversing process; By the position that driver's designated vehicle is parked, system generates the backing track line automatically according to driving experience, the driving skills of driver's reversing, makes that driving new hand also can finish the task that reversing parks cars smoothly.Advantages such as device also has ratio of performance to price height, good reliability in addition, maintenance is simple, enforcement is convenient.

Claims (5)

1. intelligent parking auxiliary device based on omnidirectional computer vision, comprise vision sensor, the microprocessor of aid decision making and the display unit that is used to show the vehicle periphery situation are used to stop, described vision sensor connects microprocessor, and described microprocessor comprises:

Image-display units is used for the signal of acquisition vision sensor, and outputs to display unit;

It is characterized in that: described vision sensor is an omnibearing vision sensor, this omnibearing vision sensor is installed in the top in the middle of the vehicle, described omnibearing vision sensor comprises the evagination catadioptric minute surface that is used for reflecting vehicle periphery field object, evagination catadioptric minute surface down, be used to prevent anaclasis and the saturated dark circles cone of light, the dark circles cone is fixed on the center of catadioptric minute surface male part, be used to support the transparent cylinder of evagination catadioptric minute surface, be used to take the camera of imaging body on the evagination catadioptric minute surface, camera facing to evagination catadioptric minute surface up;

Described microprocessor also comprises:

The transducer calibration module is used to set up the pavement image in space and the corresponding relation of the video image that is obtained; The image stretching module is used for the image of omnibearing vision sensor collection is launched, and the circular video image that reads is expanded into the panorama histogram;

Vehicle periphery barrier rim detection module, be used to adopt the Suo Beier operator to carry out rim detection, adopt the template of setting size, compute gradient between each template, the Grad of selection calculating as marginal point, connects into marginal position with each marginal point greater than the point of gradient magnitude in image;

Vehicle periphery barrier edge minimum distance computing module, be used at left and right directions, with the length of vehicle as detection line, add alarm distance detection line to start with from width from vehicle, outwards detect from left and right directions respectively, when the edge line of running into detection of straight lines and barrier when detection line has the pixel of predetermined number to intersect, judge that this place is barrier and the minimum distance of vehicle on left and right directions; On fore-and-aft direction, with the width of vehicle as detection line, add alarm distance detection line to start with from length from vehicle, outwards detect from fore-and-aft direction respectively, when the edge line of running into detection of straight lines and barrier when detection line has the pixel of predetermined number to intersect, judge that this place is barrier and the minimum distance of vehicle on fore-and-aft direction;

The barrier alarm module is used to judge that minimum distance whether less than default lower limit, as less than this lower limit, sends alarm command to alarm device;

The reversing knowledge base is used to deposit driver's reversing skill, technical ability, and the backing track line between various current location and the target location, calls during for generation backing track line;

The parking spot designated module is used for the user and wishes the position that parks cars by the input of user interface, utilize omnibearing vision sensor resulting around image information, user select target position on touch-screen on the image information of this demonstration;

Backing track line generation module, be used for detecting the current vehicles position by omnibearing vision sensor, the user is by the input target location of user interface, automatically generate a backing track line, make corresponding reversing envelope according to this backing track line, described reversing envelope is greater than the vehicle minimum path circular arc curvature of can travelling, and judges whether the reversing envelope and the image border of barrier intersect, as just definite this backing track line does not take place to intersect; Otherwise, to the intersection backing track line of making amendment takes place;

Steering wheel angle action computing module is used for according to the backing track line that is generated, and obtains destination path on the key point of this backing track line angle ψ that travels _P, the destination path angle ψ that travels _PWith steering wheel angle δ _hRelation with formula (22) expression,

ψ _P＝tan ^-1(Y _P/X _P)≈Y _P/X _P＝K _s*δ _h*t _p/2 (22)

In the following formula, Be a constant, X _P, Y _PBe the vehicle position in inertial coordinates system at a time, G _δ ^γBe the yaw velocity gain of front wheel angle input, Gs is that turning to of vehicle is gain, estimates the time t when arriving next one reversing key point under this vehicle speed condition _pAnd with this steering wheel angle δ _hOutput to display unit;

Backing track line correction module, being used for recomputating when vehicle departs from the backing track line backing track line rectifies a deviation, whether the key point of detection on the backing track line skew has taken place, if the off-set value that detects surpasses default scope, regenerate the backing track line according to the position of current vehicle.

2. the intelligent parking auxiliary device based on omnidirectional computer vision as claimed in claim 1, it is characterized in that: described catadioptric minute surface is in order to access the corresponding point with the space object coordinate, and the catadioptric minute surface adopts hyperbolic mirror to design: shown in the optical system that constitutes of hyperbolic mirror can represent by following 5 equatioies;

((X ²+Y ²)/a ²)-(Z ²/b ²)＝-1(Z＞0) (1)

$c=\sqrt{a^2+b^2}---\left(2\right)$

β＝tan ^-1(Y/X) (3)

α＝tan ^-1[(b ²+c ²)sinγ-2bc]/(b ²+c ²)cosγ (4)

$\mathrm{\&gamma;}={\tan }^{-1}[f/\sqrt{(X^2+Y^2)}]---\left(5\right)$

X in the formula, Y, Z representation space coordinate, c represents the focal length of hyperbolic mirror, and 2c represents two distances between the focus, and a, b are respectively the real axis of hyperbolic mirror and the length of the imaginary axis, and β represents the angle of incident ray on the XY plane, i.e. position angle; α represents the angle of incident ray on the XZ plane, i.e. the angle of depression, and f represents the distance of imaging plane to the virtual focus of hyperbolic mirror;

The setting height(from bottom) of omnibearing vision sensor is that the situation according to the angle of depression of omnibearing vision sensor and automobile body decides, omnibearing vision sensor can be monitored object beyond 0.35 meter on vehicle, and with the position of its omnibearing vision sensor index as setting height(from bottom); The XY plane of described omnibearing vision sensor is parallel with ground level;

According to the characteristics of the hyperboloid catadioptric mirror in the omnibearing vision sensor, by subpoint P (x, y) calculate with respect to spatially P of the barrier of vehicle center point (X, Y, Z) point, computing formula is provided by formula (11):

$x=f((b^2-c^2)X/((b^2+c^2)Z-2\mathrm{bc}\sqrt{X^2+Y^2+Z^2})---\left(11\right)$

$y=f((b^2-c^2)Y/((b^2+c^2)Z-2\mathrm{bc}\sqrt{X^2+Y^2+Z^2})$

Formula (11) represented the P on the space (X, Y, Z) the subpoint P on imaging plane (x, relation y), therefore

(x, value y) can obtain this point at P (X, Y, Z) point to obtain P from imaging plane.

3. the intelligent parking auxiliary device based on omnidirectional computer vision as claimed in claim 1 or 2, it is characterized in that: visual angle, the place ahead, visual angle, rear, left visual angle and right-hand visual angle are evenly cut apart, and show the video information of level 160 degree on each visual direction.

4. the intelligent parking auxiliary device based on omnidirectional computer vision as claimed in claim 1 or 2 is characterized in that: described image stretching module comprises:

Read the coordinate information unit, be used to read the centre coordinate of circular omnidirectional images and the inside and outside circle radius of image; The approximate expansion computing unit is used for the initial point O with the centre coordinate setting plane coordinate system of circular omnidirectional images ^*(0,0), X ^*Axle, Y ^*Axle, the internal diameter of image is r, external diameter is R, radius of a circle: r in the middle of setting ₁=(r+R)/2, the position angle is: β=tan ^-1(y ^*/ x ^*); The rectangle cylinder panoramic image is with true origin O ^*(0,0), X ^*Axle, Y ^*Axle is a plane coordinate system, is r and X with the internal diameter in the circular omnidirectional images ^*The intersection point (r, 0) of axle is as true origin O ^*Counterclockwise launch with azimuthal angle beta (0,0); Set up any some pixel coordinate P in the rectangle cylinder panoramic image ^*(x ^*, y ^*) with circular omnidirectional images in pixel coordinate Q ^*(x ^*, y ^*) corresponding relation, its calculating formula is:

x ^*＝y ^*/(tan(360x ^**/π(R+r))) (8)

y ^*＝(y ^**+r)cosβ (9)

In the following formula, x ^*, y ^*Be the pixel coordinate value of rectangle cylinder panoramic image, x ^*, y ^*Be the pixel coordinate value of circular omnidirectional images, R is the external diameter of circular omnidirectional images, and r is the internal diameter of circular omnidirectional images, and β is the position angle of circular omnidirectional images coordinate.

5. the intelligent parking auxiliary device based on omnidirectional computer vision as claimed in claim 1 or 2 is characterized in that: described image stretching module comprises:

Read the coordinate information unit, be used to read the centre coordinate of circular omnidirectional images and the inside and outside circle radius of image; Mapping matrix launches the unit, is used for the centre coordinate of circular omnidirectional images is set the initial point O of plane coordinate system ^*(0,0), X ^*Axle, Y ^*Axle, the internal diameter of image is r, and external diameter is R, and the position angle is: β=tan ^-1(y ^*/ x ^*); The rectangle cylinder panoramic image is with true origin O ^*(0,0), X ^*Axle, Y ^*Axle is a plane coordinate system, is r and X with the internal diameter in the circular omnidirectional images ^*The intersection point (r, 0) of axle is as true origin O ^*Counterclockwise launch with azimuthal angle beta (0,0); According to any some pixel coordinate Q in the circular omnidirectional images ^*(x ^*, y ^*) with the rectangle cylinder panoramic image in pixel coordinate P ^*(x ^*, y ^*) corresponding relation, set up from Q ^*(x ^*, y ^*) to P ^*(x ^*, y ^*) the mapping matrix corresponding relation, its calculating formula is:

P ^**(x ^**，y ^**)←M×Q ^*(x ^*，y ^*) (10)

In the following formula, Q ^*(x ^*, y ^*) be the matrix of each pixel coordinate on the omnidirectional images, M is the corresponding relation matrix from the omnidirectional images coordinate to rectangle cylinder panoramic image coordinate, P ^*Matrix for each pixel coordinate on the rectangle cylinder panoramic image.

Images