Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/50—Depth or shape recovery
  • G06T7/55—Depth or shape recovery from multiple images
  • G06T7/593—Depth or shape recovery from multiple images from stereo images
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
  • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
  • G06V20/586—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of parking space

Definitions

• the invention relates to a method and a device for video-based observation and measurement of the lateral surroundings around a road vehicle.
• the first driver assistance systems available on the market require a compromise between the resolution of the scanning and the size of the detected area.
• the radar sensors for an ACC application must be limited horizontally to a few degrees of detection area, while for a parking aid function in the detection of the surroundings, only a small range and resolution can be realized in addition to the vehicle using ultrasound.
• Conventional video-based systems offer a good compromise between resolution and coverage, but they generally do not provide direct distance information.
• a device for detecting the position of an agricultural vehicle carrying a scanning sensor is described in US Pat. No. 5,809,440 A1.
• the track of the vehicle is tracked using a global navigation system (GPS).
• GPS global navigation system
• the scanning optical sensor used to record the growth does not provide any distance information, only a two-dimensional cartography of the background is achieved by lining up the sensor information.
• stereo camera systems are increasingly being used, with the aid of which the object width can be calculated by means of the known geometric arrangement of the camera pair used.
• stereo image processing an object is scanned from different directions with a pair of cameras. The distance between the stereo camera system and the object is then determined taking into account the camera positions and orientations as well as the camera parameters based on the principle of triangulation.
• DE 199 26 559 A1 mentions a method and a device for the detection of objects in the vicinity of a road vehicle.
• the distance to a moving vehicle is calculated by evaluating stereo image pairs and properties of the detected objects are determined.
• distance-based image segmentation is carried out using stereo image processing.
• Object recognition is then carried out in the segmented image areas.
• EP 108 72 57 A2 an arrangement for fastening a stereo camera system in a vehicle is described, which is used to calculate a three-dimensional distance distribution for a viewed object.
• the invention has for its object to provide a novel method for video-based observation and measurement of the lateral environment of a vehicle and a vehicle with a sensor system for performing the method according to the preambles of claims 1 and 15, which has a high mechanical robustness and without can adjust high effort.
• the object is achieved according to the invention by a method and a device with the features of claims 1 and 15.
• the observation and measurement of the lateral surroundings of a vehicle takes place, primarily for the detection of parking spaces, on the one hand by means of a camera with which digital images are recorded and on the other hand by means of a DV unit which serves to provide the digital images with a time stamp and cache.
• the device further comprises a unit for detecting the vehicle's own movement in order to select image pairs on the basis of this data from the temporarily stored images.
• a further means is provided to determine the position and orientation of the camera present at the two recording times. Using an algorithm for stereo image processing, a local 3D depth image can be generated on the basis of the image pair, taking into account the position and orientation of the camera at the time of recording as part of a synthetic stereo geometry.
• the novel design of the subject matter of the invention makes this possible dynamic scenarios, such as capturing the side 3D geometry to the edge of the lane from the perspective of the actively dynamic vehicle and evaluating it to its advantage when parking.
• the inventive monocular camera system in conjunction with the method according to the invention offers the advantage of a significantly smaller space requirement.
• such a system represents an advantageous solution for integration into the vehicle and additionally offers the advantage of comparatively low costs by dispensing with a second camera.
• the mechanical robustness of monocular camera systems is comparatively higher than that of stereo cameras.
• Another advantage that single camera systems offer is a significantly reduced effort in assembly and adjustment.
• the images to be selected for image pairs need not necessarily be two immediately successive image recordings.
• the time stamp should ideally be selected in such a way that the time difference, taking into account the vehicle's own speed, corresponds to a fixed distance (stereo basis).
• the distance (stereo base) should preferably be selected such that it is in a range between 0.2 m and 1 m, depending on the current speed at the time of two image recordings. Since the average vehicle speed when parking a road vehicle is approx.
• the object of the invention can be used in a particularly advantageous manner with the aim of accumulating a global 3D depth view from the sequence of local 3D depth views obtained by means of the image pairs.
• the image data of the individual local 3D depth views which are to be assigned to the same location points of the surroundings of the vehicle, are added together and stored in a geometric data structure.
• the geometric data structure is continuously expanded in the same direction according to the direction of travel of the vehicle by the newly added image areas.
• the image data of the respective local location points to be added for a global 3D depth view are subjected to a weighting before the accumulation. Because of this weighting, the noise generated during accumulation can be reduced by subsequently filtering individual pixels.
• the volume displayed as part of the weighting by means of the accumulated 3D depth views is divided into individual volume elements and the pixels to be added are distributed to the volume elements.
• All volume elements preferably comprise a predefined, uniform volume, for example in the form of cubes.
• each node uses a 3D key to distribute the volume of the stored volumes to eight subtrees (volume elements).
• volume elements are assigned the number of pixels within the individual volume elements as a weight. For this reason, the weight of the volume elements is stored in the tree structure in addition to the data of the pixels. It is particularly advantageous to determine the average weight of all volume elements and thus to determine a total weight.
• a threshold value is formed with the total weight, on the basis of which a decision is made as to whether the pixels contained in the individual volume elements are taken into account in further processing. Only those image points are considered in which the volume element assigned to them has a weight that is equal to or greater than the threshold value. In an advantageous manner, only those volume elements that contain pixels are used to determine the total weight.
• the object of the invention can be used in a particularly advantageous manner with the aim of forming the threshold value by multiplying the total weight by a tuning factor.
• a tuning factor 1.5, the number of pixels per image was reduced from 46013 to 29792.
• the camera used in the context of the invention can be a camera which is part of another image recording system already present in the vehicle.
• the figure shows an example of a traffic scenario using the device according to the invention, for video-based observation and measurement of the lateral surroundings around a road vehicle (10).
• the measurement of the lateral 3D geometry of a moving vehicle to the edge of the road is used here to measure a parking space.
• a parking vehicle (40), a guardrail can be parked at the edge of the road . 501 or be . liehi ⁇ p _nHoro r.hiol. , _. , On w; - -; - t. . _....; - .. ...... -J - ⁇ r __.-_. _i - T - - ⁇ ⁇ __.
• a monocular camera (30) is installed in the road vehicle (10) in connection with a computer unit.
• the camera (30) is aligned such that the angle between its optical axis and the direction of travel (20) is ideally 90 °.
• the possible orientation of the camera (30) is not limited to the orientation perpendicular to the direction of travel (20), but can also be aimed at areas to be recorded at an angle of 45 ° - 135 ° to the direction of travel (20).
• the variation of the orientation of the camera can be used in an advantageous manner in particular for different design options with regard to the integration into the overall vehicle.
• the image sequence resulting from the movement of the vehicle (10) when recording with the camera (30) is provided with time stamps by means of the computer unit and temporarily stored.
• An algorithm for stereo image processing is used to generate a 3D image of the parking space from the temporarily stored images, taking into account the vehicle's own movement (10). Based on the 3D information, it can be decided, taking into account a defined parking strategy, whether the 3D space detected is suitable for parking the vehicle and, in particular, sufficient space is available for parking.
• the object of the invention is also suitable for other uses in road traffic.
• the invention is also profitably suitable for use as a warning system which measures the lateral distance of a vehicle from the edge of the road and warns the driver when the distance falls below a minimum, analogously to the exemplary embodiment mentioned above.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Multimedia (AREA)
• Traffic Control Systems (AREA)
• Image Processing (AREA)
• Image Analysis (AREA)
• Length Measuring Devices By Optical Means (AREA)
• Measurement Of Optical Distance (AREA)
• Closed-Circuit Television Systems (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (2)

Family

ID=31983990

Family Applications (1)

Country Status (5)

Cited By (7)

Families Citing this family (25)

Family Cites Families (9)

• 2002
  • 2002-09-23 DE DE10244148A patent/DE10244148A1/de not_active Withdrawn
• 2003
  • 2003-09-19 EP EP03753435A patent/EP1543475A2/de not_active Withdrawn
  • 2003-09-19 JP JP2004538958A patent/JP4327726B2/ja not_active Expired - Fee Related
  • 2003-09-19 WO PCT/EP2003/010456 patent/WO2004029877A2/de not_active Ceased
  • 2003-09-19 US US10/528,810 patent/US7365831B2/en not_active Expired - Lifetime

Also Published As

Similar Documents

Legal Events