WO2023001711A1 - Verfahren zum betreiben einer gated-kamera, steuervorrichtung zur durchführung eines solchen verfahrens, gated-kamera-vorrichtung mit einer solchen steuervorrichtung und kraftfahrzeug mit einer solchen gated-kamera-vorrichtung - Google Patents
Verfahren zum betreiben einer gated-kamera, steuervorrichtung zur durchführung eines solchen verfahrens, gated-kamera-vorrichtung mit einer solchen steuervorrichtung und kraftfahrzeug mit einer solchen gated-kamera-vorrichtung Download PDFInfo
- Publication number
- WO2023001711A1 WO2023001711A1 PCT/EP2022/069909 EP2022069909W WO2023001711A1 WO 2023001711 A1 WO2023001711 A1 WO 2023001711A1 EP 2022069909 W EP2022069909 W EP 2022069909W WO 2023001711 A1 WO2023001711 A1 WO 2023001711A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- recording
- lighting device
- optical sensor
- image
- difference
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000003287 optical effect Effects 0.000 claims abstract description 64
- 238000005286 illumination Methods 0.000 claims abstract description 55
- 230000001360 synchronised effect Effects 0.000 claims 1
- 230000004913 activation Effects 0.000 abstract description 12
- 230000001419 dependent effect Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- GNFTZDOKVXKIBK-UHFFFAOYSA-N 3-(2-methoxyethoxy)benzohydrazide Chemical compound COCCOC1=CC=CC(C(=O)NN)=C1 GNFTZDOKVXKIBK-UHFFFAOYSA-N 0.000 description 1
- FGUUSXIOTUKUDN-IBGZPJMESA-N C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 Chemical compound C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 FGUUSXIOTUKUDN-IBGZPJMESA-N 0.000 description 1
- YTAHJIFKAKIKAV-XNMGPUDCSA-N [(1R)-3-morpholin-4-yl-1-phenylpropyl] N-[(3S)-2-oxo-5-phenyl-1,3-dihydro-1,4-benzodiazepin-3-yl]carbamate Chemical compound O=C1[C@H](N=C(C2=C(N1)C=CC=C2)C1=CC=CC=C1)NC(O[C@H](CCN1CCOCC1)C1=CC=CC=C1)=O YTAHJIFKAKIKAV-XNMGPUDCSA-N 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/06—Interpretation of pictures by comparison of two or more pictures of the same area
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
- G01S17/08—Systems determining position data of a target for measuring distance only
- G01S17/10—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
- G01S17/18—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves wherein range gates are used
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/4802—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/481—Constructional features, e.g. arrangements of optical elements
- G01S7/4814—Constructional features, e.g. arrangements of optical elements of transmitters alone
- G01S7/4815—Constructional features, e.g. arrangements of optical elements of transmitters alone using multiple transmitters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
Definitions
- the invention relates to a method for operating a gated camera, a control device for carrying out such a method, a gated camera device with such a control device and a motor vehicle with such a gated camera device.
- Methods for operating a gated camera are known, in which an object is recognized on the basis of a shadow cast by the object. These methods are based on the assumption that a measure of a reflectivity of, in particular, diffusely reflecting, i.e. in particular non-luminous surfaces, in particular an albedo, of the object and an area surrounding the object is large enough so that enough photons are directed towards the gated camera reflected in order to be able to take a high-contrast picture.
- the higher the albedo the higher the contrast can be recorded using the gated camera.
- the shadow cast by the object can be detected more reliably if the albedo is high and sufficient photons for illuminating the optical sensor are reflected, in particular diffusely, by the object and the area surrounding the object.
- a disadvantage of this method is that the albedo of the surroundings of the object—in particular a road surface—can become very small, in particular as a function of a surface condition and/or surface coverage. In particular, puddles of water on a road surface and/or a very dark road surface cause the albedo to become very small and no shadow can be detected, and object recognition is therefore not possible.
- the invention is therefore based on the object of providing a method for operating a gated camera, a control device for carrying out such a method, a 2
- the object is achieved in particular by creating a method for operating a gated camera with a first lighting device, a second lighting device spaced apart from the first lighting device and an optical sensor, with activation of the first lighting device, the second lighting device and the optical Sensors are timed to each other, and the coordinated control is associated with a visible distance range.
- a first recording is made when the first lighting device is illuminated with the optical sensor by means of the coordinated control.
- a second recording is made with illumination by means of the second illumination device with the optical sensor by means of the coordinated control.
- a third recording is made with the optical sensor without illumination by means of one of the illumination devices.
- a first difference recording is then formed as the difference between the first recording and the second recording.
- a second difference recording is formed as the difference between the first recording and the third recording.
- the second difference recording is formed as the difference between the second recording and the third recording.
- the third recording corresponds to an ambient light recording, in particular a daylight recording.
- the influence of ambient light, in particular daylight is advantageously calculated from the first recording or the second recording by subtracting the third recording. This can advantageously be used to identify areas of the visible distance range in which the albedo is small, in particular close to zero, and photons from the first illumination device and/or the second illumination device are almost exclusively absorbed and/or reflected in such a way that too small a proportion of the Emitted photons reach the gated camera for exposure of the optical sensor.
- a reflection of a surface describes how photons that hit this surface are reflected back from the surface.
- a specular reflection - in particular also referred to as directed reflection - can be distinguished from a diffuse reflection. If one considers a plurality of photons, with the plurality of photons having an almost identical direction of propagation, the plurality of photons are thrown back in a reflection direction in each case when they are reflected at the surface. The plurality of reflection directions are almost identical in specular reflection, with the respective reflection directions being dependent on the propagation direction.
- the majority of reflection directions vary greatly in the case of diffuse reflection, with a main reflection direction preferably forming perpendicularly to the surface, in particular independently of the propagation direction (Lambert's law).
- the method for generating recordings by means of a temporally coordinated activation of at least one lighting device—in particular the first lighting device and/or the second lighting device—and an optical sensor is in particular a method known as a gated imaging method; in particular, the optical sensor is a camera that is only sensitive to a specific, limited time range, which is referred to as "gated activation", so the camera is a gated camera.
- the at least one lighting device--in particular the first lighting device and/or the second lighting device-- is also activated correspondingly only in a specific, selected time interval in order to illuminate a scene on the object side, in particular the visible distance range.
- a predefined number of light pulses are emitted by the first illumination device and the second illumination device, preferably each with a duration of between 5 ns and 20 ns.
- the start and end of the exposure of the optical sensor is linked to the number and duration of the emitted light pulses and a start of the illumination.
- a local distance between the at least one lighting device - in particular the first lighting device and the second lighting device - and the optical sensor is preferably known and small compared to the distance between the at least one lighting device - in particular the first lighting device and the second lighting device - or the optical sensor to the visible distance range.
- a distance between the optical sensor and the visible distance range is equal to a distance between the gated camera and the visible distance range.
- the visible distance range is that area in three-dimensional space - on the object side - which is defined by the number and duration of the light pulses of the at least one lighting device - in particular the first lighting device and/or the second lighting device - and the start of the lighting in connection with the start and the end of the exposure of the optical sensor is imaged by means of the optical sensor in a two-dimensional recording on an image plane of the optical sensor.
- the observation area is in particular the area in three-dimensional space—on the object side—which, given sufficient lighting and exposure of the optical sensor by means of the optical sensor, could be imaged in a two-dimensional recording as a whole—in particular as a maximum.
- the observation area corresponds to the entire image area of the optical sensor that can be exposed and that could theoretically be illuminated.
- the visible distance range is thus a subset of the observation range in real space.
- only a subset of the image plane of the optical sensor is exposed in the method proposed here, with this subarea of the image plane being present in particular between a start image line and an end image line.
- object-side an area in real space is addressed.
- image side an area on the image plane of the optical sensor is addressed.
- the observation range and the visible distance range are given on the object side.
- mapping laws as well as the temporal Activation of the at least one lighting device—in particular the first lighting device and/or the second lighting device—and the optical sensor associated image-side areas on the image plane.
- the optical sensor After the start and end of the exposure of the optical sensor after the start of the illumination by the at least one illumination device—in particular the first illumination device and/or the second illumination device—light pulse photons impinge on the optical sensor.
- the time interval between the end of the illumination and the beginning of the exposure increases the further away the visible distance range is from the at least one illumination device—in particular the first illumination device and/or the second illumination device—and from the optical sensor.
- the method it is therefore possible, in particular, to determine the position and the spatial width of the visible distance range, in particular a distance between the near boundary and the far boundary of the visible distance range.
- the visible distance range is specified, from which the timing of the first illumination device and/or the second illumination device on the one hand and the optical sensor on the other hand is correspondingly specified.
- the timing of the first lighting device and/or the second lighting device on the one hand and the optical sensor on the other is specified, with the visible distance range being specified accordingly.
- the first lighting device and/or the second lighting device has at least one surface emitter, in particular a so-called VCSE laser.
- the optical sensor is preferably a camera.
- the third recording is made after the first recording and after the second recording.
- the third recording is made after the first recording. After the third shot, the second shot is taken.
- the third recording is made before the first recording and the second recording.
- the process can be carried out continuously.
- the first recording, the second recording and the third recording are recorded the same number of times per unit of time.
- the first recording, the second recording and the third recording are each recorded at a time interval of less than 0.1 seconds, preferably less than 0.01 seconds.
- the recordings in particular the first recording, the second recording and the third recording—are recorded at a recording rate of at least 20 Hz and at most 90 Hz.
- a recording rate of at least 20 Hz to a maximum of 90 Hz it is not necessary to consider the own speed and/or the speed of the object found in order to determine a further coordinated activation.
- a method for image registration is applied to the first image, the second image and the third image before the first differential image and the second differential image are formed.
- An inherent movement of a motor vehicle having the gated camera is advantageously compensated for by means of the image registration.
- a method for image registration is applied to the first recording and the second recording, wherein for the formation the first difference recording, the second recording is adapted to the first recording or the first recording to the second recording.
- a method for image registration is applied to the second image and the third image, with the third image being adapted to the second image or the second image to the third image to form the second difference image.
- a method for image registration is applied to the first image and the second image, the second image being adapted to the first image or the first image to the second image to form the first differential image.
- a method for image registration is applied to the first image and the third image, with the third image being adapted to the first image or the first image to the third image to form the second difference image.
- an object is arranged in the visible distance range and if an image-side shadow cast by the object is visible in at least one image selected from the first image and the second image, advantageously in the first difference image only the image-side shadow cast by the object - not but the object itself - because the first shot and the second shot differ only in the shadow cast.
- the object can thus be detected on the basis of the shadow cast on the image side in the first difference recording.
- the object is arranged in the visible distance range and if a shadow cast by the object on the image side is not visible either in the first image or in the second image, it is not possible to easily detect the object in the first differential image.
- no shadow cast by the object on the image side can be detected if the area surrounding the object reflects the photons almost predominantly specularly away from the optical sensor—in particular, does not reflect them diffusely.
- the surroundings of the object are almost predominantly mirror-reflecting, the surroundings of the object in the first recording and/or the second recording are represented at least approximately identically to those in the third recording. In this way, the object—in particular a bright object—can advantageously be detected in the second difference recording.
- blind points are searched for in the second difference recording, reliability ranges of the recordings being defined on the basis of the blind points found.
- all pixels of the second differential recording that are not blind spots form the reliability ranges of the recordings.
- a pixel of a recording is defined as a blind spot if a luminance of the pixel is less than a predetermined threshold luminance.
- a gray value of the pixels is used as the luminance of the pixels.
- the luminance is a measure of the brightness of a pixel.
- the predetermined threshold luminance is less than 0.2, preferably less than 0.1, preferably less than 0.05, more preferably less than 0.01.
- a blind point characterizes an area of the visible distance area on the image side, which is represented almost identically, preferably identically, in the first recording and the third recording or in the second recording and the third recording.
- a reliability range advantageously characterizes an image-side range of the visible distance range, which is represented differently in the first recording and the third recording or in the second recording and the third recording.
- the objects found in the first difference recording are verified using the reliability ranges.
- a robust and reliable object detection can advantageously be implemented in this way.
- error-free object recognition in particular based on at least one shadow cast on the image side of the object to be recognized in the first difference recording, cannot be guaranteed in the blind points.
- reliable object recognition in particular based on at least one shadow cast on the image side of the object to be recognized in the first difference recording, can be implemented in the reliability areas.
- An object that is recognized in a reliability range is preferably verified.
- an object that is detected in a blind point is preferably not verified and the method is preferably restarted, with a further first recording, a further second recording and a further third recording being recorded.
- a future movement trajectory of a motor vehicle having the gated camera is determined based on the first difference recording and/or the second difference recording, i.e. in particular it is defined.
- the future movement trajectory is preferably additionally determined on the basis of the objects that have been found and, in particular, verified.
- the future movement trajectory of the motor vehicle is implemented by means of an actuator system of the motor vehicle, in particular by means of steering movements, acceleration processes and/or braking processes.
- a base point determination is carried out on an object found in the second differential recording.
- An object distance of the object from the optical sensor and/or from at least one lighting device, selected from the first lighting device and the second lighting device, can preferably be determined on the basis of an image-side base point.
- the image-side object in the second differential recording has an image of the object and an image of a reflection of the object, it being preferably assumed that in the second differential recording a Horizontal symmetry holds.
- the image-side base point is preferably estimated, in particular determined, as a horizontal image-side line between a maximum image-side extent in the vertical direction and a minimum image-side extent in the vertical direction of the image-side object.
- the object is also achieved by creating a control device that is set up to carry out a method according to the invention or a method according to one or more of the embodiments described above.
- the control device is preferably designed as a computing device, particularly preferably as a computer, or as a control unit, in particular as a control unit of a motor vehicle.
- a computing device particularly preferably as a computer
- a control unit in particular as a control unit of a motor vehicle.
- the control device is preferably set up to be operatively connected to the gated camera, in particular to the first lighting device, the second lighting device and the optical sensor, and set up to control them in each case.
- the object is also achieved by creating a gated camera device that has a gated camera that has a first illumination device, a second illumination device and an optical sensor, and a control device according to the invention or a control device according to one or more of those described above Having embodiments, wherein the first lighting device and the second lighting device are arranged at a distance from one another.
- a gated camera device In connection with the gated camera device, there are in particular the advantages that have already been explained in connection with the method and the control device.
- first lighting device and the second lighting device are arranged offset horizontally with respect to one another.
- first lighting device and the second lighting device are preferably arranged vertically offset from one another.
- a distance between the first lighting device and the second lighting device is more than 10 cm, preferably more than 20 cm, preferably more than 50 cm, preferably more than 100 cm, particularly preferably more than 150 cm.
- the distance is measured in a horizontal direction if the first lighting device and the second lighting device are arranged horizontally offset from one another.
- the distance is measured, in particular, in a vertical direction if the first illumination device and the second illumination device are arranged vertically offset from one another.
- the distance is measured in particular in an oblique direction if the first Lighting device and the second lighting device are arranged horizontally and vertically offset from one another.
- the control device is preferably operatively connected to the gated camera, in particular to the first illumination device, the second illumination device and the optical sensor, and is set up to control them in each case.
- the object is also achieved by creating a motor vehicle with a gated camera device according to the invention or a gated camera device according to one or more of the embodiments described above.
- a gated camera device according to the invention or a gated camera device according to one or more of the embodiments described above.
- the motor vehicle is designed as an autonomously driving motor vehicle.
- the motor vehicle is preferably designed as a truck.
- the motor vehicle it is also possible for the motor vehicle to be in the form of a passenger car, a commercial vehicle, or another motor vehicle.
- the motor vehicle is in the form of a truck that in particular drives autonomously.
- the optical sensor is preferably arranged above the windshield.
- the first lighting device and the second lighting device are preferably arranged in the area of the bumper and are at a distance from one another in the horizontal direction of more than 10 cm, preferably more than 20 cm, preferably more than 50 cm, preferably more than 100 cm, in particular preferably more than 150 cm.
- FIG. 1 shows a schematic representation of an exemplary embodiment of a motor vehicle
- FIG. 2 shows a flowchart of an embodiment of a method for operating a gated camera
- 3 shows a schematic representation of a first example of a first recording, a second recording, a third recording, a first differential recording and a second differential recording
- FIG. 4 shows a schematic representation of a second example of the first recording, the second recording, the third recording, the first difference recording and the second difference recording, and
- FIG. 5 shows a schematic representation of the second example of the second difference recording.
- FIG. 1 shows a schematic representation of an embodiment of a motor vehicle 1, in particular an autonomous truck, with an embodiment of a gated camera device 3.
- the gated camera device 3 has a gated camera 5 with a first lighting device 7.1, a second Illumination device 7.2 and an optical sensor 9, in particular a camera, and a control device 11.
- the control device 11 is operatively connected to the gated camera 5, in particular to the first lighting device 7.1, the second lighting device 7.2 and the optical sensor 9, in a manner that is not explicitly shown, and is set up to control them in each case.
- the first lighting device 7.1 and the second lighting device 7.2 are arranged at a distance from one another.
- the first lighting device 7.1 and the second lighting device 7.2 are particularly preferably arranged offset horizontally with respect to one another.
- the first lighting device 7.1 and the second lighting device 7.2 are preferably arranged vertically offset from one another.
- the first lighting device 7.1 and the second lighting device 7.2 preferably each have at least one surface emitter, in particular a so-called VCSE laser.
- Figure 1 shows a first illumination frustum 13.1 of the first illumination device 7.1, a second illumination frustum 13.2 of the second illumination device 7.2 and an observation area 15 of the optical sensor 9.
- An object 19 is arranged within the visible distance range 17 .
- the control device 11 is set up in particular to carry out an exemplary embodiment of a method for operating the gated camera 5, which is described in more detail in Figure 2, with the first lighting device 7.1, the second lighting device 7.2 and the optical sensor 9.
- Figure 2 shows a flowchart of an embodiment of a method for operating the gated camera 5.
- a first step a) activation of the first lighting device 7.1, the second lighting device 7.2, and the optical sensor 9 are coordinated in terms of time, the visible distance range 17 being assigned to the coordinated activation.
- a first recording 21.1 is recorded with illumination by means of the first illumination device 7.1 with the optical sensor 9 by means of the coordinated control.
- a second recording 21.2 is recorded with illumination by means of the second illumination device 7.2 with the optical sensor 9 by means of the coordinated control.
- a third recording 21.3 is recorded with the optical sensor 9 without illumination by means of one of the illumination devices 7.
- the first recording 21.1 is preferably recorded first, followed by the second recording 21.2 and then the third recording 21.3.
- the first recording 21.1 is preferably recorded first in a second chronological sequence, followed by the third recording 21.3 and then the second recording 21.2.
- the second recording 21.2 is preferably recorded first in a third chronological sequence, followed by the first recording 21.1 and then the third recording 21.3.
- the third recording 21.3 is preferably recorded first in a fifth chronological sequence, followed by the first recording 21.1 and then the second recording 21.2.
- the third recording 21.3 is recorded first, followed by the second recording 21.2 and then the first recording 21.1.
- the second recording 21.2 is preferably recorded in the second step b) with illumination by means of the second illumination device 7.2 with the optical sensor 9 by means of the coordinated control.
- the first recording 21.1 is recorded with illumination by means of the first illumination device 7.1 with the optical sensor 9 by means of the coordinated control.
- a first difference recording 23.1 is formed as the difference between the first recording 21.1 and the second recording 21.2.
- a second difference recording 23.2 is formed as the difference between the second recording 21.2 and the third recording 21.3.
- the second differential recording 23.2 is preferably formed as the difference between the first recording 21.1 and the third recording 21.3.
- a search is made for objects 19, in particular for image-side objects 19', preferably in the first difference recording 23.1.
- the object 19 is preferably detected in the first difference recording 23.1 using a shadow 25 cast on the image side.
- a search is made for objects 19, in particular for image-side objects 19', preferably in the second difference recording 23.2.
- blind points 27 are preferably searched for in the second differential image 23.2, with the blind points 27 found being used to define reliability ranges 29 of the images 21, 23.
- the objects 19 found in the first difference recording 23.1 are verified using the reliability ranges 29.
- a base point 31 on the image side is preferably determined by at least one of the objects 19 found.
- a future movement trajectory is determined using the—possibly verified—first differential recording 23.1 and/or the—possibly verified—second differential recording 23.2.
- FIG. 3 shows a schematic representation of a first example of the first recording
- FIG. 3a shows a schematic representation of the first example of the first recording
- the distance region 17' visible on the image side and the object 19' on the image side arranged therein, which forms the shadow 25 cast on the image side, can be clearly seen. Furthermore, in particular a first surface 33.1 and a second surface 33.2 can be seen within the distance area 17' visible on the image side, which appear darker in the first photograph 21.1 than the remaining distance area 17' visible on the image side.
- a road surface it is possible for a road surface to be covered with water in an object-side area of the first surface 33.1 and in an object-side area of the second surface 33.2, and thus in particular the photons are specularly reflected away from the optical sensor 9.
- FIG. 3 b shows a schematic representation of the first example of the second receptacle 21.2.
- the shadow 25 cast on the image side is not visible in FIG. 3b).
- the reason for this is the particularly horizontally spaced arrangement of the lighting devices 7, as a result of which the object forms a different shadow 25 depending on the lighting.
- FIG. 3c shows a schematic representation of the first example of the third recording 21.3, the object 19' on the image side being clearly visible.
- the third recording 21.3 is a daylight recording or an ambient light recording.
- FIG. 3d) shows a schematic representation of the first example of the first differential recording 23.1 as the difference between the first recording 21.1 and the second recording 21.2.
- the only visible element in the first difference recording 23.1 is the image-side shadow 25 of the object 19.
- the object 19 is preferably detected on the basis of the image-side shadow 25 in the first difference recording 23.1.
- FIG. 3e) shows a schematic representation of the first example of the second difference recording 23.2 as the difference between the second recording 21.2 and the third recording 21.3.
- the visible distance range 17 and the image-side object 19' are visible in the second difference recording 23.2.
- the areas outside of the visible distance range 17, the first surface 33.1 and the second surface 33.2 are shown as blind spots 27.
- all other areas of the second difference recording 33.2 that are not blind points 27 are reliability areas 29.
- Figure 4 shows a schematic representation of a second example of the first recording 21.1, the second recording 21.2, the third recording 21.3, the first differential recording 23.1 and the second differential recording 23.2.
- the first receptacle 21.1 in FIG. 4a) and the second receptacle 21.2 in FIG. 4b) are almost identical, in particular identical. Furthermore, the visible distance range 17 cannot be seen. This is the case when the scene to be observed either reflects photons almost exclusively specularly away from the optical sensor 9, in particular due to a very low albedo, for example on a wet road, and/or the environment absorbs photons and thus almost no photons are reflected at all .
- the remaining areas of the recording correspond to a recording without illumination using at least one of the illumination devices 7.
- FIG. 4 c) shows, analogously to FIG. 3 c), the third recording 21.3, which is in particular a daylight recording or an ambient light recording.
- the image-side object 19' can also be seen in the third image 21.3, with the image-side object 19' appearing less brightly in comparison to the first image 21.1 and the second image 21.2.
- FIG. 4d) shows, analogously to FIG. 3d), the first difference recording 23.1. Since the first recording 21.1 and the second recording 21.2 are almost identical, the first differential recording 23.1 is colored homogeneously and no structure, in particular no object 19, can be detected.
- FIG. 4e) shows, analogously to FIG. 3e), the second difference recording 23.2. In particular, only the image-side object 19' can be seen here. It is therefore preferably possible to detect the object 19 by means of the second differential recording 23.2.
- FIG. 5 shows a schematic representation of the second example of the second difference recording 23.2.
- the surroundings of the object 19 reflect photons in a specular manner and that a reflection 35 of the object 19 can therefore be seen in the second difference recording 23.2 in addition to the object 19' on the image side.
- the image-side object 19' and the reflection 35 are identified using horizontal symmetry, so that a base point of the object 19 can be determined.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
Description
Claims
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202280048163.XA CN117616301A (zh) | 2021-07-20 | 2022-07-15 | 选通摄像头操作方法、用于执行这种方法的控制装置、具有这种控制装置的选通摄像头装置和具有这种选通摄像头装置的机动车 |
EP22753651.3A EP4374191A1 (de) | 2021-07-20 | 2022-07-15 | Verfahren zum betreiben einer gated-kamera, steuervorrichtung zur durchführung eines solchen verfahrens, gated-kamera-vorrichtung mit einer solchen steuervorrichtung und kraftfahrzeug mit einer solchen gated-kamera-vorrichtung |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102021003728.6 | 2021-07-20 | ||
DE102021003728.6A DE102021003728B4 (de) | 2021-07-20 | 2021-07-20 | Verfahren zum Betreiben einer Gated-Kamera, Steuervorrichtung zur Durchführung eines solchen Verfahrens, Gated-Kamera-Vorrichtung mit einer solchen Steuervorrichtung und Kraftfahrzeug mit einer solchen Gated-Kamera-Vorrichtung |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023001711A1 true WO2023001711A1 (de) | 2023-01-26 |
Family
ID=82850258
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2022/069909 WO2023001711A1 (de) | 2021-07-20 | 2022-07-15 | Verfahren zum betreiben einer gated-kamera, steuervorrichtung zur durchführung eines solchen verfahrens, gated-kamera-vorrichtung mit einer solchen steuervorrichtung und kraftfahrzeug mit einer solchen gated-kamera-vorrichtung |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP4374191A1 (de) |
CN (1) | CN117616301A (de) |
DE (1) | DE102021003728B4 (de) |
WO (1) | WO2023001711A1 (de) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2161537A2 (de) * | 2008-09-05 | 2010-03-10 | Fujitsu Limited | Optische Positionsmessvorrichtung basierend auf die Projektion von Gittermustern |
DE102020002994A1 (de) * | 2020-05-19 | 2020-07-02 | Daimler Ag | Verfahren zur Messung eines Abstandes zwischen einem Objekt und einem optischen Sensor, Steuereinrichtung zur Durchführung eines solchen Verfahrens, Abstandsmessvorrichtung mit einer solchen Steuereinrichtung und Kraftfahrzeug mit einer solchen Abstandsmessvorrichtung |
US20210150279A1 (en) * | 2019-11-14 | 2021-05-20 | Zoox, Inc. | Depth data model training with upsampling, losses, and loss balancing |
DE102020004690A1 (de) * | 2020-08-03 | 2021-05-27 | Daimler Ag | Verfahren zur Erkennung von Objekten, Steuereinrichtung zur Durchführung eines solchen Verfahrens, Erkennungsvorrichtung mit einer solchen Steuereinrichtung und Kraftfahrzeug mit einer solchen Erkennungsvorrichtung |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102006060893A1 (de) | 2006-05-12 | 2007-11-15 | Adc Automotive Distance Control Systems Gmbh | Vorrichtung und Verfahren zum Bestimmen eines Freiraums vor einem Fahrzeug |
DE102019214404A1 (de) | 2019-09-20 | 2021-03-25 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Diagnostizierende beleuchtung |
-
2021
- 2021-07-20 DE DE102021003728.6A patent/DE102021003728B4/de active Active
-
2022
- 2022-07-15 CN CN202280048163.XA patent/CN117616301A/zh active Pending
- 2022-07-15 WO PCT/EP2022/069909 patent/WO2023001711A1/de active Application Filing
- 2022-07-15 EP EP22753651.3A patent/EP4374191A1/de active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2161537A2 (de) * | 2008-09-05 | 2010-03-10 | Fujitsu Limited | Optische Positionsmessvorrichtung basierend auf die Projektion von Gittermustern |
US20210150279A1 (en) * | 2019-11-14 | 2021-05-20 | Zoox, Inc. | Depth data model training with upsampling, losses, and loss balancing |
DE102020002994A1 (de) * | 2020-05-19 | 2020-07-02 | Daimler Ag | Verfahren zur Messung eines Abstandes zwischen einem Objekt und einem optischen Sensor, Steuereinrichtung zur Durchführung eines solchen Verfahrens, Abstandsmessvorrichtung mit einer solchen Steuereinrichtung und Kraftfahrzeug mit einer solchen Abstandsmessvorrichtung |
DE102020004690A1 (de) * | 2020-08-03 | 2021-05-27 | Daimler Ag | Verfahren zur Erkennung von Objekten, Steuereinrichtung zur Durchführung eines solchen Verfahrens, Erkennungsvorrichtung mit einer solchen Steuereinrichtung und Kraftfahrzeug mit einer solchen Erkennungsvorrichtung |
Also Published As
Publication number | Publication date |
---|---|
DE102021003728A1 (de) | 2023-01-26 |
CN117616301A (zh) | 2024-02-27 |
DE102021003728B4 (de) | 2023-04-20 |
EP4374191A1 (de) | 2024-05-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
DE102005016288B4 (de) | Vorrichtung zur Erfassung eines Objekts vor einem Fahrzeug | |
EP1928687B1 (de) | Verfahren und fahrerassistenzsystem zur sensorbasierten anfahrtsteuerung eines kraftfahrzeugs | |
EP0344404B1 (de) | Verfahren und Vorrichtung zur Steuerung der Türstellung einer automatischen Tür | |
DE102020002994B4 (de) | Verfahren zur Messung eines Abstandes zwischen einem Objekt und einem optischen Sensor, Steuereinrichtung zur Durchführung eines solchen Verfahrens, Abstandsmessvorrichtung mit einer solchen Steuereinrichtung und Kraftfahrzeug mit einer solchen Abstandsmessvorrichtung | |
WO2018069405A1 (de) | Kraftfahrzeug und verfahren zur 360°-umfelderfassung | |
DE102005056647B4 (de) | Fahrzeugumgebungsüberwachungsvorrichtung | |
DE102010039092B4 (de) | Verfahren und Steuergerät zum Ermitteln eines Abstandes eines Objektes von einem Fahrzeug | |
DE112014007047B4 (de) | Fahrzeugkamera-Steuervorrichtung | |
WO2013178407A1 (de) | Verfahren und vorrichtung zur verarbeitung stereoskopischer daten | |
DE102020004690A1 (de) | Verfahren zur Erkennung von Objekten, Steuereinrichtung zur Durchführung eines solchen Verfahrens, Erkennungsvorrichtung mit einer solchen Steuereinrichtung und Kraftfahrzeug mit einer solchen Erkennungsvorrichtung | |
WO2021239323A1 (de) | Verfahren zur erkennung von bildartefakten, steuereinrichtung zur durchführung eines solchen verfahrens, erkennungsvorrichtung mit einer solchen steuereinrichtung und kraftfahrzeug mit einer solchen erkennungsvorrichtung | |
DE102005056665B4 (de) | Fahrzeugumgebungsüberwachungsvorrichtung | |
EP3715779B1 (de) | Verfahren und vorrichtung zur bestimmung von verformungen an einem objekt | |
WO2017041795A1 (de) | Stereokameravorrichtung zur umgebungserfassung eines kraftfahrzeugs, kraftfahrzeug mit einer solchen stereokameravorrichtung sowie ein verfahren zur erkennung von regentropfen oder ablagerungen | |
DE102015223500A1 (de) | Verfahren und Vorrichtung zur Prüfung der Funktionalität einer außenseitigen Lichteinrichtung eines Fahrzeugs | |
DE102006004770B4 (de) | Verfahren zur bildgestützten Erkennung von Fahrzeugen im Umfeld eines Sraßenfahrzeugs | |
DE102021003728B4 (de) | Verfahren zum Betreiben einer Gated-Kamera, Steuervorrichtung zur Durchführung eines solchen Verfahrens, Gated-Kamera-Vorrichtung mit einer solchen Steuervorrichtung und Kraftfahrzeug mit einer solchen Gated-Kamera-Vorrichtung | |
WO2023247302A1 (de) | Verfahren zur ermittlung wenigstens einer korrekturfunktion für ein lidarsystem, lidar-system, fahrzeug mit wenigstens einem lidar-system, messanlage | |
DE102019116004A1 (de) | Verfahren und vorrichtung zur objekterkennung in blindzonen einer kamera | |
DE102021004516B3 (de) | Verfahren zum Betreiben einer Gated-Kamera, Steuervorrichtung zur Durchführung eines solchen Verfahrens, Gated-Kamera mit einer solchen Steuervorrichtung und Kraftfahrzeug mit einer solchen Gated-Kamera | |
EP4356163A1 (de) | Verfahren zum betreiben einer gated-kamera, steuervorrichtung zur durchführung eines solchen verfahrens, schulterblickvorrichtung mit einer solchen steuervorrichtung und kraftfahrzeug mit einer solchen schulterblickvorrichtung | |
DE102022002766B4 (de) | Verfahren zur dreidimensionalen Rekonstruktion einer Fahrzeugumgebung | |
DE102017005657A1 (de) | Vorrichtung und Verfahren zur Umgebungserfassung für ein Fahrzeug | |
DE102020007064B4 (de) | Verfahren zum Kalibrieren einer ersten Beleuchtungseinrichtung, einer zweiten Beleuchtungseinrichtung und eines optischen Sensors, Steuereinrichtung zur Durchführung eines solchen Verfahrens, Kalibrierungsvorrichtung mit einer solchen Steuereinrichtung und Kraftfahrzeug mit einer solchen Kalibrierungsvorrichtung | |
WO2022258527A1 (de) | Verfahren zum betreiben einer gated-kamera, steuervorrichtung zur durchführung eines solchen verfahrens, sichtweitenmessvorrichtung mit einer solchen steuervorrichtung und kraftfahrzeug mit einer solchen sichtweitenmessvorrichtung |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22753651 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202280048163.X Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2022753651 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2022753651 Country of ref document: EP Effective date: 20240220 |