DE102020202542A1 - Environment recognition system of a motor vehicle, method for operating an environment recognition system, computer program product and motor vehicle - Google Patents

Abstract

An environment recognition system of a motor vehicle is described, with at least one projector which is set up to project linear light structures onto an object in front of the motor vehicle, and at least one camera which is arranged in such a way that it captures an area in front of the motor vehicle, the The camera is sensitive to at least a partial range of the frequency spectrum output by the at least one projector in order to detect reflections of the light projected onto the object by the projector, the projector having a modulatable lamp arrangement, with a control being provided which is connected to the at least one camera and the at least one projector is connected, the controller having a light modulation module that is configured to modulate the light emitted by the projector so that the projector has at least one first light structure with a first modulation state and at least one second Can emit light structure with at least one different second modulation state, the controller having an evaluation module for camera data which is set up to evaluate camera data from the first modulation state and the second modulation state separately from one another.

Description

An environment recognition system of a motor vehicle, a method for operating an environment recognition system, a computer program product and a motor vehicle are described here.

Environment recognition systems of a motor vehicle, methods for operating an environment recognition system, computer program products and motor vehicles of the type mentioned at the beginning are known in the prior art.

Environment recognition systems are used in motor vehicles for various purposes, including for autonomous driving systems and for driver assistance systems such as speed limit warning systems that read speed limits from signs and lane assistance systems that have to recognize lane markings.

In addition, light projection systems are known in motor vehicles that generate light structures and project them into the environment, e.g. on signs, walls or lanes.

One application already mentioned is lane assistance systems. A first known type are active lane assistance systems that actively support a driver of a suitably equipped motor vehicle. Such systems use interventions in a steering system, for example in order to prevent an unintentional lane change, that is to say a lane change without indicating a lane change intention by actuating a blinker on the part of the driver. Passive systems, that is to say purely warning systems, are also known, for example those that generate a vibration in a steering wheel and thus draw the driver's attention to the fact that he will leave his lane if he continues to drive in the same direction. Furthermore, lane assistance systems are known which combine both of the aforementioned systems and support the driver by means of optical projections onto the road, for example by projecting solid or non-solid lines onto the road. For example, a non-solid line can be covered with a projection of a solid line if a safe lane change is not possible, for example due to another motor vehicle in a target lane. Furthermore, combinations of the aforementioned types of lane assistance systems are known.

In the case of the latter systems that work with projections, it can happen that the projection interferes with a system for recognizing the surroundings, for example by means of a front camera, so that the projection outshines a structure to be recognized, for example it can be mistaken for an actual lane marking by the optical environment recognition system or the lane marker is marked as undetectable.

From the DE 10 2011 081 357 A1 a method for controlling a headlight of a vehicle is known which comprises a step of performing an image evaluation in order to identify a reflection zone of an object located in the vicinity of the vehicle in at least one image of the surroundings of the vehicle. The method further comprises a step of providing a control signal for controlling the headlight, the control signal being designed to intensify light emission by the headlight in the direction of the reflection zone.

The task is thus to develop environment recognition systems of a motor vehicle, methods for operating an environment recognition system, computer program products and motor vehicles of the type mentioned at the beginning so that environment recognition systems that have projectors for emitting light structures and camera-based lane recognition systems can more reliably recognize lane markings on a road.

The object is achieved by an environment recognition system of a motor vehicle according to claim 1, a method for operating an environment recognition system according to the independent claim 9, a computer program product according to the independent claim 13 and a motor vehicle according to the independent claim 14 .

An environment recognition system of a motor vehicle is described below, with at least one projector which is set up to project linear light structures onto an object in front of the motor vehicle, and at least one camera which is arranged in such a way that it captures an area in front of the motor vehicle, the The camera is sensitive to at least a partial range of the frequency spectrum output by the at least one projector in order to detect reflections of the light projected onto the object by the projector, the projector having a modulatable lamp arrangement, with a control being provided which is connected to the at least one camera and the at least one projector is connected, the controller having a light modulation module which is set up to modulate the light emitted by the projector so that the projector has at least one first light structure and at least one different from the first light structure che second light structure can emit, wherein the Control has an evaluation module for camera data, which is set up to evaluate camera data from the first light structure and the second light structure separately from one another.

The camera data can be camera images, but they can also contain structured data, for example recognized objects.

The object can be, for example, a sign, a wall or a roadway. The light structure can be linear or flat, among other things.

Because the at least one projector can generate different light structures, a corresponding camera records two different images of the same scene. An image with the first light structure then differs from an image with the second light structure, so that similarities and differences can be found in the two images. The method thus allows conclusions to be drawn about the similarities in both images, in particular with regard to the road surface. These similarities can be analyzed accordingly and therefore allow the identification of constant elements, which include lane markings on a road, among other things.

Due to the fact that a corresponding motor vehicle is generally driving while the environment recognition system is being used, the recordings of the first light structure and the second light structure regularly have an offset which is compensated according to a further development. According to a further development, such an offset can be compensated for by recognizing the same image areas.

According to another development, it is possible to compensate for an offset by shifting one of the images relative to the other by a certain value, the necessary offset being dependent on the time interval between the two images and the current driving speed of the motor vehicle. In this embodiment, the controller can be set up to determine or select a suitable offset based on the speed and the time interval between the two recordings that are compared with one another. The offset can be calculated or it can be taken from a corresponding table.

According to another further refinement, the controller can be designed to compensate or correct corresponding perspective distortions in the camera data.

In other further refinements, more than two light structures can be provided, for example a continuum of light structures, one or more determination parameters of the light structure being varied, for example with regard to a light structure geometry.

In a further refinement, the projector can be integrated into a headlight of a motor vehicle.

In another further refinement, it can be provided that the first light structure and the second light structure differ in their respective color spectrum and / or in their respective brightness.

By varying the color of the light structure, it is possible that at least in one of the modulation states the color of the projected light structure differs sufficiently from the color of the lane marking on the object.

By adapting the brightnesses, it is possible to avoid overexposure to the lane marking with the darker of the at least two brightnesses, so that the lane marking becomes visible in the camera data with at least the darker of the two brightnesses.

According to a further embodiment, the darker brightness can be completely dark, so that no visible light structure is emitted.

In a further refinement, it can be provided that the first light structure and the second light structure differ with regard to the projected contours.

If the projected contours differ, they can be more easily identified in the image and distinguished from lane markings. In this case it makes sense if the controller is set up to identify the lane markings in the recorded camera data. To this end, according to a further development, it can be provided that corresponding information on the currently emitted light structure is available in the evaluation module. The light structure shown in each case can then be more easily identified in the camera data.

In a further refinement, it can be provided that the modulation module is set up to carry out a change from the first light structure to the second light structure and back to the first light structure in 1/100 s to 1/30 s.

The periods mentioned are short enough not to be seen by the human eye. This makes it possible to carry out a corresponding modulation and evaluation without the driver noticing.

In a further refinement, it can be provided that the first light structure and the second light structure differ from one another in a frequency range that is not visible to humans.

The visible frequency range of light is between 380 nm and 780 nm wavelength. If a corresponding change takes place outside of this frequency range, for example in the infrared or ultraviolet range, the corresponding modulation between the first light structure and the second light structure cannot be perceived by humans.

In a further refinement, it can be provided that the controller has a timer which is connected to the modulation module and the evaluation module.

By using a timer that is connected to the modulation module and the evaluation module, the modulation module and evaluation module can be synchronized with one another. It is also possible to generate time stamps that allow the light structure displayed in each case to be assigned to a corresponding recording.

In a further refinement, it can be provided that the controller is set up to send a status signal for the first light structure and the second light structure to the light modulator module and / or the evaluation module.

A corresponding status signal can be evaluated by the evaluation unit by associating the status signal with the corresponding camera data.

In a further refinement, it can be provided that the projector has at least one dimmable and / or color-variable illuminant and / or several illuminants that can be activated and deactivated individually or in groups.

If the projector has at least one dimmable and / or color-variable illuminant and / or several illuminants that can be activated and deactivated individually or in groups, it is possible to generate light structures of different geometries, different colors and / or different brightnesses. The controller can then be set up to control the corresponding lighting means in such a way that they are switched on or off, emit a certain brightness and / or a certain luminous color.

A first independent subject relates to a method for operating an environment recognition system of the type described above, wherein a first light structure is output from the at least one projector, a second light structure then being output from the at least one projector, the first light structure from the at least one camera and the second light structure are recorded, at least one camera recording of the first light structure being evaluated by the controller, at least one camera recording of the second light structure being evaluated by the controller, differences between the camera recording of the first light structure and the second light structure being recognized and evaluated.

This makes it possible to reliably identify lane markings on roads, even if an environment recognition system emits corresponding light structures that overlap with the lane markings.

In a first further refinement, it can be provided that overlapping areas are identified in the at least two camera recordings and lane markings are recognized in the camera recordings.

By recognizing overlapping areas, two different recordings can be appropriately aligned with one another and the recognition of lane markings on the basis of the different light structures is possible with greater reliability.

In a further refinement, it can be provided that the first light structure is emitted for 90% to 99% of the time in a specific time interval, the second light structure being emitted for the remaining time.

This makes it possible to essentially emit the first light structure that is to be perceived accordingly by the driver. The second light structure can only be displayed for a short time in order to enable the system to reliably recognize lane markings.

In a further refinement, provision can be made for the light structure to be changed if a level of confidence for the detection of lane markings falls below a predetermined value.

This makes it possible to change the light structure only if this is necessary for the reliability of the system.

Another independent subject matter relates to a device for operating an environment recognition system of the type described above, wherein a projector is set up to output a first light structure and then output a second light structure, a camera is provided for recording the first light structure and the second light structure, one Control for evaluating at least one camera recording of the first light structure and at least one camera recording of the second light structure is provided, the controller being set up to detect and evaluate differences between the camera recordings of the first light structure and the second light structure.

In a first further refinement, it can be provided that the controller is set up to identify overlapping areas in the at least two camera recordings and to recognize lane markings in the camera recordings.

In a further refinement, it can be provided that the controller is set up to control the projector in such a way that it is set up so that the first light structure is emitted for 90% to 99% of the time in a certain time interval, the controller for this purpose is set up to control the projector so that it is set up to emit the second light structure for the remaining time.

In a further refinement, it can be provided that the control is set up to determine a confidence level for the detection of lane markings, the control being further set up to control the projector so that it is set up so that a change in the light structure is performed when a level of confidence in the detection of lane markings falls below a predetermined value.

Another independent subject matter relates to a computer program product with a computer-readable storage medium on which instructions are embedded which, when executed by at least one processing unit, have the effect that the at least processing unit is set up to carry out the method of the aforementioned type.

The method can be carried out on one or more processing units, so that certain method steps are carried out on one processing unit and other method steps are carried out on at least one further processing unit, with calculated data being able to be transmitted between the processing units if necessary.

Another independent subject matter relates to a motor vehicle with a computer program product of the type described above and / or an environment recognition system of the type described above.

• 1 eine Draufsicht auf ein Kraftfahrzeug mit einem Umgebungserkennungssystem;
• 2 eine Steuerung des Umgebungserkennungssystems des Kraftfahrzeugs aus 1;
• 3 eine Draufsicht auf eine Straße mit dem Kraftfahrzeug aus 1;
• 4 Frequenzspektren zweier Lichtstrukturen, sowie
• 5 Frequenzspektren zweier Lichtstrukturen.

Further features and details emerge from the following description, in which - possibly with reference to the drawing - at least one exemplary embodiment is described in detail. Described and / or graphically represented features form the subject on their own or in any meaningful combination, possibly also independently of the claims, and in particular can additionally also be the subject of one or more separate applications. Identical, similar and / or functionally identical parts are provided with the same reference symbols. They show schematically:

• 1 a plan view of a motor vehicle with an environment recognition system;
• 2 a control of the environment recognition system of the motor vehicle 1 ;
• 3 a plan view of a road with the motor vehicle 1 ;
• 4th Frequency spectra of two light structures, as well as
• 5 Frequency spectra of two light structures.

1 shows a plan view of a motor vehicle 2 with an environment recognition system 4th (Components of the environment recognition system 4th framed with dashed lines).

The environment recognition system 4th points in 1 invisible projectors that are in headlights 6th , 8th are arranged. The projectors in the headlights 6th , 8th are set up to project linear structures with a sufficiently high contrast onto an object, here a roadway, so that they are perceptible and distinguishable by a driver. This allows the driver of the motor vehicle 2 Help can be given when changing lanes and, for example, indicate when a lane can be changed, for example by means of a projected dashed line, and when not, for example by a projected solid line, if a danger has been detected. The environment recognition system can also indicate where the lane marking is located, for example in poor visibility conditions such as a snow-covered roadway, and the like.

The environment recognition system 4th also has a front facing camera 10 on, which is an area of a roadway in front of the motor vehicle 2 can capture.

The headlights 6th , 8th as well as the camera 10 are with a controller 12th connected to the controller 12th an arithmetic unit 14th as well as a memory 16 having. In the memory 16 A computer program product is stored which, when it is processed by the processing unit 14th is loaded and executed, perform the procedure described here.

Using the camera 10 are recordings of the front of the motor vehicle 2 lying street made and in control 12th , evaluated by recognizing lane markings recorded on the road. In an alternative exemplary embodiment, this evaluation can be carried out in the camera 10 be made. Special hardware can be used for this, for example special graphics processors that are specialized in image analysis and object recognition.

The light structures generated by the projectors and projected onto the road, for example lines, can cover the lane markings so that the projections allow the camera to recognize the lane markings, at least according to the type of lane marking 10 or control 12th can prevent.

In this case, it is possible to modify the light structures with regard to their physical properties so that lane markings can be recognized by the camera 10 recorded images is possible. In the exemplary embodiment presented here, a change between different light structures takes place at regular time intervals.

In alternative embodiments, it may be possible to change the physical properties of the light structure only if a level of confidence in the detection of lane markings by the corresponding projected light structures drops too far, for example below a threshold value of 95%.

2 shows a schematic diagram of the control 12th .

The modules shown below can be program components of the aforementioned computer program product. In alternative configurations, the modules can be implemented entirely or partially by means of dedicated components, for example processors and / or circuit boards.

The camera 10 is with an evaluation module 18th the control 12th tied together. The evaluation module 18th serves to analyze the image information and to recognize the corresponding lane markings.

There is also a light modulator module 20th provided that controls a projector. The projector 22nd is in the headlight 6th arranged.

The projector 22nd has a matrix of light sources 23 on that from the light modulator module 20th can be switched on and off individually. The corresponding lamps are dimmable and can therefore emit different brightnesses. The lighting means can also be color-variable in other configurations.

There is also a timer 24 provided with the evaluation module 18th and the light modulation module 20th connected is.

Evaluation module 18th , Light modulation module 20th , Timer 24 and memory 16 are with the arithmetic unit 14th tied together. From the camera 10 Recorded images are recorded by the evaluation module 18th processed and the evaluated images, which are enriched with object information on objects identified in the recorded images, to the computing unit 14th to hand over. The arithmetic unit 14th stores this information in memory for a short time 16 away.

Then the arithmetic unit 14th a change in the light structure triggered by the light modulator module 20th is controlled accordingly, whereby the projector 22nd gives off a changed light structure.

The camera 10 takes another picture with the correspondingly changed light structure, the picture is taken by the evaluation module 18th processed as described above and sent to the processing unit 14th to hand over. The further picture can also be in the memory 16 stored or cached.

The arithmetic unit 14th can then then transfer the data for the first recording and, if necessary, the second recording from the memory 16 and compare the two data sets with each other.

There is an offset of the two recordings, which is caused by the movement of the motor vehicle 2 resulting other recording location is compensated by taking the time interval between the two recordings and the driving speed of the motor vehicle 2 taking into account the optical properties of the camera 10 an offset is determined and one of the two recordings is shifted relative to the other by the offset. If If necessary, an image distortion can also be corrected.

In an alternative embodiment, the offset can be compensated for by image recognition.

The differences found therein allow the light structures to be recognized and the light structures to be differentiated from the lane markings on the road, so that reliable recognition of the lane markings is possible.

3 shows a representation of the motor vehicle 2 the end 1 on a street 26th .

The street 26th has a lane marker 28 on that from the camera 10 of the environment recognition system 4th is recognized.

The car 2 emitted through the headlight 6th a cone of light 30th and about the projector located in it 22nd a light structure 32 , here a solid line, at the height of the lane marking 28 . Depending on the color and brightness, it may be for the camera 10 through the light structure 32 become impossible, the type of lane marking 28 reliably recognizable.

By varying the light structure 32 , in the simplest case by switching off the projector 22nd or by dimming the projector 22nd , can be achieved that the lane marking 28 stands out from the light structure.

4th shows frequency spectra of two light structures in an intensity and frequency diagram.

The light structure 32 has a first frequency spectrum that differs from a frequency spectrum of a second light structure 34 differs significantly. This changes the projections of the light structures 32 and 34 on the street 26th to the extent that a reliable detection of the lane marking 28 is possible.

5 also shows frequency spectra of two light structures.

A third light structure 36 has essentially the same spectrum as the first light structure, but with a significantly lower intensity than the light structure 32 . The third light structure 36 is therefore considerably darker than the light structure 36 . This makes it possible to change the light structure 36 from the light structure 32 in the street 26th to distinguish and a recognition of the lane marking 28 is possible.

Although the subject matter was illustrated and explained in more detail by means of exemplary embodiments, the invention is not restricted by the disclosed examples and other variations can be derived therefrom by the person skilled in the art. It is therefore clear that there is a multitude of possible variations. It is also clear that embodiments cited by way of example only represent examples which are not to be interpreted in any way as a limitation, for example, of the scope of protection, the possible applications or the configuration of the invention. Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without the To leave the scope of protection that is defined by the claims and their legal equivalents, such as a more detailed explanation in the description.

List of reference symbols

2

Motor vehicle

4th

Environment recognition system

6th

Headlights

8th

Headlights

10

camera

12th

steering

14th

Arithmetic unit

16

Storage

18th

Evaluation module

20th

Light modulator module

22nd

projector

23

Bulbs

24

timer

26th

Street

28

Lane marking

30th

Cone of light

32

Light structure

34

Light structure

36

Light structure

f

frequency

I.

intensity

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102011081357 A1 [0007]

Claims (14)

Environment recognition system of a motor vehicle (2) with at least one projector (22) which is set up to project linear light structures (32, 34, 36) onto an object (26) in front of the motor vehicle (2), and at least one camera (10) which is arranged in such a way that it detects an area in front of the motor vehicle (2), the camera (10) being sensitive to at least a partial area of the frequency spectrum output by the at least one projector (22) in order to detect reflections of the frequency spectrum from the projector (22 ) to detect light projected onto the object (26), the projector (22) having a modulatable illuminant arrangement (23), a control (12) being provided which is connected to the at least one camera (10) and the at least one projector ( 22) is connected, the controller (12) having a light modulation module (20) which is set up to modulate the light emitted by the projector (22) so that the projector (22) has at least one first light t structure (32) and at least one second light structure (34, 36) different from the first light structure (32), the controller (12) having an evaluation module (18) for camera data which is set up to receive camera data from the first light structure (32) and the second light structure (34, 36) to be evaluated separately from one another. Environment recognition system according to Claim 1 , the first light structure (32) and second light structure (34, 36) differing in their respective color spectrum (f) and / or in their respective brightness (I). Environment recognition system according to Claim 1 or 2 , wherein the first light structure (32) and the second light structure (34, 36) differ with regard to the projected contours. Environment recognition system according to one of the preceding claims, wherein the modulation module (20) is set up to carry out a change from the first light structure (32) to the second light structure (34, 36) and back to the first light structure in 1/100 s to 1/30 s . Environment recognition system according to one of the preceding claims, wherein the first light structure (32) and the second light structure (34, 36) differ from one another in a frequency range (f) which is not visible to humans. Environment recognition system according to one of the preceding claims, wherein the controller (12) has a timer (24) which is connected to the modulation module (20) and the evaluation module (18). Environment recognition system according to one of the preceding claims, wherein the controller (12) is set up to send a status signal for the first light structure (32) and the second light structure (34, 36) to the light modulation module (20) and / or the evaluation module (18) send. Environment recognition system according to one of the preceding claims, wherein the projector (22) has at least one dimmable and / or color-variable illuminant and / or several illuminants that can be activated and deactivated individually or in groups. Method for operating an environment recognition system according to one of the preceding claims, wherein a first light structure (32) is output from the at least one projector (22), a second light structure (34, 36) subsequently being output from the at least one projector (22), wherein the first light structure (32) and the second light structure (34, 36) are recorded by the at least one camera (10), with at least one camera recording of the first light structure (32) being evaluated by the controller (12), with at least one camera recording the second light structure (34, 36) is evaluated by the controller (12), differences between the camera recordings of the first light structure (32) and the second light structure (34, 36) being recognized and evaluated. Procedure according to Claim 9 , wherein overlapping areas are identified in the at least two camera recordings and lane markings (28) are recognized in the camera recordings. Procedure according to Claim 9 or 10 wherein in a given time interval the first light structure (32) is emitted for 90% to 99% of the time, the second light structure (34, 36) being emitted for the remaining time. Method according to one of the Claims 9 until 11 wherein the light structure (32, 34, 36) is changed when a confidence level for the detection of lane markings (28) falls below a predetermined value. Computer program product with a computer-readable storage medium (16) on which instructions are embedded which, when executed by at least one processing unit (14), have the effect that the at least one processing unit (14) is set up to implement the method according to one of the preceding To carry out claims. Motor vehicle with a computer program product Claim 13 and / or one Environment recognition system according to one of the Claims 1 until 9 .

Images