CN113844357A

CN113844357A - Method, device, computer program and storage medium for identifying objects in the surroundings of a motor vehicle

Info

Publication number: CN113844357A
Application number: CN202110709846.2A
Authority: CN
Inventors: K·沃尔夫
Original assignee: Robert Bosch GmbH
Current assignee: Robert Bosch GmbH
Priority date: 2020-06-25
Filing date: 2021-06-25
Publication date: 2021-12-28
Also published as: DE102020207922A1

Abstract

The invention relates to a method for identifying objects in the surroundings of a motor vehicle, wherein the motor vehicle comprises a camera for recording the surroundings and comprises a lighting device for illuminating the surroundings, characterized in that at least a first lighting situation of the objects and a second lighting situation of the objects are set by means of a defined modulation of the lighting intensity of the lighting device, wherein the objects are identified taking into account the lighting situations. The invention also relates to a device for carrying out the method.

Description

Method, device, computer program and storage medium for identifying objects in the surroundings of a motor vehicle

Technical Field

Background

Patent application DE 102008002026 a1 is known from the prior art, for example. This document relates to a device for a motor vehicle, wherein the device comprises a headlight for illuminating the surroundings of the motor vehicle with light pulses and a camera for generating an image of the surroundings of the motor vehicle. The camera and the headlight are synchronized in such a way that the camera produces an image during the dark phase of the headlight. A camera and a method for generating an image are also presented.

Furthermore, patent application US 2020001775 AA is known from the prior art. This document relates to object recognition in an automated vehicle, wherein a light quantity of an object is determined and the illumination intensity of the headlight is increased if the light quantity is less than a threshold value.

Furthermore, patent application EP 0869031 a2 is known from the prior art. This document relates to a known method for adjusting the illumination range and the illumination direction of headlights in a motor vehicle, wherein a camera observes a traffic space located in front of the motor vehicle and having a traffic lane and detects image data. Based on the detected image data, the course of the road is determined and the headlight settings are adapted accordingly. Furthermore, it is known from EP 0869031 a2 to detect oncoming vehicles and to adjust the lighting in such a way that the oncoming vehicles do not suffer from glare.

Furthermore, patent application DE 102008023853 a1 is known from the prior art. This document relates to a method for identifying an object (for example a variable information sign), in which method at least one light source of the object is illuminated in a pulsed manner and light emitted by the light source can be detected by a detector unit and the object is identified on the basis of the detection of the light, wherein, for the detection of the light source, a plurality of exposures are carried out to generate an image by the detector unit and the total duration of the exposure process with the plurality of exposures is set to be greater than the duration of the stepwise exit (Ausphase) of the light source during the period duration of the pulsed emitted light of the light source.

The control of the exposure is an important part of the camera. Especially at night, it may be difficult to find the best compromise between objects of different brightness. In addition to existing lighting (e.g., street lights and other automotive headlights), there is also its own headlight light. In particular, retro-reflectors (retro-reflectors), such as the tail lights of road signs or other traffic participants, project their headlight light almost directly into the camera and thus appear very bright. It is therefore difficult for the auxiliary function to distinguish the object that performs reflection from the object that performs self-light emission. For example, if a traffic sign is recognized as a self-emitting light source, this results in it being erroneously recognizable as an oncoming traffic participant. This results in a high beam-assisted darkening, however, whereby the retroreflection at the traffic sign disappears. It is accordingly concluded that there are no oncoming traffic participants and that the high-beam assist is again faded on, which in turn leads to the start of the cycle, i.e. the traffic sign reflects light and is recognized as a self-luminous light source. This results in unnecessary fading, which reduces the useful life of the device and reduces customer acceptance of this function.

Disclosure of Invention

In contrast, the method according to the invention advantageously enables improved recognition and classification of objects in the surroundings of the motor vehicle. According to the invention, this is achieved by the features specified in the invention. Other configurations of the present invention are described below.

The method according to the invention for identifying objects in the surroundings of a motor vehicle, wherein the motor vehicle comprises a camera for recording the surroundings and comprises a lighting device for illuminating the surroundings, is characterized in that at least a first lighting situation of the object and a second lighting situation of the object are set by means of a defined modulation of the lighting intensity of the lighting device, wherein the object is identified taking into account the lighting situation.

The camera is understood here as a front video camera, for example. An image of the surroundings of the motor vehicle is recorded corresponding to the field of view of the camera. The object may be identified within the recorded image data. Identification is understood in particular as finding and identifying or classifying. The lighting device is in particular configured as an LED headlight. Other illuminations, such as lasers, are also contemplated. In particular, a lighting device with fast steerability is advantageous. The setting of the lighting conditions at the object is understood to be that the lighting device at least partly contributes to the lighting of the object. In this case, intentionally not illuminating the object is also understood to mean setting the illumination situation.

In particular, a first lighting condition is set by means of the generated first defined lighting intensity. Furthermore, a second lighting situation is set by means of the generated second defined lighting intensities, wherein these lighting intensities are different. Accordingly, the first and second lighting conditions relate to different lighting conditions.

Modulation is understood to be, for example, a change over time. In this case, changes can be made only at a specific time or within a specific time interval (i.e., after expiration of a specific time duration). Obviously, it is also possible to consider parameters that are only indirectly time-dependent as a reference, for example frames of a camera. A modulation of the illumination intensity is understood to be a change in the illumination intensity. Here, the variation may mean a decrease. In this case, reduction to zero should also be included together. Furthermore, the modulation may also include a change over a period of time. This means that not only the binary change at a defined time instant but also, for example, a continuous change at a certain, greater number of time instants.

Advantageously, the method is configured and arranged for supporting a driver assistance system, for example for supporting an assistance function for (highly) automated driving, for example a highwaipilot, or for supporting a comfort function, for example a high beam assistance.

In an advantageous embodiment, the method is characterized in that synchronization is carried out between the camera and the illumination device for the purpose of identifying the object.

This is understood to mean that coordination is performed between the lighting device and the camera. The synchronization may be configured as a unidirectional information flow with a corresponding change on one side. Obviously, the synchronization can also be configured as a bidirectional coordination.

For example, the lighting device may be coordinated with the camera. The influence of the headlight can be varied by suitable selection and implementation of the temporal relationship of the illumination intensity to the known exposure interval of the camera. Here, a change in a direction in which the own illumination portion is faded out may be considered. This advantageously enables a dynamic adaptation of the camera to the existing externally illuminated object. Furthermore, the self-luminous object is thereby better identified. Variations in the direction of ambient lighting fade out associated with self-illuminating objects may also be considered. This enables particularly better recognition of the retroreflector. Glare from intense light sources (e.g., oncoming vehicles) may be reduced.

Advantageously, the modulation of the lighting device is only set in the following categories: within this category, these changes can be perceived by a camera or image processor (imager).

The novel imager can also be operated at high frequencies. For example, fast (binary) modulation of the headlight can be achieved in a matched imager that can be operated in the LFM (LED Flicker suppression) operating mode. If the imager switches between light-sensitive and non-light-sensitive multiple times within a (conventional) frame (common application: identification of matrix displays), the illumination can be modulated in the same grating and thus suppression or amplification of the portion of the own illumination can be achieved as well. High frequency sinusoidal modulation may also be used if the imager allows sensitivity that enables fast modulation.

In addition, the camera may also be coordinated with the lighting device. For example, in the analysis processing of image data and classification of objects, the following information may be considered: this information relates to the illumination intensity set at the moment of recording of the generated image. Here, a plurality of image recordings may also be considered. Here, comparison of differences in images may be considered in the analysis process. Obviously, comparison of the development of the difference over time can also be considered.

Alternatively or additionally, a changing setting with respect to the camera can also be implemented at the moment of the illumination modulation. In this case, for example, a change in the exposure time can be achieved during the illumination modulation. For example, the defined illumination of the object is set by setting a specific illumination intensity of the headlights, and additionally a specific exposure of the camera is set in a manner coordinated with this illumination intensity.

Of course, a bidirectional information flow and a mutual matching in the lighting device and the camera may also be considered.

In one possible embodiment, the method is characterized in that the exposure of the camera is adapted, in particular in a manner coordinated with a defined modulation of the illumination intensity of the illumination device.

This means that the exposure of the camera is set taking into account the set or to be set illumination intensity of the headlight. For example, the exposure time is set here. Alternatively or additionally, an aperture is provided. In this sense, the modulation of the illumination is synchronized with the camera exposure to obtain an optimized result.

In a preferred embodiment, the method is characterized in that the light sensitivity of the image sensor of the camera is adapted, in particular in a manner coordinated with a defined modulation of the illumination intensity of the illumination device.

The adaptation of the light sensitivity of the image sensor can be understood as a special form of adaptation of the exposure of the camera. Here, the light sensitivity of the image sensor is changed in order to improve the correct recognition or classification of the object. In a simple case, it is possible here to switch between light-sensitive and non-light-sensitive. However, this can be understood in particular as follows: in these cases, the photosensitivity can in fact be matched as required, i.e. a specific photosensitivity (i.e. an intermediate value between photosensitivity and non-photosensitivity) can be set.

In an alternative embodiment, the method is characterized in that the retroreflection at the object is determined by evaluating the illumination situation.

This is understood to mean that the method is used to identify retro-reflection. The retro-reflector is in particular a traffic sign/guideboard. In addition, retro-reflectors are also present in the tail lights of other traffic participants. It has hitherto been difficult to distinguish objects that undergo reflection from self-luminous objects. The retroreflection is determined by this method. It can thus be recognized that the object to be classified, for example, does not have its own lighting device and that the light spot sought is based on retro-reflection. In consideration of these knowledge, misclassification of objects can be reduced.

In an advantageous embodiment, the method is characterized in that the modulation pattern of the illumination intensity of the illumination device is a stepped profile, wherein in particular the illumination intensity is reduced during the modulation or no illumination is carried out during the modulation.

This is understood to mean that the illumination intensity changes substantially abruptly during the modulation. Binary modulation may also be referred to herein. The binary modulation is performed, for example, as follows: no illumination occurs for a defined short time (e.g. 1 frame). Alternatively, it is also conceivable to set the illumination intensity to a defined reduced value (for example 50%) during this time. The modulation of the illumination is thus configured, for example, such that for every nth image of the camera the object to be illuminated is illuminated with reduced or no illumination.

In a possible embodiment, the method is characterized in that the modulation of the illumination intensity of the illumination device is configured in a coded form, wherein in particular the code configuration is a fixed order of the changed illumination intensities or alternatively the code configuration is a random order of the changed illumination intensities.

This is understood to mean that the illumination changes in a specific manner. The form corresponds to a predefined code. The code may be configured in binary, for example. Both a fixed order and a random order in the variation of the illumination intensity are conceivable here. The random sequence can represent an individualized identification, which advantageously enables differentiation, for example, when a plurality of vehicles illuminate the same surrounding area or the same object in a coded manner.

In a preferred embodiment, the method is characterized in that the modulation pattern of the illumination intensity of the illumination device is a continuous variation, wherein the continuous variation pattern is, in particular, a sinusoidally varying illumination intensity.

This means that, in the case of a continuous modulation of the illumination, the illumination brightness changes continuously — not in a sudden manner as in binary modulation. For example, the continuous variation is performed in a sinusoidal shape. The change can be effected in alternating directions (wechselgericht) (i.e. with a positive and negative amplitude relative to the conventional mean value) or else in the same direction (gleichgericht) (in particular as a negative amplitude relative to the conventional mean value, i.e. a reduction in the illumination intensity). The continuous modulation can be carried out, for example, over 5 frames as a period in which the brightness in the exposure time period is sinusoidally modulated. This can be detected in the analysis process. Obviously, longer periods (e.g. 10 frames or 15 frames) and shorter periods are also contemplated.

In an alternative embodiment, the method is characterized in that the modulation of the illumination intensity of the illumination device is carried out in such a way that the illumination intensity is imperceptible to the human eye.

The method can be implemented, for example, in the form of software or hardware or in the form of a mixture of software and hardware, for example, in a control device.

The solution proposed here also proposes an apparatus which is designed to carry out, control or carry out the steps of the variants of the method proposed here in a corresponding device. The object on which the invention is based can also be achieved quickly and efficiently by the embodiment variant of the invention in the form of a device.

In this context, a device can be understood as an electrical device: the device processes the sensor signals and outputs control signals and/or data signals accordingly. The device may have an interface, which may be constructed in hardware and/or in software. In the case of a hardware configuration, this interface can be part of a so-called system ASIC, for example, which contains the various functions of the device. However, it is also possible that the interface is an integrated circuit of its own, or is at least partly composed of discrete components. In the case of a software configuration, the interface can be, for example, a software module which is present on the microcontroller together with other software modules. The following may be considered devices: control device for a camera or a lighting device, driver assistance system, lighting device configured for carrying out the method. Likewise, a camera for carrying out the method is configured.

A computer program product or a computer program is also advantageous, which has a program code that can be stored on a machine-readable carrier or storage medium (such as a semiconductor memory, a hard disk memory, or an optical memory) and is used, in particular, when the program product or the program is executed on a computer or a device, to carry out, implement, and/or manipulate the steps of the method according to one of the embodiments described above.

Drawings

It should be noted that the features listed individually in the description can be combined with one another in any technically meaningful way and illustrate further configurations of the invention. Further features and uses of the invention emerge from the description of the embodiments on the basis of the figures.

Shown from the attached drawings:

FIG. 1 shows a schematic view of one driving situation; and is

FIG. 2 shows the course of the illumination intensity in the case of a dim illumination, an

FIG. 3a shows a stepwise modulation of the illumination intensity, and

FIG. 3b shows another step-wise modulation of the illumination intensity, an

FIG. 4a shows coded modulation of the illumination intensity in a fixed order, and

FIG. 4b shows coded modulation in random order, and

fig. 5a shows a sinusoidal modulation of the illumination intensity in alternating directions, and

fig. 5b shows a sinusoidal modulation of the illumination intensity in the same direction, and

figure 6 illustrates a method according to one embodiment of the invention.

Detailed Description

Fig. 1 shows a schematic representation of a driving situation. A motor vehicle 1 with a driver assistance system 2 (e.g. high beam assistance) is shown here. The motor vehicle 1 comprises a camera 4 and a lighting device 5. There is an information connection 6 between the camera 4 and the lighting device 5. The information connection 6 may be alternating or unidirectional. The information connection 6 can also be made by means of or by means of the control device 3. Obviously, it is also possible for a plurality of control devices 3 to contribute to this. The control device 3 can also be provided for controlling the lighting device 5 and/or the camera 4 or for evaluating image data of images of the surroundings 8 generated by means of the camera 4. In this case, an object 9 (for example a traffic sign) may be present in the surroundings 8 of the motor vehicle 1, which object is illuminated by means of the light 7 with a modulated illumination intensity. The reflection of the modulated light 7 generated at the object 9 is recorded by the camera 4.

The course of the illumination intensity in the case of a dimming of the illumination is shown in fig. 2. At time t1, an object is identified, which is identified as an oncoming vehicle in the glare region of the lighting device. Thus, the high beam assists in dimming. This means that the illumination intensity is reduced so that other traffic participants are not dazzled. At time t2, it is recognized that the object is no longer present, since the object has left the glare area or the field of view of the camera, for example. Accordingly, the high beam assist automatically fades up again. This means that the illumination intensity is increased in order to achieve a good illumination of the surroundings.

It should be noted here that representing the illumination intensity as a line may relate to a simplified representation. In particular in the case of LED headlights, this does not relate to continuous illumination, but to the combined action of light pulses perceived as continuous illumination. Such optical pulses should not be confused with the modulation described later. The modulation describes a variation of the conventional illumination, e.g. a change of the conventional light pulse. In particular, the modulation is not continuous. It is, however, obvious that the modulation may be performed regularly.

Fig. 3a and 3b show a step-wise modulation of the illumination intensity. This can also be understood as binary modulation. In this case, for example, every nth image of the camera is not illuminated by the headlights — or the illumination is reduced (for example, by 50%) as shown in fig. 3 b. Thus, when the recorded image data of the camera is processed analytically, the illuminated objects appear only alternately and can therefore be detected more simply — and correctly recognized as non-self-illuminating objects.

Fig. 4a and 4b show coded modulation of illumination intensity. This means that the illumination intensity varies in a defined pattern. Thereby enabling transmission of binary codes. In this way, in object recognition, the assignment as self-luminous objects can be guaranteed even better with respect to the objects to be illuminated. The code may be performed, for example, as shown in fig. 4a, in a fixed order over a plurality of brightness levels (i.e., different illumination intensities). For example, 5 bits may be set as the length of the code. Alternatively, the code may proceed in a random order of illumination intensity modulation (or also complete light/dark modulation), for example as shown in fig. 4 b. For example, if multiple vehicles illuminate the same scene, the random coding facilitates differentiation.

Fig. 5a and 5b show continuous modulation of the illumination intensity. This means that the illumination brightness changes continuously — not abruptly as in binary modulation. The continuous modulation can be performed, for example, as a sinusoidal modulation of the illumination intensity in alternating directions, as is shown in fig. 5 a. This means that the illumination intensity which is usually to be set in this case is not only increased but also reduced. Alternatively, the continuous modulation can be performed, for example, as a sinusoidal modulation of the illumination intensity in the same direction, as is shown in fig. 5 b. This means that the change is made in only one direction, for example a reduction of the illumination intensity that is usually to be set in this case (and of course a continuous return to the normal illumination intensity).

Fig. 6 shows a schematic representation of the method steps of an embodiment of the invention. In this case, the method is started in a first step S1, for example, the driver assistance system is started for high beam assistance. In step S2, the driver assistance system is implemented. In a first condition B1 it is checked whether there are criteria for aborting the method. If this is the case (yes branch), the method is aborted in a method step S4. If this is not the case (the "no" branch), the method continues. In a second condition B2, it is checked whether there is a criterion for carrying out object recognition by means of modulated light, for example, such modulation must take place at regular time intervals or at regular frame intervals. If this is not the case (the "no" branch), the method continues generally in step S2. However, if this is the case (yes branch), object recognition is carried out in step S3 by means of modulation. For this purpose, the illumination is modulated in step S3 a. Furthermore, the exposure of the camera can also be matched in step S3b (for example, the aperture or exposure time is matched). Likewise, the light sensitivity of the image sensor of the camera may be adapted in step S3 c. The image data generated by means of the modulation is subsequently evaluated in a step S3 d.

Claims

1. Method for identifying objects in the surroundings of a motor vehicle, wherein the motor vehicle comprises a camera for recording the surroundings and comprises a lighting device for illuminating the surroundings, characterized in that at least a first lighting situation of the objects and a second lighting situation of the objects are set by means of a defined modulation of the lighting intensity of the lighting device, wherein the objects are identified taking into account the lighting situations.

2. Method according to claim 1, characterized in that for identifying the object, a synchronization is performed between the camera and the lighting device.

3. Method according to any of the preceding claims, characterized in that the exposure of the camera is adapted, in particular in a manner coordinated with a defined modulation of the illumination intensity of the illumination device.

4. Method according to any of the preceding claims, characterized in that the light sensitivity of the camera's image sensor is matched, in particular in a manner coordinated with a defined modulation of the illumination intensity of the illumination device.

5. Method according to any of the preceding claims, characterized in that the retro-reflection at the object is found by means of an analytical processing of the illumination situation.

6. Method according to any of the preceding claims, characterized in that the modulation profile of the illumination intensity of the illumination device is a stepwise variation, wherein in particular the illumination intensity is reduced during the modulation or no illumination is performed during the modulation.

7. Method according to any of the preceding claims, characterized in that the modulation of the illumination intensity of the illumination device is configured in a coded form, wherein in particular the code configuration is a fixed order of changed illumination intensities or alternatively the code configuration is a random order of changed illumination intensities.

8. Method according to any of the preceding claims, characterized in that the modulation profile of the illumination intensity of the illumination device is a continuous variation, wherein in particular the continuous variation profile is a sinusoidally varying illumination intensity.

9. Method according to any of the preceding claims, characterized in that the modulation of the illumination intensity of the illumination device is done such that the illumination intensity is not perceptible to the human eye.

10. A device (2, 3, 6) arranged for carrying out the method according to any one of claims 1 to 9.

11. A computer program arranged to perform, implement and/or handle the method (300) according to any one of claims 1 to 9 when the program is implemented by the apparatus according to claim 10.

12. A machine-readable storage medium on which a computer program according to claim 11 is stored.