EP3353013A1 - Projection of a pre-definable light pattern - Google Patents
Projection of a pre-definable light patternInfo
- Publication number
- EP3353013A1 EP3353013A1 EP16770285.1A EP16770285A EP3353013A1 EP 3353013 A1 EP3353013 A1 EP 3353013A1 EP 16770285 A EP16770285 A EP 16770285A EP 3353013 A1 EP3353013 A1 EP 3353013A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- vehicle
- projection
- pixel
- observer
- determined
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000012876 topography Methods 0.000 claims description 15
- 230000008859 change Effects 0.000 claims description 5
- 241000287107 Passer Species 0.000 claims description 4
- 230000006870 function Effects 0.000 description 29
- 238000004364 calculation method Methods 0.000 description 8
- 239000011159 matrix material Substances 0.000 description 6
- 230000004913 activation Effects 0.000 description 5
- 239000013598 vector Substances 0.000 description 5
- 238000010276 construction Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000006978 adaptation Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 230000001419 dependent effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000002310 reflectometry Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000003466 anti-cipated effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000036642 wellbeing Effects 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q1/00—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor
- B60Q1/02—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to illuminate the way ahead or to illuminate other areas of way or environments
- B60Q1/04—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to illuminate the way ahead or to illuminate other areas of way or environments the devices being headlights
- B60Q1/06—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to illuminate the way ahead or to illuminate other areas of way or environments the devices being headlights adjustable, e.g. remotely-controlled from inside vehicle
- B60Q1/08—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to illuminate the way ahead or to illuminate other areas of way or environments the devices being headlights adjustable, e.g. remotely-controlled from inside vehicle automatically
- B60Q1/085—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to illuminate the way ahead or to illuminate other areas of way or environments the devices being headlights adjustable, e.g. remotely-controlled from inside vehicle automatically due to special conditions, e.g. adverse weather, type of road, badly illuminated road signs or potential dangers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q1/00—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor
- B60Q1/02—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to illuminate the way ahead or to illuminate other areas of way or environments
- B60Q1/04—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to illuminate the way ahead or to illuminate other areas of way or environments the devices being headlights
- B60Q1/14—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to illuminate the way ahead or to illuminate other areas of way or environments the devices being headlights having dimming means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q1/00—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor
- B60Q1/26—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic
- B60Q1/50—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic for indicating other intentions or conditions, e.g. request for waiting or overtaking
- B60Q1/549—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic for indicating other intentions or conditions, e.g. request for waiting or overtaking for expressing greetings, gratitude or emotions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q1/00—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor
- B60Q1/26—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic
- B60Q1/50—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic for indicating other intentions or conditions, e.g. request for waiting or overtaking
- B60Q1/547—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic for indicating other intentions or conditions, e.g. request for waiting or overtaking for issuing requests to other traffic participants; for confirming to other traffic participants they can proceed, e.g. they can overtake
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q2300/00—Indexing codes for automatically adjustable headlamps or automatically dimmable headlamps
- B60Q2300/20—Indexing codes relating to the driver or the passengers
- B60Q2300/23—Driver's line of sight
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q2300/00—Indexing codes for automatically adjustable headlamps or automatically dimmable headlamps
- B60Q2300/30—Indexing codes relating to the vehicle environment
- B60Q2300/32—Road surface or travel path
- B60Q2300/324—Road inclination, e.g. uphill or downhill
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q2400/00—Special features or arrangements of exterior signal lamps for vehicles
- B60Q2400/50—Projected symbol or information, e.g. onto the road or car body
Definitions
- the present invention relates to a method for projecting a predeterminable light pattern by means of a lighting device of a vehicle into an environment of the vehicle. Moreover, the present invention relates to a lighting device for a vehicle with a lighting unit and a control device for controlling the lighting unit. Furthermore, the present invention relates to a motor vehicle with such a lighting device.
- Document EP 1 916 153 A2 discloses a method for displaying information, in which a projection object is produced at a projection location outside a motor vehicle by means of projection means provided on the motor vehicle. By means of a generated projection object, an anticipated future location of the motor vehicle is marked areally.
- document DE 10 2006 050 548 A1 discloses a method for warning other road users.
- a projection by means of a projection object is generated as a warning for another road user at a projection point outside the motor vehicle.
- An angle of view of a particular viewer is taken into account in order to produce an undistorted image.
- the object of the present invention is thus to provide a method with which a lighting device of a vehicle can be controlled more flexibly with regard to the surroundings of the vehicle. According to the invention, this object is achieved by a method according to claim 1.
- the invention provides a lighting device according to claim 7.
- a specifiable light pattern is projected into the surroundings of the vehicle.
- This light pattern can serve purely for the presentation of information or specifically illuminate parts of the environment for better perception by a viewer, in particular the driver.
- the perspective of a predetermined observer is explicitly taken into account.
- the projection surface e.g., projection plane
- a position of the predetermined or potential observer is determined, for example, by means of sensor technology or provided, for example, by means of already stored data.
- a pixel-by-pixel control of a lighting unit eg of a pixel spotlight
- a lighting unit eg of a pixel spotlight
- the concrete perspective or representative of the projection surface or plane and the position of the observer are taken into account.
- the predetermined observer is a driver or passenger of the vehicle or a pedestrian at a predetermined position outside the vehicle or a person in one passing vehicle.
- the head position which is indeed relevant to the actual perspective, can be obtained accurately and simply from sensor data of the vehicle interior or the vehicle environment.
- a change in position of the vehicle or of an object in the surroundings of the vehicle can be taken into account. This allows the process to be dynamically updated. In this case, for example, a trajectory of the vehicle or of the object can be used.
- a topography in front of the vehicle can be analyzed.
- the environment in front of the vehicle is sensory-recorded, and from the data, a suitable projection plane or surface is extracted.
- a property of the environment can be taken into account for the pixel-by-pixel control of the lighting unit of the lighting device.
- a property may be, for example, the nature of a projection surface, such as reflectance or granularity of the surface.
- a property of the environment may be an actual traffic density, the nature of a road, a point-of-interest, the humidity of the traffic lane, or the ambient brightness. Such a property allows the projection of the light pattern to be further individualized, thus providing even more benefits to the observer.
- the lighting device can have a plurality of headlights as the lighting unit, and in the pixel-by-pixel control of the lighting device, the headlights can be controlled such that the predeterminable light pattern results on the projection surface. This can therefore be several headlights for the generation of Light pattern, which can improve the variability or the resolution of the light pattern.
- the lighting device according to the invention can be further developed functionally by the method features described above.
- a vehicle may be equipped with this lighting device.
- Fig. 1 is a Lochantimodell
- FIG. 2 shows the geometry for the calculation of the coordinates in a side view and in a plan view
- 4 shows a diagram for method selection
- 5 shows a projection in side view
- FIG. 6 shows the projection of FIG. 5 in a front view
- Fig. 7 individual projection steps
- Fig. 1 1 is a diagram for calculating the pixel intensity.
- the embodiments described in more detail below represent preferred embodiments of the present invention. It should be noted that the individual features can be realized not only in the described combination, but also in isolation or in other technically meaningful combinations.
- the invention is based on the approach that a dipped beam should be high resolution and can be realized for example by a pixel headlight. This makes it possible, for example, to provide perspectives which are adapted to the environment. In addition, interactions with the environment can occur. Furthermore, the road topology and the distance to other vehicles can be taken into account in order to achieve a situation-dependent control of the lighting device, in particular of a vehicle.
- the central idea here is that the concrete perspective is determined by taking into account the position of the observer but also the current concrete projection plane or area in the surroundings of the vehicle for the pixel-by-pixel control of the lighting unit of a lighting device.
- a first parameter for the control of the lighting unit of the lighting device may be a change in position of the own vehicle or ego vehicle. This parameter is important so that the desired light distribution is always projected at the same horizontal and vertical distance. For example, a pitch, roll, and roll motion can be detected via acceleration sensors. For this parameter, the eye / head position X f, Y f, Z f ... may be updated as well as the viewing direction are determined in accordance with a vector r.
- a second parameter would be the position change of objects such as the lane markings. The importance of this parameter lies in the fact that the objects are references to which a light distribution can orient.
- the topography T s can be determined from this parameter.
- a third parameter would be the topography in front of the vehicle in the first place. This includes the detection of other projection surfaces except the road, such. B. a garage wall. The vertical and horizontal size v, h of the light distribution can be determined from this. The exact knowledge about the geometry of the projection surface is necessary for the calculation of a correct, rectified light image.
- the topography T s can also be determined from this parameter.
- An important parameter is also the condition of the projection surface. This includes the reflectance and the reflection properties as well as the granularity of the surface. Due to the reflection properties, the required light intensity and the image quality can be determined. The visibility S s , the reflectivity R s and the luminance L s of the projection surface can be determined from this parameter.
- the head position of the driver the driver, the passenger, a passer or a person in another vehicle. Due to the reflection properties, the required light intensity and the image quality can be determined.
- the visibility S s , the reflectivity R s and the luminance L s of the projection surface can be determined from this parameter.
- the parameter h p observeer head height
- Q viewing direction in respect to the road
- ⁇ lateral viewing direction
- b distance between the observer head and the image plane
- X f From the eye and head position of one or more observers: X f , Y f , Z f .
- the line of sight vector r can be co-determined by one or more observers.
- the speed v f of one or more observers can be determined from this.
- Another parameter can specify the current traffic density. From this parameter results, which projection distance is to be selected. At high traffic density the projection distance is shortened and at low traffic density (eg on country road) the projection distance can be increased.
- Size v, h of the light distribution can be determined from this.
- the visibility S s of a light function can also be determined from this parameter.
- a decision as to which function F should be displayed can be derived therefrom.
- another parameter concerns the type of road and, where appropriate, 'points-of-interest' (urban environment, motorway, highway, construction site, accident, etc.) Depending on this parameter, activation of the desired lighting function may take place.
- a decision as to which function F is to be displayed derive therefrom.
- a further parameter can relate to the humidity of the lane. depending on an activation of the desired light function or a consideration in the illumination can be effected.
- the light function, the reflectivity R s and the luminance L s of the projection surface can be co-determined from this parameter Furthermore, a decision as to which function F is to be displayed can be derived from this j)
- the ambient brightness (illumination of a vehicle, street lamps, twilight , Daylight) represent another parameter. With him can an adjustment of the light intensity and an activation of a desired light function done. The luminance of the ambient brightness L f can be determined therefrom.
- a parameter may be the distance and / or the coordinates to a preceding or oncoming vehicle and the vehicle topology. With this parameter, an exact display of a vehicle can take place.
- the vertical and horizontal size v, h of the light distribution can be determined from this.
- the topography T s can also be determined from this parameter.
- Another parameter may be the user request or a user request. This / this can be on the operation of a button, eye tracking, driving behavior, voice recognition, well-being and the like recognize. Thus, the activation or adaptation of a desired light function / light distribution can take place. Likewise, a decision as to which function F should be displayed can be derived therefrom. m) Also vehicle data such as GPS and speed can be an important parameter. This is an activation or adaptation of a desired light function / light distribution possible. The speed of the vehicle occupant v v can be co-determined from this. In addition, a decision as to which function F should be displayed can be derived therefrom. n) Another parameter is the interior lighting.
- the apron of the low beam can be made lighter.
- the luminance of the interior lighting L f can be determined from this parameter.
- the distance of a desired projection to the ego vehicle is conceivable as another parameter. It can be used to calculate the overlap of the left and right headlamps to correctly display the light distribution.
- the topography T s can also be determined from this parameter.
- As a further parameter can serve a decision input, which represents the decision of the driver, who should be an observer of the light function. This input can be done, for example, by pressing a button, an MMI (Human Machine Interface) or an APP. This can be decided by the driver himself, whether z. B. a logo projection from his point of view should be equalized perspective or from the perspective of a passer. This decision is the Superordinate function mapping (see below).
- the decision parameter E can be determined from this.
- Different lighting functions can be defined. These lighting functions are each assigned a corresponding viewing perspective. This assignment results in a so-called “function mapping.” For example, the "construction site light" function is assigned the driver's perspective, since the lighting of a construction site is primarily of importance to the driver.
- the projection of a logo may, for example, be assigned to a driver's perspective in a default setting, if necessary this assignment can be changed by the MMI / APP, so that, for example, also a passerby sees the logo well.
- the above table shows that the perspectives that arise for projecting a light pattern from a vehicle can be classified, for example, into four areas or categories. This results in perspectives or functions for a driver, a passenger, a foreign vehicle or a pedestrian.
- the respective function then ensures that the projection is not distorted from the corresponding perspective. In some situations, however, it may happen that the function can not be displayed for the desired perspective. Then it has to be decided whether the lighting function is provided or not.
- a camera center O c which corresponds in the present case, the optical center, for example, a headlamp.
- O w the optical center
- a 3D point cloud is to be generated, each point of which is represented, for example, by a value x w .
- Such a point is located in the real world in a coordinate system O w .
- a 2D plane can be defined by which each projection beam passes from the headlight center O c to the 3D point x w and defines a point m there.
- This pinhole camera model can be used to convert a particular image on the road into a two-dimensional image relative to the headlight plane.
- the two-dimensional coordinates can be calculated using the following equations, where u and v belong to the image plane coordinates and k to the intrinsic camera parameters. ⁇ corresponds to the extrinsic camera parameter.
- the street scene is represented by a point cloud, the values being stored in X, Y and Z.
- R and t correspond to a rotation between the camera and the world frame coordinates.
- the angle values can be converted into pixel positions for any headlight technology and stored in a corresponding image or video format.
- the method also uses a trigometrische approach for calculating the two-dimensional coordinates X p, Z p, wherein each pixel ⁇ ,, Z, is projected for a particular observer head position and direction of view on the road. It is assumed that the image plane is perpendicular to the viewing direction of the observer.
- the coordinate X for example, of the driving tool longitudinal direction or direction of travel, the coordinate Z of the roadway transverse direction and the coordinate Y associated with the height direction of the vehicle.
- Fig. 2 now shows the geometry for calculating the two-dimensional coordinates X p and Z p on a projection plane, z. B.
- h p corresponds to the observer head height
- ⁇ the viewing direction with respect to the road
- ⁇ the lateral viewing direction
- b the distance between the observer's head and the image plane
- h b represents the lower part of the image and the road.
- An error of approximately 2.5% for angles (a or ß) up to 14 ° is to be expected. This error is calculated from the distance between the pixels of the original image and those calculated from the driver's perspective using this method.
- ⁇ represents the angle between the observer (s) BO from the center of the road surface or projection plane PE, whereupon the image Bl is projected.
- the coordinates of each corner of the portion of the road surface, ie the projection plane PE on which the image Bl is projected, are:, Zi ... X n , Z n .
- an additional object OB is located on the projection plane.
- the topography T s then results according to the uppermost illustration in FIG. 3.
- the vehicle is illuminated by the two headlights View or beam direction vectors Si and S 2 symbolizes. It moves at the speed v s .
- the topography T s for the surroundings in front of the vehicle is obtained with regard to the projection beam in the side view according to the middle image of FIG. 3.
- the lower image of FIG. 3 shows the reflection beam which the observer BO perceives. This leads, together with all other reflection beams in an image plane, to an image Bl.
- the projection beam and the reflection beam result in the corresponding perspective of the observer BO.
- the observer BO can influence a decision parameter E.
- a workflow for the geometric calculation at a specific time t In a first step S1, input parameters such as the function F (eg construction site light, logo representation etc.), the topography T s or the decision parameter E (eg driver or passerby as observer) are provided. In a subsequent step S2, it is automatically or manually decided whether projecting is desired. For ex- ample, it is undesirable if the topography does not fit or if a car is driving ahead. In the subsequent step S3, when the projection is desirable physical parameters such as v f, L f, S s, R s, U, V s or the like are calculated. In the subsequent step S4, it is decided whether the projecting is still possible.
- the function F eg construction site light, logo representation etc.
- the topography T s or the decision parameter E eg driver or passerby as observer
- step S5 a query is made in step S5 as to whether projecting is still desired. If not, then step S1 is returned. If projecting is still desired, in step S6, observer parameters and projection surface parameters are extracted from the input parameters of step S1. These include the head position and the viewing direction on the one hand and the luminance on the other. It is then checked whether the projection plane is the same as the image plane. If not, the following procedure 1 is performed. If, on the other hand, the projection plane is the same as the image plane and the observer For example, looking perpendicular to the road onto which the image or light pattern is projected, method 2 is applied. The method 2 is based on the hole camera model and is much simpler than the method 1.
- the method 1 is divided into the following steps:
- Step 1 The image is binarized and those points that should be illuminated are extracted and stored as coordinates ⁇ , Z, (image plane).
- Step 2 Based on the values of X f, Y f, Z f, r, ⁇ , Xi, Zi ... X n, Z n, h, v and the equations (7) and (8), each point in ⁇ ,, Z, scaled (ie adapted to the projection plane) and projected onto the surface defined by the points Xi, Zi ... X n , Z n and adapted to the topology T s .
- ⁇ ,, Z forms a binary matrix
- T s the topography matrix
- 3rd step The surface defined by the points Xi, Zi ... X n , Z n is split into subfaces. For example, each headlamp, which together form the lighting unit of a lighting device, illuminate a sub-area.
- Step 5 Based on the headlight technology and geometry, each 2D projection (ie, the flat dot distribution) is converted into a specific image format specific to the headlight (the pixel intensity is based on the pixel intensity calculation).
- Step 6 Based on v s and v f , each image can be converted into a video sequence.
- FIGS. 5 to 7 show an example in which the Audi logo for the driver's perspective is projected onto the road.
- Fig. 5 shows the geometry in the side view.
- a headlight SW projects a light pattern onto a projection plane PE, resulting in an image Bl for the observer BO in an image plane.
- the resulting perspective here corresponds, for example, to a driver's perspective.
- the constellation of FIG. 6 results.
- the left-hand headlamp SWL, together with the right-hand headlamp SWR, should display the undistorted Audi logo for the driver or observer BO Projecting between two headlamps project.
- an original or desired light pattern is given in accordance with the illustration on the top left in FIG.
- the projection plane and the position of the observer or driver are already taken into account. Since, however, two headlights in different positions should generate the light pattern, corresponding images or drive signals corresponding to the illustrations on the right side of FIG. 7 for the left headlight SWL and the right headlight SWR must be determined. Together, for the observer BO, the reconstructed image from FIG. 7 then appears from the bottom on the left.
- the second method indicated in FIG. 4 includes the following steps:
- Step 2 Based on the values of Xi, Zi ... X n, Z n, h, v, each point Xi, Z, scaled in the image plane and passing through the points Xi, Zi ... X n, Z n defined surface and adapted to the topography T s .
- Steps 3 to 6 As in Method 1 As for Method 1, there are also different optimization possibilities for Method 2 in order to reduce latency. For example, integration of steps 2 to 4 can be done in a single matrix operation.
- Figures 8 and 9 show a possible light distribution for the two headlights. 8 shows on the left a vehicle FZ which is to produce a rectangular carpet of light on a roadway of a road. The left-hand headlamp should produce a left-hand carpet LTL and the right-hand headlamp a right-hand carpet LTR. The image in FIG. 8 right shows, in a mirrored view, the beam paths for the corners of the illuminated carpets LTL and LTR with respect to the two headlights SWL and SWR.
- FIG. 9 shows the 2D image of each of the headlights SWL and SWR on the respective headlight level. This means that in each of the headlights, those pixels must be lit that are in the respectively marked area. Together with the headlight geometry and optics as well as the selected projection plane, the carpet of light then results with the two parts LTL and LTR.
- the following explains how to calculate the pixel intensity. Based on the values L f , S s , R s , L s , T s and the parameters of observer and projection plane or road surface, the requirements of the Highway Code, the geometry of the headlights and their technology, the number of headlights and the 2D projections calculated by methods 1 and 2 can calculate the intensity per pixel needed to produce a desired light distribution on the road.
- the overlapping areas for two or more headlamps are to be determined by way of example.
- the desired light distribution x can be calculated from the sum of all overlapping images y k multiplied by the factor A k for all overlapping headlights K according to the formula:
- Fig. 10 shows above the desired light distribution in total. This consists of the light distribution of the left headlamp SWL added to the light distribution of the right headlamp SWR together.
- Fig. 1 1 shows a workflow diagram for calculating the pixel intensity.
- step S10 all physical and geometric input parameters are provided for the intensity calculation.
- step S1 1 it is checked whether overlapping areas are present. If not, a jump is made to step S12. However, if there are overlapping areas, a jump is made to step S13 where preprocessing of the data is made for, for example, increased resolution or special effects. Subsequently, it also jumps to step S12.
- step S12 it is checked whether the speeds of interest are small and the affected topography T s is very complex. If this AND condition is satisfied, a time-consuming calculation can be performed in step S13. In this case, for example, a 3D image processing with so-called "render” or "inverse rateracing” done. Subsequently, an image sequence per headlight is generated in step S14.
- step S12 if the AND condition of step S12 is not satisfied and, for example, one of the speeds is large, a predetermined criterion, e.g. B. checks the homogeneity or a desired intensity profile. Subsequently, the intensity per pixel is determined in step S16. Then, in step S17, one of the methods 1 or 2 for calculating the projection geometry and the pixel positions is performed. This is followed by a jump to step S14.
- a predetermined criterion e.g. B. checks the homogeneity or a desired intensity profile.
- light functions offer the possibility of actually improving or worsening the visibility of objects in the environment compared to other driver assistance systems, such as head-up displays.
- light assist functions in future piloted driving can help to communicate and interact with the environment.
- the driver can be supported in his vision and perception task.
- a specialized GPU Graphic Processor Unit
- a control unit on which the above method is implemented
- a sensor system for the required sensor data may be connected. It can be implemented in an architecture and programming language that is optimized for normal latency.
- the headlight it is advantageous if it is a high-resolution headlight, z. B.
- DMD Digital Micro Device
- scanner with at least 100 pixels.
- Light functions in the dipped beam are not visible on wet roads, too bright external lighting or a vehicle driving in front of the vehicle and must therefore be used in a situational manner.
Landscapes
- Engineering & Computer Science (AREA)
- Mechanical Engineering (AREA)
- Lighting Device Outwards From Vehicle And Optical Signal (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102015012416.1A DE102015012416A1 (en) | 2015-09-25 | 2015-09-25 | Projecting a predefinable light pattern |
PCT/EP2016/072725 WO2017050986A1 (en) | 2015-09-25 | 2016-09-23 | Projection of a pre-definable light pattern |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3353013A1 true EP3353013A1 (en) | 2018-08-01 |
EP3353013B1 EP3353013B1 (en) | 2020-08-12 |
Family
ID=56985627
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP16770285.1A Active EP3353013B1 (en) | 2015-09-25 | 2016-09-23 | Projection of a pre-definable light pattern |
Country Status (5)
Country | Link |
---|---|
US (1) | US10293745B2 (en) |
EP (1) | EP3353013B1 (en) |
CN (1) | CN108025672B (en) |
DE (1) | DE102015012416A1 (en) |
WO (1) | WO2017050986A1 (en) |
Families Citing this family (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102017103886A1 (en) | 2017-02-24 | 2018-08-30 | Osram Opto Semiconductors Gmbh | Arrangement for illuminating and recording a moving scene |
JP7000114B2 (en) * | 2017-10-24 | 2022-01-19 | スタンレー電気株式会社 | Optical axis change method and control system for road surface drawing for automobiles |
DE102017223441B4 (en) | 2017-12-20 | 2019-10-31 | Audi Ag | Light control device for the division of a light distribution |
DE102017223439B4 (en) * | 2017-12-20 | 2019-08-08 | Audi Ag | Warning device against dangerous situations for a motor vehicle |
DE102018200002A1 (en) * | 2018-01-02 | 2019-07-04 | Ford Global Technologies, Llc | Voice control of a vehicle light |
WO2019156244A1 (en) * | 2018-02-09 | 2019-08-15 | 大日本印刷株式会社 | Illumination device, method of designing illumination device, and designing device |
DE102018218038A1 (en) * | 2018-10-22 | 2020-04-23 | Volkswagen Aktiengesellschaft | System and method for projecting informational content for a motor vehicle |
WO2020108761A1 (en) | 2018-11-29 | 2020-06-04 | HELLA GmbH & Co. KGaA | Headlamp for a vehicle, vehicle with a headlamp and method of driving a headlamp |
FR3096824B1 (en) * | 2019-05-29 | 2021-10-01 | Valeo Vision | Method for providing a light pattern and lighting device for a motor vehicle |
CN111439195A (en) * | 2019-07-15 | 2020-07-24 | 长城汽车股份有限公司 | Method for projecting pattern by using car lamp, car lamp system and car |
CN112519670B (en) * | 2019-09-17 | 2024-03-05 | 宝马股份公司 | Reversing indication method and reversing indication system for motor vehicle and motor vehicle |
CN111385947B (en) * | 2020-03-23 | 2022-04-26 | 北京经纬恒润科技股份有限公司 | Control method and device applied to pixel lamp |
EP4139897A1 (en) * | 2020-04-24 | 2023-03-01 | Valeo Vision | Method for managing image data, and vehicle lighting system |
KR102446274B1 (en) * | 2020-11-30 | 2022-09-22 | 현대모비스 주식회사 | Apparatus for emitting road surface infromation and method thereof |
CN118265633A (en) * | 2021-11-19 | 2024-06-28 | 株式会社小糸制作所 | Lamp system, controller of lamp and control method |
FR3130937A1 (en) * | 2021-12-16 | 2023-06-23 | Valeo Vision | Adaptation of the beam of a light module according to the load of a vehicle |
US11607987B1 (en) | 2022-03-02 | 2023-03-21 | Ford Global Technologies, Llc | Multi-functional headlamp and image projector |
US12000573B2 (en) | 2022-09-30 | 2024-06-04 | Nissan North America, Inc. | Vehicle light assembly with light bar and rotatable projector |
DE102022127219A1 (en) | 2022-10-18 | 2024-04-18 | HELLA GmbH & Co. KGaA | Headlights for vehicles and control procedures |
DE102023200568A1 (en) | 2023-01-25 | 2024-07-25 | Volkswagen Aktiengesellschaft | Method for supporting a UAV by means of a motor vehicle |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102006050546B4 (en) | 2006-10-26 | 2020-07-23 | Bayerische Motoren Werke Aktiengesellschaft | Information presentation method |
DE102006050548B4 (en) | 2006-10-26 | 2014-05-08 | Bayerische Motoren Werke Aktiengesellschaft | Procedure for warning other road users |
JP5255301B2 (en) | 2008-03-12 | 2013-08-07 | 株式会社小糸製作所 | Vehicle headlamp device |
JP5719697B2 (en) * | 2011-06-10 | 2015-05-20 | 株式会社小糸製作所 | Vehicle headlamp device |
GB2499579B (en) | 2012-02-07 | 2014-11-26 | Two Trees Photonics Ltd | Lighting device |
JP5761119B2 (en) | 2012-05-17 | 2015-08-12 | 株式会社デンソー | In-vehicle lighting system |
US8733939B2 (en) | 2012-07-26 | 2014-05-27 | Cloudcar, Inc. | Vehicle content projection |
JP5790698B2 (en) | 2013-04-11 | 2015-10-07 | トヨタ自動車株式会社 | Information display device and information display method |
DE102013216318A1 (en) * | 2013-08-16 | 2015-02-19 | Volkswagen Aktiengesellschaft | A method of controlling a headlamp assembly for a vehicle and headlamp assembly |
DE102014000935A1 (en) * | 2014-01-22 | 2015-07-23 | Johannes Reusch | Method and arrangement for generating a traffic light display for vehicles |
JP6372376B2 (en) | 2014-02-10 | 2018-08-15 | 株式会社デンソー | Headlight control device |
EP3329420A1 (en) * | 2015-07-28 | 2018-06-06 | Apple Inc. | System and method for light and image projection |
-
2015
- 2015-09-25 DE DE102015012416.1A patent/DE102015012416A1/en not_active Ceased
-
2016
- 2016-09-23 CN CN201680055548.3A patent/CN108025672B/en active Active
- 2016-09-23 WO PCT/EP2016/072725 patent/WO2017050986A1/en active Application Filing
- 2016-09-23 US US15/762,375 patent/US10293745B2/en active Active
- 2016-09-23 EP EP16770285.1A patent/EP3353013B1/en active Active
Also Published As
Publication number | Publication date |
---|---|
US10293745B2 (en) | 2019-05-21 |
EP3353013B1 (en) | 2020-08-12 |
US20190016256A1 (en) | 2019-01-17 |
CN108025672A (en) | 2018-05-11 |
DE102015012416A1 (en) | 2017-03-30 |
CN108025672B (en) | 2020-05-15 |
WO2017050986A1 (en) | 2017-03-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3353013B1 (en) | Projection of a pre-definable light pattern | |
DE102014107158B4 (en) | Improved top-down imaging in a front curb visualization system | |
DE102014107156B4 (en) | System and method for providing improved perspective imaging in a front curb visualization system | |
DE112018001655T5 (en) | Display device and moving body with the display device | |
DE102008025459B4 (en) | Method and device for calibrating a vertical cut-off line generated by a headlight of a vehicle | |
EP3847499B1 (en) | Method for operating a field-of-vision display device for a motor vehicle | |
DE102010038825B4 (en) | The image display control device | |
DE102013019021B4 (en) | Method for driver assistance and motor vehicle | |
DE112017005111T5 (en) | Board display controller | |
DE102016213687B4 (en) | Method for controlling a display device for a motor vehicle, display device for a motor vehicle and motor vehicle with a display device | |
DE102017103282A1 (en) | DISPLAY CONTROL FOR ONE VEHICLE | |
DE102017103287A1 (en) | DISPLAY CONTROL FOR ONE VEHICLE | |
DE112018000858T5 (en) | Apparatus and method for displaying information | |
DE102018218038A1 (en) | System and method for projecting informational content for a motor vehicle | |
DE102018200963A1 (en) | HEAD-UP DISPLAY DEVICE AND DISPLAY CONTROL METHOD | |
DE102016213028A1 (en) | Motor vehicle with one or more headlamps | |
DE102012003158A1 (en) | Method for projecting laser light image in environment around vehicle during driving vehicle, involves projecting manipulated contents of captured image to position in predetermined environment by using laser light beams | |
DE102013219556A1 (en) | Method and device for controlling an image generation device of a head-up display | |
DE112019000329B4 (en) | DISPLAY SYSTEM, MOVABLE OBJECT AND DESIGN METHOD | |
DE102009054231A1 (en) | Head-up-display for information indication in motor vehicle, comprises indicating unit, which has display unit, in which lighting points are produced | |
DE102011003550A1 (en) | Method and device for adjusting a basic setting of a headlight of a motor vehicle | |
DE102017210219A1 (en) | A method for assisting a user of a motor vehicle in avoiding an obstacle, driver assistance device, and motor vehicle | |
DE102015012816A1 (en) | Method for calibrating a laser headlight in a motor vehicle and motor vehicle | |
EP3296795A1 (en) | Method for displaying an image object in a vehicle on a display seen inside and outside the vehicle | |
DE102016002010B4 (en) | Method for operating a projection device of a motor vehicle, projection system and motor vehicle |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20180425 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20190207 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20200519 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502016010846 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1301245 Country of ref document: AT Kind code of ref document: T Effective date: 20200915 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20200812 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201113 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201112 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201112 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201212 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502016010846 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20200930 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200923 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 |
|
26N | No opposition filed |
Effective date: 20210514 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200923 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200930 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200930 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200812 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 1301245 Country of ref document: AT Kind code of ref document: T Effective date: 20210923 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210923 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230530 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20230920 Year of fee payment: 8 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230922 Year of fee payment: 8 Ref country code: DE Payment date: 20230930 Year of fee payment: 8 |