CN115220029A - Visibility identification method and device and automobile - Google Patents
Visibility identification method and device and automobile Download PDFInfo
- Publication number
- CN115220029A CN115220029A CN202111413731.5A CN202111413731A CN115220029A CN 115220029 A CN115220029 A CN 115220029A CN 202111413731 A CN202111413731 A CN 202111413731A CN 115220029 A CN115220029 A CN 115220029A
- Authority
- CN
- China
- Prior art keywords
- visibility
- target object
- value
- preset
- distance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 18
- 238000012896 Statistical algorithm Methods 0.000 claims description 10
- 230000008447 perception Effects 0.000 abstract description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
- G01S13/06—Systems determining position data of a target
- G01S13/08—Systems for measuring distance only
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention provides a visibility identification method, a visibility identification device and an automobile, wherein the visibility identification method comprises the steps of obtaining forward radar target identification data of the automobile and image data of a forward camera; determining a distance value between a target object and the current vehicle according to the forward radar target object identification data, and inquiring a preset visibility standard database according to the distance value between the target object and the current vehicle to obtain a visibility standard value corresponding to the distance value between the target object and the current vehicle; determining a credible characteristic value of the target object through a preset recognition algorithm according to the image data of the forward camera; and comparing the credible characteristic value of the target object with the visibility standard value, and determining the current visibility value according to the comparison difference value. According to the invention, the distance of the target object is identified by using the vehicle-mounted millimeter wave radar, the image is identified by using the camera and compared with the standard visibility, the visibility of the distance from the current position to the target object is obtained, and the perception of the vehicle to the environment is improved.
Description
Technical Field
The invention relates to the technical field of visibility recognition of environment by vehicle driving assistance, in particular to a visibility recognition method and device and an automobile.
Background
At present, two kinds of visibility testing equipment are generally available on the market, namely a transmission instrument and a scattering instrument. The transmission instrument is an instrument for calculating visibility by measuring atmospheric transparency, and the principle of the transmission instrument is to select a fixed reference object and test the visibility between the fixed reference object and the fixed reference object. Scatterometers measure directly the intensity of scattered light from a small sample volume to derive visibility.
Visibility measuring devices based on transillumination and scatterometry have the following disadvantages for automobiles: the cost of vehicle hardware and the cost of related supporting services are additionally increased; for the whole vehicle, the volume and the weight of the equipment are increased; the transilluminator and scatterometer are precision optical instruments suitable for mounting on non-moving objects. However, the vehicle vibrates at any time in the running process to cause deformation, so that the equipment precision is reduced; the current solution is only suitable for locations where the relative positions of the test sample and the detector are fixed. If the visibility recognition is realized without increasing the cost and overcoming the current vehicle use conditions, the visibility problem needs to be solved by considering the vehicle-mounted radar and the camera.
Disclosure of Invention
The invention aims to provide a visibility identification method, a device and an automobile, which solve the technical problem that the visibility is identified through a vehicle-mounted radar and a camera.
In one aspect, a visibility recognition method is provided, including:
acquiring forward radar target object identification data of a vehicle and image data of a forward camera, wherein the forward radar target object identification data at least comprises distance data of a target object and the current vehicle;
determining a distance value between a target object and a current vehicle according to the forward radar target object identification data, and inquiring a preset visibility standard database according to the distance value between the target object and the current vehicle to obtain a visibility standard value corresponding to the distance value between the target object and the current vehicle;
determining a credible characteristic value of the target object through a preset identification algorithm according to the image data of the forward camera;
and comparing the credible characteristic value of the target object with the visibility standard value, and determining the current visibility value according to the comparison difference value.
Preferably, the visibility criteria database is obtained by:
selecting a training target object;
shooting an image of the training target object within a preset distance through a camera under the condition that the standard visibility is 1;
performing grey-square statistics on the training target object image through a preset statistical algorithm to obtain a credible characteristic value of the image in the corresponding distance;
and combining the plurality of training targets, the corresponding distances and the corresponding credible characteristic values into a visibility standard database.
Preferably, the determining the trusted feature value of the target object according to the image data of the forward camera through a preset recognition algorithm specifically includes:
reading image data of a forward camera;
matching the images of the forward camera through a preset image matching algorithm, and extracting pictures with common characteristics with the target objects in the visibility standard database;
and performing grey-square statistics on the extracted picture through a preset statistical algorithm to obtain a credible characteristic value corresponding to the image data.
Preferably, the obtaining the current visibility value specifically includes:
outputting the difference value between the credible characteristic value of the target object and the visibility standard value as light attenuation quantity;
and querying a preset visibility standard database according to the obtained light attenuation quantity to obtain the current visibility value.
On the other hand, a visibility identification device is also provided, which is used for realizing the visibility identification method, and comprises the following steps:
the system comprises a data acquisition module, a data acquisition module and a data processing module, wherein the data acquisition module is used for acquiring forward radar target object identification data of a vehicle and image data of a forward camera, and the forward radar target object identification data at least comprises distance data between a target object and the current vehicle;
the standard value acquisition module is used for determining a distance value between a target object and a current vehicle according to the forward radar target object identification data, and inquiring a preset visibility standard database according to the distance value between the target object and the current vehicle to obtain a visibility standard value corresponding to the distance value between the target object and the current vehicle;
the visibility determining module is used for determining a credible characteristic value of the target object through a preset recognition algorithm according to the image data of the forward camera; and comparing the credible characteristic value of the target object with the visibility standard value, and obtaining the current visibility value according to the comparison difference value.
Preferably, the standard value obtaining module is further configured to select a training target object;
shooting an image of the training target object within a preset distance through a camera under the condition that the standard visibility is 1;
performing grey-square statistics on the training target object image through a preset statistical algorithm to obtain a credible characteristic value of the image in the corresponding distance;
and combining one or more training targets, the corresponding distances and the corresponding credible characteristic values into a visibility standard database.
Preferably, the visibility determining module is further configured to read image data of a forward-facing camera;
matching the images of the forward camera through a preset image matching algorithm, and extracting pictures with common characteristics with the target objects in the visibility standard database;
and performing grey-square statistics on the extracted picture through a preset statistical algorithm to obtain a credible characteristic value corresponding to the image data.
Preferably, the visibility determining module is further configured to output a difference value between the trusted characteristic value of the target object and the visibility standard value as an optical attenuation amount;
and querying a preset visibility standard database according to the obtained light attenuation quantity to obtain the current visibility value.
On the other hand, the visibility identification method is used for identifying the visibility of the automobile.
In summary, the embodiment of the invention has the following beneficial effects:
according to the visibility identification method, the device and the automobile, the distance of the target object is identified by using the vehicle-mounted millimeter wave radar, then the image is identified through the camera and compared with the standard visibility, the visibility of the distance from the current position to the target object is obtained, the perception of the automobile to the environment is improved, and therefore the advanced driving auxiliary system of the automobile can better serve the trip of people.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is within the scope of the present invention for those skilled in the art to obtain other drawings based on the drawings without inventive exercise.
Fig. 1 is a schematic main flow chart of a visibility identification method according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of a visibility recognition apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings.
Fig. 1 is a schematic diagram of a visibility identification method according to an embodiment of the present invention.
In this embodiment, the method comprises the steps of:
acquiring forward radar target object identification data of a vehicle and image data of a forward camera, wherein the forward radar target object identification data at least comprises distance data of a target object and the current vehicle; the image data of the forward camera is image data in front of the vehicle collected by the vehicle-mounted camera, wherein the image data at least comprises the same target object as the forward radar target object identification data, and the image data is mainly used for determining the credible characteristic value of the real-time target object through the image data in the follow-up process so as to determine the real-time visibility value through comparison with the standard value in the follow-up process; the radar in this embodiment at least includes a millimeter wave radar. That is, the identification data of the object of the millimeter wave radar before the vehicle is called, in order to obtain the distance data between the object and the current vehicle, the update frequency of the distance data should be more than or equal to the frame frequency of the camera, so as to obtain the more accurate distance d between the object and the camera within the same frame number, and the time data is directly transmitted to the bus by the radar sensor.
Further, a distance value between a target object and the current vehicle is determined according to the forward radar target object identification data, and a preset visibility standard database is inquired according to the distance value between the target object and the current vehicle to obtain a visibility standard value corresponding to the distance value between the target object and the current vehicle; that is, the distance value and the credible characteristic standard value of the target object under the condition that the visibility is 1 are determined through the acquired vehicle forward millimeter wave radar target object identification data.
In a specific embodiment, the step of determining the visibility standard database comprises: selecting a training target object; shooting an image of the training target object within a preset distance through a camera under the condition that the standard visibility (GB/T33673 standard) is 1; performing grey-square statistics on the training target object image through a preset statistical algorithm to obtain a credible characteristic value of the image in the corresponding distance; and combining one or more training targets, the corresponding distances and the corresponding credible characteristic values into a visibility standard database. Wherein, the target object is the license plate of the front vehicle generally, and the credibility characteristic value is the requirement of GA36-2018 Motor vehicle license plate of the people's republic of China on the license plate.
Further, determining a credible characteristic value of the target object through a preset identification algorithm according to the image data of the forward camera; that is, the credible feature value of the target object is picked up through an image recognition algorithm.
In a specific embodiment, image data of a forward camera is read; matching the images of the forward camera through a preset image matching algorithm, and extracting images with common characteristics with the target object in the visibility standard database; and performing grey-square statistics on the extracted picture through a preset statistical algorithm to obtain a credible characteristic value corresponding to the image data. In this embodiment, the image data P0 of the forward camera can be read by the onboard ECU; matching the images of the forward camera by the vehicle-mounted ECU through an image matching method, and intercepting a picture P1 with common characteristics with a target object in the database; and performing grey-square statistics on the P1 to obtain a credible characteristic value of the picture in the corresponding distance.
Furthermore, the credible characteristic value of the target object is compared with the visibility standard value, and the current visibility value is determined according to the comparison difference value. That is, the picked-up credible feature image is compared with visibility standard data of an equal distance d to obtain the light attenuation quantity of the credible feature image and the visibility standard data of the equal distance d, the visibility is calculated according to the light attenuation quantity in a specified distance, and the calculation process can be calculated through a preset formula or obtained by inquiring a preset numerical value in a preset standard database.
Fig. 2 is a schematic view of an embodiment of a visibility recognition apparatus according to the present invention.
In this embodiment, it comprises:
the data acquisition module is used for acquiring forward radar target object identification data of the vehicle and image data of the forward camera, wherein the forward radar target object identification data at least comprises distance data of a target object and the current vehicle.
And the standard value acquisition module is used for determining a distance value between a target object and the current vehicle according to the forward radar target object identification data, and inquiring a preset visibility standard database according to the distance value between the target object and the current vehicle to obtain a visibility standard value corresponding to the distance value between the target object and the current vehicle.
Specifically, the standard value obtaining module is further configured to select a training target object; shooting an image of the training target object within a preset distance through a camera under the condition that the standard visibility is 1; performing grey-square statistics on the training target object image through a preset statistical algorithm to obtain a credible characteristic value of the image in the corresponding distance; and combining the plurality of training targets, the corresponding distances and the corresponding credible characteristic values into a visibility standard database.
The visibility determining module is used for determining a credible characteristic value of the target object through a preset recognition algorithm according to the image data of the forward camera; and comparing the credible characteristic value of the target object with the visibility standard value, and obtaining the current visibility value according to the comparison difference value.
Specifically, the visibility determining module is further configured to read image data of a forward-facing camera; matching the images of the forward camera through a preset image matching algorithm, and extracting images with common characteristics with the target object in the visibility standard database; and performing grey-square statistics on the extracted picture through a preset statistical algorithm to obtain a credible characteristic value corresponding to the image data. The visibility determining module is also used for outputting a difference value between the credible characteristic value of the target object and the visibility standard value as light attenuation; and querying a preset visibility standard database according to the obtained light attenuation quantity to obtain the current visibility value.
The invention further provides the automobile, and the visibility is identified by the visibility identification method.
For the visibility recognition device and the implementation process of the automobile, reference may be made to the specific implementation process of the visibility recognition method, which is not described herein again.
In summary, the embodiment of the invention has the following beneficial effects:
according to the visibility identification method, the device and the automobile, the distance of the target object is identified by using the vehicle-mounted millimeter wave radar, then the image is identified through the camera and compared with the standard visibility, the visibility of the distance from the current position to the target object is obtained, the perception of the automobile to the environment is improved, and therefore the advanced driving auxiliary system of the automobile can better serve the trip of people.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.
Claims (9)
1. A visibility recognition method, comprising:
acquiring forward radar target object identification data of a vehicle and image data of a forward camera, wherein the forward radar target object identification data at least comprises distance data between a target object and the current vehicle;
determining a distance value between a target object and the current vehicle according to the forward radar target object identification data, and inquiring a preset visibility standard database according to the distance value between the target object and the current vehicle to obtain a visibility standard value corresponding to the distance value between the target object and the current vehicle;
determining a credible characteristic value of the target object through a preset identification algorithm according to the image data of the forward camera;
and comparing the credible characteristic value of the target object with the visibility standard value, and determining the current visibility value according to the comparison difference value.
2. The method of claim 1, wherein the visibility criteria database is obtained by:
selecting a training target object;
shooting an image of the training target object within a preset distance through a camera under the condition that the standard visibility is 1;
performing grey-square statistics on the training target object image through a preset statistical algorithm to obtain a credible characteristic value of the image in the corresponding distance;
and combining one or more training targets, the corresponding distances and the corresponding credible characteristic values into a visibility standard database.
3. The method of claim 2, wherein the determining the confidence feature value of the object according to the image data of the forward-facing camera by a predetermined recognition algorithm specifically comprises:
reading image data of a forward camera;
matching the images of the forward camera through a preset image matching algorithm, and extracting images with common characteristics with the target object in the visibility standard database;
and carrying out grey-square statistic on the extracted picture through a preset statistic algorithm to obtain a credible characteristic value corresponding to the image data.
4. The method as claimed in claim 3, wherein said obtaining a current visibility value comprises in particular:
outputting a difference value between the credible characteristic value of the target object and the visibility standard value as light attenuation quantity;
and querying a preset visibility standard database according to the obtained light attenuation quantity to obtain the current visibility value.
5. Visibility recognition device for implementing the method according to any one of claims 1 to 4, characterized in that it comprises:
the data acquisition module is used for acquiring forward radar target object identification data of the vehicle and image data of a forward camera, wherein the forward radar target object identification data at least comprises distance data between a target object and the current vehicle;
the standard value acquisition module is used for determining a distance value between a target object and the current vehicle according to the forward radar target object identification data, and inquiring a preset visibility standard database according to the distance value between the target object and the current vehicle to obtain a visibility standard value corresponding to the distance value between the target object and the current vehicle;
the visibility determining module is used for determining a credible characteristic value of the target object through a preset identification algorithm according to the image data of the forward camera; and comparing the credible characteristic value of the target object with the visibility standard value, and obtaining the current visibility value according to the comparison difference value.
6. The apparatus of claim 5, wherein the standard value obtaining module is further configured to select a training target;
shooting an image of the training target object within a preset distance through a camera under the condition that the standard visibility is 1;
performing grey-square statistics on the training target object image through a preset statistical algorithm to obtain a credible characteristic value of the image in the corresponding distance;
and combining the plurality of training targets, the corresponding distances and the corresponding credible characteristic values into a visibility standard database.
7. The apparatus of claim 6, wherein the visibility determination module is further configured to read image data of a forward facing camera;
matching the images of the forward camera through a preset image matching algorithm, and extracting pictures with common characteristics with the target objects in the visibility standard database;
and carrying out grey-square statistic on the extracted picture through a preset statistic algorithm to obtain a credible characteristic value corresponding to the image data.
8. The apparatus of claim 7, wherein said visibility determination module is further operative to output a difference between a value of an authentic characteristic of said object and said visibility criterion value as an amount of light attenuation;
and querying a preset visibility standard database according to the obtained light attenuation quantity to obtain the current visibility value.
9. An automobile, characterized in that visibility is identified by a method according to any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111413731.5A CN115220029A (en) | 2021-11-25 | 2021-11-25 | Visibility identification method and device and automobile |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111413731.5A CN115220029A (en) | 2021-11-25 | 2021-11-25 | Visibility identification method and device and automobile |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115220029A true CN115220029A (en) | 2022-10-21 |
Family
ID=83606858
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111413731.5A Pending CN115220029A (en) | 2021-11-25 | 2021-11-25 | Visibility identification method and device and automobile |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115220029A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070008182A1 (en) * | 2005-06-08 | 2007-01-11 | Denso Corporation | Image processing device for vehicle |
| DE102011105074A1 (en) * | 2011-06-21 | 2011-12-22 | Daimler Ag | Method for determining visual range for vehicle, involves determining surrounding of vehicle by camera, where contrast dimensions are determined for object depicted in images, where visual range is determined from contrast dimensions |
| JP2018136713A (en) * | 2017-02-21 | 2018-08-30 | マツダ株式会社 | Driver's visual field estimation device and vehicle control device |
| US10228460B1 (en) * | 2016-05-26 | 2019-03-12 | Rockwell Collins, Inc. | Weather radar enabled low visibility operation system and method |
| CN110542909A (en) * | 2019-09-03 | 2019-12-06 | 北京领骏科技有限公司 | Atmospheric visibility detection method and electronic equipment |
| CN112017243A (en) * | 2020-08-26 | 2020-12-01 | 大连信维科技有限公司 | Medium visibility identification method |
-
2021
- 2021-11-25 CN CN202111413731.5A patent/CN115220029A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070008182A1 (en) * | 2005-06-08 | 2007-01-11 | Denso Corporation | Image processing device for vehicle |
| DE102011105074A1 (en) * | 2011-06-21 | 2011-12-22 | Daimler Ag | Method for determining visual range for vehicle, involves determining surrounding of vehicle by camera, where contrast dimensions are determined for object depicted in images, where visual range is determined from contrast dimensions |
| US10228460B1 (en) * | 2016-05-26 | 2019-03-12 | Rockwell Collins, Inc. | Weather radar enabled low visibility operation system and method |
| JP2018136713A (en) * | 2017-02-21 | 2018-08-30 | マツダ株式会社 | Driver's visual field estimation device and vehicle control device |
| CN110542909A (en) * | 2019-09-03 | 2019-12-06 | 北京领骏科技有限公司 | Atmospheric visibility detection method and electronic equipment |
| CN112017243A (en) * | 2020-08-26 | 2020-12-01 | 大连信维科技有限公司 | Medium visibility identification method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP3760068B2 (en) | Image recognition device | |
| JP5294995B2 (en) | Distance measuring device and distance measuring method | |
| US10545228B2 (en) | Object identification device | |
| KR20120079144A (en) | A calibration apparatus, a distance measurement system, a calibration method and a calibration program | |
| CN106885622B (en) | A kind of big visual field multiple spot three-dimensional vibrating measurement method | |
| US20040046866A1 (en) | Method for determining visibility | |
| CN111372005B (en) | Automatic exposure compensation method and system for TOF camera module | |
| KR20170039465A (en) | System and Method for Collecting Traffic Information Using Real time Object Detection | |
| CN110824188A (en) | Speed measuring method and device for highway vehicles, coder-decoder and storage device | |
| CN116148801B (en) | Millimeter wave radar-based target detection method and system | |
| CN113325408B (en) | Noise estimation method, device and related equipment | |
| CN115220029A (en) | Visibility identification method and device and automobile | |
| CN113302511B (en) | Interference processing method and device | |
| CN114998329B (en) | Precision stamping quality analysis system of radio frequency shielding case of electronic communication equipment | |
| KR101892105B1 (en) | Data analysis method in ground penetrating measure | |
| JP2008299787A (en) | Vehicle detector | |
| CN217981856U (en) | Laser radar test system and vehicle | |
| JPS63188741A (en) | Measuring instrument for range of visibility | |
| CN116188316A (en) | Water area defogging method based on fog concentration sensing | |
| CN113674212B (en) | Handle assembly detection method and device | |
| CN116382389A (en) | System and method for controlling temperature and humidity inside automatic medicine vending machine | |
| CN110969859B (en) | Automatic zooming method and device for vehicle snapshot | |
| CN113534074A (en) | Positioning method and positioning device of ADAS calibration equipment | |
| WO2020101595A3 (en) | System and method for automatic calibration of vehicle position determining device in a traffic system | |
| JP3142489B2 (en) | Target motion analysis method and target motion analysis device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |