CN109684944B - Obstacle detection method, obstacle detection device, computer device, and storage medium - Google Patents
Obstacle detection method, obstacle detection device, computer device, and storage medium Download PDFInfo
- Publication number
- CN109684944B CN109684944B CN201811501437.8A CN201811501437A CN109684944B CN 109684944 B CN109684944 B CN 109684944B CN 201811501437 A CN201811501437 A CN 201811501437A CN 109684944 B CN109684944 B CN 109684944B
- Authority
- CN
- China
- Prior art keywords
- obstacle
- value
- trust function
- sensor
- existence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
- G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The application provides an obstacle detection method, an obstacle detection device, computer equipment and a storage medium, wherein the method comprises the following steps: determining a first trust function corresponding to a first obstacle in a current detection area of a first sensor according to data currently acquired by the first sensor, wherein each element in the trust function is used for representing the existence credibility, the non-existence credibility and the unknown credibility of the obstacle respectively; judging whether the total existence confidence level value corresponding to the first obstacle is larger than a threshold value or not according to the value of each element in the historical trust function corresponding to the first obstacle and the value of each element in the first trust function; and if so, determining that the first obstacle exists in the current detection area of the first sensor. According to the method, the trust function of the obstacle is obtained according to the current detection data of the sensor, the trust function is subjected to existence fusion with the historical trust function, the existence credibility of the obstacle is judged according to the fusion result, the reliability of the obstacle detection result is improved, and the false alarm rate is reduced.
Description
Technical Field
The present application relates to the field of information processing technologies, and in particular, to a method and an apparatus for detecting an obstacle, a computer device, and a storage medium.
Background
As vehicle technology has developed, driving may be assisted by mounting various sensors on the vehicle. In the existing intelligent driving technology, different sensors are mostly used for detecting obstacles, and driving is assisted according to the detection result of the sensors. Specifically, when any sensor on the vehicle detects an obstacle, an alarm prompt is given out to achieve the purpose of avoiding the obstacle.
However, in this obstacle detection method, the sensor is affected by the working scene and the processing speed, so that a false alarm condition is easily caused, and when the false alarm rate of the autonomous vehicle is too high, the normal running of the vehicle is affected.
Disclosure of Invention
The application provides a method and a device for detecting obstacles, computer equipment and a storage medium, which are used for solving the problem of high false alarm rate of the obstacle detection method in the related technology.
An embodiment of an aspect of the present application provides an obstacle detection method, including:
determining a first trust function corresponding to a first obstacle in a current detection area of a first sensor according to data currently acquired by the first sensor, wherein each element in the trust function is used for representing the existence credibility, the non-existence credibility and the unknown credibility of the obstacle respectively;
judging whether the total existence confidence level value corresponding to the first obstacle is larger than a threshold value or not according to the value of each element in the historical trust function corresponding to the first obstacle and the value of each element in the first trust function;
and if so, determining that the first obstacle exists in the current detection area of the first sensor.
According to the obstacle detection method, a first trust function corresponding to a first obstacle in a current detection area of a first sensor is determined according to data obtained by the first sensor currently, wherein each element in the trust function is used for representing the existence credibility, the non-existence credibility and the unknown credibility of the obstacle respectively, whether the total existence credibility value corresponding to the first obstacle is larger than a threshold value or not is judged according to the value of each element in a historical trust function corresponding to the first obstacle and the value of each element in the first trust function, and if the total existence credibility value is larger than the threshold value, the first obstacle is determined to exist in the current detection area of the first sensor. Therefore, the trust function corresponding to the obstacle is obtained according to the current detection data of the sensor, the trust function is subjected to existence fusion with the historical trust function, whether the obstacle exists in the detection area is determined according to the fused obstacle existence confidence value, the obstacle existence confidence level is judged according to the fusion result of the historical data and the current detection data of the sensor, the reliability of the obstacle detection result is improved, and the false alarm rate is reduced.
An embodiment of another aspect of the present application provides an obstacle detection device, including:
the first determination module is used for determining a first trust function corresponding to a first obstacle in a current detection area of a first sensor according to data currently acquired by the first sensor, wherein each element in the trust function is used for representing the existence credibility, the non-existence credibility and the unknown credibility of the obstacle respectively;
the first judging module is used for judging whether the total existence confidence level value corresponding to the first obstacle is larger than a threshold value or not according to the value of each element in the historical trust function corresponding to the first obstacle and the value of each element in the first trust function;
and the second determination module is used for determining that the first obstacle exists in the current detection area of the first sensor when the total existence reliability value corresponding to the first obstacle is greater than a threshold value.
According to the obstacle detection device, a first trust function corresponding to a first obstacle in a current detection area of a first sensor is determined according to data obtained by the first sensor currently, wherein each element in the trust function is used for representing the existence reliability, the non-existence reliability and the unknown reliability of the obstacle respectively, whether the total existence reliability value corresponding to the first obstacle is larger than a threshold value or not is judged according to the value of each element in a historical trust function corresponding to the first obstacle and the value of each element in the first trust function, and if the total existence reliability value is larger than the threshold value, the first obstacle is determined to exist in the current detection area of the first sensor. Therefore, the trust function corresponding to the obstacle is obtained according to the current detection data of the sensor, the trust function is subjected to existence fusion with the historical trust function, whether the obstacle exists in the detection area is determined according to the fused obstacle existence confidence value, the obstacle existence confidence level is judged according to the fusion result of the historical data and the current detection data of the sensor, the reliability of the obstacle detection result is improved, and the false alarm rate is reduced.
Another embodiment of the present application provides a computer device, including a processor and a memory;
wherein the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the obstacle detection method according to the embodiment of the above aspect.
Another embodiment of the present application proposes a non-transitory computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements an obstacle detection method as described in an embodiment of the above-mentioned aspect.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flowchart of an obstacle detection method according to an embodiment of the present disclosure;
fig. 2 is a schematic flow chart of another obstacle detection method according to an embodiment of the present disclosure;
fig. 3 is a schematic flowchart of another obstacle detection method according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an obstacle detection device according to an embodiment of the present disclosure;
FIG. 5 illustrates a block diagram of an exemplary computer device suitable for use in implementing embodiments of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
An obstacle detection method, an apparatus, a computer device, and a storage medium of the embodiments of the present application are described below with reference to the accompanying drawings.
The embodiment of the application provides an obstacle detection method aiming at the problem that a false alarm rate is high in an obstacle detection method in intelligent driving in the related art.
According to the obstacle detection method, the trust function is obtained according to the current detection data of the sensor, the trust function and the historical trust function are subjected to existence fusion, whether the obstacle exists in the detection area is determined according to the fused obstacle existence confidence value, so that the obstacle existence confidence level is judged according to the fusion result of the historical data and the current detection data of the sensor, the reliability of the obstacle detection result is improved, and the false alarm rate is reduced.
Fig. 1 is a schematic flow chart of an obstacle detection method according to an embodiment of the present application.
The obstacle detection method can be used for automatically driving the vehicle, and obstacle detection is carried out in the driving process of the vehicle so as to guarantee normal driving of the vehicle.
As shown in fig. 1, the obstacle detection method includes:
A variety of sensors, e.g., cameras, lidar, millimeter wave radar, etc., may be mounted on the vehicle, each sensor having a corresponding detection area. For example, cameras are mounted on both the left and right sides of the vehicle to detect obstacles in a certain range of areas on the left and right sides of the vehicle.
In this embodiment, the pre-trained neural network model may be utilized to process the detection data of the first sensor, so as to obtain the probability that the first obstacle exists in the current detection area of the first sensor.
For example, if the first sensor is a camera in front of the vehicle, the probability of the existence of the obstacle in the front area can be obtained by using a neural network model according to the image currently acquired by the camera.
After determining the probability of the first obstacle existing in the current detection area of the first sensor, the first trust function may be obtained according to the existence probability of the first obstacle. And each element of the trust function is used for representing the existence credibility, the non-existence credibility and the unknown credibility of the obstacle respectively.
For example, a trust function includes three elements, m (exit), m (unexsit), and m (unkown), and may be expressed as m ═ m (exit), m (unexsit), and m (unkown). Wherein m (exit) represents the obstacle existence reliability, i.e. the reliability of the existence of the obstacle; m (unexsit) represents the confidence that the obstacle is absent, i.e., the confidence that the obstacle is absent; m (unsown) represents the confidence that the obstacle is unknown, i.e., the confidence that it is uncertain whether the obstacle is present.
If the first trust function is denoted as ms1＝{ms1(exsit),ms1(unexsit),ms1(unsown) }, the probability of the presence of the first obstacle is ps1Element value m that specifies the reliability of the presence of the first obstacles1(exsit)＝ps1The element value of the first obstacle without confidence is ms1(unexsit) ═ 0, and the element value of the first obstacle without confidence is ms1(unkown)＝1-ps1。
And step 102, judging whether the total existence confidence level value corresponding to the first obstacle is greater than a threshold value according to the value of each element in the historical trust function corresponding to the first obstacle and the value of each element in the first trust function.
In this embodiment, the history trust function also includes three elements of the existence confidence level, the non-existence confidence level, and the unknown confidence level of the characterization obstacle. Then, the historical trust function corresponding to the first obstacle may be obtained according to the historical detection data of the first sensor, or may be obtained according to the detection results of all the sensors that detect the first obstacle.
In this embodiment, the historical trust function corresponding to the first obstacle and the first function may be fused according to a preset synthesis rule, so as to obtain a total existence confidence value corresponding to the first obstacle. As shown in the formula (1),
wherein m ist1Representing historical trustTrust function m obtained by fusing function and first functionhRepresenting a historical trust function, m, corresponding to the first obstacles1A first trust function corresponding to the first obstacle, a, B, C ∈ { extit, inexit, unkown }, where unkown # a ═ a, K is a normalization function,
wherein D, E is belonged to { exsit, unexsit, unkown }.
When A is exit, B is exit, and C is unkown; b is unkown, C is exit; and B and C both exsit, B and C satisfy A. Then, according to the formula (1), the total existence certainty value corresponding to the first obstacle is
Wherein, K is 1- [ m ═ mh(unexsit)·ms1(exsit)+mh(exsit)·ms1(unexsit)]. Therefore, the total existence certainty factor value corresponding to the first obstacle can be calculated by formula (1), and then the total existence certainty factor value is compared with the threshold value to judge whether the total existence certainty factor value corresponding to the first obstacle is greater than the threshold value, that is, whether the total existence certainty factor of the first obstacle is greater than the threshold value.
The threshold may be set according to actual needs, and this embodiment does not limit the threshold.
When the total existence reliability value corresponding to the first obstacle is larger than the threshold value, the existence reliability of the first obstacle is relatively high, and the first obstacle can be considered to exist in the current detection area of the first sensor, so that the obstacle existence prompt is sent.
In the embodiment of the application, the trust function corresponding to the first obstacle is obtained according to the data currently acquired by the first sensor, then the total existence confidence value of the first obstacle in the detection area is obtained by combining the trust function corresponding to the first obstacle, if the total existence confidence value is greater than the threshold value, the first obstacle is considered to exist in the detection area corresponding to the first sensor, and therefore whether the obstacle exists in the current detection area of the first sensor is determined according to the existence fusion result of the historical data detection result and the detection result of the first sensor, the reliability of the obstacle detection result is greatly improved, and the false alarm rate is reduced.
For the above embodiment, before determining whether the total existence confidence value corresponding to the first obstacle is greater than the threshold value, the historical trust function corresponding to the first obstacle may be generated.
As a possible implementation manner, the historical detection data of the first sensor may be synthesized based on a preset synthesis rule, so as to obtain a historical trust function corresponding to the first obstacle. The preset rule for synthesis may be a rule shown in the above formula (1).
During specific implementation, an initialized historical trust function can be obtained according to historical detection data of the first sensor at a certain moment, and the trust function obtained according to the historical detection data of the first sensor is sequentially fused with the historical trust function of the previous time according to the formula (1) in time to obtain the historical trust function corresponding to the first obstacle.
As another possible implementation manner, based on a preset synthesis rule, historical detection data of all sensors that detect the first obstacle is synthesized, so as to obtain a historical trust function corresponding to the first obstacle.
When the method is implemented, the corresponding trust functions obtained according to the historical detection data of the sensors can be fused according to the formula (1) to obtain the historical trust function corresponding to the first sensor.
In the embodiment of the application, the historical detection data of the first sensor or the historical detection data of each sensor detecting the first obstacle are synthesized to obtain the historical trust function corresponding to the first obstacle, so that the detection result of the historical data is fused with the current detection result of the first sensor, the reliability of the obstacle detection result can be greatly improved, and the false alarm rate is reduced.
According to the embodiments, the first trust function obtained according to the data acquired by the first sensor can represent the credibility of the first sensor for the existence of the first obstacle in the current detection area. Then, in an embodiment of the present application, before determining the total existence confidence value corresponding to the first obstacle according to the historical trust function and the first trust function, it may be determined whether the confidence value corresponding to the first obstacle is greater than a threshold value according to the first trust function.
Specifically, whether the value of an element used for representing the obstacle existence reliability in the first trust function is larger than a threshold value is judged. If the first sensor current detection area is larger than the threshold value, the fact that the first obstacle exists in the first sensor current detection area can be determined only according to the element values used for representing the obstacle existence reliability in the first trust function. If the total confidence level value is smaller than the threshold value, the total confidence level value corresponding to the first obstacle can be judged whether to be larger than the threshold value or not by combining the historical trust function of the first obstacle.
For example, the first trust function for the first obstacle is ms1＝{ms1(exsit)、ms1(unexsit)、ms1(nkown) }, then m is judgeds1(exit) if m is greater than the thresholds1(exit) is greater than the threshold, indicating the presence of a first obstacle in the current detection area of the first sensor. If ms1(exit) is less than or equal to a threshold, which may be in accordance with a first trust function ms1And historical trust function mhObtaining a total existence certainty value m corresponding to the first obstaclet1(exsit), the specific calculation method can be found in the above example, and m is judgedt1(exit) if the first obstacle is larger than the threshold value, and if the first obstacle is larger than the threshold value, the first obstacle is determined to exist in the current detection area of the first sensor.
Based on the above embodiment, it is determined whether the total existence confidence level value corresponding to the first obstacle is greater than the threshold value according to the value of each element in the first trust function and the value of each element in the historical trust function. And if the total existing reliability value is smaller than or equal to the threshold value, updating the value of each element in the historical trust function by using the value of each element in the first trust function.
When the method is implemented, the first trust function and the historical trust function can be fused according to the formula (1) to obtain an updated historical trust function. It is easy to understand that the fusion process is to calculate the values of the elements in the updated historical trust function for characterizing the existence confidence, the non-existence confidence and the unknown confidence of the first obstacle.
Assume that the updated historical trust function is mt1Is represented by mt1＝{mt1(exsit),mt1(unexsit),mt1(unsown) value m of an element for characterizing the existence confidence of the first obstaclet1The (exit) is the total existence certainty value obtained according to the first trust function and the historical trust function in the above embodiment, and then only the non-existence certainty value and the unknown certainty value need to be calculated.
Updated historical trust function mt1Value of the element for characterizing the non-existence confidence of the first obstacle
Values of elements for characterizing unknown credibility of first obstacle
Thus, according to the updated historical trust function mt1The values of the elements in the database can obtain the updated historical trust function.
In practical application, detection areas of different sensors mounted on a vehicle may overlap, for example, detection areas of a front camera and a front radar overlap, so that the front camera and the front radar may both detect the same obstacle, and then whether the reliability of the existence of the same obstacle is greater than a threshold value is determined according to detection data of the two sensors.
Based on the above analysis, in an embodiment of the application, after the values of the elements in the historical trust function corresponding to the first obstacle are updated, it may be further determined whether the area where the first obstacle is currently located is the detection area of the second sensor. Fig. 2 is a schematic flow chart of another obstacle detection method according to an embodiment of the present disclosure.
As shown in fig. 2, the obstacle detection method further includes:
In this embodiment, whether the first obstacle is in the detection area of the second sensor may be determined based on the position information of the first obstacle. The second sensor may be a sensor having an overlapping area with the detection area of the first sensor, and the second sensor may be one or a plurality of sensors.
And if the current area of the first obstacle is the detection area of the second sensor, determining a second trust function corresponding to the first obstacle according to the current detection data of the second sensor.
Specifically, the pre-trained neural network model can be used for processing the data currently acquired by the second sensor to obtain the probability that the first obstacle exists in the detection area of the second sensor, and then the value of each element in the second trust function is determined according to the probability.
For example, the probability of the presence of a first obstacle in the detection area of the second sensor is ps2Then a second trust function ms2＝{ms2(exsit),ms2(unexsit),ms2(nkown) } ms2(exsit)＝ps2，ms2(unexsit)＝0，ms2(unkown)＝1-ps2。
And step 203, judging whether the total existence confidence level value corresponding to the first barrier is greater than a threshold value according to the updated values of the elements in the historical trust function and the updated values of the elements in the second trust function.
After the second trust function is determined, the total existence confidence value corresponding to the first barrier is determined by combining the values of all elements in the updated historical trust function, and then whether the total existence confidence value is larger than a threshold value is judged.
In this embodiment, a method for determining the total existing certainty factor value corresponding to the first obstacle by using the updated value of each element in the historical trust function and the updated value of each element in the second trust function is similar to the method for determining the total existing certainty factor value corresponding to the first obstacle according to the value of each element in the historical trust function of the first obstacle and the value of each element in the first trust function in the above embodiment.
For example, the updated historical trust function is mt1The second trust function is ms2Then there is always a confidence value according to equation (1) above
Wherein, K is 1- [ m ═ mt1(unexsit)·ms2(exsit)+mt1(exsit)·ms2(unexsit)]. Then, m is judgedt2(exsit) is greater than a threshold.
And 204, if the total existence reliability value corresponding to the first obstacle is greater than the threshold value, determining that the first obstacle exists in the current detection area of the first sensor.
If the total existence reliability value of the first obstacle is larger than the threshold, the reliability that the first obstacle exists in the current detection area of the first sensor is relatively high, the first obstacle can be determined to exist in the current detection area of the first sensor, and an obstacle existence prompt is sent.
In the embodiment of the application, when a first obstacle exists in the detection area of the second sensor, whether the first obstacle exists in the detection area of the first sensor is determined by using the detection data of the second sensor and the updated historical trust function, so that whether the obstacle exists in the detection area is judged by fusing the existence of the second trust function and the updated historical trust function, the reliability of the obstacle detection result can be greatly improved, and the false alarm rate is reduced.
In the above embodiment, if it is determined that the total existence reliability value corresponding to the first obstacle is smaller than or equal to the threshold value according to the updated historical trust function and the second trust function, the value of each element in the updated historical trust function may be updated according to the value of each element in the second trust function, that is, the value of each element in the historical trust function corresponding to the first obstacle is updated.
For example, the updated historical trust function is mt1The second trust function is ms2Can be obtained according to the above formula (1)
Wherein m ist2Is according to ms2To mt1And carrying out the updated historical trust function, namely the historical trust function corresponding to the first barrier. In the above embodiment, the total existence certainty value m corresponding to the first obstacle is obtained according to the updated value of each element in the historical trust function and the value of each element in the second trust functiont2(exit), i.e. mt2The value of the element for representing the existence reliability of the first obstacle only needs to be calculatedt2(unexsit) and mt2(Unkown), concrete calculation method and mt1(unexsit) and mt1(inkwn) is similar in calculation method, and thus will not be described herein.
In this embodiment, when the total obstacle existence reliability value is less than or equal to the threshold, the existence of the obstacle may be continuously updated using the detection results of the other sensors that detect the obstacle, so that the reliability of the detection result is improved, and the false alarm rate is reduced.
In practical applications, there may be cases where the detection results are different due to sensors having overlapping detection areas. For example, there may be an overlap of the detection areas of the two sensors, with one of the sensors detecting an obstacle but the other sensor not detecting the obstacle. Based on this, in this embodiment, when determining the second trust function according to the data acquired by the second sensor, it may be determined whether the first obstacle exists in the detection area of the second sensor, and then, according to the determination result, the value of the element used for representing the unknown reliability of the first obstacle in the second trust function is determined. Fig. 3 is a schematic flow chart of another obstacle detection method according to an embodiment of the present application.
As shown in fig. 3, before determining the second trust function corresponding to the first obstacle, the obstacle detection method further includes:
In this embodiment, according to the data currently acquired by the second sensor, the obstacles existing in the detection area corresponding to the second sensor may be determined first, and then it may be determined whether the detected obstacles include the first obstacle.
If the first obstacle does not exist in the detection area of the second sensor, the first obstacle may be blocked by other obstacles, and the probability that the first obstacle is currently blocked by the second obstacle may be determined according to the position information of the first obstacle currently detected by the first sensor and the position information of the second obstacle currently detected by the second sensor.
If a first obstacle is present in the second sensor detection area, the value of the element in the second confidence function used for representing the non-existence confidence level of the first obstacle is zero.
In this embodiment, the element value used for characterizing the existence reliability of the first obstacle in the second trust function is equal to 1 minus the probability that the first obstacle is currently covered by the second obstacle. In the second confidence function, the value of the element for representing the existence confidence of the first obstacle and the value of the element with unknown confidence may be obtained by using the existence probability of the first obstacle in the detection area of the second sensor.
For example, the probability that a first obstacle is occluded by a second obstacle is pzThen m in the second trust functions2(unexsit)＝1-pz. That is, the greater the probability that the first obstacle is blocked by the second obstacle, the higher the first obstacle existence reliability.
In the embodiment of the application, when the second trust function is calculated, whether the first obstacle exists in the detection area of the second sensor is judged, and when the first obstacle does not exist, the value of an element used for representing the existence absence reliability of the first obstacle in the second trust function is determined according to the shielding probability of the first obstacle, so that the existence accuracy of the first obstacle is improved.
In order to implement the above embodiments, an obstacle detection device is further provided in the embodiments of the present application. Fig. 4 is a schematic structural diagram of an obstacle detection device according to an embodiment of the present application.
As shown in fig. 4, the obstacle detecting device includes: a first determining module 410, a first judging module 420 and a second determining module 430.
The first determining module 410 is configured to determine, according to data currently acquired by the first sensor, a first trust function corresponding to a first obstacle in a current detection area of the first sensor, where each element in the trust function is used to represent existence reliability, non-existence reliability, and unknown reliability of the obstacle;
a first determining module 420, configured to determine whether a total existence confidence value corresponding to the first obstacle is greater than a threshold value according to a value of each element in the historical trust function corresponding to the first obstacle and a value of each element in the first trust function;
the second determining module 430 is configured to determine that the first obstacle exists in the current detection area of the first sensor when the total existence reliability value corresponding to the first obstacle is greater than the threshold.
In a possible implementation manner of the embodiment of the present application, the apparatus further includes: a synthesis module to:
synthesizing historical detection data of the first sensor based on a preset synthesis rule to generate a historical trust function corresponding to the first obstacle;
alternatively, the first and second electrodes may be,
and synthesizing historical detection data of all sensors detecting the first obstacle based on a preset synthesis rule to generate a historical trust function corresponding to the first obstacle.
In a possible implementation manner of the embodiment of the present application, the apparatus further includes:
the second judgment module is used for judging whether the element value used for representing the existence credibility of the barrier in the first trust function is larger than a threshold value or not;
and the third determination module is used for determining that the first obstacle exists in the current detection area of the first sensor when the value of the element used for representing the obstacle existence reliability in the first trust function is larger than the threshold value.
In a possible implementation manner of the embodiment of the present application, the apparatus further includes:
and the updating module is used for updating the value of each element in the historical trust function corresponding to the first obstacle according to the value of each element in the historical trust function and the value of each element in the first trust function when the total existence trust value corresponding to the first obstacle is smaller than or equal to the threshold value.
In a possible implementation manner of the embodiment of the present application, the apparatus further includes:
the third judgment module is used for judging whether the area where the first barrier is located is the detection area of the second sensor or not;
the fourth determination module is used for determining a second trust function corresponding to the first obstacle according to data currently acquired by the second sensor when the current area where the first obstacle is located is the detection area of the second sensor;
the first determining module 420 is further configured to determine whether the total existence confidence value corresponding to the first obstacle is greater than a threshold value according to the updated value of each element in the historical trust function and the value of each element in the second trust function;
the second determining module 430 is further configured to determine that the first obstacle exists in the current detection area of the first sensor when the total existence reliability value corresponding to the first obstacle is greater than the threshold.
In a possible implementation manner of the embodiment of the present application, the apparatus may further include:
the fourth judging module is used for judging whether the first barrier exists in the detection area of the second sensor according to the data currently acquired by the second sensor;
the fifth determining module is used for determining the probability that the first obstacle is shielded by the second obstacle currently according to the position information of the first obstacle currently detected by the first sensor and the position information of the second obstacle currently detected by the second sensor when the first obstacle does not exist in the detection area of the second sensor;
and the sixth determining module is used for determining the element value used for representing the non-existence credibility of the first obstacle in the second trust function according to the probability that the first obstacle is currently shielded by the second obstacle.
In a possible implementation manner of the embodiment of the application, the sixth determining module is further configured to determine that, when there is a first obstacle in the detection area of the second sensor, an element value used for characterizing the non-existence reliability of the first obstacle in the second trust function is 0.
In a possible implementation manner of the embodiment of the present application, the update module is further configured to:
and when the total existence confidence level value corresponding to the first obstacle is smaller than or equal to the threshold value, updating the value of each element in the history trust function corresponding to the first obstacle according to the updated value of each element in the history trust function and the updated value of each element in the second trust function.
It should be noted that the foregoing explanation of the embodiment of the obstacle detection method is also applicable to the obstacle detection apparatus of this embodiment, and therefore will not be described herein again.
According to the obstacle detection device, a first trust function corresponding to a first obstacle in a current detection area of a first sensor is determined according to data obtained by the first sensor currently, wherein each element in the trust function is used for representing the existence reliability, the non-existence reliability and the unknown reliability of the obstacle respectively, whether the total existence reliability value corresponding to the first obstacle is larger than a threshold value or not is judged according to the value of each element in a historical trust function corresponding to the first obstacle and the value of each element in the first trust function, and if the total existence reliability value is larger than the threshold value, the first obstacle is determined to exist in the current detection area of the first sensor. Therefore, the trust function is obtained according to the current detection data of the sensor, the trust function and the historical trust function are subjected to existence fusion, whether the obstacle exists in the detection area is determined according to the fused obstacle existence credibility value, the obstacle existence credibility is judged according to the fusion result of the historical data and the current detection data of the sensor, the reliability of the obstacle detection result is improved, and the false alarm rate is reduced.
In order to implement the foregoing embodiments, an embodiment of the present application further provides a computer device, including a processor and a memory;
wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, for implementing the obstacle detection method as described in the above embodiments.
FIG. 5 illustrates a block diagram of an exemplary computer device suitable for use in implementing embodiments of the present application. The computer device 12 shown in fig. 5 is only an example and should not bring any limitation to the function and scope of use of the embodiments of the present application.
As shown in FIG. 5, computer device 12 is in the form of a general purpose computing device. The components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
Computer device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally perform the functions and/or methodologies of the embodiments described herein.
Computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with computer device 12, and/or with any devices (e.g., network card, modem, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Moreover, computer device 12 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public Network such as the Internet) via Network adapter 20. As shown, network adapter 20 communicates with the other modules of computer device 12 via bus 18. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 16 executes various functional applications and data processing, for example, implementing the methods mentioned in the foregoing embodiments, by executing programs stored in the system memory 28.
In order to implement the above embodiments, the present application also proposes a non-transitory computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the obstacle detection method as described in the above embodiments.
In the description of the present specification, the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (9)
1. An obstacle detection method, comprising:
determining a first trust function corresponding to a first obstacle in a current detection area of a first sensor according to data currently acquired by the first sensor, wherein each element in the trust function is used for representing the existence credibility, the non-existence credibility and the unknown credibility of the obstacle respectively;
judging whether the total existence confidence level value corresponding to the first obstacle is larger than a threshold value or not according to the value of each element in the historical trust function corresponding to the first obstacle and the value of each element in the first trust function;
if so, determining that the first obstacle exists in the current detection area of the first sensor;
if the current value is less than or equal to the preset value, updating the value of each element in the historical trust function corresponding to the first obstacle according to the value of each element in the historical trust function and the value of each element in the first trust function;
after the updating of the values of the elements in the historical trust function corresponding to the first obstacle, the method further includes:
judging whether the current area of the first barrier is the detection area of a second sensor;
if so, determining a second trust function corresponding to the first barrier according to the data currently acquired by the second sensor;
judging whether the total existence confidence level value corresponding to the first barrier is larger than a threshold value or not according to the value of each element in the updated historical trust function and the value of each element in the second trust function;
and if so, determining that the first obstacle exists in the current detection area of the first sensor.
2. The method of claim 1, wherein prior to determining whether the total presence confidence value for the first obstacle is greater than a threshold value, further comprising:
synthesizing historical detection data of the first sensor based on a preset synthesis rule to generate a historical trust function corresponding to the first obstacle;
alternatively, the first and second electrodes may be,
and synthesizing historical detection data of all sensors which detect the first obstacle based on a preset synthesis rule to generate a historical trust function corresponding to the first obstacle.
3. The method of claim 1, wherein prior to determining whether the total presence confidence value for the first obstacle is greater than a threshold value, further comprising:
judging whether the element value used for representing the existence credibility of the obstacle in the first trust function is larger than a threshold value or not;
and if so, determining that the first obstacle exists in the current detection area of the first sensor.
4. The method of claim 1, wherein prior to determining the second trust function corresponding to the first obstacle, further comprising:
judging whether the first barrier exists in the detection area of the second sensor or not according to the data currently acquired by the second sensor;
if not, determining the probability that the first obstacle is shielded by the second obstacle currently according to the position information of the first obstacle currently detected by the first sensor and the position information of the second obstacle currently detected by the second sensor;
and determining element values used for representing the non-existence credibility of the first obstacle in the second trust function according to the probability that the first obstacle is currently shielded by the second obstacle.
5. The method of claim 4, wherein after determining whether the first obstacle is present within the detection area of the second sensor, further comprising:
and if so, determining that the value of an element used for representing the non-existence credibility of the first obstacle in the second trust function is 0.
6. The method of any of claims 1-5, wherein said determining whether the total presence confidence value for the first obstacle is greater than a threshold value further comprises:
and if not, updating the value of each element in the historical trust function corresponding to the first obstacle according to the value of each element in the updated historical trust function and the value of each element in the second trust function.
7. An obstacle detection device, comprising:
the first determination module is used for determining a first trust function corresponding to a first obstacle in a current detection area of a first sensor according to data currently acquired by the first sensor, wherein each element in the trust function is used for representing the existence credibility, the non-existence credibility and the unknown credibility of the obstacle respectively;
the first judging module is used for judging whether the total existence confidence level value corresponding to the first obstacle is larger than a threshold value or not according to the value of each element in the historical trust function corresponding to the first obstacle and the value of each element in the first trust function;
the second determination module is used for determining that the first obstacle exists in the current detection area of the first sensor when the total existence reliability value corresponding to the first obstacle is larger than a threshold value;
the updating module is used for updating the value of each element in the historical trust function corresponding to the first obstacle according to the value of each element in the historical trust function and the value of each element in the first trust function when the total existence trust value corresponding to the first obstacle is smaller than or equal to a threshold value;
the third judgment module is used for judging whether the area where the first barrier is located is the detection area of the second sensor or not;
a fourth determining module, configured to determine, when the current area where the first obstacle is located is a detection area of a second sensor, a second trust function corresponding to the first obstacle according to data currently acquired by the second sensor;
the first judging module is further configured to judge whether the total existence reliability value corresponding to the first obstacle is greater than a threshold value according to the value of each element in the updated historical trust function and the value of each element in the second trust function;
the second determining module is further configured to determine that the first obstacle exists in the current detection area of the first sensor when the total existence reliability value corresponding to the first obstacle is greater than a threshold value.
8. A computer device comprising a processor and a memory;
wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory for implementing the obstacle detection method according to any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the obstacle detection method according to any one of claims 1 to 6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110903942.0A CN113743228B (en) | 2018-12-10 | 2018-12-10 | Obstacle existence detection method and device based on multi-data fusion result |
| CN201811501437.8A CN109684944B (en) | 2018-12-10 | 2018-12-10 | Obstacle detection method, obstacle detection device, computer device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811501437.8A CN109684944B (en) | 2018-12-10 | 2018-12-10 | Obstacle detection method, obstacle detection device, computer device, and storage medium |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110903942.0A Division CN113743228B (en) | 2018-12-10 | 2018-12-10 | Obstacle existence detection method and device based on multi-data fusion result |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109684944A CN109684944A (en) | 2019-04-26 |
| CN109684944B true CN109684944B (en) | 2021-09-03 |
Family
ID=66187373
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811501437.8A Active CN109684944B (en) | 2018-12-10 | 2018-12-10 | Obstacle detection method, obstacle detection device, computer device, and storage medium |
| CN202110903942.0A Active CN113743228B (en) | 2018-12-10 | 2018-12-10 | Obstacle existence detection method and device based on multi-data fusion result |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110903942.0A Active CN113743228B (en) | 2018-12-10 | 2018-12-10 | Obstacle existence detection method and device based on multi-data fusion result |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN109684944B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113281760A (en) * | 2021-05-21 | 2021-08-20 | 阿波罗智能技术(北京)有限公司 | Obstacle detection method, obstacle detection device, electronic apparatus, vehicle, and storage medium |
| CN113253299B (en) * | 2021-06-09 | 2022-02-01 | 深圳市速腾聚创科技有限公司 | Obstacle detection method, obstacle detection device and storage medium |
| US11624831B2 (en) | 2021-06-09 | 2023-04-11 | Suteng Innovation Technology Co., Ltd. | Obstacle detection method and apparatus and storage medium |
| CN115690739B (en) * | 2022-10-31 | 2024-03-26 | 阿波罗智能技术(北京)有限公司 | Multi-sensor fusion obstacle presence detection method and automatic driving vehicle |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102870005B (en) * | 2010-05-19 | 2015-07-08 | 三菱电机株式会社 | Obstacle detection device |
| CN104965202A (en) * | 2015-06-18 | 2015-10-07 | 奇瑞汽车股份有限公司 | Barrier detection method and device |
| CN108693541A (en) * | 2017-04-03 | 2018-10-23 | 福特全球技术公司 | Obstacle detection system and method |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE10330011B4 (en) * | 2003-07-03 | 2005-05-12 | Eads Deutschland Gmbh | Procedure for obstacle detection and terrain classification |
| EP3261078A1 (en) * | 2011-05-23 | 2017-12-27 | ION Geophysical Corporation | Marine threat monitoring and defense system |
| US8848978B2 (en) * | 2011-09-16 | 2014-09-30 | Harman International (China) Holdings Co., Ltd. | Fast obstacle detection |
| US8855442B2 (en) * | 2012-04-30 | 2014-10-07 | Yuri Owechko | Image registration of multimodal data using 3D-GeoArcs |
| CN106951847B (en) * | 2017-03-13 | 2020-09-29 | 百度在线网络技术（北京）有限公司 | Obstacle detection method, apparatus, device and storage medium |
| US10552691B2 (en) * | 2017-04-25 | 2020-02-04 | TuSimple | System and method for vehicle position and velocity estimation based on camera and lidar data |
| CN107341087A (en) * | 2017-06-29 | 2017-11-10 | 上海德衡数据科技有限公司 | A kind of O＆M pre-alarm and prevention system architecture based on multi-sensor information fusion |
| CN107830869B (en) * | 2017-11-16 | 2020-12-11 | 百度在线网络技术（北京）有限公司 | Information output method and apparatus for vehicle |
| CN108062368B (en) * | 2017-12-08 | 2021-05-07 | 北京百度网讯科技有限公司 | Full data translation method, device, server and storage medium |
| CN108319982A (en) * | 2018-02-06 | 2018-07-24 | 贵州电网有限责任公司 | A kind of power-line patrolling unmanned plane Fusion obstacle detection method |
| CN108280445B (en) * | 2018-02-26 | 2021-11-16 | 江苏裕兰信息科技有限公司 | Method for detecting moving objects and raised obstacles around vehicle |
| CN108909624B (en) * | 2018-05-13 | 2021-05-18 | 西北工业大学 | Real-time obstacle detection and positioning method based on monocular vision |
| CN108898055A (en) * | 2018-05-24 | 2018-11-27 | 长安大学 | A kind of mobile robot obstacle detection method of detection information fusion |
| CN108828527B (en) * | 2018-06-19 | 2021-04-16 | 驭势（上海）汽车科技有限公司 | Multi-sensor data fusion method and device, vehicle-mounted equipment and storage medium |
-
2018
- 2018-12-10 CN CN201811501437.8A patent/CN109684944B/en active Active
- 2018-12-10 CN CN202110903942.0A patent/CN113743228B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102870005B (en) * | 2010-05-19 | 2015-07-08 | 三菱电机株式会社 | Obstacle detection device |
| CN104965202A (en) * | 2015-06-18 | 2015-10-07 | 奇瑞汽车股份有限公司 | Barrier detection method and device |
| CN108693541A (en) * | 2017-04-03 | 2018-10-23 | 福特全球技术公司 | Obstacle detection system and method |
Non-Patent Citations (2)
| Title |
|---|
| 基于改进信度函数分配方法的煤种判别技术;赵亮宇 等;《华北电力大学学报》;20110731;第38卷(第4期);摘要、第2节 * |
| 智能车辆近场物体探测及其状态识别方法研究;鲍阚;《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》;20160915(第9期);第二章 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113743228B (en) | 2023-07-14 |
| CN109684944A (en) | 2019-04-26 |
| CN113743228A (en) | 2021-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109684944B (en) | Obstacle detection method, obstacle detection device, computer device, and storage medium | |
| CN111709975B (en) | Multi-target tracking method, device, electronic equipment and storage medium | |
| EP3065087B1 (en) | Information processing apparatus, image capturing apparatus, control system applicable to moveable apparatus, information processing method, and carrier medium of program of method | |
| CN109738884B (en) | Object detection method and device and computer equipment | |
| US11126875B2 (en) | Method and device of multi-focal sensing of an obstacle and non-volatile computer-readable storage medium | |
| JP2016081525A (en) | Vehicular image recognition system and corresponding method | |
| CN109766867B (en) | Vehicle running state determination method and device, computer equipment and storage medium | |
| CN111814746A (en) | Method, device, equipment and storage medium for identifying lane line | |
| US11688078B2 (en) | Video object detection | |
| CN112084810A (en) | Obstacle detection method and device, electronic equipment and storage medium | |
| US11436876B2 (en) | Systems and methods for diagnosing perception systems of vehicles based on temporal continuity of sensor data | |
| CN111937036A (en) | Method, apparatus, and computer-readable storage medium having instructions for processing sensor data | |
| CN111383246B (en) | Scroll detection method, device and equipment | |
| CN110647818A (en) | Identification method and device for shielding target object | |
| JP2021033510A (en) | Driving assistance device | |
| CN109635868B (en) | Method and device for determining obstacle type, electronic device and storage medium | |
| US9418443B2 (en) | Apparatus and method for detecting obstacle | |
| CN114537521A (en) | Parking assistance apparatus and method | |
| CN110706257B (en) | Identification method of effective characteristic point pair, and camera state determination method and device | |
| CN111832347B (en) | Method and device for dynamically selecting region of interest | |
| WO2022264533A1 (en) | Detection-frame position-accuracy improving system and detection-frame position correction method | |
| CN115703471A (en) | Driver attention and hand placement system and method | |
| CN109583511B (en) | Speed fusion method and device | |
| CN110133624B (en) | Unmanned driving abnormity detection method, device, equipment and medium | |
| KR101886856B1 (en) | System and method for data combining based on result of non-rigid object tracking on multi-sensor seeker |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |