CN110647801A - Method and device for setting region of interest, storage medium and electronic equipment - Google Patents

Method and device for setting region of interest, storage medium and electronic equipment Download PDF

Info

Publication number
CN110647801A
CN110647801A CN201910723121.1A CN201910723121A CN110647801A CN 110647801 A CN110647801 A CN 110647801A CN 201910723121 A CN201910723121 A CN 201910723121A CN 110647801 A CN110647801 A CN 110647801A
Authority
CN
China
Prior art keywords
vehicle
region
interest
lane
setting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910723121.1A
Other languages
Chinese (zh)
Inventor
周志爽
孔凡忠
陈新
李彪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BAIC Motor Co Ltd
Beijing Automotive Group Co Ltd
Beijing Automotive Research Institute Co Ltd
Original Assignee
BAIC Motor Co Ltd
Beijing Automotive Research Institute Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BAIC Motor Co Ltd, Beijing Automotive Research Institute Co Ltd filed Critical BAIC Motor Co Ltd
Priority to CN201910723121.1A priority Critical patent/CN110647801A/en
Publication of CN110647801A publication Critical patent/CN110647801A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/56Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/56Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/584Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Traffic Control Systems (AREA)

Abstract

The disclosure relates to a method and a device for setting a region of interest, a storage medium and an electronic device. The method comprises the following steps: acquiring a vehicle running image detected by a vehicle detection device; judging the lane of the vehicle on the image; and setting the region of interest on the image according to the lane where the vehicle is located and the running speed of the vehicle. By adopting the method, the calculation amount of the acquired data can be reduced by limiting the partial area in the driving image as the interested area and further performing calculation processing on the data in the interested area.

Description

Method and device for setting region of interest, storage medium and electronic equipment
Technical Field
The present disclosure relates to the field of data processing, and in particular, to a method and an apparatus for setting a region of interest, a storage medium, and an electronic device.
Background
With the development of artificial intelligence, the automobile industry is continuously innovated, and the automatic driving technology of vehicles is rapidly developed.
In order to realize operations such as automatic deceleration until stopping when the automobile meets an obstacle like a person, or continuous driving while avoiding the obstacle, a vision sensor needs to be installed on the automobile. Currently, a video camera, an ultrasonic radar, a millimeter wave radar, a laser radar, and the like are used as a vision sensor in an automatic driving system of a vehicle. However, in any of the sensors, there is a problem that the amount of data calculation is large because the amount of data to be acquired is large.
Disclosure of Invention
An object of the present disclosure is to provide a method, an apparatus, a storage medium, and an electronic device for setting a region of interest, so as to solve the problems in the prior art.
In order to achieve the above object, according to a first aspect of the present disclosure, there is provided a method of setting a region of interest, including:
acquiring a vehicle running image detected by a vehicle detection device;
judging the lane of the vehicle on the image;
and setting the region of interest on the image according to the lane where the vehicle is located and the running speed of the vehicle.
Optionally, the setting the region of interest on the image according to the lane where the vehicle is located and the driving speed of the vehicle includes:
setting the width of the region of interest according to the lane where the vehicle is located;
setting the length of the region of interest according to the running speed of the vehicle.
Optionally, the setting the width of the region of interest according to the lane where the vehicle is located includes:
judging whether the vehicle is in a turning driving state or not;
if the vehicle is not in a turning driving state, setting the width of the region of interest according to the lane where the vehicle is located and an adjacent lane, wherein the adjacent lane is the adjacent lane on the left side and/or the right side of the lane where the vehicle is located;
and if the vehicle is in a turning driving state, setting the width of the region of interest according to all lanes of the road where the vehicle is located in the image.
Optionally, the setting the length of the region of interest according to the driving speed of the vehicle includes:
detecting a running speed of the vehicle;
if the running speed is smaller than a first speed threshold value, taking a first preset length as the length of the region of interest;
if the running speed is greater than the first speed threshold value and less than a second speed threshold value, determining the length of the region of interest according to the running speed and a preset mapping relation between the running speed of the vehicle and the length of the region of interest, wherein the length of the region of interest is positively correlated with the running speed between the first speed threshold value and the second speed threshold value;
and if the running speed is greater than the second speed threshold value, taking a second preset length as the length of the region of interest.
Optionally, the setting the region of interest on the image according to the lane where the vehicle is located and the driving speed of the vehicle includes:
and if the vehicle is in a lane changing driving state, setting the region of interest according to the driving speed of the vehicle from an image area of a lane in the driving direction where the vehicle is located before the lane changing is successful and an image area of a lane where the vehicle is located after the lane changing is successful.
Optionally, if the vehicle is in a turning driving state, the boundary of the region of interest in the vehicle driving direction is adjusted according to the curvature of the vehicle turning driving.
According to a second aspect of the embodiments of the present disclosure, there is provided an apparatus for setting a region of interest, the apparatus comprising:
the acquisition module is used for acquiring a vehicle running image detected by a detection device of a vehicle;
the judging module is used for judging the lane of the vehicle on the image;
and the setting module is used for setting the region of interest on the image according to the lane where the vehicle is located and the running speed of the vehicle.
Optionally, the setting module includes:
the first setting submodule is used for setting the width of the region of interest according to the lane where the vehicle is located;
and the second setting submodule is used for setting the length of the region of interest according to the running speed of the vehicle.
Optionally, the first setting sub-module includes:
the judgment submodule is used for judging whether the vehicle is in a turning driving state or not;
the first setting submodule is further used for setting the width of the region of interest according to the lane where the vehicle is located and an adjacent lane when the vehicle is not in a turning driving state, wherein the adjacent lane is the adjacent lane on the left side and/or the right side of the lane where the vehicle is located;
the first setting submodule is further used for setting the width of the region of interest according to all lanes of a road where the vehicle is located in the image when the vehicle is in a turning driving state.
Optionally, the second setting sub-module includes:
the detection submodule is used for detecting the running speed of the vehicle;
the second setting submodule is also used for taking a first preset length as the length of the region of interest when the running speed is smaller than a first speed threshold value;
the second setting submodule is further used for determining the length of the region of interest according to the running speed and a preset mapping relation between the running speed and the length of the region of interest when the running speed is larger than the first speed threshold and smaller than a second speed threshold, wherein the length of the region of interest is positively correlated with the running speed between the first speed threshold and the second speed threshold;
the second setting submodule is further configured to use a second preset length as the length of the region of interest when the driving speed is greater than the second speed threshold.
Optionally, the setting module is further configured to set the region of interest according to a driving speed of the vehicle from an image area of a lane in a driving direction where the vehicle is located before the vehicle successfully changing lanes and an image area of a lane after the vehicle successfully changing lanes when the vehicle is in a lane changing driving state.
Optionally, the apparatus comprises:
and the turning module is used for adjusting the boundary of the region of interest in the vehicle driving direction according to the curvature of the vehicle turning driving when the vehicle is in a turning driving state.
According to a third aspect of embodiments of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the method of the first aspect.
According to a fourth aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to implement the steps of the method of the first aspect.
By adopting the technical scheme, the lane of the vehicle on the driving image is judged by acquiring the driving image of the vehicle detected by the detection device, and then the region of interest is set on the driving image according to the lane of the vehicle and the driving speed of the vehicle. The method can convert all data detected by the detection device into the image to obtain the current driving image of the vehicle, further set an interested area in the driving image according to the lane where the vehicle is located and the driving speed of the vehicle, and process the data in the range of the interested area, and the data in the interested area is less than the data contained in the original driving image, so the method reduces the calculation amount when processing the data.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:
fig. 1 is a flowchart illustrating a method of setting a region of interest according to an exemplary embodiment of the present disclosure.
Fig. 2 is a flowchart illustrating another method of setting a region of interest according to an exemplary embodiment of the present disclosure.
Fig. 3 is a schematic view illustrating a vehicle driving image according to an exemplary embodiment of the present disclosure.
Fig. 4 is a schematic view illustrating another vehicle travel image according to an exemplary embodiment of the present disclosure.
Fig. 5 is a schematic view illustrating another vehicle travel image according to an exemplary embodiment of the present disclosure.
Fig. 6 is a schematic view illustrating another vehicle travel image according to an exemplary embodiment of the present disclosure.
Fig. 7 is a schematic view illustrating another vehicle travel image according to an exemplary embodiment of the present disclosure.
Fig. 8 is a block diagram illustrating an apparatus for setting a region of interest according to an exemplary embodiment of the present disclosure.
Fig. 9 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
The following detailed description of specific embodiments of the present disclosure is provided in connection with the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
It is noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
In order to make it easier for those skilled in the art to understand the technical solutions provided by the embodiments of the present disclosure, an application background of the embodiments of the present disclosure will be described first, and the following embodiments of the present disclosure are applied to a vehicle having an auxiliary automatic driving function or a vehicle having an automatic driving function.
The method in the following embodiments of the present disclosure may be a method for setting a region of interest for a data model or an image acquired by different sensors such as a camera, an ultrasonic radar, a millimeter wave radar, and a laser radar, and the present disclosure is not limited thereto. The following embodiments of the present disclosure take an image into which laser radar collected data is converted as an example, and the method described in the present disclosure is exemplified in detail.
The following is a brief explanation of the relevant concepts involved in the present disclosure.
In the region of interest (ROI), in computer vision and image processing techniques, a region to be processed is delineated from a processed image in the form of a square frame, a circle, an ellipse, an irregular polygon, and the like, and is called a region of interest.
The high-precision map, also called high-precision electronic map, refers to a high-precision and fine defined map. The refined definition refers to the need to format and store various traffic elements in the traffic scene, including road network data, lane lines, traffic signs and other data of the traditional map. The high-precision electronic map contains a large amount of driving assistance information, and the most important is accurate three-dimensional representation (centimeter-level precision) of a road network. Such as the geometry of the road surface, the location of road sign lines, point cloud models of the surrounding road environment, etc. With these high-precision three-dimensional representations, the on-board robot can accurately determine its current position by comparing on-board GPS, IMU, LiDAR or camera data.
The automatic driving vehicle grade is divided into 6 grades from L0-L5 according to grading standards, the L0 grade is manual driving, the driver completely performs operations such as braking, steering, accelerator stepping, power transmission and the like, and the driver is required to judge the danger. L1-L5 were ranked according to the maturity of the autopilot.
The laser radar is a radar system that detects a characteristic amount such as a position and a velocity of a target by emitting a laser beam. The transmitter of the laser radar transmits a beam of laser, and the beam of the laser meets an object and returns to the laser receiver after diffuse reflection, so that data of the laser radar are obtained.
The point cloud data (point cloud data) records the scanning data in the form of points, each point includes three-dimensional coordinates, and some points may include color information (RGB) or reflection Intensity information (Intensity). That is, point cloud data refers to a collection of vectors in a three-dimensional coordinate system. In the embodiment of the present disclosure, the point cloud data refers to data acquired by a laser radar, wherein the single point cloud data includes an offset in a direction of a three-dimensional coordinate X, Y, Z and a reflection intensity, and the offset in the direction of X, Y, Z is calculated by using a position where the laser radar is installed as an origin.
Based on the above application background, a method for setting a region of interest provided by the present disclosure is described below.
Fig. 1 is a flowchart illustrating a method of setting a region of interest according to an exemplary embodiment of the present disclosure, as illustrated in fig. 1, the method including:
s101, acquiring a vehicle running image detected by a vehicle detection device.
The detection device can be a camera, an ultrasonic sensor, an infrared sensor, a radar sensor, a laser sensor and the like which are installed on the vehicle, and can also be a monitoring device on other remote terminals. The present disclosure is not limited thereto.
The detection device may illustratively be a device incorporating a lidar system. By installing the laser radar above the roof of the vehicle or around the vehicle and then using the laser radar to perform 360-degree periodic scanning on the vehicle and the environment around the vehicle, the laser radar can be 16-beam or 32-beam or 64-beam laser radar with different beams. And after the laser radar scans the surrounding environment of the vehicle, processing the collected point cloud data. Specifically, the point cloud data acquired by the laser radar may be subjected to coordinate conversion, preprocessing such as denoising, clustering processing or data modeling, and finally, image conversion is performed on the point cloud data to obtain a scanned image, so that a driving image in the driving process of the vehicle can be obtained.
As another example, the detection device may be a camera that acquires environmental information of the vehicle and surroundings thereof by installing the camera above a roof of the vehicle or around the vehicle. Then, the image acquired by the camera is preprocessed and analyzed, and a new image coordinate system can be additionally established to match with the subsequent processing of the method in the embodiment of the disclosure.
The environmental information refers to all information around the vehicle, including lane information, driving state information of the vehicle on the lane, traffic sign information on the road, and obstacles on the road.
And S102, judging the lane of the vehicle on the image.
Specifically, for example, the position of the vehicle may be determined using a high-precision map, and then the position of the vehicle in the travel image may be obtained in combination with the travel image obtained in step S101. The high-precision map has three-dimensional representation of a road network, so that the position information of the lane where the vehicle is located can be determined by combining the high-precision map.
S103, setting the region of interest on the image according to the lane where the vehicle is located and the running speed of the vehicle.
The automatic driving of the vehicle needs to detect the surrounding environment of the vehicle, and determines whether the vehicle can perform operations such as acceleration or deceleration, overtaking or lane changing, turning or obstacle avoidance according to surrounding environment information. There is a certain range of surrounding environments that affect safe driving of the vehicle, for example, when the vehicle is driving straight on the rightmost lane, and there is no drivable road on the right side. The area that influences the safe driving of the vehicle in this case is the environmental information on the lanes directly in front of and directly behind the vehicle and on the left side of the vehicle, because the vehicle is already on the rightmost lane on the road, and in this case, the vehicle is driven without a conditional turn to the right according to the driving regulations, i.e. the environmental information on the right side has no influence on the safe driving of the vehicle. However, the vehicle can perform lane change driving to the left lane, and due to traffic driving regulations, the lane change cannot cross two lanes at a time, so that only the adjacent left lane in the left environment of the lane affects the safe driving of the vehicle. In addition, the front-rear safe distance of the vehicle, and the safe distance to prevent rear-end collision are related to the running speed of the vehicle. Therefore, when the vehicle travels straight on the rightmost lane, the range that affects the safe travel of the vehicle is the partial range on the lanes directly in front and directly behind the vehicle and on the left side of the vehicle, and the travel speed of the vehicle. In this way, in one possible embodiment, after determining the lane in which the vehicle is located and the traveling speed of the vehicle, an environmental range that ensures safe traveling of the vehicle can be calculated on the acquired traveling image. Specifically, it may be a vehicle travel image detected by the detection device that combines high-precision map information and a vehicle as in step S102 described above, on which a detection range of safe travel of the vehicle is defined, which may be taken as the region of interest. By performing calculation analysis on the data in the region of interest, the environmental state in the range can be determined according to the analysis result, so that the automatic driving system of the vehicle can perform corresponding safe operation. Similarly, the driving behavior of the driver may be assisted by performing calculation analysis on the data in the region of interest and determining the environmental state in the range based on the result of the analysis processing.
By adopting the method, a detection range for ensuring the safe running of the vehicle is defined on the running image of the vehicle according to the lane where the vehicle is located and the running speed of the vehicle, the range is used as the region of interest, and the scanning data in the region of interest is calculated, so that compared with the method for performing undifferentiated processing on all data in the running image in the prior art, the method has the advantages that the calculation amount of the detected data is reduced under the condition of ensuring the safe running of the vehicle, and the data processing efficiency is improved.
Fig. 2 is a flowchart illustrating another method of setting a region of interest according to an exemplary embodiment of the present disclosure, as shown in fig. 2, the method including:
s201, setting the width of the region of interest according to the lane where the vehicle is located.
As can be seen from the step S103, the lane position where the vehicle is located is related to the detection area range for ensuring safe driving of the vehicle, and specifically, the lane position where the vehicle is located is related to the width of the region of interest.
In a possible implementation, setting the width of the region of interest according to the lane in which the vehicle is located may further include:
and judging whether the vehicle is in a turning driving state or not.
The vehicle is judged to be in a turning driving state, whether the vehicle is about to turn can be known, and the road is curved when the vehicle turns, and at the moment, besides a view field blind area factor, a plurality of unforeseen safety factors exist, so that the region-of-interest width can be further limited in consideration of safety when the vehicle turns or is about to turn.
If the vehicle is not in a turning driving state, setting the width of the region of interest according to the lane where the vehicle is located and an adjacent lane, wherein the adjacent lane is the adjacent lane on the left side and/or the right side of the lane where the vehicle is located.
Since the lane position where the vehicle is located is related to the width of the region of interest, the region of interest can be set by the lane where the vehicle is located. The width of the region of interest may specifically be set according to the lane in which the vehicle is located and the adjacent lane. Illustratively, the vehicle shown in fig. 3 is a traveling image on a road having three lanes in one direction, the region of interest is a diagonally filled region in fig. 3, and the arrow direction is the traveling direction of the vehicle. When the vehicle is driving on the rightmost lane in the driving direction, at this time, the adjacent lane is the left adjacent lane of the lane where the vehicle is located, then the width sum of the lane where the vehicle is located and the left adjacent lane thereof may be used as the width value of the region of interest, or the width sum of the lane where the vehicle is located and the left adjacent lane may be used as a reference value, and the width value of the region of interest is obtained by appropriately increasing or decreasing.
For another example, as shown in fig. 4, when the vehicle is traveling on the leftmost lane in the traveling direction, at this time, the adjacent lane is the right adjacent lane of the lane where the vehicle is located, the width sum of the lane where the vehicle is located and the right adjacent lane thereof may be used as the width value of the region of interest, and the width sum of the lane where the vehicle is located and the right adjacent lane thereof may also be used as a reference value, and the width value of the region of interest is obtained by appropriately increasing or decreasing the width sum.
For another example, as shown in fig. 5, when the vehicle is traveling on the middle lane in the traveling direction, at this time, the adjacent lanes are the left and right adjacent lanes of the lane where the vehicle is located, the width sum of the lane where the vehicle is located and the left and right adjacent lanes thereof may be used as the width value of the region of interest, and the width sum of the lane where the vehicle is located and the left and right adjacent lanes thereof may also be used as a reference value, and the width value of the region of interest is obtained by appropriately increasing or decreasing the width sum.
It should be noted that, since the region of interest is a region defined on the driving image obtained by converting data detected by the vehicle detection device, the above-mentioned actual width value of the lane may not be the same as the measurement unit of the width value of the region of interest, but the lane information and other traffic indication information on the driving image may be distinguished by combining the high-precision map, and thus, the width of the corresponding region of interest may be set on the driving image according to the lane where the vehicle is located and the adjacent lane.
And if the vehicle is in a turning driving state, setting the width of the region of interest according to all lanes of the road where the vehicle is located in the image.
When the vehicle is in a turning travel state, a priority is given to a safety issue when the vehicle turns. In a possible case, the width of the region of interest may be set according to the lane where the vehicle is located and the adjacent lane of the lane where the vehicle is located, and specifically, the method for setting the width of the region of interest according to the lane where the vehicle is located and the adjacent lane when the vehicle is not in the turning driving state is the same as the above-mentioned method, and details are not described here.
In another possible case, all lanes in the driving direction of the vehicle may be used as the width range of the region of interest, for example, when the vehicle is driving around a corner, regardless of the lane position, the lane in the driving direction is used as the range covered by the region of interest. Therefore, by adopting the method, all road condition information on the turning road section and all driving state information of all vehicles can be known by detecting the information on all lanes, and the vehicles can be ensured to safely drive on the turning road section.
S202, setting the length of the region of interest according to the running speed of the vehicle.
It will be appreciated by those skilled in the art that the vehicle is kept on the way to the vehicle ahead a necessary separation distance, i.e. a safe separation distance of the vehicle, which is related to the driving speed of the vehicle, to prevent rear-end accidents from occurring at all times. That is, in order to achieve safe automatic driving of the vehicle, it is necessary to determine whether there are other vehicles or other obstacles within the separation distance range. Therefore, in one possible case, the length of the region of interest, which includes at least the above-described range of the safe distance, may be set according to the traveling speed of the vehicle.
Illustratively, the length of the region of interest is combined with the width of the region of interest obtained in the above step S201, so that the region of interest can be filled with oblique lines as shown in fig. 3.
In a possible embodiment, the step of setting the length of the region of interest according to the driving speed of the vehicle may further comprise the steps of:
the running speed of the vehicle is detected.
Generally, the faster the vehicle speed, the greater the vehicle weight, and the longer the safe distance between vehicles. Therefore, by detecting the traveling speed of the vehicle, the length of the region of interest is set according to the different speeds of the vehicle.
And if the running speed is less than a first speed threshold value, taking a first preset length as the length of the region of interest.
In a possible case, the running speed of the vehicle is low, and in order to ensure the running safety of the vehicle, a minimum length needs to be set as the length of the region of interest. The first preset length may be used as the length of the region of interest when the running speed of the vehicle is less than the first speed threshold by detecting the running speed of the vehicle. Illustratively, according to the regulations related to the traffic safety law in China, when a motor vehicle runs on a highway, the speed of the motor vehicle is lower than 100 kilometers per hour, and at the minimum, 60 kilometers per hour, the motor vehicle should keep a safe distance of not less than 50 meters with a previous motor vehicle on the same lane, so that 80 kilometers per hour can be set as a first speed threshold, and the first preset length is set as 50 meters.
And if the running speed is greater than the first speed threshold value and less than a second speed threshold value, determining the length of the region of interest according to the running speed and a preset mapping relation between the running speed of the vehicle and the length of the region of interest, wherein the length of the region of interest is positively correlated with the running speed between the first speed threshold value and the second speed threshold value.
In particular, in addition to complying with traffic safety law-related regulations, the length range of the region of interest in the embodiments of the present disclosure is related to the performance of the detection device employed, whereas the recognition rate of the object by the detection device directly affects the accuracy of processing the data within the region of interest. And the detection devices with different models are adopted, so that the recognition rate of the objects at the same relative position in the detection range is different. Furthermore, the recognition rate for objects at different relative positions within their detection range is also different for the same detection device. Generally, in the case of the same detection device, the object at a position farther from the device has a lower recognition rate for the object.
It should be noted that, when the detection device is used for detection, the vehicle speed affects the recognition rate of the object at the same relative position as the detection device, and in a possible case, the faster the vehicle speed, the wider the range of the object meeting the recognition rate requirement.
In a possible implementation manner, the driving speed of the vehicle and the mapping relationship between the driving speed of the vehicle and the length of the region of interest can be set in advance according to the model of the detection device and the relevant regulations of the traffic safety law under the condition of guaranteeing the identification rate. For example, the mapping relationship may be that when the traveling speed of the vehicle is greater than a first speed threshold and less than a second speed threshold, the length of the region of interest is in positive correlation with the traveling speed between the first speed threshold and the second speed threshold, and may be directly related by an exponential positive correlation or a quadratic curve, etc. Wherein the first speed threshold may be 80 km per hour as determined in accordance with the method described above. A second speed threshold of 120 km per hour may also be set. When the speed of the vehicle is in the range of 80 to 120 roads per hour, the length of the region of interest is positively correlated with the data in the range, for example, the length of the region of interest may be 60 meters when the speed is 80 kilometers per hour, the length of the region of interest may be 80 meters when the speed is 90 kilometers per hour, and the length of the region of interest may be 110 meters when the speed is 100 kilometers per hour.
And if the running speed is greater than the second speed threshold value, taking a second preset length as the length of the region of interest.
Since the detection device also has the farthest detection distance, for example, the laser radar with model number HDL-64, the farthest detection distance is 120 meters; the laser radar with the model number of HDL-32 detects the farthest distance of 100 meters; the laser radar of model VLP-16, which detects the farthest distance of 100 meters. Therefore, when the speed is greater than the second speed threshold value, the second preset length is preset to be 100 meters or 110 meters and the like after comprehensive consideration is given to the identification rate and the safety distance of the detection device.
It should be noted that the range meeting the requirement of the identification rate is greater than or equal to the safe distance of the vehicle.
The length of the region of interest is defined based on the travel speed of the vehicle and a safe distance associated with the travel speed of the vehicle. This method reduces the amount of calculation processing of data detected by the detection means, compared with conventional methods, while ensuring safety.
By adopting the technical scheme, the lane where the vehicle is located is determined, the lane which is included in the width range which influences the safe driving of the vehicle when the vehicle is located in the lane is analyzed, and then the width of the region of interest is determined according to the lanes. Similarly, the safe distance of the vehicle at the current running speed can be known through the running speed of the vehicle-checking vehicle, and the regulation of relevant traffic rules is also considered, so that the length of the region of interest is determined according to the running speeds of different vehicles. By adopting the method, the data in the region of interest is limited to be used as the data processing region, the calculation amount of the data is reduced, and meanwhile, the safe distance of the vehicle is also considered, and the driving safety of the vehicle is improved.
In a possible case, the region of interest is set on the driving image according to the lane where the vehicle is located and the driving speed of the vehicle, and the following method can be adopted:
and if the vehicle is in the lane changing driving state, setting the region of interest according to the driving speed of the vehicle from the image area of the lane in the driving direction before the vehicle successfully changes the lane and the image area of the lane after the vehicle successfully changes the lane.
Specifically, when the vehicle needs to perform lane change driving, only the environmental information on the lane where the vehicle is located before the lane change and the lane after the lane change is successful may be focused, for example, as shown in fig. 3, when the vehicle is located in the rightmost lane, if the vehicle changes the lane to the middle lane, the region of interest may be a region filled with oblique lines as shown in fig. 3.
In another possible implementation manner, according to the general lane change operation habit of the driver, the autonomous vehicle may also perform lane change in the same manner, that is, when the vehicle needs to perform lane change driving, only the environmental information on the lane where the vehicle is located before the lane change and in front of the vehicle and the environmental information on the target lane after the lane change is successful may be focused, for example, as shown in fig. 6 as the oblique line filling area. Therefore, under the condition of ensuring safe lane changing driving, the range of the region of interest can be further reduced, and the data processing amount is further reduced.
It should be noted that, when the vehicle is in lane change driving, only the width of the region of interest is affected, and the length of the region of interest is still determined by the method for determining the length of the region of interest according to the driving speed of the vehicle. And after the lane change is successful, the region of interest is still set according to the flying method in step S103.
In one possible embodiment, if the vehicle is in a turning driving state, the boundary of the region of interest in the driving direction of the vehicle may be adjusted according to the curvature of the turning driving of the vehicle.
Wherein the curvature of the vehicle in turning is the same as or related to the curve.
Illustratively, when the vehicle is about to enter a turning road for driving, the boundary of the region of interest is adjusted accordingly according to the curvature of the road, for example, as shown in fig. 7. In a possible case, the region of interest shown in fig. 7 can be appropriately narrowed or enlarged, which is not limited by the present disclosure.
By adopting the method, the interesting area can be adaptively adjusted to follow the curvature of the road, so that the driving information, the obstacle information and the like of other vehicles in the curve can be accurately identified. Compared with the traditional method, the detection data in the region of interest obtained by the method are all effective data, and accurate road environment information is provided for safe driving or automatic driving of the automobile.
In summary, in the method detailed in the embodiment of the present disclosure, the region of interest is set according to the lane where the vehicle is located and the driving speed information of the vehicle, and the region of interest may be further defined according to the determination that the lane where the vehicle is located is the lane where the vehicle is located after the lane change is successful and the vehicle is about to enter a special road (i.e., a curve), where the definition may be to expand or reduce the range of the region of interest. Compared with the prior art, the method not only reduces the data processing amount, but also improves the validity of the data in the region of interest, thereby ensuring the driving safety of the vehicle.
Based on the same inventive concept, the disclosure also provides an apparatus for setting a region of interest, for implementing the steps of the method for setting a region of interest provided by the above method embodiment, as shown in fig. 8, the apparatus 800 includes:
an obtaining module 810, configured to obtain a vehicle driving image detected by a detection device of a vehicle;
a determining module 820, configured to determine a lane where the vehicle is located on the image;
a setting module 830, configured to set the region of interest on the image according to a lane where the vehicle is located and a driving speed of the vehicle.
By adopting the device, a detection range for ensuring the safe running of the vehicle is defined on the running image of the vehicle according to the lane where the vehicle is located and the running speed of the vehicle, the range is used as the region of interest, and the scanning data in the region of interest is calculated, so that compared with the method for performing undifferentiated processing on all data in the running image in the prior art, the device also reduces the calculation amount of the detection data under the condition of ensuring the safe running of the vehicle and improves the efficiency of data processing.
Optionally, the setting module includes:
a first setting sub-module 831 for setting the width of the region of interest according to the lane in which the vehicle is located;
a second setting submodule 832 for setting a length of the region of interest according to a traveling speed of the vehicle.
Optionally, the first setting sub-module includes:
a judgment submodule 8311 configured to judge whether the vehicle is in a turning driving state;
the first setting submodule 831 is further configured to set the width of the region of interest according to the lane where the vehicle is located and an adjacent lane when the vehicle is not in a turning driving state, where the adjacent lane is an adjacent lane on the left side and/or the right side of the lane where the vehicle is located;
the first setting sub-module 831 is further configured to set the width of the region of interest according to all lanes of a road on which the vehicle is located in the image when the vehicle is in a turning driving state.
Optionally, the second setting sub-module 832 includes:
a detection submodule 8321 for detecting a running speed of the vehicle;
the second setting submodule 832 is further configured to take a first preset length as the length of the region of interest when the driving speed is less than a first speed threshold;
the second setting submodule 832 is further configured to determine the length of the region of interest according to the running speed and a preset mapping relationship between the running speed and the vehicle running speed and the length of the region of interest when the running speed is greater than the first speed threshold and less than a second speed threshold, wherein the length of the region of interest is positively correlated with the running speed between the first speed threshold and the second speed threshold;
the second setting submodule 832 is further configured to use a second preset length as the length of the region of interest when the driving speed is greater than the second speed threshold.
Optionally, the setting module 830 is further configured to set the region of interest according to the driving speed of the vehicle from an image area of a lane in the driving direction where the vehicle is located before the vehicle successfully changing lanes and an image area of a lane after the vehicle successfully changing lanes when the vehicle is in the lane changing driving state.
Optionally, the apparatus comprises:
a turning module 840 configured to adjust a boundary of the region of interest in the vehicle driving direction according to a curvature of the vehicle turning driving when the vehicle is in a turning driving state.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
The present disclosure also provides a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the method of setting a region of interest provided by the present disclosure.
Fig. 9 is a block diagram illustrating an electronic device 400 according to an example embodiment. As shown in fig. 9, the electronic device 400 may include: a processor 401 and a memory 402. The electronic device 400 may also include one or more of a multimedia component 403, an input/output (I/O) interface 404, and a communications component 405.
Wherein, the processor 401 is configured to control the overall operation of the electronic device 400 to complete all or part of the steps in the above-mentioned method for setting a region of interest. The memory 402 is used to store various types of data to support operation at the electronic device 400, such as instructions for any application or method operating on the electronic device 400 and application-related data, such as contact data, transmitted and received messages, pictures, audio, video, and so forth. The Memory 402 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk or optical disk. The multimedia components 403 may include a screen and an audio component. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 402 or transmitted through the communication component 405. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 404 provides an interface between the processor 401 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 405 is used for wired or wireless communication between the electronic device 400 and other devices. Wireless communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, which is not limited herein. The corresponding communication component 405 may therefore include: Wi-Fi module, Bluetooth module, NFC module, etc.
In an exemplary embodiment, the electronic Device 400 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the steps of one of the above-described methods of setting a region of interest.
In another exemplary embodiment, a computer readable storage medium comprising program instructions which, when executed by a processor, carry out the steps of a method of setting a region of interest as described above is also provided. For example, the computer readable storage medium may be the memory 402 comprising program instructions executable by the processor 401 of the electronic device 400 to perform a method of setting a region of interest as described above.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.
It should be noted that the various features described in the above embodiments may be combined in any suitable manner without departing from the scope of the invention. In order to avoid unnecessary repetition, various possible combinations will not be separately described in this disclosure.
In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.

Claims (10)

1. A method of setting a region of interest, the method comprising:
acquiring a vehicle running image detected by a vehicle detection device;
judging the lane of the vehicle on the image;
and setting the region of interest on the image according to the lane where the vehicle is located and the running speed of the vehicle.
2. The method according to claim 1, wherein the setting of the region of interest on the image according to the lane in which the vehicle is located and the traveling speed of the vehicle includes:
setting the width of the region of interest according to the lane where the vehicle is located;
setting the length of the region of interest according to the running speed of the vehicle.
3. The method of claim 2, wherein setting the width of the region of interest according to the lane in which the vehicle is located comprises:
judging whether the vehicle is in a turning driving state or not;
if the vehicle is not in a turning driving state, setting the width of the region of interest according to the lane where the vehicle is located and an adjacent lane, wherein the adjacent lane is the adjacent lane on the left side and/or the right side of the lane where the vehicle is located;
and if the vehicle is in a turning driving state, setting the width of the region of interest according to all lanes of the road where the vehicle is located in the image.
4. The method according to claim 2, wherein the setting the length of the region of interest according to the traveling speed of the vehicle includes:
detecting a running speed of the vehicle;
if the running speed is smaller than a first speed threshold value, taking a first preset length as the length of the region of interest;
if the running speed is greater than the first speed threshold value and less than a second speed threshold value, determining the length of the region of interest according to the running speed and a preset mapping relation between the running speed of the vehicle and the length of the region of interest, wherein the length of the region of interest is positively correlated with the running speed between the first speed threshold value and the second speed threshold value;
and if the running speed is greater than the second speed threshold value, taking a second preset length as the length of the region of interest.
5. The method according to claim 1, wherein the setting of the region of interest on the image according to the lane in which the vehicle is located and the traveling speed of the vehicle includes:
and if the vehicle is in a lane changing driving state, setting the region of interest according to the driving speed of the vehicle from an image area of a lane in the driving direction where the vehicle is located before the lane changing is successful and an image area of a lane where the vehicle is located after the lane changing is successful.
6. The method according to any one of claims 1-5, comprising:
and if the vehicle is in a turning driving state, adjusting the boundary of the region of interest in the driving direction of the vehicle according to the curvature of the turning driving of the vehicle.
7. An apparatus for setting a region of interest, the apparatus comprising:
the acquisition module is used for acquiring a vehicle running image detected by a detection device of a vehicle;
the judging module is used for judging the lane of the vehicle on the image;
and the setting module is used for setting the region of interest on the image according to the lane where the vehicle is located and the running speed of the vehicle.
8. The apparatus of claim 7, wherein the setup module comprises:
the first setting submodule is used for setting the width of the region of interest according to the lane where the vehicle is located;
and the second setting submodule is used for setting the length of the region of interest according to the running speed of the vehicle.
9. A computer-readable storage medium, on which computer program instructions are stored, which program instructions, when executed by a processor, carry out the steps of the method according to any one of claims 1 to 6.
10. An electronic device, comprising:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to carry out the steps of the method of any one of claims 1 to 6.
CN201910723121.1A 2019-08-06 2019-08-06 Method and device for setting region of interest, storage medium and electronic equipment Pending CN110647801A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910723121.1A CN110647801A (en) 2019-08-06 2019-08-06 Method and device for setting region of interest, storage medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910723121.1A CN110647801A (en) 2019-08-06 2019-08-06 Method and device for setting region of interest, storage medium and electronic equipment

Publications (1)

Publication Number Publication Date
CN110647801A true CN110647801A (en) 2020-01-03

Family

ID=68990019

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910723121.1A Pending CN110647801A (en) 2019-08-06 2019-08-06 Method and device for setting region of interest, storage medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN110647801A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111231982A (en) * 2020-01-08 2020-06-05 中国第一汽车股份有限公司 Obstacle identification method and device for intelligent driving, vehicle and storage medium
CN111626206A (en) * 2020-05-27 2020-09-04 北京百度网讯科技有限公司 High-precision map construction method and device, electronic equipment and computer storage medium
CN111797701A (en) * 2020-06-10 2020-10-20 东莞正扬电子机械有限公司 Road obstacle sensing method and system for vehicle multi-sensor fusion system
CN113438427A (en) * 2020-03-23 2021-09-24 三星电机株式会社 Camera and control device for vehicle
CN113466877A (en) * 2020-03-30 2021-10-01 北京轻舟智航智能技术有限公司 Method and device for real-time object detection and electronic equipment
CN115359460A (en) * 2022-10-20 2022-11-18 小米汽车科技有限公司 Image recognition method and device for vehicle, vehicle and storage medium

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102951151A (en) * 2011-08-24 2013-03-06 现代摩比斯株式会社 Lane maintaining auxiliary system for vehicles and method thereof
CN102997900A (en) * 2011-09-15 2013-03-27 歌乐株式会社 Vehicle systems, devices, and methods for recognizing external worlds
CN103448634A (en) * 2012-05-31 2013-12-18 通用汽车环球科技运作有限责任公司 Dynamic guideline overlay with image cropping
CN104554269A (en) * 2013-10-17 2015-04-29 现代摩比斯株式会社 Region of interest setting device and method thereof
CN104742912A (en) * 2013-12-27 2015-07-01 比亚迪股份有限公司 Lane deviation detection method and device
CN104760593A (en) * 2014-01-03 2015-07-08 现代摩比斯株式会社 Apparatus for assisting in lane change and operating method thereof
CN105620489A (en) * 2015-12-23 2016-06-01 深圳佑驾创新科技有限公司 Driving assistance system and real-time warning and prompting method for vehicle
CN106503636A (en) * 2016-10-12 2017-03-15 同济大学 A kind of road sighting distance detection method of view-based access control model image and device
CN107462223A (en) * 2017-08-08 2017-12-12 长安大学 Driving sight distance self-operated measuring unit and measuring method before a kind of highway is turned
CN109855639A (en) * 2019-01-15 2019-06-07 天津大学 Unmanned method for planning track based on forecasting-obstacle and MPC algorithm
CN109959364A (en) * 2017-12-25 2019-07-02 比亚迪股份有限公司 Vehicle carries out the method and apparatus for automatically correcting range error based on lane line

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102951151A (en) * 2011-08-24 2013-03-06 现代摩比斯株式会社 Lane maintaining auxiliary system for vehicles and method thereof
CN102997900A (en) * 2011-09-15 2013-03-27 歌乐株式会社 Vehicle systems, devices, and methods for recognizing external worlds
CN103448634A (en) * 2012-05-31 2013-12-18 通用汽车环球科技运作有限责任公司 Dynamic guideline overlay with image cropping
CN104554269A (en) * 2013-10-17 2015-04-29 现代摩比斯株式会社 Region of interest setting device and method thereof
CN104742912A (en) * 2013-12-27 2015-07-01 比亚迪股份有限公司 Lane deviation detection method and device
CN104760593A (en) * 2014-01-03 2015-07-08 现代摩比斯株式会社 Apparatus for assisting in lane change and operating method thereof
CN105620489A (en) * 2015-12-23 2016-06-01 深圳佑驾创新科技有限公司 Driving assistance system and real-time warning and prompting method for vehicle
CN106503636A (en) * 2016-10-12 2017-03-15 同济大学 A kind of road sighting distance detection method of view-based access control model image and device
CN107462223A (en) * 2017-08-08 2017-12-12 长安大学 Driving sight distance self-operated measuring unit and measuring method before a kind of highway is turned
CN109959364A (en) * 2017-12-25 2019-07-02 比亚迪股份有限公司 Vehicle carries out the method and apparatus for automatically correcting range error based on lane line
CN109855639A (en) * 2019-01-15 2019-06-07 天津大学 Unmanned method for planning track based on forecasting-obstacle and MPC algorithm

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111231982A (en) * 2020-01-08 2020-06-05 中国第一汽车股份有限公司 Obstacle identification method and device for intelligent driving, vehicle and storage medium
CN111231982B (en) * 2020-01-08 2021-05-04 中国第一汽车股份有限公司 Obstacle identification method and device for intelligent driving, vehicle and storage medium
CN113438427A (en) * 2020-03-23 2021-09-24 三星电机株式会社 Camera and control device for vehicle
CN113466877A (en) * 2020-03-30 2021-10-01 北京轻舟智航智能技术有限公司 Method and device for real-time object detection and electronic equipment
CN113466877B (en) * 2020-03-30 2024-03-01 北京轻舟智航智能技术有限公司 Real-time object detection method and device and electronic equipment
CN111626206A (en) * 2020-05-27 2020-09-04 北京百度网讯科技有限公司 High-precision map construction method and device, electronic equipment and computer storage medium
CN111797701A (en) * 2020-06-10 2020-10-20 东莞正扬电子机械有限公司 Road obstacle sensing method and system for vehicle multi-sensor fusion system
CN111797701B (en) * 2020-06-10 2024-05-24 广东正扬传感科技股份有限公司 Road obstacle sensing method and system for vehicle multi-sensor fusion system
CN115359460A (en) * 2022-10-20 2022-11-18 小米汽车科技有限公司 Image recognition method and device for vehicle, vehicle and storage medium

Similar Documents

Publication Publication Date Title
US12005904B2 (en) Autonomous driving system
CN110647801A (en) Method and device for setting region of interest, storage medium and electronic equipment
CN111874006B (en) Route planning processing method and device
US10935976B2 (en) Blinker judgment device and autonomous driving system
US11157751B2 (en) Traffic guide object recognition device, traffic guide object recognition method, and storage medium
US20160325750A1 (en) Travel control apparatus
US10795374B2 (en) Vehicle control device
CN112208533B (en) Vehicle control system, vehicle control method, and storage medium
US11548443B2 (en) Display system, display method, and program for indicating a peripheral situation of a vehicle
WO2020259284A1 (en) Obstacle detection method and device
JP2024045402A (en) Vehicle control device, vehicle control method, vehicle control program
US11042759B2 (en) Roadside object recognition apparatus
CN111094096A (en) Vehicle control device, vehicle control method, and program
CN111824141B (en) Display control device, display control method, and storage medium
CN113525378B (en) Vehicle control device, vehicle control method, and storage medium
US20220237921A1 (en) Outside environment recognition device
US11468767B2 (en) Map information system
US20190161090A1 (en) Vehicle control device, vehicle control method, and recording medium
CN113611008B (en) Vehicle driving scene acquisition method, device, equipment and medium
CN112550263B (en) Information processing device, vehicle system, information processing method, and storage medium
CN113511220B (en) Vehicle control device, vehicle control method, and storage medium
US20240010242A1 (en) Signal processing device and signal processing method
US20230294702A1 (en) Control device, control method, and storage medium
US20240059288A1 (en) Vehicle control device, storage medium storing computer program for vehicle control, and method for controlling vehicle
CN117622200A (en) Vehicle control device, storage medium storing computer program for controlling vehicle, and vehicle control method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200103

RJ01 Rejection of invention patent application after publication