CN112215145A - Tunnel lighting lamp cleaning effect monitoring method based on image comparison - Google Patents

Tunnel lighting lamp cleaning effect monitoring method based on image comparison Download PDF

Info

Publication number
CN112215145A
CN112215145A CN202011085735.0A CN202011085735A CN112215145A CN 112215145 A CN112215145 A CN 112215145A CN 202011085735 A CN202011085735 A CN 202011085735A CN 112215145 A CN112215145 A CN 112215145A
Authority
CN
China
Prior art keywords
tunnel
image
lamp
cleaning
lighting lamp
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011085735.0A
Other languages
Chinese (zh)
Inventor
陈建忠
丁浩
李文锋
杨孟
胡居义
李科
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Merchants Chongqing Communications Research and Design Institute Co Ltd
Original Assignee
China Merchants Chongqing Communications Research and Design Institute Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Merchants Chongqing Communications Research and Design Institute Co Ltd filed Critical China Merchants Chongqing Communications Research and Design Institute Co Ltd
Priority to CN202011085735.0A priority Critical patent/CN112215145A/en
Publication of CN112215145A publication Critical patent/CN112215145A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30232Surveillance

Abstract

The invention relates to a tunnel lighting lamp cleaning effect monitoring method based on image comparison, and belongs to the field of tunnel lamp cleaning. Cleaning a tunnel lighting lamp, acquiring image information of the cleaned lighting lamp by using a high-definition camera, and taking the image information as a standard image of the lighting lamp; the method comprises the following steps that high-definition cameras are respectively installed at the head and the tail of the tunnel lighting lamp cleaning vehicle and are used for acquiring image information before and after the tunnel lighting lamp cleaning vehicle is used for cleaning lamps; the method comprises the steps of extracting image information of the tunnel lighting lamp according to an image processing method by combining the obtained image information, calculating the similarity between the standard image information and the image information before cleaning, and between the standard image information and the image information after cleaning, and evaluating the cleaning effect of the tunnel lighting lamp according to the ratio of the two similarities, wherein the evaluation result can effectively describe the cleaning effect of the tunnel lighting lamp and is beneficial to improving the tunnel maintenance management level.

Description

Tunnel lighting lamp cleaning effect monitoring method based on image comparison
Technical Field
The invention belongs to the field of tunnel lamp cleaning, and relates to a tunnel lighting lamp cleaning effect monitoring method based on image comparison.
Background
Under the special environment of a closed highway tunnel, the surface of the tunnel lighting lamp is easy to be covered with smoke dust, so that the lighting efficiency is reduced, the road surface illumination is reduced, the driving safety in the tunnel is influenced, and the tunnel lighting lamp needs to be cleaned regularly. The method is used for testing the lighting effect of the tunnel lighting lamp before and after cleaning, quantifying the improvement effect of the tunnel lighting after cleaning of the lighting lamp and having important engineering significance for maintenance of the highway tunnel lighting lamp, and therefore the method for testing and evaluating the cleaning effect of the highway tunnel lighting lamp is provided.
Disclosure of Invention
In view of the above, the present invention provides a method for monitoring a cleaning effect of a tunnel lighting fixture based on image comparison.
In order to achieve the purpose, the invention provides the following technical scheme:
a tunnel lighting lamp cleaning effect monitoring method based on image comparison comprises the following steps:
1) acquiring an image;
2) processing an image;
3) and (6) evaluating the results.
Optionally, the step 1) specifically includes:
respectively installing a high-definition camera at the head part and the tail part of the tunnel lamp cleaning vehicle, wherein the high-definition cameras are used for acquiring the change of lamp image information before and after the tunnel lamp is cleaned;
tunnel lamp cleaning vehicle for tunnel illuminationBefore the lamps are cleaned, a certain lamp in the tunnel is cleaned in a manual cleaning mode, and after the lamps are cleaned, a high-definition camera is used for shooting and recording to obtain standard image information I after the tunnel lamps are cleanedE
The tunnel lamp cleaning vehicle is used for cleaning the tunnel lighting lamp, the high-definition camera on the tunnel lamp cleaning vehicle is used for photographing the tunnel lamp to be cleaned, and image information I of the tunnel lamp is obtainedAThe shooting angle is opposite to the lighting lamp;
the tunnel lamp cleaning vehicle adopts cleaning equipment to clean the lamp X, and utilizes a vehicle tail high-definition camera to shoot and record image information I of the cleaned lighting lamp at the same angleB
Optionally, the position and the angle of the high-definition camera are the same as those of the high-definition camera on the cleaning vehicle.
Optionally, the step 2) is specifically:
acquiring high-definition image characteristics;
and extracting effective areas of shot image information, calibrating a rectangular frame where a tunnel lighting lamp is located in the image by means of an OpenCV (open computer vision library) in deep learning, and extracting.
Optionally, the 3) is specifically:
comparing the high-definition camera A and the high-definition camera B with the tunnel lighting lamp image I obtained by shooting after manual cleaningA、IBAnd IEComparing the images I separatelyAAnd IEImage IBAnd IEAccording to the similarity between the image information before and after the tunnel lamp is cleaned and the image information under standard cleaning, a lamp cleaning rate index is introduced to describe the cleaning effect of the tunnel lighting lamp;
two images IxAnd IyThe similarity between the two is measured based on the brightness, contrast and structure, and the expressions are respectively
Figure BDA0002720276440000021
c1=(k1L)2 (2)
In the formula, muxAs an image IxThe mean value of (a); mu.syAs an image IyThe mean value of (a); k is a radical of1Constant, default to 0.01; l is the value range of the pixel value;
Figure BDA0002720276440000022
c2=(k2L)2 (4)
in the formula (I), the compound is shown in the specification,
Figure BDA0002720276440000023
as an image IxThe variance of (a);
Figure BDA0002720276440000024
as an image IyThe variance of (a); k is a radical of2Constant, default to 0.03; l is the value range of the pixel value;
Figure BDA0002720276440000025
in the formula, c3Is constant, by default, take c2Half of (1);
combining brightness, contrast and structure to obtain image IxAnd IyThe similarity between the two is expressed as
SSIM(x,y)=[l(x,y)·c(x,y)·s(x,y)] (6)
Figure BDA0002720276440000026
In the formula, SSIM (x, y) is image IxAnd IyThe similarity between the two images is that the SSIM (x, y) value is between 0 and 1, and the closer the similarity is to 1, the more similar the two images are; otherwise they are not similar;
respectively calculate to obtain images IAAnd IESimilarity between SSIM (I)A,IE) Image IBAnd IESimilarity between SSIM (I)B,IE) Then the expression of the index of the cleanness rate of the tunnel lighting lamp is
Figure BDA0002720276440000031
In the formula, q is the cleaning rate of the tunnel lighting lamp; the larger the q value is, the more similar the cleaned lamp image is to the lamp image cleaned by manual cleaning, and the better the cleaning effect is; the poorer the cleaning effect.
The invention has the beneficial effects that: the method and the device test the lighting effect of the tunnel lighting lamp before and after cleaning, and quantify the improvement effect of the tunnel lighting after the lighting lamp is cleaned.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the means of the instrumentalities and combinations particularly pointed out hereinafter.
Drawings
For the purposes of promoting a better understanding of the objects, aspects and advantages of the invention, reference will now be made to the following detailed description taken in conjunction with the accompanying drawings in which:
FIG. 1 is a flow chart of the present invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention in a schematic way, and the features in the following embodiments and examples may be combined with each other without conflict.
Wherein the showings are for the purpose of illustrating the invention only and not for the purpose of limiting the same, and in which there is shown by way of illustration only and not in the drawings in which there is no intention to limit the invention thereto; to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if there is an orientation or positional relationship indicated by terms such as "upper", "lower", "left", "right", "front", "rear", etc., based on the orientation or positional relationship shown in the drawings, it is only for convenience of description and simplification of description, but it is not an indication or suggestion that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes, and are not to be construed as limiting the present invention, and the specific meaning of the terms may be understood by those skilled in the art according to specific situations.
As shown in fig. 1, a method for monitoring the cleaning effect of a tunnel lighting fixture based on image comparison is provided.
1) Image acquisition method
And acquiring high-definition image information. The high-definition cameras are respectively arranged at the head part and the tail part of the tunnel lamp cleaning vehicle and used for acquiring the change of the image information of the lamp before and after the tunnel lamp is cleaned. Before cleaning tunnel lighting lamps, a tunnel lamp cleaning vehicle cleans a certain lamp in a tunnel in a manual cleaning mode, and after the tunnel lamp is cleaned, a high-definition camera (the position and the angle of the camera are the same as those of the cleaning vehicle) is used for photographing and recording to obtain standard image information I of the cleaned tunnel lampE(ii) a Then, the tunnel lamp cleaning vehicle is used for cleaning the tunnel lighting lamp, and the tunnel lamp is usedThe high-definition camera on the cleaning vehicle shoots the tunnel lamp to be cleaned and obtains the image information IAThe shooting angle of the lamp is opposite to that of the lighting lamp; then the tunnel lamp cleaning vehicle adopts cleaning equipment to clean the lamp X, and utilizes a vehicle tail high-definition camera to shoot and record image information I for the cleaned lighting lamp at the same angleB
2) Image processing method
And acquiring high-definition image characteristics. The high-definition camera is far away from the tunnel lighting lamp, and can shoot other equipment in the tunnel when an image is obtained, so that the acquisition of the image characteristics of the lamp at the back is influenced, and therefore the shot image information needs to be effectively extracted in an area.
3) Method for evaluating results
Comparing the high-definition camera A and the high-definition camera B with the tunnel lighting lamp image I obtained by shooting after manual cleaningA、IBAnd IEComparing the images I separatelyAAnd IEImage IBAnd IEAccording to the similarity between the image information before and after the tunnel lamp is cleaned and the image information under standard cleaning, the cleaning effect of the tunnel lighting lamp is described by introducing the index of the lamp cleaning rate.
Two images IxAnd IyThe similarity between the two is measured based on the brightness, contrast and structure, and the expressions are respectively
Figure BDA0002720276440000041
c1=(k1L)2 (2)
In the formula, muxAs an image IxThe mean value of (a); mu.syAs an image IyThe mean value of (a); k is a radical of1Constant, default to 0.01; and L is the value range of the pixel value.
Figure BDA0002720276440000051
c2=(k2L)2 (4)
In the formula (I), the compound is shown in the specification,
Figure BDA0002720276440000052
as an image IxThe variance of (a);
Figure BDA0002720276440000053
as an image IyThe variance of (a); k is a radical of2Constant, default to 0.03; and L is the value range of the pixel value.
Figure BDA0002720276440000054
In the formula, c3Is constant, by default, take c2Half of that.
Then combining brightness, contrast and structure, image I can be obtainedxAnd IyThe similarity between the two is expressed as
SSIM(x,y)=[l(x,y)·c(x,y)·s(x,y)] (6)
Figure BDA0002720276440000055
In the formula, SSIM (x, y) is image IxAnd IyThe similarity between the two images is that the SSIM (x, y) value is between 0 and 1, and the closer the similarity is to 1, the more similar the two images are; otherwise they are not similar.
Respectively calculate to obtain images IAAnd IESimilarity between SSIM (I)A,IE) Image IBAnd IESimilarity between SSIM (I)B,IE) Then the expression of the index of the cleanness rate of the tunnel lighting lamp is
Figure BDA0002720276440000056
In the formula, q is the cleaning rate of the tunnel lighting lamp; the larger the q value is, the more similar the cleaned lamp image is to the lamp image cleaned by manual cleaning, and the better the cleaning effect is; the poorer the cleaning effect.
Finally, the above embodiments are only intended to illustrate the technical solutions of the present invention and not to limit the present invention, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions, and all of them should be covered by the claims of the present invention.

Claims (5)

1. A tunnel lighting lamp cleaning effect monitoring method based on image comparison is characterized in that: the method comprises the following steps:
1) acquiring an image;
2) processing an image;
3) and (6) evaluating the results.
2. The method for monitoring the cleaning effect of the tunnel lighting lamp based on the image comparison as claimed in claim 1, wherein: the step 1) is specifically as follows:
respectively installing a high-definition camera at the head part and the tail part of the tunnel lamp cleaning vehicle, wherein the high-definition cameras are used for acquiring the change of lamp image information before and after the tunnel lamp is cleaned;
the tunnel lamp cleaning vehicle cleans a certain lamp in a tunnel in a manual cleaning mode before cleaning a tunnel lighting lamp, and after the tunnel lamp is cleaned, a high-definition camera is used for shooting and recording to obtain standard image information I after the tunnel lamp is cleanedE
The tunnel lamp cleaning vehicle is used for cleaning the tunnel lighting lamp, the high-definition camera on the tunnel lamp cleaning vehicle is used for photographing the tunnel lamp to be cleaned, and image information I of the tunnel lamp is obtainedATaking a pictureThe angle is opposite to the lighting lamp;
the tunnel lamp cleaning vehicle adopts cleaning equipment to clean the lamp X, and utilizes a vehicle tail high-definition camera to shoot and record image information I of the cleaned lighting lamp at the same angleB
3. The method for monitoring the cleaning effect of the tunnel lighting lamp based on the image comparison as claimed in claim 2, wherein: the position and the angle of the high-definition camera are the same as those of the high-definition camera on the cleaning vehicle.
4. The method for monitoring the cleaning effect of the tunnel lighting lamp based on the image comparison as claimed in claim 2, wherein: the 2) is specifically as follows:
acquiring high-definition image characteristics;
and extracting effective areas of shot image information, calibrating a rectangular frame where a tunnel lighting lamp is located in the image by means of an OpenCV (open computer vision library) in deep learning, and extracting.
5. The method for monitoring the cleaning effect of the tunnel lighting lamp based on the image comparison as claimed in claim 4, wherein: the 3) is specifically as follows:
comparing the high-definition camera A and the high-definition camera B with the tunnel lighting lamp image I obtained by shooting after manual cleaningA、IBAnd IEComparing the images I separatelyAAnd IEImage IBAnd IEAccording to the similarity between the image information before and after the tunnel lamp is cleaned and the image information under standard cleaning, a lamp cleaning rate index is introduced to describe the cleaning effect of the tunnel lighting lamp;
two images IxAnd IyThe similarity between the two is measured based on the brightness, contrast and structure, and the expressions are respectively
Figure FDA0002720276430000021
c1=(k1L)2 (2)
In the formula, muxAs an image IxThe mean value of (a); mu.syAs an image IyThe mean value of (a); k is a radical of1Constant, default to 0.01; l is the value range of the pixel value;
Figure FDA0002720276430000022
c2=(k2L)2 (4)
in the formula (I), the compound is shown in the specification,
Figure FDA0002720276430000023
as an image IxThe variance of (a);
Figure FDA0002720276430000024
as an image IyThe variance of (a); k is a radical of2Constant, default to 0.03; l is the value range of the pixel value;
Figure FDA0002720276430000025
in the formula, c3Is constant, by default, take c2Half of (1);
combining brightness, contrast and structure to obtain image IxAnd IyThe similarity between the two is expressed as
SSIM(x,y)=[l(x,y)·c(x,y)·s(x,y)] (6)
Figure FDA0002720276430000026
In the formula, SSIM (x, y) is image IxAnd IySimilarity between the two, SSIM (x, y) is takenThe value is between 0 and 1, and the closer the similarity is to 1, the more similar the two images are; otherwise they are not similar;
respectively calculate to obtain images IAAnd IESimilarity between SSIM (I)A,IE) Image IBAnd IESimilarity between SSIM (I)B,IE) Then the expression of the index of the cleanness rate of the tunnel lighting lamp is
Figure FDA0002720276430000027
In the formula, q is the cleaning rate of the tunnel lighting lamp; the larger the q value is, the more similar the cleaned lamp image is to the lamp image cleaned by manual cleaning, and the better the cleaning effect is; the poorer the cleaning effect.
CN202011085735.0A 2020-10-12 2020-10-12 Tunnel lighting lamp cleaning effect monitoring method based on image comparison Pending CN112215145A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011085735.0A CN112215145A (en) 2020-10-12 2020-10-12 Tunnel lighting lamp cleaning effect monitoring method based on image comparison

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011085735.0A CN112215145A (en) 2020-10-12 2020-10-12 Tunnel lighting lamp cleaning effect monitoring method based on image comparison

Publications (1)

Publication Number Publication Date
CN112215145A true CN112215145A (en) 2021-01-12

Family

ID=74053614

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011085735.0A Pending CN112215145A (en) 2020-10-12 2020-10-12 Tunnel lighting lamp cleaning effect monitoring method based on image comparison

Country Status (1)

Country Link
CN (1) CN112215145A (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20090128055A (en) * 2008-06-10 2009-12-15 엘지전자 주식회사 Projection apparatus and controling method the same
CN101996406A (en) * 2010-11-03 2011-03-30 中国科学院光电技术研究所 No-reference structural sharpness image quality evaluation method
CN105516713A (en) * 2015-12-24 2016-04-20 招商局重庆交通科研设计院有限公司 Image quality evaluation method of road traffic closed-circuit television based on machine vision
CN106903093A (en) * 2017-03-08 2017-06-30 中国民航大学 A kind of embedded autonomous cleaning Vehicle of navaid light fixture and control method
CN108109147A (en) * 2018-02-10 2018-06-01 北京航空航天大学 A kind of reference-free quality evaluation method of blurred picture
CN108550146A (en) * 2018-04-11 2018-09-18 北京印刷学院 A kind of image quality evaluating method based on ROI
CN110549254A (en) * 2019-09-27 2019-12-10 北京爱德空港设备工程有限公司 Integrated navigation aid lamp cleaning and detecting vehicle
CN110916574A (en) * 2019-10-18 2020-03-27 上海善解人意信息科技有限公司 Sweeping robot system and sweeping robot control method

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20090128055A (en) * 2008-06-10 2009-12-15 엘지전자 주식회사 Projection apparatus and controling method the same
CN101996406A (en) * 2010-11-03 2011-03-30 中国科学院光电技术研究所 No-reference structural sharpness image quality evaluation method
CN105516713A (en) * 2015-12-24 2016-04-20 招商局重庆交通科研设计院有限公司 Image quality evaluation method of road traffic closed-circuit television based on machine vision
CN106903093A (en) * 2017-03-08 2017-06-30 中国民航大学 A kind of embedded autonomous cleaning Vehicle of navaid light fixture and control method
CN108109147A (en) * 2018-02-10 2018-06-01 北京航空航天大学 A kind of reference-free quality evaluation method of blurred picture
CN108550146A (en) * 2018-04-11 2018-09-18 北京印刷学院 A kind of image quality evaluating method based on ROI
CN110549254A (en) * 2019-09-27 2019-12-10 北京爱德空港设备工程有限公司 Integrated navigation aid lamp cleaning and detecting vehicle
CN110916574A (en) * 2019-10-18 2020-03-27 上海善解人意信息科技有限公司 Sweeping robot system and sweeping robot control method

Similar Documents

Publication Publication Date Title
CN109084350B (en) Range hood with optical filtering function visual detection module and range hood concentration detection method
CN106782264B (en) Mould group color spot prosthetic device and method based on PCGPU
CN110850109B (en) Method for measuring vehicle speed based on fuzzy image
CN105900418A (en) Quality assessment of OLED stack films
WO2018010386A1 (en) Method and system for component inversion testing
CN104732900A (en) Pixel defect detection method and device
WO2016091041A1 (en) Method for performing dynamic evaluation on lighting glare of road by using imaging brightness meter
CN109165602A (en) A kind of black smoke vehicle detection method based on video analysis
CN1444183A (en) Method and device for tracking moving object in image
CN109756697A (en) A kind of computer room information technoloy equipment intelligent fault cruising inspection system
CN110231018A (en) A kind of structure light measurement method, apparatus and computer readable storage medium
CN112215145A (en) Tunnel lighting lamp cleaning effect monitoring method based on image comparison
US8224060B2 (en) Image processing method, paint inspection method and paint inspection system
CN106128112B (en) Night bayonet vehicle identifies grasp shoot method
Suder Parameters evaluation of cameras in embedded systems
CN107454319A (en) Image processing method, device, mobile terminal and computer-readable recording medium
CN109285151B (en) AI intelligent dimming method
TW201419168A (en) A method and system for license plate recognition under non-uniform illumination
Hanji et al. Hdr4cv: High dynamic range dataset with adversarial illumination for testing computer vision methods
CN216410941U (en) Image positioning auxiliary device suitable for vehicle-mounted drop hammer deflectometer
CN105654494A (en) Smog detection method for detecting smog in video image
CN109889818A (en) A kind of camera module light source test method
CN113762122A (en) Raindrop detection algorithm based on stroboscopic photograph
CN109003273A (en) A kind of car light light guide consistency detecting method
CN113259595A (en) Image acquisition method, image acquisition device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210112