CN116721039A - Image preprocessing method applied to automatic optical defect detection - Google Patents

Image preprocessing method applied to automatic optical defect detection Download PDF

Info

Publication number
CN116721039A
CN116721039A CN202310989295.9A CN202310989295A CN116721039A CN 116721039 A CN116721039 A CN 116721039A CN 202310989295 A CN202310989295 A CN 202310989295A CN 116721039 A CN116721039 A CN 116721039A
Authority
CN
China
Prior art keywords
image
channel
formula
defect detection
histogram
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310989295.9A
Other languages
Chinese (zh)
Other versions
CN116721039B (en
Inventor
胡冰
程坦
刘涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhongkehaituo Wuxi Technology Co ltd
Original Assignee
Zhongkehaituo Wuxi Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhongkehaituo Wuxi Technology Co ltd filed Critical Zhongkehaituo Wuxi Technology Co ltd
Priority to CN202310989295.9A priority Critical patent/CN116721039B/en
Publication of CN116721039A publication Critical patent/CN116721039A/en
Application granted granted Critical
Publication of CN116721039B publication Critical patent/CN116721039B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/40Image enhancement or restoration by the use of histogram techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20132Image cropping
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Abstract

The application discloses an image preprocessing method applied to automatic optical defect detection, which relates to the technical field of image processing and comprises the steps of collecting an original image, inputting the original image, stretching a color channel, mapping a stretched RGB image to an HIS space and carrying out self-adaptive Gamma correction based on weighted histogram equalization. According to the Gamma self-adaptive correction method based on weighted histogram equalization, the contrast of the image can be enhanced on the premise of retaining the natural colors and abundant details of the original image in the pretreatment process of defect detection, and the problem that the contrast of the image cannot be enhanced on the premise of retaining the natural colors and abundant details of the original image in the pretreatment process of defect detection in the existing industrial detection is solved.

Description

Image preprocessing method applied to automatic optical defect detection
Technical Field
The application relates to the technical field of image processing, in particular to an image preprocessing method applied to automatic optical defect detection.
Background
In the industrial inspection process, due to the influence of the manufacturing process of the target plate, the color of the substrate and the illumination environment, the acquired image samples have deviations of exposure or color in and among the plates, which are important factors affecting the inspection effect, so that the enhancement operation of the input image is necessary before the inspection method is executed, and the reduction of the color of the image and the increase of the contrast of the image are essential requirements for image enhancement. However, in the existing industrial detection, in the pretreatment process of defect detection, the contrast of an image cannot be enhanced on the premise of retaining the natural color and abundant details of an original image. For this purpose, the application provides an image preprocessing method applied to automated optical defect detection.
Disclosure of Invention
Aiming at the defects of the prior art, the application provides an image preprocessing method applied to automatic optical defect detection, and a Gamma self-adaptive correction method based on weighted histogram equalization, which can enhance the contrast of an image on the premise of keeping the natural color and rich details of an original image in the preprocessing process of defect detection.
In order to achieve the above purpose, the application is realized by the following technical scheme: an image preprocessing method applied to automatic optical defect detection, comprising the following steps:
step one, collecting an original image: acquiring a defect image by utilizing an automatic defect detection mechanism, acquiring a corresponding non-defect image through image registration, enabling the non-defect image to be aligned with the defect image, and taking the defect image as an original image;
step two, inputting an original image: the input width and the input height are respectively and />RGB original image of (2), the number of pixels in the image is +.>Each color channel is limited to between 0 and 255, and the input color image expression is:
in the formula ,for pixel coordinates in each channel, +.>For inputting image coordinates>Pixel coordinates for the red channel,/>For green channel pixel coordinates +.>Pixel coordinates for blue channel, +.>
Step three, stretching the color channel: stretching each channel in the image within the stretching range of the maximum amplitude value tolerated by each channel;
mapping the stretched RGB image to an HIS space: only carrying out contrast enhancement processing on an I channel representing brightness, and retaining original information of an H channel and an S channel;
step five, self-adaptive Gamma correction based on weighted histogram equalization: the Gamma correction formula is used for carrying out self-adaptive correction on the weighted histogram equalization, and the expression is as follows:
wherein ,is a Gamma correction value; />Is a rounding operation; />A brightness level for an input image; />A maximum brightness level for an input image; />Accumulating relevant parameters of a distribution function for brightness weighting in an original image;
step six, converting the image: and combining the I channel after the self-adaptive Gamma correction with other two channels in the HIS space, and then converting the I channel back to the RGB color space to acquire an enhanced image.
As a further aspect of the present application, in step one, the automated defect detection mechanism uses an area camera with a 25mm lens as an acquisition sensor, uses an AOI light source to provide illumination, moves the camera through a motion control card, captures images of different areas on the surface of the detected object, each captured image having a resolution of 1280×1024 and a real scene coverage of 42.7mm×34.1mm, repeatedly confirms the captured defect-free images by at least 5 workers, and sets them as template images for defect detection.
As a further aspect of the present application, in the third step, stretching each channel in the image, and pixels in the stretched red channel, green channel, and blue channel may be expressed as:
in the formula :、/>maximum, minimum obtainable for all pixel positions in the red channel, respectively +.>、/>Maximum, minimum obtainable for all pixel positions in the green channel,/v>、/>Maximum, minimum available for all pixel positions in the blue channel.
In the fourth step, as a further scheme of the present application, a mapping formula for mapping the RGB image after the amplitude stretching into the HSI space is:
in the formula ,for HSI space, +.>In order to map the function of the function,for RGB image +.>Is jointly represented by the following three formulas:
in the formula :for the hue of the pixel location +.>For the color saturation of the pixel location, +.>For the brightness of the pixel location +.>Is the red pixel location in the cartesian coordinate system, and (2)>Is the red pixel location in the cartesian coordinate system, and (2)>Is the red pixel location in the cartesian coordinate system, and (2)>Is the hue position in the conical space model.
As a further aspect of the present application, in step five,the formula of (2) is:
wherein ,accumulating a distribution function for the luminance weights in the original function, < >>The relevant parameters of the distribution function are accumulated for the brightness weighting in the original image.
As a further aspect of the present application, in step five, the formula of the weighted cumulative distribution function of the brightness in the original image is calculated as:
in the formula ,for the variables->Maximum value of>For weighted probability density function, +.>Is the sum of weighted probability density functions.
In the fifth step, a weighted histogram distribution function is further constructed to modify the statistical histogram, where the weighted histogram distribution function formula is:
;/>
in the formula ,、/>the minimum probability density function and the maximum probability density function after the histogram modification are respectively +.>As a probability density function>Is->For correction parameters +.>By using the clipped statistical histogram, the histogram clipping operation is used to control the level of contrast enhancement.
As a further aspect of the present application, in the weighted histogram distribution function formula, the correction parameters areBy using the statistical histogram after clipping to carry out adaptive setting, the correction parameter setting formula is as follows:
wherein ,the formula of (2) is:
in the formula ,is->And (5) a cut histogram.
As a further aspect of the application, inIn the formula of->The formula for performing histogram clipping is:
in the formula ,for clipping limits, the clipping limits are taken as the mean value that occurs at each brightness level, and the clipping limits are formulated as:
the application has the beneficial effects that: the image preprocessing method applied to the automatic optical defect detection has the following beneficial effects: by adopting the Gamma self-adaptive correction method based on weighted histogram equalization, the contrast of the image can be enhanced and visual artifacts can be removed on the premise of keeping the natural colors and rich details of the original image in the preprocessing process of defect detection.
Drawings
FIG. 1 is a schematic diagram of an automated defect detection mechanism motion control card;
fig. 2 is a process contrast diagram for collecting too bright and too dark image samples, with the left side being a schematic diagram for processing too bright image samples and the right side being a schematic diagram for processing too dark image samples.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The application provides an image preprocessing method applied to automatic optical defect detection, which comprises the following steps:
step one, collecting an original image: acquiring a defect image by utilizing an automatic defect detection mechanism, acquiring a corresponding non-defect image through image registration, enabling the non-defect image to be aligned with the defect image, and taking the defect image as an original image;
step two, inputting an original image: the input width and the input height are respectively and />RGB original image of (2), the number of pixels in the image is +.>Each color channel is limited to between 0 and 255, and the input color image expression is:
in the formula ,for pixel coordinates in each channel, +.>For inputting image coordinates>Pixel coordinates for the red channel,/>For green channel pixel coordinates +.>Pixel coordinates for blue channel, +.>
Step three, stretching the color channel: stretching each channel in the image within the stretching range of the maximum amplitude value tolerated by each channel;
mapping the stretched RGB image to an HIS space: only carrying out contrast enhancement processing on an I channel representing brightness, and retaining original information of an H channel and an S channel;
step five, self-adaptive Gamma correction based on weighted histogram equalization: the Gamma correction formula is used for carrying out self-adaptive correction on the weighted histogram equalization, and the expression is as follows:
wherein ,is a Gamma correction value; />Is a rounding operation; />A brightness level for an input image; />A maximum brightness level for an input image; />Accumulating relevant parameters of a distribution function for brightness weighting in an original image;
step six, converting the image: and combining the I channel after the self-adaptive Gamma correction with other two channels in the HIS space, and then converting the I channel back to the RGB color space to acquire an enhanced image.
It should be noted that, since the human visual system is sensitive to the brightness intensity change, in the subsequent contrast enhancement operation, only the I channel representing the brightness is subjected to enhancement processing, and the original information of the H channel and the S channel is retained, because the information in the two channels does not directly affect the human perception of the image contrast.
As shown in fig. 1, in the first step, the automated defect detecting mechanism uses an area camera with a 25mm lens as an acquisition sensor, uses an AOI light source to provide illumination, moves the camera through a motion control card to capture images of different areas on the surface of the detected object, each captured image has a resolution of 1280×1024 and a real scene coverage of 42.7mm×34.1mm, and repeatedly confirms the captured defect-free images by at least 5 workers and sets them as template images for defect detection.
Specifically, in the third step, stretching each channel in the image, and the pixels in the stretched red channel, green channel, and blue channel may be expressed as:
in the formula :、/>maximum, minimum obtainable for all pixel positions in the red channel, respectively +.>、/>Maximum, minimum obtainable for all pixel positions in the green channel,/v>、/>Maximum and minimum values available for all pixel positions in the blue channel; the stretched image is more informative in color and is also ready for subsequent contrast enhancement operations.
Further, in the fourth step, a mapping formula for mapping the RGB image after the amplitude stretching into the HSI space is:
in the formula ,for HSI space, +.>In order to map the function of the function,for RGB image +.>Is jointly represented by the following three formulas:
in the formula :for the hue of the pixel location +.>For the color saturation of the pixel location, +.>For the brightness of the pixel location +.>Is the red pixel location in the cartesian coordinate system, and (2)>Is the red pixel location in the cartesian coordinate system, and (2)>Is the red pixel location in the cartesian coordinate system, and (2)>Is the hue position in the conical space model.
In the fifth step, the first step is performed,the formula of (2) is:
wherein ,accumulating a distribution function for the luminance weights in the original function, < >>The relevant parameters of the distribution function are accumulated for the brightness weighting in the original image.
Specifically, in step five, the formula of the weighted cumulative distribution function of the luminance in the original image is calculated as:
in the formula ,for the variables->Maximum value of>For weighted probability density function, +.>Is the sum of weighted probability density functions.
Specifically, in the fifth step, a weighted histogram distribution function is further constructed to modify the statistical histogram to reduce the adverse reaction, where the weighted histogram distribution function formula is:
;/>
in the formula ,、/>the minimum probability density function and the maximum probability density function after the histogram modification are respectively +.>As a probability density function>Is->For correction parameters +.>By using the clipped statistical histogram, the histogram clipping operation is used to control the level of contrast enhancement.
It should be noted that, in the weighted histogram distribution function formula, the correction parametersBy using the statistical histogram after clipping to carry out adaptive setting, the correction parameter setting formula is as follows:
wherein ,the formula of (2) is: />; in the formula ,/>Is a cut histogram.
Further, inIn the formula of->The formula for performing histogram clipping is:
in the formula ,for clipping limits, the clipping limits are taken as the mean value that occurs at each brightness level, and the clipping limits are formulated as:
according to the Gamma self-adaptive correction method based on weighted histogram equalization, the contrast of the image can be enhanced and visual artifacts can be removed on the premise of keeping natural colors and rich details of the original image in the pretreatment process of defect detection.
Examples. As shown in fig. 2, the sample of the too bright and too dark images is taken as the input test image, while the image enhanced by the method presented herein has better image quality, does not generate any visual artifacts, and the detected object in the image looks clearer and the color is brighter.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Finally: the foregoing description of the preferred embodiments of the application is not intended to limit the application, but to enable any modification, equivalent or improvement to be made without departing from the spirit and principles of the application.

Claims (9)

1. An image preprocessing method applied to automatic optical defect detection, which is characterized by comprising the following steps:
step one, collecting an original image: acquiring a defect image by utilizing an automatic defect detection mechanism, acquiring a corresponding non-defect image through image registration, enabling the non-defect image to be aligned with the defect image, and taking the defect image as an original image;
step two, inputting an original image: the input width and the input height are respectively and />RGB original image of (2), the number of pixels in the image is +.>Each color channel is limited to between 0 and 255, and the input color image expression is:
in the formula ,for pixel coordinates in each channel, +.>For inputting image coordinates>Pixel coordinates for the red channel,/>For green channel pixel coordinates +.>Pixel coordinates for the blue channel; />;/>
Step three, stretching the color channel: stretching each channel in the image within the stretching range of the maximum amplitude value tolerated by each channel;
mapping the stretched RGB image to an HIS space: only carrying out contrast enhancement processing on an I channel representing brightness, and retaining original information of an H channel and an S channel;
step five, self-adaptive Gamma correction based on weighted histogram equalization: the Gamma correction formula is used for carrying out self-adaptive correction on the weighted histogram equalization, and the expression is as follows:
wherein ,is a Gamma correction value; />Is a rounding operation; />A brightness level for an input image; />A maximum brightness level for an input image; />Accumulating relevant parameters of a distribution function for brightness weighting in an original image;
step six, converting the image: and combining the I channel after the self-adaptive Gamma correction with other two channels in the HIS space, and then converting the I channel back to the RGB color space to acquire an enhanced image.
2. An image preprocessing method applied to automated optical defect detection according to claim 1, wherein in step one, an automated defect detection mechanism uses an area camera with a 25mm lens as an acquisition sensor, uses an AOI light source to provide illumination, moves the camera through a motion control card, captures images of different areas on the surface of the detected object, each captured image has a resolution of 1280×1024, a real scene coverage of 42.7mm×34.1mm, repeatedly confirms the captured defect-free images by at least 5 workers, and sets it as a template image for defect detection.
3. An image preprocessing method for use in automated optical defect detection according to claim 2, wherein in step three, stretching is performed on each channel in the image, and pixels in the stretched red channel, green channel and blue channel can be represented as:
in the formula :、/>maximum, minimum obtainable for all pixel positions in the red channel, respectively +.>、/>Maximum, minimum obtainable for all pixel positions in the green channel,/v>、/>Maximum, minimum available for all pixel positions in the blue channel.
4. The method for image preprocessing used in automated optical defect detection according to claim 1, wherein in step four, a mapping formula for mapping the RGB image subjected to the amplitude stretching into the HSI space is:
in the formula ,for HSI space, +.>For the mapping function +.>For RGB image +.>Is jointly represented by the following three formulas:
in the formula :for the hue of the pixel location +.>For the color saturation of the pixel location, +.>For the brightness of the pixel location +.>Is the red pixel location in the cartesian coordinate system, and (2)>Is the red pixel location in the cartesian coordinate system, and (2)>Is the red pixel location in the cartesian coordinate system, and (2)>Is a conical spaceTone position in the model.
5. An image preprocessing method for use in automated optical defect detection as defined in claim 1, wherein in step five,the formula of (2) is:
wherein ,accumulating a distribution function for the luminance weights in the original function, < >>The relevant parameters of the distribution function are accumulated for the brightness weighting in the original image.
6. The method of claim 5, wherein in step five, the formula of the weighted cumulative distribution function of brightness in the original image is:
in the formula ,for the variables->Maximum value of>For weighted probability density function, +.>Is the sum of weighted probability density functions.
7. The method of claim 6, wherein in step five, a weighted histogram distribution function is further constructed to modify the statistical histogram, and the weighted histogram distribution function formula is:
;/>
in the formula ,、/>the minimum probability density function and the maximum probability density function after the histogram modification are respectively,as a probability density function>Is->Is a correction parameter.
8. The method of claim 7, wherein the correction parameters are in a weighted histogram distribution function formulaCorrection by adaptive setting using clipped statistical histogramThe positive parameter setting formula is:
wherein ,the formula of (2) is:
in the formula ,is a cut histogram.
9. The method for image preprocessing used in automated optical defect detection according to claim 8, wherein, in the following stepsIn the formula of->The formula for performing histogram clipping is:
in the formula ,for histogram->For clipping limits, the clipping limits are taken as the mean value that occurs at each brightness level, and the clipping limits are formulated as:
CN202310989295.9A 2023-08-08 2023-08-08 Image preprocessing method applied to automatic optical defect detection Active CN116721039B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310989295.9A CN116721039B (en) 2023-08-08 2023-08-08 Image preprocessing method applied to automatic optical defect detection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310989295.9A CN116721039B (en) 2023-08-08 2023-08-08 Image preprocessing method applied to automatic optical defect detection

Publications (2)

Publication Number Publication Date
CN116721039A true CN116721039A (en) 2023-09-08
CN116721039B CN116721039B (en) 2023-11-03

Family

ID=87873729

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310989295.9A Active CN116721039B (en) 2023-08-08 2023-08-08 Image preprocessing method applied to automatic optical defect detection

Country Status (1)

Country Link
CN (1) CN116721039B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117495845A (en) * 2023-12-25 2024-02-02 宁德时代新能源科技股份有限公司 Blue gel detection method and device, electronic equipment and medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109146811A (en) * 2018-08-14 2019-01-04 长沙全度影像科技有限公司 A kind of Adaptive contrast enhancement method of color image
CN109146826A (en) * 2018-06-27 2019-01-04 华南理工大学 A kind of image enchancing method and device
CN113989147A (en) * 2021-10-26 2022-01-28 安徽大学 Endoscope image enhancement method based on histogram equalization and improved unsharp mask
CN114331896A (en) * 2021-12-30 2022-04-12 安徽大学 Image enhancement method and system based on image decomposition and spectral transformation
CN116468636A (en) * 2023-04-20 2023-07-21 无锡唐古半导体有限公司 Low-illumination enhancement method, device, electronic equipment and readable storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109146826A (en) * 2018-06-27 2019-01-04 华南理工大学 A kind of image enchancing method and device
CN109146811A (en) * 2018-08-14 2019-01-04 长沙全度影像科技有限公司 A kind of Adaptive contrast enhancement method of color image
CN113989147A (en) * 2021-10-26 2022-01-28 安徽大学 Endoscope image enhancement method based on histogram equalization and improved unsharp mask
CN114331896A (en) * 2021-12-30 2022-04-12 安徽大学 Image enhancement method and system based on image decomposition and spectral transformation
CN116468636A (en) * 2023-04-20 2023-07-21 无锡唐古半导体有限公司 Low-illumination enhancement method, device, electronic equipment and readable storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117495845A (en) * 2023-12-25 2024-02-02 宁德时代新能源科技股份有限公司 Blue gel detection method and device, electronic equipment and medium

Also Published As

Publication number Publication date
CN116721039B (en) 2023-11-03

Similar Documents

Publication Publication Date Title
CN104749184B (en) Automatic optical detection method and system
CN111563889B (en) Liquid crystal screen Mura defect detection method based on computer vision
CN101350933B (en) Method for regulating lighteness of filmed display screen based on image inductor
CN103716547A (en) Smart mode photographing method
CN101140365A (en) Unevenness inspection method, method for manufacturing display panel, and unevenness inspection apparatus
CN116721039B (en) Image preprocessing method applied to automatic optical defect detection
CN108037142B (en) Mask optical defect detection method based on image gray value
CN112583999B (en) Method for detecting lens dirt of camera module
CN107404647A (en) Camera lens condition detection method and device
CN112419261B (en) Visual acquisition method and device with abnormal point removing function
CN109801231B (en) Image processing method of electrophoresis electronic paper detection equipment
CN104749801B (en) High Precision Automatic optical detecting method and system
CN101312494B (en) Method for computing camera response curve and synthesizing image with large dynamic range and apparatus therefor
CN113657335A (en) Mineral phase identification method based on HSV color space
CN111563869B (en) Stain test method for quality inspection of camera module
CN113450289B (en) Method for automatically enhancing low illumination of face image in passenger traffic scene
CN109448012A (en) A kind of method for detecting image edge and device
CN108833873A (en) A kind of luminance uniformity motion compensation process
CN112183158B (en) Cereal type identification method of cereal cooking equipment and cereal cooking equipment
CN109028234B (en) Range hood capable of identifying smoke grade
JP2004226272A (en) Method and apparatus for detecting stain defect
TWI780465B (en) Defect-inspecting method of goggles and a system thereof
JP3433333B2 (en) Defect inspection method
JP4036048B2 (en) Defect identification method
JP3941403B2 (en) Image density unevenness detection method and inspection apparatus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant