CN114998355A - Production defect identification method and device for sealing rubber ring - Google Patents
Production defect identification method and device for sealing rubber ring Download PDFInfo
- Publication number
- CN114998355A CN114998355A CN202210941738.2A CN202210941738A CN114998355A CN 114998355 A CN114998355 A CN 114998355A CN 202210941738 A CN202210941738 A CN 202210941738A CN 114998355 A CN114998355 A CN 114998355A
- Authority
- CN
- China
- Prior art keywords
- pixel point
- image
- sliding window
- sub
- rubber ring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007789 sealing Methods 0.000 title claims abstract description 133
- 230000007547 defect Effects 0.000 title claims abstract description 40
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 23
- 230000002159 abnormal effect Effects 0.000 claims abstract description 123
- 230000011218 segmentation Effects 0.000 claims abstract description 12
- 238000004364 calculation method Methods 0.000 claims description 22
- 239000000126 substance Substances 0.000 claims description 17
- 238000009795 derivation Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 2
- 101100391182 Dictyostelium discoideum forI gene Proteins 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000003292 glue Substances 0.000 description 1
- 239000012535 impurity Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20192—Edge enhancement; Edge preservation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a method and a device for identifying production defects of a sealing rubber ring, belonging to the technical field of data processing, and the method comprises the following steps: acquiring an image of the sealing rubber ring, and dividing the image into a plurality of sub-images through the center of a circle of the sealing rubber ring in the image; if the abnormal probability value of any sub-image is larger than the preset abnormal probability threshold value, marking the sub-image as an abnormal sub-image; respectively carrying out linear enhancement on the abnormal sub-images by utilizing the first difference corresponding to each edge pixel point of each abnormal sub-image and the second difference corresponding to each internal pixel point in each abnormal sub-image to obtain enhanced sub-images, and carrying out threshold segmentation on the enhanced sealing ring images to identify the defect areas of the sealing rubber ring; according to the invention, the edge pixel points of the abnormal subimage in the sealing rubber ring image and the pixel points in the sealing rubber ring area are respectively enhanced, so that the accurate identification of the sealing rubber ring defect area is realized.
Description
Technical Field
The invention belongs to the technical field of data processing, and particularly relates to a method and a device for identifying production defects of a sealing rubber ring.
Background
The sealing rubber ring is an important industrial product in China, and is available at a connecting port in a plurality of machine equipment. The sealing rubber ring can have various defects in mass production, such as: the defects of material shortage, air bubble, impurity, flow mark, mold mark, glue overflow and the like are needed, so a series of quality detection is needed before the product is delivered.
However, the quality detection of the sealing rubber ring at present mainly depends on the traditional manual detection method, and the method not only needs a large amount of labor force to improve the production cost, but also easily causes the problems of large quality detection error, low efficiency, different standards among workers and the like, so that a new defect detection method of the sealing rubber ring is needed to replace the traditional manual detection method.
Disclosure of Invention
The invention provides a method and a device for identifying production defects of a sealing rubber ring.
The invention relates to a method for identifying production defects of a sealing rubber ring, which adopts the following technical scheme: the method comprises the following steps:
acquiring an image of the sealing rubber ring;
acquiring edge pixel points in the sealing rubber ring image, calculating the coordinates of the circle center of the sealing rubber ring according to the coordinate values of the edge pixel points, and dividing the sealing rubber ring image into a plurality of sub-images by making a plurality of straight lines through the circle center of the sealing rubber ring;
establishing a gray level histogram according to the gray level value of the pixel point in each subimage, and calculating the abnormal probability value of each subimage by using the gray level range corresponding to the first peak in the gray level histogram of each subimage;
if the abnormal probability value of any sub-image is larger than a preset abnormal probability threshold value, marking the sub-image as an abnormal sub-image, and if the abnormal probability value of any sub-image is smaller than the preset abnormal probability threshold value, marking the sub-image as a normal sub-image;
fitting an edge curve path of edge pixel points in each abnormal subimage, and taking an included angle between the tangent direction of each edge pixel point on the edge curve path and the horizontal positive direction as a reference angle;
establishing a sliding window by taking any edge pixel point on an edge curve path as a central pixel point, taking the rest pixel points in the sliding window as neighborhood pixel points of the central pixel point, taking an included angle between a connecting line of each neighborhood pixel point and the central pixel point in the sliding window and the positive horizontal direction as a neighborhood angle of each neighborhood pixel point in the sliding window, and determining the membership probability of each neighborhood pixel point in the sliding window according to the difference value of the neighborhood angle of each neighborhood pixel point in the sliding window and a reference angle corresponding to the neighborhood pixel point;
taking the membership probability of each neighborhood pixel point in each sliding window as the weight of the gray value of the neighborhood pixel point, calculating a first difference degree of the gray values of all the pixel points in the sliding window, and taking the first difference degree as the first difference degree of the center pixel point of the sliding window, thereby obtaining the first difference degree of each edge pixel point of each abnormal sub-image;
acquiring a sealing rubber ring area in a sealing rubber ring image, establishing a sliding window by taking any internal pixel point in the sealing rubber ring area as a central pixel point, calculating second difference of gray values of all pixel points in the sliding window, and taking the second difference as a second difference corresponding to each internal pixel point in an abnormal sub-image;
and respectively carrying out linear enhancement on the abnormal subimage according to the first difference of each edge pixel point and the second difference of each internal pixel point in the abnormal subimage to obtain an enhanced subimage, and carrying out threshold segmentation on the enhanced subimage to identify the defect area of the sealing rubber ring.
Further, the calculating the abnormal probability value of each sub-image by using the gray value range corresponding to the first peak in the gray value histogram of each sub-image includes:
performing polynomial fitting on the gray level histogram of each sub-image to obtain a gray level histogram function, performing derivation on the gray level histogram function, and taking a point which is first negative and then positive in derivative staggered multiplication as a valley point;
taking the distance between two valley points corresponding to the first peak in the gray level histogram of each sub-image as the gray level range of the sealing rubber ring area in the sub-image;
and calculating the abnormal probability value of each subimage according to the gray value range of the sealing rubber ring area in each subimage.
Further, the calculation formula of the anomaly probability value of each sub-image is as follows:
wherein the content of the first and second substances,
a valley point which represents the large gray value corresponding to the first peak in the gray histogram;
a valley point which represents the small gray value corresponding to the first peak in the gray histogram;
representing an anomaly probability value for each region;
is a hyperbolic tangent function.
Further, the calculation formula for determining the membership probability of each neighborhood pixel point in the sliding window according to the difference value between the neighborhood angle of each neighborhood pixel point in the sliding window and the reference angle corresponding to the neighborhood pixel point is as follows:
wherein the content of the first and second substances,
to show the second in the sliding window
Neighborhood angles of the individual neighborhood pixel points;
to show the second in the sliding window
A reference angle corresponding to each neighborhood pixel point;
representing the difference between the neighborhood angle and the reference angle;
to show the second in the sliding window
And (4) the membership probability of each neighborhood pixel.
Further, the calculating a first difference degree of gray values of all pixel points in the sliding window by using the membership probability of each neighborhood pixel point in each sliding window as the weight of the gray value of the neighborhood pixel point comprises:
taking the membership probability of each neighborhood pixel point in each sliding window as the weight of the gray value of the neighborhood pixel point to obtain the weighted gray value of each neighborhood pixel point in the sliding window;
calculating the mean value of gray values of all pixel points in each sliding window;
and calculating the difference between the weighted gray value of each neighborhood pixel point in each sliding window and the mean value of the gray values of all the pixel points in the sliding window, and taking the mean value of all the obtained difference values as the first difference of the gray values of all the pixel points in the sliding window.
Further, the calculation formula of the first difference degree is as follows:
wherein the content of the first and second substances,
to show the second in the sliding window
Membership probability of each neighborhood pixel point;
to show the second in the sliding window
Gray values of the neighborhood pixel points;
representing the mean value of gray values of all pixel points in the sliding window;
representing the total number of all pixel points in the sliding window;
representing a first degree of difference.
Further, the calculation formula of the second difference degree is as follows:
wherein the content of the first and second substances,
to show the second in the sliding window
Gray values of the neighborhood pixels;
representing the mean value of gray values of all pixel points in the sliding window;
representing the total number of all pixel points in the sliding window;
representing a second degree of difference.
Further, the performing linear enhancement on the abnormal sub-image according to the first difference of each edge pixel point and the second difference of each internal pixel point in the abnormal sub-image to obtain an enhanced sub-image includes:
performing linear enhancement on the gray value of each edge pixel point of the abnormal sub-image according to all the obtained first difference degrees to obtain an enhanced edge pixel point;
performing linear enhancement on the gray value of each internal pixel point of the abnormal sub-image according to all the obtained second difference degrees to obtain enhanced pixel points in the sealing ring area;
and adding the enhanced edge pixel points and the enhanced seal ring area pixel points to obtain enhanced sub-images.
A production defect recognition device of a sealing rubber ring comprises:
the image acquisition module is used for acquiring an image of the sealing rubber ring; the device is used for acquiring edge pixel points in the sealing rubber ring image, calculating the center coordinates of the sealing rubber ring according to the coordinate values of the edge pixel points, and dividing the sealing rubber ring image into a plurality of sub-images by making a plurality of straight lines through the center of the sealing rubber ring;
the abnormal subimage determining module is used for establishing a gray level histogram according to the gray level value of the pixel point in each subimage and calculating the abnormal probability value of each subimage by utilizing the gray level range corresponding to the first peak in the gray level histogram of each subimage; if the abnormal probability value of any sub-image is larger than a preset abnormal probability threshold value, marking the sub-image as an abnormal sub-image, and if the abnormal probability value of any sub-image is smaller than the preset abnormal probability threshold value, marking the sub-image as a normal sub-image;
the membership probability calculation module is used for fitting an edge curve path of edge pixel points in each abnormal subimage and taking an included angle between the tangential direction of each edge pixel point on the edge curve path and the positive horizontal direction as a reference angle; the method comprises the steps of establishing a sliding window by taking any edge pixel point on an edge curve path as a central pixel point, taking the rest pixel points in the sliding window as neighborhood pixel points of the central pixel point, taking the included angle between the connecting line of each neighborhood pixel point and the central pixel point in the sliding window and the positive horizontal direction as the neighborhood angle of the neighborhood pixel point in the sliding window, and determining the membership probability of each neighborhood pixel point in the sliding window according to the difference value of the neighborhood angle of each neighborhood pixel point in the sliding window and the reference angle corresponding to the neighborhood pixel point;
the first difference calculation module is used for calculating first differences of gray values of all pixel points in the sliding window by taking the membership probability of each neighborhood pixel point in each sliding window as the weight of the gray value of the neighborhood pixel point, and taking the first differences as the first differences of the center pixel point of the sliding window, so as to obtain the first differences of each edge pixel point of each abnormal subimage;
the second difference degree calculation module is used for acquiring a sealing rubber ring area in the sealing rubber ring image, establishing a sliding window by taking any internal pixel point in the sealing rubber ring area as a central pixel point, calculating second difference degrees of gray values of all pixel points in the sliding window, and taking the second difference degrees as second difference degrees corresponding to each internal pixel point in an abnormal sub-image;
and the defect area identification module is used for respectively carrying out linear enhancement on the abnormal subimage according to the first difference degree of each edge pixel point in the abnormal subimage and the second difference degree of each internal pixel point to obtain an enhanced subimage, and carrying out threshold segmentation on the enhanced subimage to identify the defect area of the sealing rubber ring.
The invention has the beneficial effects that:
the invention provides a method for identifying production defects of a sealing rubber ring, which comprises the steps of dividing the same sealing rubber ring image into a plurality of sub-images, calculating the abnormal probability value of each sub-image, marking any sub-image as an abnormal sub-image when the abnormal probability value of the sub-image is larger than a preset abnormal probability threshold value, and then calculating the selected abnormal sub-image, so that the calculated amount can be obviously reduced. After the abnormal subimages are selected, edge pixel points in the abnormal subimages and pixel points in the enhanced sealing ring area in the abnormal subimages are respectively enhanced to obtain enhanced subimages, and finally threshold segmentation is carried out on the enhanced subimages to identify the defect area of the sealing rubber ring.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flow chart illustrating the general steps of an embodiment of a method for identifying a production defect of a sealing rubber ring according to the present invention;
FIG. 2 is a schematic diagram showing an included angle between a tangential direction of each edge pixel point and a horizontal positive direction as a reference angle in the present invention;
fig. 3 is a schematic diagram of a sliding window established by using any edge pixel point as a center pixel point in the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
An embodiment of the method for identifying the production defects of the sealing rubber ring, as shown in fig. 1, comprises the following steps:
and S1, acquiring the sealing rubber ring image.
The invention utilizes image acquisition equipment to acquire images of the sealing rubber ring, and acquisition devices including a camera, a sampling platform, a light setting, a bracket and the like are required to be arranged during acquisition. And acquiring an image of the sealing rubber ring by using a camera, and performing weighted average graying processing on the acquired image of the sealing rubber ring to obtain a grayscale image of the sealing rubber ring.
S2, edge pixel points in the sealing rubber ring image are obtained, the center coordinates of the sealing rubber ring are calculated according to the coordinate values of the edge pixel points, and the sealing rubber ring image is divided into a plurality of sub-images by making a plurality of straight lines through the center of the sealing rubber ring.
And after the gray level image of the sealing rubber ring is acquired, reasonably partitioning the sealing rubber ring. Because the O-shaped sealing rubber ring is symmetrical and approximately round, the gray image of the O-shaped sealing rubber ring can be divided into a plurality of areas, and the center of the O-shaped sealing rubber ring needs to be determined before division. Firstly, extracting edge pixel points in an image of the sealing rubber ring by using an edge extraction algorithm, and then calculating the circle center coordinate of the sealing rubber ring according to the maximum value of the abscissa and the minimum value of the abscissa of the edge pixel points of the sealing rubber ring, and the maximum value of the ordinate and the minimum value of the ordinate of the edge pixel points of the sealing rubber ring, wherein the calculation formula of the circle center coordinate of the sealing rubber ring is shown as the following formula:
wherein the content of the first and second substances,
representing the maximum value of the abscissa of the pixel point at the edge of the sealing rubber ring;
expressing the minimum value of the abscissa of the pixel point at the edge of the sealing rubber ring;
the center abscissa of the sealing rubber ring is represented;
expressing the maximum value of the vertical coordinate of the pixel point at the edge of the sealing rubber ring;
expressing the minimum value of the vertical coordinate of the pixel point at the edge of the sealing rubber ring;
circle center longitudinal direction of indicating sealing rubber ringAnd (4) coordinates.
After the center coordinates of the sealing rubber ring are calculated, a plurality of straight lines are made through the center of the sealing rubber ring to divide the sealing rubber ring image into a plurality of sub-images.
S3, establishing a gray level histogram according to the gray level value of the pixel point in each sub-image, and calculating the abnormal probability value of each sub-image by using the gray level range corresponding to the first peak in the gray level histogram of each sub-image.
The method for calculating the abnormal probability value of each sub-image by using the gray value range corresponding to the first peak in the gray value histogram of each sub-image comprises the following steps: performing polynomial fitting on the gray level histogram of each sub-image to obtain a gray level histogram function, performing derivation on the gray level histogram function, and taking a point which is first negative and then positive in derivative staggered multiplication as a valley point; taking the distance between two valley points corresponding to the first peak in the gray level histogram of each sub-image as the gray level range of the sealing rubber ring area in the sub-image; and calculating the abnormal probability value of each subimage according to the gray value range of the sealing rubber ring area in each subimage.
According to the method, a gray level histogram is established according to the gray level value of the pixel point in each sub-image, and the gray level histogram is subjected to smooth denoising treatment. Because the defects in the sealing rubber ring are mainly detected, and the gray value in the sealing rubber ring is lower relative to the background area, the established gray histogram is analyzed. Due to the special structure of the sealing rubber ring, the constructed gray level histogram is mainly divided into two wave crests, one wave crest corresponds to the inner area of the rubber ring, and the other wave crest corresponds to the background area. Therefore, if a defect occurs in the sealing rubber ring, the range of the gray scale value in the corresponding rubber ring in the sub-image is enlarged. Therefore, the abnormal probability value of each sub-image is calculated according to the gray value range corresponding to the first peak in the gray value histogram of each sub-image.
The calculation formula of the anomaly probability value of each sub-image is as follows:
wherein the content of the first and second substances,
a valley point which represents the large gray value corresponding to the first peak in the gray histogram;
a valley point which represents the small gray value corresponding to the first peak in the gray histogram;
representing a probability value of the abnormality for each region;
is a hyperbolic tangent function for
Is limited to
Within the range.
And S4, if the abnormal probability value of any sub-image is greater than the preset abnormal probability threshold, marking the sub-image as an abnormal sub-image, and if the abnormal probability value of any sub-image is less than the preset abnormal probability threshold, marking the sub-image as a normal sub-image.
Calculating the abnormal probability value of each subimage individually, and setting the abnormal probability threshold value as
. And comparing the abnormal probability value of each sub-image with an abnormal probability threshold, if the abnormal probability value of any sub-image is greater than a preset abnormal probability threshold, marking the sub-image as an abnormal sub-image, and if the abnormal probability value of any area is less than the preset abnormal probability threshold, marking the sub-image as a normal sub-image. Threshold of probability of anomaly
The method can be determined according to the specific implementation situation of an implementer by detecting the gray value conversion range of a normal sealing rubber ring, and the experimental value is given in the scheme as follows:
。
and S5, fitting an edge curve path of edge pixel points in each abnormal sub-image, and taking an included angle between the tangential direction of each edge pixel point on the edge curve path and the positive horizontal direction as a reference angle.
The edge curve path of the edge pixel points in each abnormal subimage is an arc-shaped edge curve path, each edge pixel point on the arc-shaped edge curve path is tangent to an arc-shaped edge curve, and the included angle between the tangent direction of each edge pixel point and the positive horizontal direction is used as a reference angle
The reference angle is shown in fig. 2.
S6, establishing a sliding window by taking any edge pixel point on the edge curve path as a center pixel point, taking the rest pixel points in the sliding window as neighborhood pixel points of the center pixel point, taking the included angle between the connecting line of each neighborhood pixel point and the center pixel point in the sliding window and the positive horizontal direction as the neighborhood angle of each neighborhood pixel point in the sliding window, and determining the membership probability of each neighborhood pixel point in the sliding window according to the difference value of the neighborhood angle of each neighborhood pixel point in the sliding window and the reference angle corresponding to the neighborhood pixel point.
In the invention, any edge pixel point on the edge curve path is used as a central pixel point to establish
The sliding window is arranged on the top of the sliding window,
and the pixel points in the sliding window are neighborhood pixel points of the central pixel point. When the sliding window operation is performed, when the neighborhood pixels of the central pixel are counted for any edge pixel on the edge curve path, it is necessary to calculate the neighborhood pixels of the central pixelWhether the sliding window contains the background area pixel points or not is considered, and if the sliding window contains the background area pixel points, the gray value of the background area pixel points can affect the gray value difference degree of the pixel points in the sliding window. Therefore, the edge curve path of the sealing rubber ring is calculated by combining the structure of the sealing rubber ring, and the sliding window slides in the anticlockwise direction of the edge curve path of the sealing rubber ring. In a sliding window established by taking any edge pixel point on an edge curve path as a central pixel point, neighborhood pixel points of the central pixel point contain background region pixel points, and therefore the membership probability of each neighborhood pixel point in the sliding window belonging to a sealing rubber ring region needs to be calculated.
When calculating the membership probability of each neighborhood pixel point in the sliding window, the included angle between the connecting line of any neighborhood pixel point and the central pixel point in the sliding window and the positive horizontal direction is calculated as a neighborhood angle. As shown in fig. 3, is established by using any edge pixel as the center pixel
A sliding window, wherein the central pixel point in the sliding window is also any edge pixel point, and the included angle between the connecting line of the neighborhood pixel point 1 and the central pixel point and the positive horizontal direction is 0 degree, namely
0 deg. Similarly, the included angles between the connecting lines of the neighborhood pixel points 2, the neighborhood pixel points 3, the neighborhood pixel points 4, the neighborhood pixel points 5, the neighborhood pixel points 6, the neighborhood pixel points 7, the neighborhood pixel points 8 and the central pixel point and the horizontal positive direction are respectively obtained, namely, the neighborhood angle corresponding to each neighborhood pixel point is (
45°、
90°、
135°、
180°、
135°、
90°、
45°）。
The calculation formula of the membership probability of each neighborhood pixel point in the sliding window is shown as the following formula:
wherein the content of the first and second substances,
representing the neighborhood angle;
represents a reference angle;
representing the difference between the neighborhood angle and the reference angle;
to show the second in the sliding window
And (4) the membership probability of each neighborhood pixel.
S7, taking the membership probability of each neighborhood pixel point in each sliding window as the weight of the gray value of the neighborhood pixel point, calculating the first difference of the gray values of all the pixel points in the sliding window, and taking the first difference as the first difference of the center pixel point of the sliding window, thereby obtaining the first difference of each edge pixel point of each abnormal sub-image.
The method for calculating the gray value of each pixel point in the sliding window by taking the membership probability of each neighborhood pixel point in the sliding window as the weight of the gray value of the neighborhood pixel point comprises the following steps: taking the membership probability of each neighborhood pixel point in the sliding window as the weight of the gray value of the neighborhood pixel point to obtain the weighted gray value of each neighborhood pixel point in the sliding window; calculating the mean value of gray values of all pixel points in the sliding window; and calculating the difference between the weighted gray value of each neighborhood pixel point in the sliding window and the mean value of the gray values of all the pixel points in the sliding window, and taking the mean value of all the obtained difference values as the first difference of the gray values of all the pixel points in the sliding window.
The invention obtains the membership probability of each neighborhood pixel point in the sliding window through the step S6
Taking the membership probability of each neighborhood pixel point in the sliding window as the weight of the gray value of the neighborhood pixel point to obtain the weighted gray value of each neighborhood pixel point in the sliding window
. And calculating the first difference degree of the gray values of all the pixel points in the sliding window according to the weighted gray value of each neighborhood pixel point.
The calculation formula of the first difference is shown as follows:
wherein the content of the first and second substances,
to show the second in the sliding window
Membership probability of each neighborhood pixel point;
to show the second in the sliding window
Gray values of the neighborhood pixels;
representing the mean value of gray values of all pixel points in the sliding window;
representing the total number of all pixel points in the sliding window;
representing a first degree of difference.
S8, obtaining a sealing rubber ring area in the sealing rubber ring image, establishing a sliding window by taking any internal pixel point in the sealing rubber ring area as a central pixel point, calculating second difference of gray values of all pixel points in the sliding window, and taking the second difference as a second difference corresponding to each internal pixel point in the abnormal sub-image.
The calculation formula of the second difference degree is shown as follows:
wherein the content of the first and second substances,
indicating first in the sliding window
Gray values of the neighborhood pixel points;
representing the mean value of gray values of all pixel points in the sliding window;
representing the total number of all pixel points in the sliding window;
to representA second degree of difference.
S9, respectively carrying out linear enhancement on the abnormal subimage according to the first difference of each edge pixel point and the second difference of each internal pixel point in the abnormal subimage to obtain an enhanced subimage, and carrying out threshold segmentation on the enhanced subimage to identify the defect area of the sealing rubber ring.
The method for obtaining the enhanced sub-image by respectively performing linear enhancement on the abnormal sub-image according to the first difference of each edge pixel point and the second difference of each internal pixel point in the abnormal sub-image comprises the following steps: performing linear enhancement on the gray value of each edge pixel point of the abnormal sub-image according to all the obtained first difference degrees to obtain an enhanced edge pixel point; performing linear enhancement on the gray value of each internal pixel point of the abnormal sub-image according to all the obtained second difference degrees to obtain enhanced pixel points in the sealing ring area; and adding the enhanced edge pixel points and the enhanced seal ring area pixel points to obtain enhanced sub-images.
The formula for the linear enhancement is shown below:
wherein the content of the first and second substances,
representing the gray value of the edge pixel point;
represents the slope of the linear stretch;
representing a first degree of difference;
and expressing the gray value of the edge pixel point after enhancement.
Wherein, the first and the second end of the pipe are connected with each other,
representing a first degree of difference;
is the maximum value of the gray value range after linear stretching;
indicating the slope of the linear stretch.
Wherein the content of the first and second substances,
representing the gray value of pixel points in the sealing ring area;
represents the slope of the linear stretch;
representing a second degree of difference;
and expressing the gray value of the pixel point in the enhanced sealing ring area.
Wherein the content of the first and second substances,
representing a second degree of difference;
is the maximum value of the gray value range after linear stretching;
indicating the slope of the linear stretch.
And after the enhanced seal ring image is obtained, performing threshold segmentation on the enhanced seal ring image to identify a defect area of the seal rubber ring. Setting a threshold value
，
And dividing the enhanced seal ring image into three categories, namely a background category, a seal rubber ring normal area category and a seal rubber ring abnormal area category. The expression is as follows:
wherein the content of the first and second substances,
representing the gray value before threshold segmentation in the enhanced seal ring image;
representing the gray value after threshold segmentation in the enhanced seal ring image;
，
for setting the threshold, the implementer can set the threshold by himself, and the empirical value is given in the invention:
=40，
。
a production defect recognition device of a sealing rubber ring comprises:
the image acquisition module is used for acquiring an image of the sealing rubber ring; the device is used for acquiring edge pixel points in the sealing rubber ring image, calculating the center coordinates of the sealing rubber ring according to the coordinate values of the edge pixel points, and dividing the sealing rubber ring image into a plurality of sub-images by making a plurality of straight lines through the center of the sealing rubber ring;
the abnormal sub-image determining module is used for establishing a gray level histogram according to the gray level value of the pixel point in each sub-image and calculating the abnormal probability value of each sub-image by utilizing the gray level range corresponding to the first peak in the gray level histogram of each sub-image; if the abnormal probability value of any sub-image is larger than the preset abnormal probability threshold, marking the sub-image as an abnormal sub-image, and if the abnormal probability value of any sub-image is smaller than the preset abnormal probability threshold, marking the sub-image as a normal sub-image;
the membership probability calculation module is used for fitting an edge curve path of edge pixel points in each abnormal subimage and taking an included angle between the tangential direction of each edge pixel point on the edge curve path and the positive horizontal direction as a reference angle; the system is used for establishing a sliding window by taking any edge pixel point on an edge curve path as a central pixel point, taking the rest pixel points in the sliding window as neighborhood pixel points of the central pixel point, taking an included angle between a connecting line of each neighborhood pixel point and the central pixel point in the sliding window and a positive horizontal direction as a neighborhood angle of each neighborhood pixel point in the sliding window, and determining the membership probability of each neighborhood pixel point in the sliding window according to the difference value of the neighborhood angle of each neighborhood pixel point in the sliding window and a reference angle corresponding to the neighborhood pixel point;
the first difference calculating module is used for calculating first differences of gray values of all pixels in the sliding window by taking the membership probability of each neighborhood pixel in each sliding window as the weight of the gray value of the neighborhood pixel, and taking the first differences as the first differences of the center pixel of the sliding window, so as to obtain the first differences of each edge pixel of each abnormal subimage;
the second difference degree calculation module is used for acquiring a sealing rubber ring area in the sealing rubber ring image, establishing a sliding window by taking any internal pixel point in the sealing rubber ring area as a central pixel point, calculating second difference degrees of gray values of all pixel points in the sliding window, and taking the second difference degrees as second difference degrees corresponding to each internal pixel point in an abnormal sub-image;
and the defect area identification module is used for respectively carrying out linear enhancement on the abnormal subimage according to the first difference degree of each edge pixel point in the abnormal subimage and the second difference degree of each internal pixel point to obtain an enhanced subimage, and carrying out threshold segmentation on the enhanced subimage to identify the defect area of the sealing rubber ring.
In summary, the present invention provides a method and an apparatus for identifying a production defect of a rubber seal ring, which are configured to divide a same rubber seal ring image into a plurality of sub-images, calculate an abnormal probability value of each sub-image, mark any sub-image as an abnormal sub-image when the abnormal probability value of the sub-image is greater than a preset abnormal probability threshold, and then calculate the selected abnormal sub-image, so as to significantly reduce the calculation amount. After the abnormal subimages are selected, edge pixel points in the abnormal subimages and pixel points in the enhanced sealing ring area in the abnormal subimages are respectively enhanced to obtain enhanced subimages, and finally threshold segmentation is carried out on the enhanced subimages to identify the defect area of the sealing rubber ring.
Claims (9)
1. A production defect identification method of a sealing rubber ring is characterized by comprising the following steps:
acquiring an image of the sealing rubber ring;
acquiring edge pixel points in the sealing rubber ring image, calculating the center coordinates of the sealing rubber ring according to the coordinate values of the edge pixel points, and dividing the sealing rubber ring image into a plurality of sub-images by making a plurality of straight lines through the center of the sealing rubber ring;
establishing a gray level histogram according to the gray level value of the pixel point in each subimage, and calculating the abnormal probability value of each subimage by using the gray level range corresponding to the first peak in the gray level histogram of each subimage;
if the abnormal probability value of any sub-image is larger than a preset abnormal probability threshold value, marking the sub-image as an abnormal sub-image, and if the abnormal probability value of any sub-image is smaller than the preset abnormal probability threshold value, marking the sub-image as a normal sub-image;
fitting an edge curve path of edge pixel points in each abnormal subimage, and taking an included angle between the tangent direction of each edge pixel point on the edge curve path and the horizontal positive direction as a reference angle;
establishing a sliding window by taking any edge pixel point on an edge curve path as a central pixel point, taking the rest pixel points in the sliding window as neighborhood pixel points of the central pixel point, taking an included angle between a connecting line of each neighborhood pixel point and the central pixel point in the sliding window and the positive horizontal direction as a neighborhood angle of each neighborhood pixel point in the sliding window, and determining the membership probability of each neighborhood pixel point in the sliding window according to the difference value of the neighborhood angle of each neighborhood pixel point in the sliding window and a reference angle corresponding to the neighborhood pixel point;
taking the membership probability of each neighborhood pixel point in each sliding window as the weight of the gray value of the neighborhood pixel point, calculating a first difference degree of the gray values of all the pixel points in the sliding window, and taking the first difference degree as the first difference degree of the center pixel point of the sliding window, thereby obtaining the first difference degree of each edge pixel point of each abnormal sub-image;
acquiring a sealing rubber ring area in a sealing rubber ring image, establishing a sliding window by taking any internal pixel point in the sealing rubber ring area as a central pixel point, calculating second difference of gray values of all pixel points in the sliding window, and taking the second difference as a second difference corresponding to each internal pixel point in an abnormal sub-image;
and respectively carrying out linear enhancement on the abnormal subimage according to the first difference degree of each edge pixel point and the second difference degree of each internal pixel point in the abnormal subimage to obtain an enhanced subimage, and carrying out threshold segmentation on the enhanced subimage to identify the defect area of the sealing rubber ring.
2. The method for identifying the production defects of the sealing rubber ring according to claim 1, wherein the step of calculating the abnormal probability value of each sub-image by using the gray value range corresponding to the first peak in the gray value histogram of each sub-image comprises the following steps:
performing polynomial fitting on the gray level histogram of each sub-image to obtain a gray level histogram function, performing derivation on the gray level histogram function, and taking a point which is first negative and then positive in derivative staggered multiplication as a valley point;
taking the distance between two valley points corresponding to the first peak in the gray level histogram of each sub-image as the gray level range of the sealing rubber ring area in the sub-image;
and calculating the abnormal probability value of each sub-image according to the gray value range of the sealing rubber ring area in each sub-image.
3. The method for identifying the production defects of the sealing rubber ring according to claim 2, wherein the calculation formula of the abnormal probability value of each sub-image is as follows:
wherein the content of the first and second substances,
a valley point which represents the large gray value corresponding to the first peak in the gray histogram;
a valley point which represents the small gray value corresponding to the first peak in the gray histogram;
representing an anomaly probability value for each region;
is a hyperbolic tangent function.
4. The method for identifying the production defects of the sealing rubber ring according to claim 1, wherein the calculation formula for determining the membership probability of each neighborhood pixel point in the sliding window according to the difference value between the neighborhood angle of each neighborhood pixel point in the sliding window and the reference angle corresponding to the neighborhood pixel point is as follows:
wherein the content of the first and second substances,
to show the second in the sliding window
Neighborhood angles of the individual neighborhood pixel points;
to show the second in the sliding window
A reference angle corresponding to each neighborhood pixel point;
representing the difference between the neighborhood angle and the reference angle;
to show the second in the sliding window
And (4) the membership probability of each neighborhood pixel.
5. The method for identifying the production defects of the sealing rubber ring according to claim 4, wherein the step of calculating the first difference degree of the gray values of all the pixel points in the sliding window by taking the membership probability of each neighborhood pixel point in each sliding window as the weight of the gray value of the neighborhood pixel point comprises the following steps:
taking the membership probability of each neighborhood pixel point in each sliding window as the weight of the gray value of the neighborhood pixel point to obtain the weighted gray value of each neighborhood pixel point in the sliding window;
calculating the mean value of gray values of all pixel points in each sliding window;
and calculating the difference between the weighted gray value of each neighborhood pixel point in each sliding window and the mean value of the gray values of all the pixel points in the sliding window, and taking the mean value of all the obtained difference values as the first difference of the gray values of all the pixel points in the sliding window.
6. The method for identifying the production defects of the sealing rubber ring according to claim 5, wherein the calculation formula of the first difference is as follows:
wherein the content of the first and second substances,
to show the second in the sliding window
Membership probability of each neighborhood pixel point;
to show the second in the sliding window
Gray values of the neighborhood pixels;
representing the mean value of gray values of all pixel points in the sliding window;
representing the total number of all pixel points in the sliding window;
representing a first degree of difference.
7. The method for identifying the production defects of the sealing rubber ring according to claim 6, wherein the calculation formula for calculating the second difference degree of the gray values of all the pixel points in the sliding window is as follows:
wherein the content of the first and second substances,
to show the second in the sliding window
Gray values of the neighborhood pixels;
representing the mean value of gray values of all pixel points in the sliding window;
representing the total number of all pixel points in the sliding window;
representing a second degree of difference.
8. The method for identifying the production defects of the sealing rubber ring according to claim 1, wherein the step of respectively performing linear enhancement on the abnormal sub-image according to the first difference degree of each edge pixel point and the second difference degree of each internal pixel point in the abnormal sub-image to obtain an enhanced sub-image comprises the following steps:
performing linear enhancement on the gray value of each edge pixel point of the abnormal sub-image according to all the obtained first difference degrees to obtain an enhanced edge pixel point;
performing linear enhancement on the gray value of each internal pixel point of the abnormal sub-image according to all the obtained second difference degrees to obtain enhanced pixel points in the sealing ring area;
and adding the enhanced edge pixel points and the enhanced seal ring area pixel points to obtain enhanced sub-images.
9. A production defect recognition device of sealing rubber circle which characterized in that includes:
the image acquisition module is used for acquiring an image of the sealing rubber ring; the device is used for acquiring edge pixel points in the sealing rubber ring image, calculating the center coordinates of the sealing rubber ring according to the coordinate values of the edge pixel points, and dividing the sealing rubber ring image into a plurality of sub-images by making a plurality of straight lines through the center of the sealing rubber ring;
the abnormal sub-image determining module is used for establishing a gray level histogram according to the gray level value of the pixel point in each sub-image and calculating the abnormal probability value of each sub-image by utilizing the gray level range corresponding to the first peak in the gray level histogram of each sub-image; if the abnormal probability value of any sub-image is larger than the preset abnormal probability threshold, marking the sub-image as an abnormal sub-image, and if the abnormal probability value of any sub-image is smaller than the preset abnormal probability threshold, marking the sub-image as a normal sub-image;
the membership probability calculation module is used for fitting an edge curve path of edge pixel points in each abnormal subimage and taking the included angle between the tangential direction of each edge pixel point on the edge curve path and the positive horizontal direction as a reference angle; the system is used for establishing a sliding window by taking any edge pixel point on an edge curve path as a central pixel point, taking the rest pixel points in the sliding window as neighborhood pixel points of the central pixel point, taking an included angle between a connecting line of each neighborhood pixel point and the central pixel point in the sliding window and a positive horizontal direction as a neighborhood angle of each neighborhood pixel point in the sliding window, and determining the membership probability of each neighborhood pixel point in the sliding window according to the difference value of the neighborhood angle of each neighborhood pixel point in the sliding window and a reference angle corresponding to the neighborhood pixel point;
the first difference calculation module is used for calculating first differences of gray values of all pixel points in the sliding window by taking the membership probability of each neighborhood pixel point in each sliding window as the weight of the gray value of the neighborhood pixel point, and taking the first differences as the first differences of the center pixel point of the sliding window, so as to obtain the first differences of each edge pixel point of each abnormal subimage;
the second difference degree calculation module is used for acquiring a sealing rubber ring area in the sealing rubber ring image, establishing a sliding window by taking any internal pixel point in the sealing rubber ring area as a central pixel point, calculating second difference degrees of gray values of all pixel points in the sliding window, and taking the second difference degrees as second difference degrees corresponding to each internal pixel point in an abnormal sub-image;
and the defect region identification module is used for respectively carrying out linear enhancement on the abnormal subimages according to the first difference degree of each edge pixel point in the abnormal subimages and the second difference degree of each internal pixel point to obtain enhanced subimages, and carrying out threshold segmentation on the enhanced subimages to identify the defect region of the sealing rubber ring.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210941738.2A CN114998355B (en) | 2022-08-08 | 2022-08-08 | Production defect identification method and device for sealing rubber ring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210941738.2A CN114998355B (en) | 2022-08-08 | 2022-08-08 | Production defect identification method and device for sealing rubber ring |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114998355A true CN114998355A (en) | 2022-09-02 |
| CN114998355B CN114998355B (en) | 2023-06-30 |
Family
ID=83023160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210941738.2A Active CN114998355B (en) | 2022-08-08 | 2022-08-08 | Production defect identification method and device for sealing rubber ring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114998355B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115200797A (en) * | 2022-09-19 | 2022-10-18 | 山东超华环保智能装备有限公司 | Leakage detection system for zero leakage valve |
| CN115375588A (en) * | 2022-10-25 | 2022-11-22 | 山东旗胜电气股份有限公司 | Power grid transformer fault identification method based on infrared imaging |
| CN115393354A (en) * | 2022-10-27 | 2022-11-25 | 江苏集宿智能装备有限公司 | Laser cutting defect identification method |
| CN116452596A (en) * | 2023-06-20 | 2023-07-18 | 深圳市捷超行模具有限公司 | Visual detection method for glue overflow of injection molding of silica gel toy |
| CN116883270A (en) * | 2023-07-04 | 2023-10-13 | 广州医科大学附属第四医院(广州市增城区人民医院) | Soft mirror clear imaging system for lithotripsy operation |
| CN117764893A (en) * | 2024-01-08 | 2024-03-26 | 惠州市惠阳区兆顺益五金加工有限公司 | Numerical control machine tool wear identification method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120013747A1 (en) * | 2010-07-16 | 2012-01-19 | Primax Electronics Ltd. | Image testing method of image pickup device and image testing apparatus using such method |
| CN114708325A (en) * | 2022-03-31 | 2022-07-05 | 南通华烨塑料工业有限公司 | Method for quickly positioning rubber production problem based on rubber blooming defect |
-
2022
- 2022-08-08 CN CN202210941738.2A patent/CN114998355B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120013747A1 (en) * | 2010-07-16 | 2012-01-19 | Primax Electronics Ltd. | Image testing method of image pickup device and image testing apparatus using such method |
| CN114708325A (en) * | 2022-03-31 | 2022-07-05 | 南通华烨塑料工业有限公司 | Method for quickly positioning rubber production problem based on rubber blooming defect |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115200797A (en) * | 2022-09-19 | 2022-10-18 | 山东超华环保智能装备有限公司 | Leakage detection system for zero leakage valve |
| CN115200797B (en) * | 2022-09-19 | 2022-12-16 | 山东超华环保智能装备有限公司 | Leakage detection system for zero leakage valve |
| CN115375588A (en) * | 2022-10-25 | 2022-11-22 | 山东旗胜电气股份有限公司 | Power grid transformer fault identification method based on infrared imaging |
| CN115393354A (en) * | 2022-10-27 | 2022-11-25 | 江苏集宿智能装备有限公司 | Laser cutting defect identification method |
| CN116452596A (en) * | 2023-06-20 | 2023-07-18 | 深圳市捷超行模具有限公司 | Visual detection method for glue overflow of injection molding of silica gel toy |
| CN116452596B (en) * | 2023-06-20 | 2023-09-22 | 深圳市捷超行模具有限公司 | Visual detection method for glue overflow of injection molding of silica gel toy |
| CN116883270A (en) * | 2023-07-04 | 2023-10-13 | 广州医科大学附属第四医院(广州市增城区人民医院) | Soft mirror clear imaging system for lithotripsy operation |
| CN116883270B (en) * | 2023-07-04 | 2024-03-22 | 广州医科大学附属第四医院(广州市增城区人民医院) | Soft mirror clear imaging system for lithotripsy operation |
| CN117764893A (en) * | 2024-01-08 | 2024-03-26 | 惠州市惠阳区兆顺益五金加工有限公司 | Numerical control machine tool wear identification method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114998355B (en) | 2023-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114998355A (en) | Production defect identification method and device for sealing rubber ring | |
| CN108764229B (en) | Water gauge image automatic identification method based on computer vision technology | |
| CN105067638B (en) | Tire fetal membrane face character defect inspection method based on machine vision | |
| CN109377485B (en) | Machine vision detection method for instant noodle packaging defects | |
| CN108898610B (en) | Object contour extraction method based on mask-RCNN | |
| CN111709909B (en) | General printing defect detection method based on deep learning and model thereof | |
| CN107798326B (en) | Contour vision detection method | |
| CN108257128B (en) | Establishment method of pulmonary nodule detection device based on 3D convolutional neural network | |
| CN112529875A (en) | Photovoltaic module glass burst early warning method and system based on artificial intelligence | |
| CN107490582B (en) | Assembly line workpiece detection system | |
| CN105574533B (en) | A kind of image characteristic extracting method and device | |
| CN111080615B (en) | PCB defect detection system and detection method based on convolutional neural network | |
| CN110032946B (en) | Aluminum/aluminum blister packaging tablet identification and positioning method based on machine vision | |
| CN114972356B (en) | Plastic product surface defect detection and identification method and system | |
| CN114820625B (en) | Automobile top block defect detection method | |
| CN112465809B (en) | Mold defect detection method based on image recognition, computer readable storage medium and device | |
| CN113808180B (en) | Heterologous image registration method, system and device | |
| CN115147418B (en) | Compression training method and device for defect detection model | |
| CN114627080B (en) | Vehicle stamping accessory defect detection method based on computer vision | |
| CN105740872A (en) | Image feature extraction method and apparatus | |
| CN115797361B (en) | Aluminum template surface defect detection method | |
| CN111462057A (en) | Transmission line glass insulator self-explosion detection method based on deep learning | |
| CN112085709A (en) | Image contrast method and equipment | |
| CN113487563B (en) | EL image-based self-adaptive detection method for hidden cracks of photovoltaic module | |
| CN108805854B (en) | Method for rapidly counting tablets and detecting completeness of tablets in complex environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |