CN109580630A - A kind of visible detection method of component of machine defect - Google Patents
A kind of visible detection method of component of machine defect Download PDFInfo
- Publication number
- CN109580630A CN109580630A CN201811334772.3A CN201811334772A CN109580630A CN 109580630 A CN109580630 A CN 109580630A CN 201811334772 A CN201811334772 A CN 201811334772A CN 109580630 A CN109580630 A CN 109580630A
- Authority
- CN
- China
- Prior art keywords
- component
- image
- machine
- defect
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007547 defect Effects 0.000 title claims abstract description 92
- 238000001514 detection method Methods 0.000 title claims abstract description 31
- 238000013461 design Methods 0.000 claims abstract description 16
- 238000000034 method Methods 0.000 claims description 43
- 238000004458 analytical method Methods 0.000 claims description 21
- 238000012545 processing Methods 0.000 claims description 21
- 238000005516 engineering process Methods 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 14
- 238000003325 tomography Methods 0.000 claims description 14
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 230000011218 segmentation Effects 0.000 claims description 12
- 230000008859 change Effects 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000005457 optimization Methods 0.000 claims description 6
- 230000003595 spectral effect Effects 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 6
- 235000009508 confectionery Nutrition 0.000 claims description 4
- 238000003708 edge detection Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 3
- 238000001228 spectrum Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 2
- 239000000203 mixture Substances 0.000 claims description 2
- 230000008569 process Effects 0.000 description 9
- 230000000694 effects Effects 0.000 description 7
- 230000006872 improvement Effects 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 6
- 238000007689 inspection Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000004069 differentiation Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 241001270131 Agaricus moelleri Species 0.000 description 1
- 206010061619 Deformity Diseases 0.000 description 1
- 229910000831 Steel Inorganic materials 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004886 process control Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
- G01B11/005—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates coordinate measuring machines
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N23/00—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00
- G01N23/02—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material
- G01N23/04—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material and forming images of the material
- G01N23/046—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material and forming images of the material using tomography, e.g. computed tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N2223/00—Investigating materials by wave or particle radiation
- G01N2223/10—Different kinds of radiation or particles
- G01N2223/101—Different kinds of radiation or particles electromagnetic radiation
- G01N2223/1016—X-ray
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N2223/00—Investigating materials by wave or particle radiation
- G01N2223/40—Imaging
- G01N2223/401—Imaging image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Pathology (AREA)
- Analytical Chemistry (AREA)
- Immunology (AREA)
- General Health & Medical Sciences (AREA)
- Biochemistry (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Geometry (AREA)
- Radiology & Medical Imaging (AREA)
- Pulmonology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of visible detection methods of component of machine defect, the rejected region on inside component of machine and surface can be subjected to continuous scanning, and the shape of the area for obtaining surface and internal defect, defect center-of-mass coordinate and defect is handled the image of acquisition and is analyzed, it allows the operator to be clearly seen that rejected region;Then the texture on component of machine surface, the threedimensional model of color character and internal structure are restored by three-dimensional reconstruction reverse engineering design, to show its component of machine surface and internal flaw position and indicated range, and the three-dimensional reconfiguration system of defect is rendered as three-dimensional PDF document.
Description
Technical field
The present invention relates to visible detection method more particularly to a kind of visible detection methods of component of machine defect.
Background technique
With the development for seeing industrial technology, mechanical part processing method show high-precision, at high speed, automation development become
Gesture, conventional contacts detection method speed is slower, is easy to cause to damage to part, can reduce measurement accuracy, is based on computer vision
Measuring technique have many advantages, such as at high speed, real-time it is good, it is non-by touching, low cost, be widely used in various parts process lack
Sunken Precision measurement.
Detection and examine be one of activity most basic in manufacturing process, by detect and inspection activity provide product and its
The quality information of manufacturing process implements control according to manufacturing process of these information to product --- it is modified and compensates work
It is dynamic, minimize waster with reprocessed products rate, the stability of guarantee product quality forming process and its output product
Consistency;Traditional detection and inspection relies primarily on people, and completes mainly by manual mode, its not only time-consuming but also cost makes to produce
Period increases, and the production cost increases;Meanwhile traditional inspection and detection activity mainly carries out after processing and manufacturing process,
Once detecting waster, loss has occurred;In addition, the information based on artificial detection, usually comprises the influence of the error of people,
Manufacturing process is controlled by such information, it is not only just implementable after process, but also the error of people's difference can be introduced;Therefore,
This information realization cannot be relied in real time or in machine process control;New detection and inspection is often with a variety of advanced sensing skills
Based on art, and it is easy to same computer system and combines;Under suitable software support, this kind of automatic detection or checking system
Data sampling and processing, feature extraction and identification and a variety of analyses can be automatically completed and calculated.
The present invention provides a kind of visible detection method of component of machine defect, using 3-D scanning technology and tomoscan
The advantage reduction component of machine defect sturcture of the high-acruracy survey of technology and the accurate-sizes of broken parts, elaborate position with
Carry out convenient for subsequent finer work.
Summary of the invention
The purpose of the present invention is overcome the deficiencies of the prior art and provide a kind of vision-based detection side of component of machine defect
Method carries out continuous scanning by rejected region to component of machine inside and surface, and to the image of acquisition carry out processing and
The shape for analyzing the area for obtaining surface and internal defect, defect center-of-mass coordinate and defect allows the operator to clear
Rejected region is seen by Chu;Then texture, the color on component of machine surface are restored by three-dimensional reconstruction reverse engineering design
The threedimensional model of feature and internal structure, to show its component of machine surface and internal flaw position and defect model
It encloses, and the three-dimensional reconfiguration system of defect is rendered as three-dimensional PDF document.
To achieve the above object, the technical solution of the invention is as follows: a kind of vision-based detection side of component of machine defect
Method, it is characterised in that: this method comprises:
3-D scanning is carried out to component of machine surface using three-dimensional coordinate measuring instrument and close shot video camera and is closely imaged simultaneously
Store image;
The noise and retroreflective regions that influence characteristics of image are denoised using median filter method, threshold is carried out by ostu algorithm
The automatic segmentation of value carries out binary conversion treatment to image;
Pixel edge is first obtained using Candy edge detection algorithm, two dimension is then utilized using space matrix edge positioning mode
Spatial gradation matrix determines the edge sub-pixel of marginal position to refine to edge, and then by slightly positioning edge to essence
Inside Accuracy Refinement to pixel, to enhance the surface smoothness and edge sharpening of image;
Then the color for using focusing solutions analysis component of machine surface carries out image line to the component of machine that analysis is completed
The analysis for managing feature, matches the image texture characteristic after optimization, convenient for the subsequent company to component of machine surface texture
The analysis of continuous property judges the continuity of component of machine surface texture and color character using identification technology, and to zero mechanical
The non-edge for being smoothly connected region in part surface carries out registration matching primitives, by computer to the continuous of component of machine surface
Property is analyzed, and component of machine surface image after analysis is stored;
Three-dimensional tomographic is carried out to component of machine using tomography thin layer acquisition method, the faultage image of acquisition is become by gray scale
It changes processing and carries out image enhancement, binary conversion treatment then is carried out to the image after image enhancement by Binarization methods, it is then right
Defect area, defect mass center and defect shape inside image are calculated and are stored;
The three of the texture on component of machine surface, color character and internal structure are restored by three-dimensional reconstruction reverse engineering design
Dimension module, to show its component of machine surface and internal flaw position and indicated range;
The location and shape of component of machine defect sturcture and broken parts are restored, and the three-dimensional reconfiguration system of defect is presented
For three-dimensional PDF document.
As a further improvement of the present invention, described to utilize three-dimensional coordinate measuring instrument and close shot video camera to component of machine
Surface carries out 3-D scanning and closely images and store image to be specifically zero mechanical using three-dimensional laser scanner scanning
When part surface, the three-dimensional laser scanning system of grade and submillimeter level is comprehensively utilized, is carried out with millimetre-sized spatial digitizer
Whole control, carries out local data's acquisition with the spatial digitizer of submillimeter level, then in conjunction with close-range photogrammetry, realizes mechanical
The whole of component surface information is left and taken.
It is further limited as what 3-D scanning and short distance imaged, firstly, being swashed using grade and submillimeter level three-dimensional
Optical scanning instrument obtains the intensive point data of component of machine entirety, is compared by multidate point data, obtains component of machine
The deformation and defect shape of surface partial structurtes;Then close-range photogrammetry is used, is obtained by way of projection or texture expansion
The striograph for obtaining component of machine, by the slight change of image recognition technology detection texture color.
As a further improvement of the present invention, it is used in described image identification technology method particularly includes: be to close first
Scape image is corrected and pre-processes, and obtains the calibration image on component of machine surface, then extracts certain pixel pair in image
Then the curve of spectrum answered carries out PCA dimension-reduction treatment to image, image is reduced to 3 dimensions as the spectral signature at the point, use
Feature extractor based on CNN carries out the extraction of feature, using the feature extracted as space characteristics, finally by spectral signature with
Space characteristics carry out fusion linear feature and form spectrum-sky characteristic set, and SVM is used to be classified to obtain machinery zero as classifier
The result of color continuity between parts surface.
As a further improvement of the present invention, optimizing processing to the noise and retroreflective regions that influence characteristics of image includes
Main includes point cloud, noise reduction except superfluous, data segmentation, filtering, spots cloud optimization processing.
As a further improvement of the present invention, the continuity judgment method tool of the surface texture between component of machine surface
Body is with the following method:
Firstly, statistics angle point quantity: one arc length threshold values A is as the standard compared for design, passes through the mean arc between data point
Long weighting obtains A value;When the neighbor distance of continuous multiple points is continuously less than or equal to A, then being considered as the region, there are 1 angles
Point, and the data point in angle steel joint region is marked;
Then, reject the redundant points in Texture Points cloud: one angle threshold values B of design calculates adjacent 3 points Pti-1、Pt、Pti+1, 1
The angle a of two vectors composed by≤i≤N-1i, wherein N is the positive integer greater than 1, if aiIt, then will be intermediate greater than threshold values B
Data point Pt is deleted, until all offset points carry out differentiation completion;
Finally, connection whole story endpoint Pt0PtN-1, each point is calculated to line distance hi, and calculate the number with angle point label
Strong point PtiWith adjacent two data point Pti-1、Pti+1The gained angle a of composed two vectorsi, with intermediate data points distance
hiRatio ai/hi, there is minimum value min(ai/hi) intermediate data points be 1 known angle point, delete adjacent area data
The angle point label of point, complex texture section is split, and subtract 1 for angle point counter, continues this to new texture section
Step often judges that an angle point, counter subtract 1, differentiates until counter is kept to 0 stopping.
The main mesh of image enhancement processing is carried out in the present invention to the original mechanical components inner scanning image acquired
The local characteristics being in prominent faultage image, original unsharp internal flaw is apparent from, it is different in enlarged image
Difference between object features makes to enhance the differentiation between different piece in image, obscures the feature of irrelevant portions, formed anti-
Difference comparison, so that the parts of images quality for needing to study in image, information content are more abundant, thus more efficiently and effectively to adding
It is partially identified by force, reaches the demand to image analysis research, reinforce internal flaw interpretation and recognition effect.
Greyscale transformation image processing techniques is carried out in spatial domain, and spatial domain processing can be expressed from the next:
G (x, y)=T [f (x, y)], wherein f (x, y) is input picture, and g (x, y) is the image after processing, and T is in point (x, y)
Field on a kind of operator about f for defining, operator can be applied to independent image or image collection, in one width figure of spatial domain
A neighborhood as in about point (x, y), from a pixel to one other pixel, movement is defeated to generate one in the picture for neighborhood
Image out, processing step are as follows:
1) neighborhood origin first shifts to one other pixel from a pixel, to the pixel application operator T in neighborhood, and produces in the position
Raw output exports value of the image g at these coordinates and is equal to (x, y) being original in f for the position (x, y) being arbitrarily designated
The result of neighborhood of a point application operator T;
2) origin of neighborhood continuously moves to next position, and the circulating repetition above process can generate next output image g
Value, the process are first since the upper left position of input picture, are then handled pixel-by-pixel in a manner of horizontal sweep, often
Secondary a line.
The specifically used inverse transform of grey scale change in the present invention, available tonal range are the piece image of [0, L-1]
Reverse image, which is given by: s=L-1-r;Make the gray level for inverting piece image in this way, it can be with
The egative film of equivalent photo is obtained, this seed type is used in particular for enhancing embedded white or grey in the dark areas of piece image
Details, especially when occupying an leading position in black area size.
After image enhancement, the component of machine internal image defect for the ease of obtaining to tomoscan carries out prominent aobvious
Show, set 0 or 255 for the gray value of the pixel on image, that is, whole image is showed into apparent black and white effect,
To more significantly highlight internal flaw.
On the basis of image enhancement, binary conversion treatment is carried out to image;Binary conversion treatment is specifically chosen most in the present invention
Big Ostu method.
The gray scale of 256 brightness degrees is handled by choosing threshold values appropriate in binarization, so as to
Obtain the image of reflection image local or global feature;In image procossing, binary conversion treatment image analyzes subsequent image
Very important effect.
Firstly, image binaryzation process can effectively prepare for being further processed for image, image is made there was only 0 He
255 two states reduce data volume, and can highlight the objective contour to be obtained;Secondly, processing and analysis of binary figure
The premise of picture is that gray level image is first carried out binary conversion treatment, binary image can be just got, be conducive to do image in this way
The set property of subsequent analysis, image is only related with the position of point that pixel value is 0 or 255, does not further relate to the multistage of pixel
Value makes processing become simple, and data processing and decrement it is small.
The binary conversion treatment of image is exactly that the gray scale of the point on image is set to 0 or 255, and all gray scales are greater than or equal to valve
The pixel of value is defined as wanting highlighted part, and gray value is 255 expressions, and the pixel of rest part is then identified as
Other than region, gray value sets 0, indicates background or and uninterested region;It is apparent black exactly to say that whole image shows
White effect, ideal bianry image, general using the region closed, the boundary definition of connection does not overlap in order to obtain.
After image binaryzation is handled, inside component of machine, the black hole got, is exactly component of machine
Internal defect;Statistical is carried out to component of machine internal flaw elemental area by calculating internal flaw black region
Defect area, defect mass center and the defect shape data of analysis you can get it component of machine internal flaw.
The defect area to inside image, defect mass center and defect shape, which are calculated, specifically obtains machinery zero
Tomoscan image internal flaw inside part, loops through the boundary of each connected region, then calculates defect boundary institute envelope surface
Long-pending and marking of defects mass center simultaneously seeks defect center-of-mass coordinate;Then each closure defect boundary of circular treatment obtains boundary coordinate,
The points for calculating description enclosed region boundary, obtain the shape of defect.
As a further improvement of the present invention, tomography thin layer acquisition method is specifically with the following method: being set using tomoscan
It is standby that thin layer scanning storage is carried out to component of machine, segmentation result is inputted, and the segmentation side appropriate according to image feature selection
Method constructs contour surface using segmentation result;Then, two-dimensional slice image is analyzed and is handled, the system that scanning is obtained
Column two-dimensional slice image constructs 3D data volume by tomography interpolation.
As a further improvement of the present invention, CT equipment includes sweep test, computer system and image display storage system
System;The sweep test is made of X-ray tube, detector and scanning support;The computer system is will to scan the information being collected into
Data carry out storage operation and construct threedimensional model;Described image shows that storage system is the figure that will be handled, rebuild through computer
As being shown in display screen.
Three-dimensional reconstruction reverse engineering design is by carrying out conversed analysis to component of machine structure, to deduce and obtain machine
The internal structure of tool components, so that the continuity to component of machine internal structure is assessed.
Wherein, three-dimensional reconstruction reverse engineering design is a kind of product design technology reproducing processes, i.e., to a target product
Conversed analysis and research are carried out, to deduce and obtain process flow, institutional framework, functional characteristic and the technical specification of the product
Equal design elements, to produce similar product.
The present invention carries out continuous scanning by the rejected region to component of machine inside and surface, and then to acquisition
Image is handled and is analyzed the shape of the area for obtaining surface and internal defect, defect center-of-mass coordinate and defect, so that
Operator can be clearly seen that rejected region;Then the component of machine surface image and tomography obtained to 3-D scanning is swept
The component of machine internal image that face obtains is fitted, and is then restored by three-dimensional reconstruction reverse engineering design zero mechanical
The threedimensional model of the texture on part surface, color character and internal structure lacks to show its component of machine surface and inside
Position and indicated range are fallen into, and the three-dimensional reconfiguration system of defect is rendered as three-dimensional PDF document.
The beneficial effects of the present invention are:
1, the present invention takes 3-D scanning technology and layer scanning technology to machinery zero when detecting component of machine internal flaw
Parts surface and inside are scanned, and are then rebuild reverse engineering design by 3D again and are restored component of machine surface and inside
Structure three-dimensional model, to show internal defects position and indicated range.
2, the present invention obtains the intensive of component of machine entirety using grade and submillimeter level three-dimensional laser scanner device
Point cloud data is compared by multidate point cloud data, obtains the deformation and defect shape of component of machine partial structurtes;Then,
Using close-range photogrammetry, the orthophotoquad of component of machine is obtained by way of projection or texture expansion, by image
Identification technology detects the slight change of texture color, these are greatly facilitated, and subsequent mechanical component surface is non-to be smoothly connected area
The matching in domain improves computer to the accuracy of component of machine Surface Defect Recognition.
3, the present invention first obtains pixel edge using Candy edge detection algorithm, is then positioned using spatial moment edge
Method determined using two-dimensional space gray matrix the edge sub-pixel of marginal position to be refined to edge, and then by slightly to essence
Edge precision is refine to inside pixel, the surface smoothness and edge sharpening of image can be effectively enhanced.
Detailed description of the invention
The attached drawing for constituting a part of the invention is used to provide further understanding of the present invention, schematic reality of the invention
It applies example and its explanation is used to explain the present invention, do not constitute improper limitations of the present invention;In the accompanying drawings:
Fig. 1 is a kind of schematic diagram of the visible detection method of component of machine defect of the present invention.
Fig. 2 is a kind of flow diagram of the visible detection method of component of machine defect of the present invention.
Specific embodiment
The present invention and its specific embodiment are described in further detail with reference to the accompanying drawing.
A kind of visible detection method of component of machine defect, this method comprises:
3-D scanning is carried out to component of machine surface using three-dimensional coordinate measuring instrument and close shot video camera and is closely imaged simultaneously
Store image;
The noise and retroreflective regions that influence characteristics of image are denoised using median filter method, threshold is carried out by ostu algorithm
The automatic segmentation of value carries out binary conversion treatment to image;
Pixel edge is first obtained using Candy edge detection algorithm, two dimension is then utilized using space matrix edge positioning mode
Spatial gradation matrix determines the edge sub-pixel of marginal position to refine to edge, and then by slightly positioning edge to essence
Inside Accuracy Refinement to pixel, to enhance the surface smoothness and edge sharpening of image;
Then the color for using focusing solutions analysis component of machine surface carries out image line to the component of machine that analysis is completed
The analysis for managing feature, matches the image texture characteristic after optimization, convenient for the subsequent company to component of machine surface texture
The analysis of continuous property judges the continuity of component of machine surface texture and color character using identification technology, and to zero mechanical
The non-edge for being smoothly connected region in part surface carries out registration matching primitives, by computer to the continuous of component of machine surface
Property is analyzed, and component of machine surface image after analysis is stored;
Three-dimensional tomographic is carried out to component of machine using tomography thin layer acquisition method, the faultage image of acquisition is become by gray scale
It changes processing and carries out image enhancement, binary conversion treatment then is carried out to the image after image enhancement by Binarization methods, it is then right
Defect area, defect mass center and defect shape inside image are calculated and are stored;
The three of the texture on component of machine surface, color character and internal structure are restored by three-dimensional reconstruction reverse engineering design
Dimension module, to show its component of machine surface and internal flaw position and indicated range;
The location and shape of component of machine defect sturcture and broken parts are restored, and the three-dimensional reconfiguration system of defect is presented
For three-dimensional PDF document.
Wherein, 3-D scanning and low coverage are carried out to component of machine surface using three-dimensional coordinate measuring instrument and close shot video camera
It is specifically from imaging and storing image: when scanning mechanical component surface using three-dimensional laser scanner, comprehensively utilizes millimeter
The three-dimensional laser scanning system of grade and submillimeter level carries out whole control with millimetre-sized spatial digitizer, with submillimeter level
Spatial digitizer carries out local data's acquisition, then in conjunction with close-range photogrammetry, realizes the whole of component of machine surface information
It leaves and takes.
Above step specifically: firstly, being obtained using grade and submillimeter level three-dimensional laser scanner device zero mechanical
The intensive point data of part entirety, is compared by multidate point data, is obtained the deformation of component of machine surface partial structurtes and is lacked
Disfigurement shape;Then close-range photogrammetry is used, the striograph of component of machine is obtained by way of projection or texture expansion, is borrowed
Help the slight change of image recognition technology detection texture color.
Wherein, it is used in image recognition technology method particularly includes: it is that close shot image is corrected and is pre-processed first,
Obtain the calibration image on component of machine surface, then extract image in the corresponding curve of spectrum of certain pixel as the point at
Then spectral signature carries out PCA dimension-reduction treatment to image, image is reduced to 3 dimensions, carried out using the feature extractor based on CNN
Spectral signature and space characteristics are finally carried out linear character and melted by the extraction of feature using the feature extracted as space characteristics
Conjunction forms spectrum-sky characteristic set, and SVM is used to be classified to obtain color continuity between component of machine surface as classifier
Result.
Wherein, to the noise and retroreflective regions that influence characteristics of image to optimize processing include main include a point cloud,
Noise reduction is except superfluous, data segmentation, filtering, spots cloud optimization processing;Binary conversion treatment is specifically chosen maximum variance between clusters.
Wherein, defect area, defect mass center and the defect shape inside image are calculated and is stored specifically: being obtained
Tomoscan image internal flaw inside component of machine, loops through the boundary of each connected region, then calculates defect side
Boundary's institute's envelope surface product and marking of defects mass center simultaneously seek defect center-of-mass coordinate;Then each closure defect boundary of circular treatment obtains
Boundary coordinate calculates the points on description enclosed region boundary, the shape of defect is obtained, finally to the defect inside the image sought
Area, defect mass center and defect shape data are stored.
Wherein, tomography thin layer acquisition method includes CT equipment acquisition data and three-dimensional reconstruction, and the CT equipment acquisition data are
Component of machine is subjected to thin layer scanning storage, then 3 d structure model is restored by three-dimensional reconstruction reverse engineering design.
Wherein, tomography thin layer acquisition method is specifically with the following method: being carried out using tomographic apparatus to component of machine
Thin layer scanning storage, segmentation result is inputted, and is split according to characteristics of image to image, is constructed using segmentation result equivalent
Face;Then, two-dimensional slice image is analyzed and is handled, a series of two-dimensional slice images that scanning obtains are inserted by tomography
Value construction interior three-dimensional data structure.
Tomographic apparatus includes that sweep test, computer system and image show storage system;The sweep test is by X
Spool, detector and scanning support composition;The computer system is that the information data for being collected into scanning carries out storage operation;Institute
It states image and shows that storage system is that the image for handling, rebuilding through computer is shown in display screen.
The present invention carries out continuous scanning by the rejected region to component of machine inside and surface, and then to acquisition
Image is handled and is analyzed the shape of the area for obtaining surface and internal defect, defect center-of-mass coordinate and defect, so that
Operator can be clearly seen that rejected region;Then the component of machine surface image and tomography obtained to 3-D scanning is swept
The component of machine internal image that face obtains is fitted, and is then restored by three-dimensional reconstruction reverse engineering design zero mechanical
The threedimensional model of the texture on part surface, color character and internal structure lacks to show its component of machine surface and inside
Position and indicated range are fallen into, and the three-dimensional reconfiguration system of defect is rendered as three-dimensional PDF document.
The foregoing is only a preferred embodiment of the present invention, but scope of protection of the present invention is not limited thereto,
Anyone skilled in the art in the technical scope disclosed by the present invention, according to the technique and scheme of the present invention and its
Inventive concept is subject to equivalent substitution or change, should be covered by the protection scope of the present invention.
Claims (10)
1. a kind of visible detection method of component of machine defect, it is characterised in that: this method comprises:
3-D scanning is carried out to component of machine surface using three-dimensional coordinate measuring instrument and close shot video camera and is closely imaged simultaneously
Store image;
The noise and retroreflective regions that influence characteristics of image are denoised using median filter method, threshold is carried out by ostu algorithm
The automatic segmentation of value carries out binary conversion treatment to image;
Pixel edge is first obtained using Candy edge detection algorithm, space matrix edge positioning mode is then used, utilizes two
Dimension space gray matrix determines the edge sub-pixel of marginal position, to refine to edge, and then by slightly to essence by edge
Positioning accuracy is refine to inside pixel, to enhance the surface smoothness and edge sharpening of image;
Then the color for using focusing solutions analysis component of machine surface carries out image line to the component of machine that analysis is completed
The analysis for managing feature, matches the image texture characteristic after optimization, convenient for the subsequent company to component of machine surface texture
The analysis of continuous property judges the continuity of component of machine surface texture and color character using identification technology, and to zero mechanical
The non-edge for being smoothly connected region in part surface carries out registration matching primitives, by computer to the continuous of component of machine surface
Property is analyzed, and component of machine surface image after analysis is stored;
Three-dimensional tomographic is carried out to component of machine using tomography thin layer acquisition method, the faultage image of acquisition is become by gray scale
It changes processing and carries out image enhancement, binary conversion treatment then is carried out to the image after image enhancement by Binarization methods, it is then right
Defect area, defect mass center and defect shape inside image are calculated and are stored;
The three of the texture on component of machine surface, color character and internal structure are restored by three-dimensional reconstruction reverse engineering design
Dimension module, to show its component of machine surface and internal flaw position and indicated range;
The location and shape of component of machine defect sturcture and broken parts are restored, and the three-dimensional reconfiguration system of defect is presented
For three-dimensional PDF document.
2. a kind of visible detection method of component of machine defect according to claim 1, it is characterised in that: described to utilize three
Dimension coordinate measuring apparatus and close shot video camera carry out 3-D scanning to component of machine surface and closely image and store image tool
Body is: when scanning mechanical component surface using three-dimensional laser scanner, comprehensively utilizing the three-dimensional of grade and submillimeter level
Laser scanning system carries out whole control with millimetre-sized spatial digitizer, carries out part with the spatial digitizer of submillimeter level
Data acquisition realizes that the whole of component of machine surface information is left and taken then in conjunction with close-range photogrammetry.
3. a kind of visible detection method of component of machine defect according to claim 2, it is characterised in that: firstly, utilizing
Grade and submillimeter level three-dimensional laser scanner device obtain the intensive point data of component of machine entirety, are counted by multidate
According to comparison, the deformation and defect shape of component of machine surface partial structurtes are obtained;Then close-range photogrammetry is used, throwing is passed through
Shadow or the mode of texture expansion obtain the striograph of component of machine, by the subtle change of image recognition technology detection texture color
Change.
4. a kind of visible detection method of component of machine defect according to claim 1, it is characterised in that: described image is known
It is used in other technology method particularly includes: be that close shot image is corrected and is pre-processed first, obtain component of machine surface
Calibration image, then extract image in the corresponding curve of spectrum of certain pixel as the spectral signature at the point, then to figure
As carrying out PCA dimension-reduction treatment, image is reduced to 3 dimensions, the extraction of feature is carried out using the feature extractor based on CNN, will be extracted
Spectral signature and space characteristics are finally carried out fusion linear feature and form spectrum-sky feature set by feature out as space characteristics
It closes, SVM is used to be classified to obtain the result of color continuity between component of machine surface as classifier.
5. a kind of visible detection method of component of machine defect according to claim 1, it is characterised in that: described pair of influence
It include main includes a point cloud, noise reduction except superfluous, data segmentation, filter that the noise and retroreflective regions of characteristics of image, which optimize processing,
Wave, spots cloud optimization processing.
6. a kind of visible detection method of component of machine defect according to claim 5, it is characterised in that: the binaryzation
Processing specific choice maximum variance between clusters.
7. a kind of visible detection method of component of machine defect according to claim 1, it is characterised in that: described to image
Internal defect area, defect mass center and defect shape is calculated and is stored specifically: obtaining tomography inside component of machine
Scan image internal flaw loops through the boundary of each connected region, then calculates defect boundary institute envelope surface product and label lacks
It falls into mass center and seeks defect center-of-mass coordinate;Then each closure defect boundary of circular treatment obtains boundary coordinate, calculates description and closes
The points for closing zone boundary, obtain the shape of defect, finally to defect area, defect mass center and the defect inside the image sought
Shape data is stored.
8. a kind of visible detection method of component of machine defect according to claim 1, it is characterised in that: the tomography is thin
Layer acquisition method includes CT equipment acquisition data and three-dimensional reconstruction, and the CT equipment acquisition data are that component of machine is carried out thin layer
Scanning storage, then internal structure threedimensional model is restored by three-dimensional reconstruction reverse engineering design.
9. a kind of visible detection method of component of machine defect according to claim 8, it is characterised in that: the tomography is thin
Layer acquisition method be specifically with the following method: carrying out thin layer scanning storage to component of machine using tomographic apparatus, will divide
As a result it inputs, and the dividing method appropriate according to image feature selection, constructs contour surface using segmentation result;Then, to two dimension
Sectioning image is analyzed and is handled, and a series of two-dimensional slice images that scanning is obtained construct three-dimensional data by tomography interpolation
Body.
10. a kind of visible detection method of component of machine defect according to claim 9, it is characterised in that: tomoscan
Equipment includes that sweep test, computer system and image show storage system;The sweep test is by X-ray tube, detector and sweeps
Retouch frame composition;The computer system is that the information data for being collected into scanning carries out storage operation;Described image display storage
System is that the image for handling, rebuilding through computer is shown in display screen.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811334772.3A CN109580630B (en) | 2018-11-10 | 2018-11-10 | Visual inspection method for defects of mechanical parts |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811334772.3A CN109580630B (en) | 2018-11-10 | 2018-11-10 | Visual inspection method for defects of mechanical parts |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109580630A true CN109580630A (en) | 2019-04-05 |
CN109580630B CN109580630B (en) | 2022-02-18 |
Family
ID=65922026
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201811334772.3A Expired - Fee Related CN109580630B (en) | 2018-11-10 | 2018-11-10 | Visual inspection method for defects of mechanical parts |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109580630B (en) |
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110052569A (en) * | 2019-05-22 | 2019-07-26 | 安庆市吉安汽车零件锻轧有限公司 | A kind of intelligence gear shaft forging rolling technique |
CN110197479A (en) * | 2019-05-23 | 2019-09-03 | 中国科学院力学研究所 | A kind of modified quality determining method in surface |
CN110400286A (en) * | 2019-06-05 | 2019-11-01 | 山东科技大学 | The detection localization method of metal needle in a kind of X ray CT image |
CN110738180A (en) * | 2019-10-21 | 2020-01-31 | 易思维(杭州)科技有限公司 | Method for evaluating signal accuracy and system precision in detection process |
CN110806736A (en) * | 2019-11-19 | 2020-02-18 | 北京工业大学 | Method for detecting quality information of forge pieces of die forging forming intelligent manufacturing production line |
CN112304960A (en) * | 2020-12-30 | 2021-02-02 | 中国人民解放军国防科技大学 | High-resolution image object surface defect detection method based on deep learning |
CN112484664A (en) * | 2020-11-26 | 2021-03-12 | 江苏国和智能科技有限公司 | Defect identification device and method based on laser three-dimensional scanning |
CN112967280A (en) * | 2021-04-07 | 2021-06-15 | 苏州工业职业技术学院 | Three-dimensional visual defect detection method based on CT |
CN113793317A (en) * | 2021-09-13 | 2021-12-14 | 合肥艾克斯特智能科技有限公司 | Plate part identification code spraying method |
CN114034704A (en) * | 2021-10-18 | 2022-02-11 | 武汉科技大学 | Waste roller three-dimensional detection device and method based on multi-source data fusion |
CN114092506A (en) * | 2022-01-20 | 2022-02-25 | 北京微链道爱科技有限公司 | Scanning device with 3D (three-dimensional) characteristics and scanning method |
CN114740005A (en) * | 2021-12-23 | 2022-07-12 | 西北工业大学 | SiCfQuantitative evaluation method for machining surface quality of/SiC ceramic matrix composite |
CN116595214A (en) * | 2023-07-14 | 2023-08-15 | 中国航发北京航空材料研究院 | Component non-contact automatic shape correction system based on image processing |
CN117871415A (en) * | 2024-03-11 | 2024-04-12 | 天津大学四川创新研究院 | Exposure type structural flaw detection system and method based on parallel light source |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103239256A (en) * | 2013-05-29 | 2013-08-14 | 中国人民解放军第三军医大学第一附属医院 | Virtual X-ray imaging method and virtual X-ray imaging system for human body bone joint |
CN104266587A (en) * | 2014-09-22 | 2015-01-07 | 电子科技大学 | Three-dimensional measurement system and method for obtaining actual 3D texture point cloud data |
CN105912874A (en) * | 2016-04-29 | 2016-08-31 | 青岛大学附属医院 | Liver three-dimensional database system constructed on the basis of DICOM (Digital Imaging and Communications in Medicine) medical image |
CN106204733A (en) * | 2016-07-22 | 2016-12-07 | 青岛大学附属医院 | Liver and the three-dimensional constructing system of kidney CT image associating |
-
2018
- 2018-11-10 CN CN201811334772.3A patent/CN109580630B/en not_active Expired - Fee Related
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103239256A (en) * | 2013-05-29 | 2013-08-14 | 中国人民解放军第三军医大学第一附属医院 | Virtual X-ray imaging method and virtual X-ray imaging system for human body bone joint |
CN104266587A (en) * | 2014-09-22 | 2015-01-07 | 电子科技大学 | Three-dimensional measurement system and method for obtaining actual 3D texture point cloud data |
CN105912874A (en) * | 2016-04-29 | 2016-08-31 | 青岛大学附属医院 | Liver three-dimensional database system constructed on the basis of DICOM (Digital Imaging and Communications in Medicine) medical image |
CN106204733A (en) * | 2016-07-22 | 2016-12-07 | 青岛大学附属医院 | Liver and the three-dimensional constructing system of kidney CT image associating |
Non-Patent Citations (1)
Title |
---|
金炎芳: "基于三维CT图像的涡轮叶片缺陷检测技术研究", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 * |
Cited By (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110052569A (en) * | 2019-05-22 | 2019-07-26 | 安庆市吉安汽车零件锻轧有限公司 | A kind of intelligence gear shaft forging rolling technique |
CN110197479A (en) * | 2019-05-23 | 2019-09-03 | 中国科学院力学研究所 | A kind of modified quality determining method in surface |
CN110400286A (en) * | 2019-06-05 | 2019-11-01 | 山东科技大学 | The detection localization method of metal needle in a kind of X ray CT image |
CN110738180A (en) * | 2019-10-21 | 2020-01-31 | 易思维(杭州)科技有限公司 | Method for evaluating signal accuracy and system precision in detection process |
CN110738180B (en) * | 2019-10-21 | 2022-05-17 | 易思维(杭州)科技有限公司 | Method for evaluating signal accuracy and system precision in detection process |
CN110806736B (en) * | 2019-11-19 | 2021-10-15 | 北京工业大学 | Method for detecting quality information of forge pieces of die forging forming intelligent manufacturing production line |
CN110806736A (en) * | 2019-11-19 | 2020-02-18 | 北京工业大学 | Method for detecting quality information of forge pieces of die forging forming intelligent manufacturing production line |
CN112484664A (en) * | 2020-11-26 | 2021-03-12 | 江苏国和智能科技有限公司 | Defect identification device and method based on laser three-dimensional scanning |
CN112304960A (en) * | 2020-12-30 | 2021-02-02 | 中国人民解放军国防科技大学 | High-resolution image object surface defect detection method based on deep learning |
CN112967280A (en) * | 2021-04-07 | 2021-06-15 | 苏州工业职业技术学院 | Three-dimensional visual defect detection method based on CT |
CN113793317A (en) * | 2021-09-13 | 2021-12-14 | 合肥艾克斯特智能科技有限公司 | Plate part identification code spraying method |
CN114034704A (en) * | 2021-10-18 | 2022-02-11 | 武汉科技大学 | Waste roller three-dimensional detection device and method based on multi-source data fusion |
CN114740005A (en) * | 2021-12-23 | 2022-07-12 | 西北工业大学 | SiCfQuantitative evaluation method for machining surface quality of/SiC ceramic matrix composite |
CN114092506A (en) * | 2022-01-20 | 2022-02-25 | 北京微链道爱科技有限公司 | Scanning device with 3D (three-dimensional) characteristics and scanning method |
CN114092506B (en) * | 2022-01-20 | 2022-04-15 | 北京微链道爱科技有限公司 | Scanning device with 3D (three-dimensional) characteristics and scanning method |
CN116595214A (en) * | 2023-07-14 | 2023-08-15 | 中国航发北京航空材料研究院 | Component non-contact automatic shape correction system based on image processing |
CN117871415A (en) * | 2024-03-11 | 2024-04-12 | 天津大学四川创新研究院 | Exposure type structural flaw detection system and method based on parallel light source |
Also Published As
Publication number | Publication date |
---|---|
CN109580630B (en) | 2022-02-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109580630A (en) | A kind of visible detection method of component of machine defect | |
CN109087274B (en) | Electronic device defect detection method and device based on multi-dimensional fusion and semantic segmentation | |
CN109242888B (en) | Infrared and visible light image fusion method combining image significance and non-subsampled contourlet transformation | |
CN107230203A (en) | Casting defect recognition methods based on human eye vision attention mechanism | |
CN109559324A (en) | A kind of objective contour detection method in linear array images | |
CN111354047B (en) | Computer vision-based camera module positioning method and system | |
CN111602177B (en) | Method and apparatus for generating a 3D reconstruction of an object | |
CN109410139A (en) | Inside a kind of historical relic and surface disease digital assay appraisal procedure | |
CN106327451A (en) | Image restorative method of ancient animal fossils | |
CN115861409B (en) | Soybean leaf area measuring and calculating method, system, computer equipment and storage medium | |
CN105787870A (en) | Graphic image splicing fusion system | |
Chen et al. | A color-guided, region-adaptive and depth-selective unified framework for Kinect depth recovery | |
CN106780718A (en) | A kind of three-dimensional rebuilding method of paleontological fossil | |
Anusree et al. | Characterization of sand particle morphology: state-of-the-art | |
CN113390357B (en) | Rivet levelness measuring method based on binocular multi-line structured light | |
CN117218192A (en) | Weak texture object pose estimation method based on deep learning and synthetic data | |
Chen | Image Edge Detection Algorithm of Machined Parts Based on Mathematical Morphology | |
Li et al. | MCFDFusion: Multi-focus image fusion based on multi-scale cross-difference and focus detection | |
CN113554688B (en) | O-shaped sealing ring size measurement method based on monocular vision | |
CN115880371A (en) | Method for positioning center of reflective target under infrared visual angle | |
Jazayeri et al. | Interest operators in close-range object reconstruction | |
Chidambaram | Edge Extraction of Color and Range Images | |
Zhou et al. | A fast and robust visual detection method with binary template matching for glass bottle body | |
CN111626299A (en) | Outline-based digital character recognition method | |
Bao et al. | Research on machine vision size measurement method based on particle weight |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20220218 |