KR101562988B1 - Apparatus and method for detecting surface defects of hot billet - Google Patents

Apparatus and method for detecting surface defects of hot billet Download PDF

Info

Publication number
KR101562988B1
KR101562988B1 KR1020140054726A KR20140054726A KR101562988B1 KR 101562988 B1 KR101562988 B1 KR 101562988B1 KR 1020140054726 A KR1020140054726 A KR 1020140054726A KR 20140054726 A KR20140054726 A KR 20140054726A KR 101562988 B1 KR101562988 B1 KR 101562988B1
Authority
KR
South Korea
Prior art keywords
image
pixel
weight
detecting
hot material
Prior art date
Application number
KR1020140054726A
Other languages
Korean (ko)
Inventor
윤종필
박창현
Original Assignee
주식회사 포스코
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 주식회사 포스코 filed Critical 주식회사 포스코
Priority to KR1020140054726A priority Critical patent/KR101562988B1/en
Application granted granted Critical
Publication of KR101562988B1 publication Critical patent/KR101562988B1/en

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical techniques
    • G01B11/30Measuring arrangements characterised by the use of optical techniques for measuring roughness or irregularity of surfaces
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • G01N21/88Investigating the presence of flaws or contamination
    • G01N21/8851Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • G01N21/88Investigating the presence of flaws or contamination
    • G01N21/8851Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques

Abstract

The present invention relates to an image forming apparatus, comprising: a photographing unit photographing a hot material to obtain an original image of the surface of the hot material; An artificial neural network unit for detecting a second weight through an artificial neural network learned based on the first weight and a characteristic parameter of the original image; And an inverse transform unit for inversely transforming the transformed image by applying the second weight to generate a filtered image. A binarization unit for binarizing the filtered image to extract a binarized image and a defect detector for detecting surface defects present on the surface of the hot material from the binarized image, It is possible to distinguish surface defects and scales according to the surface characteristics and to effectively detect surface defects.

Description

TECHNICAL FIELD [0001] The present invention relates to a surface defect detecting apparatus and a surface defect detecting method,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a surface defect detecting apparatus and a surface detecting method for a hot material, and more particularly, to a surface defect detecting apparatus and a surface detecting method for hot material for distinguishing surface defects from scales.

The hot rolling process is a process of manufacturing a strip by heating a material such as a slab, a bloom, or a billet produced in a continuous casting process and then rolling it.

Hot materials such as slabs, blooms, or billets can entail surface defects due to a variety of factors. Especially. On the surface of the billet, corners, bushing defects, surface scabs, scratches, hole defects, etc. may occur. Such surface defects of the hot workpiece may lead to defects such as cracking of the steel sheet in subsequent processes such as rolling. In order to supply a good surface quality material to the post-process, it is necessary to perform surface quality inspection and defect removal work on the hot material.

However, since there are many scales such as scales on the surface of the hot hot material in the hot state, it is difficult to distinguish between the scale and the surface defects occurring on the surface of the actual hot material. Further, since the surface quality inspection depends on the naked eye, the surface defect detection performance is low, and in particular, the error detection rate due to the scale is high, so that the reliability of the defect detection is poor.

Accordingly, Korean Patent No. 10-1272005 (published on May 31, 2013) discloses a surface defect detection apparatus and a surface detection method for distinguishing between actual defects and scales.

1 is a schematic view showing a surface defect detection apparatus for hot work.

Referring to FIG. 1, the surface defect detection apparatus of the present document may include a lighting apparatus 10, a camera 20, an image processing apparatus 30, and a defect detection apparatus 40.

The illuminating device 10 can irradiate the surface of the hot workpiece 1 with light. The camera 20 can take an image of the surface of the hot workpiece 1 through the light irradiated to the surface of the hot workpiece 1. [ At this time, the illumination device 10 may be a light emitting diode (LED), and in particular, a blue LED may be used. The camera 20 can take an image of the surface of the hot workpiece 1 to generate an original image. The original image captured by the camera 20 can be transmitted to the image processing apparatus 30. [

The image processing apparatus 30 can generate a binarized image by binarizing the original image. A black and white image of surface defects can be obtained through the binarized image.

But. Such a surface defect detection apparatus has a problem in that its ability to distinguish between surface defects and scale is poor. This is because surface characteristics vary according to the type of steel, and surface defects are uniformly detected using fixed parameter values (weight, etc.) in the filtering process of the original image.

Korean Registered Patent No. 10-1272005 (Announced 2013.06.05)

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above-mentioned problems, and an object of the present invention is to provide a hot material capable of detecting surface defects by effectively distinguishing surface defects and scales corresponding to surface characteristics of various steel types, An object of the present invention is to provide a surface defect detecting apparatus and a surface detecting method.

The problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems not mentioned here can be understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a method of manufacturing a hot material, including: a photographing unit photographing a hot material to obtain an original image of a surface of the hot material; a converting unit converting the original image by applying a first weight according to a frequency; An artificial neural network unit for detecting a second weight through an artificial neural network learned based on a first weight and a characteristic parameter of the original image; and an inverse transform unit for inversely transforming the transformed image by applying the second weight, And a defect detector for detecting a surface defect existing on the surface of the hot work from the binarized image, wherein the binary image is a binary image of the binary image, A surface defect detecting apparatus of the present invention can be provided.

Preferably, the characteristic parameters of the original image may be Skewness, Mean, Standard Deviation, Smoothness, Uniformity, and Entropy.

Advantageously, said first weight may be a value determined through an optimization algorithm.

Preferably, the binarizing unit may generate an energy image by squaring the pixel value of the filtered image, smoothing the energy image, comparing the gray level of the smoothed energy image with a threshold value, If the gray level of the pixel is smaller than the threshold value, the pixel is associated with 1, and if the gray level of the pixel is larger than the threshold value, the binarized image can be extracted by associating the pixel with 0.

According to another aspect of the present invention, there is provided a method of manufacturing a hot material, comprising the steps of: a) capturing a hot material to obtain an original image of a surface of the hot material; b) transforming the original image by applying a first weight, Detecting a second weight through an artificial neural network learned based on the first weight and a characteristic parameter of the original image and generating a filtered image by inversely transforming the transformed image by applying the second weight, ) Extracting a binary image by binarizing the filtered image, and d) detecting a surface defect existing on the surface of the hot workpiece from the binarized image. .

Preferably, the characteristic parameters of the original image may be Skewness, Mean, Standard Deviation, Smoothness, Uniformity, and Entropy.

Advantageously, said first weight can be determined through an optimization algorithm.

Preferably, the step d) includes the steps of generating an energy image by providing pixel values of the filtered image, smoothing the energy image, and converting the gray level of the pixels of the smoothed energy image Comparing the pixel with the threshold value and associating the pixel with 1 if the gray level of the pixel is smaller than the threshold value and extracting the binarized image with the pixel corresponding to 0 if the gray level of the pixel is larger than the threshold value can do.

According to an embodiment of the present invention, a filtered image is acquired by applying a new weight through an artificial neural network learned by a characteristic parameter of an original image in a process of filtering an original image of a hot material, It is possible to distinguish between surface defects and scales according to the present invention, thereby effectively detecting surface defects.

1 is a schematic view showing a surface defect detection apparatus for hot work.
2 is a view showing a surface defect detection apparatus according to a preferred embodiment of the present invention,
FIG. 3 is a diagram showing the artificial neural network shown in FIG. 2,
4 is a view for explaining a filtering process of an original image,
FIG. 5 is a diagram illustrating a source image, a filtered image, and a binary image of a billet,
6 is a diagram illustrating a surface defect detection method according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating a method S300 for extracting the binarized image shown in FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS The objectives, specific advantages, and novel features of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG. The terms and words used in the present specification and claims should not be construed to be limited to ordinary or dictionary terms and the inventor should properly define the concept of the term in order to describe its own invention in the best way. The present invention should be construed in accordance with the meaning and concept consistent with the technical idea of the present invention. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

2 is a view showing a surface defect detecting apparatus according to a preferred embodiment of the present invention.

2, a surface defect detection apparatus 100 according to an exemplary embodiment of the present invention includes a photographing unit 110, a filtering unit 120, A binarization unit 130, and a defect detection unit 140.

First, the photographing unit 110 can photograph the surface of the hot workpiece 1 to acquire an original image. The photographing unit 110 can photograph the hot work 1 in real time and can be installed in an appropriate number so that the entire width of the hot work 1 can be photographed in consideration of the imaging angle of the camera. Here, the hot material may be a billet.

The filtering unit 120 may convert the original image acquired by the photographing unit 110 to generate a filtered image enhanced with the defect signal.

The filtering unit 120 separates the original image according to the frequency through a Wavelet transform. In general, the defective portion has a high frequency component, and a change in the surface brightness depending on the illumination and the steel type, which is not a defect, has a low frequency component. Therefore, after the original image is separated by frequency by the wavelet transform, the defective high frequency region is emphasized by increasing the weight relatively, and the filtered image is generated by inversely transforming the region by relatively reducing the weight, So that it is possible to more effectively detect surface defects.

The filtering unit 120 may include a transform unit 121, an artificial neural network unit 122, and an inverse transform unit 123.

The transforming unit 121 transforms the original image by applying a first weight according to the frequency. At this time, the first weight may be determined through a trial and error. Or Particle Swarm Optimization (PSO), Genetic Algorithm (GA), etc.), and the like.

However, because the surface image of the billet varies, the optimized weight may vary depending on the sample. Accordingly, the second weight, which is a new weight, can be detected and applied through the artificial neural network unit 122.

The artificial neural network unit 122 can detect the second weight through an artificial neural network (ANN) based on the optimized first weight and the characteristic parameters of the original image.

FIG. 3 is a diagram showing an artificial neural network shown in FIG. 2. FIG.

As shown in FIG. 3, the artificial light network includes a first weight W1 applied to wavelet transform, a characteristic parameter of the original image, Skewness, Mean, Standard Deviation, (Smoothness), Uniformity, and entropy.

The artificial neural network unit 122 detects the second weight W2 through the learned artificial neural network based on the received first weight.

The inverse transform unit 123 may generate a filtered image through inverse transform using the second weight W2

4 is a view for explaining a filtering process of an original image.

Referring to FIG. 4, the original image (FIG. 4 (a)) can be separated by frequency as shown in FIG. 4 (b) through wavelet transformation. The three regions 52, 53, and 54 in FIG. 4B are high-frequency regions, and one region 51 is a low-frequency region. When this one area 51 is again subjected to wavelet transformation, it can be separated into three high frequency areas 51b, 51c and 51d and one low frequency area 51a as shown in Fig. 4 (c). When the low frequency region is set to 0, the low frequency region is removed from the original image and the defective high frequency region can be emphasized.

At this time, when the second weight W2 is applied in the inverse conversion process, the surface defect detection performance can be remarkably improved in accordance with various surface characteristics.

The binarization unit 130 may extract the binarized image by binarizing the filtered image in the filtering unit 120.

Binarization of an image is represented by two values of 0 and 1, which are color values that are distributed to pixels in various ways. That is, only one candidate group that may correspond to a surface defect in the original image can be expressed as 1, and the remaining part can be expressed as 0. The part represented by 1 is represented by white, and the part represented by 0 is represented by black. The binarized image can be represented as a black and white image.

5 is a photograph showing a source image, a filtered image, and a binarized image for a billet.

The binarization unit 130 squares the pixel values of the filtered image (FIG. 5B) of the original image (FIG. 5A) to generate an energy image (FIG. Then, the binarization unit 130 smoothens the energy image, and then compares the gray level of the pixel of the smoothed energy image (FIG. 5 (d)) with the threshold value to determine whether the gray level of the pixel is greater than the threshold value If the pixel is small, the pixel corresponds to 1, and if the gray level of the pixel is larger than the threshold value, the pixel is associated with 0 and the binary image (eh 5) is extracted.

Here, the gray level is a numerical value indicating the degree of lightness and darkness of a pixel and can be generally expressed by 8 bits. That is, the brightness can be represented by 0 to 255 according to the degree of brightness of black and white. Here, when black and black are set to 0 and 255, respectively, the larger the gray value is, the brighter the smaller the gray value is. Therefore, the brightness and darkness of the pixel can be determined based on the threshold value.

The defect detection unit 140 can detect surface defects from the binarized image extracted by the binarization unit 130. Specifically, the defect detector 140 can detect surface defects in consideration of the size of the white regions, the lengths and angles of the major and minor axes, and the like.

FIG. 6 illustrates a method of detecting a surface defect according to an embodiment of the present invention. FIG. 7 illustrates a method S300 of extracting a binarized image shown in FIG.

Referring to FIGS. 6 and 7, in the surface defect detection method according to the preferred embodiment of the present invention, the photographing unit (110 in FIG. 2) captures the hot material to obtain an original image of the surface of the hot material (S100)

Next, the filtering unit (120 in FIG. 2) transforms the original image by applying a first weight according to a frequency, and outputs a second weight through an artificial neural network learned based on the first weight and the characteristic parameters of the original image And a second weight is applied to generate a filtered image by inversely transforming the transformed image in step S200.

In this case, the characteristic parameters of the original image may be as skewness, mean, standard deviation, smoothness, uniformity, and entropy.

Next, the binarization unit 130 binarizes the filtered image to extract a binarized image (S300). The binarization unit 130 may generate an energy image by providing a pixel value of the filtered image (S310) Thereafter, the energy image may be smoothed (S320). If the gray level of the pixel of the smoothed energy image is compared with the threshold value, if the gray level of the pixel is smaller than the threshold value, the pixel is matched to 1 , And if the gray level of the pixel is larger than the threshold value, the binarized image can be extracted by associating the pixel with 0 (S330)

Next, the defect detecting unit 140 detects surface defects present on the surface of the hot work from the binarized image (S400)

As used in this embodiment, the term " portion " refers to a hardware component such as software or an FPGA (field-programmable gate array) or ASIC, and 'part' performs certain roles. However, 'part' is not meant to be limited to software or hardware. &Quot; to " may be configured to reside on an addressable recording medium and may be configured to play back one or more processors. Thus, by way of example, 'parts' may refer to components such as software components, object-oriented software components, class components and task components, and processes, functions, , Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functions provided in the components and components may be further combined with a smaller number of components and components or further components and components. In addition, the components and components may be implemented to play back one or more CPUs in a device or a secure multimedia card.

Hereinafter, an apparatus and method for detecting surface defects of a hot material according to one preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: Surface defect detection device
110:
120: Filtering unit
121:
122: artificial neural network
123: Inverse transform unit
130: binarization unit
140:

Claims (8)

A photographing unit photographing a hot material to obtain an original image of the surface of the hot material;
An artificial neural network unit for detecting a second weight through an artificial neural network learned based on the first weight and a characteristic parameter of the original image; And an inverse transform unit for inversely transforming the transformed image by applying the second weight to generate a filtered image.
A binarization unit for binarizing the filtered image to extract a binarized image;
A defect detector for detecting a surface defect existing on the surface of the hot workpiece from the binarized image,
And a surface defect detecting device for detecting the surface defect of the hot material.
The method of claim 1,
Wherein the characteristic parameter of the original image is a skewness, a mean, a standard deviation, a smoothness, a uniformity, and an entropy.
The method according to claim 1,
Wherein the first weight is a value determined through an optimization algorithm.
The method according to claim 1,
The binarization unit may generate an energy image by squaring the pixel value of the filtered image, smoothing the energy image, and comparing the gray level of the smoothed energy image with a threshold value, If the gray level of the pixel is greater than the threshold value, the pixel is associated with 0. If the gray level of the pixel is greater than the threshold value, the pixel is associated with 0 to extract the binarized image.
a) capturing a hot material to obtain an original image of the surface of the hot material;
b) transforming the original image by applying a first weight according to a frequency, detecting a second weight through an artificial neural network learned based on the first weight and a characteristic parameter of the original image, Transforming the transformed image to generate a filtered image;
c) binarizing the filtered image to extract a binarized image;
d) detecting a surface defect existing on the surface of the hot work from the binarized image
And detecting a surface defect of the hot material.
6. The method of claim 5,
Wherein the characteristic parameter of the original image is a skewness, a mean, a standard deviation, a smoothness, a uniformity, and an entropy.
The method of claim 5,
Wherein the first weight is a value determined through an optimization algorithm.
6. The method of claim 5,
The step d)
Generating an energy image by providing pixel values of the filtered image;
Smoothing the energy image;
Comparing the gray level of the pixel of the smoothed energy image with a threshold value and associating the pixel with 1 if the gray level of the pixel is less than a threshold value and setting the pixel to 0 if the gray level of the pixel is greater than a threshold value; And extracting the binarized image
And detecting a surface defect of the hot material.
KR1020140054726A 2014-05-08 2014-05-08 Apparatus and method for detecting surface defects of hot billet KR101562988B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020140054726A KR101562988B1 (en) 2014-05-08 2014-05-08 Apparatus and method for detecting surface defects of hot billet

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020140054726A KR101562988B1 (en) 2014-05-08 2014-05-08 Apparatus and method for detecting surface defects of hot billet

Publications (1)

Publication Number Publication Date
KR101562988B1 true KR101562988B1 (en) 2015-10-23

Family

ID=54427486

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020140054726A KR101562988B1 (en) 2014-05-08 2014-05-08 Apparatus and method for detecting surface defects of hot billet

Country Status (1)

Country Link
KR (1) KR101562988B1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101863196B1 (en) * 2017-07-24 2018-06-01 한국생산기술연구원 An Apparatus and A Method For Detecting A Defect On Surfaces Based On Deep Learning
CN109781733A (en) * 2017-11-15 2019-05-21 欧姆龙株式会社 Flaw detection apparatus, defect detecting method and computer readable storage medium
KR102008973B1 (en) * 2019-01-25 2019-08-08 (주)나스텍이앤씨 Apparatus and Method for Detection defect of sewer pipe based on Deep Learning

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012032369A (en) 2010-07-29 2012-02-16 Sharp Corp Defect identification method, defect identification apparatus, program, and recording medium

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012032369A (en) 2010-07-29 2012-02-16 Sharp Corp Defect identification method, defect identification apparatus, program, and recording medium

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101863196B1 (en) * 2017-07-24 2018-06-01 한국생산기술연구원 An Apparatus and A Method For Detecting A Defect On Surfaces Based On Deep Learning
CN109781733A (en) * 2017-11-15 2019-05-21 欧姆龙株式会社 Flaw detection apparatus, defect detecting method and computer readable storage medium
CN109781733B (en) * 2017-11-15 2022-02-22 欧姆龙株式会社 Defect inspection apparatus, defect inspection method, and computer-readable storage medium
KR102008973B1 (en) * 2019-01-25 2019-08-08 (주)나스텍이앤씨 Apparatus and Method for Detection defect of sewer pipe based on Deep Learning

Similar Documents

Publication Publication Date Title
EP1400922B1 (en) Print inspection method
JP5086970B2 (en) Wood appearance inspection device, wood appearance inspection method
JP6052590B2 (en) Surface inspection apparatus and surface inspection method for automobile body
KR101224164B1 (en) Pre- processing method and apparatus for license plate recognition
CN101464418A (en) Flaw detection method and apparatus
JP2018096908A (en) Inspection device and inspection method
KR101562988B1 (en) Apparatus and method for detecting surface defects of hot billet
CN113869300A (en) Workpiece surface defect and character recognition method and system based on multi-vision fusion
CN111122590A (en) Ceramic surface defect detection device and detection method
JP2014020926A (en) Surface defect inspection device and surface defect inspection method
KR101272005B1 (en) Apparatus and method for detecting surface defects of hot billet
JP2019168388A (en) Image inspection method and image inspection device
JP2009139133A (en) Flaw detection method and flaw detector
CN107980094B (en) Visual detection system and method
JP2005165387A (en) Method and device for detecting stripe defective of picture and display device
US10997743B2 (en) Attachable matter detection apparatus
KR101543896B1 (en) Apparatus and method of detecting scap defect
KR101675532B1 (en) Apparatus and Method for detecting defect of thick steel plate
JP5264956B2 (en) Two-dimensional code reading apparatus and method
JP2009080004A (en) Inspection device
JP4863050B2 (en) Color unevenness defect inspection method and apparatus
KR101538696B1 (en) Apparatus for detecting surface crack of hot slab
JP4364773B2 (en) Inspection method of printed matter
US9311710B1 (en) Method for measuring width of gap of display module
TWI595224B (en) A method for inspecting color defects on contact lenses and a system thereof

Legal Events

Date Code Title Description
E701 Decision to grant or registration of patent right
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20181001

Year of fee payment: 4

FPAY Annual fee payment

Payment date: 20191002

Year of fee payment: 5