KR20210050168A - Method For Applying Learning Data Augmentaion To Deep Learning Model, Apparatus And Method For Classifying Images Using Deep Learning - Google Patents

Abstract

The present invention relates to an image classification apparatus and a method thereof, which use a Gabor filter to suggest a learning data extension method for extending limited learning data even if learning data for deep learning-based image processing are insufficient, and can be applied to an image classification process which is executed in machine vision of the manufacturing industry. The image classification apparatus comprises a memory and a processor. The present invention can secure sufficient learning data by easily and conveniently extending learning data by using a filter property change technique for changing filter parameters of a Gabor filter. The present invention can also improve image classification success rates by improving the reliability of a deep learning model by learning the deep learning model based on learning data sufficiently secured by the filter property change technique of the Gabor filter.

Description

Method For Applying Learning Data Augmentaion To Deep Learning Model, Apparatus And Method For Classifying Images Using Deep Learning}

The present invention proposes a learning data expansion method capable of securing sufficient learning data by expanding limited learning data using a Gabor filter even if training data for deep learning-based image processing is insufficient. It relates to an image classification apparatus and a method applicable to the image classification process executed in the machine vision of the manufacturing industry.

The development of artificial intelligence machine learning is expected to have a very large and widespread ripple effect in that it opens the possibility for automation of intellectual activities. Machine learning technology, which has recently developed rapidly around deep learning, greatly narrows the gap between the level of demand for practical use and actual artificial intelligence technology, predicting the emergence of various intelligent systems.

Deep learning is a technology that learns high-level information from data and is mainly based on deep neural networks. The core methodologies of deep learning include pre-training algorithms, convolutional networks (CNNs), and cyclic neural networks (RNNs).

Deep learning is applied to various fields such as computer vision, speech recognition, autonomous vehicles, robotics, natural language processing, and so on, showing excellent performance overcoming existing methods, and outstanding in computer vision and pattern recognition. have.

In [Patent Literature 1] to [Patent Literature 4], in order to increase the performance of an application using deep learning, various techniques for arbitrarily processing and increasing limited learning data have been developed.

However, the above patent documents do not propose a technique for extending learning data optimized for machine vision.

Machine vision is used to inspect final products in various manufacturing industries.For example, in the field of inspecting surface defects of products using automated facilities, it is often difficult to obtain sufficient training data to apply deep learning models. . In this case, since the performance of an application using deep learning cannot be guaranteed, a technique for securing training data optimized for the machine vision field and a technique for classifying images based on the training data need to be developed.

[Patent Document 1] Korean Patent Registration No. 10-1843066 (registered in the title of the invention "Method and device for classifying data using data expansion in machine learning", 2018.03.22) [Patent Document 2] Korean Patent Registration No. 10-1828011 (the title of the invention, "Method and Classification Device for Classifying the State of Objects Included in Images", registered on February 13, 2018) [Patent Document 3] Korean Patent Laid-Open Patent No. 10-2018-0130925 (the title of the invention "Artificial intelligence device for automatically generating learning images for machine learning and its control method", published on Dec. 10, 2018) [Patent Document 4] Korean Patent Laid-Open Patent No. 10-2019-0021095 (the title of the invention "Image Classification Device and Method, and Image Learning Device for Image Classification", published on Mar. 5, 2019)

none

An object of the present invention is to provide a learning data extension method for applying to a deep learning model that generates a plurality of extended images changed to have similarity to a single image of the original by using a filter characteristic change technique that changes the filter parameters of the Gabor filter. have.

Another object of the present invention is to secure sufficient training data by applying a filter characteristic change technique to all single images belonging to a prepared training dataset, and to apply to a deep learning model by performing a training process with the secured training data. It is to provide an image classification apparatus and method for generating weights and classifying images using a deep learning model to which the weights are applied.

A training data extension method for applying to a deep learning model according to the present invention for achieving the above object includes: a preparation step of preparing a training dataset in advance; An extension step of generating an extended image using a filter characteristic change technique for changing a filter parameter of a Gabor filter for a single image of the training dataset; And a learning step of training a deep learning model based on the expanded image generated in the expansion step.

In addition, in the expansion step, the filter characteristic changing method is characterized in that a filter parameter corresponding to at least one of a shape, a direction, a center frequency, and a bandwidth of a kernel is changed.

In addition, in the expansion step, learning data is expanded according to one of a first expansion mode in which training data is expanded using a normal Gabor filter and a second expansion mode in which training data is expanded using a modified Gabor filter. It is characterized by that.

In addition, the normal Gabor filter applied to the first extended mode is characterized by defined by Equations 1 and 2 below.

[Equation 1]

[Equation 2]

Where x and y are the coordinate values in the rectangular coordinate system of the two-dimensional image, λ is the wavelength controlling the sine function of the Gabor filter kernel, θ is the directionality of the kernel function, ψ is the phase difference, σ is the standard deviation of the Gaussian function, and γ is the Gaussian function. It is a factor that determines the shape of the filter.

In addition, the modified Gabor filter applied to the second extended mode is defined by Equation 3 below.

[Equation 3]

Where x and y are the coordinate values in the rectangular coordinate system of the two-dimensional image, θ is the direction of the kernel function, θ _x is the rotation angle in the x-axis direction _{of the kernel function, θ y} is the rotation angle in the y-axis direction of the kernel function, and σ is the Gaussian function. The standard deviation of, Sa and Sb are scale factors that determine the amplitude of the kernel frequency, where Sa is a constant from 1 to 50, Sb is a constant from 6 to 300, C is a constant from 0 to 2, and D is the distance away from the center. It is a constant from 0 to 50.

An image classification apparatus using deep learning according to the present invention for achieving the above object includes: a memory having a training data set storage unit for storing a training data set and a weight storage unit for storing weight information; Data for receiving a single image of the training dataset, generating an extended image by changing a filter parameter of a Gabor filter, and storing weight information derived from the result of training a deep learning model based on the generated extended image in the weight storage unit A processor having an image classification module for performing image classification processing by a deep learning-based diagnostic program on the input image acquired by the learning module and the image acquisition device and provided through the user interface using the weight information stored in the weight information; It characterized in that it comprises a.

In addition, the data learning module specifies a kernel by setting a design value of a filter parameter of a gabor filter, and generates an extended image by a filtering operation of the specified kernel, and a training dataset including the extended image. It characterized in that it comprises a learning unit for training a deep learning model.

In addition, the learning data extension unit is a normal kernel setter and a normal kernel generator for extending learning data according to a first expansion mode using a normal Gabor filter, and a modification for expanding learning data according to a second expansion mode using a modified Gabor filter. It characterized in that it includes a kernel configurator and a modified kernel generator.

In addition, the normal kernel configurator sets at least one of a kernel size, a directionality θ of a kernel function, and a standard deviation σ of a Gaussian function as a filter parameter design value of a normal gabor filter, and the normal kernel generator A normal kernel is generated according to a filter parameter design value of a normal gabor filter, and a plurality of extended images are generated by filtering a single image received from the generated normal kernel.

In addition, the modified kernel setting group transformation Gabor filters filter parameter design value as a kernel size, kernel functions direction θ, the rotational angle standard deviation σ, y-axis direction of the kernel x axis of the function the rotation angle θ _x, the kernel functions of the Gaussian function of the θ _y , scale factors Sa and Sb for determining the amplitude of the kernel frequency, and at least one of constants C and D are set, and the modified kernel generator is based on a design value of a filter parameter of a modified Gabor filter set by the modified kernel setter. A modified kernel is generated, and a plurality of extended images are generated by filtering a single image input from the generated modified kernel.

In addition, the image classification module is an image supply unit that supplies an image to be classified obtained by an image acquisition device and provided through a user interface, and a deep learning model in which the weight stored in the weight storage unit is reflected with respect to the image supplied from the image supply unit. And a deep learning execution unit for classifying an image using a diagnostic program based on and an image output unit for outputting the images classified by the deep learning execution unit and classification information.

In addition, it is characterized in that the deep learning execution unit uses any one of LeNet-5, Alex-Net, and Google-Net as a deep learning model.

An image classification method using deep learning according to the present invention for achieving the above object comprises: a preparation step of preparing a training dataset prepared in advance; An expansion step of expanding the learning data according to a first expansion mode using a normal Gabor filter or a second expansion mode using a modified Gabor filter in order to expand limited training data belonging to the prepared training data set; A learning step of performing a learning process in a deep learning model for the extended training data; A storage step of storing weight information derived from a learning result of the learning process; A deep learning execution step of classifying an image using a deep learning-based diagnostic program that reflects weight information obtained in a learning process with respect to the image to be classified that is acquired by the image acquisition device and input through a user interface; And an output step of outputting the image and classification information classified by the deep learning execution.

In addition, the normal Gabor filter applied to the first extended mode is characterized by defined by Equations 1 and 2 below.

[Equation 1]

[Equation 2]

Where x and y are the coordinate values in the rectangular coordinate system of the two-dimensional image, λ is the wavelength controlling the sine function of the Gabor filter kernel, θ is the directionality of the kernel function, ψ is the phase difference, σ is the standard deviation of the Gaussian function, and γ is the Gaussian function. It is a factor that determines the shape of the filter.

In addition, the modified Gabor filter applied to the second extended mode is defined by Equation 3 below.

[Equation 3]

Where x and y are the coordinate values in the rectangular coordinate system of the two-dimensional image, θ is the direction of the kernel function, θ _x is the rotation angle in the x-axis direction _{of the kernel function, θ y} is the rotation angle in the y-axis direction of the kernel function, and σ is the Gaussian function. The standard deviation of, Sa and Sb are scale factors that determine the amplitude of the kernel frequency, where Sa is a constant from 1 to 50, Sb is a constant from 6 to 300, C is a constant from 0 to 2, and D is the distance away from the center. It is a constant from 0 to 50.

In addition, it is characterized in that any one of LeNet-5, Alex-Net, and Google-Net is used as a deep learning model in the learning step.

According to the present invention as described above, it is possible to secure sufficient learning data by easily and simply expanding the learning data by using a filter characteristic changing technique for changing the filter parameter of the Gabor filter.

In addition, the present invention can improve the success rate of image classification by improving the reliability of the deep learning model by learning a deep learning model based on training data sufficiently secured by a filter characteristic change method of a Gabor filter.

In addition, the present invention can be applied to a diagnostic program of an inspection system that inspects product defects in the manufacturing industry, and can improve the accuracy of defect inspection by image classification processing executed in a diagnostic program based on a deep learning model.

1 is a flowchart showing an image classification method using deep learning according to an embodiment of the present invention;
2 is a block diagram of an image classification apparatus using deep learning according to an embodiment of the present invention.
3 is a detailed block diagram of a main part of an image classification apparatus using deep learning according to an embodiment of the present invention;
4 is a detailed block diagram of the learning data expansion unit of FIG. 3;
FIG. 5 is a diagram for explaining an operation of expanding an image of a number '0' in a first expansion mode by using a normal Gabor filter by the learning data expansion unit of FIG. 3;
FIG. 6 is a diagram illustrating a principle of generating a plurality of extended images from a single image in a first extended mode using a normal Gabor filter by the training data extension of FIG. 3; FIG.
FIG. 7 is a diagram illustrating a principle of generating a plurality of extended images from a single image in a second extended mode using a modified Gabor filter by the learning data extension part of FIG. 3; FIG.
8 is a diagram illustrating a plurality of extended images obtained when a first extended mode and a second extended mode are applied to the same input image according to an embodiment of the present invention.

Hereinafter, the present invention will be described by describing embodiments of the present invention with reference to the accompanying drawings.

The same reference numerals shown in each drawing indicate the same members. In addition, in describing the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted. In addition, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

Machine Vision acquires an image of a product through hardware such as cameras, optical systems, and lighting installed on a production line, and performs image processing through a diagnostic program that analyzes and inspects the acquired image.

The present invention is characterized in that the design value of a filter parameter of a Gabor filter is varied to create and expand training data for application to a deep learning model.

In addition, the present invention can be applied to a diagnostic program of an inspection system that inspects product defects in the manufacturing industry, and by providing sufficient training data to increase the reliability of the deep learning model, the diagnostic program based on the deep learning model can be applied. It is possible to improve the accuracy of defect inspection by image classification processing.

The Gabor filter has filter parameters related to a tunable orientation and center frequency, and a radial frequency bandwidth in the spatial and frequency domains. Depending on how the filter parameters are set, the shape, orientation, center frequency, and bandwidth of the kernel are changed, and the characteristics of the kernel function for the Gabor filter are determined. Similar images can be obtained by passing through the filtering operation performed in the function.

In the present invention, by varying the design value of the filter parameter, a number of new images with similarity to the input single image are created using different kernel functions set, and secured through data expansion using a Gabor filter in a pre-prepared training dataset. The new image is included as training data. In the present invention, images similar to the original image may be created by variously changing a kernel function representing the filter characteristics of a Gabor filter, and the newly created images may be included in the training data of the deep learning model to be reinforced.

The present invention performs deep learning based on the reinforced training dataset, calculates weights for application to image classification processing, and stores weight information obtained from the execution result. In this way, the weight information is updated when new training data is added.

Referring to FIG. 1, the image classification method using deep learning according to an embodiment of the present invention includes a preparation step 10 of preparing a training dataset prepared in advance, and normal to expand limited training data belonging to the prepared training dataset. Expansion step 20 of expanding training data according to a first expansion mode using a Gabor filter or a second expansion mode using a modified Gabor filter, a learning step of performing a learning process in a deep learning model targeting the expanded training data (30), a storage step of storing weight information derived as a result of the learning process (40), deep learning that classifies the image using a diagnostic program based on deep learning for the image to be classified using the weight obtained from the learning process An execution step 50, and an output step 60 of outputting images and classification information classified by deep learning execution.

2 and 3, the image classification apparatus using deep learning according to the present invention may be implemented as a computing device including a user interface 100, a processor 101, and a memory 104.

Through the user interface 100, the user inputs overall user commands related to image classification, such as whether to execute data learning, select an extension mode, select a deep learning model, and input a target image.

The memory 104 includes a deep learning solution 105 that stores a deep learning-based diagnostic program, a training dataset storage unit 110 that stores training data for training a deep learning model, and a weight obtained as a result of learning by the deep learning model. It includes a weight storage unit 140 for storing information.

The processor 101 receives training data prepared in advance from the training data set storage unit 110 and stores weight information obtained as a result of the training in the weight storage unit 140. The processor 101 includes a data learning module 102 and an image classification module 103.

The data learning module 102 includes a learning data expansion unit 120 for expanding learning data, and a learning unit 130 for learning based on the expanded learning data.

The image classification module 103 includes an image supply unit 150 for supplying an image to be classified obtained from an image acquisition device and provided through the user interface 100, and a weight storage unit for the image supplied from the image supply unit 150 ( 140) a deep learning execution unit 160 that classifies images using a diagnostic program based on a deep learning model reflecting the weights stored in), an image output unit that outputs images and classification information classified by the deep learning execution unit 160 Includes 170.

Referring to FIG. 4, the data learning module 120 includes a normal kernel setter 121, a normal kernel generator 122, a modified kernel setter 123, and a modified kernel generator 124.

The data learning module 120 receives an extended mode selection command from the user interface unit 100 and expands the learning data according to the first extended mode or the second extended mode.

The data learning module 120 can be broadly divided into learning data expansion by a first expansion mode using a normal Gabor filter and expansion of learning data by a second expansion mode using a modified Gabor filter. In the first extended mode, training data is extended using the normal kernel configurator 121 and the normal kernel generator 122.

The normal kernel configurator 121 changes filter parameters for specifying the kernel of the normal gabor filter.

The normal Gabor filter can be expressed as the following [Equation 1].

In [Equation 1], x'and y'can be expressed as the following [Equation 2].

Where x and y are the coordinate values in the rectangular coordinate system of the two-dimensional image, λ is the wavelength controlling the sine function of the Gabor filter kernel, θ is the directionality of the kernel function, ψ is the phase difference, σ is the standard deviation of the Gaussian function, and γ is the Gaussian function. It is a factor that determines the shape of the filter.

The normal kernel configurator 121 provides design values for filter parameters of at least one normal gabor filter to the normal kernel generator 122. The number of filter parameters is not limited. In the embodiment, design values for kernel size, σ, and θ are provided as filter parameters of the normal Gabor filter, but the number and targets of filter parameters are determined in consideration of the image characteristics of the classification target and the performance of the image acquisition device. You can change it.

The normal kernel generator 122 generates a normal kernel, which is a kernel of a normal heirloom filter, based on the design values of the supplied filter parameters. For example, the first normal kernel (GK1) is generated by the kernel size of 11X11, σ=2, θ=π/6, and the kernel size is 11 X 11 (pixel), σ= 2, the second normal kernel GK2 is generated by θ=π/3, and n normal kernels may be generated in this manner.

The normal kernel generator 122 receives a training dataset prepared in advance from the training dataset storage unit 110, and receives a single image of the original individually. The normal kernel generator 122 performs filtering operations on n normal kernels generated for a single image that is individually input, and performs a filtering operation on n extended images (extended image 1, extended image 2, ..., extended image n) similar to the original image. ). The generated n expanded images are provided to the learning unit 130.

In the second extended mode, training data is extended using the modified kernel setter 123 and the modified kernel generator 124.

The modified kernel configurator 123 changes filter parameters for specifying the kernel of the modified gabor filter.

The transformed Gabor filter can be transformed so that the input image is partially warped or a reduced image is repeatedly displayed around the center away from the kernel.

In an embodiment, the modified Gabor filter can be expressed as the following [Equation 3].

Where x and y are the coordinate values in the rectangular coordinate system of the two-dimensional image, θ is the direction of the kernel function, θ _x is the rotation angle in the x-axis direction _{of the kernel function, θ y} is the rotation angle in the y-axis direction of the kernel function, and σ is the Gaussian function. The standard deviation of, Sa and Sb are scale factors that determine the amplitude of the kernel frequency, where Sa is a constant from 1 to 50, Sb is a constant from 6 to 300, C is a constant from 0 to 2, and D is the distance away from the center. It is a constant from 0 to 50.

The modified kernel design value setter 123 provides design values for filter parameters of at least one modified Gabor filter to the normal kernel generator 122. The number of filter parameters is not limited. _{In the embodiment, design values for kernel size, σ, θ, θ x} , θ _y , Sa, Sb, C, D were provided as filter parameters of the modified Gabor filter, but image characteristics and image acquisition devices to be classified The number and target of filter parameters can be changed in consideration of the performance of the filter.

The modified kernel generator 124 generates a modified kernel, which is a kernel of the modified Gabor filter, based on the design values of the provided filter parameters. For example, given as σ=5, θ=π/4, θ _x =π/30, θ _y =0, C=1.4, D=15, the first transformation kernel (MK1) is generated, and σ =5, θ=π/4, θ _x =0, θ _y =π/30, C=1.4, D=15 The second transforming kernel (MK2) is generated by the same method, and n A transform kernel can be created.

The modified kernel generator 124 individually receives an original single image corresponding to a pre-prepared training dataset from the training dataset storage unit 110. The transformed kernel generator 124 performs filtering operations on n transformed kernels generated targeting a single image that is individually input to generate n extended images similar to the original image (extended image 1, extended image 2, ..., extended image n). ). The generated n expanded images are provided to the learning unit 130.

The learning unit 130 performs training on the deep learning model based on the extended training data using the normal Gabor filter in the first extended mode, or based on the extended training data using the modified Gabor filter in the second extended mode. As a result, deep learning models are trained. In this way, the weight information obtained by performing the learning process of the deep learning model by the learning unit 130 is stored in the weight storage unit 140 and applied to the deep learning execution unit 160.

In the embodiment, LeNet-5 is applied as a deep learning model used by the learning unit 130 and the deep learning execution unit 160, but the deep learning model is not specified, and as another deep learning model, Alex Net , Google Net, etc. can also be applied.

The training dataset provided by the training dataset storage unit 110 is a collection of information for learning deep learning.

In general, a dataset of the Modified National Institute of Standards and Technology database (MNIST), which is widely used in machine learning, can be adopted. The MNIST dataset contains 60,000 train images and 10,000 test images. Half of the training images and half of the test images were collected from the National Institute of Standards and Technology (NIST) train dataset, and half of the remaining training images and half of the test images were collected from NIST's test dataset. ).

The MNIST dataset is size-normalized and certered, and is a 28×28 gray image. [Table 1] below shows sample images and quantities of each number in the MNIST dataset.

Train dataset Test dataset (teat dateset) number example image Quantity example image Quantity 0

5,923 pieces

Chapter 980 One

6,742 pieces

1,135 pieces 2

5,958 pieces

1,032 pieces 3

6,131 pieces

1,010 pieces 4

5,842 pieces

Chapter 982 5

5,421 pieces

Chapter 892 6

5,918 pieces

Chapter 958 7

6,265 pieces

1,028 pieces 8

5,851 pieces

Chapter 974 9

5,949 pieces

1,009 pieces sum 60,000 sheets 10,000 sheets

Referring to FIG. 5, when the learning data expansion unit 120 expands the learning data in the first expansion mode, the first to fourth kernel functions (GK1, GK2, GK3, GK4) for the input image of the number '0' ) Can be used to create the first to fourth extended images A1, A2, A3, A4.

As another example, referring to FIG. 6, as a training data set, when the training data of numbers 0 to 9 has a total of 500 pieces of 50 pieces each, and the training data expansion unit 120 expands the training data in the first expansion mode It will be described by way of example. Here, by setting σ constant and varying the kernel size and θ, you can create 8 extended images for each number of training data. In other words, for a total of 50 training data for the number '0', 8 extended images are created each to create a total of 400 extended images, and 450 pieces of training images for the number '0' including 50 original images. Can be secured.

When the training data set storage unit 110 has limited training data and therefore absolutely lacks training data, an operation of securing sufficient extended data using the second extended mode will be described.

FIG. 7 is a diagram for explaining a principle of generating a plurality of expanded images from a single image in a second expansion mode using a modified Gabor filter by the learning data expansion unit of FIG. 3.

In FIG. 7, when the filter parameters applied to the kernel K1 of an arbitrary Gabor filter are σ=5 and θ=π/4, the rotation angle θ _x =π/30 in the x-axis direction of the kernel function and y of the kernel function When the axial rotation angle θ _y = 0 is set, a modified kernel (K21) is created, and similarly, the rotation angle in the x-axis direction of the kernel function θ _x = 0 and the rotation angle in the y-axis direction of the kernel function θ _y =π/30 When set to, a modified kernel (K22) is created. In addition, when the filter parameters applied to the kernel K1 of an arbitrary Gabor filter are σ=5 and θ=π/4, a modified kernel K3 is created when C=1.4 and D=15 are set. Here, D denotes the distance between the four regions (H) of the repeated shape apart from the center of the kernel.

A new kernel K41 may be generated by a combination of the modified kernel K21 and the kernel K3, and a new kernel K42 may be generated by a combination of the modified kernel K22 and the kernel K3.

8 is a diagram illustrating a plurality of extended images obtained when a first extended mode and a second extended mode are applied to the same input image according to an embodiment of the present invention.

For the input image (A0), in the first extended mode, the filter parameter of the normal Gabor filter is set constant as σ=2, and then θ=π/6, θ=π/3, θ=2π/3, respectively. Three modified expanded images (NA1, NA2, and N3) are generated.

In addition, in the second extended mode for the input image (A0), σ=2, θ _x =π/6, θ _y =0, C=0, D=0 as filter parameters of the modified Gabor filter are set constant and then θ In each case set to =π/6, θ=π/3, and θ=2π/3, three modified expanded images (MA1, MA2, M3) are generated. In the second extended mode, since the number of filter parameters of the modified Gabor filter is large, much more training data can be extended than in the first extended mode.

Hereinafter, a process of expanding learning data using a normal Gabor filter and a modified Gabor filter according to the present invention will be verified, and this will be described.

[Test Example 1]

-Training dataset used: MNIST dataset

-Number of training images: 500 each for numbers 0-9 in the MNIST dataset

-Number of test images: 500 each for numbers 0-9 in the MNIST dataset

-Deep Learning Model: LeNet-5

-Image classification success rate: 97%

input




Average image classification success rate 0 One 2 3 4 5 6 7 8 9

Print 0 482 0 2 0 0 One 5 One One 0 One 0 494 3 0 0 0 2 One 0 3 2 2 2 491 0 One 0 One 5 0 0 3 2 2 0 498 0 10 0 12 10 One 4 0 0 0 0 486 0 2 2 One 0 5 4 One 0 0 0 488 5 0 5 7 6 3 0 One 0 0 0 485 0 0 0 7 One One 3 One 0 0 0 476 7 6 8 2 0 0 One One One 0 One 474 0 9 4 0 0 0 12 0 0 2 2 483 Sum 500 500 500 500 500 500 500 500 500 500 image
Classification
Success rate (%)
96.4
98.8
98.2
99.6
97.2
97.6
97.0
95.2
94.8
96.6
97

[Table 2] shows a total of 5000 pieces of 500 training images for each of the numbers 0 to 9 in the MNIST dataset, and training in the deep learning model LeNet-5, and a total of 5000 pieces of 500 test images for each of the numbers 0 to 9 are categorized. The images were classified as. For example, as a result of image classification for 500 test images with the number '0', 482 images judged as '0', images judged as '1'/'4' are judged as 0 and '7'. 1 image, 2 images determined as '2'/'3'/'8', 3 images determined as '6', 3 images determined as '5'/'9' It appeared in four chapters. As a result of classifying images from 0 to 9, the average image classification success rate is 97%.

[Test Example 2]

-Training dataset used: MNIST dataset

-Number of training images: 50 each for numbers 0-9 in the MNIST dataset

-Number of test images: 500 each for numbers 0-9 in the MNIST dataset

-Deep Learning Model: LeNet-5

-Image classification success rate: 89%

input




Average image classification success rate 0 One 2 3 4 5 6 7 8 9

Print 0 478 0 2 0 2 One 10 0 5 One One 2 496 22 8 6 6 5 6 28 11 2 2 2 423 2 2 6 0 8 One One 3 0 0 14 421 One 5 One 3 12 One 4 0 0 10 0 448 3 12 0 10 10 5 10 0 3 21 0 464 9 One 32 7 6 4 0 6 2 6 5 462 0 10 0 7 3 2 20 32 3 7 0 477 25 34 8 0 0 0 4 0 3 One 0 355 0 9 One 0 0 10 32 0 0 5 22 435 Sum 500 500 500 500 500 500 500 500 500 500 image
Classification
Success rate (%)
95.6
99.2
84.6
84.2
89.6
92.8
92.4
95.4
71.0
87.0
89

[Table 3] shows a total of 500 pieces of 50 training images for each of the numbers 0 to 9 in the MNIST dataset, and training in the deep learning model LeNet-5, and a total of 5000 pieces of 500 test images for each of the numbers 0 to 9 are categorized. The images were classified as. For example, as a result of image classification for 500 test images with the number '0', 478 images judged as '0', 0 images were judged as '3'/'4'/'8', and ' 1 image is determined as '9', 2 images are determined as '1'/'2', 3 images are determined as '7', 4 images are determined as '6', and '5' The number of images judged as' appeared as 10. As a result of classifying images from 0 to 9, the average image classification success rate is 89%.

It can be seen that [Test Example 2] has a relatively small number of learning images compared to [Test Example 1], so that the success rate of image classification is significantly lowered.

[Test Example 3]

-Used training dataset: expanded training image using normal heirloom filter

-Number of training images: 450 each for numbers 0-9 from the expanded training image dataset

-Number of test images: 500 each for numbers 0-9 from the expanded training image dataset

-Deep Learning Model: LeNet-5

-Image classification success rate: 92%

input




Average image classification success rate 0 One 2 3 4 5 6 7 8 9

Print 0 481 0 One 0 0 4 3 One One One One 0 486 0 0 One 0 One 2 4 5 2 2 4 460 One 2 3 0 9 2 3 3 2 0 15 435 0 8 2 5 8 2 4 0 2 10 0 474 0 4 0 2 11 5 3 2 2 39 3 472 11 6 26 19 6 6 4 2 0 One 4 478 One 8 0 7 0 One 3 8 One 2 0 460 11 13 8 3 One 6 13 One 7 One 2 430 2 9 3 0 One 4 17 0 0 14 8 444 Sum 500 500 500 500 500 500 500 500 500 500 image
Classification
Success rate (%)
96.2
97.2
92.0
87.0
94.8
94.4
95.6
92.0
86.0
88.8
92

[Table 4] shows a total of 4500 training images, each 450 training images for numbers 0 to 9, selected from the training dataset expanded using the normal heirloom filter, and trained on the deep learning model LeNet-5, and test images from the numbers 0 to 9 The images were classified with a total of 5000 sheets of 500 sheets each. For example, as a result of image classification for 500 test images with the number '0', 481 images were determined as '0', 0 images were determined as '1'/'4'/'7', and ' 2 images were determined as 2'/'3', 3 images were determined as '5'/'8'/'9', and 6 images were determined as '6'. As a result of classifying images from 0 to 9, the average image classification success rate was 92%.

[Test Example 4]

-Used training dataset: expanded training image using transformed heirloom filter

-Number of training images: 450 each for numbers 0-9 from the expanded training image dataset

-Number of test images: 500 each for numbers 0-9 from the expanded training image dataset

-Deep Learning Model: LeNet-5

-Image classification success rate: 94%

input




Average image classification success rate 0 One 2 3 4 5 6 7 8 9

Print 0 484 0 2 0 One One 8 One 4 One One 0 498 2 One One 0 3 6 8 6 2 3 2 460 0 0 One 0 8 One One 3 One 0 18 464 0 7 2 One 6 2 4 One 0 One 0 453 0 One One 0 3 5 2 0 2 18 0 482 8 2 11 8 6 4 0 4 0 9 5 477 0 10 0 7 0 0 8 12 4 0 0 462 14 16 8 2 0 2 3 0 3 One 2 432 One 9 3 0 One 2 32 One 0 17 14 462 Sum 500 500 500 500 500 500 500 500 500 500 image
Classification
Success rate (%)
96.8
99.6
92.0
92.8
90.6
96.4
95.4
92.4
86.4
92.4
94

[Table 5] shows a total of 4500 training images, each 450 training images for numbers 0 to 9, selected from the training dataset expanded using the transformed Gabor filter, and trained in the deep learning model LeNet-5, and test images from numbers 0 to 9 The images were classified with a total of 5000 sheets of 500 sheets each as the target of classification. For example, as a result of image classification for 500 test images with the number '0', 484 images determined as '0', 0 images determined as '1'/'7', and '3'/' One image for each of 4', two for each of '5'/'8', three for each of '2'/'9', and one for each of '6' Appeared in 4 chapters. As a result of classifying images from 0 to 9, the average image classification success rate is 94%. [Test Example 4] was a case in which the learning data was expanded using a modified Gabor filter, and the image classification success rate was 2% higher under the same conditions as in [Test Example 3].

In a situation where it is difficult to secure enough training data in machine vision, and in a situation where deep learning can only be performed using limited training data in reality, securing sufficient training data becomes a decisive factor in increasing the success rate of image classification. As described above, the image classification success rate was 97% in [Test Example 1] and significantly dropped to 89% in [Test Example 2], which is the main factor due to lack of training data used for deep learning learning in [Test Example 2]. to be.

[Test Example 3] applying the learning data expansion method using a normal Gabor filter and [Test Example 4] using the learning data expansion method using a modified Gabor filter exhibited relatively high image classification success rates compared to [Test Example 2]. I got it.

The present invention uses a filter characteristic change method that changes the filter parameter of the Gabor filter to easily and conveniently expand the training data to secure sufficient training data, and learns a deep learning model based on the acquired training data to learn a deep learning model. By improving the reliability, the success rate of image classification can be improved.

From this point of view, securing sufficient learning data is of paramount importance in a situation in which machine vision has limited learning data, and the present invention can be usefully utilized as a method to satisfy this.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that other specific forms can be easily modified without changing the technical spirit or essential features of the present invention. will be.

100: user interface 101: processor
102: data learning module 103: image classification module
104: memory 105: deep learning solution
110: training data set storage unit 120: training data expansion unit
130: learning unit 140: weight storage unit
150: image supply unit 160: deep learning execution unit
170: image output unit

Claims (16)

A preparation step of preparing a training dataset in advance;
An extension step of generating an extended image using a filter characteristic change technique for changing a filter parameter of a Gabor filter for a single image of the training dataset; And
And a learning step of training a deep learning model based on the expanded image generated in the expansion step. The method of claim 1,
In the expansion step, the filter characteristic change method changes a filter parameter corresponding to at least one of a shape, a direction, a center frequency, and a bandwidth of a kernel. The method of claim 1,
In the expansion step, expanding the learning data according to any one of a first expansion mode for expanding training data using a normal Gabor filter and a second expansion mode for expanding training data using a modified Gabor filter. Learning data extension method for application to a deep learning model characterized by. The method of claim 3,
The normal Gabor filter applied to the first extended mode is defined by Equation 1 and Equation 2 below.
[Equation 1]

[Equation 2]

Where x and y are the coordinate values in the rectangular coordinate system of the two-dimensional image, λ is the wavelength controlling the sine function of the Gabor filter kernel, θ is the directionality of the kernel function, ψ is the phase difference, σ is the standard deviation of the Gaussian function, and γ is the Gaussian function. It is a factor that determines the shape of the filter. The method of claim 3,
The modified Gabor filter applied to the second extended mode is defined by Equation 3 below.
[Equation 3]

Where x and y are the coordinate values in the rectangular coordinate system of the two-dimensional image, θ is the direction of the kernel function, θ _x is the rotation angle in the x-axis direction _{of the kernel function, θ y} is the rotation angle in the y-axis direction of the kernel function, and σ is the Gaussian function. The standard deviation of, Sa and Sb are scale factors that determine the amplitude of the kernel frequency, where Sa is a constant from 1 to 50, Sb is a constant from 6 to 300, C is a constant from 0 to 2, and D is the distance away from the center. It is a constant from 0 to 50. A memory having a training data set storage unit for storing a training data set and a weight storage unit for storing weight information;
Data for receiving a single image of the training dataset, generating an extended image by changing a filter parameter of a Gabor filter, and storing weight information derived from the result of training a deep learning model based on the generated extended image in the weight storage unit A processor having an image classification module for performing image classification processing by a deep learning-based diagnostic program on the input image acquired by the learning module and the image acquisition device and provided through the user interface using the weight information stored in the weight information; Image classification apparatus using deep learning, characterized in that it comprises a. The method of claim 6,
The data learning module has a learning data expansion unit configured to specify a kernel by setting a design value of a filter parameter of a Gabor filter, and to generate an expanded image by a filtering operation of the specified kernel, and a training dataset including the expanded image. An image classification apparatus using deep learning, comprising a learning unit that trains a deep learning model. The method of claim 6,
The learning data extension unit is a normal kernel setter and a normal kernel generator for extending learning data according to a first expansion mode using a normal Gabor filter, and a modified kernel for expanding learning data according to a second expansion mode using a modified Gabor filter. An image classification apparatus using deep learning, comprising: a setter and a modified kernel generator. The method of claim 8,
The normal kernel configurator sets at least one of a kernel size, a directionality θ of a kernel function, and a standard deviation σ of a Gaussian function as a design value of a filter parameter of a normal Gabor filter,
The normal kernel generator generates a normal kernel according to a filter parameter design value of a normal Gabor filter set by the normal kernel configurator,
An image classification apparatus using deep learning, characterized in that for generating a plurality of extended images by filtering a single image received from the generated normal kernel. The method of claim 8,
The modified kernel setting group filter parameters of the modified Gabor filter design value as a kernel size, kernel functions direction θ, a Gaussian function by a standard deviation σ, rotating the x axis of the kernel functions each θ _x, the rotation angle θ y-axis direction of the kernel functions of the _y , scale factors Sa and Sb for determining the amplitude of the kernel frequency, and at least one of constants C and D are set,
The modified kernel generator generates a modified kernel according to a filter parameter design value of a modified Gabor filter set by the modified kernel setter,
An image classification apparatus using deep learning, characterized in that filtering a single image received from the generated modified kernel to generate a plurality of extended images. The method of claim 6,
The image classification module includes an image supply unit that supplies an image to be classified obtained by an image acquisition device and provided through a user interface, and a deep learning model in which a weight stored in the weight storage unit is reflected with respect to the image supplied from the image supply unit. An image classification apparatus using deep learning, comprising: a deep learning execution unit for classifying images using a diagnostic program based on the deep learning program, and an image output unit for outputting the images classified by the deep learning execution unit and classification information. The method of claim 11,
An image classification apparatus using deep learning, characterized in that the deep learning execution unit uses any one of LeNet-5, Alex-Net, and Google-Net as a deep learning model. A preparation step of preparing a training dataset prepared in advance;
An expansion step of expanding the learning data according to a first expansion mode using a normal Gabor filter or a second expansion mode using a modified Gabor filter in order to expand limited training data belonging to the prepared training data set;
A learning step of performing a learning process in a deep learning model for the extended training data;
A storage step of storing weight information derived as a result of learning in the learning process;
A deep learning execution step of classifying an image using a deep learning-based diagnostic program that reflects weight information obtained in a learning process with respect to the image to be classified, which is acquired by the image acquisition device and input through a user interface;
An image classification method using deep learning comprising; an output step of outputting the image classified by the deep learning execution and the classification information. The method of claim 13,
An image classification method using deep learning, characterized in that the normal Gabor filter applied to the first extended mode is defined by Equations 1 and 2 below.
[Equation 1]

[Equation 2]

Where x and y are the coordinate values in the rectangular coordinate system of the two-dimensional image, λ is the wavelength that adjusts the sine function of the Gabor filter kernel, θ is the directionality of the kernel function, ψ is the phase difference, σ is the standard deviation of the Gaussian function, and γ is It is a factor that determines the shape of the filter. The method of claim 13,
An image classification method using deep learning, characterized in that the modified Gabor filter applied to the second extended mode is defined by Equation 3 below.
[Equation 3]

Where x and y are the coordinate values in the rectangular coordinate system of the two-dimensional image, θ is the direction of the kernel function, θ _x is the rotation angle in the x-axis direction _{of the kernel function, θ y} is the rotation angle in the y-axis direction of the kernel function, and σ is the Gaussian function. The standard deviation of, Sa and Sb are scale factors that determine the amplitude of the kernel frequency, where Sa is a constant from 1 to 50, Sb is a constant from 6 to 300, C is a constant from 0 to 2, and D is the distance away from the center. It is a constant from 0 to 50. The method of claim 13,
Image classification method using deep learning, characterized in that any one of LeNet-5, Alex-Net, and Google-Net is used as a deep learning model in the learning step.

Images