KR20220108865A - Object classfication and counting method using Genetic Algorithm and CNN - Google Patents

Abstract

The present invention relates to an object classification and aggregation method using genetic algorithm and CNN, and more specifically, to an object detection method using genetic algorithm and CNN, wherein in a general object detection process, an object aggregation and classification process, which are concurrently performed, are separated into two steps, and the genetic algorithm is used in the process of aggregating objects, and a CNN is used in the process of classifying the objects, so that the amount of computation and optimization time can be greatly reduced, and the accuracy of object classification can be improved.

Description

Object classfication and counting method using Genetic Algorithm and CNN}

The present invention relates to an object classification and aggregation method using a genetic algorithm and CNN, and more particularly, the aggregation and classification process simultaneously performed in a general object detection process is performed by dividing the object aggregation and classification process into two steps, Object detection using a genetic algorithm and CNN that uses a genetic algorithm in the aggregation process of it's about how

In general, object detection refers to computer vision technology for accurately finding the type and location of a meaningful object such as a person, animal, or vehicle in a photographed image or video. In general, in order to recognize an object, a method of finding a candidate area for a detection target and predicting the type and location of the object in the candidate area through a learned model is mainly used.

For this process, information on the image or image and the object class and the object position (bounding box) within the image or image is required.

Object detection as described above is applied in numerous computer vision fields including image recovery and video surveillance. Methods for object detection are generally classified into machine learning-based approaches or deep learning-based approaches, and recently, among them, An object detection technique based on a convolutional neural network (CNN) is widely used.

The convolutional neural network (hereinafter, referred to as 'CNN') is a type of deep neural network (DNN), and is completely connected to one or several convolutional layers and a pooling layer. It is composed of fully connected layers, has a structure suitable for learning two-dimensional data, and can be trained through a backpropagation algorithm.

Such CNNs are widely used in various application fields such as object classification in images and object detection. Recently, CNN is a method that detects and classifies objects in manufacturing plants or distribution centers and aggregates the number of classified objects at the same time. How to use it is being studied.

However, when classification and aggregation of objects, that is, counting the number of objects, are simultaneously performed using CNN, the training process takes a lot more time than just the classification task, and the amount of computation during training is also huge, making it difficult to apply in practice. .

In addition, although the accuracy of object classification is high, there is a problem in that an error may occur or the accuracy may be greatly reduced in the counting process, such as when an object is recognized as an object by optical conditions such as light and shadow.

In order to solve such a problem, a separate image pre-processing task must be performed. Image pre-processing using image processing takes a long time to be optimized, and the optimized pre-processing process operates only on specific data to classify various products. and it is difficult to be used simultaneously for aggregation.

In addition, since both the input and output used for training are images, it is difficult to evaluate the efficiency of the preprocessing result because evaluation is possible only through additional training and verification processes using the corresponding result.

1. Republic of Korea Patent Publication No. 10-1828011 (2018. 02. 21. Announcement) 2. Republic of Korea Patent Publication No. 10-2097905 (2020. 04. 06. Announcement)

The present invention has been devised to solve the problems of the prior art as described above, and an object of the present invention is to separate the object aggregation and classification process into two steps, but in the object aggregation process, a genetic algorithm is used, and the object In the classification process of , it is to provide a genetic algorithm and an object detection method using CNN that can significantly reduce the amount of computation and optimization time using CNN and improve the accuracy of object classification at the same time.

In addition, the present invention provides an object detection method using a genetic algorithm and CNN that automatically configures and optimizes hyperparameters of an image processing process using an image of an object and types and sequences of operations through learning using a genetic algorithm. There is another purpose to providing.

In addition, the present invention provides a genetic algorithm and an object detection method using CNN that can improve the accuracy of object classification by enabling the image processed by the optimized image processing process to be used for object type classification using CNN. has another purpose.

The present invention for achieving the above objects,

A first step of counting the number of objects present in an image containing a plurality of objects through image processing optimized by a genetic algorithm, and a CNN trained using individual images of objects and a second step of classifying the types of objects present in the image.

In this case, the image including the plurality of objects is characterized in that it is an image including the objects stacked in a horizontal or vertical direction.

In addition, the second step is characterized in that the type of object is classified by the trained CNN by receiving the image obtained through the image processing optimized in the first step.

In addition, the first step includes a first image acquisition step of acquiring an image including a plurality of objects, and a type of operation used for image processing for the acquired image and parameters used for the operation into chromosomes and genes, respectively. A first learning step of automating the optimization process of image processing through learning the genetic algorithm used, and an aggregation step of counting the number of objects by processing the image obtained by the image processing optimized in the first learning step characterized in that

In addition, the aggregation step is characterized in that it comprises an individual image extraction step of extracting an individual image of each object by performing contour detection on the image processed by the optimized image processing.

In addition, the optimized image processing method learned by the genetic algorithm in the first learning step includes a blur operation step of applying a blur effect to the image including the plurality of objects, and an erosion operation on the image to which the blur effect is applied. The erode operation step, the whitening operation step for whitening the image on which the erosion operation has been performed, the dilate operation step for performing the dilation operation on the whitened image, the binary operation step for binarizing the image on which the dilation operation has been performed into black and white, and binarization It is characterized in that it includes a close operation step of performing a closing operation on the image.

In addition, the second step includes a second image acquisition step of cutting and obtaining an image of an individual object from an image including a plurality of objects, and a second learning step of performing CNN learning using the acquired image of the individual object; and a classification step of classifying the type of object included in the image processed through image processing optimized by the genetic algorithm in the first step using the CNN trained in the second learning step.

According to the present invention, the aggregation and classification process of objects, which are simultaneously performed in the general object detection process, is divided into two steps, and the amount of computation can be reduced by using a genetic algorithm in the object aggregation process. The time required for optimization can be greatly reduced, and in the object classification process, the image processed by the optimized image processing process can be used for object classification using CNN. have an effect

1 is a block diagram conceptually illustrating an object classification and aggregation method using a genetic algorithm and CNN according to the present invention.
2 is a flowchart sequentially illustrating an object classification and aggregation method using a genetic algorithm and CNN according to the present invention.
3 is a diagram conceptually illustrating an image processing process optimized as a learning result in the first learning step of the present invention shown in FIG. 2 .

Hereinafter, preferred embodiments of an object classification and aggregation method using a genetic algorithm and CNN according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram conceptually illustrating an object classification and aggregation method using a genetic algorithm and CNN according to the present invention, and FIG. 2 is a flowchart sequentially illustrating an object classification and aggregation method using a genetic algorithm and CNN according to the present invention. , FIG. 3 is a diagram conceptually illustrating an image processing process optimized as a learning result in the first learning step of the present invention shown in FIG. 2 .

The present invention divides the aggregation and classification process of objects simultaneously in the general object detection process into two steps, but uses a genetic algorithm in the object aggregation process and CNN in the object classification process. It relates to an object detection method using a genetic algorithm and CNN that can significantly reduce the amount of computation and optimization time and improve the accuracy of object classification, and the configuration of which is largely the first step, as shown in FIG. (S100) and a second step (S200).

First, the first step (S100) relates to a step of processing an image including a plurality of objects by image processing to count the number of objects present in the image, and the image processing process used in this case is a genetic algorithm (Genetic). Algorithm), and in the first step ( S100 ), the number of objects present in the image can be accurately counted after an image processing process optimized through learning by a genetic algorithm is performed.

At this time, the plurality of objects included in the image may be objects in a state of being stacked in a horizontal or vertical direction, where the objects stacked in the horizontal direction are in a state in which a plurality of objects are placed in a horizontal direction as shown in FIG. 3 . It means a state in which the objects are stacked in the up and down directions, and the vertically stacked object means a state in which a plurality of objects are stacked in the left and right directions, such as books arranged on a bookshelf, in a vertical direction.

Hereinafter, an object stacked in a horizontal or vertical direction will be collectively referred to as a stacked object.

In addition, when classifying and counting objects in the present invention, it is recognized as an object by optical conditions such as light or shadow, causing an error in the counting process or a problem that the accuracy may be greatly reduced. Since most of the objects are in a stacked state, an image used for image processing, that is, an image including a plurality of objects, will be described based on an image including a plurality of stacked objects.

On the other hand, the first step (S100) may include a first image acquisition step (S110), a first learning step (S120) and an aggregation step (S130), wherein the first image acquisition step (S110) is a camera, etc. A step of acquiring an image or image of a state in which a plurality of objects photographed by the photographing means of can

At this time, when the form input to the computer is a moving picture captured by a photographing means, it may be supplied to an image processing program in the form of a captured image, and other elements included in the image using a normal image editing program such as Photoshop It is also possible to remove all of them and supply only the image of the stacked object to the image processing program.

Next, the first learning step (S120) is a step of automating the optimization process of image processing for the stacked object image input through the first image obtaining step (S110) through learning. A genetic algorithm may be used.

More specifically, the genetic algorithm is a meta-heuristic-based algorithm that aims to find an optimal solution to a given problem by imitating the evolutionary process of living things. It has the advantage of being able to find the optimal solution within.

Such a genetic algorithm can largely include five steps: generation of the initial generation, evaluation of fitness of the first generation, crossing and mutation of the first generation for generation of the next generation, evaluation of fitness of the next generation, and crossing and mutation of the current generation for generation of the next generation. Since a more specific learning method of such a genetic algorithm may be the same as a method used in various existing optimization problems, a detailed description thereof will be omitted.

In the present invention, since the optimization of image processing is a solution to be obtained with a genetic algorithm, the optimization of image processing is achieved by learning a genetic algorithm using the chromosome as the type of operation used for image processing and the gene as the parameters used for the operation. The process can be automated, and when the learning of the genetic algorithm is completed, hyperparameters of the image processing process using the image of an object, the type and sequence of operations, etc. can be automatically configured and optimized.

Next, in the aggregation step ( S130 ), the image input through the first image acquisition step ( S110 ) is processed using the image processing technique optimized by the genetic algorithm learned in the first learning step ( S120 ). In the step of counting the number, when an image of a state in which a plurality of objects are stacked is input, the learned genetic algorithm automatically creates an optimized image processing process to process the input image, and calculates the number of objects from the processed result. It may be configured to identify and aggregate.

More specifically, FIG. 3 shows an example of an optimized image processing process automatically generated by the genetic algorithm learned in the first learning step (S120), which is input through the first image acquisition step (S110). A blur operation step of applying a blur effect to the stacked object image, an erode operation step of performing an erosion operation on the image to which the blur effect has been applied, a whitening operation step of whitening the image on which the erosion operation has been performed, and an expansion operation on the whitened image A dilate operation step of performing , a binary operation step of binarizing the image on which the dilation operation is performed into black and white, and a close operation step of performing a closing operation on the binarized image may be sequentially included.

Since the operation steps of the image processing process as described above are already used for image processing in various fields, that is, pre-processing of an acquired image, a more detailed description thereof will be omitted.

In addition, although not shown, the order of operations performed in the image processing process is changed or not illustrated according to the color, size, etc. of the stacked object included in the image input through the first image obtaining step S110 Other operations of may be additionally performed.

The stacked object image processed by the automatically generated optimization image processing process as described above is used to determine the number of stacked objects. It may further include an extraction step (S132).

In more detail, the individual image extraction step ( S132 ) is a step of extracting individual images of an object by performing contour detection on the stacked object image processed by the optimization image processing process to extract individual images of the object. When individual images are extracted through , the accuracy of counting the number of objects can be further improved.

In addition, the individual images of the objects extracted as described above may be used to classify the types of objects in the second step S200 to be described later, which will be described later.

Next, the second step (S200) is a step for classifying the type of object included in the image of the stacked object. In the second step (S200), a convolutional neural network, that is, a CNN (Convolutinal Neural Network) can be used .

In more detail, the second step (S200) may include a second image acquisition step (S210), a second learning step (S220), and a classification step (S230). First, the second image acquisition step (S210) ) is a step of cutting and obtaining an image of an individual object from an image of a stacked object, and the obtained image of an individual object may be used for CNN learning in the second learning step (S220).

That is, in the second image acquisition step (S210), from the image of the stacked object obtained in the first image acquisition step (S110), images of individual objects are cropped using an image editing program such as Photoshop, and CNN learning is performed. can be entered into the program for

Next, the second learning step (S220) is a step of performing CNN learning using the image of an individual object obtained through the second image obtaining step (S210). As described above, object detection technology based on CNN learning Since it is widely used, a detailed description thereof will be omitted.

However, in the present invention, the first learning step (S120) using the image of the stacked object obtained in the first image obtaining step (S110), that is, learning through a genetic algorithm, and the second image obtaining step (S210) Since the second learning step (S220) using images of individual objects, that is, CNN learning, can be performed simultaneously, the overall learning time required for automating object classification and aggregation can be shortened.

Next, the classification step (S230) is a step of classifying the type of object included in the image using the learning, that is, the trained CNN in the second learning step (S220). As described above, the first step (S100) can utilize the processed image through image processing optimized by the genetic algorithm.

More specifically, the stacked object image processed through the image processing process automatically optimized by the genetic algorithm in the first step (S100) is extracted as an individual image by contour detection in the individual image extraction step (S132). It is used for counting the number of objects and is automatically input into a program for performing CNN at the same time and may be configured to be used for classification of object types by CNN in the classification step (S230).

Accordingly, since the image processed by the optimized image processing process can be used for classification of object types using CNN, the accuracy of object classification can be further improved.

Therefore, according to the object classification and aggregation method using the genetic algorithm and CNN according to the present invention as described above, the aggregation and classification process of objects simultaneously performed in the general object detection process is divided into two steps, In the object aggregation process, the amount of computation can be reduced by using a genetic algorithm, and the time required for optimization can be greatly reduced. In the object classification process, the image processed by the optimized image processing process can be It has various advantages such as being able to improve the accuracy of object classification by allowing it to be used for classification tasks.

Although the above-described embodiments have been described with respect to the most preferred examples of the present invention, it is not limited to the above-described embodiments, and it is apparent to those skilled in the art that various modifications are possible without departing from the technical spirit of the present invention.

The present invention relates to an object classification and aggregation method using a genetic algorithm and CNN, and more particularly, the aggregation and classification process simultaneously performed in a general object detection process is performed by dividing the object aggregation and classification process into two steps, Object detection using a genetic algorithm and CNN that uses a genetic algorithm in the aggregation process of it's about how

S100: first step S110: first image acquisition step
S120: first learning step S130: counting step
S132: individual image extraction step S200: second step
S210: second image acquisition step S220: second learning step
S230: Classification stage

Claims (7)

A first step of counting the number of objects present in an image including a plurality of objects through image processing optimized by a genetic algorithm;
Object classification and aggregation method using a genetic algorithm and CNN, characterized in that it comprises a second step of classifying the type of object existing in the image including the plurality of objects by a CNN trained using individual images of the object .
The method of claim 1,
The object classification and counting method using a genetic algorithm and CNN, characterized in that the image including the plurality of objects is an image including the objects in a state of being horizontally or vertically stacked.
The method of claim 1,
The second step is an object classification and aggregation method using a genetic algorithm and CNN, characterized in that the type of object is classified by a trained CNN by receiving the image obtained through the image processing optimized in the first step.
The method of claim 1,
The first step is
A first image acquisition step of acquiring an image including a plurality of objects;
A first learning step of automating the optimization process of image processing through genetic algorithm learning using the type of operation used for image processing for the acquired image and parameters used for the operation as chromosomes and genes, respectively;
Object classification and aggregation method using a genetic algorithm and CNN, characterized in that it comprises an aggregation step of processing the obtained image by image processing optimized in the first learning step and counting the number of objects.
5. The method of claim 4,
The aggregation step is an object classification and aggregation method using a genetic algorithm and CNN, characterized in that it comprises an individual image extraction step of extracting an individual image of each object by performing contour detection on the image processed by the optimized image processing.
5. The method of claim 4,
The optimized image processing method learned by the genetic algorithm in the first learning step includes a blur operation step of applying a blur effect to the image including the plurality of objects, and an erode operation that performs an erosion operation on the image to which the blur effect is applied. step, a whitening operation step of whitening the image on which the erosion operation has been performed, a dilate operation step of performing a dilation operation on the whitened image, a binary operation step of binarizing the image on which the dilation operation has been performed into black and white, and the binarized image Object classification and aggregation method using a genetic algorithm and CNN, characterized in that it includes a close operation step of performing a closing operation on the .
The method of claim 1,
The second step is
A second image acquisition step of cutting out and acquiring an image of an individual object from an image including a plurality of objects;
A second learning step of performing CNN learning using the acquired image of an individual object;
Genetic algorithm comprising a classification step of classifying the type of object included in the image processed through image processing optimized by the genetic algorithm in the first step using the CNN trained in the second learning step and object classification and aggregation method using CNN.

Images