KR102285240B1 - Method to train model - Google Patents

Abstract

According to one embodiment of the present disclosure, disclosed is a model learning method. The model learning method may comprise: an operation of calculating an input image using a neural network model; an operation of comparing a prediction box outputted from the neural network model with a ground truth box and calculating an accuracy of the prediction box; and an operation of performing learning of the neural network model by performing back propagation for the neural network model based on the accuracy.

Description

How to train a model {METHOD TO TRAIN MODEL}

The present invention relates to a method for learning a model, and more particularly, to a method for learning a model based on a prediction box and a measurement box.

Recently, artificial intelligence systems have been used in various fields. Unlike the existing rule-based smart system, an artificial intelligence system is a system in which a machine learns, judges, and becomes smarter by itself. As artificial intelligence systems are used, the recognition rate improves and users can understand user preferences more accurately, so existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

Artificial intelligence technology consists of machine learning (eg, deep learning) and elemental technologies using machine learning.

Machine learning is an algorithm technology that categorizes/learns the characteristics of input data by itself, and element technology uses machine learning algorithms such as deep learning to simulate functions such as cognition and judgment of the human brain. It consists of technical fields such as understanding, reasoning/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology for recognizing and applying/processing human language/text, and includes natural language processing, machine translation, dialogue system, question and answer, and speech recognition/synthesis. Visual understanding is a technology for recognizing and processing objects like human vision, and includes object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image improvement, and the like. Inferential prediction is a technique for logically reasoning and predicting by judging information, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, recommendation, and the like. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data generation/classification) and knowledge management (data utilization). Motion control is a technology for controlling autonomous driving of a vehicle and movement of a robot, and includes motion control (navigation, collision, driving), manipulation control (action control), and the like.

In order to perform machine learning in various fields, there is a demand in the art to improve learning efficiency and speed for artificial neural networks.

Korean Patent Application Laid-Open No. 2020-0021408 discloses a method for learning a neural network.

The present disclosure has been made in response to the above-mentioned background art, and an object of the present disclosure is to provide a model learning method.

A model learning method according to an embodiment of the present disclosure for realizing the above-described task, the method comprising: calculating an input image using a neural network model; calculating accuracy of the prediction box by comparing a prediction box output from the neural network model with a ground truth box; and training the neural network model by performing backpropagation on the neural network model based on the accuracy.

In an alternative embodiment of the operations to perform the following operations for providing a model learning method, a prediction box output from the neural network model is compared with a ground truth box to determine the prediction box. The operation of calculating the accuracy may include calculating the accuracy of the prediction box based on an area of a sum area and an intersection area of the prediction box and the actual measurement box.

In an alternative embodiment of the operations to perform the following operations for providing a model learning method, a prediction box output from the neural network model is compared with a ground truth box to determine the prediction box. The operation of calculating the accuracy may include: determining an intersection of the prediction box and the actual measurement box as a first reference point; comparing a vertex of the prediction box with an edge of the measured box, and determining the vertex as a second reference point when the vertex is located inside the measured box; and calculating the accuracy of the prediction box based on the first reference point and the second reference point.

In an alternative embodiment of the operations to perform the following operations for providing a model learning method, the operation of calculating the accuracy of the prediction box based on the first reference point and the second reference point includes: the first reference point and and calculating an area of an intersection area between the prediction box and the actual measurement box based on the second reference point.

In an alternative embodiment of the operations to perform the following operations for providing a model learning method, a prediction box output from the neural network model is compared with a ground truth box to determine the prediction box. The operation of calculating the accuracy may include an operation of calculating the accuracy by a number of times corresponding to the number of the actual measurement boxes when two or more actual measurement boxes exist.

In an alternative embodiment of the operations to perform the following operations for providing a model learning method, a prediction box output from the neural network model is compared with a ground truth box to determine the prediction box. The operation of calculating the accuracy may include calculating the accuracy of the two or more prediction boxes by comparing each of the two or more prediction boxes output from the neural network model with the two or more actual measurement boxes.

In an alternative embodiment of the operations to perform the following operations for providing a model learning method, a prediction box output from the neural network model is compared with a ground truth box to determine the prediction box. The operation of calculating the accuracy may include an operation of determining one prediction box corresponding to each of the two or more actual measurement boxes based on the actual measurement box.

In an alternative embodiment of the operations to perform the following operations for providing a model learning method, the operation of determining one prediction box corresponding to each of the two or more actual measurement boxes based on the actual measurement box may include: determining a grid including each of the actual measurement boxes; matching each of one prediction box and one actual measurement box based on the grid; and calculating the accuracy of the matched prediction box and the actual measurement box.

In an alternative embodiment of the operations to perform the following operations for providing a model learning method, the accuracy of the two or more prediction boxes by comparing each of the two or more prediction boxes output from the neural network model with the two or more actual measurement boxes The operation of calculating may include extracting one accuracy value that is a basis of the backpropagation based on the accuracy of the two or more prediction boxes.

As a computer program stored in a computer readable storage medium according to an embodiment of the present disclosure for realizing the above-described problem, the computer program, when executed on one or more processors, performs the following operations for performing chest image reading The operation may include: calculating an input image using a neural network model; calculating accuracy of the prediction box by comparing a prediction box output from the neural network model with a ground truth box; and training the neural network model by performing backpropagation on the neural network model based on the accuracy.

As a server according to an embodiment of the present disclosure for realizing the above-described problem, the processor including one or more cores; network department; and memory; including, wherein the processor calculates an input image using a neural network model, and compares a prediction box output from the neural network model with a ground truth box to determine the accuracy of the prediction box. The neural network model may be trained by calculating, and performing backpropagation on the neural network model based on the accuracy.

As a computer-readable recording medium storing a data structure corresponding to a parameter of a neural network that is at least partially updated in the learning process according to an embodiment of the present disclosure for realizing the above-described task, the operation of the neural network is based at least in part on the parameter, the neural network comprising: computing an input image using a neural network model; calculating accuracy of the prediction box by comparing a prediction box output from the neural network model with a ground truth box; and training the neural network model by performing backpropagation on the neural network model based on the accuracy.

The present disclosure may provide a model training method.

1 is a diagram illustrating a block diagram of a computing device that performs operations for performing model learning according to an embodiment of the present disclosure.
2 is a diagram exemplarily illustrating a calculation method for performing model learning according to an embodiment of the present disclosure.
3 is a diagram exemplarily illustrating a calculation method for performing model learning according to an embodiment of the present disclosure.
4 is a diagram exemplarily illustrating a flowchart of a model learning method according to an embodiment of the present disclosure.
5 is a block diagram of a computing device according to an embodiment of the present disclosure.

Various embodiments are now described with reference to the drawings. In this specification, various descriptions are presented to provide an understanding of the present disclosure. However, it is apparent that these embodiments may be practiced without these specific descriptions.

The terms “component,” “module,” “system,” and the like, as used herein, refer to a computer-related entity, hardware, firmware, software, a combination of software and hardware, or execution of software. For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, a thread of execution, a program, and/or a computer. For example, both an application running on a computing device and the computing device may be a component. One or more components may reside within a processor and/or thread of execution. A component may be localized within one computer. A component may be distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures stored therein. Components may communicate via a network such as the Internet with another system, for example via a signal having one or more data packets (eg, data and/or signals from one component interacting with another component in a local system, distributed system, etc.) may communicate via local and/or remote processes depending on the data being transmitted).

In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from context, "X employs A or B" is intended to mean one of the natural implicit substitutions. That is, X employs A; X employs B; or when X employs both A and B, "X employs A or B" may apply to either of these cases. It should also be understood that the term “and/or” as used herein refers to and includes all possible combinations of one or more of the listed related items.

Also, the terms "comprises" and/or "comprising" should be understood to mean that the feature and/or element in question is present. However, it should be understood that the terms "comprises" and/or "comprising" do not exclude the presence or addition of one or more other features, elements and/or groups thereof. Also, unless otherwise specified or unless the context is clear as to designating a singular form, the singular in the specification and claims should generally be construed to mean "one or more."

Those skilled in the art will further appreciate that the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or combinations of both. It should be recognized that they can be implemented with To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Descriptions of the presented embodiments are provided to enable those skilled in the art to use or practice the present invention. Various modifications to these embodiments will be apparent to those skilled in the art of the present disclosure. The generic principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present invention is not limited to the embodiments presented herein. This invention is to be construed in its widest scope consistent with the principles and novel features presented herein.

In an embodiment of the present disclosure, the server may include other configurations for performing a server environment of the server. The server may include any type of device. The server is a digital device, and may be a digital device equipped with a processor, such as a laptop computer, a notebook computer, a desktop computer, a web pad, and a mobile phone, and having a computing capability with a memory. The server may be a web server that processes the service. The above-described types of servers are merely examples, and the present disclosure is not limited thereto.

In this specification, neural network, artificial neural network, and network function may often be used interchangeably.

1 is a diagram illustrating a block diagram of a computing device that performs an operation for providing a model learning method according to an embodiment of the present disclosure.

An image may contain one or more objects. The processor 120 may detect one or more objects included in the image by using the neural network model. The processor 120 may have to utilize a pre-trained neural network model to detect one or more objects.

The pre-trained network model of the present disclosure may be trained in at least one of supervised learning, unsupervised learning, and semi-supervised learning. Learning the network function is to minimize the error in the output. In learning the network function, iteratively input the training data to the network function. In the training of the network function, the output of the network function and the error of the target are calculated for the training data. The weight of each node of the network function can be updated by backpropagating the error of the network function from the output layer of the network function to the input layer in the direction to reduce the error in the learning of the network function.

The processor 120 may calculate an error by inputting the labeled training data to the network function, and comparing the classification, which is an output of the network function, with the label of the training data. The calculated error can be backpropagated from the output layer to the input layer, which is the reverse in the network function. According to the backpropagation from the output layer to the input layer, the connection weight of each node of each layer may be updated. A change amount of the connection weight of each node to be updated may be determined according to a learning rate. The computation of network functions on input data and backpropagation of errors can constitute a learning cycle. The learning rate may be applied differently according to the number of repetitions of the learning cycle of the network function. For example, in the early stages of learning a network function, a high learning rate can be used to ensure that the network function quickly acquires a certain level of performance, thereby increasing efficiency. For example, in the late learning phase of a network function, a low learning rate can be used to increase the accuracy. Image segmentation may include, for example, image processing of separating a part of an image to be distinguished from another part based on pixels, edges, colors, etc. extracted from the image. The classification that is the output of the network function may be pixel-by-pixel image segmentation. The image data may be classified by acquiring a scoring value in units of pixels of an image, and performing image segmentation based on each pixel scoring value. Image segmentation may refer to image processing for separating a part of an image to be distinguished from another part.

If the object included in the image is a rotated object instead of in a forward direction, learning performance for the rotated object may deteriorate. The processor 120 may perform learning on the neural network model in consideration of the rotation angle of the object. In the model learning method according to an embodiment of the present disclosure, when learning a rotation object, not only the position of the object but also the rotation angle of the object may be considered. That is, it is possible to train the neural network model by backpropagating the information about the rotation angle.

A model learning method according to an embodiment of the present disclosure may improve learning performance and speed for a rotated object. The model learning method according to an embodiment of the present disclosure may improve accuracy of a neural network model by resolving learning ambiguity with respect to angle. In addition, when compared with the existing angle learning method, it is possible to perform the learning of the angle stably.

The processor 120 may calculate the input image using a neural network model. The input image may be an image including one or more objects. The object included in the input image may be at least partially rotated instead of in a forward direction. For example, the person object included in the input image may not be a person standing at an angle of 180 degrees, but may be at least a partially rotated object. Specific description of the above-described object is merely an example, and the present disclosure is not limited thereto.

The above-described neural network model may be a deep neural network. Throughout this specification, the terms neural network, network function, and neural network may be used interchangeably. A deep neural network (DNN) may refer to a neural network including a plurality of hidden layers in addition to an input layer and an output layer. Deep neural networks can be used to identify the latent structures of data. In other words, it can identify the potential structure of photos, texts, videos, voices, and music (for example, what objects are in the photos, what the text and emotions are, what the texts and emotions are, etc.) . Deep neural networks include a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), and a deep belief network (DBN). , Q network, U network, Siamese network, and the like.

A convolutional neural network is a kind of deep neural network, and includes a neural network including a convolutional layer. Convolutional neural networks are a type of multilayer perceptorns designed to use minimal preprocessing. A CNN can consist of one or several convolutional layers and artificial neural network layers combined with them. CNNs can additionally utilize weights and pooling layers. Thanks to this structure, CNN can fully utilize the input data of the two-dimensional structure. Convolutional neural networks can be used to recognize objects in images. A convolutional neural network can process image data by representing it as a matrix with dimensions. For example, in the case of RGB (red-green-blue) encoded image data, each R, G, and B color may be represented as a two-dimensional (eg, two-dimensional image) matrix. That is, the color value of each pixel of the image data may be a component of the matrix, and the size of the matrix may be the same as the size of the image. Accordingly, image data can be represented by three two-dimensional matrices (three-dimensional data array).

In a convolutional neural network, a convolutional process (input/output of the convolutional layer) can be performed by moving the convolutional filter and multiplying the convolutional filter and matrix components at each position of the image. The convolutional filter may be composed of an n*n matrix. A convolutional filter may be composed of a fixed type filter that is generally smaller than the total number of pixels in the image. That is, when an m*m image is input as a convolutional layer (for example, a convolutional layer having a size of n*n of a convolutional filter), a matrix representing n*n pixels including each pixel of the image This convolutional filter can be multiplied by a component (ie, the product of each component of a matrix). A component matching the convolutional filter may be extracted from the image by multiplication with the convolutional filter. For example, a 3*3 convolutional filter for extracting upper and lower linear components from an image can be configured as [[0,1,0], [0,1,0], [0,1,0]] can When the 3*3 convolutional filter for extracting the upper and lower linear components from the image is applied to the input image, the upper and lower linear components matching the convolutional filter may be extracted and output from the image. The convolutional layer may apply a convolutional filter to each matrix (ie, R, G, B colors for R, G, and B coded images) for each channel representing the image. The convolutional layer may extract a feature matching the convolutional filter from the input image by applying the convolutional filter to the input image. The filter value of the convolutional filter (ie, the value of each component of the matrix) may be updated by backpropagation in the learning process of the convolutional neural network.

A subsampling layer is connected to the output of the convolutional layer to simplify the output of the convolutional layer, thereby reducing the amount of memory and computation. For example, if you input the output of the convolutional layer to a pooling layer with a 2*2 max pooling filter, you can compress the image by outputting the maximum value included in each patch for every 2*2 patch in each pixel of the image. can The above-described pooling may be a method of outputting a minimum value from a patch or an average value of a patch, and any pooling method may be included in the present disclosure.

A convolutional neural network may include one or more convolutional layers and subsampling layers. The convolutional neural network may extract features from an image by repeatedly performing a convolutional process and a subsampling process (eg, the aforementioned max pooling, etc.). Through iterative convolutional and subsampling processes, neural networks can extract global features from images.

An output of the convolutional layer or the subsampling layer may be input to a fully connected layer. A fully connected layer is a layer in which all neurons in one layer and all neurons in neighboring layers are connected. The fully connected layer may mean a structure in which all nodes of each layer are connected to all nodes of other layers in a neural network.

In an embodiment of the present disclosure, in order to perform segmentation on medical data included in a medical image, the neural network may include a deconvolutional neural network (DCNN). A deconvolutional neural network performs an operation similar to that calculated in the reverse direction of a convolutional neural network. The deconvolutional neural network may output features extracted from the convolutional neural network as a feature map related to original data. A description of a specific configuration for a convolutional neural network is discussed more specifically in US Patent No. US9870768B2, which is incorporated herein by reference in its entirety.

The processor 120 may calculate the accuracy of the prediction box by comparing a prediction box output from the neural network model with a ground truth box.

The training data may include a training image and a labeling result for an object included in the training image. A labeling result for the training image may be an actual measurement box. The actual measurement box may be a box indicating an actual position of an object included in an image. The actual measurement box may be a labeling result for an object included in an image. The actual measurement box may mean a result of displaying the actual location of an object included in the training image with respect to the training image, which is the training data.

The prediction box may be a result output by calculating an input image using a neural network model. The processor 120 may calculate the input image using the neural network model, predict the position of the object included in the input image, and output a prediction box for the object.

The processor 120 may train the neural network model by using a difference between the actual measurement box, which is an actual correct answer, and the prediction box, which is a result of calculation using the neural network model. The processor 120 may allow the neural network model to learn the difference between the output of the neural network model and the actual correct answer.

The processor 120 may calculate the accuracy of the prediction box based on an area of a sum area and an intersection area of the prediction box and the actual measurement box. The processor 120 may calculate the sum area area, which is a value obtained by subtracting the area of the intersection area from the sum of the areas of the prediction box and the actual measurement box. The processor 120 may calculate the area of the intersection area between the prediction box and the actual measurement box. The processor 120 may calculate the accuracy of the prediction box by dividing the area of the sum area by the area of the intersection area. That is, the processor 120 may output the accuracy of the prediction box by calculating (area of the intersection area)/(area of the sum area), that is, an intersection over union (IOU) value. The processor 120 may perform learning on the neural network model by backpropagating the accuracy value of the prediction box.

Hereinafter, a method for the processor 120 to calculate the accuracy value of the prediction box will be described in detail. Hereinafter, it will be described with reference to FIG. 2 . 2 is a diagram exemplarily illustrating a prediction box 220 and a measurement box 210 .

The processor 120 may determine the intersection of the prediction box 220 and the actual measurement box 210 as the first reference points 232 , 234 , 236 , and 238 .

The processor 120 compares the vertex of the prediction box 220 with the edge of the actual measurement box 210 , and when the vertex is located inside the actual measurement box 210 , sets the vertex to the second It can be determined as the reference point 242 .

The processor 120 may calculate the accuracy of the prediction box 220 based on the first reference point 232 , 234 , 236 , 238 and the second reference point 242 .

The processor 120 calculates the area of the intersection of the prediction box 220 and the actual measurement box 210 based on the first reference points 232 , 234 , 236 , 238 and the second reference point 242 . can

The processor 120 may calculate the area of regions connected by lines connecting the first reference points 232 , 234 , 236 , and 238 to the second reference points 242 . The processor 120 calculates the area of the intersection area using the first reference point and the second reference point, thereby accurately deriving the calculation result of the boxes for the rotated objects.

Even when learning the rotated objects, the processor 120 can easily and quickly perform learning by setting the first reference point and the second reference point. In the case of the existing rotating object learning methods, by calculating the angle change of the rotating object, learning efficiency is lowered, and there are disadvantages that excessive calculation is required. When the learning method according to an embodiment of the present disclosure is used, a difference between a prediction box and an actual measurement box for an object rotated accurately at a high speed may be calculated.

When two or more actual measurement boxes exist, the processor 120 may calculate the accuracy by a number of times corresponding to the number of actual measurement boxes. The processor 120 may perform an accuracy calculation as much as the number of actual measurement boxes. For example, when there are three actual measurement boxes labeled in the training data, three accuracy calculations may be performed. In the case of the existing learning method, when there are three actual measurement boxes and three prediction boxes, the neural network model was trained by performing 3*3, that is, 9 accuracy calculations. In the case of the learning method of a neural network model according to an embodiment of the present disclosure, an amount of unnecessary computation can be reduced by performing an accuracy calculation only as much as the number of actually measured boxes.

The processor 120 may determine one prediction box corresponding to each of the two or more actual measurement boxes based on the actual measurement box. In order to perform an accuracy calculation as many as the number of actual measurement boxes, the processor 120 may determine one measurement box and one prediction box corresponding to the measurement box to perform the accuracy operation. That is, accuracy calculation may be performed on the first measured box with the first prediction box, and accuracy calculation may not be performed between the first measured box and the second predicted box, and between the first measured box and the third predicted box.

The accuracy calculation will be described with reference to FIG. 3 . 3 is a diagram exemplarily illustrating a prediction box and an actual measurement box. 3 shows three prediction boxes and three actual measurement boxes.

The processor 120 may determine a grid including each of the two or more actual measurement boxes. The processor 120 may determine one grid including the center point of the actual measurement box. The black dot at the center of each box shown in FIG. 3 may be the center point of the actually measured box. The central point shown in FIG. 3 is an illustrative point to explain the learning method, and this point may not be indicated in the actual learning process.

The processor 120 may match each of one prediction box and one actual measurement box based on the grid. The processor 120 may determine one prediction box predicted for the grid based on the grid determined for the actual measurement box. The processor 120 may determine one grid based on the center point of the actual measurement box, and determine one prediction box with respect to the determined grid to perform an accuracy calculation between the corresponding actual measurement box and the prediction box. The processor 120 may calculate the accuracy of the matched prediction box and the actual measurement box. The accuracy calculation for the matched prediction box and the actual measurement box may be an IOU operation as described above.

The processor 120 may not perform an accuracy calculation between other prediction boxes located outside the corresponding grid and the actual measurement box corresponding to the corresponding grid.

3 shows three actual measurement boxes and three prediction boxes, respectively. In the case of the existing learning methods, since there are 3 actual measurement boxes and 3 prediction boxes, there is a disadvantage that 9 accuracy calculations must be performed. When performing 9 accuracy calculations, the consumption of computing power is considerably increased, and the accuracy may also decrease.

In the case of the model learning method according to an embodiment of the present disclosure, since there are three measured boxes, a grid corresponding to each of the three measured boxes is determined, and one prediction box corresponding to the grid is determined, It is possible to perform an accuracy calculation between the three actual measurement boxes and one prediction box corresponding thereto. That is, according to the learning method of a neural network model according to an embodiment of the present disclosure, since it is enough to perform an accuracy calculation three times, the consumption of computing power is also much less, so that computational efficiency can be achieved, and between unnecessary boxes Since there is no need to perform the calculation, the accuracy of the calculation can also be achieved.

The processor 120 may calculate the accuracy of the two or more prediction boxes by comparing each of the two or more prediction boxes output from the neural network model with the two or more actual measurement boxes. For example, in the case of FIG. 3 , the processor 120 may calculate the accuracy of each of the three boxes.

The processor 120 may extract one accuracy value that is the basis of the backpropagation based on the accuracy of the two or more prediction boxes. The processor 120 may extract one accuracy value by using a sum operation on the accuracy of two or more prediction boxes. For example, in the case of FIG. 3 , the processor 120 calculates a first accuracy value for the first measured box, a second accuracy value for the second measured box, and a third accuracy value for the third measured box, respectively. can do. The processor 120 may extract one accuracy value by using an average value of the first to third accuracy values. The processor 120 may extract one accuracy value in various methods other than the above-described average value. The detailed description of the above-described accuracy value extraction is merely an example, and the present disclosure is not limited thereto.

The processor 120 may train the neural network model by performing backpropagation on the neural network model based on the accuracy. The processor 120 may train the neural network model using the accuracy value. In the case of teacher learning, by backpropagating an error, which is a difference value between a label, which is a correct answer included in the training data, and an output value of the neural network model, the neural network model can minimize such an error.

The computing device 100 for providing a model learning method according to an embodiment of the present disclosure may include a network unit 110 , a processor 120 , and a memory 130 .

The network unit 110 may transmit/receive an image, etc. according to an embodiment of the present disclosure to another computing device, a server, and the like. In addition, the network unit 110 may enable communication between a plurality of computing devices so that operations for learning a model are distributed in each of the plurality of computing devices. The network unit 110 may enable communication between a plurality of computing devices to distribute calculations for model learning using a network function.

The network unit 110 according to an embodiment of the present disclosure may operate based on any type of wired/wireless communication technology currently used and implemented, such as short-distance (short-range), long-distance, wired and wireless, etc., and may operate in other networks as well. can be used

The processor 120 may include one or more cores, and a central processing unit (CPU) of a computing device, a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU). unit) and the like, and may include a processor for learning the model. The processor 120 may read a computer program stored in the memory 130 to provide an object detection method according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor 120 may perform calculation to provide an object detection method.

The memory 130 may store a computer program for providing a model learning method according to an embodiment of the present disclosure, and the stored computer program may be read and driven by the processor 120 .

The memory 130 according to embodiments of the present disclosure may store a program for the operation of the processor 120 , and may temporarily or permanently store input/output data or events. The memory 130 may store data related to a display and sound. The memory 130 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory), and a RAM. (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic It may include at least one type of storage medium among a disk and an optical disk.

A computer-readable medium storing a data structure is disclosed according to an embodiment of the present disclosure.

The data structure may refer to the organization, management, and storage of data that enables efficient access and modification of data. A data structure may refer to an organization of data to solve a specific problem (eg, data retrieval, data storage, and data modification in the shortest time). A data structure may be defined as a physical or logical relationship between data elements designed to support a particular data processing function. The logical relationship between data elements may include a connection relationship between user-defined data elements. Physical relationships between data elements may include actual relationships between data elements physically stored on a computer-readable storage medium (eg, persistent storage). A data structure may specifically include a set of data, relationships between data, and functions or instructions applicable to data. Through an effectively designed data structure, a computing device can perform an operation while using the resources of the computing device to a minimum. Specifically, the computing device may increase the efficiency of operations, reads, insertions, deletions, comparisons, exchanges, and retrievals through effectively designed data structures.

A data structure may be classified into a linear data structure and a non-linear data structure according to the type of the data structure. The linear data structure may be a structure in which only one piece of data is connected after one piece of data. The linear data structure may include a list, a stack, a queue, and a deck. A list may mean a set of data in which an order exists internally. The list may include a linked list. The linked list may be a data structure in which data is linked in such a way that each data is linked in a line with a pointer. In a linked list, a pointer may contain information about a link with the next or previous data. A linked list may be expressed as a single linked list, a doubly linked list, or a circularly linked list according to a shape. A stack can be a data enumeration structure with limited access to data. A stack can be a linear data structure in which data can be processed (eg, inserted or deleted) at only one end of the data structure. The data stored in the stack may be a data structure LIFO-Last in First Out. A queue is a data listing structure that allows limited access to data. Unlike a stack, the queue may be a data structure that comes out later (FIFO-First in First Out) as data stored later. A deck can be a data structure that can process data at either end of the data structure.

The nonlinear data structure may be a structure in which a plurality of data is connected after one data. The nonlinear data structure may include a graph data structure. A graph data structure may be defined as a vertex and an edge, and the edge may include a line connecting two different vertices. A graph data structure may include a tree data structure. The tree data structure may be a data structure in which one path connects two different vertices among a plurality of vertices included in the tree. That is, it may be a data structure that does not form a loop in the graph data structure.

Throughout this specification, computational model, neural network, network function, and neural network may be used interchangeably. Hereinafter, the neural network is unified and described. The data structure may include a neural network. And the data structure including the neural network may be stored in a computer-readable medium. Data structures, including neural networks, also include preprocessed data for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and the neural network. It may include a loss function for learning of . A data structure comprising a neural network may include any of the components disclosed above. That is, the data structure including the neural network includes preprocessed data for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and the neural network It may be configured to include all or any combination thereof, such as a loss function for learning of . In addition to the above-described configurations, a data structure including a neural network may include any other information that determines a characteristic of a neural network. In addition, the data structure may include all types of data used or generated in the operation process of the neural network, and is not limited to the above. Computer-readable media may include computer-readable recording media and/or computer-readable transmission media. A neural network may be composed of a set of interconnected computational units, which may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is configured by including at least one or more nodes.

The data structure may include data input to the neural network. A data structure including data input to the neural network may be stored in a computer-readable medium. The data input to the neural network may include learning data input in a neural network learning process and/or input data input to the neural network in which learning is completed. Data input to the neural network may include pre-processing data and/or pre-processing target data. The preprocessing may include a data processing process for inputting data into the neural network. Accordingly, the data structure may include data to be pre-processed and data generated by pre-processing. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

The data structure may include the weights of the neural network. (In this specification, a weight and a parameter may be used interchangeably.) And a data structure including a weight of a neural network may be stored in a computer-readable medium. The neural network may include a plurality of weights. The weight may be variable, and may be changed by the user or algorithm in order for the neural network to perform a desired function. For example, when one or more input nodes are interconnected to one output node by respective links, the output node sets values input to input nodes connected to the output node and links corresponding to the respective input nodes. A data value output from the output node may be determined based on the weight. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

By way of example and not limitation, the weight may include a weight variable in a neural network learning process and/or a weight in which neural network learning is completed. The variable weight in the neural network learning process may include a weight at the start of the learning cycle and/or a variable weight during the learning cycle. The weight for which neural network learning is completed may include a weight for which a learning cycle is completed. Accordingly, the data structure including the weight of the neural network may include a data structure including the weight variable in the neural network learning process and/or the weight in which the neural network learning is completed. Therefore, it is assumed that the above-described weights and/or combinations of weights are included in the data structure including the weights of the neural network. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

The data structure including the weights of the neural network may be stored in a computer-readable storage medium (eg, memory, hard disk) after being serialized. Serialization can be the process of converting a data structure into a form that can be reconstructed and used later by storing it on the same or a different computing device. The computing device may serialize the data structure to send and receive data over the network. A data structure including weights of the serialized neural network may be reconstructed in the same computing device or in another computing device through deserialization. The data structure including the weights of the neural network is not limited to serialization. Furthermore, the data structure including the weights of the neural network is a data structure to increase the efficiency of computation while using the resources of the computing device to a minimum (e.g., B-Tree, Trie, m-way search tree, AVL tree, Red-Black Tree). The foregoing is merely an example, and the present disclosure is not limited thereto.

The data structure may include hyper-parameters of the neural network. In addition, the data structure including the hyperparameters of the neural network may be stored in a computer-readable medium. The hyperparameter may be a variable variable by a user. Hyperparameters are, for example, learning rate, cost function, number of iterations of the learning cycle, weight initialization (e.g., setting the range of weight values subject to weight initialization), Hidden Unit The number (eg, the number of hidden layers, the number of nodes of the hidden layer) may be included. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

4 is a flowchart for performing a model learning method according to an embodiment of the present disclosure.

The computing device 100 may calculate 410 the input image using the neural network model.

The computing device 100 may calculate ( 420 ) the accuracy of the prediction box by comparing a prediction box output from the neural network model with a ground truth box. The computing device 100 may calculate the accuracy of the prediction box based on an area of a sum area and an intersection area of the prediction box and the actual measurement box.

The computing device 100 may determine an intersection of the prediction box and the actual measurement box as a first reference point. The computing device 100 may compare a vertex of the prediction box with an edge of the actual measurement box, and when the vertex is located inside the actual measurement box, determine the vertex as the second reference point. The computing device 100 may calculate the accuracy of the prediction box based on the first reference point and the second reference point. The computing device 100 may calculate an area of an intersection area between the prediction box and the actual measurement box based on the first reference point and the second reference point.

When two or more actual measurement boxes exist, the computing device 100 may calculate the accuracy by a number of times corresponding to the number of actual measurement boxes. Computing device 100 is the two output from the neural network model

By performing backpropagation on the neural network model based on the accuracy, the accuracy of the two or more prediction boxes may be calculated by comparing the neural anomaly prediction box and two or more actual measurement boxes, respectively.

The computing device 100 may determine one prediction box corresponding to each of the two or more actual measurement boxes based on the actual measurement box. The computing device 100 may determine a grid including each of the two or more actual measurement boxes. The computing device 100 may match one prediction box and one actual measurement box, respectively, based on the grid. The computing device 100 may calculate the accuracy of the matched prediction box and the actual measurement box.

The computing device 100 may extract one accuracy value that is the basis of the backpropagation based on the accuracy of the two or more prediction boxes.

The computing device 100 may train 430 the network model.

The model learning method according to an embodiment of the present disclosure may be implemented by modules, circuits, means and logic that perform the above operations.

5 is a block diagram of a computing device according to an embodiment of the present disclosure.

5 depicts a simplified, general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

Although the present disclosure has been described above generally in the context of computer-executable instructions that may be executed on one or more computers, those skilled in the art will appreciate that the present disclosure may be implemented in combination with other program modules and/or as a combination of hardware and software. will be.

Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In addition, those skilled in the art will appreciate that the methods of the present disclosure can be applied to single-processor or multiprocessor computer systems, minicomputers, mainframe computers as well as personal computers, hand-held computing devices, microprocessor-based or programmable consumer electronics, etc. (each of which is It will be appreciated that other computer system configurations may be implemented, including those that may operate in connection with one or more associated devices.

The described embodiments of the present disclosure may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Computers typically include a variety of computer-readable media. Any medium accessible by a computer may be a computer-readable medium. Computer-readable media includes volatile and non-volatile media, transitory and non-transitory media, removable and non-removable media. By way of example, and not limitation, computer-readable media may include computer-readable storage media and computer-readable transmission media. Computer-readable storage media includes volatile and non-volatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. includes media. Computer storage media may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disk (DVD) or other optical disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device or other magnetic storage device; or any other medium that can be accessed by a computer and used to store the desired information.

Computer-readable transmission media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as other transport mechanisms, and includes all information delivery media. do. The term modulated data signal means a signal in which one or more of the characteristics of the signal is set or changed so as to encode information in the signal. By way of example, and not limitation, computer-readable transmission media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above are also intended to be included within the scope of computer-readable transmission media.

An example environment 1100 for implementing various aspects of the present disclosure is shown including a computer 1102 , the computer 1102 including a processing unit 1104 , a system memory 1106 , and a system bus 1108 . do. The system bus 1108 couples system components, including but not limited to system memory 1106 , to the processing device 1104 . The processing device 1104 may be any of a variety of commercially available processors. Dual processor and other multiprocessor architectures may also be used as processing unit 1104 .

The system bus 1108 may be any of several types of bus structures that may further be interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. System memory 1106 includes read only memory (ROM) 1110 and random access memory (RAM) 1112 . A basic input/output system (BIOS) is stored in non-volatile memory 1110, such as ROM, EPROM, EEPROM, etc., which is the basic input/output system (BIOS) that helps transfer information between components within computer 1102, such as during startup. contains routines. RAM 1112 may also include high-speed RAM, such as static RAM, for caching data.

Computer 1102 may also be configured for external use within an internal hard disk drive (HDD) 1114 (eg, EIDE, SATA) - this internal hard disk drive 1114 may also be configured for external use within a suitable chassis (not shown). Yes, magnetic floppy disk drive (FDD) 1116 (eg, for reading from or writing to removable diskette 1118), and optical disk drive 1120 (eg, CD-ROM) for reading from, or writing to, disk 1122, or other high capacity optical media such as DVD. The hard disk drive 1114 , the magnetic disk drive 1116 , and the optical disk drive 1120 are connected to the system bus 1108 by the hard disk drive interface 1124 , the magnetic disk drive interface 1126 , and the optical drive interface 1128 , respectively. ) can be connected to The interface 1124 for external drive implementation includes at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

These drives and their associated computer readable media provide non-volatile storage of data, data structures, computer executable instructions, and the like. In the case of computer 1102, drives and media correspond to storing any data in a suitable digital format. Although the description of computer readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will use zip drives, magnetic cassettes, flash memory cards, cartridges, etc. It will be appreciated that other tangible computer-readable media such as etc. may also be used in the exemplary operating environment and any such media may include computer-executable instructions for performing the methods of the present disclosure.

A number of program modules may be stored in the drive and RAM 1112 , including an operating system 1130 , one or more application programs 1132 , other program modules 1134 , and program data 1136 . All or portions of the operating system, applications, modules, and/or data may also be cached in RAM 1112 . It will be appreciated that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

A user may enter commands and information into the computer 1102 via one or more wired/wireless input devices, for example, a pointing device such as a keyboard 1138 and a mouse 1140 . Other input devices (not shown) may include a microphone, IR remote control, joystick, game pad, stylus pen, touch screen, and the like. Although these and other input devices are often connected to the processing unit 1104 through an input device interface 1142 that is connected to the system bus 1108, parallel ports, IEEE 1394 serial ports, game ports, USB ports, IR interfaces, It may be connected by other interfaces, etc.

A monitor 1144 or other type of display device is also coupled to the system bus 1108 via an interface, such as a video adapter 1146 . In addition to the monitor 1144, the computer typically includes other peripheral output devices (not shown), such as speakers, printers, and the like.

Computer 1102 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1148 via wired and/or wireless communications. Remote computer(s) 1148 may be workstations, computing device computers, routers, personal computers, portable computers, microprocessor-based entertainment devices, peer devices, or other common network nodes, and are typically connected to computer 1102 . Although it includes many or all of the components described for it, only memory storage device 1150 is shown for simplicity. The logical connections shown include wired/wireless connections to a local area network (LAN) 1152 and/or a larger network, eg, a wide area network (WAN) 1154 . Such LAN and WAN networking environments are common in offices and companies, and facilitate enterprise-wide computer networks, such as intranets, all of which can be connected to a worldwide computer network, for example, the Internet.

When used in a LAN networking environment, the computer 1102 is connected to the local network 1152 through a wired and/or wireless communication network interface or adapter 1156 . Adapter 1156 may facilitate wired or wireless communication to LAN 1152 , which also includes a wireless access point installed therein for communicating with wireless adapter 1156 . When used in a WAN networking environment, the computer 1102 may include a modem 1158, be connected to a communication computing device on the WAN 1154, or establish communications over the WAN 1154, such as over the Internet. have other means. A modem 1158 , which may be internal or external and a wired or wireless device, is coupled to the system bus 1108 via a serial port interface 1142 . In a networked environment, program modules described for computer 1102 or portions thereof may be stored in remote memory/storage device 1150 . It will be appreciated that the network connections shown are exemplary and other means of establishing a communication link between the computers may be used.

Computer 1102 may be associated with any wireless device or object that is deployed and operates in wireless communication, for example, printers, scanners, desktop and/or portable computers, portable data assistants (PDAs), communications satellites, wireless detectable tags. It operates to communicate with any device or place, and phone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network or may simply be an ad hoc communication between at least two devices.

Wi-Fi (Wireless Fidelity) makes it possible to connect to the Internet, etc. without a wire. Wi-Fi is a wireless technology such as cell phones that allows these devices, eg, computers, to transmit and receive data indoors and outdoors, ie anywhere within range of a base station. Wi-Fi networks use a radio technology called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks may operate in unlicensed 2.4 and 5 GHz radio bands, for example, at 11 Mbps (802.11a) or 54 Mbps (802.11b) data rates, or in products that include both bands (dual band). there is.

One of ordinary skill in the art of this disclosure will understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, instructions, information, signals, bits, symbols and chips that may be referenced in the above description are voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields particles or particles, or any combination thereof.

A person of ordinary skill in the art of the present disclosure will recognize that the various illustrative logical blocks, modules, processors, means, circuits and algorithm steps described in connection with the embodiments disclosed herein include electronic hardware, (convenience For this purpose, it will be understood that it may be implemented by various forms of program or design code (referred to herein as "software") or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. A person skilled in the art of the present disclosure may implement the described functionality in various ways for each specific application, but such implementation decisions should not be interpreted as a departure from the scope of the present disclosure.

The various embodiments presented herein may be implemented as methods, apparatus, or articles of manufacture using standard programming and/or engineering techniques. The term "article of manufacture" includes a computer program or media accessible from any computer-readable device. For example, computer-readable media include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical disks (eg, CDs, DVDs, etc.), smart cards, and flash memory. devices (eg, EEPROMs, cards, sticks, key drives, etc.). Also, various storage media presented herein include one or more devices and/or other machine-readable media for storing information.

It is understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. Based on design priorities, it is understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure. The appended method claims present elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art of the present disclosure. The generic principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the embodiments presented herein, but is to be construed in the widest scope consistent with the principles and novel features presented herein.

Claims (12)

A model training method comprising:
calculating an input image including at least one rotated object using a neural network model;
determining a first reference point and a second reference point by comparing a prediction box output from the neural network model and a ground truth box;
calculating the accuracy of the prediction box based on the first reference point and the second reference point; and
training the neural network model by performing backpropagation on the neural network model based on the accuracy;
comprising,
How to train a model.
The method of claim 1,
Comparing a prediction box output from the neural network model and a ground truth box, and calculating the accuracy of the prediction box,
calculating accuracy of the prediction box based on an area of a sum area and an intersection area of the prediction box and the actual measurement box;
comprising,
How to train a model.
The method of claim 1,
The determining action is
determining an intersection of the prediction box and the actual measurement box as a first reference point; and
comparing a vertex of the prediction box with an edge of the measured box, and determining the vertex as a second reference point when the vertex is located inside the measured box;
comprising,
How to train a model.
The method of claim 1,
The operation of calculating the accuracy of the prediction box is,
calculating an area of an intersection area between the prediction box and the actual measurement box based on the first reference point and the second reference point;
comprising,
How to train a model.
The method of claim 1,
Comparing a prediction box output from the neural network model and a ground truth box, and calculating the accuracy of the prediction box,
an operation of calculating accuracy by a number of times corresponding to the number of actual measurement boxes when two or more actual measurement boxes exist;
comprising,
How to train a model.
The method of claim 1,
Comparing a prediction box output from the neural network model and a ground truth box, and calculating the accuracy of the prediction box,
calculating accuracy of the two or more prediction boxes by comparing each of the two or more prediction boxes output from the neural network model and the two or more actual measurement boxes;
comprising,
How to train a model.
The method of claim 1,
Comparing a prediction box output from the neural network model and a ground truth box, and calculating the accuracy of the prediction box,
Determining one prediction box corresponding to each of the two or more actual measurement boxes based on the actual measurement box;
comprising,
How to train a model.
8. The method of claim 7,
The operation of determining one prediction box corresponding to each of two or more actual measurement boxes on the basis of the actual measurement box,
determining a grid including each of the two or more actual measurement boxes;
matching each of one prediction box and one actual measurement box based on the grid; and
calculating the accuracy of the matched prediction box and the actual measurement box;
comprising,
How to train a model.
7. The method of claim 6,
Comparing each of the two or more prediction boxes output from the neural network model and the two or more actual measurement boxes, the operation of calculating the accuracy of the two or more prediction boxes comprises:
extracting one accuracy value that is a basis of the backpropagation based on the accuracy of the two or more prediction boxes;
comprising,
How to train a model.
A computer program stored in a computer readable storage medium, wherein the computer program, when executed on one or more processors, performs the following operations for training a neural network model, the operation comprising:
calculating an input image including at least one rotated object using a neural network model;
determining a first reference point and a second reference point by comparing a prediction box output from the neural network model and a ground truth box;
calculating the accuracy of the prediction box based on the first reference point and the second reference point; and
training the neural network model by performing backpropagation on the neural network model based on the accuracy;
comprising,
A computer program stored on a computer readable storage medium.
As a server,
a processor including one or more cores;
network department; and
Memory;
including,
The processor is
Calculate an input image including at least one rotated object using a neural network model,
determining a first reference point and a second reference point by comparing a prediction box output from the neural network model with a ground truth box,
calculating the accuracy of the prediction box based on the first reference point and the second reference point, and
Training the neural network model by performing backpropagation on the neural network model based on the accuracy,
server.
delete

Images