KR101803471B1 - Deep learning system and learning method using of convolutional neural network based image patterning - Google Patents

Deep learning system and learning method using of convolutional neural network based image patterning Download PDF

Info

Publication number
KR101803471B1
KR101803471B1 KR1020160017501A KR20160017501A KR101803471B1 KR 101803471 B1 KR101803471 B1 KR 101803471B1 KR 1020160017501 A KR1020160017501 A KR 1020160017501A KR 20160017501 A KR20160017501 A KR 20160017501A KR 101803471 B1 KR101803471 B1 KR 101803471B1
Authority
KR
South Korea
Prior art keywords
image
pattern
unit
input
learning
Prior art date
Application number
KR1020160017501A
Other languages
Korean (ko)
Other versions
KR20170096298A (en
Inventor
변진영
이상준
Van Hau Nguyen
전재욱
Original Assignee
성균관대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 성균관대학교 산학협력단 filed Critical 성균관대학교 산학협력단
Priority to KR1020160017501A priority Critical patent/KR101803471B1/en
Publication of KR20170096298A publication Critical patent/KR20170096298A/en
Application granted granted Critical
Publication of KR101803471B1 publication Critical patent/KR101803471B1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06F15/18—
    • G06F17/30598—
    • G06F17/30705—
    • G06F17/30784—
    • G06N99/005—

Abstract

The present invention relates to a deep learning system using image patterning based on a convolutional neural network and an image learning method using the same, which includes an image input unit for inputting an input image; A patterning module for generating an input image received from the image input unit as a plurality of patterned pattern images; A CNN learning unit based on a convolution neural network (CNN) that learns an input image received from an image input unit and a pattern image received from a patterning module; A CNN execution unit for receiving learning information from the CNN learning unit and an input image received from the image input unit; And a final classifier for receiving image information from the CNN executing unit and classifying objects of the image information according to types.
The present invention provides an image learning apparatus capable of enhancing the quality of image learning information that is vulnerable to various environmental problems (shaking, illuminance, noise, degradation of recognition rate, etc.) and a deep learning system using the same.

Figure 112016014996819-pat00005

Description

TECHNICAL FIELD [0001] The present invention relates to a deep learning system using image patterning based on convolutional neural networks, and an image learning method using the same. [0002]

More particularly, the present invention relates to a depth learning system based on a convolutional neural network (CNN) using not only RGB values but also patterned image information similar to a human brain, and a deep learning system using the deep learning system To an image learning method.

CNN (Convolutional Neural Network) mimics the human brain structure, and the CRNN (Convolutional Recursive Neural Network) is modeled as a cluster of single CNNs, just as the human brain consists of neurons that are the smallest units.

Conventionally, CNN apparatuses have been made using only RGB input information, and there is no process of extracting additional feature information when only RGB images are used. Therefore, there is an environmental problem that is vulnerable to image rotation, illumination change, and noise. There was a problem.

1 is a diagram illustrating a general deep learning system structure based on a convolutional neural network (CNN). As shown in FIG. 1, an input image for learning is input from the image input unit 110, and the learning image is directly transmitted to the learning unit CNN 112. The learning information 114 obtained through the learning unit CNN 112 and the new input image through the image input unit 110 are transmitted to the execution unit CNN 120. The final classification unit 130 finally determines the type of the object Classify.

2 is a schematic diagram of a general convolution neural network (CNN) structure used for deep running. FIG. 2 is a schematic diagram of a portion corresponding to the learning unit CNN 112 and the execution unit CNN 120 in FIG. 1, and CNNs are composed of multiple layers intertwined like a human neuron. That is, a single CNN is used multiple times to form a general CNN structure used for deep running.

FIG. 3 is a schematic diagram of a general single convolution neural network (CNN) structure, and FIG. 4 is a diagram illustrating an input / output structure of a single convolution neural network based deep learning system.

3 and 4, the input information 410 corresponds to an input 310 of a general single CNN structure 300 and the output information 420 corresponds to an output of a general single CNN structure 300, (312). In the prior art convolution neural network based deep learning system, a weight 412 obtained through learning is applied to input information 410, and these values are summed to obtain an output 420.

In other words, as illustrated in FIGS. 1 to 4, when using only an RGB image in a conventional deep convolution neural network-based deep drawing system, when an image input system is newly installed or when an input image is shaken, There may arise a problem that the illuminance change may occur when the illuminance is used in an environment where the illuminance variation is large such as outdoors, and the recognition rate may decrease when the characteristic information is insufficient. Also, environmental problems can be extracted through the layers of CRNN, which can deepen the problem.

Korean Patent Publication No. 2000-0023915 (published date: May 06, 2000) Korean Registered Patent No. 10-1114135 (Registered Date: February 01, 2012)

The deep learning system using the image patterning based on the convolutional neural network according to the present invention and the image learning method using the deep learning system have the following problems.

First, the present invention is to provide a deep learning system using image pattern information based on convolutional neural network (CNN) capable of improving the quality of image learning information that is vulnerable to various environmental problems (shaking, illuminance, noise, to be.

Second, the present invention provides a method and apparatus for inputting images through various routes and patterning and combining input images to generate various features and a large amount of image data, thereby obtaining a more accurate and high- A learning system and a learning method therefor.

The present invention has been made in view of the above problems, and it is an object of the present invention to provide an apparatus and method for controlling the same.

According to a first aspect of the present invention, there is provided an image processing apparatus including an image input unit for inputting an input image; A patterning module for generating an input image received from the image input unit as a plurality of patterned pattern images; A CNN learning unit based on a convolution neural network (CNN) that learns an input image received from an image input unit and a pattern image received from a patterning module; A CNN execution unit for receiving learning information from the CNN learning unit and an input image received from the image input unit; And a final classifier for receiving image information from the CNN executing unit and classifying objects of the image information according to types.

Preferably, the image input unit is a device connected to the camera to receive a direct input image, or to receive the input image from the wireless network or the Internet network, and the patterning module is configured to input the image received from the image input unit to a plurality of patterns A pattern classifying unit classifying the image into an image; And a pattern combining unit for receiving and combining at least one pattern image from the pattern classifying unit to generate a combined image.

In addition, it is preferable that the patterning module classify the image received from the image input unit into a plurality of corresponding pattern images so as to minimize the influence on each environmental condition, and the pattern image may be classified into a local binary pattern, A local ternary pattern image, a local differential pattern image, and a local tetra pattern image.

The CNN learning unit may include a feature extraction module for extracting features of the image received from the patterning module and the image input unit; And a feature combining unit for generating learning information by combining the extracted features with at least one or more features. The patterning module generates an input image received from the image input unit as a plurality of patterned pattern images, It is preferable to transmit the pattern image to the execution unit.

The apparatus may further include a learning information storage unit that stores learning information learned by the learning unit and transmits the stored learning information to the execution unit, and the learning information is weight information of the pattern image.

The execution unit may include: an input image from the image input unit; Input image data of pattern image through patterning module; And the learning information generated from the learning unit is received. Preferably, the convolutional neural network is a single neural network structure including a plurality of input layers and an output layer. The convolutional neural network includes a plurality of input layers, It is desirable to have multiple neural network structures composed of multiple output layers.

According to a second aspect of the present invention, there is provided a depth learning system comprising: (a) inputting an input image by a video input unit; (b) generating an input image received from the image input unit as a plurality of patterned pattern images by the patterning module; (c) learning the input image received from the CNN learning unit image input unit and the pattern image received from the patterning module based on a Convolution Neural Network (CNN); (d) the CNN execution unit receives the learning information from the CNN learning unit and the input image received from the image input unit; And (e) the final classification unit receives the image information from the CNN execution unit and classifies the object of the image information by type.

In the step (c), the feature extraction module extracts each feature of the image received from the patterning module and the image input unit; And generating the learning information by combining at least one or more features extracted by the combining unit, wherein the patterning module includes a plurality of matching units for minimizing the influence on each environmental condition in the image received from the image input unit, It is preferable to classify them into pattern images.

In addition, preferably, the pattern image includes at least one of a local binary pattern, a local ternary pattern, a local derivative pattern image, and a local tetra pattern image And may include at least one.

In the step (d), the input image and the input image from the image input unit may receive data of the pattern image through the patterning module and the learning information generated from the learning unit.

A third aspect of the present invention features a computer program stored in a medium for executing an image learning method using convolutional neural network based image patterning according to claim 13 in combination with hardware.

The deep learning system using the image patterning based on the convolutional neural network according to the present invention and the image learning method using the deep learning system have the following effects.

First, the present invention provides a convolutional neural network (CNN) capable of enhancing the quality of image learning information that is vulnerable to various environmental problems (shaking, illuminance, noise, recognition rate degradation, etc.) by patterning the input image in a pattern robust to each environmental condition. The present invention also provides a deep learning system using the image pattern information based on the image pattern information and a learning method therefor.

Second, the present invention provides a convolutional neural network (CNN) capable of inputting images through various routes and patterning input images to generate various features and a large amount of image data to acquire more accurate and high- The present invention also provides a deep learning system using the image pattern information based on the image pattern information and a learning method therefor.

Third, the present invention combines at least two patterns of n pattern images to generate a plurality of combined pattern image data, and transmits the pattern image data to the CNN synthesis unit to generate pattern image data combined with patterns robust to environmental conditions, It becomes possible to acquire more selective and rich data as data.

Fourth, since it is possible to correct an error that may occur in extracting features from each layer of the convolution neural network (CNN) and to apply different patterns to each layer layer, (CNN) based image patterning and a learning method therefor.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

1 is a diagram illustrating a general deep learning system structure based on a convolutional neural network (CNN).
2 is a schematic diagram of a general convolution neural network (CNN) structure used for deep running.
3 is a schematic diagram of a general single convolutional neural network (CNN) structure.
4 is a diagram showing an input / output structure of a single convolution neural network (CNN) -based deep learning system.
5 is a block diagram of a deep learning system using image patterning based on convolutional neural networks according to an embodiment of the present invention.
6 is a block diagram illustrating a patterning module used in a deep learning system using convolutional neural network-based image patterning according to an embodiment of the present invention.
FIG. 7 is a pattern diagram of a local binary pattern (LBP) process as a patterning technique used in a patterning module according to an embodiment of the present invention.
FIG. 8 is a pattern diagram of a local ternary pattern (LTP) process as a patterning technique used in the patterning module according to an embodiment of the present invention.
FIG. 9 is a pattern diagram of local diffractive pattern (LDP) processing as a patterning technique used in the patterning module according to the embodiment of the present invention.
10 is a schematic diagram of a single CNN structure applied to a deep learning system using image patterning according to an embodiment of the present invention.
11 is a schematic diagram of a single CNN structure applied to a deep running system according to an embodiment of the present invention.
12 is a schematic diagram of a multiple CNN structure applied to a deep learning system according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Wherever possible, the same or similar parts are denoted using the same reference numerals in the drawings.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms as used herein include plural forms as long as the phrases do not expressly express the opposite meaning thereto.

Means that a particular feature, region, integer, step, operation, element and / or component is specified and that other specific features, regions, integers, steps, operations, elements, components, and / It does not exclude the existence or addition of a group.

All terms including technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Predefined terms are further interpreted as having a meaning consistent with the relevant technical literature and the present disclosure, and are not to be construed as ideal or very formal meanings unless defined otherwise.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

5 is a block diagram of a deep learning system using image patterning based on convolutional neural networks according to an embodiment of the present invention. As shown in FIG. 5, the deep learning system using the convolutional neural network based image patterning according to the embodiment of the present invention includes an image input unit 412 for inputting an input image; A patterning module 414 for generating an input image received from the image input unit 412 as a plurality of patterned pattern images; A CNN learning unit 416 based on a convolutional neural network (CNN) that learns an input image received from the image input unit 412 and a pattern image received from the patterning module 414; A CNN executing unit 430 receiving learning information from the CNN learning unit 416 and an input image received from the image input unit 412; And a final classifier 440 for receiving image information from the CNN executing unit 430 and classifying objects of the image information according to classes.

As another embodiment of the present invention, an image learning method using image patterning based on a convolutional neural network uses a deep learning system of FIG. 5, wherein (a) an image input unit 412 inputs an input image ; (b) generating an input image received from the image input unit 412 as a plurality of patterned pattern images by the patterning module 414; (c) the CNN learning unit 416 learns an input image received from the image input unit 412 and a pattern image received from the patterning module 414 based on a convolutional neural network (CNN); (d) receiving the learning information from the CNN learning unit 416 and the input image received from the image input unit 412; And (e) the final classification unit 440 receives the image information from the CNN execution unit 430 and classifies objects of the image information according to the types.

5, the input information obtained through the image input unit 412 and the patterning module 414 in the deep learning system and the image learning method using the deep learning system according to the embodiment of the present invention are structured as a CNN structure And the CNN learning unit 416 and the CNN executing unit 430 correspond to an output unit for outputting pattern images and weight information. That is, a value obtained by applying a weight obtained by learning to input information for each input information and a value obtained by applying a pattern weight to pattern information obtained by patterning the relationship between input information are obtained and added. Finally, the sum of the input values is finally added to obtain the output information.

Here, Deep Learning technology is an artificial intelligence (AI) technology that enables a computer to think and learn like a human being, and enables a machine to learn and solve complex nonlinear problems based on artificial neural network theory. Deep learning is a technology that allows a human brain to learn patterns in a lot of data and then learn the machine to imitate the information processing method of distinguishing objects so that the computer can distinguish objects.

Deep learning technology enables computers to recognize, deduce, and judge themselves without having to set all criteria, and it can be widely used for voice, image recognition, and photo analysis.

In other words, deep learning is a combination of several nonlinear transformation techniques that use a high level of abstraction (machine learning) that attempts to summarize key content or functions in large amounts of data or complex data learning algorithms, and can be said to be a field of machine learning that teaches computers how people think in a large framework.

When there is any data, it is represented by a form that can be understood by the computer (for example, pixel information is expressed as a column vector in the case of images), and many studies As a result of these efforts, various deep learning techniques such as deep neural networks, convolutional deep neural networks, and deep belief networks have been developed for computer vision, speech recognition, Natural language processing, voice / signal processing, and so on.

In particular, the importance of deep learning technology in the field of image recognition and object recognition, which are the main fields of computer vision, is increasing. As with the expansion of automatic speech translation and understanding field of automatic speech recognition field, It is expanding to the more challenging field of captioning. Auto image captioning is a field where deep learning is used as a core technology. Examples of applications include a car-mounted computer learned through deep learning to understand a 360 ° camera screen.

Convolutional neural networks (CNN) have been mainly used for recognition of 2D images and have been used for recognition problems. The first is the local receptive field, The second is to share the weights representing these features in the entire image area. In this way, it is possible to reduce the number of parameters and to share the feature regardless of the position of the image. .

The third feature is that the process of creating a higher layer by stacking one layer and reducing the number of nodes becomes more generalized as it goes to the upper layer. Recently deep research has been attracting much attention, and CNN has been deep structured like convolutional deep beliefnetworks (deep CNN) with convoluted RBM (restricted Boltzmann machine) layer. It is showing.

As described above, in order to solve the problem that the quality of the learning information due to various environmental problems is low, the embodiment of the present invention classifies and combines the input image into a strong pattern image in each environment, It is possible to improve the quality of the image learning information that is vulnerable to the shake, the illuminance, the noise, the degradation of the recognition rate, etc., as well as correct errors that may occur in the process of extracting features from each layer of the convolution neural network (CNN) (CNN) -based image patterning that can acquire better learning information in that different patterns can be applied to each layer layer, thereby providing a deep learning system using image patterning based on convolutional neural network (CNN).

The image input unit 412 illustrated in FIG. 5 is a device that is connected to the camera and receives a direct input image, receives the input image from the wireless network or the Internet network, receives the image directly through the camera, Or receive images through a mobile device or a PC via a wireless network or an Internet network. By receiving images through various routes as described above, it is possible to acquire more accurate and high-level video learning data through the learning apparatus according to the embodiment of the present invention by performing various functions and functions of a large amount of video data as big data .

6 is a block diagram illustrating a patterning module 414 used in a deep learning system using convolutional neural network-based image patterning according to an embodiment of the present invention. 6, the patterning module 414 includes a pattern classifying unit 414a for classifying an image received from the image input unit 412 into a plurality of pattern images; And a pattern combining unit 414b for receiving and combining at least one pattern image from the pattern classifying unit 414a to generate a combined image.

Here, the pattern classifying unit 414a classifies the inputted image into pattern images and classifies them into a plurality of corresponding pattern images so as to minimize the influence on each environmental condition in the image received from the image input unit 412 . That is, an image learning apparatus capable of improving the quality of image learning information that is vulnerable to various environmental problems (shaking, illuminance, noise, degradation of recognition rate, etc.) by classifying input images into pattern images robust to respective environments, .

The pattern combining unit 414b combines pattern images patterned from the pattern classifying unit 414a with each other to generate patterned combined image data. 6, when pattern images of a plurality of patterns are combined by combining at least two patterns of n pattern images and then transmitted to the CNN learning unit 416, It is possible to acquire more selective and rich data as the learning data by generating the pattern image data combined with the patterns robust to the environmental conditions.

That is, the patterning module 414 applied to the embodiment of the present invention generates n patterned images using the same input image, and uses them in combination. For example, when there is an environmental problem in which the angle of the camera can be changed and an image is taken outdoors, the input image received from the image input unit 412 is received by the patterning module 414 and rotated inside the patterning module 414 And transmits it to the CNN learning unit 416, so that more selective data acquisition and excellent learning data can be generated.

The pattern image in which the input image is patterned and classified by the patterning module 414 applied to the embodiment of the present invention includes a local binary pattern image, a local ternary pattern image, It is possible to acquire a robust pattern image corresponding to a specific environmental condition by using an image pattern processing technique such as a local differential pattern image and a local Tetra Pattern image.

FIG. 7 is a pattern diagram of a local binary pattern (LBP) process used in the patterning module 414 according to an embodiment of the present invention. As shown in FIG. 7, when the input image is shaken, an angle difference of the input image occurs. In order to form a pattern robust to such an environmental condition, the patterning module 414 generates a local binary pattern image To solve this problem.

Here, as shown in FIG. 7, the Local Binary Pattern refers to a pattern in which the value of the center pixel of the window is compared with the value of the surrounding pixel and is represented by 0 and 1. That is, a value larger than the center pixel value is represented by 1, a smaller value is represented by 0, and a number of 1 is obtained. When the number of images is compared only by this number, image data can be compared regardless of the direction, It is possible to acquire a pattern image that is robust to environmental conditions in which a difference occurs.

FIG. 8 is a pattern diagram of a local ternary pattern (LTP) process as a patterning technique used in the patterning module according to an embodiment of the present invention. As shown in FIG. 8, when using in an environment with a large variation in illumination, such as outdoor, a problem arises due to variation in illumination. In order to form a pattern resistant to such an environmental condition, Local Ternary Pattern This pattern is solved by image.

Local Ternary Pattern refers to a pattern that adds a threshold value (k) to a local binary pattern (LBP) and is denoted by -1, 0, 1. It is possible to obtain a pattern image which is robust to the environmental conditions of the change in illumination in that the size is twice as large as that of the local binary pattern (LBP) but is proved to be robust against the variation in illumination.

FIG. 9 is a pattern diagram of local diffractive pattern (LDP) processing as a patterning technique used in the patterning module according to the embodiment of the present invention. As shown in FIG. 9, when the feature information is insufficient, a problem occurs in that the recognition rate is low. In order to form a pattern robust against such environmental conditions, the patterning module 414 generates a local differential pattern, Patterns by image are solved.

As shown in FIG. 9, the Local Derivative Pattern can provide more specific information in a pattern using derivative direction variations, so that it is robust against environmental conditions in which characteristic information is insufficient The pattern image can be obtained.

In addition, as another embodiment of the present invention, a local Tetra Pattern image processing technique may be used as the patterning technique used in the patterning module 414. The local Tetra Pattern image processing technique combines the features of the local binary pattern (LBP), the local strikeout pattern (LTP) and the local differential pattern (LDP), and the relationship between the center pixel value and the surrounding pixel values Is a four-directional pattern. This makes it possible to minimize the influence on the input image which is vulnerable to various environmental conditions in that it can obtain the specified information, is robust to angles and illuminance, and has more feature information.

The CNN learning unit 416 used in the deep learning system according to the embodiment of the present invention includes a feature extraction module 416a for extracting features of an image received from the patterning module 414 and the image input unit 412, And a feature combination unit 416b that combines the extracted features into at least one or more features to generate learning information.

That is, the CNN learning unit 416 receives the input image and patterned pattern image data through the patterning module 414, extracts various features based on the convolution neural network structure, And transmits the generated learning information to the CNN executing unit 430. [

For example, assuming that 100 pieces of data are input to the CNN learning unit 416 and 25 pieces are compressed through a feature extraction process using the feature extraction module 416a, 50 pieces of data are sent to the outside of the CNN learning unit 416 to acquire learning information applied as a weight.

As shown in FIG. 5, the deep learning system using image patterning based on the convolutional neural network according to an embodiment of the present invention includes a CNN learning unit 416, a CNN learning unit 416, (Weight value) corresponding to the learning information (weight value) corresponding to the learning information (weight value) corresponding to the learning information (weight) in the CNN executing unit 430. The CNN executing unit 430, The pattern image obtained through the patterning module 420 is multiplied by the weight and then transmitted to the final classifier 440. The final classifier 440 performs an algorithm such as soft-max classification To classify objects.

10 is a schematic diagram of a single CNN structure applied to a deep learning system using image patterning according to an embodiment of the present invention.

As shown in FIG. 10, the input information 510 corresponds to an input unit of a general single CNN structure, and the output 530 corresponds to an output unit of a general single CNN structure. A value obtained by applying the weight 512 obtained through learning to the input information 510 and a value obtained by applying the pattern weight 522 to the pattern information 520 obtained by patterning the relationship of the input information 510 are obtained for each input information 510, Sum. Finally, the sum of the input values is finally summed to obtain the output 530 information.

FIG. 11 is a schematic view of a single CNN structure applied to a deep learning system according to an embodiment of the present invention, and FIG. 12 is a schematic diagram of a multiple CNN structure applied to a deep learning system according to an embodiment of the present invention.

As shown in FIGS. 11 and 12, a deep-running system according to an embodiment of the present invention can be applied to a single CNN structure including a plurality of input layers and an output layer. In the conventional CNN structure, the input information is simply multiplied by a weight (parameter value obtained through a training process), and the result is multiplied. In the single or multi-CNN structure proposed in the embodiment of the present invention, In addition, there is a difference in that it constitutes a convolutional neural network.

In the embodiment of the present invention, in order to use the relation between the input information in the apparatus, it is preferable to use the relationship between the input information (for example, in the case of LBP, generally, the relationship between the eight input values and their values, Pattern information can be used as an input.

The embodiments and the accompanying drawings described in the present specification are merely illustrative of some of the technical ideas included in the present invention. Accordingly, the embodiments disclosed herein are for the purpose of describing rather than limiting the technical spirit of the present invention, and it is apparent that the scope of the technical idea of the present invention is not limited by these embodiments. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

412: image input unit, 414: patterning module
416: CNN learning section 430: CNN execution section
440: Final classification section

Claims (18)

A video input unit for inputting an input video;
A first and a second patterning module for respectively generating an input image received from the image input unit as a plurality of pattern images;
A CNN learning unit based on a Convolution Neural Network (CNN) that learns an input image received from the image input unit and a pattern image received from the first patterning module to generate learning information;
A CNN execution unit which receives learning information from the CNN learning unit, an input image received from the image input unit, and a pattern image from the second patterning module, multiplies the input image and the pattern image by the weight of the learning information, ; And
And a final classifier for receiving the image information from the CNN executing unit and classifying the objects of the image information according to the type of the object. The deep learning system using convolutional neural network based image patterning,
The CNN learning unit,
A feature extraction module for extracting features of an image received from the second patterning module and the image input unit; And
And a feature combination unit for generating learning information by combining at least one feature extracted from the feature extraction module,
Further comprising a learning information storage unit for storing learning information learned by the CNN learning unit and transmitting the stored learning information to the CNN execution unit,
Wherein the learning information is weight information of a pattern image,
Wherein each of the first and second patterning modules comprises: a pattern classifying unit for classifying an image received from the image input unit into a plurality of pattern images; And a pattern combiner for receiving and combining at least one pattern image from the pattern classifier to generate a combined image. The method according to claim 1,
Wherein the image input unit comprises:
It is connected to the camera to receive a direct input image,
Wherein the apparatus is a device capable of receiving and receiving input from a wireless network or an internet network. delete The method according to claim 1,
Wherein each of the first and second patterning modules comprises:
Wherein the plurality of pattern images are classified into a plurality of corresponding pattern images so as to minimize the influence on each environmental condition in the image received from the image input unit. The method according to claim 1,
In the pattern image,
A local differential pattern image, a local binary pattern image, a local binary pattern image, a local ternary pattern image, a local differential pattern image, and a local tetra pattern image. Deep Learning System using Image Patterning Based on Convolution Neural Network. delete The method according to claim 1,
Wherein the patterning module comprises:
Wherein the input image received from the image input unit is generated as a plurality of patterned pattern images, and the generated pattern image is transmitted to the CNN executing unit. delete delete delete The method according to claim 1,
Wherein the convolutional neural network is a single neural network structure composed of a plurality of input layers and an output layer, and the deep learning system using convolutional neural network based image patterning. The method according to claim 1,
Wherein the convolution neural network is a multiple neural network structure composed of a plurality of input layers and a plurality of output layers. By using the deep learning system of claim 1,
(a) inputting the input image by the image input unit;
(b) generating a plurality of pattern images of the input image received from the image input unit by the first patterning module;
(c) the CNN learning unit learns an input image received from the image input unit and a pattern image received from the first patterning module based on a Convolution Neural Network (CNN);
(d) The CNN execution unit receives the learning information from the CNN learning unit, the input image received from the image input unit, and the pattern image from the second patterning module, multiplies the input image and the pattern image by the weight of the learning information, ; And
(e) a final classification unit receives image information from the CNN execution unit and classifies the objects of the image information according to types, and the image classification method using the image patterning based on the convolutional neural network,
The step (c)
Extracting features of an image received from the first patterning module and the image input unit; And
And generating learning information by combining the extracted features with at least one or more features,
Wherein the learning information is weight information of a pattern image,
The step (d)
Wherein the input image and the input image from the image input unit are transmitted through the second patterning module to receive data of the pattern image and the learning information generated from the CNN learning unit. Using image learning method. delete 14. The method of claim 13,
Wherein the patterning module comprises:
Wherein the image is classified into a plurality of corresponding pattern images so as to minimize the influence on each environmental condition in the image received from the image input unit. 14. The method of claim 13,
In the pattern image,
A local differential pattern image, a local binary pattern image, a local binary pattern image, a local ternary pattern image, a local differential pattern image, and a local tetra pattern image. Image Learning Method Using Image Patterning Based on Convolution Neural Network. delete 13. A computer program stored in a medium for executing an image learning method using convolutional neural network based image patterning of claim 13 in combination with hardware.
KR1020160017501A 2016-02-15 2016-02-15 Deep learning system and learning method using of convolutional neural network based image patterning KR101803471B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020160017501A KR101803471B1 (en) 2016-02-15 2016-02-15 Deep learning system and learning method using of convolutional neural network based image patterning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020160017501A KR101803471B1 (en) 2016-02-15 2016-02-15 Deep learning system and learning method using of convolutional neural network based image patterning

Publications (2)

Publication Number Publication Date
KR20170096298A KR20170096298A (en) 2017-08-24
KR101803471B1 true KR101803471B1 (en) 2017-12-01

Family

ID=59758043

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020160017501A KR101803471B1 (en) 2016-02-15 2016-02-15 Deep learning system and learning method using of convolutional neural network based image patterning

Country Status (1)

Country Link
KR (1) KR101803471B1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20200073078A (en) 2018-12-13 2020-06-23 주식회사 픽스트리 Image processing device of learning parameter based on machine Learning and method of the same
KR20200073079A (en) 2018-12-13 2020-06-23 주식회사 픽스트리 Image processing device of learning parameter based on machine Learning and method of the same
US10825185B2 (en) 2018-05-14 2020-11-03 Electronics And Telecommunications Research Institute Method and apparatus for performing segmentation of small objects within moving pictures in real-time

Families Citing this family (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102188521B1 (en) * 2017-10-27 2020-12-08 전북대학교산학협력단 Method and Apparatus for Identification of Crops and Weeds with Neural Network Learning
KR102073406B1 (en) * 2017-10-30 2020-02-05 동국대학교 산학협력단 Method and system for detect electric arc occurrence position
KR102578826B1 (en) * 2017-11-08 2023-09-15 삼성전자주식회사 Neural network device and operation method of the same
KR101932008B1 (en) * 2017-12-29 2018-12-24 (주)제이엘케이인스펙션 Image analysis apparatus and method based on feature and context of image
KR102050780B1 (en) * 2018-01-10 2019-12-02 한국과학기술원 Method and Server Apparatus for Delivering Content Based on Content-aware Using Neural Network
KR102121462B1 (en) * 2018-01-25 2020-06-10 안준범 A device that divides the body skin into the type of skin required for the procedure through image analysis using convolution neural network
KR102155154B1 (en) * 2018-02-01 2020-09-11 주식회사 스칼라웍스 Method for taking artistic photograph using drone and drone having function thereof
KR102210137B1 (en) * 2018-03-30 2021-02-02 한국전력공사 Apparatus and method for discrimiating high temperature equipment degradation level using artificial intelligence
KR102190483B1 (en) * 2018-04-24 2020-12-11 주식회사 지디에프랩 System for compressing and restoring picture based on AI
KR102108953B1 (en) * 2018-05-16 2020-05-11 한양대학교 산학협력단 Robust camera and lidar sensor fusion method and system
KR102108951B1 (en) * 2018-05-16 2020-05-11 한양대학교 산학협력단 Deep learning-based object detection method and system utilizing global context feature of image
JP2020524418A (en) 2018-05-21 2020-08-13 ジーディーエフラボ カンパニー リミテッド VOD service system based on AI video learning platform
KR101997909B1 (en) * 2018-05-21 2019-07-08 주식회사 지디에프랩 Program and recording medium for extracting ai image learning parameters for resolution restoration
KR102557572B1 (en) 2018-05-23 2023-07-24 한국전자통신연구원 Artificial neural network device and operating method for the same
KR102122893B1 (en) * 2018-06-18 2020-06-15 주식회사 바탕색이엔지 System and method for autonomous crack evaluation of structure based on uav mounted-hybrid image scanning
KR102077396B1 (en) * 2018-07-16 2020-02-13 재단법인 다차원 스마트 아이티 융합시스템 연구단 Method for generating license plate image using noise pattern and apparatus therefor
KR101926181B1 (en) * 2018-08-01 2018-12-06 (주)한그린테크 Apparatus and method for measuring freshness based on machine-learning
KR102180569B1 (en) 2018-08-08 2020-11-18 부산대학교 산학협력단 Apparatus for Enhancing Contrast and Resolution in a Grating Interferometer by Machine Learning
KR102100304B1 (en) * 2018-08-30 2020-05-18 충북대학교 산학협력단 Method for identifying snake using image patternize
KR102092475B1 (en) * 2018-10-16 2020-03-23 고려대학교 산학협력단 Method and application for animal species classification
KR20200080391A (en) * 2018-12-18 2020-07-07 주식회사 포스코 System and method for predicting material properties using metal microstructure images based on deep running
KR102053582B1 (en) * 2019-05-28 2019-12-06 (주) 지오아이티 Method of ground coverage classification by using image pattern learning based on deep learning
KR102273149B1 (en) * 2019-05-29 2021-07-07 주식회사 허그케어앤테라퓨틱스 Method for conitive therapy based on artifical intelligence
KR102265525B1 (en) * 2019-05-31 2021-06-16 재단법인 경북아이티융합 산업기술원 Method and system for diagnosing skin based on an artificial intelligence
KR20210037199A (en) 2019-09-27 2021-04-06 한국전력공사 Apparatus for dividing, tagging an image and for detecting defect of facilities using the same
KR102166835B1 (en) 2019-10-28 2020-10-16 주식회사 루닛 Method for Training Neural Network and Device Thereof
KR102365504B1 (en) 2019-12-20 2022-02-21 주식회사 엘지씨엔에스 A method and apparatus for determining purchase attribute of customer
KR102130076B1 (en) * 2020-02-19 2020-07-03 주식회사 지디에프랩 Method for improving the resolution of streaming files based on the learning importance of feature areas
KR20220033150A (en) 2020-09-09 2022-03-16 한국과학기술연구원 유럽연구소 Method and apparatus for non-invasive real-time monitoring bevior patterns of ecological species for ecotoxicology risk assessment
KR102341236B1 (en) * 2021-03-22 2021-12-20 미소에이스 주식회사 Indentation inspection system based on deep lrarning and control method of indentation inspection system
WO2023277319A1 (en) * 2021-07-02 2023-01-05 가천대학교 산학협력단 Artificial intelligence-based classification apparatus and method therefor

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Ahmed, Faisal, et al. "Automated weed classification with local pattern-based texture descriptors." Int. Arab J. Inf. Technol. 11.1, 2014.
Anuradha, Koneru, and Manoj Kumar Tyagi. "A novel method of face recognition using lbp, ltp and gabor features." International Journal of Scientific & Technology Research 1.5, 2012.*
Levi, Gil, and Tal Hassner. "Emotion recognition in the wild via convolutional neural networks and mapped binary patterns." Proceedings of the 2015 ACM on International Conference on ICMI, 2015.*
Zhu, Zhenyao, et al. "Deep learning identity-preserving face space." Proceedings of the IEEE International Conference on Computer Vision. 2013.

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10825185B2 (en) 2018-05-14 2020-11-03 Electronics And Telecommunications Research Institute Method and apparatus for performing segmentation of small objects within moving pictures in real-time
KR20200073078A (en) 2018-12-13 2020-06-23 주식회사 픽스트리 Image processing device of learning parameter based on machine Learning and method of the same
KR20200073079A (en) 2018-12-13 2020-06-23 주식회사 픽스트리 Image processing device of learning parameter based on machine Learning and method of the same

Also Published As

Publication number Publication date
KR20170096298A (en) 2017-08-24

Similar Documents

Publication Publication Date Title
KR101803471B1 (en) Deep learning system and learning method using of convolutional neural network based image patterning
Rao et al. Selfie video based continuous Indian sign language recognition system
Hu et al. A novel image steganography method via deep convolutional generative adversarial networks
Van Den Oord et al. Pixel recurrent neural networks
Do et al. Forensics face detection from GANs using convolutional neural network
Ronneberger et al. U-net: Convolutional networks for biomedical image segmentation
Liu et al. Multi-objective convolutional learning for face labeling
Lerch-Hostalot et al. Unsupervised steganalysis based on artificial training sets
WO2021129181A1 (en) Portrait segmentation method, model training method and electronic device
CN113674140B (en) Physical countermeasure sample generation method and system
CN107590432A (en) A kind of gesture identification method based on circulating three-dimensional convolutional neural networks
CN106599863A (en) Deep face identification method based on transfer learning technology
KR20180004898A (en) Image processing technology and method based on deep learning
US20180232606A1 (en) Sensory information providing apparatus, video analysis engine, and method thereof
CN110929099B (en) Short video frame semantic extraction method and system based on multi-task learning
US11544571B2 (en) Optimizing unsupervised generative adversarial networks via latent space regularizations
CN109711411B (en) Image segmentation and identification method based on capsule neurons
Chung et al. Traffic sign recognition in harsh environment using attention based convolutional pooling neural network
CN111833360B (en) Image processing method, device, equipment and computer readable storage medium
Rafique et al. Deep fake detection and classification using error-level analysis and deep learning
Alam et al. Two dimensional convolutional neural network approach for real-time bangla sign language characters recognition and translation
US20210227223A1 (en) System and methods for artificial intelligence explainability via symbolic generative modeling
CN113657272B (en) Micro video classification method and system based on missing data completion
CN112990154B (en) Data processing method, computer equipment and readable storage medium
Wang et al. Face hallucination using multisource references and cross‐scale dual residual fusion mechanism

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
AMND Amendment
E601 Decision to refuse application
AMND Amendment
X701 Decision to grant (after re-examination)
GRNT Written decision to grant