KR101896357B1 - Method, device and program for detecting an object - Google Patents

Abstract

The present invention provides a method, a device and a program for detecting an object. The method for detecting an object by the device comprises the following steps of: obtaining an image including a detection target object; segmenting the obtained image into a plurality of cells; obtaining information on at least one region estimated to contain the object for each of the plurality of segmented cells by using a plurality of layers included in a network for detecting the object; obtaining reliability for each of the at least one region obtained; and detecting the object based on the obtained reliability and an overlapped region of the at least one region obtained.

Description

METHOD, DEVICE AND PROGRAM FOR DETECTING AN OBJECT,

The present invention relates to a method, a device and a program for detecting an object.

In general, documents are widely used as scanned images for archiving and management of documents in government offices, financial institutions, and corporations. In particular, since the cost of archiving documents has increased, the management of documents using images has become commonplace in most financial sectors since the late 1990s.

It is the resident registration number which occupies the most part of the personal information to be protected in the document which is kept and managed in this image form.

The resident registration number in the document is variously recorded in the form of a printed matter and a handwritten form, and it is difficult to specify the recorded form recorded in various forms.

Many companies and financial institutions have introduced automatic document detection systems that contain personal information to protect personal information. However, the objects to be detected are confined to electronic documents such as text, word, and Korean alphabet, Documents are not detected and there is a high risk of security incidents.

Patent Document 1 recognizes text in an image by extracting and marking personal information by recognizing text in the image, and extracts and marks personal information based on the recognized result.

In Patent Document 2, the text in the image is recognized and the recognition result is used to determine the probability of the personal information using the probability similarity, and then the personal information is extracted.

Patent Document 3 classifies the types of documents by using template matching of scanned document images and extracts the personal information existing at positions corresponding to the types of the classified documents. That is, personal information is detected through document classification in a document image having a formal format.

In the case of Patent Document 1 and Patent Document 2, there is a problem in that it is inefficient to prevent information leakage which requires fast processing in terms of using character recognition technology, and its performance is limited to a character recognition rate. On the other hand, in the case of Patent Document 3, although quick processing is possible, classification of documents using template matching or the like must precede classification, which limits the application to a document image of an unspecified format.

An object detection method using an ordinary convolution neural network detects an object using a classifier, and after a window of a predetermined size is generated, a window is moved and classified as an entire image. That is, the classifier moves the window of a certain size vertically and horizontally to search the entire image. This has the problem that it is impossible to process quickly.

Patent Registration No. 10-1401028 (Registered on May 22, 2014) Patent Registration No. 10-1721063 (registered on March 23, 2013) Published Japanese Patent Application No. 10-2015-0130253 (published Nov. 23, 2015)

A problem to be solved by the present invention is to provide a method, a device and a program for detecting an object.

The problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems which are not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a method for a device to detect an object, comprising: obtaining an image including a detection object; segmenting the obtained image into a plurality of cells; Obtaining information about one or more regions estimated to contain the object for each of the plurality of divided cells using a plurality of layers included in a network for detecting the object, Obtaining reliability for each of the regions, and detecting the object based on the obtained reliability and the overlapping region of the obtained one or more regions.

The dividing may further include removing noise of the obtained image, dividing the noise-removed image into one or more blocks, and dividing each of the divided one or more blocks into the plurality of cells .

In addition, the information on the at least one area includes a center point coordinate, a width, and a height of the at least one area, and the at least one center point coordinate, width, And normalized based on the width and height of the block.

In addition, the object to be detected may include one or more classes, and the step of acquiring information on the one or more regions may include acquiring information on one or more regions estimated to include the object on the basis of the one or more classes . ≪ / RTI >

The step of detecting the object may further include the steps of acquiring at least one region having a reliability higher than a predetermined threshold for each of the obtained one or more regions and using at least one region having reliability higher than the predetermined threshold, And detecting the object.

The detecting of the object may include calculating a position of the region including the object using the overlap region of at least one region having reliability higher than the predetermined threshold.

The plurality of layers included in the network may include an output layer outputting information on the at least one region and the reliability of the at least one region, A neuron representing the location, size and reliability of each of the above regions, and a neuron representing the class of the object.

In addition, the detection object may include at least one of a resident registration number and other personal information included in the image.

According to an aspect of the present invention, there is provided a device including a memory for storing one or more instructions, a processor for executing the one or more instructions stored in the memory, the processor executing the one or more instructions Thereby dividing the obtained image into a plurality of cells, and using a plurality of layers included in a network for detecting the object, for each of the plurality of divided cells Acquiring information about one or more regions estimated to include the object, obtaining reliability for each of the obtained one or more regions, and based on the obtained reliability and the overlapping region of the obtained one or more regions, And detects the object.

According to an aspect of the present invention, there is provided a computer readable recording medium readable with a computer, which is capable of performing the method of claim 1. [

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention, it is possible to efficiently extract the irregular-shaped personal information included in the image by extracting the resident registration number included in the scanned document image based on the convolution neural network.

Also, by using artificial neural network, it is possible to improve the performance of personal information leakage prevention system by extracting resident registration number robust to unstructured form and format.

Also, in detecting an object based on a convolutional neural network, processing speed can be improved by dividing an image into cells.

In addition, the processing speed can be improved by limiting the area to be detected in the preprocessing process.

The effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

1 is a diagram for explaining a neural network used for detecting an object in a device according to an embodiment.
2 is a flow chart illustrating a method for a device to detect an object according to an embodiment.
3 is a block diagram of a device that detects an object in accordance with one embodiment.
4 is a block diagram of a processor in accordance with one embodiment.
5 is a block diagram of a data learning unit according to an embodiment.
6 is a block diagram illustrating a device according to one embodiment.
FIGS. 7 to 11 are diagrams for explaining a method for a device to perform learning for detecting an object and to detect an object according to the disclosed embodiment. FIG.
12 is a block diagram of a data recognition unit according to an embodiment.
13 is a block diagram illustrating a device according to one embodiment.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. It should be understood, however, that the invention is not limited to the disclosed embodiments, but may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, Is provided to fully convey the scope of the present invention to a technician, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. The terms " comprises "and / or" comprising "used in the specification do not exclude the presence or addition of one or more other elements in addition to the stated element. Like reference numerals refer to like elements throughout the specification and "and / or" include each and every combination of one or more of the elements mentioned. Although "first "," second "and the like are used to describe various components, it is needless to say that these components are not limited by these terms. These terms are used only to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical scope of the present invention.

Unless defined otherwise, all terms (including technical and scientific terms) used herein may be used in a sense that is commonly understood by one of ordinary skill in the art to which this invention belongs. In addition, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

As used herein, the term "part" or "module" refers to a hardware component, such as a software, FPGA, or ASIC, and a "component" or "module" performs certain roles. However, "part" or " module " is not meant to be limited to software or hardware. A "module " or " module " may be configured to reside on an addressable storage medium and configured to play back one or more processors. Thus, by way of example, "a" or " module " is intended to encompass all types of elements, such as software components, object oriented software components, class components and task components, Microcode, circuitry, data, databases, data structures, tables, arrays, and variables, as used herein. Or " modules " may be combined with a smaller number of components and "parts " or " modules " Can be further separated.

The terms spatially relative, "below", "beneath", "lower", "above", "upper" And can be used to easily describe a correlation between an element and other elements. Spatially relative terms should be understood in terms of the directions shown in the drawings, including the different directions of components at the time of use or operation. For example, when inverting an element shown in the figures, an element described as "below" or "beneath" of another element may be placed "above" another element . Thus, the exemplary term "below" can include both downward and upward directions. The components can also be oriented in different directions, so that spatially relative terms can be interpreted according to orientation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining a neural network used for detecting an object in a device according to an embodiment.

In one embodiment, the device may obtain an image that includes an object. For example, a device may receive an image including an object from an external device, or may photograph an object through a photographing device provided in the device to obtain an image including the object.

A device according to an exemplary embodiment may detect an object included in an image using a neural network 100. FIG. Here, the neural network 100 extracts various attribute information of an image using the result of statistical machine learning, and sets an algorithm for detecting, identifying, and / or determining objects in the image based on the extracted attribute information .

The neural network 100 may also be implemented as software or an engine for executing the algorithm set described above. A neural network embodied in software, engine, or the like may be executed by a processor in a device (not shown) or a processor in a server (not shown).

The neural network 100 according to one embodiment can detect objects in an image by abstracting various attributes contained in the image input to the neural network 100. [ In this case, abstracting the attributes in the image may be to detect attribute information from the image and to determine key attributes that can represent the object from the detected attribute information.

In addition, the neural network 100 may include an input layer 110, an output layer 130, and a plurality of layers 122 to 128 therebetween.

The device can extract the attribute information of the image using the neural network 100. [ The attribute information of the image may include color, edge, polygon, saturation, brightness, color temperature, blur, sharpness, contrast, This is only an example, and the attribute information of the image is not limited to the above example.

On the other hand, the device may be a smart phone, a tablet PC, a PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, Kiosks, MP3 players, digital cameras, consumer electronics, and other mobile or non-mobile computing devices. The device may also be a wearable device, such as a watch, eyeglasses, a hair band, and a ring, with communication and data processing capabilities.

2 is a flow chart illustrating a method for a device to detect an object according to an embodiment.

In step S210, the device acquires an image including the object to be detected.

For example, the detection object may mean a resident registration number included in the image, but is not limited thereto.

In one embodiment, the object to be detected includes one or more classes. For example, a class means an object of a different kind. For example, the object to be detected includes a handwritten resident registration number written in handwriting, and a resident registration number in the form of a typed (or printed) print. In addition, the object to be detected may include various kinds of personal information such as fingerprints, names, and addresses, but is not limited thereto.

In one embodiment, the image may comprise a scanned or photographed document image.

In one embodiment, the device removes noise from the acquired image. For example, a device may remove unwanted white space contained in an image, a portion that does not contain information, or an unnecessary pixel included in an image during a scan or photographing.

In one embodiment, the device can match the images. For example, the device may rotate the image or adjust the size of the image to match the image to a predetermined standard.

In one embodiment, the device can remove noise and segment the matched image into one or more blocks.

For example, a device can divide an image into blocks based on the distribution of black pixels contained in the image. Since a large part of the scanned document is blank or meaningless, only the divided blocks are used for object detection to improve the processing speed.

In one embodiment, the device may generate a normalized image using the partitioned block. For example, a device can produce a normalized image with the same horizontal and vertical lengths as a color image with three channels to handle both monochrome and color images.

In step S220, the device divides the image obtained in step S210 into a plurality of cells.

In one embodiment, the device may divide each of the one or more blocks into which the image was divided in step S210 into a plurality of cells.

A specific method of dividing the block into a plurality of cells will be described later.

In step S230, the device obtains information on one or more areas estimated to contain the detection object for each of the plurality of cells divided in step S220 using a plurality of layers included in the network for detecting the object do.

In one embodiment, the information for the one or more regions includes the center point coordinates, width, and height of one or more regions.

In one embodiment, the one or more center point coordinates, width, and height may be normalized information based on the width and height of the block to which each of at least one of the one or more blocks divided in step S210 belongs. For example, one or more center point coordinates, width, and height may be normalized to a value between 0 and 1 based on the width and height of the block.

In one embodiment, the device obtains information about one or more regions that are estimated to include the object of detection by one or more classes.

In addition, the device obtains reliability for each of the one or more regions obtained.

In one embodiment, the device obtains, for each of the one or more regions obtained, one or more regions having a confidence that is above a predetermined threshold. The device detects an object using one or more regions having a reliability higher than a predetermined threshold value.

For example, the device can exclude a region that does not have a reliability higher than a predetermined threshold among the acquired one or more regions, and can detect an object through analysis only for regions having a reliability higher than a predetermined threshold.

In one embodiment, the device detects the object based on the overlap region of the acquired one or more regions. For example, if there is a part where many areas overlap in the acquired one or more areas, the probability that an object is detected in the corresponding part is high. Thus, a device can detect an object based on an overlapping area of one or more areas.

In one embodiment, the device may detect an object based on an overlapping region of regions having a reliability greater than a predetermined threshold.

In one embodiment, a neural network used in a device includes a plurality of layers, and includes an output layer that outputs information about one or more regions and reliability of one or more regions.

In one embodiment, the output layer includes a neuron representing the location, size and reliability of each of the one or more regions for each of the plurality of segmented cells and a neuron representing the class of the detected object.

3 is a block diagram of a device that detects an object in accordance with one embodiment.

The processor 310 may include one or more cores (not shown) and a connection path (e.g., a bus, etc.) to transmit and receive signals to and / or from a graphics processing unit (not shown) .

According to one embodiment, the processor 310 may process one or more instructions contained in the neural network 100 in parallel.

For example, the processor 310 may use the plurality of layers included in the neural network 100 to obtain attribute information such as color, edge, polygon, saturation, brightness, color temperature, blur, sharpness, .

The processor 310 in accordance with one embodiment performs the object detection method described with respect to FIG. 2 by executing one or more instructions stored in memory.

For example, the processor 310 may be configured to obtain an image containing the object to be detected by executing one or more instructions stored in memory, to divide the obtained image into a plurality of cells, Acquiring information on at least one region estimated to contain the object for each of the plurality of divided cells using a plurality of layers, obtaining reliability for each of the obtained one or more regions, Based on the obtained reliability and the overlapping area of the obtained one or more areas.

The processor 310 may include a random access memory (RAM) (not shown) and a read only memory (ROM) for temporarily and / or permanently storing signals (or data) , Not shown). In addition, the processor 310 may be implemented as a system-on-chip (SoC) including at least one of a graphics processing unit, a RAM, and a ROM.

The memory 320 may store programs (one or more instructions) for processing and control of the processor 310. Programs stored in the memory 320 may be divided into a plurality of modules according to functions. According to one embodiment, the memory 320 may include a data learning unit and a data recognition unit. The data learning unit and the data recognizing unit may each include a neural network module or may share one neural network module.

The neural network module may include a plurality of layers. The plurality of layers included in the neural network module may each include one or more instructions for detecting at least one attribute information from the image and abstracting the detected at least one attribute information.

For example, the first to Nth layers 122 to 128 may each comprise a convolution layer including one or more instructions for extracting attribute information of an image from an image, and / or a convolution layer, And a pooling layer that includes one or more instructions that determine the state of the processor.

Referring to FIG. 4, the processor 330 according to one embodiment may include a data learning unit 410 and a data recognition unit 420.

The data learning unit 410 may learn a criterion for detecting an object included in the image. For example, the data learning unit 410 can learn parameters of at least one layer included in the neural network 100 using learning data used for detecting an object.

The data recognition unit 420 can detect an object in the image based on the learned criterion through the data learning unit 410. [

At least one of the data learning unit 410 and the data recognition unit 420 may be manufactured in at least one hardware chip form and mounted on the device. For example, at least one of the data learning unit 410 and the data recognition unit 420 may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI) Or an application processor) or a graphics-only processor (e.g., a GPU), and may be mounted on various devices as described above.

In this case, the data learning unit 410 and the data recognition unit 420 may be mounted on one device or may be mounted on separate devices, respectively. For example, one of the data learning unit 410 and the data recognition unit 420 may be included in the device, and the other may be included in the server. The data learning unit 410 and the data recognition unit 420 may provide the model information constructed by the data learning unit 410 to the data recognition unit 420 through a wired or wireless connection, 420 may be provided to the data learning unit 410 as additional learning data.

At least one of the data learning unit 410 and the data recognition unit 420 may be implemented as a software module. When at least one of the data learning unit 410 and the data recognition unit 420 is implemented as a software module (or a program module including an instruction), the software module may be a computer-readable, And may be stored in non-transitory computer readable media. In this case, at least one software module may be provided by an operating system (OS) or by a predetermined application. Alternatively, some of the at least one software module may be provided by an operating system (OS), and some of the software modules may be provided by a predetermined application.

5 is a block diagram of a data learning unit according to an embodiment.

5, a data learning unit 410 according to some embodiments includes a data obtaining unit 510, a preprocessing unit 520, a learning data selecting unit 530, a model learning unit 540, 550). However, this is only an example, and the data learning unit 410 may be configured with fewer components than the above-described configurations, or other components other than the above-described configurations may be additionally included in the data learning unit 410. [

The data acquisition unit 510 may acquire at least one of an image and a moving image. Here, the moving image may be composed of a plurality of images. In one example, the data acquisition unit 510 may acquire an image from an external device capable of communicating with a device including the data learning unit 410 or a device including the learning unit 410.

The preprocessing unit 520 can preprocess the acquired image so that the acquired image can be used for learning for object detection. The preprocessing unit 520 can process the acquired image into a predetermined format so that the model learning unit 540, which will be described later, can use the image obtained for learning for object detection.

The learning data selection unit 530 can select an image required for learning from the preprocessed data. The selected image may be provided to the model learning unit 540. The learning data selecting unit 530 can select an image required for learning from among the preprocessed images according to the set criteria.

The model learning unit 540 can learn the data recognition model. For example, the data learning unit 410 can learn parameters of at least one layer included in the neural network 100 using learning data used for detecting an object.

In addition, the model learning unit 540 can learn the data recognition model through reinforcement learning using, for example, feedback as to whether the detection result of the object according to the learning is correct.

Further, when the data recognition model is learned, the model learning unit 540 can store the learned data recognition model. In this case, the model learning unit 540 can store the learned data recognition model in the memory of the device including the data recognition unit 420. [ Alternatively, the model learning unit 540 may store the learned data recognition model in the memory of the device including the data recognition unit 420 to be described later. Alternatively, the model learning unit 540 may store the learned data recognition model in a memory of a server connected to the device via a wired or wireless network.

In this case, the memory in which the learned data recognition model is stored may also store instructions or data associated with, for example, at least one other component of the device. The memory may also store software and / or programs. The program may include, for example, a kernel, a middleware, an application programming interface (API) and / or an application program (or "application").

The model evaluation unit 550 may input evaluation data to the data recognition model and allow the model learning unit 540 to learn again when the detection result output from the evaluation data does not satisfy the predetermined criterion. In this case, the evaluation data may be predetermined data for evaluating the data recognition model. Here, the evaluation data may include a matching ratio between the detected object and the actual object based on the data recognition model.

On the other hand, when there are a plurality of learned data recognition models, the model evaluation unit 550 evaluates whether each of the learned moving image recognition models satisfies a predetermined criterion, and if the model satisfying the predetermined criterion is a final data recognition model You can decide.

At least one of the data acquisition unit 510, the preprocessor 520, the learning data selection unit 530, the model learning unit 540, and the model evaluation unit 550 in the data learning unit 410 may include at least one And can be mounted on a device. For example, at least one of the data acquisition unit 510, the preprocessor 520, the learning data selection unit 530, the model learning unit 540, and the model evaluation unit 550 may be an artificial intelligence (AI) Or may be implemented as part of a conventional general-purpose processor (e.g., a CPU or an application processor) or a graphics-only processor (e.g., a GPU) and loaded on various devices as described above.

The data acquisition unit 510, the preprocessor 520, the learning data selection unit 530, the model learning unit 540, and the model evaluation unit 550 may be mounted on one device, Respectively. For example, some of the data acquisition unit 510, the preprocessor 520, the learning data selection unit 530, the model learning unit 540, and the model evaluation unit 550 are included in the device, .

At least one of the data acquisition unit 510, the preprocessing unit 520, the learning data selection unit 530, the model learning unit 540, and the model evaluation unit 550 may be implemented as a software module. At least one of the data acquisition unit 510, the preprocessor 520, the learning data selection unit 530, the model learning unit 540, and the model evaluation unit 550 is a software module (or a program including an instruction) Module), the software module may be stored in a computer-readable, readable non-transitory computer readable media. Also, in this case, the at least one software module may be provided by an operating system (OS) or by a predetermined application. Alternatively, some of the at least one software module may be provided by an Operating System (OS), and some of the software modules may be provided by a predetermined application.

6 is a block diagram illustrating a device according to one embodiment.

6, the device 300 includes an image preprocessing unit 330, a learning data editing and generating unit 332, a learning data storage unit 334, a learning unit 336, and a learning result storage unit 338 .

In one embodiment, the image preprocessing unit 330, the learning data editing and generating unit 332, the learning data storage unit 334, the learning unit 336, and the learning result storage unit 338 are shown in FIGS. 4 and 5 May be understood as an embodiment of the data learning unit 410 shown.

Therefore, the image preprocessing unit 330, the learning data editing and generating unit 332, the learning data storage unit 334, the learning unit 336, and the learning unit 336 included in the device 300 and the device 300 shown in FIG. The contents already described with respect to the data learning unit 410 of FIGS. 4 and 5 are the same as those of the device 300 and the device 300 shown in FIG. 6, The learning data editing and generating unit 332, the learning data storage unit 334, the learning unit 336, and the learning result storage unit 338 included in the image processing unit 330 of FIG.

Hereinafter, the operation of the device 300 shown in Fig. 6 will be described with reference to Figs. 7 to 11. Fig.

FIGS. 7 to 11 are diagrams for explaining a method for a device to perform learning for detecting an object and to detect an object according to the disclosed embodiment. FIG.

In one embodiment, the device 300 generates learning data 600 for neural network learning and uses it to learn the neural network and store the learning results. For example, the neural network may be a convolution based neural network.

In one embodiment, the image preprocessing unit 330 removes noise from the scanned document image, and then divides the image into a plurality of blocks based on the distribution of black pixels. Referring to Figure 8, each of the blocks 710-740 that are divided from the scanned document image 700 is shown.

Each of the divided blocks is normalized to the input value of the input layer 110 of the neural network 100. The image preprocessing unit 330 generates a normalized image 610 portion of the learning data 600 using the divided blocks.

In one embodiment, the normalized image is a color image with three channels to process both monochrome and color images, and is normalized to a predetermined size of equal width and height.

The learning data editing and generating unit 332 receives an input from a user and sets an object area on the image and a type (or a class) of the corresponding area, and creates mask data 620 using the set.

For example, the mask data 620 is composed of the x and y coordinates of the center point 812 of the object area 810 and the width 814 and the height 816 of the area, as shown in Fig. For example, the center point 812 coordinates of the object region 810 may be normalized between 0 and 1 based on the width and height of the segmented block 800.

The width 814 and height 816 of the region may also be normalized between 0 and 1 based on the width and height of the segmented block 800.

Therefore, the object area information included in the mask data 620 includes information of the center point 812 of the area, the width 814 and the height 816 of the area, and each information is information normalized between 0 and 1.

The mask data 620 is used as data for map learning in neural network learning.

In one embodiment, the learning data storage unit 334 automatically generates and stores a plurality of learning data through a conversion operation such as adding noise, rotation, and movement to the generated learning data. At this time, the learning data storage unit 334 may be formed as a storage capable of distributed parallel processing.

The learning unit 336 learns the neural network 100 using the generated learning data. In one embodiment, learning of the neural network 100 is done by supervised learning. The map learning is a method of inputting input data and corresponding output data together into a neural network and updating the weights of connected edges so that output data corresponding to the input data is output. For example, the neural network according to the disclosed embodiments may update the connection weights between artificial neurons using delta rules and error-domain propagation learning.

The learning result storage unit 338 stores weight values as learning results of the neural network 100 and configuration information of the neural network 100 as learning result data.

10 is a diagram illustrating a method for detecting an object in accordance with the disclosed embodiment.

In the disclosed embodiment, to improve the detection speed, the divided block image 800 is divided into cells 802 of a predetermined size, and object detection is performed based on each cell 802.

For example, the device 300 divides the divided block 800 image into w * h cells, and extracts n estimated object regions and reliability for each divided cell.

Referring to Fig. 11, an example of the configuration of the output layer 130 is shown.

For example, the output layer 130 is composed of convolution neural networks that output n estimated object areas and reliability for each cell for each cell.

The neuron 900 corresponding to one of the divided cells 804 includes a neuron 901 representing the abscissa x of the center point of the estimated object region, a neuron 902 representing the ordinate y, A neuron 904 representing the height of the region and a neuron 905 representing the reliability are sequentially included and may consist of neurons 906 and 907 for classification.

Then, the device 300 can estimate an optimal object region using the overlapping portion and the reliability between the object regions estimated for each cell.

At this time, the object region 810 has the coordinates x, y of the center point 812 of the region such as the structure of the mask data for learning, the width 814 of the region, and the height 816 of the region. And also has reliability for each estimated object region 810.

Thus, the final output layer 130 of the neural network 100 for object region detection is configured in the form of a convolution layer having an output of magnitude w * h * (n * 5 + c). In this case, c is the number of classes to classify, and the number of filters in the convolution layer is n * 5 + c.

In one embodiment, the number of classes may be two, including the resident registration number in the printed typography format and the resident registration number in the written handwriting format. In addition, classes can be added to detect other types of personal information, such as fingerprints and names, and personal information in various formats can be detected.

For example, the input layer 110 may be configured to have a size of 608 * 608 in three channels to receive an image normalized as a result of processing by the image preprocessing unit 330 as an input value.

For example, the first layer 122 included in the plurality of layers 122 to 128 receives an input value of the input layer 110 as a convolution layer, and the size of the window is 3 and the size of the filter is 32. [

For example, the second layer 124 included in the plurality of layers 122 to 128 is a pulling layer, and receives the output value of the convolution layer, and the downscale size is 2.

Subsequent layers 126 and 128 may be omitted and the third layer 126 may be used as the convolution layer according to an embodiment and the downsampling of the pooling layer may be used to prevent lower resolution of the detection result , The fourth layer 128 may not be used.

The output value of the output layer 130 has a size of a cell to divide the divided image for object detection and is configured to have reliability and information of n object regions estimated for each class.

Accordingly, the neural network 100 according to the disclosed embodiment receives the preprocessed image, passes through each layer, and then obtains coordinate information and reliability of an area estimated as an object in the image in the output layer 130. [ At this time, only the case where the reliability is equal to or more than the threshold value is selected as the extracted region, thereby completing the object detection process.

12 is a block diagram of a data recognition unit according to an embodiment.

12, the data recognition unit 420 according to some embodiments includes a data acquisition unit 1010, a preprocessing unit 1020, a recognition data selection unit 1030, a recognition result providing unit 1040, 1050 < / RTI >

The data acquisition unit 1010 can acquire an image necessary for the detection of the object and the preprocessing unit 1020 can preprocess the acquired image so that the acquired image can be used for detection of the object. The preprocessing unit 1020 may process the acquired image into a predetermined format so that the recognition result providing unit 1040, which will be described later, can use the image obtained for the object detection. The recognition data selection unit 1030 can select an image required for object detection from the preprocessed data. The selected data may be provided to the recognition result providing unit 1040.

The recognition result providing unit 1040 may apply the selected image to the neural network according to an embodiment to detect an object in the image. Here, the neural network may include a plurality of layers as described above. A method of detecting an object in an image by applying an image to a neural network may correspond to the method described above with reference to FIGS.

The recognition result providing unit 1040 may provide detection information of at least one object included in the image.

The model updating unit 1050 updates the information on the evaluation so that the parameters of one or more layers included in the neural network are updated based on the evaluation of the detection result of the object provided by the recognition result providing unit 1040, To the model learning unit 1040 described above.

At least one of the data acquiring unit 1010, the preprocessing unit 1020, the recognition data selecting unit 1030, the recognition result providing unit 1040 and the model updating unit 1050 in the data recognizing unit 1020 is a It can be manufactured in the form of one hardware chip and mounted on the device. For example, at least one of the data acquisition unit 1010, the preprocessing unit 1020, the recognition data selection unit 1030, the recognition result providing unit 1040, and the model updating unit 1050 may be a dedicated hardware chip for artificial intelligence Or may be implemented as part of an existing general purpose processor (e.g., a CPU or an application processor) or a graphics-only processor (e.g., a GPU) and loaded on the various devices described above.

The data acquisition unit 1010, the preprocessing unit 1020, the recognition data selection unit 1030, the recognition result providing unit 1040 and the model updating unit 1050 may be mounted on one device, Devices, respectively. For example, some of the data acquisition unit 1010, the preprocessing unit 1020, the recognition data selection unit 1030, the recognition result providing unit 1040, and the model updating unit 1050 are included in the device, May be included in the server.

At least one of the data acquisition unit 1010, the preprocessing unit 1020, the recognition data selection unit 1030, the recognition result providing unit 1040, and the model updating unit 1050 may be implemented as a software module. At least one of the data acquisition unit 1010, the preprocessing unit 1020, the recognition data selection unit 1030, the recognition result providing unit 1040 and the model updating unit 1050 is a software module (or an instruction) Program modules), the software modules may be stored in a computer-readable, readable non-transitory computer readable media. In this case, at least one software module may be provided by an operating system (OS) or by a predetermined application. Alternatively, some of the at least one software module may be provided by an operating system (OS), and some of the software modules may be provided by a predetermined application.

13 is a block diagram illustrating a device according to one embodiment.

13, the device 300 includes a neural network learning result load unit 340, a neural network generator 342, an image preprocessor 344, a detection unit 346, and a detection result application unit 348.

In one embodiment, the neural network learning result load unit 340, the neural network generating unit 342, the image preprocessing unit 344, the detection unit 346, and the detection result application unit 348 are the same as the data shown in Figs. 4 and 12 And can be understood as an embodiment of the recognition section 420. [

Accordingly, the neural network learning result load unit 340, the neural network generation unit 342, the image preprocessing unit 344, the detection unit 346, and the detection result application unit 342 included in the device 300 and the device 300 shown in FIG. The contents already described with respect to the data learning unit 410 of FIGS. 4 and 5 are the same as those of the device 300 shown in FIG. 13 and the device 300 shown in FIG. The neural network learning result load unit 340, the neural network generation unit 342, the image preprocessing unit 344, the detection unit 346, and the detection result application unit 348.

The neural network learning result load unit 340 reads the learning result data stored in the learning result storage unit 338 of FIG. At this time, the read information includes configuration information of the convolution-based neural network and weight information between neurons in each layer.

The neural network generator 342 generates a neural network for object detection on the basis of the configuration information and the weight information of the neural network read through the neural network learning result load unit 340.

The neural network learning result load unit 340 and the neural network generator 342 may be included so that the data recognition unit 420 can operate independently of the data recognition unit 410.

The image preprocessing unit 344 removes noise from the input image in the same manner as the image preprocessing unit 330 in FIG. 6, and performs block segmentation and normalization for object detection.

The detection unit 346 performs an object detection operation using the divided input data for each block generated from the image preprocessing unit 344 and the convolution based neural network generated from the neural network generating unit 342.

Based on the object detection information generated by the detection unit 346, the detection result application unit 348 calculates the position of the object using the position information of each block divided from the image.

The steps of a method or algorithm described in connection with the embodiments of the present invention may be embodied directly in hardware, in software modules executed in hardware, or in a combination of both. The software module may be a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD- May reside in any form of computer readable recording medium known in the art to which the invention pertains.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, You will understand. Therefore, it should be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

300: device
310: Processor
320: Memory

Claims (10)

Acquiring an image including a detection target object;
Segmenting the obtained image into a plurality of cells;
Performing object detection on each of the plurality of divided cells using a plurality of layers included in a network for detecting the object, wherein the object detection is performed for each of the plurality of divided cells, Acquiring information about a region;
Obtaining reliability for each of the obtained one or more regions; And
Detecting the object based on the obtained reliability and the overlapping area of the obtained one or more areas; / RTI >
Wherein the dividing step comprises:
Dividing the image into one or more blocks based on a distribution of pixels included in the obtained image; And
And dividing each of the divided one or more blocks into the plurality of cells. delete The method according to claim 1,
Wherein the information on the at least one area includes a center point coordinate, a width, and a height of the at least one area,
The one or more center point coordinates, width,
Characterized in that the device is normalized with respect to the width and height of the block to which each of the at least one of the divided one or more blocks belongs. The method according to claim 1,
Wherein the detection object includes one or more classes,
Wherein obtaining information about the one or more regions comprises:
And obtaining information about at least one region that is estimated to include the object for each of the one or more classes. The method according to claim 1,
Wherein detecting the object comprises:
Obtaining, for each of the obtained one or more areas, one or more areas having a reliability higher than a predetermined threshold; And
Detecting the object using one or more regions having reliability higher than the predetermined threshold; Wherein the device detects an object. 6. The method of claim 5,
Wherein detecting the object comprises:
Calculating a position of an area including the object using an overlap region of at least one region having a reliability higher than or equal to the predetermined threshold value; Wherein the device detects an object. The method according to claim 1,
Wherein a plurality of layers included in the network include an output layer for outputting information on the one or more areas and reliability of the one or more areas,
Wherein the output layer comprises:
A neuron representing a location, size and reliability of each of the one or more areas for each of the plurality of divided cells, and a neuron representing a class of the object. The method according to claim 1,
The object to be detected,
Wherein the device comprises at least one of a resident registration number and other personal information included in the image. A memory for storing one or more instructions;
And a processor executing the one or more instructions stored in the memory,
The processor executing the one or more instructions,
Acquiring an image including a detection object,
Dividing the obtained image into a plurality of cells,
Performing object detection on each of the plurality of divided cells using a plurality of layers included in a network for detecting the object, wherein the object detection is performed for each of the plurality of divided cells, Acquires information on the area,
Acquiring reliability for each of the obtained one or more areas,
Detecting the object based on the obtained reliability and the overlapping area of the obtained one or more areas,
And dividing the obtained image into a plurality of cells,
Dividing the image into one or more blocks based on a distribution of pixels included in the acquired image, and dividing each of the divided one or more blocks into the plurality of cells. A computer program stored in a computer readable recording medium in combination with a computer which is hardware and which is capable of performing the method of claim 1. Description:

Images