KR20200005324A - Method and system for recognizing images based on machine learning - Google Patents

Abstract

Disclosed are a machine learning-based image recognition method and a machine learning-based image recognition system. The machine learning-based image recognition method according to an embodiment of the present invention comprises the steps of: recording data about a background fragment selected by processing image for training in a machine learning DB; determining whether the background fragment is included in an input actual image; and retrieving the data from the machine learning DB when the background fragment is included in the actual image and outputting information recognized in relation to the actual image on a screen by using the retrieved data.

Description

Machine learning based image recognition method and machine learning based image recognition system {METHOD AND SYSTEM FOR RECOGNIZING IMAGES BASED ON MACHINE LEARNING}

The present invention relates to a technology for integrating a background with an object from a camera image based on a machine learning. The present invention selects an advantageous background piece that can serve as an object in a learning image and machine learning the same. The present invention relates to a machine learning based image recognition method and a machine learning based image recognition system which can be combined with an object to improve the recognition performance of an input real image.

1 is a diagram illustrating an example of recognizing an image in a machine learning system according to an exemplary embodiment.

Referring to FIG. 1, in the machine learning system 100 according to an exemplary embodiment, a mechanism for processing a learning image used for machine learning is mainly object-oriented.

For example, the machine learning system 100 has been studied as an object to be divided or removed in consideration of all the objects other than a specified object such as a bicycle, an airplane, a person, etc. in a learning image as a background.

For example, in an image captured at a distance from a certain area, the machine learning system 100 may divide the image into 'house' and 'road' and 'forest', but not 'house' but 'road' We couldn't learn by specifying elements that could act as a single object in the background, such as' and 'forest'.

In addition, the shadow of the object in the image was regarded as the background, and the processing for classifying or recognizing the background caused the performance deterioration.

Conventional machine learning system 100 has been applied to a variety of classification technology, location technology, multiple sensing technology, segmentation technology, matching technology. For example, classification technology is a technology for classifying an image having an object 'cat', and a location locating technology is a technology for specifying the position of an object 'cat' in an image, and multiple sensing technology is a 'cat' and ' It is a technology for recognizing various objects such as 'duck' and 'dog' at the same time, and the segmentation technology may refer to a technology for determining, dividing, and separating the shape of an object in each image.

Therefore, in various systems using the machine learning system 100 described above, when the object cannot be extracted from the input image, it may be in a very unstable state.

Accordingly, if a form that cannot be defined as an object is detected in the image by the existing object-oriented machine learning system 100, a technology is required to recognize the information in the image by using it as a background fragment capable of acting as an object. have.

An embodiment of the present invention is to extract a part of the background (hereinafter, 'background fragment') from the learning image as a meaningful object, to solve the problem of not recognizing the background in the existing object-centered machine learning The aim is to improve recognition performance by combining objects with background fragments.

In addition, an embodiment of the present invention aims to more accurately calculate the position of each object by using the relationship information between the object and the background pieces, and obtain motion information of the camera based on this.

Machine learning-based image recognition method according to an embodiment of the present invention, the step of recording the data on the background pieces selected by processing the learning image in the machine learning DB, and determines whether the background pieces are included in the actual image input And retrieving the data from the machine learning DB if the determination result is included, and outputting information recognized in relation to the actual image to the screen using the retrieved data.

In addition, the machine learning-based image recognition system according to an embodiment of the present invention, the learning processing unit for recording the data on the background pieces selected by processing the learning image, in the machine learning DB, and the input in the actual image, the background And a recognition processor configured to determine whether a piece is included, and if the determination result is included, retrieve the data from the machine learning DB and use the retrieved data to output information recognized in relation to the real image on a screen. do.

According to an embodiment of the present invention, by extracting a part of the background (hereinafter, 'background fragment') from the learning image as a meaningful object, to solve the problem of not recognizing the background in the existing object-oriented machine learning In addition, recognition performance and accuracy can be improved by combining objects with background fragments.

According to an embodiment of the present invention, even when an object is not extracted from the input real image, it is possible to provide an environment that can recognize navigation information such as position and direction using the learned background fragment.

According to an embodiment of the present invention, the position of each object may be more accurately calculated by using the relationship information between the object and the background pieces, and the motion information of the camera may be obtained based on this.

According to an embodiment of the present invention, a system for referencing digital terrain information, a system for finding a matching reference point in an image, a system for extracting navigation information from an image, and an integrated recognition system applicable to machine learning-based applications may be provided. Can be.

1 is a diagram illustrating an example of recognizing an image in a machine learning system according to an exemplary embodiment.
2 is a block diagram illustrating a configuration of a machine learning based image recognition system according to an exemplary embodiment of the present invention.
3A is a diagram illustrating a configuration of a window region to be extracted from a training image in an image recognition system according to an exemplary embodiment of the present invention.
3B illustrates an example of extracting a plurality of window regions from a training image in an image recognition system according to an exemplary embodiment of the present invention.
FIG. 4 is a diagram illustrating a process of machine learning a background piece using an image for learning in an image recognition system according to an embodiment of the present invention.
5 is a diagram illustrating a process of outputting information by recognizing a background fragment from an actual image in an image recognition system according to an exemplary embodiment of the present invention.
6 is a diagram illustrating a process of sequentially extracting a plurality of window regions in an image recognition system according to an embodiment of the present invention.
7A and 7B illustrate an example of extracting a window region by dividing a background region into a plurality of grids in an image recognition system according to an exemplary embodiment of the present invention.
8 is a diagram illustrating a process of constructing a machine learning DB by selecting a background piece from a learning image in an image recognition system according to an embodiment of the present invention.
9 is a flowchart illustrating a procedure of a machine learning based image recognition method according to an embodiment of the present invention.

Hereinafter, a machine learning based image recognition method and an image recognition system according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the embodiments. Like reference numerals in the drawings denote like elements.

2 is a block diagram illustrating a configuration of a machine learning based image recognition system according to an exemplary embodiment of the present invention.

Referring to FIG. 2, the image recognition system 200 according to an embodiment of the present invention may include a learning processor 210, a machine learning DB 220, and a recognition processor 230.

The learning processor 210 records data related to the background pieces selected by processing the learning image in the machine learning DB 220.

That is, the learning processor 210 may select a part of the background area in the learning image input during the machine learning process as a 'background fragment' capable of acting as an object.

Here, the background area may refer to the entire area excluding a specified object such as a bicycle, a person, a dolphin, a car, an airplane, a house, and the like in the learning image, and the learning processor 210 may extract the object from the learning image or the learning image. Even if no object is extracted from, the background fragment can be selected.

In detail, the learning processor 210 extracts at least one window area that is a candidate for background fragments from the background area of the training image, and selects a window area of which the recognition performance is high from the extracted window area. It can be selected as a possible background piece.

For example, referring to FIGS. 3A and 3B, the learning processor 210 may have a rectangular window area Window 1 having a plurality of parameters shown in FIG. 3A from the learning image 320 shown in FIG. 3B. Window 2, Window 3, and Window 4) 310 may be extracted.

In this case, the learning processor 210 may find a large amount of the learning images 320 in real data or virtually synthesize the learning images according to at least one set shooting condition among shadow, resolution, brightness, color, position, and direction. 320) can be prepared.

The plurality of parameters may be at least one of the horizontal size of the window area 310 and the relationship information (eg, distance and spacing) between the vertical size, the horizontal movement width, the vertical movement width, the center point position, the rotation angle, and other window regions. have.

The value of each parameter may be set in advance, but may be changed in the process of extracting each window region 310.

For example, referring to the learning images 630 and 640 of FIG. 6, the learning processing unit 210 is configured between four window areas (Window 1, Window 2, Window 3, and Window 4) set in the learning image 630. The distribution of the window area is set by adjusting the center point position according to the relationship information (for example, distance, spacing, direction, etc.), and the parameters of each window area are adjusted to adjust the size of the window area as in the training image 640. Can be changed.

In addition, although the shape of the window area is not limited to a rectangle and may be any shape such as a triangle or a circle, the present specification exemplifies extracting the window area of the rectangle to facilitate control of the window area by adjusting parameters.

The learning processor 210 may select a background fragment from a part or the entire area of the background of the learning image, and when selecting a background fragment capable of acting as an object in the entire area, the object map of the learning image (the moon surface) Can be generated automatically.

The learning processor 210 may extract the window area from the background area of the learning image in various ways.

For example, the learning processor 210 may divide the background area into a lattice structure and set the parameter for n window areas to be extracted from the background area divided into the lattice structure.

For example, referring to FIG. 7A, the learning processor 210 includes a background region, an entire background region (lattice 1), and an area in which the background region is divided into four quarters (lattices 2 to 5). Can be divided into a grid of.

The learning processor 210 may extract a predetermined number n of window areas from the background area.

Referring to FIG. 7B, the learning processor 210 sets the center point positions and horizontal and vertical sizes of a total of four window regions to be extracted from the background region of the grid structure, for example, selecting the window regions Window 1 and Window 2 from the grid 1. The window regions (Window 3, Window 4) can be extracted from the grid 5.

As another example, the learning processor 210 may divide the background area into a plurality of areas such as east, west, south, and north according to directions, and divide and extract each area into detailed areas according to the size.

In addition, the learning processor 210 may extract the remaining window area according to a positional relationship or a distance relationship with another window area by using the previously extracted window area.

Specifically, referring to FIG. 6, the learning processor 210 extracts an arbitrary area in the background area of the learning images 610 to 640 as the first window area Window 2, and a center point for the first window area. Extracts a second window region Window 1 located on a circle C2 within a radius m, and locates a third window located within a predetermined distance from a line L1 connecting the center points of the first and second window regions. Areas (Window 4, Window 3) can be extracted.

The learning processor 210 may select the background pieces from the extracted n window areas. For example, the learning processor 210 extracts a window region including at least a background region in a training image, and extracts the extracted window region when the number of extraction of the window region reaches a predetermined m (the m is one or more natural numbers). Each of the m window areas may be selected as the background fragment.

That is, as shown in FIG. 4, when all m (m = 4) window regions selected from the background region of the learning image are selected, the learning processor 210 may select each window region as the background fragment.

In detail, the learning processor 210 may calculate a center point position error calculated at the middle or the end of the convolutional neural network layer based on the extracted n window regions, and the rotation angle of the window region based on the center point. A combination of at least one of an error, whether the actual shooting area is identified, a horizontal size error and a vertical size error, and a distance error between the window areas to calculate a performance using a window area other than the window area used for learning. The process of deleting the window area that does not satisfy the minimum reference value, extracting the window area as much as the deleted number, and calculating the performance of the additional extracted window area may be repeated.

Here, the performance may include at least one of a detection rate for the unspecified window area and an accuracy of position identification using the extracted window area.

In particular, the accuracy indicates whether the sensed window area can serve as an object in the background, and may be set to a value equal to or greater than a minimum reference value used to determine accuracy in conventional general object extraction. That is, the learning processor 210 may select a window region having the accuracy of the general object level as the background fragment.

The learning processor 210 may delete the window area whose calculated performance does not satisfy the minimum reference value and exclude the candidate from the background fragment. In addition, the learning processor 210 may additionally extract the window area as much as the deleted number, and calculate the performance of the additional extracted window area.

According to an exemplary embodiment, the learning processor 210 sorts the performance of the n window areas in ascending order using verification data provided separately from the learning image, and then reselects the window areas having the higher performance as the background pieces. You may.

In this case, the learning processor 210 may adjust the m in consideration of the performance of the learning image.

For example, the learning processor 210 considers an average value of the performance of each of the m window regions extracted from the learning image as the average performance of the learning image, and if the performance is higher than the minimum reference value, adjusts to increase m to minimize the minimum. A window with slightly lower performance than the reference value can be adopted.

When the m window areas are selected as the background pieces, the learning processor 210 may assign at least one of a unique ID and a name to each window area.

For example, the learning processor 210 may refer to the center point position, the horizontal size, the vertical size, and the direction, to give the window region Window 1 a name “northern region B zone”, and to give the window region Window 2 a name. 'Eastern Zone D' may be assigned, the name 'Southern Zone A' may be assigned to the window region (Window 3), and the name 'West Zone C' 'may be assigned to the window region (Window 4).

The learning processor 210 may include data including at least one of a unique ID, a name, a center point position, a horizontal size, a vertical size, a rotation angle of a window region, and performance (detection rate and accuracy) assigned to a window region selected as a background fragment. It can be recorded in the machine learning DB (220).

The machine learning DB 220 processes and detects a learning image and records data about the identified object, the selected background fragment, and artificial neural network coefficients (weights, biases, etc.) and the hierarchical structure resulting from learning the object and the background fragment. , Keep.

For example, the machine learning DB 220 may record at least one object data among the size, shape, position in the image, direction, and relationship with other objects of the extracted object in association with the corresponding object.

In addition, the machine learning DB 220 may record at least one piece of data of a parameter of a selected background piece, a unique ID, a name, an object, or a relationship with another background piece in association with the background piece.

The recognition processor 230 determines whether the background fragment is included in the actual image to be input.

The recognition processor 230 calculates whether the determination result is included using the artificial neural network coefficients and the hierarchical structure learned from the machine learning DB 220. The data corresponding to the background fragment is searched for, and the recognition information related to the actual image is output to the screen using the searched data.

For example, referring to FIG. 5, the recognition processor 230 determines whether there is a background fragment in the inputted real image 510, and if a background fragment exists in the real image 510, the recognition processor 230 corresponds to the corresponding background fragment. The recorded data 520 may be retrieved from the machine learning DB 220 and output on the screen.

In this way, the recognition processor 230 may recognize the navigation information such as the position and the direction by using the learned background fragment even when the object is not identified in the input real image.

In addition, the recognition processor 230 generates and outputs recognition information regarding a movement of a photographing camera or an object recognized from the real image, based on at least one of a position, a size, a rotation angle, and a center point position associated with the background fragment. can do.

That is, the recognition processor 230 may calculate the position of each object more accurately by using the relationship information between the object and the background pieces, and acquire the motion information of the camera based on this.

As another example, when an object is identified in an actual image, the recognition processor 230 generates the recognition information by combining the object data recorded about the object in the machine learning DB 220 with the data on the background pieces, and outputs the recognition information. can do.

For example, the recognition processor 230 may re-learn background fragments using artificial neural network coefficients and structures that have learned objects in the machine learning DB 220. As a result, the data related to the background fragment may be combined with the object data related to the object ('plane') and output on the screen as the information 520 recognized in relation to the actual image.

As described above, the recognition processor 230 extracts a part of the background (hereinafter, the background fragment) excluding the object from the learning image as a meaningful object, and solves the problem of not recognizing the background in the existing object-oriented machine learning. In addition, recognition performance and accuracy can be improved by combining objects with background fragments.

3A is a diagram illustrating a configuration of a window region to be extracted from a training image in an image recognition system according to an embodiment of the present invention, and FIG. 3B is a diagram illustrating an example of extracting a plurality of window regions from a training image. .

3A and 3B, the image recognition system according to an exemplary embodiment of the present invention includes a rectangular window area (Window 1, having a plurality of parameters shown in FIG. 3A) from a background area of the training image 320. Window 2, Window 3, and Window 4) 310 may be extracted.

Here, the window area 310 is not limited to a rectangle and may have any shape such as a triangle or a circle, and the plurality of parameters may include a center point position, a rotation angle of the window area based on the center point, a horizontal size and a vertical size, and a distance between the window areas. There may be at least one.

4 is a diagram illustrating a process of machine learning a background piece using an image for learning in an image recognition system according to an embodiment of the present invention.

FIG. 4 illustrates a detailed process of recording data on a background fragment selected by processing a learning image in a learning processing unit 400 in a machine learning-based image recognition system according to an embodiment of the present invention, in a machine learning DB. have.

For example, the learning processor 400 may extract a predetermined number of window areas by selecting a position and a size of a window area that is a candidate for background fragments from the learning image (see 320 of FIG. 3B) input for machine learning. have.

In detail, the learning processor 400 selects the size, position, and number of window regions in the learning image for machine learning. In order to configure the window area, the above-described configurable parameters of the window area are used. If the learning time and the computing performance of the learning system are inferior, the window area may be configured by randomly determining the parameter value.

The best thing to do is to use the data generated in as many different combinations as possible for every area of the image. In particular, in the absence of the atmosphere, such as the moon surface, only the lighting conditions of the sun are the only variables. In such a static environment, the learning processor 400 finds the largest background fragment that can serve as an object at a time, and automatically maps the object map of the entire lunar surface area. Can be generated.

In addition, the learning processor 400 may configure a determination network for calculating the accuracy of the recognition of the extracted window region based on the existing machine learning method, position, direction, and size.

The learning processor 400 learns the data generated according to the parameter configuration of the window area by using a conventional machine learning method (eg, CNN in the case of an image), but determines the position, direction, A discriminant network (calculation) that calculates accuracy based on size, mAP, etc., can be attached at the end.

Here, the calculation formula may use existing YOLO, SSD, Faster RCNN, etc., and the learning processor 400 may calculate accuracy using the number of detected (extracted) window regions, the position, direction, and size of each window region. It may be.

In addition, the learning processor 400 may sort the window areas in ascending order according to the calculated accuracy, and select a window area having an accuracy higher than the minimum reference value as the background fragment.

In this case, the learning processor 400 may sort the accuracy for each window area calculated in ascending order using the verification data classified separately from the learning use.

The learning processor 400 considers (selects) a background fragment that can serve as an object when the accuracy of each window region exceeds a minimum reference value, and assigns a name or a unique ID to identify the window region selected as the background fragment. can do.

Here, the learning processor 400 uses the same reference accuracy as the minimum reference value applied when determining the accuracy of an object (for example, "airplane") extracted from the training image in the existing object-oriented machine learning system as an object in the training image. And background pieces can be handled at the same level.

Thereafter, the learning processor 400 may assign a name or a unique ID to each of the window areas selected as the background pieces, and record the name or unique ID in the machine learning DB.

For example, the learning processor 400 assigns the name 'Northern region B zone' to the window region Window 1, gives the name 'Eastern region D region' to the window region Window 2, and sets the window region (Window). 3) The name 'Southern Region A' may be assigned, and the name 'West Region C' may be assigned to the window area (Window 4).

5 is a diagram illustrating a process of outputting information by recognizing a background fragment from an actual image in an image recognition system according to an exemplary embodiment of the present invention.

FIG. 5 illustrates a detailed process of outputting the information 520 recognized in the actual image 510 input by the recognition processor 500 in the machine learning-based image recognition system according to an embodiment of the present invention. It is.

Here, the recognition processing unit 500 is a real photographing, which is recognized from the inputted real image 510 based on the position, direction, size, and recognition evaluation of the background fragment using a machine learning system that has been transferred learning. Various information 520 may be generated and output, including the movement of the region and the photographing camera or the movement of the object.

When the real image 510 is input, the recognition processor 500 may classify a general object (“airplane”) and a background fragment and recognize and output the information 520 in the real image 510. The information 520 may be an object and a background fragment itself, or may be information obtained by combining data (name or unique ID, etc.) recorded about the object and the background fragment during a machine learning process.

In another example, the recognition processing unit 500 may further learn four transition zones previously selected from the conventional system in which airplanes, spacecrafts, vehicles, and the like are learned, or learn them from the beginning. Plane ”) on the image 510, the location and name of the surrounding area may be output on the screen as a recognition result 520.

6 is a diagram illustrating a process of sequentially extracting a plurality of window regions in an image recognition system according to an embodiment of the present invention.

Referring to FIG. 6, in the case of the learning images 610 to 640 in which an object is not detected, the image recognition system may extract a plurality of window areas that may serve as objects in the background area.

In this case, the image recognition system may sequentially extract the remaining window regions according to a positional relationship or a distance relation with other window regions using the previously extracted window region.

That is, the image recognition system already knows the relationship between the center point of the crater of the moon surface, which is the background area, and the window area in the machine learning process. You can detect window areas sequentially.

In detail, the image recognition system detects a portion of the background area that has different shades, patterns, and colors as the window area ('Window 1') in order to select the crater as the background fragment in the background area 'moon surface'. Can be.

In addition, when a window area ('Window 2') is detected, the image recognition system has a crater center around a circle ('C1') within a predetermined distance from the detected center point of 'Window 2', and then again within a certain distance. A second window area ('Window 1') can be detected around the circle 'C2'.

In addition, the image recognition system may include a crater center point and other window regions ('Window 3' and 'Window 4') from the line (L1) connecting the center points of the two detected window regions ('Window 2' and 'Window 1'). Find the location of.

If the image recognition system detects a predetermined number ('four') of window areas, the camera may obtain motion information of the camera photographing the actual image using triangulation and bundle adjustment.

In addition, the image recognition system changes (corrects) the size of each window area by adjusting parameter values, and when the size of the window area is changed, obtains motion information of the camera capturing the corresponding image by changing the size of each window area. Can be.

The image recognition system selects each window area as a background piece, assigns a name and a unique ID, and records the window area together with the camera motion information in the machine learning DB.

As a result, the image recognition system can improve the positional identification accuracy of the object compared to the conventional system, and can easily obtain the navigation information and the camera motion information even when the object is not detected.

7A and 7B illustrate an example of extracting a window region by dividing a background region into a plurality of grids in an image recognition system according to an exemplary embodiment of the present invention.

7A and 7B, the image recognition system according to an exemplary embodiment of the present invention may include a grid of p background regions (p is one or more natural numbers, for example, p = 5) of a training image. The window region may be extracted by dividing and setting a center point position, a horizontal size, and a vertical size in the grid of the p regions.

For example, the image recognition system includes five background regions of the training image including the entire background region (lattice 1) and the region (lattice 2 to grid 5) that is divided into four quarters of the background region, as shown in FIG. 7A. Can be divided into a grid of areas.

Also, as shown in FIG. 7B, the image recognition system sets the center point position and the horizontal and vertical size of the window area to be extracted from each grid, for example, extracts the window areas Window 1 and Window 2 from grid 1, You can extract window regions (Window 3, Window 4).

As another example, when the image recognition system divides a background area into three grids, the image recognition system sets a center point position of two window regions in grid 1, sets a center point position of one window region in grid 2, If the center point position is not set at 3, the total number of extractions of the window area may be '3' when summed by the grids.

8 is a diagram illustrating a configuration of an image recognition system according to another embodiment of the present invention.

Referring to FIG. 8, the image recognition system 800 according to the exemplary embodiment of the present invention may include a learning processor 810, a recognition processor 820, and a machine learning DB 830.

The learning processor 810 includes at least a background region in the training image, extracts n (where n is a natural number of 1 or more) window regions, and selects the number of candidate regions selected in consideration of the performance calculated for each of the n window regions. When the number of m (m is one or more natural numbers) set by the user is reached, each of the m candidate areas may be selected as the background pieces and maintained in the machine learning DB 830.

For example, the learning processor 810 may calculate, for each of the n window regions, a center point position error and a center point reference calculated at the middle or the end of the artificial neural network layer based on the convolutional neural network (“artificial neural network A” of FIG. 8). The performance may be calculated by combining at least one of a rotation angle error of one window area, whether or not an actual photographing area is identified, a horizontal size error and a vertical size error, and a distance error between the window area.

In addition, the learning processor 810 may select a window area whose performance satisfies the minimum reference value from among n window areas as a candidate area.

In addition, the learning processor 810 may select, as the candidate region, a window region that is higher than the minimum reference value and aligned above when the n window regions are arranged in ascending order according to the performance.

For example, when n is '10' and m is set to '3', the learning processing unit 810 sequentially extracts 10 window regions from the background region divided into a grid structure as one window region. Alternatively, ten window regions may be randomly extracted without specifying a position in the background region.

The learning processor 810 may calculate the performance (including accuracy and detection rate) of the extracted 10 window areas, and select all six window areas whose calculated performance is equal to or greater than the minimum reference value 'c' as a candidate area. Alternatively, when the ten window regions are sorted in ascending order according to the calculated performance, the upper three window regions can be selected as candidate regions. As the number of the selected candidate regions reaches m predetermined values ('3'), the learning processor 810 may select m candidate regions having excellent performance ('3') as the background pieces.

In this case, if the number of candidate regions does not reach the m number, the learning processor 810 deletes the window regions whose performance does not satisfy the minimum reference value, and additionally extracts the window regions by the deleted number. In addition, the performance may be calculated for the additional extracted window region.

The learning processor 810 may adjust to increase the m when a value obtained by averaging the performance of each of the m candidate areas is larger than a minimum reference value.

In other words, when the performance average of each candidate region selected as the background fragment is significantly greater than the minimum reference value, the learning processor 810 may further select the background fragment from the remaining candidate regions not selected as the background fragment. m can be increased.

The learning processing unit 810 further selects the candidate areas in order from the window areas arranged above when the n window areas are sorted in ascending order according to the performance, until the increased m is reached, and the addition is performed. The candidate region selected by can be further selected as the background fragment.

The learning processor 810 determines the minimum reference value in consideration of the performance calculated for the objects in the machine learning DB 820, and calculates the background pieces when the background pieces are recorded in the machine learning DB 820. In further consideration of performance, the minimum reference value can be adjusted.

In other words, the minimum reference value may be determined by using the performance of objects that are learned through existing general machine learning and maintained in the machine learning DB 820, and thus candidates having the same performance (accuracy and detection rate) as the objects. The area can be selected as a background piece.

Background pieces recorded in the machine learning DB (830) can be learned in the artificial neural network ('artificial neural network B' of Figure 8) through the transfer machine learning, the learning processing unit 810 is the artificial through the existing machine learning The minimum reference value may be feedbacked by calculating a performance of the object trained on the neural network and the background fragment, and using an average value or a minimum value of the performance of the object and the background fragment.

Through this, the learning processor 810 can maintain the performance of at least the previously selected objects and background pieces, and can increase the average performance of the window areas selected as the background pieces through repetitive machine learning. You can increase the accuracy.

The learning processor 810 may recognize the performance of the learning image, and adjust the m and n in consideration of the recognized performance.

In this way, the learning processor 810 may determine the optimized m and n for selecting the background pieces for the learning images having different resolutions and contrasts according to the photographing region, the weather environment, and the time zone.

When the actual image is input, the recognition processor 820 may perform a function of recognizing information using the object and the background fragment recorded in the machine learning DB 830.

Hereinafter, the workflow of the machine learning-based image recognition system 200 according to embodiments of the present invention will be described in detail.

9 is a flowchart illustrating a procedure of a machine learning based image recognition method according to an embodiment of the present invention.

The machine learning based image recognition method according to the present embodiment may be performed by the machine learning based image recognition system 200 described above.

Referring to FIG. 9, in step 910, the image recognition system 200 records a background fragment selected by processing a learning image in a machine learning DB.

That is, the image recognition system 200 extracts at least one window region that is a candidate for background fragments from the background region of the training image, and selects a window region of which the recognition accuracy is high among the extracted window regions. It can be selected as a possible background piece.

For example, referring to FIGS. 3A and 3B, the image recognition system 200 may include four window regions having a rectangular shape having a plurality of parameters shown in FIG. 3A from the training image 320 illustrated in FIG. 3B. You can extract Window 1, Window 2, Window 3, and Window 4).

In addition, the image recognition system 200 extracts a window region to include a portion different from the background region having different brightness by sunlight or the like in the background region ('moon surface'), or a portion having a different shade such as a crater (pit). After extracting the window area to include, the remaining window area may be extracted according to a positional relationship or a distance relationship with another window area by using the previously extracted window area.

In addition, when the image recognition system 200 extracts a predetermined number ('four') of window regions from the background region of the training image, the accuracy calculated for each window region exceeds the minimum reference value. The window area can be selected as the background fragment.

The accuracy may be calculated according to a calculation formula based on one of YOLO, SSD, and Faster RCNN, which are one of deep learning based FAST object discovery techniques.

The image recognition system 200 associates at least one piece of data of a parameter, a size, a shape, a position in an image, a direction, an object, or another background fragment of the selected background fragment with the background fragment and machine learning DB 220. Can be written on.

Similarly, the image recognition system 200 associates at least one object data among the size, shape, position, direction, and relationship with other objects of the object extracted by processing the learning image and the machine learning DB 220. Can be written on.

In operation 920, the image recognition system 200 determines whether an actual image is input. If the actual image is not input, step 920 is repeated to wait for input of the actual image.

When an actual image is input, in operation 930, the image recognition system 200 determines whether the background fragment is included in the actual image.

When the background fragment is included in the real image, in operation 940, the image recognition system 200 generates information recognized from the real image by using the data about the background fragment and outputs it to the screen.

For example, referring to FIG. 5, the image recognition system 200 determines whether a background fragment exists in the input real image 510, and if a background fragment exists in the real image 510, the image recognition system 200 corresponds to the background fragment. The recorded data 520 may be retrieved from the machine learning DB 220 and output on the screen.

In this way, the image recognition system 200 may recognize navigation information such as position and direction by using the learned background fragments even when the object is not extracted from the input real image.

In addition, the image recognition system 200 may calculate the position of each object more accurately by using the relationship information between the object and the background pieces, and acquire motion information of the camera based on this.

In addition, the image recognition system 200 may combine the data on the background pieces with the object data on the object ('plane'), and output the information on the screen as information recognized in relation to the actual image.

As described above, according to the present invention, it is possible to improve the recognition performance of the input real image by selecting and machine learning beneficial background pieces that can act as objects in the learning image and combining them with the object.

Method according to an embodiment of the present invention can be implemented in the form of program instructions that can be executed by various computer means may be recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

200: image recognition system
210: learning processing unit
220: machine learning DB
230: recognition processing unit

Claims (18)

Recording the selected background fragment by processing the learning image in a machine learning DB;
Determining whether the background fragment is included in an actual input image; And
If included as a result of the determination,
Retrieving data corresponding to the background fragment from the machine learning DB, and outputting recognition information related to the actual image on the screen by using the retrieved data.
Machine learning based image recognition method comprising a. The method of claim 1,
Extracting n window regions including at least a background region in the learning image, wherein n is one or more natural numbers; And
When the number of candidate regions selected in consideration of the calculated performance for each of the n window regions reaches m determined by the user (where m is a natural number of 1 or more),
Selecting each of the m candidate regions as the background fragment
Machine learning based image recognition method further comprising. The method of claim 2,
Adjusting the m if the average value of the performance of each of the m candidate areas is greater than or equal to a threshold than the minimum reference value;
Further selecting the candidate region from among window regions arranged at an upper side when the n window regions are sorted in ascending order according to the performance until the increased m is reached; And
Further selecting the additionally selected candidate region as the background fragment
Machine learning based image recognition method further comprising. The method of claim 3,
Determining the minimum reference value in consideration of the performance calculated on the object in the machine learning DB; And
Adjusting the minimum reference value by further considering performance calculated with respect to the background fragment when the background fragment is recorded in the machine learning DB;
Machine learning based image recognition method further comprising. The method of claim 2,
For each of the n window areas, the center point position error calculated at the middle or the end of the convolutional neural network layer based on the convolutional neural network, the rotation angle error of the window area based on the center point, the presence or absence of the actual photographing area, and the horizontal size error And calculating at least one of the vertical size error and the distance error between the window area to calculate the performance.
Selecting a window area whose performance satisfies a minimum reference value among the n window areas as the candidate area;
If the number of candidate regions does not reach the m number,
Deleting window areas whose performance does not satisfy the minimum reference value, and further extracting the window areas by the deleted number; And
Repeating the calculating step for the additionally extracted window region
Machine learning based image recognition method further comprising. The method of claim 5,
The calculating step,
Calculating the performance, including at least one of a detection rate regarding a ratio of a background fragment selected from the window region extracted from the training image, and an accuracy of position identification using the window region selected as the background fragment;
Machine learning based image recognition method comprising a. The method of claim 5,
Arranging the performance for each of the n window regions in ascending order using the verification data provided separately from the training image, and selecting the window region having the higher performance as the background fragment.
Machine learning based image recognition method further comprising. The method of claim 2,
Recognizing the performance of the learning image; And
Adjusting m and n in consideration of the recognized performance
Machine learning based image recognition method further comprising. The method of claim 2,
The recording step,
At least one of a unique ID, a name, a center point position, a horizontal size, a vertical size, a rotation angle of a window region, a performance, a distance between different background fragments, and an artificial neural network coefficient and structure after machine learning Recording the data in the machine learning DB corresponding to the background pieces;
Machine learning based image recognition method comprising a. The method of claim 2,
Extracting the window area,
Dividing the background area into a grid structure; And
Setting a center point position of the n window regions to be extracted from the background region divided into the grid structure, and at least one parameter of a horizontal size, a vertical size, and a rotation angle
Machine learning based image recognition method comprising a. The method of claim 1,
The outputting step,
Creating and outputting a movement of a photographing camera or an object as the recognition information based on at least one of a position, a size, a direction, a center point, and another distance between the background pieces with respect to the background pieces;
Machine learning based image recognition method comprising a. The method of claim 1,
When an object is extracted from the real image,
The outputting step,
Creating and outputting the recognition information by combining the object data recorded about the object with the data on the background pieces;
Machine learning based image recognition method comprising a. A learning processor that records the selected background pieces by processing the learning image in a machine learning DB; And
It is determined whether the background fragment is included in the actual image to be input. If the determination result is included, the data corresponding to the background fragment is retrieved from the machine learning DB, and the searched data is used to search for the background image. Recognition processing unit for outputting the recognition information on the screen
Machine learning based image recognition system comprising a. The method of claim 13,
The learning processing unit,
Extracting n window regions including at least a background region in the training image, wherein n is a natural number of 1 or more;
When the number of candidate regions selected in consideration of the calculated performance for each of the n window regions reaches m determined by the user (where m is a natural number of 1 or more),
Each of the m candidate regions is selected as the background fragment.
Machine learning based image recognition system. The method of claim 14,
The learning processing unit,
If the value obtained by averaging the performance of each of the m candidate areas is greater than or equal to the threshold value, the m value is adjusted to increase.
Further selecting the candidate region from among window regions arranged at an upper side when the n window regions are sorted in ascending order according to the performance until the increased m is reached,
Selecting the additionally selected candidate region as the background fragment
Machine learning based image recognition system. The method of claim 15,
The learning processing unit,
The minimum reference value is determined in consideration of the performance calculated for the object in the machine learning DB, and when the background piece is recorded in the machine learning DB, the minimum reference value is further considered in consideration of the performance calculated for the background fragment. To adjust
Machine learning based image recognition system. The method of claim 14,
The learning processing unit,
For each of the n window areas, the center point position error calculated at the middle or the end of the convolutional neural network layer based on the convolutional neural network, the rotation angle error of the window area based on the center point, the presence or absence of the actual photographing area, and the horizontal size error And calculating at least one of the vertical size error and the distance error between the window area to calculate the performance,
Selecting a window area whose performance satisfies a minimum reference value among the n window areas as the candidate area,
If the number of candidate regions does not reach the m number,
Deleting the window area whose performance does not satisfy the minimum reference value, and extracting the window area as much as the deleted number;
For the additionally extracted window area, calculate the performance
Machine learning based image recognition system. The method of claim 13,
The recognition processing unit,
Based on at least one piece of data of the position, size, direction, center point, and distance between the other background fragments with respect to the background fragments, the motion of the photographing camera or the object is created and output as the recognition information.
Machine learning based image recognition system.

Images