KR102522399B1 - Method for detecting object and system therefor - Google Patents

Abstract

The present invention relates to an object identification method for recognizing an object based on a feature map extracted using a convolutional neural network (CNN) and a system therefor. A method for identifying an object in an image analysis system according to an embodiment of the present invention includes extracting a plurality of reference feature maps of different sizes from an original image by performing a plurality of convolution operations on an original image; checking regions where objects are displayed for each of the reference feature maps, and setting a plurality of candidate regions having different shapes in an original image based on the checked expression regions; calculating a probability that the object is displayed for each of the candidate regions and calculating an overlapping ratio between each candidate region; and recognizing a region where an object is located in an original image based on the calculated object presentation probability and overlap ratio.

Description

Method for detecting object and system therefor}

The present invention relates to a method for identifying an object through image analysis, and more particularly, to a method for identifying an object based on a feature map extracted using a convolutional neural network (CNN), and a system therefor. It is about.

A technology for recognizing an object by analyzing an image received from a camera has been developed and used. As such an object recognition technology, a vehicle number recognition technology and a user recognition technology may be representatively placed.

In order to accurately recognize a user, convolutional neural network (CNN) technology is used. The convolutional neural network technology constructs a feature map for an image through a plurality of convolution and pooling steps, and recognizes a desired object in an image using the feature map.

Object recognition technology using such a convolutional neural network is mainly used in an environment in which a fixed camera is mounted. That is, in object recognition technology using a convolutional neural network, a camera is installed at a certain position and height, and an object is recognized by analyzing an image received from the camera at a certain shooting angle. However, since this object recognition method takes an image captured in a limited range of shooting angles as an analysis target, accuracy and recognition speed are degraded when the shooting angle deviates from the existing shooting angle. In other words, in an environment where images are received from various angles, such as from a mobile camera and a wearable camera, when the existing object recognition technology is applied, the accuracy of the object recognition result is not high and it takes a lot of time to detect the object. There is a problem.

The present invention has been proposed to solve these conventional problems, and an object of the present invention is to provide an object identification method and a system therefor capable of quickly and accurately identifying an object even in an environment where a mobile camera is applied.

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by the examples of the present invention. It will also be readily apparent that the objects and advantages of the present invention may be realized by means of the instrumentalities and combinations indicated in the claims.

In order to achieve the above object, a method for identifying an object in an image analysis system according to a first aspect of the present invention performs a plurality of convolution operations on an original image to obtain a plurality of reference feature maps having different sizes from the original image. Extracting from; checking regions where objects are displayed for each of the reference feature maps, and setting a plurality of candidate regions having different shapes in an original image based on the checked expression regions; calculating a probability that the object is displayed for each of the candidate regions and calculating an overlapping ratio between each candidate region; and recognizing a region where an object is located in an original image based on the calculated object presentation probability and overlap ratio.

The recognizing may include recognizing a region having the highest probability among overlapping candidate regions as the region where the object is located, when an overlapping ratio between the candidate regions exceeds a preset threshold ratio.

In the recognizing, if there are a plurality of candidate regions having an object presentation probability exceeding a preset threshold and an overlap ratio of each of the candidate regions is equal to or less than a preset threshold, the object is located in the plurality of candidate regions. area can be recognized.

The setting of the plurality of candidate regions in the original image may include: identifying regions in the original image that match regions in which objects of the reference feature map are expressed, and calculating center coordinates of the identified regions; and setting a plurality of candidate regions having different shapes based on the center coordinates in the original image.

The method may include, after the recognizing step, generating direction information indicating a direction to the recognized object from a reference location; and providing the generated direction information to a user.

The extracting of the plurality of reference feature maps from the original image may include extracting a reference feature map having a minimum size from the original image by performing a plurality of convolution operations on the original image; Upsampling a feature map used as an input value of a most recently used convolution operation, and extracting a new feature map by performing a concatenation operation between the upsampled feature map and a previously extracted feature map; and extracting one or more reference feature maps having a size greater than the reference feature map of the minimum size from the original image by performing a convolution operation on the feature maps extracted through the concatenation operation.

According to a second aspect of the present invention for achieving the above object, an image analysis system for analyzing an image to identify an object includes: an image receiving unit for receiving an original image; a feature map extraction unit extracting a plurality of reference feature maps having different sizes from the original image by performing a plurality of convolution operations on the original image; and a region in which an object is displayed for each reference feature map is identified, a plurality of candidate regions having different shapes are set in an original image based on the identified expression region, and a probability that the object is represented for each of the candidate regions and an object recognizing unit that recognizes a region where an object is located in an original image based on the calculated object expression probability and overlap ratio after calculating an overlapping ratio between candidate regions.

The present invention has the advantage of analyzing an image received from a mobile camera, quickly and accurately detecting an object in the image, and providing a service according to the object confirmation to the user in real time.

In addition, the present invention analyzes different reference feature maps to select a plurality of candidate regions of different shapes, and according to the overlapping degree and probability of the selected candidate regions, single or multiple objects in the original image By recognizing , there is an advantage in accurately identifying not only large objects but also small objects.

In addition, the present invention has an effect of providing convenience for the user to move to the object in a real space by providing the direction information of the object to the user.

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and serve to further understand the technical idea of the present invention together with specific details for carrying out the invention, so the present invention is described in such drawings should not be construed as limited to
1 is a diagram illustrating an environment to which an image analysis system according to an embodiment of the present invention is applied.
2 is a diagram showing the configuration of an image analysis system according to an embodiment of the present invention.
3 is a flowchart illustrating a method of generating a reference feature map in an image analysis system according to an embodiment of the present invention.
4 is a flowchart illustrating a method of identifying an object in an image analysis system according to an embodiment of the present invention.
5 is a diagram illustrating a plurality of candidate regions.
6 is a diagram illustrating a region in which an object is recognized.
7 is a diagram illustrating an image of recognizing a person indoors.
8 is a diagram illustrating that an image analysis system is applied to a parking management service.

The above-described objects, features and advantages will become more apparent through the following detailed description in conjunction with the accompanying drawings, and accordingly, those skilled in the art to which the present invention belongs can easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an environment to which an image analysis system according to an embodiment of the present invention is applied.

As shown in FIG. 1 , the video analysis system 200 according to an embodiment of the present invention communicates with each of a plurality of cameras 120 and user terminals 110 through a network 300 . The network 300 includes a local area wireless communication network such as Wi-Fi, a 4G mobile communication network, a 5G mobile communication network, and a wired communication network.

The user terminal 110 is a mobile communication device such as a smart phone or a tablet computer, and may receive direction information of an object from the image analysis system 200 . In addition, the user terminal 110 may capture an image through a built-in camera and transmit the image to the image analysis system 200 .

The camera 120 is a movable image capture device, for example, a network camera. The camera 120 may capture a surrounding image and transmit it to the image analysis system 200 . The camera 120 may be mounted on a wearable device worn by a user. Alternatively, the camera 120 may be a camera capable of capturing 360 degrees. The camera 120 may divide the entire image into a plurality of pieces (eg, into quarters) and capture the image, and transmit the divided image to the image analysis system 200 .

The image analysis system 200 analyzes the image received from the user terminal 110 or the camera 120 and detects and recognizes an object in the image. The image analysis system 200 performs convolution operations on the received original image several times to generate a plurality of reference feature maps of different sizes, and uses the reference feature maps to determine whether an object in the original image is area can be recognized.

2 is a diagram showing the configuration of an image analysis system according to an embodiment of the present invention.

As shown in FIG. 2, the video analysis system 200 includes an image receiver 210, a screen divider 220, a feature map extractor 230, an object recognizer 240, a direction notification unit 250, and It includes the database 260, and these components may be configured with hardware or software, or implemented through a combination of hardware and software.

In addition, the image analysis system 200 may include one or more processors and memory, and may include an image receiver 210, a screen divider 220, a feature map extractor 230, an object recognizer 240, and a direction notification. The unit 250 may be loaded (stored) in the memory in the form of a program executed by the processor.

The database 260 is a mass storage means such as storage and stores various data such as image data. Also, the database 260 may store object recognition information and may store building drawings. The database 260 may store coordinate reference data matching the region of each reference feature map with the coordinate region of the original image. Here, the reference feature map means a feature map used as a reference for object detection among feature maps. In addition, the database 260 may store object pattern information representing objects such as people, animals, license plates, and the like.

The image receiver 210 receives image data from either the camera 120 or the user terminal 110 . The image receiver 210 may store the received image in the database 260 . The image receiving unit 210 may receive divided images from the camera 120 as images.

When an image is received from the camera 120 or the user terminal 110, the screen divider 220 divides the image into a plurality of regions of a preset size. The screen division unit 220 may not perform the division of the screen by itself when an already divided image is received from the camera 120 . In addition, when an already divided image is received from the camera 120, the screen divider 220 may change the resolution of the divided image if there is an image that does not satisfy a preset resolution.

The feature map extractor 230 extracts a plurality of feature maps by performing a multi-step convolution operation on the segmented original image, and selects a plurality of reference feature maps having different sizes from among the extracted feature maps. . Details of selecting a reference feature map in the feature map extractor 230 will be described later with reference to FIG. 3 .

The object recognizing unit 240 recognizes objects such as people, license plates, and animals in the original image by analyzing reference feature maps having different sizes. The object recognizing unit 240 analyzes each reference feature map, sets a plurality of candidate regions where the object is predicted to be located in the original image, and based on the probability and overlapping degree of each candidate region, the object among the candidate regions. Identify one or more regions where is located. Details will be described later with reference to FIG. 4 .

The direction notification unit 250 compares the current location of the user terminal 110 with the location of the object recognized by the object recognizing unit 240, and generates direction information in which the object is located based on the user terminal 110. , transmits the generated direction information to the user terminal 110. At this time, the direction notification unit 250 may check the building drawing where the user is located in the database 260 based on the location of the user terminal 110 and transmit the building drawing and direction information to the user terminal 110 together.

3 is a flowchart illustrating a method of generating a reference feature map in an image analysis system according to an embodiment of the present invention.

Referring to FIG. 3 , the image receiving unit 210 receives an image captured by the camera 120 or the user terminal 110 (S301). When the image receiver 210 receives an image from the camera 120 or the user terminal 110, the screen divider 220 may divide the image into a preset size.

Then, the feature map extractor 230 sets the image divided by the screen divider 220 or the image received from the image receiver 210 as an original image, and uses the original image as an input value to the first convolution layer. input, and a first feature map is extracted from the original image (S303). At this time, a preset filter (eg, 3×3 filter) and a preset stride may be applied to the first convolution layer, and the feature map extractor 230 does not perform pooling. A reduced first feature map compared to the original image may be extracted from the original image using the first convolution layer and the stride. The feature map extractor 230 may extract a first feature map by repeatedly performing convolution on the original image using a plurality of different first convolution layers. For example, in the first convolution operation, the original image is input to the first convolution layer having the first filter, and in the second operation, the result value of the immediately preceding first convolution layer may be input as an input value. In this case, the filter of the first convolution layer may be different from or the same as the filter used immediately before.

Next, the feature map extractor 230 extracts a second feature map by performing a convolution operation on the extracted first feature map as an input value (S305). That is, the feature map extractor 230 inputs the first feature map as an input value of the 2-1 convolution layer, extracts the 2-1 feature map and the 2-2 feature map, and extracts the extracted 2-1 feature map. A second feature map is extracted by summing the feature map and the 2-2 feature map. A 1×1 filter may be applied to the 2-1 convolution layer, and a 3×3 filter and a padding operation may be applied to the 2-2 convolution layer. In addition, the 2-1 convolutional layer is repeatedly used a preset number of times to extract the 2-1 feature map, and the 2-2 convolutional layer is repeatedly used a preset number of times A 2-2 feature map may be extracted. When the 2-1 convolution layer is repeatedly used, the first feature map is input as an input value to the 2-1 convolution layer in the first operation, and from the second operation, the immediately preceding operation result value is the second-1 convolution layer. It is input as an input value of 1 convolution layer. Similarly, when the 2-2 convolution layer is repeatedly applied, the first feature map is input as an input value to the 2-2 convolution layer in the first operation, and the immediately preceding operation result value is provided in the second operation. It is input as the input value of the 2-2 convolution layer.

Next, the feature map extractor 230 extracts a third feature map from the second feature map by pooling the second feature map (S307). In this case, the feature map extractor 230 may generate a third feature map by reducing the second feature map using max pooling or mean pooling. Due to pooling, the third feature map is smaller in size than the second feature map.

Then, the feature map extractor 230 extracts a fourth feature map by inputting the pooled and reduced third feature map as an input value to a third convolution layer, and converts the fourth feature map into a first reference feature map. Select (S309). In this case, a 1×1 filter may be applied to the third convolution. The first reference feature map has the smallest size among reference feature maps.

Next, the feature map extractor 230 performs first upsampling using the second feature map and the third feature map used as input values to the last convolution (ie, third convolution) layer to obtain A fifth feature map enlarged in size from the first reference feature map is extracted (S311).

Next, the feature map extractor 230 performs a concatenation operation on the primary upsampled first feature map and the second feature map, and then performs a convolution operation on the result of the concatenation operation to obtain a second reference feature map. is selected (S313). That is, the feature map extractor 230 performs a concatenation operation on the fifth feature map and the second feature map to extract a 5-1 feature map, and uses the extracted 5-1 feature map as a fourth convoluted feature map. A sixth feature map is extracted by inputting it to the solution layer as an input value, and this sixth feature map is selected as the second reference feature map. A 1×1 filter or a 2×2 filter may be applied to the fourth convolution layer.

Subsequently, the feature map extractor 230 performs secondary upsampling on the 5-1 feature map used as an input value to the last convolution (ie, 4th convolution) layer so that the feature map is larger than the second reference feature map. An enlarged seventh feature map may be extracted (S315). Further, the feature map extractor 230 performs a concatenation operation on the second upsampled seventh feature map and the first feature map, and performs a convolution operation on the result of the concatenation operation to select a third reference feature map. Do (S317). That is, the feature map extractor 230 extracts a 7-1 feature map by performing a concatenation operation on the first feature map and the seventh feature map, and converts the extracted 7-1 feature map into a fifth convolve. After extracting the eighth feature map by inputting it to the solution layer as an input value, the eighth feature map is selected as the third reference feature map. A 1×1 filter or a 2×2 filter may be applied to the fifth convolution layer.

As described above, the feature map extractor 230 reduces the feature map through convolution or pooling, selects the smallest first reference feature map, and uses the first and second upsampled feature maps, A medium-sized second reference feature map and a maximum-sized third reference feature map are selected.

Meanwhile, in the above-described embodiment, the number of upsampling is described as being twice, but the number of upsampling can be flexibly changed in proportion to the number of times the original image is reduced, and the number of reference feature maps is also equal to that of the original image. It can be changed in proportion to the number of reductions.

4 is a flowchart illustrating a method of identifying an object in an image analysis system according to an embodiment of the present invention.

5 is a diagram illustrating a plurality of candidate regions.

Referring to FIGS. 4 and 5 , the object recognizing unit 240 recognizes an area indicated by an object in a plurality of reference feature maps having different sizes (S401). At this time, the object recognizer 240 compares the data of the reference feature map with the object pattern stored in the database 260, determines whether or not there is a region represented by the object for each reference feature map, and identifies the region represented by the object as the reference feature map. you can check very much.

Next, when it is confirmed that the object exists in the reference feature map, the object recognition unit 240 checks the expression area of the object recognized in the reference feature map in the original image, and determines the expression area of the object identified in the original image. The center coordinates are checked (S403). The object recognizing unit 240 checks the database 260 for coordinate reference data matching the region of the reference feature map and the coordinate region of the original image, and determines the region where the object is located in the original image based on the checked coordinate reference data. Check, and calculate the coordinates of the center for this area. In other words, since the size of the reference feature map and the actual original image are different, the object recognition unit 240 refers to the coordinate reference data in order to match and recognize the expression area of the object recognized from the reference feature map in the original image. .

Subsequently, when an object is recognized in one or more reference feature maps, the object recognizing unit 240 sets a plurality of candidate regions having different shapes in the original image, centered on the coordinates of the center of the object recognized in the original image (S405). ). In this case, the object recognizing unit 240 may determine the size of each candidate region based on the size of the reference feature map from which the object is recognized. That is, the object recognizer 240 may set candidate regions of different sizes in the original image for each reference feature map that is a basis for object recognition.

To elaborate, a difference in the success rate of recognizing an object may occur depending on the size of the reference feature map. That is, when an object is recognized using the first reference feature map having a small size, the success rate of recognizing a small-looking object in the original image is reduced, but the success rate of recognizing a large-sized object is increased. On the other hand, when recognizing an object using the third reference feature map having the largest size, the recognition success rate for an object that looks small in the original image may be high, but the success rate of completely recognizing a region of a large object is may appear relatively low. In this way, since the sizes of the reference feature maps are different from each other and the recognition rate is different according to the size of the object, the candidate region of the object recognized through the first reference feature map having the smallest size is set to be the largest in the original image and has the largest size. A candidate region of an object recognized through the large third reference feature map is set to the smallest size in the original image.

In FIG. 5, the center coordinates of an object (ie, a person) recognized through the first reference feature map are indicated by reference numeral 51, and the center coordinates of objects recognized based on the second reference feature map are indicated by reference numeral 52. The center coordinates of the object recognized based on the feature map are exemplified by reference numeral 53. Referring to FIG. 5, three candidate regions 51a, 51b, and 51c having the largest area based on the center coordinate of reference numeral 51 are set on the screen, and have an intermediate size based on the center coordinate of reference numeral 52. Three candidate regions 52a, 52b, and 52c having ? are set on the screen. In addition, three candidate regions 53a, 53b, and 53c having the smallest area based on the center coordinate of reference numeral 53 are set on the screen. When the first reference feature map and the second reference feature map are used, the largest object (ie, a person) may be recognized, but a small object may not be recognized. In addition, when a plurality of candidate regions 51a, 51b, 51c, 52a, 52b, 52c, 53a, 53b, and 53c set using each reference feature map are set in the original image, the candidate regions 51a, 51b, and 51c , 52a, 52b, 52c, 53a, 53b, 53c) may differ from each other in that the object is fully expressed. That is, an object may be completely expressed in a specific candidate region, but only a part of the object may be expressed in another candidate region.

Referring back to FIG. 4 , the object recognizing unit 240 calculates a probability that an object is displayed in each candidate region (S407). The probability is a numerical value of the probability that an object can appear in the candidate area intact, and can be set from 1 to 0. For example, if all objects appear in a specific candidate region, the probability of the candidate region may be calculated as 1, and if the object does not appear in the candidate region at all, the probability may be set to 0, and the object may be partially present in the candidate region. When is expressed, the probability can be calculated between 0 and less than 1 according to the expressed ratio. Referring to the candidate regions of FIG. 5 as an example, the object recognizing unit 240 may calculate probabilities for each of nine candidate regions.

The object recognizing unit 240 leaves candidate regions having a probability greater than or equal to a preset threshold value (eg, 0.5) among each candidate region in the original image, and candidate regions having a probability less than the set value in the original image. The candidate region is filtered by removing it (S409).

Next, when the filtering of the candidate regions is completed, the object recognition unit 240 determines whether there is an overlapping region between the candidate regions (S411), and if there are overlapping candidate regions, the overlapping ratio of the corresponding candidate regions is determined. It is determined whether or not a preset threshold ratio (eg, 50%) is exceeded (S413).

The object recognizing unit 240 sets the candidate regions as an overlapping group, when a plurality of overlapping candidate regions exist with an overlapping ratio exceeding a threshold ratio. The object recognizing unit 240 recognizes a candidate region having the highest object expression probability among the overlapping groups as a region where an object exists (hereinafter, referred to as an 'object region'), and removes the remaining candidate regions from the original image ( S415). FIG. 6 is a diagram illustrating a region in which an object is recognized. Referring to FIGS. 5 and 6 , a region having the highest probability among overlapping candidate regions is finally selected.

Meanwhile, the object recognition unit 240 recognizes each of the candidate regions appearing on the screen as an object region when a plurality of overlapping candidate regions does not exceed a threshold ratio (S419). In this case, the object recognizing unit 240 may recognize a plurality of object areas from the original image as shown in FIG. 6 .

When the object area is recognized, the object recognizing unit 240 analyzes the object represented in the object area to identify the state of the object (eg, upright, bent, sitting, etc.) or to identify the object, or identify the object. can also be tracked.

In addition, when recognizing an object, the direction notification unit 250 may generate direction information representing the direction of the recognized object from the reference location and transmit the direction information to the user terminal (S417). In this case, the reference location may be the location of the user terminal 110 requesting the location of the object. As an embodiment, the direction notification unit 250 may receive a message requesting a location of an object from the user terminal 110, and interwork with a known positioning system (not shown) to determine the location of the user terminal 110. After measuring the location, the direction in which the recognized object exists based on the location of the user terminal 110 may be checked, and information about this direction may be transmitted to the user terminal 110 . In this case, the direction notification unit 250 extracts a drawing in which the user terminal 110 is located based on the location of the user terminal 110 from the database 260, and transmits direction information together with the drawing to the user terminal 110. can be sent to At this time, the direction notification unit 250 may additionally display the location of the object on the drawing.

7 is a diagram illustrating an image of a person recognized indoors, and the image analysis system 200 according to the present invention can accurately recognize even a person measured far from the camera, and the person's state (ie, upright, sitting) etc.) can be identified.

8 is a diagram illustrating that an image analysis system is applied to a parking management service.

Referring to FIG. 8 , the video analysis system 200 may recognize a vehicle license plate as an object, and also provide direction information 81 of the object and an indoor drawing based on the user's location to the user terminal 110 of the user. can do.

While this specification contains many features, such features should not be construed as limiting the scope of the invention or the claims. Also, features described in separate embodiments in this specification may be implemented in combination in a single embodiment. Conversely, various features that are described in this specification in a single embodiment may be implemented in various embodiments individually or in combination as appropriate.

Although actions are described in a particular order in the drawings, it should not be understood that such actions are performed in the specific order as shown, or that the actions are performed in a series of sequential order, or that all described actions are performed to achieve a desired result. . Multitasking and parallel processing can be advantageous in certain circumstances. In addition, it should be understood that the division of various system components in the above-described embodiments does not require such division in all embodiments. The program components and systems described above may generally be implemented as a package in a single software product or multiple software products.

The method of the present invention as described above may be implemented as a program and stored in a recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.) in a computer-readable form. Since this process can be easily performed by a person skilled in the art to which the present invention belongs, it will not be described in detail.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention to those skilled in the art to which the present invention belongs, and thus the above-described embodiments and It is not limited by drawings.

110: user terminal 120: camera
200: video analysis system 210: video receiver
220: screen division unit 230: feature map extraction unit
240: object recognition unit 250: direction notification unit
260: database 300: network

Claims (13)

As a method for identifying an object in an image analysis system,
extracting a plurality of reference feature maps having different sizes by performing a plurality of convolution operations in which an output value of a previous convolution operation is input as an input value of a next convolution operation using an original image as an initial input value;
Identifying a region in which an object is displayed for each of the reference feature maps, and setting a plurality of candidate regions having different shapes in an original image for each reference feature map based on coordinates of the identified expression region and coordinate reference data. ;
calculating a probability that the object is displayed for each of the candidate regions and calculating an overlapping ratio between each candidate region; and
Recognizing a region where an object is located in an original image based on the calculated object expression probability and overlap ratio;
The size of each candidate region set in the original image is determined according to the size of a reference feature map based on object recognition, and candidate regions based on the largest reference feature map are candidate regions based on other reference feature maps. and candidate regions based on the smallest reference feature map are larger than candidate regions based on other reference feature maps. According to claim 1,
The step of recognizing is
and recognizing an area having the highest probability among the overlapping candidate areas as an area where an object is located when an overlapping ratio between the candidate areas exceeds a preset threshold ratio. According to claim 1,
The step of recognizing is
If there are a plurality of candidate regions having an object presentation probability exceeding a preset threshold, and the overlapping ratio of each candidate region is equal to or less than a preset threshold ratio, the plurality of candidate regions are recognized as regions where objects are located. How to check an object with . According to claim 1,
The step of setting the plurality of candidate regions in the original image,
checking an area in the original image that matches an area where an object of the reference feature map is displayed, and calculating center coordinates of the checked area; and
and setting a plurality of candidate regions having different shapes in the original image for each reference feature map based on the center coordinates. delete According to claim 1,
After the recognition step,
generating direction information indicating a direction to the recognized object based on a location of a user terminal requesting the location of the object; and
Providing the generated direction information to the user terminal; object confirmation method comprising a. According to claim 1,
The step of extracting the plurality of reference feature maps,
extracting a reference feature map having a minimum size by performing a plurality of convolution operations in which an output value of a previous convolution operation is input as an input value of a next convolution operation using the original image as an initial input value;
Upsampling a feature map used as an input value of a most recently used convolution operation, and extracting a new feature map by performing a concatenation operation between the upsampled feature map and a previously extracted feature map; and
and extracting at least one reference feature map having a larger size than the reference feature map of the minimum size by performing a convolution operation on the feature map extracted through the concatenation operation. An image analysis system that analyzes an image to identify an object,
an image receiving unit for receiving an original image;
Feature map extraction to extract a plurality of reference feature maps of different sizes by performing a plurality of convolution operations in which the output value of the previous convolution operation is input as an input value of the next convolution operation using the original image as an initial input value wealth; and
A region in which an object is expressed is checked for each reference feature map, and a plurality of candidate regions having different shapes are set in the original image for each reference feature map based on the coordinates of the confirmed expression region and the coordinate reference data. An object that calculates the probability that the object is displayed for each candidate region, calculates the overlap ratio between each candidate region, and then recognizes the region where the object is located in the original image based on the calculated object expression probability and overlap ratio. Recognition unit; including,
The object recognition unit,
The size of each candidate region set in the original image is determined according to the size of a reference feature map based on object recognition, and candidate regions based on the largest reference feature map are candidate regions based on other reference feature maps. An image analysis system that sets candidate regions based on a reference feature map having the smallest size to be larger than candidate regions based on other reference feature maps. According to claim 8,
The object recognition unit,
When the overlapping ratio of the candidate regions exceeds a preset threshold ratio, the image analysis system, characterized in that for recognizing a region having the highest probability among the overlapping candidate regions as the region where the object is located. According to claim 8,
The object recognition unit,
If there are a plurality of candidate regions having an object presentation probability exceeding a preset threshold, and the overlap ratio of each candidate region is less than or equal to a preset threshold ratio, image analysis recognizing the plurality of candidate regions as regions where objects are located. system. According to claim 8,
The object recognition unit,
In the original image, a region that matches the region where the object of the reference feature map is displayed is checked, and after calculating the coordinates of the center of the checked region, a plurality of images having different shapes for each reference feature map are calculated based on the center coordinates. An image analysis system, characterized in that for setting a candidate region in the original image. According to claim 8,
A video further comprising a; direction notification unit generating direction information indicating a direction to the recognized object based on the location of the user terminal requesting the location of the object, and providing the generated direction information to the user terminal. analysis system. According to claim 8,
The feature map extractor,
Using the original image as the first input value, a plurality of convolution operations are performed in which the output value of the previous convolution operation is input as the input value of the next convolution operation to extract a reference feature map of the minimum size, and the last used convolution Upsampling the feature map used as the input value of the convolution operation, extracting a new feature map by performing a concatenation operation between the upsampled feature map and the previously extracted feature map, and then performing a convolution operation on the feature map extracted through the concatenation operation. The image analysis system, characterized in that for extracting one or more reference feature maps having a larger size than the reference feature map of the minimum size.

Images