JP7287650B2 - Image recognition device, image recognition method, and program - Google Patents

Description

The present invention relates to an image recognition apparatus and image recognition method for recognizing an object from an image, and further to a program for realizing these.

In recent years, an image recognition device that performs image recognition using a machine learning model has been developed (see Patent Documents 1 and 2, for example). According to such an image recognition device, it is possible to detect pre-learned persons, animals, automobiles, etc. from images. For this reason, the image recognition device is used in a video monitoring system, an accident prevention system mounted on a vehicle, and the like.

Here, processing in a conventional image recognition apparatus will be described with reference to FIG. FIG. 11 is a flowchart showing processing performed by a conventional image recognition apparatus. This image recognition device also has a machine learning model for recognizing specific objects. Machine learning models are built by deep learning.

As shown in FIG. 11, the image recognition device first acquires image data from an external imaging device or storage device (step S1). The acquired image data is stored in a memory or the like installed in the image recognition device.

Next, the image recognition device designates a range of a portion of the obtained image that may contain the detection target (step S2). Specifically, in step S2, the image recognition device designates the range by designating the coordinates of the upper left corner of the screen, the width of the region, and the height of the region. Also, in step S2, the entire image may be designated instead of the partial designation of the acquired image.

Next, the image recognition apparatus sets a rectangular area in which horizontal and vertical resolutions are preset within the range specified in step S2, and cuts out an image of the set area (step S3). ). Step S3 is repeatedly executed a plurality of times as described later. Also, the setting of the rectangular area is performed while sliding the position by the set number of pixels each time it is executed. This method is called a sliding method, and the rectangular area is called a sliding window.

Also, in this method, the sliding window starts from the upper left corner of the specified range and is first slid horizontally by a set number of pixels. At the position where it is slid for several minutes, it is slid further from the left end to the right end. Also, the number of pixels to be set as the amount of sliding is set so that the edge portions of positionally adjacent sliding windows overlap.

Next, the image recognition device inputs the image cut out in step S3 to the machine learning model, performs inference on the object in the image, and calculates a score indicating the likelihood that the object is a specific object. (step S4).

Next, the image recognition device compares the score calculated in step S4 with the score previously calculated in step S4 for another sliding window. Then, the image recognition device saves the score with the higher value, the coordinates of the sliding window with the higher score, and the image identification number of this sliding window (step S5).

Next, the image recognition device determines whether or not steps S3 to S5 have been performed for all the ranges specified in step S2 (step S6). As a result of the determination in step S6, if steps S3 to S5 have not been executed for all the ranges specified in step S2, the image recognition apparatus slides the sliding window and performs step S3 again, as described above. Execute.

On the other hand, as a result of the determination in step S6, if steps S3 to S5 have been executed for all the ranges specified in step S2, the image recognition device stores the saved score, coordinates, and image identification number. is output to the outside.

As described above, in the conventional image recognition apparatus, inference using a learning model is performed for each sliding window, and image recognition is performed.

JP 2012-243155 A JP 2014-041427 A

However, the conventional image recognition apparatus has a problem that the processing efficiency is low and it is difficult to improve the processing speed. Specifically, the conventional image recognition device performs inference for each sliding window as described above. Each sliding window is set so as to overlap another adjacent sliding window. For this reason, the conventional image recognition apparatus redundantly executes inference for the overlapping portion, and performs useless processing. As a result, the above-mentioned problem arises.

An example of an object of the present invention is to provide an image recognition apparatus, an image recognition method, and a program capable of solving the above problems and improving processing efficiency in image recognition using a machine learning model.

In order to achieve the above object, an image recognition device according to one aspect of the present invention includes:
a feature map generation unit that generates a feature map of an image to be recognized by using a convolutional layer of a machine learning model for detecting a specific object from an image;
A virtual window is set on the feature map, and while the window is slid by a set amount, the area in the window of the feature map is displayed at a plurality of predetermined positions by the full coupling of the machine learning model. a score calculation unit that inputs to the layer and calculates a score indicating the possibility that the specific object exists in the region for each of the predetermined positions;
characterized by comprising

Further, in order to achieve the above object, an image recognition method according to one aspect of the present invention includes:
(a) using a convolutional layer of a machine learning model for detecting specific objects from an image to generate a feature map of the image to be image recognized;
(b) setting a virtual window on the feature amount map, and sliding the window by a set amount, at a plurality of predetermined positions, the area within the window of the feature amount map, the machine learning model; inputting into the fully connected layer of and calculating, for each of the predetermined positions, a score indicating the possibility that the specific object exists in the region;
characterized by having

Furthermore, in order to achieve the above object, the program in one aspect of the present invention is
to the computer,
(a) using a convolutional layer of a machine learning model for detecting specific objects from an image to generate a feature map of the image to be image recognized;
(b) setting a virtual window on the feature amount map, and sliding the window by a set amount, at a plurality of predetermined positions, the area within the window of the feature amount map, the machine learning model; inputting into the fully connected layer of and calculating, for each of the predetermined positions, a score indicating the possibility that the specific object exists in the region;
is characterized by executing

As described above, according to the present invention, it is possible to improve processing efficiency in image recognition using a machine learning model.

FIG. 1 is a block diagram showing a schematic configuration of an image recognition device according to Embodiment 1 of the present invention. FIG. 2 is a block diagram specifically showing the configuration of the image recognition device according to Embodiment 1 of the present invention. FIG. 3 is a diagram showing an example of a feature quantity map obtained in Embodiment 1 of the present invention. FIG. 4 is a diagram showing an example of a discriminator (machine learning model) used in Embodiment 1 of the present invention. FIG. 5 is a flowchart showing the operation of the image recognition device according to Embodiment 1 of the present invention. FIG. 6 is a block diagram specifically showing the configuration of the image recognition device according to Embodiment 2 of the present invention. FIG. 7 is a diagram showing an example of a discriminator (machine learning model) according to Embodiment 2 of the present invention. FIG. 8 is a diagram showing an example of data output from the convolution layer of the discriminator according to Embodiment 2 of the present invention. FIG. 9 is a flowchart showing the operation of the image recognition device according to Embodiment 2 of the present invention. FIG. 10 is a block diagram showing an example of a computer that implements the image recognition device according to the embodiment of the present invention. FIG. 11 is a flowchart showing processing performed by a conventional image recognition apparatus.

(Embodiment 1)
An image recognition apparatus, an image recognition method, and a program according to Embodiment 1 of the present invention will be described below with reference to FIGS. 1 to 5. FIG.

[Device configuration]
First, using FIG. 1, a schematic configuration of an image recognition apparatus according to Embodiment 1 of the present invention will be described. FIG. 1 is a block diagram showing a schematic configuration of an image recognition device according to Embodiment 1 of the present invention.

An image recognition device 10 according to the first embodiment shown in FIG. 1 is a device for detecting a specific object from an image. As shown in FIG. 1 , the image recognition device 10 includes a feature quantity map generator 11 and a score calculator 12 .

The feature map generator 11 uses a convolutional layer of a machine learning model for detecting a specific object from an image to generate a feature map of an image to be recognized.

The score calculation unit 12 sets a virtual window on the feature quantity map, and while sliding this window by a set amount, at a plurality of predetermined positions, the area within the window of the feature quantity map is processed by the machine learning model. Enter the fully connected layer. Then, from the result of this input processing, the score calculation unit 12 calculates a score indicating the possibility that a specific object exists in the area within this window for each predetermined position.

As described above, in the first embodiment, first, a feature map is generated using the convolutional layer of the machine learning model, and then processing is performed using a sliding window on this feature map. For this reason, as compared with the conventional art, redundant processing is greatly reduced, so according to the first embodiment, it is possible to improve processing efficiency in image recognition using a machine learning model.

Next, the configuration and functions of the image recognition device 10 according to the first embodiment will be specifically described with reference to FIGS. 2 to 4. FIG. FIG. 2 is a block diagram specifically showing the configuration of the image recognition device according to Embodiment 1 of the present invention. FIG. 3 is a diagram showing an example of a feature quantity map obtained in Embodiment 1 of the present invention. FIG. 4 is a diagram showing an example of a discriminator (machine learning model) used in Embodiment 1 of the present invention.

As shown in FIG. 2, in the present embodiment, the image recognition apparatus 10 includes a target image setting unit 13 and a feature map storage unit 14 in addition to the feature map generation unit 11 and the score calculation unit 12 described above. , a recognition processing unit 15, and a discriminator 20, which is a machine learning model.

Note that the discriminator 20 is provided in the image recognition device 10 in the example of FIG. 2, but is not limited to this example. The discriminator 20 may be provided in a device other than the image recognition device 10 .

The target image setting unit 13 first acquires image data of an image to be subjected to image recognition. Subsequently, the target image setting unit 13 sets a range to be subjected to image recognition in the image specified by the acquired image data.

Specifically, the target image setting unit 13 specifies a range in which the recognition target may be included in the image, and sets the range. Further, the range in which the recognition target may be included is specified, for example, by detecting the contour of the object in the image and specifying the range in which the detected contour exists. Furthermore, the setting of the range is performed by setting the upper left coordinate, width and height of the set range. In addition, the target image setting unit 13 can also set the entire image of the image data as a target range for image recognition.

In this embodiment, the feature map generation unit 11 generates a feature map for the range set by the target image setting unit 13 using the convolution layer of the classifier 20 (see FIG. 4). Also, the feature map generation unit 11 stores the generated feature map in the feature map storage unit 14 . The feature map storage unit 14 is, for example, a memory, and stores feature maps in its storage area.

Specifically, as shown in FIG. 3, the feature amount map generation unit 11 first divides the image data in the range set by the target image setting unit 13 into N rows (lines) of all pixels in the horizontal direction. Take out every minute (N: any natural number). Subsequently, the feature map generation unit 11 sequentially inputs the extracted N rows of image data to the convolution layer of the discriminator 20 . Thereby, a feature quantity map is generated every N lines. FIG. 3 shows image data for N lines and a feature map generated therefrom. Also, in the feature quantity map, grids indicate pixels, and ▪ indicate feature quantities.

In the present embodiment, the score calculation unit 12 first retrieves feature maps for N lines from the feature map storage unit 14 . Subsequently, the score calculation unit 12 slides the set virtual window on the extracted N-line feature amount map by a set amount, and at a plurality of predetermined positions, the area within the window of the feature amount map. , are input to the fully connected layer of the discriminator 20 (see FIG. 4). Then, the score calculation unit 12 sets the output result of the fully connected layer for each predetermined position as the score for each predetermined position. Note that in FIG. 3, the rectangular dashed lines indicate virtual windows.

Further, as shown in FIG. 4, the discriminator 20 includes convolution layers 21 to 24 and a fully connected layer 25 . In the example of FIG. 4, image data is first input to convolutional layer 21 and the output of convolutional layer 21 is input to convolutional layer 22 and convolutional layer 24 . The convolution layer 24 performs size conversion on input data and outputs the input data after size conversion. Furthermore, the output of the convolutional layer 22 is input to the convolutional layer 23, and the output of the convolutional layer 23 and the output of the convolutional layer 24 are combined to form a feature map.

When the area within the window of the feature map is input by the score calculation unit 12, the fully connected layer 25 identifies the input area, and for each class, the object in the image corresponds to that class. Calculate the probability that The score calculation unit 12 uses the output probability as a score.

The recognition processing unit 15 first identifies the score with the largest value among the scores calculated for each predetermined position by the score calculation unit 12 and the predetermined position at that time. Then, the recognition processing unit 15 outputs the specified score and the predetermined position as a result of image recognition. Based on this output result, it can be determined whether or not a specific object exists in the image.

[Device operation]
Next, the operation of the image recognition device 10 according to Embodiment 1 will be described with reference to FIG. FIG. 5 is a flowchart showing the operation of the image recognition device according to Embodiment 1 of the present invention. 1 to 4 will be referred to as needed in the following description. Further, in Embodiment 1, the image recognition method is implemented by operating the image recognition device 10 . Therefore, the description of the image recognition method in the first embodiment is replaced with the description of the operation of the image recognition apparatus 10 below.

As shown in FIG. 5, first, the target image setting unit 13 acquires image data of an image to be subjected to image recognition (step A1). Next, the target image setting unit 13 sets a range to be subjected to image recognition in the image of the image data acquired in step A1 (step A2).

Next, the feature map generator 11 extracts image data for N lines from the image data of the range set in step A2 (step A3). Subsequently, the feature map generation unit 11 inputs the extracted image data for N lines to the convolution layer of the classifier 20 to generate a feature map (step A4). Also, the feature map generation unit 11 stores the generated feature map in the feature map storage unit 14 .

Next, the score calculation unit 12 retrieves feature maps for N lines from the feature map storage unit 14 . Then, the score calculation unit 12 inputs regions within the window of the feature map at a plurality of predetermined positions to the fully connected layer of the classifier 20 while sliding the virtual window on the extracted feature map. Then, the score is calculated (step A5).

Next, the recognition processing unit 15 identifies the score with the largest value and the predetermined position at that time from among the scores calculated for each predetermined position in step A5, and identifies the identified score and the predetermined position. position is output as a result of image recognition (step A6).

Next, the recognition processing unit 15 determines whether or not the processes of steps A3 to A6 have been completed for all the ranges set in step A2 (step A7).

As a result of the determination in step A7, if the processing of steps A3 to A6 has not been completed for all the ranges set in step A2, the recognition processing unit 15 causes the feature map generation unit 11 to execute step A3 again. . As a result, the feature map generation unit 11 extracts image data for N lines located below the previous image data.

On the other hand, if the result of determination in step A7 is that the processing of steps A3 to A6 has been completed for all the ranges set in step A2, the processing in the image recognition apparatus is completed.

[Effects of Embodiment 1]
As described above, in the first embodiment, a feature map is generated for each N lines of an image, and a score is calculated using a fully connected layer for each feature map of N lines. Therefore, unlike the conventional technique, redundant feature quantity maps are not generated. Therefore, according to the first embodiment, it is possible to improve processing efficiency in image recognition using a machine learning model.

[program]
The program in the first embodiment may be any program that causes a computer to execute steps A1 to A7 shown in FIG. By installing this program in a computer and executing it, the image recognition apparatus 10 and the image recognition method according to the first embodiment can be realized. In this case, the processor of the computer functions as a feature quantity map generation unit 11, a score calculation unit 12, a target image setting unit 13, and a recognition processing unit 15, and performs processing.

Also, the program in this embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the feature quantity map generation unit 11, the score calculation unit 12, the target image setting unit 13, and the recognition processing unit 15, respectively.

(Embodiment 2)
Next, an image recognition device, an image recognition method, and a program according to Embodiment 2 of the present invention will be described with reference to FIGS. 6 to 9. FIG.

[Device configuration]
First, with reference to FIG. 6, a schematic configuration of an image recognition device according to Embodiment 2 of the present invention will be described. FIG. 6 is a block diagram specifically showing the configuration of the image recognition device according to Embodiment 2 of the present invention.

As shown in FIG. 6, the image recognition device 30 according to the second embodiment has the same configuration as the image recognition device 10 according to the first embodiment, but differs in the following points. The following description focuses on differences from the first embodiment.

First, in the image recognition apparatus 30 according to the second embodiment, padding data is added to the learning data used to construct the discriminator (machine learning model) 20, so that extra data is added to the feature quantity map. is added. For this reason, unlike the image recognition device 10 according to the first embodiment, the image recognition device 30 includes a correction data generation section 31 and a correction data storage section 32 .

The correction data generation unit 31 generates correction data and stores the generated correction data in the correction data storage unit 32 . Correction data is data for correcting the data added to the feature map by padding added to the learning data. In the second embodiment, the score calculation unit 12 corrects the feature quantity map using the correction data generated by the correction data generation unit 31, and calculates the score using the corrected feature quantity map.

Further, the discriminator 20 used in the second embodiment has the characteristic that the size of the input data becomes smaller when it is output. Therefore, as described above, padding data is added to the learning data used to construct the discriminator 20 .

Padding is to add new data (padding data) to the input data so that the size of the data input to the convolutional layer and the size of the data output from it do not change. For example, let the kernel of a convolutional layer have both width and height K (width w=height h=K). In this case, if the padding data is not added, the size of the data output from the convolutional layer will be "-(K-1)" from the input size in both width and height. Specifically, when K=3, both the width and the height are reduced by "2".

Therefore, by adding padding data by (K-1)/2 pixels to the outside of the rectangle of the input data, it is possible to make the substantial size of the data the same at the time of input and output. Also, padding data is generally added not only to the first convolutional layer (input layer), but also to each lower convolutional layer. Also, normally, data with a data value of zero is used as padding data. The padding in this case is called "zero padding".

Here, the functions of the feature quantity map generation unit 11 and the correction data generation unit 31 according to the second embodiment will be described more specifically with reference to FIGS. 7 and 8. FIG. FIG. 7 is a diagram showing an example of a discriminator (machine learning model) according to Embodiment 2 of the present invention. FIG. 8 is a diagram showing an example of data output from the convolution layer of the discriminator according to Embodiment 2 of the present invention.

As mentioned above, padding with fixed values, such as zero padding, can cause the output results from the convolutional layers to change at the bounds of the rectangle. Therefore, in the present embodiment, the feature map generation unit 11 adds padding data to the image data in the range set by the target image setting unit 13, and inputs the image data after padding to the convolution layer 21. .

For this reason, in the second embodiment, also in the learning data used to construct the discriminator 20, peripheral pixel data of the image serving as the learning data is padded. For example, suppose the rectangular size of the sliding window is 64 pixels by 80 pixels and the size of the convolutional layer kernel is K=3. In this case, in machine learning, an image of 66 pixels×82 pixels including one pixel (=(K−1)/2) around the image serving as learning data is used.

By the way, it is difficult to specify appropriate padding data values in convolutional layers other than the input layer (convolutional layer 21). Since the convolutional layer 24 only performs size conversion on the input data, its kernel is 1×1, and no padding occurs in the convolutional layer 24 .

Therefore, in the second embodiment, the correction data generator 31 generates correction data corresponding to the padding data of the convolution layers 22 and 23, as shown in FIG. Further, the correction data generator 31 has correction blocks corresponding to the number of corresponding convolution layers. In the example of FIG. 8, the correction data generator 31 has a correction block 31a and a correction block 31b. Further, the correction data generation unit 31 can execute the same processing as in the convolutional layer, thereby suppressing delay in processing due to correction data generation.

Here, as shown in FIG. 8, the output of the convolutional layer (input layer) 21 is p[n,m], the output of the convolutional layer 22 is q[n,m], and the output of the convolutional layer 23 is R[n, m]. Let r[n,m] be the output of the convolutional layer 23 after being corrected by the correction data generated by the correction data generation unit 31 . Note that n indicates a row and m indicates a column.

In this case, the corrected output r[n,m] is the same as when zero padding is performed. Therefore, taking r[1,4] as an example, it is calculated by Equation 1 below.

[Number 1]
r[1,4] = q[0,4]*w2[0,1]+q[0,5]*w2[0,2]
+q[1,4]*w2[1,1]+q[1,5]*w2[1,2]
+q[2,4]*w2[2,1]+q[2,5]*w2[2,2]

On the other hand, in the second embodiment, as described in the first embodiment, the data for N lines is the object of convolution, so the real data is used as the padding data. Therefore, the output R[1,4] of the convolutional layer 23 is calculated by Equation 2 below.

[Number 2]
R[1,4] = q[0,3]*w2[0,0] + { q[0,4]*w2[0,1]+q[0,5]*w2[0,2] }
+q[1,3]*w2[1,0] + { q[1,4]*w2[1,1]+q[1,5]*w2[1,2] }
+q[2,3]*w2[2,0] + { q[2,4]*w2[2,1]+q[2,5]*w2[2,2] }
= r[1,4] + { q[0,3]*w2[0,0] + q[1,3]*w2[1,0] + q[2,3]*w2[2,0] }

Also, if r[1,4] in Equation 2 is represented by R[1,4], Equation 3 below is obtained.

[Number 3]
r[1,4] = R[1,4] - { q[0,3]*w2[0,0] + q[1,3]*w2[1,0] + q[2,3]* w2[2,0]}

Here, if Cr[1,4] is set in { } in Equation 3, Equation 3 can be expressed by Equation 4 below.

[Number 4]
r[1,4] = R[1,4] - Cr[1,4]

C[1,4] in Equation 4 above is correction data for correcting padding. In the second embodiment, the correction block 31b of the correction data generation section 31 generates this correction data and stores it in the correction data storage section 32. FIG.

The output q[n,m] of convolutional layer 22 is also affected by the padding in convolutional layer 21 . However, the output from the convolutional layer 21 does not contain padding data. Therefore, the correction block 31a also performs the same processing as the correction block 31b described above.

[Device operation]
Next, the operation of the image recognition device 30 according to the second embodiment will be explained using FIG. FIG. 9 is a flowchart showing the operation of the image recognition device according to Embodiment 2 of the present invention. 6 to 8 will be referred to as needed in the following description. Also in the second embodiment, the image recognition method is implemented by operating the image recognition device 30 . Therefore, the description of the image recognition method in the second embodiment is replaced with the description of the operation of the image recognition device 30 below.

As shown in FIG. 9, first, the target image setting unit 13 acquires image data of an image to be subjected to image recognition (step B1). Next, the target image setting unit 13 sets a range to be subjected to image recognition in the image of the image data acquired in step A1 (step B2).

Next, the feature map generator 11 extracts image data for N lines from the image data of the range set in step B2 (step B3). Subsequently, the feature map generation unit 11 inputs the extracted N lines of image data to the convolution layer of the classifier 20 to generate a feature map (step B4). Also, the feature map generation unit 11 stores the generated feature map in the feature map storage unit 14 .

Next, the correction data generation unit 31 generates correction data in order to correct the data added to the feature map by padding, and stores the generated correction data in the correction data storage unit 32 (step B5). Note that step B5 may be executed in the same manner as step B4.

Next, the score calculation unit 12 retrieves N lines of feature quantity maps from the feature quantity map storage unit 14 , and further extracts correction data from the correction data storage unit 32 . Then, the score calculation unit 12 corrects the extracted feature amount map using the correction data (step B6).

Next, the score calculation unit 12, while sliding the virtual window on the corrected feature map, divides the region within the window of the feature map into the fully connected layer of the classifier 20 at a plurality of predetermined positions. to calculate the score (step B7).

Next, the recognition processing unit 15 identifies the score with the largest value and the predetermined position at that time from among the scores calculated for each predetermined position in step B7, and identifies the identified score and the predetermined position. position is output as a result of image recognition (step B8).

Next, the recognition processing unit 15 determines whether or not the processes of steps B3 to B8 have been completed for all the ranges set in step B2 (step B9).

As a result of the determination in step B9, if the processing of steps B3 to B8 has not been completed for all the ranges set in step B2, the recognition processing unit 15 causes the feature map generation unit 11 to execute step B3 again. . As a result, the feature map generation unit 11 extracts image data for N lines located below the previous image data.

On the other hand, if the result of determination in step B9 is that the processing of steps B3 to B8 has been completed for all the ranges set in step B2, the processing in the image recognition device is terminated.

[Effects of Embodiment 2]
As described above, according to the second embodiment, when the discriminator 20 that requires padding for learning data is used, the padding data can be corrected. can be suppressed. Also in the second embodiment, as in the first embodiment, it is possible to improve the processing efficiency in image recognition using a machine learning model.

[program]
The program in the second embodiment may be any program that causes a computer to execute steps B1 to B9 shown in FIG. By installing this program in a computer and executing it, the image recognition apparatus 30 and the image recognition method according to the second embodiment can be realized. In this case, the processor of the computer functions as the feature quantity map generator 11, the score calculator 12, the target image setter 13, the recognition processor 15, and the correction data generator 31, and performs processing.

Also, the program in this embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the feature quantity map generation unit 11, the score calculation unit 12, the target image setting unit 13, the recognition processing unit 15, and the correction data generation unit 31. .

(Modification)
Although the discriminator 20 illustrated in FIGS. 3 and 4 is used in the first and second embodiments described above, the discriminator is not particularly limited in the first and second embodiments. In particular, in Embodiment 1, discriminators that do not require padding may be used.

(physical configuration)
A computer that implements the image recognition apparatus by executing the programs in the first and second embodiments will now be described with reference to FIG. FIG. 10 is a block diagram showing an example of a computer that implements the image recognition device according to the embodiment of the present invention.

As shown in FIG. 10 , computer 110 includes CPU 111 , main memory 112 , storage device 113 , input interface 114 , display controller 115 , data reader/writer 116 and communication interface 117 . These units are connected to each other via a bus 121 so as to be able to communicate with each other. Further, the computer 110 may include a GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), or ASIC (Application Specific IC) in addition to or instead of the CPU 111 .

The CPU 111 expands the programs (codes) of the present embodiment stored in the storage device 113 into the main memory 112 and executes them in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). Also, the program in the present embodiment is provided in a state stored in computer-readable recording medium 120 . It should be noted that the program in this embodiment may be distributed on the Internet connected via communication interface 117 .

Further, as a specific example of the storage device 113, in addition to a hard disk drive, a semiconductor storage device such as a flash memory can be cited. Input interface 114 mediates data transmission between CPU 111 and input devices 118 such as a keyboard and mouse. The display controller 115 is connected to the display device 119 and controls display on the display device 119 .

Data reader/writer 116 mediates data transmission between CPU 111 and recording medium 120 , reads programs from recording medium 120 , and writes processing results in computer 110 to recording medium 120 . Communication interface 117 mediates data transmission between CPU 111 and other computers.

Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital); magnetic recording media such as flexible disks; An optical recording medium such as a ROM (Compact Disk Read Only Memory) can be used.

It should be noted that the image recognition apparatus according to the present embodiment can also be realized by using hardware corresponding to each part instead of a computer in which a program is installed. Furthermore, the image recognition device may be partly implemented by a program and the rest by hardware.

Some or all of the above-described embodiments can be expressed by (Appendix 1) to (Appendix 12) described below, but are not limited to the following descriptions.

(Appendix 1)
a feature map generation unit that generates a feature map of an image to be recognized by using a convolutional layer of a machine learning model for detecting a specific object from an image;
A virtual window is set on the feature map, and while the window is slid by a set amount, the area in the window of the feature map is displayed at a plurality of predetermined positions by the full coupling of the machine learning model. a score calculation unit that inputs to the layer and calculates a score indicating the possibility that the specific object exists in the region for each of the predetermined positions;
An image recognition device characterized by comprising:

(Appendix 2)
The image recognition device according to Appendix 1,
further comprising a target image setting unit that acquires image data and sets a target range for the image recognition in the image specified by the acquired image data,
The image recognition apparatus, wherein the feature map generation unit generates the feature map for a set range.

(Appendix 3)
The image recognition device according to appendix 1 or 2,
identifying the score having the largest value and the predetermined position at that time from the scores calculated for each of the predetermined positions, and outputting the identified score and the predetermined position; further comprising a processing unit;
An image recognition device characterized by:

(Appendix 4)
The image recognition device according to any one of Appendices 1 to 3,
Padding data is added to the learning data used to build the machine learning model, and when extra data is added to the feature map, the added data is corrected. , a correction data generation unit that generates correction data,
The score calculation unit corrects the feature amount map using the generated correction data, and calculates the score using the corrected feature amount map.
An image recognition device characterized by:

(Appendix 5)
(a) using a convolutional layer of a machine learning model for detecting specific objects from an image to generate a feature map of the image to be image recognized;
(b) setting a virtual window on the feature amount map, and sliding the window by a set amount, at a plurality of predetermined positions, the area within the window of the feature amount map, the machine learning model; inputting into the fully connected layer of and calculating, for each of the predetermined positions, a score indicating the possibility that the specific object exists in the region;
An image recognition method characterized by comprising:

(Appendix 6)
The image recognition method according to appendix 5,
(c) acquiring image data, and setting a range to be subjected to image recognition in the image specified by the acquired image data;
generating the feature quantity map for the set range in step (a);
An image recognition method characterized by:

(Appendix 7)
The image recognition method according to appendix 5 or 6,
(d) identifying the score having the largest value and the predetermined position at that time among the scores calculated for each of the predetermined positions, and outputting the identified score and the predetermined position; further comprising the step of
An image recognition method characterized by:

(Appendix 8)
The image recognition method according to any one of Appendices 5 to 7,
(e) When padding data is added to the learning data used to build the machine learning model, and extra data is added to the feature map, the added data is corrected. generating correction data for
In step (b), correcting the feature quantity map using the generated correction data, and calculating the score using the corrected feature quantity map;
An image recognition method characterized by:

(Appendix 9)
to the computer,
(a) using a convolutional layer of a machine learning model for detecting specific objects from an image to generate a feature map of the image to be image recognized;
(b) setting a virtual window on the feature amount map, and sliding the window by a set amount, at a plurality of predetermined positions, the area within the window of the feature amount map, the machine learning model; inputting into the fully connected layer of and calculating, for each of the predetermined positions, a score indicating the possibility that the specific object exists in the region;
The program that causes the to run.

(Appendix 10)
The program according to Appendix 9,
to the computer;
(c) obtaining image data and setting a range to be subjected to image recognition in an image specified by the obtained image data;
generating the feature quantity map for the set range in step (a);
A program characterized by

(Appendix 11)
The program according to Appendix 9 or 10,
to the computer;
(d) identifying the score having the largest value and the predetermined position at that time among the scores calculated for each of the predetermined positions, and outputting the identified score and the predetermined position; do, cause the step to be executed further,
A program characterized by

(Appendix 12)
The program according to any one of Appendices 9 to 11,
to the computer;
(e) When padding data is added to the learning data used to build the machine learning model, and extra data is added to the feature map, the added data is corrected. further executing the step of generating correction data for
In step (b), correcting the feature quantity map using the generated correction data, and calculating the score using the corrected feature quantity map;
A program characterized by

As described above, according to the present invention, it is possible to improve processing efficiency in image recognition using a machine learning model. INDUSTRIAL APPLICABILITY The present invention is useful for various systems that require image recognition.

10 Image Recognition Apparatus (Embodiment 1)
11 feature map generation unit 12 score calculation unit 13 target image setting unit 14 feature map storage unit 15 recognition processing unit 20 classifier which is a machine learning model 21 to 24 convolution layer 25 fully connected layer 30 image recognition device (embodiment 2)
31 correction data generation unit 31a, 31b correction block 32 correction data storage unit 110 computer 111 CPU
112 main memory 113 storage device 114 input interface 115 display controller 116 data reader/writer 117 communication interface 118 input device 119 display device 120 recording medium 121 bus

Claims (9)

a feature map generation unit that generates a feature map of an image to be recognized by using a convolutional layer of a machine learning model for detecting a specific object from an image;
A virtual window is set on the feature map, and while the window is slid by a set amount, the area in the window of the feature map is displayed at a plurality of predetermined positions by the full coupling of the machine learning model. a score calculation unit that inputs to the layer and calculates a score indicating the possibility that the specific object exists in the region for each of the predetermined positions;
Padding data is added to the learning data used to build the machine learning model, and when extra data is added to the feature map, the added data is corrected. , a correction data generation unit that generates correction data;
with
The score calculation unit corrects the feature amount map using the generated correction data, and calculates the score using the corrected feature amount map.
An image recognition device characterized by: The image recognition device according to claim 1,
further comprising a target image setting unit that acquires image data and sets a target range for the image recognition in the image specified by the acquired image data,
The image recognition apparatus, wherein the feature map generation unit generates the feature map for a set range. The image recognition device according to claim 1 or 2,
identifying the score having the largest value and the predetermined position at that time from the scores calculated for each of the predetermined positions, and outputting the identified score and the predetermined position; further comprising a processing unit;
An image recognition device characterized by: (a) using a convolutional layer of a machine learning model for detecting specific objects from an image to generate a feature map of the image to be image recognized;
(b) setting a virtual window on the feature amount map, and sliding the window by a set amount, at a plurality of predetermined positions, the area within the window of the feature amount map, the machine learning model; inputting into the fully connected layer of and calculating, for each of the predetermined positions, a score indicating the possibility that the specific object exists in the region;
(e) When padding data is added to the learning data used to build the machine learning model, and extra data is added to the feature map, the added data is corrected. generating correction data for
has
In step (b), correcting the feature quantity map using the generated correction data, and calculating the score using the corrected feature quantity map;
An image recognition method characterized by: The image recognition method according to claim 4 ,
(c) acquiring image data, and setting a range to be subjected to image recognition in the image specified by the acquired image data;
generating the feature quantity map for the set range in step (a);
An image recognition method characterized by: The image recognition method according to claim 4 or 5 ,
(d) identifying the score having the largest value and the predetermined position at that time among the scores calculated for each of the predetermined positions, and outputting the identified score and the predetermined position; further comprising the step of
An image recognition method characterized by: to the computer,
(a) using a convolutional layer of a machine learning model for detecting specific objects from an image to generate a feature map of the image to be image recognized;
(b) setting a virtual window on the feature amount map, and sliding the window by a set amount, at a plurality of predetermined positions, the area within the window of the feature amount map, the machine learning model; inputting into the fully connected layer of and calculating, for each of the predetermined positions, a score indicating the possibility that the specific object exists in the region;
(e) When padding data is added to the learning data used to build the machine learning model, and extra data is added to the feature map, the added data is corrected. generating correction data for
and
In step (b), correcting the feature quantity map using the generated correction data, and calculating the score using the corrected feature quantity map;
program. The program according to claim 7 ,
to the computer;
(c) obtaining image data and setting a range to be subjected to image recognition in an image specified by the obtained image data;
generating the feature quantity map for the set range in step (a);
A program characterized by The program according to claim 7 or 8 ,
to the computer;
(d) identifying the score having the largest value and the predetermined position at that time among the scores calculated for each of the predetermined positions, and outputting the identified score and the predetermined position; do, cause the step to be executed further,
A program characterized by

Images