JP2022546883A - Target object recognition method, apparatus and system - Google Patents

Abstract

The present invention provides a target object recognition method, apparatus and system. The method includes: cropping a target image including a plurality of stacked target objects awaiting recognition from the acquired image; adjusting a height of the target image to a predetermined height; extracting a feature map of an image; segmenting the feature map along a dimension corresponding to the height direction of the target image to obtain features of a predetermined number of segments; and recognizing a target object based on the feature of each segment among the features of , wherein the height direction of the target image is the direction in which the plurality of target objects awaiting recognition are stacked. [Selection drawing] Fig. 1

Description

<Cross-citation of related applications>
The present invention claims priority from Singapore Patent Application entitled "Target Object Recognition Method, Apparatus, and System" and application number 10202007347V, filed on August 1, 2020; is incorporated herein by reference in its entirety.
The present invention relates to the technical field of computer vision, and more particularly to a target object recognition method, apparatus and system.

In daily production and life, we usually need to recognize several target objects. Taking the entertainment scene of desktop games as an example, in some desktop games, it is necessary to recognize the game coins on the desktop so that the information on the type and number of game coins can be obtained. However, the traditional recognition method has a low recognition accuracy rate.

One aspect of the present invention provides a target object recognition method. The target object recognition method includes: cutting out a target image including a plurality of stacked target objects awaiting recognition from a collected image; adjusting the height of the target image to a predetermined height; segmenting the feature map along a dimension corresponding to the height direction of the target image to obtain features of a predetermined number of segments; recognizing a target object based on the feature of each segment of the features of the number of segments, wherein the height direction of the target image is the direction in which the plurality of target objects awaiting recognition are stacked. be.

Referring to any one embodiment of the present invention, adjusting the height of the target image to the predetermined height may increase the height of the target image until the width of the target image after scaling is the predetermined width. scaling the height and width in equal proportions; and, if the height of the target image after scaling is greater than the predetermined height, scaling until the height of the target image after reduction is equal to the predetermined height. and reducing the height and width of the scaled target image proportionally.

Referring to any one embodiment of the present invention, adjusting the height of the target image to the predetermined height may increase the height of the target image until the width of the target image after scaling is the predetermined width. scaling the height and width equally; and padding the scaled target image using a first pixel if the height of the scaled target image is less than the predetermined height. thereby causing the height of the filled target image to be the predetermined height.

With reference to any one embodiment of the present invention, the recognition pending target objects in said target image are sheet-like objects, the thickness of each recognition pending target object is equal, and said plurality of recognition pending target objects is stacked along the thickness direction, and the predetermined height is an integer multiple of the thickness.

Referring to any one embodiment of the present invention, both the feature map extraction and target object recognition are performed by a neural network, which uses a sample image and its label information to has been trained.

Referring to any one embodiment of the present invention, the sample image label information includes a label type for each target object in the sample image, and the neural network performs a performing feature extraction to obtain a feature map of the resized sample image, and recognizing a target object in the sample image based on the features of each segment obtained by segmenting the feature map; obtaining a prediction type for each target object in a sample image; and parameter values for the neural network based on the prediction type for each target object in the sample image and the label type for each target object in the sample image. It is trained by coordinating and

Referring to any one embodiment of the present invention, the label information of the sample images further includes a number of target objects of each label type, and adjusting the parameter values of the neural network includes: a prediction type of each target target in said sample image, a label type of each target target in said sample image, a number of target targets of each label type in said sample image, and a number of target targets of each prediction type in said sample image; adjusting the parameter values of the neural network based on.

Referring to any one embodiment of the present invention, the sample image label information further includes a total number of target objects in the sample image, and adjusting parameter values of the neural network includes: the sum of the prediction type of each target object in the sample image, the label type of each target object in the sample image, the number of target objects of each prediction type in the sample image, and the total number of target objects in the sample image; adjusting the parameter values of the neural network based on.

Referring to any one embodiment of the present invention, the target object recognition method comprises testing the trained neural network, and based on the results of the testing, each type of sorting the recognition accuracy of the target object, obtaining a result of sorting the recognition accuracy; and according to the result of the test, sorting the misrecognition rate of each type of target object according to the neural network, misrecognition. Further comprising obtaining a rate reordering result, and further training the neural network based on the recognition accuracy reordering result and the false recognition rate reordering result.

One aspect of the present invention provides a target object recognizer. The target object recognition device includes an acquisition unit for cutting out a target image including a plurality of stacked target objects awaiting recognition from the acquired image, and a height adjustment unit for adjusting the height of the target image to a predetermined height. an adjusting unit, an extracting unit for extracting a feature map of an adjusted target image, segmenting the feature map along a dimension corresponding to the height direction of the target image to obtain a predetermined number of segmented features; and a recognition unit for recognizing a target object based on the feature of each segment among the features of the predetermined number of segments, wherein the height direction of the target image is , the direction in which the plurality of target objects awaiting recognition are stacked.

With reference to any one embodiment of the present invention, the adjustment unit scales the height and width of the target image by equal proportions until the width of the target image after scaling reaches a predetermined width, and If the height of the target image after scaling is greater than the predetermined height, then adjust the height and width of the target image after scaling until the height of the target image after reduction is equal to the predetermined height. Shrink by ratio.

With reference to any one embodiment of the present invention, the adjustment unit scales the height and width of the target image by equal proportions until the width of the target image after scaling reaches a predetermined width, and When the height of the target image after scaling is smaller than the predetermined height, filling the target image after scaling using the first pixel reduces the height of the filled target image to the predetermined height. Set to the desired height.

With reference to any one embodiment of the present invention, the pending target objects in said target image are sheet-like objects, the thickness of each pending target object is equal, and the multiple pending target objects are: The target object is stacked along the thickness direction, and the predetermined height is an integer multiple of the thickness.

Referring to any one embodiment of the present invention, both the feature map extraction and target object recognition are performed by a neural network, which uses a sample image and its label information to has been trained.

Referring to any one embodiment of the present invention, the sample image label information includes a label type for each target object in the sample image, the target object recognition device further comprising a training unit, wherein the A training unit performs feature extraction on a resized sample image to obtain a feature map of the resized sample image, and segmenting the feature map based on the features of each segment. performing recognition of target objects in the sample images to obtain a prediction type for each target object in the sample images; and training the neural network by adjusting parameter values of the neural network based on the label type.

Referring to any one embodiment of the present invention, the label information of the sample images further comprises a number of target objects of each label type, and the training unit comprises a prediction type of each target object in the sample images. and based on the label type of each target object in the sample image, the number of target objects of each label type in the sample image, and the number of target objects of each prediction type in the sample image, performing Adjust parameter values.

Referring to any one embodiment of the present invention, the sample image label information further includes a total number of target objects in the sample image, and the training unit predicts each target object in the sample image. the label type of each target object in the sample image; the sum of the number of target objects of each prediction type in the sample image; and the total number of target objects in the sample image. Adjust parameter values.

Referring to any one embodiment of the present invention, the target object recognizer further comprises a test unit for testing the trained neural network, and based on the results of the test: Sorting the recognition accuracy of each type of target object according to the neural network, obtaining a result of sorting the recognition accuracy, and misrecognising each type of target object according to the neural network according to the result of the test. Reordering the rates, obtaining a reordering result of the misrecognition rate, and further training the neural network based on the reordering result of the recognition accuracy and the reordering result of the misrecognition rate.

One aspect of the invention provides an electronic device. The electronic device comprises a processor and a memory for storing processor-executable instructions, and the processor invokes the instructions stored in the memory to perform the operations according to any one of the embodiments of the invention. is configured to implement the target object recognition method of

One aspect of the invention provides a computer-readable storage medium. Computer program instructions are stored on the computer readable storage medium and, when the computer program instructions are executed by a processor, the method of target object recognition according to any one of the present inventions is performed.

Target object recognition methods, apparatus, electronic devices, and storage media according to one or more embodiments of the present invention adjust the height of a target image retrieved from an acquired image to a predetermined height, and adjust the extracting a feature map of a target image, segmenting the feature map along a dimension corresponding to the height direction of the target image, and obtaining features of a predetermined number of segments; Recognition of the target object is performed based on the features of each segment of the features. Since the segment features obtained by segmentation correspond to the feature map of each target object, it is possible to avoid the influence of the number of target objects on the recognition accuracy by performing target object recognition based on the segmentation features. and the target object recognition accuracy is improved.

It is to be understood that the above general description and the following detailed description are merely exemplary and explanatory and are not intended to limit the invention.

The drawings herein are incorporated into and constitute a part of the specification. These drawings, showing embodiments consistent with the present invention, are used to interpret the solution unit of the present invention together with the description of the specification.
4 is a flow chart of a target object recognition method in accordance with at least one embodiment of the present invention; FIG. 4 is a schematic diagram of a plurality of target objects loaded upright in a target object recognition method according to at least one embodiment of the present invention; FIG. 4 is a schematic diagram of a plurality of side-mounted target objects in a target object recognition method according to at least one embodiment of the present invention; 1 is a block diagram of a target object recognizer in accordance with at least one embodiment of the present invention; FIG. 1 is a block diagram of an electronic device in accordance with at least one embodiment of the invention; FIG.

In order that the present invention may be better understood by those skilled in the art, the following clearly and completely describes several embodiments of the present invention in combination with the drawings. Obviously, the described embodiments are merely some possible embodiments of the invention. All other embodiments obtained by a person skilled in the art based on one or more embodiments of the present invention without laborious steps should fall within the protection scope of the present invention.

The terminology used in the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the present invention and the appended claims, the singular forms "a", "said" and "the" are intended to include the plural forms unless the context clearly dictates otherwise. Intend. It should be understood that the term "and/or" as used herein refers to any and all possible combinations of one or more of the associated listed items. In addition, the term "at least one" in the text refers to any one of a plurality of types or any combination of at least two of a plurality of types.

It should be understood that although the terms first, second, third, etc. are used in the present invention to describe various types of information, these information are not limited to these terms. These terms are only used to distinguish between similar types of information. For example, first information may be referred to as second information, and similarly, second information may be referred to as first information, without departing from the scope of the present invention. This is context dependent. For example, the word "if" as used herein may be interpreted as "when" or "when" or "depending on the particular circumstances".

FIG. 1 is a flowchart of a target object recognition method according to at least one embodiment of the invention. As shown in FIG. 1, the method may include steps 101-105.

In step 101, a target image is segmented from the acquired image, said target image comprising a plurality of stacked target objects awaiting recognition.

In some common situations, the targets awaiting recognition are sheet-like objects of various shapes, such as game coins, and the thickness (height) of each target is usually the same. Multiple target objects awaiting recognition are typically stacked along the thickness direction. As shown in FIG. 2A, a plurality of game coins are vertically stacked (stand stacked), the height direction (H) of the target image is the vertical direction, and the width direction (H) of the target image is the vertical direction. W) is the direction perpendicular to the height direction (H) of the target image. As shown in FIG. 2B, multiple game coins may be stacked (float stacked) along the horizontal direction, the height direction (H) of the target image is the horizontal direction, and the target image The width direction (W) of is the direction perpendicular to the height direction (H) of the target image.

A target object awaiting recognition may be a target object placed in the target area. The target area may be a flat surface (eg, a desktop), a container (eg, a box), or the like. An image of the target area may be acquired via an image acquisition device, such as a camera or camera head, in the vicinity of the target area.

In an embodiment of the present invention, a deep learning network, such as RCNN (Region Convolutional Neural Network), may be used to detect the collected images to obtain a target object detection result, said detection result being a detection frame. There may be. A detection frame may segment a target image comprising a plurality of stacked target objects awaiting recognition from the acquired image. As can be appreciated by those skilled in the art, RCNN is merely an example, other deep learning networks may be employed to perform target detection, and the present invention is not limited thereto.

At step 102, the height of the target image is adjusted to a predetermined height.

However, the height direction of the target image is the direction in which the plurality of recognition-waiting target objects are stacked. The predetermined height may be an integer multiple of the thickness of the target object awaiting recognition. Taking the stacked game coins shown in FIGS. 2A and 2B as an example, the stacking direction of the game coins shown in FIGS. 2A and 2B can be determined as the height direction of the target image. Accordingly, the radial direction of the game coin is determined as the width direction of the target image.

At step 103, a feature map of the adjusted target image is extracted.

For an adjusted target image, a pre-trained feature extraction network may be used to obtain a feature map of the adjusted target image. However, the feature extraction network may include multiple convolutional layers, or multiple convolutional and pooling layers, or the like. Multi-layer feature extraction gradually transforms low-level features into medium-level or high-level features to improve the expressiveness of the target image, which is advantageous for subsequent processing.

At step 104, the feature map is segmented along the height direction of the target image to obtain features for a predetermined number of segments.

By segmenting the feature map in the height direction of the target image, features of a predetermined number of segments can be obtained. However, each segment feature may be considered as corresponding to one target object. However, said predetermined number is also the maximum number of target objects awaiting recognition.

In one example, the feature map may include multiple dimensions, such as channel dimension, height dimension, width dimension, batch dimension, etc., and the format of the feature map is, for example, [BCHW] can be expressed as where B denotes the batch dimension, C denotes the channel dimension, H denotes the height dimension, and W denotes the width dimension. However, the directions indicated by the height and width dimensions of the feature map may be determined based on the height and width directions of the target image.

At step 105, target object recognition is performed based on the features of each segment of the predetermined number of segment features.

Since the features of each segment correspond to one target object, performing feature target recognition on a segment-by-segment basis is more efficient than performing target recognition directly using the feature map of the target image. , and improve the recognition accuracy of the target object in the target image.

In some embodiments, a target image containing multiple target objects upright (referred to as a side view image) may be captured via an image acquisition device mounted to the side of the target area, or may be viewed above the target area. A target image (referred to as a plan view image) including a plurality of sideways standing target objects may be captured via an image acquisition device provided in the .

In some embodiments, the height of the target image may be adjusted in the following manner.

First, a predetermined height and a predetermined width corresponding to the target image are obtained and used to perform size conversion on the target image. However, the predetermined width may be set according to the average width of the target objects, and the predetermined height may be set according to the average height of the target objects and the maximum number of target objects awaiting recognition. good.

In one example, the height and width of the target image may be scaled proportionally until the width of the target image is a predetermined width. However, scaling at the same ratio means enlarging or reducing the target image while maintaining the ratio between the height and width of the target image. However, the units of the predetermined width and the predetermined height may be pixels or other units, but the present invention is not limited to this.

If the width of the target image after scaling reaches the predetermined width but the height of the target image after scaling is greater than the predetermined height, the above-mentioned steps are performed until the height of the target image after reduction is equal to the predetermined height. Reduce the height and width of the scaled target image proportionally.

For example, if the target object is a game coin, the predetermined width may be set to 224 pixels based on the average width of the game coin, and the average height of the game coin and the game waiting for recognition. Based on the maximum number of coins, for example 72, the predetermined height may be set to 1344 pix. First, the width of the target image may be adjusted to 224 pixels, and the height of the target image may be adjusted in equal proportions. When the height after adjustment is greater than 1344 pix, the height of the target image after adjustment is adjusted again to make the height of the target image 1344 pix, and the width of the target image is adjusted at an equal ratio. is intended to adjust the height of the target image to a predetermined height of 1344 pix. If the height after adjustment is equal to 1344 pix, the height of the target image is adjusted to a predetermined height of 1344 pix without requiring adjustment again.

In one example, the height and width of the target image are proportionally scaled until the width of the target image is a predetermined width. When the width of the scaled target image reaches a predetermined width but the height of the scaled target image is smaller than the predetermined height, the first pixel is used to fill the scaled target image. so that the height of the target image after filling is a predetermined height.

However, the first pixel may be a pixel with a pixel value of zero, that is, a black pixel. The first pixel may be set with another pixel value. Specific pixel values do not affect the effectiveness of embodiments of the present invention.

As an example, the target object is still a game coin, the predetermined width is 224 pix, the predetermined height is 1344 pix, and the maximum number is 72. First, the width of the target image is scaled to 224 pix, and the height of the target image is may be scaled proportionally. If the height of the target image after scaling is less than 1344 pix, the portion of height less than 1344 pix is filled with black pixels so that the height of the target image after filling is 1344 pix. When the height of a target image after filling is equal to 1344 pix, it is realized to adjust the height of the target image to a predetermined height of 1344 pix without needing filling.

After adjusting the height of the target image to a predetermined height, the feature map of the adjusted target image is segmented along a dimension corresponding to the height direction of the target image to obtain features of a predetermined number of segments. You may

Taking the feature map [B C H W] as an example, for example, the feature map [B C H W] is defined in the H dimension (height dimension) based on a predetermined number, namely 72, which is the maximum number of target objects awaiting recognition. segment at. When the height of the target image after adjustment is smaller than the predetermined height, the target image is filled so that the height reaches the predetermined height. When the height of the target image after adjustment is greater than the predetermined height, the height of the target image is adjusted to the predetermined height by equal ratio reduction. It is obtained based on the image. Further, since the predetermined height is set according to the maximum number of target objects awaiting recognition, the feature map is segmented according to the maximum number, and the acquired feature map of each segment is transferred to each target object. and performing target recognition based on the feature map of each segment, the impact of the number of targets can be reduced, improving the accuracy of each target recognition.

In some embodiments, for a predetermined number of segment features in the filled target image obtained by segmenting the filled target image, filling with the first pixel when classifying the segment features. The feature classification result of the segment corresponding to the marked region is empty. For example, for segment features corresponding to regions filled with black pixels, it can be determined that the classification results corresponding to these segment features are empty. The difference between the maximum number of target objects and the number of empty classification results may determine the number of non-empty classification results contained in the target image, or the number of non-empty segment features corresponding to the target object. You may directly recognize the number of classification results where . By doing so, the number of target objects contained in the target image can be determined based on the number of non-empty classification results obtained.

Assuming that the maximum number of target objects awaiting recognition is 72, if the feature map of the target image is divided into 72 segments and the target object is recognized based on the feature map of each segment, 72 classification results can be obtained. be. If the target image contains a black pixel fill region, the classification result corresponding to the feature map of the segment of the fill region will be empty. For example, when 16 empty classification results are obtained, 56 non-empty classification results are obtained. By doing so, it can be determined that the target image contains 56 target objects.

As can be understood by those skilled in the art, the predetermined width parameter, the predetermined height parameter, and the maximum number of target objects awaiting recognition parameter are all examples, and the specific numerical values of these parameters are It may be specifically set according to demand. Embodiments of the present invention are not limited in this respect.

In some embodiments, both feature map extraction and target object recognition are performed by a neural network, which was trained using sample images and their label information. The neural networks may include feature extraction networks and classification networks. However, the feature extraction network extracts a feature map of the resized target image, and the classification network performs target object recognition based on the features of each segment of a predetermined number of segment features. However, the sample image includes multiple target objects.

In one example, the label information of the sample images includes the label type of each target object in the sample images, and the neural network was trained by the following operations. The operation includes performing feature extraction on the resized sample image to obtain a feature map of the resized sample image, and segmenting the feature map based on the features of each segment obtained. performing recognition of target objects in the sample images to obtain a prediction type for each target object in the sample images; and adjusting parameter values of the neural network based on the label type.

Taking game coins as an example, each game coin type is associated with a denomination, and game coins of the same denomination belong to the same type. For a sample image containing a plurality of game coins stacked upright, said sample image is labeled with the denomination of each game coin. A neural network is trained to recognize the target object based on the denomination labeled sample images. The neural network obtains the denomination of each game coin by prediction based on the sample image, and the difference between the prediction type and the label type determines, for example, the parameter value of the feature extraction network and the parameter value of the classification network. and to complete training when the difference between prediction type and label type is less than a set threshold or when the number of iterations reaches a set number.

In one example, the sample image label information further includes the number of target objects for each label type. In such a case, the prediction type of each target object in the sample image, the label type of each target object in the sample image, the number of target objects of each label type in the sample image, and and adjusting the parameter values of the neural network based on the number of target targets of each prediction type.

Taking a plurality of game coins still stacked upright as an example, the sample image is labeled with the denomination information of each game coin and the number information of game coins of each denomination. Based on sample images labeled with the above information, a neural network is trained to recognize the target object. The neural network obtains the denomination of each game coin and the number of game coins of the same denomination by prediction based on the sample image. Adjust the parameter values of the neural network based on the difference between the prediction result and the label information.

In one example, the sample image label information further includes the total number of target objects in the sample image. In such a case, the prediction type of each target object in the sample image, the label type of each target object in the sample image, the sum of the number of target objects of each prediction type in the sample image, and the sample and adjusting the parameter values of the neural network based on the total number of target objects in the image.

Taking a plurality of game coins still stacked upright as an example, the sample image is labeled with the denomination information of each game coin and the total number of game coins information. Based on sample images labeled with the above information, a neural network is trained to recognize the target object. The neural network obtains the denomination of each game coin and the total number of game coins (ie, the prediction result) by prediction based on the sample image. Adjust the parameter values of the neural network based on the difference between the prediction result and the label information.

In one example, the label information of the sample image includes the label type of each target object in the sample image, the number of target objects of each label type in the sample image, and the total number of target objects in the sample image. In such a case, the prediction type of each target object in the sample image, the label type of each target object in the sample image, the number of target objects of each label type in the sample image, and Adjusting parameter values of the neural network based on the number of target objects of each prediction type, the sum of the number of target objects of each prediction type in the sample image and the total number of target objects in the sample image.

Taking a plurality of game coins still stacked upright as an example, the sample image is labeled with the denomination information of each game coin, the number information of game coins of various denominations and the total number of game coins information. Based on sample images labeled with the above information, a neural network is trained to recognize the target object. The neural network obtains the denomination of each game coin, the number of game coins of different denominations and the total number of game coins by prediction based on the sample image. Adjust the parameter values of the neural network based on the difference between the prediction result and the label information.

In an embodiment of the present invention, the loss function used to train the neural network includes at least one of cross-entropy loss, target number loss of each type, and target target total number loss. That is, the loss function can also include the number loss of each type of target objects and the total loss of the target objects in addition to the cross-entropy loss, thus improving the ability to recognize the number of target objects.

In some embodiments, the neural network for recognizing types and numbers of target objects according to embodiments of the present invention is better suited for real-world situations by augmenting the training data when training the neural network. may be applied. For example, horizontally reversing the sample image, rotating the sample image at a set angle, performing color conversion on the sample image, and performing brightness conversion on the sample image. Any one term or multiple terms may be adopted to perform data expansion.

A target object recognition method according to embodiments of the present invention may be used to recognize multiple types of target objects by recognizing the target object using segmented feature maps. , the recognition accuracy of each type of target object does not degrade as the type variety increases.

In some embodiments, testing the trained neural network, sorting the recognition accuracy of each type of target object according to the neural network based on the results of the testing, and sorting the recognition accuracy results into and sorting the misrecognition rate of each type of target object according to the neural network based on the result of the test, obtaining the sorting result of the misrecognition rate, the sorting result of the recognition accuracy and the The neural network may be further trained based on the misrecognition rate reordering results.

A two-dimensional table may be used to store the sorted result of recognition accuracy and the sorted result of false recognition rate of each type of target object. For example, the results of sorting recognition accuracies may be stored in a table in order from top to bottom, and the results of sorting recognition error rates may be stored in a table in order from left to right. By further training the types within the set range in the table, for example, the types within the first three columns located in the third row of the table, the recognition accuracy and accuracy rate of the neural network for these types can be improved. Improve.

FIG. 3 is a block diagram of a target object recognizer in accordance with at least one embodiment of the invention. As shown in FIG. 3, the apparatus includes an acquisition unit 301 for cropping a target image including a plurality of stacked target objects awaiting recognition from an acquired image, and a height of the target image to a predetermined height. and an extraction unit 303 for extracting a feature map of said adjusted target image, segmenting said feature map along a dimension corresponding to the height direction of said target image. a segmentation unit 303 for obtaining a predetermined number of segment features; and a recognition unit 305 for recognizing a target object based on the features of each segment among the predetermined number of segment features. wherein the height direction of the target image is the direction in which the plurality of target objects awaiting recognition are stacked.

In some embodiments, the adjustment unit 302 scales the height and width of the target image proportionally until the width of the scaled target image is a predetermined width, and the width of the scaled target image is If the predetermined width is reached but the scaled target image height is greater than the predetermined height, the scaled target image height and Reduce width proportionally.

In some embodiments, the adjustment unit 302 scales the height and width of the target image proportionally until the width of the scaled target image is a predetermined width, and the width of the scaled target image is When the predetermined width is reached, but the height of the target image after scaling is smaller than the predetermined height, the target image after filling is filled by using the first pixel to fill the target image after scaling. Set the height to the desired height.

In some embodiments, the pending target objects in the target image are sheet-like objects, the thickness of each pending target object is equal, and the plurality of pending target objects has a thickness of The predetermined height is an integral multiple of the thickness of the target object awaiting recognition.

In some embodiments, both feature map extraction and target object recognition are performed by a neural network, which was trained using sample images and their label information.

In some embodiments, the sample image label information includes a label type for each target object in the sample image, and the apparatus further comprises a training unit, wherein the training unit is a resized sample image. and obtaining a feature map of the resized sample image; and recognizing a target object in the sample image based on the features of each segment obtained by segmenting the feature map. to obtain a prediction type for each target object in the sample image; and based on the prediction type for each target object in the sample image and the label type for each target object in the sample image, the neural network training the neural network by adjusting the parameter values of .

In some embodiments, the label information of the sample images further includes a number of target objects of each label type, and the training unit includes a prediction type for each target object in the sample images and a prediction type for each target object in the sample images. Specifically, a prediction type of each target object in the sample image and a label type of each target object in the sample image when adjusting the parameter values of the neural network based on the label type of the target object and and adjusting the parameter values of the neural network based on the number of target objects of each label type in the sample image and the number of target objects of each prediction type in the sample image.

In some embodiments, the label information of the sample images further includes a total number of target objects in the sample images, and the training unit determines a prediction type for each target object in the sample images and a prediction type for each target object in the sample image; a label type for each target object in the sample image; Adjust the parameter values of the neural network based on the sum of the number of target objects of each prediction type in a sample image and the total number of target objects in the sample image.

In some embodiments, the apparatus further comprises a testing unit for testing the trained neural network and, based on the results of the testing, determining each type of target for the neural network. Sorting the recognition accuracy of the object, obtaining a result of sorting the recognition accuracy, sorting the misrecognition rate of each type of target object according to the neural network according to the result of the test, and sorting the misrecognition rate A permutation result is obtained, and the neural network is further trained based on the permutation result of the recognition accuracy and the permutation result of the misrecognition rate.

FIG. 4 is a block diagram of an electronic device in accordance with at least one embodiment of the invention. As shown in FIG. 4, the electronic device may comprise a processor and memory for storing processor-executable instructions. However, the processor implements the target object recognition method according to any one of the embodiments of the invention by executing the instructions.

At least one embodiment of the invention further provides a computer-readable storage medium. Computer program instructions are stored on the computer-readable storage medium and, when the program instructions are executed by a processor, the method for target object recognition according to any one embodiment of the invention is performed.

As will be appreciated by those skilled in the art, one or more embodiments of the invention may be provided as a method, system or computer program product. Accordingly, one or more embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. . One or more embodiments of the present invention may also include one or more computer-usable storage media (including magnetic disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code. may take the form of a computer program product implemented in, but not limited to,

In the present invention, "and/or" means including at least one of both. For example, "A and/or B" includes the three forms A, B, and "A and B."

Each embodiment of the present invention will be described in a progressive manner, each embodiment will focus on the differences from other embodiments, and the same or similar parts of each embodiment will refer to each other. do it. In particular, since the data processing device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method embodiment.

The foregoing has described specific embodiments of the invention. Other implementations are within the scope of the appended claims. In some cases, the actions or things recited in the claims can be performed in a different order than in the examples and still achieve desirable results. Also, the steps depicted in the figures do not require that the particular order or sequential order shown must be used to achieve the desired result. Multitasking and parallel processing are also possible or advantageous in some embodiments.

Embodiments of the themes and functional operations described in this invention can be digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this invention and their structural equivalents, or It can be implemented in one or more combinations thereof. Embodiments of the subject matter described in the present invention are implemented by one or more computer programs, i.e., coded on a tangible, non-transitory program carrier, to be executed by or to operate a data processing apparatus. It can be implemented as one or more modules in controlled computer program instructions. Alternatively or additionally, program instructions may be encoded in a man-made transmission signal, such as a device-generated electrical, optical or electromagnetic signal. Such signals are generated to encode information and be transmitted to appropriate receiver equipment for execution by data processing equipment. A computer storage medium may be a device readable storage device, a device readable storage substrate, a random or serial access memory device, or a combination of one or more thereof.

The processes and logic flows described in the present invention are implemented by one or more programmable computers executing one or more computer programs to operate and respond to input data by generating output. function may be performed. The processing and logic flow may be performed by dedicated logic circuits, such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits), and the device may also be implemented as dedicated logic circuits.

Computers suitable for the execution of a computer program include, for example, general and/or special purpose microprocessors, or any other type of central processing unit. The central processing unit typically receives instructions and data from read-only memory and/or random access memory. The basic unit of a computer comprises a central processing unit for implementing and executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices, such as magnetic, magneto-optical, or optical disks, for storing data, or the computer can operate on this mass storage device. to receive data from or transmit data to, or combine the two situations. However, computers do not necessarily have such devices. The computer may also be connected to another device such as a mobile phone, personal digital assistant (PDA), mobile audio or video player, game console, global positioning system (GPS) receiver, or universal serial bus (USB) flash, for example. A memory driver may be incorporated into the portable storage device. The above are just some examples.

Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices such as semiconductor memory devices (e.g. EPROM, EEPROM and flash memory devices), magnetic disks (e.g. internal hard disk or removable disk), magneto-optical disks and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by or integrated into dedicated logic circuitry.

Although the present invention includes a large amount of specific implementation details, these details should not be construed as limiting the invention, but are primarily used to describe the features of the specific embodiment of the invention. . Certain features described in multiple embodiments of the invention may be implemented in combination in a single embodiment. On the other hand, various features that are described in a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Also, although features as described above may play a role in, and thus be claimed to protect in, some combinations, one or more features from the claimed combination may be In some cases may be omitted from the combination and the claimed combination may refer to sub-combinations or variations of sub-combinations.

Similarly, although the figures depict operations in a particular order, it is understood that these operations may be performed in the particular order shown or may be performed in sequence or all illustrated operations may be performed to obtain a desired result. should not be understood as requiring In some cases, multitasking and parallel processing can be advantageous. Also, the separation of various system modules and units in the above embodiments should not be understood to require such separation in all embodiments. Moreover, it will be appreciated that the program units and systems described may typically be integrated into a single software product or encapsulated as multiple software products.

Thus, a specific embodiment of the theme has been described. Other implementations are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desired results. Moreover, the operations depicted in the figures do not necessarily yield the desired results in the particular order or sequential order presented. In some embodiments, multitasking and parallel processing can be advantageous.

The above are some embodiments of the present invention and are not intended to limit the present invention. Any change, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included within the scope of the present invention.

Claims (20)

A target object recognition method comprising:
segmenting from the acquired image a target image comprising a plurality of stacked target objects awaiting recognition;
adjusting the height of the target image to a predetermined height;
extracting a feature map of the adjusted target image;
segmenting the feature map along a dimension corresponding to the height direction of the target image to obtain features for a predetermined number of segments;
performing target object recognition based on each segment feature of the predetermined number of segment features;
A target object recognition method, wherein the height direction of the target image is a direction in which the plurality of target objects waiting for recognition are stacked. Adjusting the height of the target image to the predetermined height includes:
scaling the height and width of the target image in equal proportions until the width of the target image after scaling reaches a predetermined width;
If the height of the target image after scaling is greater than the predetermined height, then adjust the height and width of the target image after scaling until the height of the target image after reduction is equal to the predetermined height. 2. The method of claim 1, comprising: shrinking by a ratio. Adjusting the height of the target image to the predetermined height includes:
scaling the height and width of the target image in equal proportions until the width of the target image after scaling reaches a predetermined width;
When the height of the target image after scaling is smaller than the predetermined height, filling the target image after scaling using a first pixel reduces the height of the filled target image. 2. The method of claim 1, further comprising: reaching said predetermined height. the target object awaiting recognition in the target image is a sheet-like object, the thickness of each target object awaiting recognition is equal, and the plurality of target objects awaiting recognition are stacked along a thickness direction; and,
The target object recognition method according to claim 1, wherein the predetermined height is an integral multiple of the thickness. 2. The feature map extraction and target object recognition are both performed by a neural network, and the neural network is trained using sample images and their label information. 5. The method of target object recognition according to any one of items 4 to 4. the sample image label information includes a label type for each target object in the sample image;
The neural network is
performing feature extraction on the size-adjusted sample image to obtain a feature map of the size-adjusted sample image;
recognizing a target object in a sample image based on the feature of each segment obtained by segmenting the feature map to obtain a prediction type of each target object in the sample image;
adjusting parameter values of the neural network based on the prediction type of each target object in the sample image and the label type of each target object in the sample image. 6. The target object recognition method according to claim 5. the sample image label information further includes a target number of targets for each label type;
Adjusting parameter values of the neural network includes:
a prediction type of each target object in the sample image; a label type of each target object in the sample image; a number of target objects of each label type in the sample image; and a target object of each prediction type in the sample image. 7. The method of claim 6, comprising adjusting parameter values of the neural network based on the number of . the sample image label information further includes a total number of target objects in the sample image;
Adjusting parameter values of the neural network includes:
a prediction type for each target object in the sample image; a label type for each target object in the sample image; a sum of the number of target objects of each prediction type in the sample image; 7. The method of claim 6, comprising adjusting parameter values of the neural network based on a total number of testing the trained neural network;
rearranging the recognition accuracies of each type of target object according to the neural network according to the results of the testing to obtain a recognition accuracy rearrangement result;
rearranging the misrecognition rate of each type of target object according to the neural network according to the result of the test, and obtaining a rearrangement result of the misrecognition rate;
6. The method of claim 5, further comprising: further training the neural network based on the recognition accuracy permutation results and the false recognition rate permutation results. A target object recognizer comprising:
an acquisition unit for segmenting from the acquired image a target image comprising a plurality of stacked target objects awaiting recognition;
an adjustment unit for adjusting the height of the target image to a predetermined height;
an extraction unit for extracting a feature map of the adjusted target image;
a segmentation unit for segmenting the feature map along a dimension corresponding to the height direction of the target image to obtain features of a predetermined number of segments;
a recognition unit for recognizing a target object based on features of each segment of the predetermined number of segment features;
The target object recognition device, wherein the height direction of the target image is a direction in which the plurality of target objects waiting for recognition are stacked. The adjustment unit is
scaling the height and width of the target image by equal proportions until the width of the target image after scaling reaches a predetermined width; and
If the height of the target image after scaling is greater than the predetermined height, then adjust the height and width of the target image after scaling until the height of the target image after reduction is equal to the predetermined height. 11. The target object recognizer of claim 10, wherein the target object recognition device is scaled down by a ratio. The adjustment unit is
scaling the height and width of the target image by equal proportions until the width of the target image after scaling reaches a predetermined width; and
When the height of the target image after scaling is smaller than the predetermined height, filling the target image after scaling using a first pixel reduces the height of the filled target image. 11. The target object recognition device according to claim 10, wherein the predetermined height is set. the target object awaiting recognition in the target image is a sheet-like object, the thickness of each target object awaiting recognition is equal, and the plurality of target objects awaiting recognition are stacked along a thickness direction; and,
The target object recognition device according to claim 10, wherein the predetermined height is an integral multiple of the thickness. 10. The feature map extraction and target object recognition are both performed by a neural network, and the neural network is trained using sample images and their label information. 14. A target object recognizer according to any one of Claims 1-13. the sample image label information includes a label type for each target object in the sample image;
The target object recognition device further comprises a training unit,
The training unit comprises:
performing feature extraction on the size-adjusted sample image to obtain a feature map of the size-adjusted sample image;
recognizing a target object in a sample image based on the feature of each segment obtained by segmenting the feature map to obtain a prediction type of each target object in the sample image;
training the neural network by adjusting parameter values of the neural network based on the prediction type of each target object in the sample image and the label type of each target object in the sample image. 15. The target object recognizer of claim 14. the sample image label information further includes a target number of targets for each label type;
The training unit comprises: a prediction type for each target object in the sample image; a label type for each target object in the sample image; a number of target objects of each label type in the sample image; 16. The target object recognizer of claim 15, wherein the parameter values of the neural network are adjusted based on the number of target objects of predictive type. the sample image label information further includes a total number of target objects in the sample image;
The training unit comprises: a prediction type for each target object in the sample image; a label type for each target object in the sample image; a sum of the number of target objects of each prediction type in the sample image; 16. The target object recognizer of claim 15, wherein the parameter values of the neural network are adjusted based on the total number of target objects in the target object. The target object recognition device further comprises a test unit,
The test unit includes:
testing said trained neural network;
rearranging the recognition accuracy of each type of target object according to the neural network according to the result of the test, obtaining a result of the rearrangement of the recognition accuracy;
sorting the misrecognition rate of each type of target object according to the neural network according to the results of the test, obtaining the sorting result of the misrecognition rate;
15. The target object recognizer of claim 14, wherein the neural network is further trained based on the permutation result of the recognition accuracy and the permutation result of the misrecognition rate. an electronic device,
a processor;
a memory for storing processor-executable instructions;
10. Electronic device, characterized in that the processor is arranged to implement the method of any one of claims 1 to 9 by executing the instructions. A computer readable storage medium,
Computer program instructions are stored on the computer readable storage medium, and when the computer program instructions are executed by a processor, a target object recognition method according to any one of claims 1 to 9 is performed. A computer-readable storage medium characterized by:

Images