CN117576670A - Fine-grained recognition method based on cascade neural network and target spatiotemporal continuity - Google Patents

Abstract

The application provides a fine granularity identification method based on cascading neural network and target space-time continuity, wherein the method comprises the steps of obtaining multiple frames of images to be identified with continuous time; performing target detection on each image to be identified by using a target detection algorithm to obtain the position distribution of all the interested targets; tracking the interested target by utilizing a target tracking algorithm based on the space-time continuity of the target so as to determine the newly appeared interested target in each image to be identified; carrying out fine-grained recognition on the newly-appearing interested targets by utilizing a fine-grained recognition algorithm to obtain recognition results of all the interested targets; based on the position distribution and the recognition result, the position information and the fine granularity classification result of the interested target of each image to be recognized are obtained, so that efficient fine granularity recognition of the target is realized.

Description

Fine-grained recognition method based on cascade neural network and target spatiotemporal continuity

Technical field

This application relates to the field of target recognition, and in particular to a fine-grained recognition method based on cascade neural networks and target spatiotemporal continuity.

Background technique

Video surveillance can provide rich and intuitive scene information, making it easier for people to remotely understand environmental conditions and changes, especially targets of interest in the environment. Target recognition monitoring based on computer vision has the advantages of high efficiency, low cost, and wide coverage, and is widely used in transportation, security, ecological protection and other fields. The rapid development of artificial intelligence technology in recent years has provided very effective technical support for target recognition. It can automatically and intelligently identify and classify targets in images, greatly improve work efficiency, reduce manual intervention, and achieve intelligent monitoring and management. .

In practical applications, it is often necessary not only to analyze the general category of the target (such as birds, vehicles), that is, coarse-grained identification, but also to determine the subdivided types of the target (such as white cranes, gray cranes), that is, fine-grained identification of the target. . At present, there are many studies on coarse-grained target recognition, and many model algorithms based on neural networks have emerged. These methods can achieve good recognition results through a single frame of image. However, fine-grained recognition still faces many problems. On the one hand, fine-grained distinction between different targets requires a more complex network structure to extract fine features. If fine-grained recognition is performed on each frame of a continuous video, a huge amount of calculation will be generated, resulting in the inability to recognize in real time, resulting in The recognition efficiency is low; on the other hand, due to factors such as occlusion and blur, effective features of the target cannot be extracted based on a single frame image, which will lead to low recognition accuracy.

Contents of the invention

The present application aims to solve, at least to a certain extent, one of the technical problems in the related art.

To this end, the first purpose of this application is to propose a fine-grained identification method based on cascaded neural networks and target spatiotemporal continuity to achieve efficient fine-grained identification of targets.

The second purpose of this application is to propose a fine-grained recognition system based on cascaded neural networks and target spatiotemporal continuity.

The third object of this application is to provide an electronic device.

The fourth object of this application is to provide a computer-readable storage medium.

In order to achieve the above purpose, the first embodiment of the present application proposes a fine-grained identification method based on cascade neural network and target spatiotemporal continuity, which includes the following steps:

Obtain multiple frames of images to be recognized that are continuous in time;

Use the target detection algorithm to perform target detection on each image to be recognized to obtain the location distribution of all targets of interest;

Based on the spatio-temporal continuity of the target, use a target tracking algorithm to track the target of interest to determine new targets of interest appearing in each image to be recognized;

Use a fine-grained identification algorithm to perform fine-grained identification of the newly emerged targets of interest to obtain identification results for each target of interest;

Based on the position distribution and the recognition result, the position information and fine-grained classification results of the target of interest in each image to be recognized are obtained.

In the method of the first aspect of the present application, the multiple frames of images to be identified are obtained directly or through video.

In the method of the first aspect of the present application, the target detection algorithm adopts a lightweight target detection algorithm.

In the method of the first aspect of the present application, based on the spatio-temporal continuity of the target, the target tracking algorithm is used to track the target of interest to determine the newly emerging target of interest in each image to be identified, including: The appearance information and motion information of the target of interest are used, and the target tracking algorithm is used to match the target of interest in the images to be identified that are associated with continuous time, and the continuous trajectory of the target of interest is obtained, and then the new target of interest in each image to be recognized is determined. .

In the method of the first aspect of the present application, it also includes: after obtaining the recognition results of each object of interest in the image to be recognized in the current frame, judging the reliability of the recognition results; if there are objects of interest whose reliability does not meet the requirements , then a fine-grained recognition algorithm is used for fine-grained recognition of the corresponding objects of interest in the next frame of the image to be recognized for which the reliability does not meet the requirements, and the new objects of interest that appear in the next frame of the image to be recognized.

In order to achieve the above purpose, the second embodiment of the present application proposes a fine-grained recognition system based on cascaded neural networks and target spatio-temporal continuity, including:

The acquisition module is used to acquire multiple consecutive frames of images to be recognized;

The target detection module is used to use the target detection algorithm to perform target detection on each image to be recognized to obtain the location distribution of all targets of interest;

A target tracking module, used to track the target of interest based on the spatio-temporal continuity of the target using a target tracking algorithm to determine newly emerging targets of interest in each image to be identified;

A fine-grained identification module, used to perform fine-grained identification of the newly emerged targets of interest using a fine-grained identification algorithm to obtain identification results of each target of interest;

An output module is configured to obtain the position information and fine-grained classification results of the target of interest in each image to be recognized based on the position distribution and the recognition result.

In the system of the second aspect of the present application, the acquisition module acquires the multiple frames of images to be recognized directly or through video.

In the system of the second aspect of the present application, in the target detection module, the target detection algorithm adopts a lightweight target detection algorithm.

In the system of the second aspect of the present application, the target tracking module is specifically used to: use a target tracking algorithm to match the target of interest in the associated time-continuous images to be identified based on the appearance information and motion information of the target of interest, The continuous trajectory of the target of interest is obtained, and then the new target of interest appearing in each image to be recognized is determined.

In the system of the second aspect of the present application, the fine-grained recognition module is also used to: after obtaining the recognition results of each object of interest in the image to be recognized in the current frame, determine the reliability of the recognition results; if there is a reliable For the target of interest whose reliability does not meet the requirements, the target of interest corresponding to the target of interest whose reliability does not meet the requirements is used in the next frame of the image to be recognized, and the new target of interest that appears in the next frame of the image to be recognized is used. Granular recognition algorithm performs fine-grained recognition.

To achieve the above purpose, a third embodiment of the present application provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the The computer executes instructions stored in the memory to implement the method proposed in the first aspect of this application.

In order to achieve the above object, the fourth embodiment of the present application proposes a computer-readable storage medium. Computer-executable instructions are stored in the computer-readable storage medium. When the computer-executable instructions are executed by a processor, they are used to implement the present invention. Apply the method proposed by the first aspect.

The fine-grained identification method, system, electronic equipment and storage medium provided by this application are based on cascade neural networks and target spatio-temporal continuity. The target detection algorithm, target tracking algorithm and fine-grained identification algorithm are used to form a cascade neural network. First, the target is used The detection algorithm quickly detects the target of interest in the image to be identified, and then uses the target tracking algorithm to track the target of interest in each image to be identified to obtain the new target of interest, and then performs fine-grained identification of the newly emerged target of interest. , which can avoid repeated fine-grained identification of targets, thereby reducing the amount of calculation and achieving more efficient fine-grained identification of targets.

Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.

Description of the drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

Figure 1 is a schematic flowchart of a fine-grained identification method based on cascaded neural networks and target spatiotemporal continuity provided by an embodiment of the present application;

Figure 2 is a recognition result diagram of the first frame of three consecutive frames of images to be recognized provided by the embodiment of the present application;

Figure 3 is a recognition result diagram of the second frame of three consecutive frames of images to be recognized provided by the embodiment of the present application;

Figure 4 is a recognition result diagram of the third frame of three consecutive frames of images to be recognized provided by the embodiment of the present application;

Figure 5 is a block diagram of a fine-grained recognition system based on cascaded neural networks and target spatiotemporal continuity provided by an embodiment of the present application.

Detailed ways

The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application, but should not be construed as limiting the present application.

The fine-grained identification method and system based on cascaded neural networks and target spatiotemporal continuity according to the embodiments of the present application will be described below with reference to the accompanying drawings.

In view of the various problems existing in current target classification methods, such as fine-grained distinction between different targets, a more complex network structure is needed to extract fine features. If fine-grained recognition is performed on each frame of a continuous video, huge problems will occur. The amount of calculation makes it impossible to recognize in real time, which leads to low recognition efficiency; and due to factors such as occlusion and blur, effective features of the target cannot be extracted based on a single frame image, which leads to low recognition accuracy. The embodiments of this application provide a fine-grained identification method based on cascaded neural networks and target spatiotemporal continuity to achieve efficient and accurate fine-grained identification of targets.

Figure 1 is a schematic flowchart of a fine-grained identification method based on a cascade neural network and target spatiotemporal continuity provided by an embodiment of the present application. As shown in Figure 1, the fine-grained identification method based on cascade neural network and target spatiotemporal continuity includes the following steps:

Step S101: Acquire multiple consecutive frames of images to be recognized.

In step S101, multiple frames of images to be recognized can be obtained directly or through video. The video can be an online real-time video or an offline stored video.

In step S101, when obtaining multiple frames of time-continuous images to be identified through video, frames may be extracted from the video to form multiple frames of time-continuous images to be identified according to actual needs.

In step S101, if the image size of the image to be recognized does not meet the input requirements of the target detection algorithm, each frame of the image to be recognized needs to be preprocessed (such as cropping) to adjust the image size to adapt to the target detection network (also known as target detection network). detection algorithm) input requirements.

Step S102: Use a target detection algorithm to perform target detection on each image to be recognized to obtain the location distribution of all targets of interest.

In step S102, the target detection algorithm may adopt a lightweight target detection algorithm. Among them, the lightweight target detection algorithm uses lightweight networks with lower complexity and fewer parameters, such as YOLO and MobileNet (mobile neural network architecture), which can quickly detect the location distribution of all targets of interest in each image to be recognized.

In step S102, the target detection algorithm is a model trained in advance. The target of interest is related to the label during training. For example, the label during training is "bird", then all the objects of interest in the image to be recognized obtained by target detection in this step are all the birds in the image to be recognized.

In step S102, the target detection algorithm only detects the position of the target of interest in the image to be recognized, and does not include the type information of the target of interest.

Step S103: Based on the spatio-temporal continuity of the target, use the target tracking algorithm to track the target of interest to determine the newly emerging target of interest in each image to be recognized.

In step S103, the target tracking algorithm may adopt a neural network-based tracking algorithm, such as DeepSORT (multiple target tracking algorithm), to improve the robustness and accuracy of tracking.

Specifically, in step S103, based on the spatio-temporal continuity of the target, a target tracking algorithm is used to track the target of interest to determine the newly emerging target of interest in each image to be recognized, including: appearance information for the target of interest and Based on the motion information, the target tracking algorithm is used to match the target of interest in the images to be identified with continuous correlation time, and the continuous trajectory of the target of interest is obtained, and then the new target of interest appearing in each image to be recognized is determined.

Among them, the target tracking method uses the spatial connection information of the target of interest (which can be referred to as the target for short), such as appearance information and motion information, to match targets in continuous frames of associated time, obtain the continuous trajectory of each target, and manage target data information and content at the same time. Including: assigning different labels to different targets, assigning new labels to newly appearing targets, and removing disappearing targets from the current matching library.

Step S104: Use a fine-grained recognition algorithm to perform fine-grained recognition of newly emerged targets of interest to obtain recognition results for each target of interest.

In step S104, the fine-grained recognition model uses a neural network with a deeper network level to extract more refined target features and achieve distinction between different fine-grained categories (such as category information). The fine-grained recognition algorithm is a model trained in advance. The type information of the target of interest is related to the label when training the model. For example, when the target of interest is "bird" and the fine-grained recognition algorithm training labels include bird species information such as "gray crane" and "white crane", then the recognition result of any target of interest obtained after fine-grained recognition in this step is ( Also called fine-grained classification result) is one of the corresponding "grey crane", "white crane", etc.

In step S104, the input data of the fine-grained recognition algorithm is the new objects of interest that appear in each frame of the image to be recognized. Specifically, for the first frame of the image to be recognized, all the objects of interest are new objects of interest that appear. , at this time, the fine-grained recognition algorithm is used to perform fine-grained recognition of all the objects of interest in the first frame of the image to be recognized; for each subsequent frame of the image to be recognized, taking any frame as the current frame, compared with the corresponding previous frame The newly added target of interest in the frame is the new target of interest that appears in the image to be recognized in the current frame. At this time, a fine-grained recognition algorithm is used to perform fine-grained recognition of the new target of interest that appears in the image to be recognized in the current frame. This can avoid repeated fine-grained identification of targets, thereby reducing the amount of calculation and achieving efficient fine-grained identification of targets.

In addition, considering that due to factors such as occlusion and blur, effective features of the target cannot be extracted based on a single frame image, which will lead to low recognition accuracy. In step S104, it also includes: after obtaining the recognition results of each object of interest in the image to be recognized in the current frame, judging the reliability of the recognition results; if there is an object of interest whose reliability does not meet the requirements, then judging whether the reliability does not meet the requirements. The required target of interest corresponding to the target of interest in the next frame of the image to be recognized, and the new target of interest appearing in the next frame of the image to be recognized, are fine-grained identified using a fine-grained recognition algorithm.

The reliability of the recognition results can be determined by using the network prediction score. The higher the score, the higher the reliability. If the prediction score is greater than or equal to the score threshold (for example, 0.7), the reliability meets the requirements, otherwise the reliability does not meet the requirements. If the reliability does not meet the requirements, it will be marked and fine-grained recognition will be performed again in the next frame until the reliability is high, and the new recognition results will be used to update the historical results to improve the recognition accuracy in cases of occlusion, blur, etc.

In step S104, the fine-grained recognition algorithm only performs fine-grained recognition on new targets of interest or targets of interest with poor previous recognition results, so as to use the spatiotemporal continuity of the targets of interest to avoid repeated fine-grained recognition and reduce the risk of The amount of calculation increases operational efficiency. At the same time, for targets of interest in complex scenes such as occlusion and blur, multi-frame recognition results are fused based on recognition reliability measurement to enhance the stability and accuracy of recognition.

Step S105: Based on the position distribution and recognition results, the position information and fine-grained classification results of the objects of interest in each image to be recognized are obtained.

Specifically, in step S105, the position distribution of the objects of interest obtained by using the target detection network in each image to be recognized is summarized, as well as the recognition results of each object of interest in each image to be recognized, and finally all the objects of interest in all the images to be recognized are obtained. Target location information and fine-grained classification results.

In order to verify the effect of the method of this application, an experiment was conducted using the fine-grained identification of birds as an example. The specific contents are as follows:

Step A: Obtain bird video data through real-time video surveillance equipment, and perform frame extraction processing on the video at 10 frames per second to obtain time-continuous image data. Then, the image size is converted to 448*448 to be input to the target detection algorithm.

Step B: Use the lightweight target detection algorithm YOLOv5 to perform target detection on each image data that is continuous in time, thereby obtaining the location distribution information of all targets of interest. For example, the YOLOv5s6 network is used. YOLOv5s6 is a relatively lightweight model, consisting of 4 convolutional layers, 8 Bottleneck layers (bottleneck layers) and 1 output layer, with a total parameter amount of approximately 10.5M. This enables YOLOv5s6 to achieve fast and efficient real-time target detection in resource-constrained environments.

Step C: Use the target tracking algorithm DeepSORT to track the detected target of interest, use appearance information and motion information to match and associate targets in consecutive frames, and obtain the continuous trajectory of the target. At the same time, the target data information is managed, and different targets are assigned different label, and assign new labels to newly appearing targets. DeepSort achieves more accurate correlation matching by combining motion and appearance information, uses Kalman filtering to process the correlation of each frame, and uses the Hungarian algorithm for correlation measurement, which has high tracking performance; at the same time, it uses neural networks to extract features, which improves the detection of missing features. and robustness under occlusion conditions.

Step D: Use the fine-grained recognition algorithm to conduct fine-grained recognition of the target of interest, and at the same time determine the reliability of the recognition results. The fine-grained recognition model uses convnext-base_4xb32_cmb200, which is based on a convolutional neural network architecture, which contains 4 parallel branches, each branch has 32 convolution kernels. Through this architecture, the model is able to extract image features at different levels. This step only performs fine-grained identification of new targets of interest that appear in each frame or targets of interest that were previously recognized with poor reliability. The recognition reliability is determined by the prediction score. The higher the score, the higher the reliability. If the prediction score is lower than 0.7, that is, the reliability is low, it is marked and fine-grained recognition is performed again in the next frame until the credibility is high, and the new recognition results are used to update the historical results. The spatio-temporal continuity of the target is used to avoid repeated fine-grained recognition, reduce the amount of calculation and improve operating efficiency, while improving the recognition accuracy in situations such as occlusion and blur.

Step E: Finally, use the target detection position distribution results and fine-grained recognition and classification results to obtain the position information and fine-grained classification results of the target of interest in each image data.

Figure 2 is a recognition result diagram of the first frame of three consecutive frames of images to be recognized provided by the embodiment of the present application. Figure 3 is a recognition result diagram of the second frame of three consecutive frames of images to be recognized provided by the embodiment of the present application. Figure 4 is a recognition result diagram of the third frame of three consecutive frames of images to be recognized provided by the embodiment of the present application.

Combining Figures 2 to 4, we can see that: a gray crane first appeared in the picture in Figure 2 and was successfully located and identified; then two gray cranes appeared almost simultaneously on the right side of the picture in Figure 3, among which the closer gray crane appeared. It was successfully located and identified. The gray cranes that were far away were not recognized because they were partially blocked. As the target moved, the gray cranes that initially appeared in Figure 4 disappeared from the screen. All the gray cranes that were originally blocked appeared and were successfully located. and identification, which proves the effectiveness of this method.

In order to implement the above embodiments, this application also proposes a fine-grained recognition system based on cascaded neural networks and target spatiotemporal continuity.

Figure 5 is a block diagram of a fine-grained recognition system based on cascaded neural networks and target spatiotemporal continuity provided by an embodiment of the present application.

As shown in Figure 5, the fine-grained recognition system based on cascaded neural networks and target spatio-temporal continuity includes an acquisition module 11, a target detection module 12, a target tracking module 13, a fine-grained recognition module 14 and an output module 15, where:

The acquisition module 11 is used to acquire multiple frames of images to be recognized that are continuous in time;

The target detection module 12 is used to perform target detection on each image to be identified using a target detection algorithm to obtain the location distribution of all targets of interest;

The target tracking module 13 is used to track the target of interest based on the spatio-temporal continuity of the target using the target tracking algorithm to determine the newly emerging target of interest in each image to be recognized;

The fine-grained identification module 14 is used to perform fine-grained identification of newly emerged targets of interest using a fine-grained identification algorithm to obtain identification results of each target of interest;

The output module 15 is used to obtain the position information and fine-grained classification results of the target of interest in each image to be recognized based on the position distribution and recognition results.

Further, in a possible implementation of the embodiment of the present application, the acquisition module 11 directly acquires or acquires multiple frames of images to be recognized through video.

Further, in a possible implementation manner of the embodiment of the present application, in the target detection module 12, the target detection algorithm adopts a lightweight target detection algorithm.

Further, in a possible implementation of the embodiment of the present application, the target tracking module 13 is specifically configured to use a target tracking algorithm to match the appearance information and motion information of the target of interest in the associated time-continuous images to be identified. target of interest, obtain the continuous trajectory of the target of interest, and then determine the new target of interest appearing in each image to be recognized.

Further, in a possible implementation of the embodiment of the present application, the fine-grained recognition module 14 is also used to: after obtaining the recognition results of each object of interest in the image to be recognized in the current frame, determine the reliability of the recognition results; If there is an object of interest whose reliability does not meet the requirements, then the corresponding object of interest in the next frame of the image to be recognized for the object of interest whose reliability does not meet the requirements, and the new object of interest that appears in the next frame of the image to be recognized are The target uses a fine-grained identification algorithm for fine-grained identification.

It should be noted that the foregoing explanation of the embodiment of the fine-grained identification method based on a cascade neural network and target spatio-temporal continuity is also applicable to the fine-grained identification system based on a cascade neural network and target spatio-temporal continuity in this embodiment. No further details will be given here.

In the embodiment of this application, a target detection algorithm, a target tracking algorithm and a fine-grained recognition algorithm are used to form a cascade neural network. First, the target detection algorithm is used to quickly detect the target of interest in the image to be identified, and then the target tracking algorithm is used to realize each target to be identified. The tracking of the target of interest in the recognition image obtains the newly emerging target of interest, and then performs fine-grained identification of the newly emerged target of interest. This can avoid repeated fine-grained identification of the target, thereby reducing the amount of calculation and achieving a more efficient Efficient fine-grained target identification. In addition, for targets in complex scenes such as occlusion and blur, based on recognition reliability judgment, multi-frame recognition results can be integrated to enhance the stability and accuracy of recognition. That is, through cascaded neural networks and target spatiotemporal continuity, efficient and accurate fine-grained target recognition is achieved.

In order to implement the above embodiments, this application also proposes an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the foregoing implementation. The method provided in the example.

In order to implement the above embodiments, this application also proposes a computer-readable storage medium. Computer-executable instructions are stored in the computer-readable storage medium. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided in the foregoing embodiments.

In order to implement the above embodiments, this application also proposes a computer program product, which includes a computer program. When the computer program is executed by a processor, the method provided by the foregoing embodiments is implemented.

The collection, storage, use, processing, transmission, provision and disclosure of the information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good customs.

To be clear, personal information from users should be collected for lawful and reasonable purposes and not shared or sold outside of those lawful uses. In addition, such collection/sharing should be carried out after receiving the informed consent of the user, including but not limited to informing the user to read the user agreement/user notice and sign an agreement/authorization including authorization-related user information before using the function. In addition, take any necessary steps to safeguard and secure access to such Personal Information Data and ensure that others who have access to Personal Information Data comply with its privacy policies and procedures.

This application is expected to provide implementation solutions for users to selectively prevent the use or access of personal information data. That is, the present disclosure contemplates that hardware and/or software may be provided to prevent or block access to such personal information data. Risks are minimized by limiting data collection and deleting data once the personal information data is no longer needed. In addition, when applicable, such personal information is de-identified to protect user privacy.

In the foregoing description of various embodiments, reference to the description of the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific description in conjunction with the embodiment or example Features, structures, materials or characteristics are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine different embodiments or examples and features of different embodiments or examples described in this specification unless they are inconsistent with each other.

In addition, the terms “first” and “second” are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of this application, "plurality" means at least two, such as two, three, etc., unless otherwise expressly and specifically limited.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments, or portions of code that include one or more executable instructions for implementing customized logical functions or steps of the process. , and the scope of the preferred embodiments of the present application includes additional implementations in which functions may be performed out of the order shown or discussed, including in a substantially simultaneous manner or in the reverse order, depending on the functionality involved, which shall It should be understood by those skilled in the technical field to which the embodiments of this application belong.

The logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered a sequenced list of executable instructions for implementing the logical functions, and may be embodied in any computer-readable medium, For use by, or in combination with, instruction execution systems, devices or devices (such as computer-based systems, systems including processors or other systems that can fetch instructions from and execute instructions from the instruction execution system, device or device) or equipment. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wires (electronic device), portable computer disk cartridges (magnetic device), random access memory (RAM), Read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, and subsequently edited, interpreted, or otherwise suitable as necessary. process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present application can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if it is implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: discrete logic gate circuits with logic functions for implementing data signals; Logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

Those of ordinary skill in the art can understand that all or part of the steps involved in implementing the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The program can be stored in a computer-readable storage medium. When executed, one of the steps of the method embodiment or a combination thereof is included.

In addition, each functional unit in various embodiments of the present application can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software function modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

The storage media mentioned above can be read-only memory, magnetic disks or optical disks, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above-mentioned embodiments are illustrative and cannot be understood as limitations of the present application. Those of ordinary skill in the art can make modifications to the above-mentioned embodiments within the scope of the present application. The embodiments are subject to changes, modifications, substitutions and variations.

Claims (12)

1. A fine-grained identification method based on cascaded neural networks and target spatiotemporal continuity, which is characterized by including the following steps: Obtain multiple frames of images to be recognized that are continuous in time; Use the target detection algorithm to perform target detection on each image to be recognized to obtain the location distribution of all targets of interest; Based on the spatio-temporal continuity of the target, use a target tracking algorithm to track the target of interest to determine new targets of interest appearing in each image to be recognized; Use a fine-grained identification algorithm to perform fine-grained identification of the newly emerged targets of interest to obtain identification results for each target of interest; Based on the position distribution and the recognition result, the position information and fine-grained classification results of the target of interest in each image to be recognized are obtained. 2. The fine-grained recognition method based on cascaded neural network and target spatiotemporal continuity according to claim 1, characterized in that the multiple frames of images to be recognized are obtained directly or through video. 3. The fine-grained identification method based on cascaded neural networks and target spatio-temporal continuity according to claim 1, characterized in that the target detection algorithm adopts a lightweight target detection algorithm. 4. The fine-grained identification method based on cascade neural network and target spatio-temporal continuity according to claim 1, characterized in that, based on the target spatio-temporal continuity, a target tracking algorithm is used to track the target of interest, To identify emerging objects of interest in each image to be recognized, including: In view of the appearance information and motion information of the target of interest, the target tracking algorithm is used to match the target of interest in the images to be identified that are associated with continuous time, and the continuous trajectory of the target of interest is obtained, and then the newly emerged interesting objects in each image to be recognized are determined. Target. 5. The fine-grained identification method based on cascaded neural networks and target spatiotemporal continuity according to claim 1, characterized in that it also includes: After obtaining the recognition results of each object of interest in the image to be recognized in the current frame, determine the reliability of the recognition results; If there is an object of interest whose reliability does not meet the requirements, then the corresponding object of interest in the next frame of the image to be recognized for the object of interest whose reliability does not meet the requirements, and the new object of interest that appears in the next frame of the image to be recognized are The target uses a fine-grained identification algorithm for fine-grained identification. 6. A fine-grained recognition system based on cascaded neural networks and target spatiotemporal continuity, which is characterized by: The acquisition module is used to acquire multiple consecutive frames of images to be recognized; The target detection module is used to use the target detection algorithm to perform target detection on each image to be recognized to obtain the location distribution of all targets of interest; A target tracking module, used to track the target of interest based on the spatio-temporal continuity of the target using a target tracking algorithm to determine newly emerging targets of interest in each image to be identified; A fine-grained identification module, used to perform fine-grained identification of the newly emerged targets of interest using a fine-grained identification algorithm to obtain identification results of each target of interest; An output module is configured to obtain the position information and fine-grained classification results of the target of interest in each image to be recognized based on the position distribution and the recognition result. 7. The fine-grained recognition system based on cascaded neural networks and target spatio-temporal continuity according to claim 6, characterized in that the acquisition module directly acquires or acquires the multiple frames of images to be recognized through video. 8. The fine-grained recognition system based on cascaded neural network and target spatio-temporal continuity according to claim 6, characterized in that in the target detection module, the target detection algorithm adopts a lightweight target detection algorithm. 9. The fine-grained recognition system based on cascade neural network and target spatiotemporal continuity according to claim 6, characterized in that the target tracking module is specifically used for: In view of the appearance information and motion information of the target of interest, the target tracking algorithm is used to match the target of interest in the images to be identified that are associated with continuous time, and the continuous trajectory of the target of interest is obtained, and then the newly emerged interesting objects in each image to be recognized are determined. Target. 10. The fine-grained recognition system based on cascaded neural network and target spatio-temporal continuity according to claim 6, characterized in that the fine-grained recognition module is also used to: After obtaining the recognition results of each object of interest in the image to be recognized in the current frame, determine the reliability of the recognition results; If there is an object of interest whose reliability does not meet the requirements, then the corresponding object of interest in the next frame of the image to be recognized for the object of interest whose reliability does not meet the requirements, and the new object of interest that appears in the next frame of the image to be recognized are The target uses a fine-grained identification algorithm for fine-grained identification. 11. An electronic device, characterized by comprising: a processor, and a memory communicatively connected to the processor; The memory stores computer execution instructions; The processor executes computer-executable instructions stored in the memory to implement the method according to any one of claims 1-5. 12. A computer-readable storage medium, characterized in that computer-executable instructions are stored in the computer-readable storage medium, and when executed by a processor, the computer-executable instructions are used to implement any one of claims 1-5. method described in the item.