WO2022247343A1

WO2022247343A1 - Recognition model training method and apparatus, recognition method and apparatus, device, and storage medium

Info

Publication number: WO2022247343A1
Application number: PCT/CN2022/075119
Authority: WO
Inventors: 苏翔博; 王健; 孙昊
Original assignee: 北京百度网讯科技有限公司
Priority date: 2021-05-28
Filing date: 2022-01-29
Publication date: 2022-12-01
Also published as: JP2023530796A; KR20220110321A; CN113326773A

Abstract

A recognition model training method and apparatus, a recognition method and apparatus, a device, and a storage medium. A recognition model training method comprises: inputting an image to be processed into a recognition model to be trained; outputting at least one feature map of said image by means of a feature output layer of said recognition model; obtaining, according to the at least one feature map by means of a head portion of said recognition model, prediction data of a first target object in said image, and prediction data of a second target object associated with the prediction data of the first target object in said image; and according to the prediction data of the first target object, the prediction data of the second target object, annotation data of the first target object, and annotation data of the second target object, optimizing said recognition model to obtain a trained recognition model.

Description

Recognition model training method, recognition method, device, equipment and storage medium

This application claims the priority of the Chinese patent application submitted to the China Patent Office on May 28, 2021, with the application number 202110591890.8, and the title of the invention is "recognition model training method, recognition method, device, equipment and storage medium", the entire content of which Incorporated in this application by reference.

technical field

The present disclosure relates to the technical field of artificial intelligence, in particular to the technical fields of deep learning and computer vision, and can be applied in smart cities and smart traffic scenarios.

Background technique

Target recognition is an important means and purpose of image processing. Through target recognition, objects in videos, static images, human bodies, animals and other target objects can be recognized, and various uses such as identity authentication and security inspection can be realized based on the recognition results. .

With the development of computer technology, in various scenarios that need to be applied to target recognition technology, with the diversification of application purposes and the diversification of recognized target objects, multiple models are often required to achieve the purpose of target object recognition. How to improve the efficiency of the model for processing images to be processed is a problem that needs to be improved.

Contents of the invention

The present disclosure provides a recognition model training method, a recognition method, a device, a device and a storage medium.

According to an aspect of the present disclosure, a recognition model training method is provided, including:

Input the image to be processed into the recognition model to be trained;

Output at least one feature map of the image to be processed through the feature output layer of the recognition model to be trained;

Obtain the prediction data of the first target object in the image to be processed according to at least one feature map through the head of the recognition model to be trained, and the second target object associated with the prediction data of the first target object in the image to be processed forecast data;

According to the predicted data of the first target object, the predicted data of the second target object, the labeled data of the first target object and the labeled data of the second target object, the recognition model to be trained is optimized to obtain the trained recognition model.

According to another aspect of the present disclosure, an identification method is provided, including:

Input the image to be recognized into the recognition model, and obtain the prediction data of the first target object in the image to be recognized and the prediction data of the second target object associated with the first target object. The recognition model is provided by any embodiment of the present disclosure The trained recognition model.

According to another aspect of the present disclosure, a recognition model training device is provided, including:

The first input module is used to input the image to be processed into the recognition model to be trained;

The feature map module is used to output at least one feature map of the image to be processed through the feature output layer of the recognition model to be trained;

The prediction data module is used to obtain the prediction data of the first target object in the image to be processed and the prediction data of the first target object in the image to be processed according to at least one feature map through the head of the recognition model to be trained associated predicted data for the second target object;

The training module is used to optimize the recognition model to be trained according to the prediction data of the first target object, the prediction data of the second target object, the label data of the first target object and the label data of the second target object, and obtain the recognition after training Model.

According to another aspect of the present disclosure, an identification device is provided, including:

The second input module is used to input the image to be recognized into the recognition model, and obtain the prediction data of the first target object in the image to be recognized and the prediction data of the second target object associated with the first target object. The recognition model is any of the present disclosure. An embodiment provides a trained recognition model.

According to another aspect of the present disclosure, an electronic device is provided, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the method in any embodiment of the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method in any embodiment of the present disclosure.

According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method in any embodiment of the present disclosure is implemented.

According to the technology of the present disclosure, the prediction data of the first target object and the second target object can be obtained through the recognition model to be trained, and the recognition model to be trained can be optimized and trained according to the prediction data and label data, and the obtained recognition model is accurate to the first The recognition of the target object and the second target object related to the first target object can realize the associated recognition of at least two target objects, make full use of the information provided in the image to be recognized, and output more recognition results with a smaller number of models , to improve the deployment and recognition efficiency of the model.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood through the following description.

Description of drawings

The accompanying drawings are used to better understand the present solution, and do not constitute a limitation to the present disclosure. in:

FIG. 1 is a schematic diagram of a recognition model training method according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a recognition model training method according to another embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a recognition model training method according to another embodiment of the present disclosure;

Fig. 4 is a schematic diagram of a recognition model training method according to an example of the present disclosure;

Fig. 5 is a schematic diagram of a recognition model training method according to another example of the present disclosure;

Fig. 6 is a schematic diagram of data processing according to an example of the present disclosure;

Fig. 7 is a schematic diagram of identification according to an example of the present disclosure;

Fig. 8 is a schematic diagram of a recognition model training device according to an embodiment of the present disclosure;

Fig. 9 is a schematic diagram of a recognition model training device according to another embodiment of the present disclosure;

Fig. 10 is a schematic diagram of a recognition model training device according to another embodiment of the present disclosure;

Fig. 11 is a schematic diagram of a recognition model training device according to an embodiment of the present disclosure;

Fig. 12 is a schematic diagram of a recognition model training device according to an embodiment of the present disclosure;

Fig. 13 is a block diagram of an electronic device for implementing the recognition model training method of the embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Embodiments of the present disclosure first provide a recognition model training method, as shown in FIG. 1 , including:

Step S11: Input the image to be processed into the recognition model to be trained;

Step S12: Output at least one feature map of the image to be processed through the feature output layer of the recognition model to be trained;

Step S13: Obtain the prediction data of the first target object in the image to be processed and the prediction data of the first target object in the image to be processed according to at least one feature map through the head of the recognition model to be trained associated predicted data for the second target object;

Step S14: According to the predicted data of the first target object, the predicted data of the second target object, the labeled data of the first target object and the labeled data of the second target object, optimize the recognition model to be trained to obtain the trained recognition model.

In this embodiment, the image to be processed may be an image containing a target object to be identified. The target object to be recognized can be any object, such as a person, a human face, a human eye, a human body, an animal, a still life, etc.

In the case where the feature output layer of the recognition model to be trained outputs two or more feature maps, the sizes of the two or more feature maps are different.

In this embodiment, the head of the recognition model to be trained may be a layer structure in the recognition model to be trained. After the feature output layer of the recognition model to be trained outputs at least one feature map, at least one feature map is input to the head of the recognition model to be trained, and the head of the recognition model to be trained outputs the prediction data and Predicted data for the second target object.

In this embodiment, the first target object and the second target object may be target objects in the image to be recognized. The first target object may be a target object that has a geometric relationship with the second target object.

For example, when the first target object is a human face, the second target object can be a human body related to the human face, that is, when the first target object is the face of person A, the second target object can be the face of person A. human body. For another example, when the first target object is a human eye, the second target object can be a human face related to the human eye, that is, when the first target object is the eyes of person A, the second target object can be person A face.

In this embodiment, the first target object and the second target object may contain or be included in each other.

The prediction data of the first target object may include identification data of the first target object, such as whether the first target object exists in the image to be identified, its position, and the like.

The prediction data of the first target object may also include data such as characteristics, attributes, and quality of the first target object. For example, the size level, integrity level, shape level, etc. of the first target object.

The prediction data of the first target object may include various kinds of prediction data.

In this implementation, the types of the predicted data of the first target object and the predicted data of the second target object may be the same or different.

Through the head of the recognition model to be trained, the prediction data of the first target object in the image to be processed and the prediction data of the second target object can be obtained according to at least one feature map, which can be for each pixel of the feature map, output For predicting the data of the first target object and the data for predicting the second target object, according to the data of all pixels, the predicted data of the first target object and the predicted data of the second target object are obtained.

The predicted data of the first target object and the predicted data of the second target object can be used in scenarios such as smart city and intelligent transportation.

In this embodiment, the prediction data of the first target object and the second target object can be obtained through the recognition model to be trained, and the recognition model to be trained is optimized and trained according to the prediction data and label data, and the obtained recognition model is accurate to the first target The recognition of the object and the second target object related to the first target object can realize the associated recognition of at least two target objects, make full use of the information provided in the image to be recognized, and output more recognition results with a smaller number of models, Improve model deployment and identification efficiency.

In one embodiment, the predicted data of the first target object includes the classification predicted data of the first target object and the attribute predicted data of the first target object; the predicted data of the second target object includes the predicted data of the second target object and the predicted data of the first target object. Two attribute prediction data of the target object.

In this embodiment, the classification prediction data of the first target object may be used to determine whether a certain area of the image to be recognized is the first target object. The attribute prediction data of the first target object may be a parameter for judging the presentation quality of the first target object in the image to be recognized. For example, the classification prediction data of , may be the determination data of the first target object, such as whether the first target object exists in the image, the anchor frame surrounding the first target object, and the like.

The classification prediction data of the second target object may be the same as the classification prediction data of the first target object, or may be different from the classification prediction data of the first target object. The property prediction data of the second target object may be the same as the property prediction data of the first target object, or may be different from the property prediction data of the first target object.

In this embodiment, the classification prediction data of the first target object and the attribute prediction data of the first target object can be obtained, and the classification prediction data and attribute prediction data of the second target object can also be obtained, so that at least two objects to be identified can be identified. Correlating target objects for joint output, so that not only can the recognition results of a large number of target objects be obtained through a small number of models, but also the process of recognizing the first target object and the second target object can be integrated to achieve a better recognition effect.

In one embodiment, as shown in FIG. 2 , through the head of the recognition model to be trained, the predicted data of the first target object and the predicted data of the second target object associated with the predicted data of the first target object are output. ,include:

Step S21: For each pixel of the feature map, output the anchor box prediction data of the first target object and the anchor box prediction data of the second target object;

Step S22: Output the predicted data of the first target object and the predicted data of the second target object according to the predicted data of the anchor frame of the first target object and the predicted data of the anchor frame of the second target object.

In this embodiment, for each pixel of the feature map, the anchor frame prediction data of the first target object may include data such as the probability that the pixel is the first target object. The anchor frame prediction data of the second target object may be the same as the anchor frame prediction data of the first target object.

According to the anchor point frame prediction data of the first target object, the boundary points of the first target object can be determined in the feature map, thereby forming an anchor point frame surrounding the first target object, and determining the prediction data of the first target object according to the anchor point frame . The prediction data for the second target object can be generated in the same manner as the prediction data for the first target object.

The above steps S21 and S22 can be performed by the head of the recognition model to be trained.

In this embodiment, the anchor point frame prediction data for predicting the prediction data of the first target object and the second target object can be generated for each pixel of the feature map, so that the anchor point frame surrounding the target object can be obtained later, and more Accurately output the prediction data of the first target object and the second target object according to the information such as the anchor frame.

In one embodiment, the first target object is a human face; the second target object is a human body.

In scenarios such as security big data systems, it is often necessary to detect and track natural persons appearing in surveillance video streams, and store and record key frame images. Wherein, the key frame image may contain at least one of a human face and a human body, and may be used for subsequent human face and/or human body retrieval. In the embodiment of the present disclosure, the first target object can be set to be a human face, and the second target object can be set to be a human body, so that it can be used for human body and face detection in a video stream capture system, and associate human bodies and human faces belonging to the same natural person , at the same time, in the video containing an uninterrupted trajectory of natural persons, select a frame image that is most suitable for recognition and store it in the database to provide important and high-quality information for subsequent tracking, retrieval, security and other operations.

In this embodiment, the human face in the image to be recognized and the human body related to the human face can be recognized, so as to realize association recognition.

In one embodiment, the feature output layer includes a backbone network and a feature pyramid network; through the feature output layer of the recognition model to be trained, output at least one feature map of the image to be processed, as shown in Figure 3, including:

Step S31: Output multiple first feature maps of the image to be processed through the backbone network (Backbone);

Step S32: input N second feature maps in multiple first feature maps into a feature pyramid network (Feature Pyramid Network, FPN), where N is an integer not less than 1;

Step S33: output N third feature maps through the feature pyramid network;

Step S34: Use N third feature maps as feature maps.

In this embodiment, the N second feature maps may be generated based on the N first feature maps with smaller sizes in the first feature map. For example, the backbone network outputs 5 first feature maps, F1, F2, F3, F4, and F5. Among them, the size of F1-F5 gradually becomes smaller, and the feature pyramid network outputs F6, F7, and F8 respectively according to F3, F4, and F5. A second feature map.

The backbone network may include a multi-layer CNN (Convolutional Neural Networks, Convolutional Neural Network) sub-network, which can perform a convolution operation on the image to be recognized input to the recognition model to be trained to obtain multiple first feature maps.

The feature pyramid network can perform further convolution operations on the N first feature maps, so that the high-level semantic information in the image to be recognized can be integrated into the feature maps to obtain N second feature maps.

The N second feature maps may be feature maps of different sizes, which can be used to identify target objects of different sizes in the image to be recognized. For example, feature maps of smaller sizes can be used to identify target objects of larger sizes, Larger-sized feature maps can be used to recognize smaller-sized target objects.

In this embodiment, through the backbone network and feature pyramid network of the recognition model to be trained, the feature map of the image to be recognized can be obtained, so that the recognition and correlation of the first target object and the associated second target object can be performed subsequently according to the feature map. data prediction.

An embodiment of the present disclosure also provides an image recognition method, as shown in FIG. 4 , including:

Step S41: Input the image to be recognized into the recognition model, and obtain the prediction data of the first target object in the image to be recognized and the prediction data of the second target object associated with the first target object. The recognition model is any embodiment of the present disclosure The provided trained recognition model.

In this embodiment, the trained recognition model can be used to recognize the image to be recognized, and the associated prediction data of the first target object and the second target object can be obtained, so that a relatively small number of models can be used to obtain more prediction results.

In one embodiment, the image to be identified is a frame image in the video to be identified; the identification method also includes:

According to the prediction data of the first target object and the prediction data of the second target object, key frame images in the video to be recognized are obtained.

In this embodiment, according to the predicted data of the first target object and the predicted data of the second target object, the image frame with the best overall quality of the first target object and the second target object in the video to be recognized can be determined as the key frame image.

Also according to the prediction data of the first target object and the prediction data of the second target object, respectively determine the image frame with the best overall quality of the first target object and the second target object in the video to be recognized as the key frame image of the first target object and the key image frame of the second target object.

In this embodiment, the key image frames in the video to be recognized are obtained through the prediction data of the first target object and the second target object, so that face recognition, living body recognition, human body recognition, face tracking, Operations such as human body tracking can achieve better results in various scenarios and fields such as intelligent security and intelligent identification.

In an example of the present disclosure, the recognition model training method can be applied to face and human body recognition, and may include steps as shown in Figure 5:

Step S51: Obtain an image to be recognized.

Specifically, image frames may be extracted from real-time video streams of cameras in surveillance or other scenes, and may be extracted frame by frame, or may be extracted at set intervals. The extracted image frames are first preprocessed and scaled to a fixed size, such as 416×416, and the uniform RGB mean value (such as [104, 117, 123]) is subtracted, so that the size and RGB mean value of each image to be recognized are in the recognition model to be trained The training process is unified, thereby enhancing the robustness of the trained recognition model.

Step S52: Input the image to be recognized into the model.

The preprocessed image to be recognized can be sent to the recognition model to be trained for calculation.

Step S53: Obtain the feature map of the image to be recognized.

The input data of the recognition model to be trained can be the image preprocessed in step S52 above, and processed by the backbone network to obtain first feature maps of different depths and scales. The structure of the backbone network can be the same as the backbone network of the YOLO Unified Real-Time Object Detection (You Only Look Once: Unified, Real-Time Object Detection) model, which can specifically include sub-networks with convolution calculation functions. The sub-networks can be, for example, DarkNet, Networks such as ResNet.

The smaller N sheets of the first feature map output by the backbone network are input into the feature pyramid network. Through FPN, the N first feature maps output by the backbone network are fused with each other through corresponding paths, and finally N feature maps of different scales are obtained. These N feature maps of different sizes can be used to perceive targets of different scales on the image from large to small.

Step S54: Obtain the first target object prediction data and the second target object prediction data.

In this example, the head of the recognition model to be trained is connected after the feature pyramid network, and the head may include a combination of several convolutional layers-activation layers-batch processing layers.

In this example, it can be preset that each feature map pixel position produces at least one anchor point frame with different size ratios, and a result can be regressed based on the anchor point frame. Each anchor box corresponds to an intermediate output data with a length of (5+N+M). The number of channels of the intermediate output data is 5+N+M), indicating the prediction (conf, x, y, w, h, class) of the target detection frame based on the anchor frame and the predicted value of the attribute. conf indicates the confidence of the target contained in the anchor box, x, y, w, h are the coordinates and scale of the normalized detection frame; class is a vector with a dimension of N, indicating that the probability of the target belonging to a certain category corresponds to Values within a vector of category indices; predicted values for attributes are vectors of length M.

The definition of the relationship between the human body and the human face can be as follows: the human body frame and the human face frame belonging to the same natural person are a group of related human body-face frames. When generating the training target, the label of the human body frame is generated on the anchor point (pixel) corresponding to the center point of each human body frame, and if the natural person corresponding to the human body frame has a face in the image, then on the same anchor point Generate the face frame associated with the human body.

When predicting, the pre-processed image will be input to the network to obtain all the body frames in the image and the corresponding face frames of the body, as well as the attributes corresponding to the body frames and the attributes corresponding to the face frames. The attributes corresponding to the body frame may include whether it is truncated, whether it is abnormal, the degree of occlusion, and the orientation. The attributes corresponding to the face frame may include quality, pitch angle, yaw angle and roll angle.

Use the human body frame and face frame for joint tracking, select human body key frames in different orientations without truncation, no abnormality, and low occlusion from the trajectory, and store them in the library, and select face key frames with high quality scores and small angles to store them in The stored key frame images can be used for subsequent operations related to target objects such as face retrieval.

The embodiments of the present disclosure can be applied to human face and/or recognition, only one deep learning model extraction is performed on the image to be recognized, and the detection frames of all human bodies and human faces on the image to be recognized, human body attributes, human face attributes, and human body and human body attributes are obtained. Correspondence between faces. Compared with using a single-stage model to simultaneously perform association detection and output of attribute results, the embodiments of the present disclosure reduce the computing resource overhead to the greatest extent, and at the same time directly output the association relationship between the face and the human body from the model, without the need to separate the face and the human body. Association judgment.

In an example of the present disclosure, the structure of the recognition model may be as shown in FIG. 6 , including a backbone network 61 , a feature pyramid network (FPN) 62 , and a head 63 . In the model training phase, according to the data output by the head 63, a loss (Loss) for optimizing the recognition model to be trained can be obtained. Through the backbone network 61, multiple first feature maps are output according to the image to be recognized, specifically C1, C2, C3, C4, and C5, and the size relationship is: C1>C2>C3>C4>C5. Through FPN62, C3, C4, and C5 are fused and calculated. For example, according to the order corresponding to C3, at least one of the processed first feature maps can be fused, and the second feature map P3 is output; according to the order corresponding to C4, fusion At least one of the processed first feature maps is output as a second feature map P4; according to the sequence corresponding to C5, at least one of the processed first feature maps is fused to output a second feature map P5. The head 63 can include a convolutional layer conv3×3, the number of output channels of the head is twice the number of input channels, and the output data can be respectively: face prediction data conv3×3C, 3(K+5+4), human body Prediction data conv3×3C, 3(K+5+4).

Among them, C is the number of feature channels input to the head, k is the number of categories, 5 is (x, y, w, h, conf), 4 is the three angles and quality of the face, and 11 is the 4 attributes of the human body Corresponding vector: whether the human body is normal (No, Yes); whether it is truncated (No, Yes); degree of occlusion (no occlusion, slight occlusion, severe occlusion); orientation (front, back, left side, right side).

According to the face prediction data and human body prediction data, the face quality-related data can be obtained respectively: face box (Face Box), face score (Face Score), face angle (Face Angle), face quality (Face Quality); and the quality-related data of the human body: Human Box, Human Score, and Human Quality.

For example, according to the recognition results shown in FIG. 7 , the face frame 71 and the human body frame 72 can be obtained, and the quality-related data of the face and human body can be obtained at the same time: normal human body, no occlusion, no truncation, and front. The frame image with the largest joint NMS (Non-Maximum Suppression, non-maximum suppression value) in the video to be identified can be selected as the key frame image.

The embodiment of the present disclosure also provides a recognition model training device, as shown in FIG. 8 , including:

The first input module 81 is used to input the image to be processed into the recognition model to be trained;

The feature map module 82 is used to output at least one feature map of the image to be processed through the feature output layer of the recognition model to be trained;

The prediction data module 83 is used to obtain the prediction data of the first target object in the image to be processed according to at least one feature map through the head of the recognition model to be trained, and the prediction of the first target object in the image to be processed Prediction data of the data-associated second target object;

The training module 84 is used to optimize the recognition model to be trained according to the predicted data of the first target object, the predicted data of the second target object, the labeled data of the first target object and the labeled data of the second target object, and obtain the trained Identify the model.

In one embodiment, as shown in Figure 9, the prediction data module includes:

The first prediction unit 91 is configured to output the anchor frame prediction data of the first target object and the anchor frame prediction data of the second target object for each pixel of the feature map;

The second prediction unit 92 is configured to output the predicted data of the first target object and the predicted data of the second target object according to the predicted data of the anchor frame of the first target object and the predicted data of the anchor frame of the second target object.

In one embodiment, as shown in Figure 10, the feature output layer includes a backbone network and a feature pyramid network; the feature map module includes:

The first feature map unit 101 is configured to output multiple first feature maps of the image to be processed through the backbone network;

The first feature map input unit 102 is used to input N second feature maps in multiple first feature maps into the feature pyramid network, where N is an integer not less than 1;

The second feature map unit 103 is used to output N second feature maps through the feature pyramid network;

The second feature map processing unit 104 is configured to use N second feature maps as feature maps.

An embodiment of the present disclosure also provides an image recognition device, as shown in FIG. 11 , including:

The second input module 111 is used to input the image to be recognized into the recognition model, and obtain the prediction data of the first target object in the image to be recognized and the prediction data of the second target object associated with the first target object. The recognition model is the disclosure The trained recognition model provided by any one of the embodiments.

In one embodiment, the image to be recognized is a frame image in the video to be recognized; as shown in Figure 12, the recognition device also includes:

The key frame image module 121 is configured to obtain a key frame image in the video to be recognized according to the prediction data of the first target object and the prediction data of the second target object.

For the functions of each unit, module or submodule in each device in the embodiment of the present disclosure, refer to the corresponding description in the above method embodiment, and details are not repeated here.

According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

FIG. 13 shows a schematic block diagram of an example electronic device 130 that may be used to implement embodiments of the present disclosure. Electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 13 , the electronic device 130 includes a computing unit 131 that can be executed according to a computer program stored in a read-only memory (ROM) 132 or loaded from a storage unit 138 into a random access memory (RAM) 133 Various appropriate actions and treatments. In the RAM 133, various programs and data necessary for the operation of the electronic device 130 can also be stored. The calculation unit 131, the ROM 132, and the RAM 133 are connected to each other through a bus 134. An input-output (I/O) interface 135 is also connected to the bus 134 .

Multiple components in the electronic device 130 are connected to the I/O interface 135, including: an input unit 136, such as a keyboard, a mouse, etc.; an output unit 137, such as various types of displays, speakers, etc.; a storage unit 138, such as a magnetic disk, an optical disk etc.; and a communication unit 139, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 139 allows the electronic device 130 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

The computing unit 131 may be various general and/or special purpose processing components having processing and computing capabilities. Some examples of computing units 131 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 131 executes various methods and processes described above, such as a recognition model training method. For example, in some embodiments, the recognition model training method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 138 . In some embodiments, part or all of the computer program can be loaded and/or installed on the electronic device 130 via the ROM 132 and/or the communication unit 139. When the computer program is loaded into the RAM 133 and executed by the computing unit 131, one or more steps of the identification model training method described above can be performed. Alternatively, in other embodiments, the computing unit 131 may be configured in any other appropriate way (for example, by means of firmware) to execute the recognition model training method.

Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips Implemented in a system of systems (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor Can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.

Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special purpose computer, or other programmable data processing devices, so that the program codes, when executed by the processor or controller, make the functions/functions specified in the flow diagrams and/or block diagrams Action is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

To provide for interaction with the user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user. ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Input from the user may be received through acoustic input, voice input, or tactile input.

The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN) and the Internet.

A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, each step described in the present disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in the present disclosure can be achieved, no limitation is imposed herein.

The specific implementation manners described above do not limit the protection scope of the present disclosure. It should be apparent to those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

The above are only preferred implementations of the present invention, and it should be noted that the above preferred implementations should not be regarded as limiting the present invention, and the scope of protection of the present invention should be based on the scope defined in the claims. For those skilled in the art, without departing from the spirit and scope of the present invention, some improvements and modifications can also be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

A recognition model training method, comprising:

Input the image to be processed into the recognition model to be trained;

Outputting at least one feature map of the image to be processed through the feature output layer of the recognition model to be trained;

Obtain the prediction data of the first target object in the image to be processed according to the at least one feature map through the head of the recognition model to be trained, and the prediction data of the first target object in the image to be processed and the first object in the image to be processed Predicted data of a second target object associated with the predicted data of the object;

According to the predicted data of the first target object, the predicted data of the second target object, the labeled data of the first target object and the labeled data of the second target object, the recognition model to be trained is optimized to obtain the trained recognition model.
The method according to claim 1, wherein the predicted data of the first target object comprises the category predicted data of the first target object and the attribute predicted data of the first target object; the predicted data of the second target object The prediction data includes prediction data of the second target object and property prediction data of the second target object.
The method according to any one of claims 1 or 2, wherein the prediction data of the first target object and the prediction data of the first target object are output through the head of the recognition model to be trained Predicted data of the second target object associated with the data, including:

For each pixel of the feature map, output anchor box prediction data of the first target object and anchor box prediction data of the second target object;

Outputting the predicted data of the first target object and the predicted data of the second target object according to the predicted data of the anchor frame of the first target object and the predicted data of the anchor frame of the second target object.
The method according to any one of claims 1-3, wherein the first target object is a human face; the second target object is a human body.
The method according to any one of claims 1-4, wherein the feature output layer includes a backbone network and a feature pyramid network; the feature output layer through the recognition model to be trained outputs the to-be-processed At least one feature map of the image, including:

Outputting multiple first feature maps of the image to be processed through the backbone network;

Input N second feature maps in the plurality of first feature maps into the feature pyramid network, where N is an integer not less than 1;

Through the feature pyramid network, output N third feature maps;

The N third feature maps are used as the feature maps.
A method of identification comprising:

Inputting the image to be recognized into the recognition model, obtaining the prediction data of the first target object in the image to be recognized and the prediction data of the second target object associated with the first target object, the recognition model is claim 1- The trained recognition model described in any one of 5.
The method according to claim 6, wherein the image to be recognized is a frame image in a video to be recognized; the method further comprises:

According to the prediction data of the first target object and the prediction data of the second target object, key frame images in the video to be recognized are obtained.
A recognition model training device, comprising:

The first input module is used to input the image to be processed into the recognition model to be trained;

A feature map module, configured to output at least one feature map of the image to be processed through the feature output layer of the recognition model to be trained;

A prediction data module, configured to obtain the prediction data of the first target object in the image to be processed according to the at least one feature map through the head of the recognition model to be trained, and the predicted data for a second target object associated with predicted data for the first target object;

The training module is used to optimize the recognition model to be trained according to the predicted data of the first target object, the predicted data of the second target object, the labeled data of the first target object and the labeled data of the second target object, and obtain the trained recognition model.
The apparatus according to claim 8, wherein the predicted data of the first target object comprises the category predicted data of the first target object and the attribute predicted data of the first target object; the predicted data of the second target object The prediction data includes prediction data of the second target object and property prediction data of the second target object.
The device according to any one of claims 8 or 9, wherein the predicted data module comprises:

The first prediction unit is configured to output the anchor frame prediction data of the first target object and the anchor frame prediction data of the second target object for each pixel of the feature map;

The second prediction unit is configured to output the prediction data of the first target object and the second target according to the prediction data of the anchor frame of the first target object and the prediction data of the anchor frame of the second target object. Object prediction data.
The device according to any one of claims 8-10, wherein the first target object is a human face; the second target object is a human body.
The device according to any one of claims 8-11, wherein the feature output layer includes a backbone network and a feature pyramid network; the feature map module includes:

The first feature map unit is configured to output a plurality of first feature maps of the image to be processed through the backbone network;

The first feature map input unit is configured to input N second feature maps in the plurality of first feature maps into the feature pyramid network, where N is an integer not less than 1;

The second feature map unit is used to output N third feature maps through the feature pyramid network;

The second feature map processing unit is configured to use the N third feature maps as the feature maps.
An identification device comprising:

The second input module is configured to input the image to be recognized into the recognition model, and obtain the prediction data of the first target object in the image to be recognized and the prediction data of the second target object associated with the first target object, the The recognition model is the trained recognition model described in any one of claims 8-12.
The device according to claim 13, wherein the image to be recognized is a frame image in a video to be recognized; the device further comprises:

A key frame image module, configured to obtain a key frame image in the video to be recognized according to the prediction data of the first target object and the prediction data of the second target object.
An electronic device, characterized in that it comprises:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor, so that the at least one processor can perform any one of claims 1-7. Methods.
A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to any one of claims 1-7.
A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.