CN115311680A - Human body image quality detection method and device, electronic equipment and storage medium - Google Patents
Human body image quality detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115311680A CN115311680A CN202210819601.XA CN202210819601A CN115311680A CN 115311680 A CN115311680 A CN 115311680A CN 202210819601 A CN202210819601 A CN 202210819601A CN 115311680 A CN115311680 A CN 115311680A
- Authority
- CN
- China
- Prior art keywords
- image
- human body
- sample
- quality detection
- quality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Abstract
The embodiment of the invention provides a human body image quality detection method, which comprises the following steps: acquiring an image to be detected, wherein the image to be detected comprises a target human body image; carrying out human body image quality detection on the to-be-detected image through the trained human body quality detection model to obtain a quality detection result of the target human body image; the human body quality detection model is obtained by training according to the sample human body image, the image quality label corresponding to the sample human body image and the human body segmentation label corresponding to the sample human body image. The implicit relation between the human body images and the human body image quality is learned through training the human body quality detection model, meanwhile, training of the human body quality detection model is assisted through the sample human body images and the corresponding human body segmentation labels, the human body quality detection model is made to learn the incidence relation between the human body segmentation and the human body image quality, and therefore the accuracy of a quality detection result of the human body quality detection model is improved.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a human body image quality detection method and device, electronic equipment and a storage medium.
Background
The digital management of personnel as important constituent units in digital cities is an essential part of the digital cities. Personnel's digital management mainly processes through the visual information to personnel, forms corresponding management scheme, for example files the management to personnel's snapshot image, carries out follow-up analysis and seeks through the personnel's image that files, can improve personnel's information management efficiency. The personnel snapshot image is filed and managed, namely the face image and the body image of the snapshot personnel are filed and filed, the filing management is influenced by the image quality, and the higher the image quality is, the better the filing management effect is. For human body image filing, the existing method generally adopts pedestrian re-identification and human body attribute identification, and the human body quality is a key factor influencing the effects of the pedestrian re-identification and the human body attribute identification. The commonly used human body quality evaluation method is based on human body key points for evaluation, however, the human body key points themselves may be inaccurate in some images, and further the human body quality evaluation is inaccurate.
Disclosure of Invention
The embodiment of the invention provides a human body image quality detection method, aiming at solving the problem of inaccurate human body quality evaluation in the prior art. Through sample human body images and corresponding image quality labels, the human body quality detection model is trained to learn the implicit relation between the human body images and the human body image quality, so that the human body quality detection model can output the quality detection result of a target human body image according to the implicit relation, meanwhile, the training of the human body quality detection model is assisted through the sample human body images and the corresponding human body segmentation labels, so that the human body quality detection model learns the incidence relation between human body segmentation and the human body image quality, and the accuracy of the quality detection result of the human body quality detection model is improved.
In a first aspect, an embodiment of the present invention provides a method for detecting quality of a human body image, where the method includes:
acquiring an image to be detected, wherein the image to be detected comprises a target human body image;
carrying out human body image quality detection on the to-be-detected image through the trained human body quality detection model to obtain a quality detection result of the target human body image;
the human body quality detection model is obtained by training according to a sample human body image, an image quality label corresponding to the sample human body image and a human body segmentation label corresponding to the sample human body image.
Optionally, before the human body image quality detection is performed on the image to be detected through the trained human body quality detection model to obtain the quality detection result of the target human body image, the method further includes:
acquiring a sample image and an initial human body quality detection model;
performing image segmentation on the sample image to obtain a sample human body image, and labeling the sample human body image according to a segmentation result to obtain a human body segmentation label corresponding to the sample human body image;
carrying out image quality annotation on the sample image to obtain a corresponding image quality label;
constructing a training data set based on the sample human body image, the human body segmentation label and the image quality label;
and training the initial human body quality detection model according to the training data set to obtain the trained human body quality detection model.
Optionally, the obtaining a sample image includes a plurality of sample human body images, performing image segmentation on the sample image to obtain a sample human body image, and labeling the sample human body image according to a segmentation result to obtain a human body segmentation label corresponding to the sample human body image, including:
performing image segmentation on the sample image to obtain a first sample human body image and a second sample human body image, wherein the first sample human body image is a human body image which is centered in the sample image and has the largest area;
and carrying out classification and labeling on the first sample human body image and the second sample human body image to obtain human body segmentation labels corresponding to the first sample human body image and the second sample human body image.
Optionally, the image quality labeling of the sample image to obtain a corresponding image quality label includes:
performing image processing on the first sample body image by a preset image processing method to obtain a third sample image;
according to the preset image processing method, carrying out first image quality annotation on the third sample image;
calculating an area ratio between the second sample human body image and the first sample human body image;
according to the area ratio, carrying out second image quality annotation on the sample image;
and obtaining an image quality label corresponding to the sample image based on the first image quality label and the second image quality label.
Optionally, the image processing the first sample body image by a preset image processing method to obtain a third sample image includes:
performing truncation processing on the first same human body image according to a preset truncation direction and a preset truncation ratio to obtain a truncated human body image, wherein the truncation direction and the truncation ratio are also used for performing truncation image quality annotation on the truncated human body image;
carrying out shielding processing on the first same body image according to a preset shielding direction and a shielding proportion to obtain a shielded body image, wherein the shielding direction and the shielding proportion are also used for carrying out shielding image quality marking on the shielded body image;
carrying out fuzzy processing on the first sample human body image according to a preset fuzzy parameter to obtain a fuzzy human body image, wherein the fuzzy parameter is also used for carrying out fuzzy image quality marking on the fuzzy human body image;
and obtaining a third sample image based on the truncated human body image, the shielded human body image and the blurred human body image.
Optionally, the initial human quality detection model includes a backbone network, a human segmentation branch network and a human quality branch network, and the training of the initial human quality detection model according to the training data set to obtain the trained human quality detection model includes:
extracting common features of the sample images through the backbone network;
predicting the common characteristics through the human body segmentation branch network to obtain human body segmentation prediction, and predicting the common characteristics through the human body quality branch network to obtain image quality prediction;
calculating a first loss between the human segmentation prediction and the human segmentation label, and calculating a second loss between the image quality prediction and the image quality label;
and performing iterative adjustment on parameters of the initial human body quality detection model according to the first loss and the second loss to obtain the trained human body quality detection model.
Optionally, the iteratively adjusting parameters of the initial human quality detection model according to the first loss and the second loss to obtain the trained human quality detection model includes:
according to the first loss and the second loss, network parameters corresponding to the backbone network, the human body segmentation branch network and the human body quality branch network are adjusted through random gradient descent;
and when the first loss and the second loss are minimum or the iteration times reach preset times, stopping training, and deleting the trained human body segmentation branch network to obtain the trained human body quality detection model.
In a second aspect, an embodiment of the present invention provides an apparatus for detecting human body image quality, where the apparatus includes:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring an image to be detected, and the image to be detected comprises a target human body image;
the detection module is used for detecting the quality of the human body image of the image to be detected through the trained human body quality detection model to obtain the quality detection result of the target human body image;
the human body quality detection model is obtained by training according to a sample human body image, an image quality label corresponding to the sample human body image and a human body segmentation label corresponding to the sample human body image.
In a third aspect, an embodiment of the present invention provides an electronic device, including: the human body image quality detection method comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor executes the computer program to realize the steps in the human body image quality detection method provided by the embodiment of the invention.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the human body image quality detection method provided by the embodiment of the present invention.
In the embodiment of the invention, an image to be detected is obtained, wherein the image to be detected comprises a target human body image; carrying out human body image quality detection on the to-be-detected image through the trained human body quality detection model to obtain a quality detection result of the target human body image; the human body quality detection model is obtained by training according to a sample human body image, an image quality label corresponding to the sample human body image and a human body segmentation label corresponding to the sample human body image. Through sample human body images and corresponding image quality labels, the human body quality detection model is trained to learn the implicit relationship between the human body images and the human body image quality, so that the human body quality detection model can output the quality detection result of a target human body image according to the implicit relationship, meanwhile, the training of the human body quality detection model is assisted through the sample human body images and corresponding human body segmentation labels, so that the human body quality detection model learns the association relationship between human body segmentation and the human body image quality, and the accuracy of the quality detection result of the human body quality detection model is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a human body image quality detection method according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of an initial human quality testing model according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a human body image quality detection apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of a human body image quality detection method according to an embodiment of the present invention, and as shown in fig. 1, the human body image quality detection method includes the following steps:
101. and acquiring an image to be detected.
In the embodiment of the invention, the image to be detected comprises a target human body image, and the image to be detected can be a picture or a video. The image to be detected can be obtained by uploading by a user or by snapshotting through image acquisition equipment, and it needs to be explained that the image to be detected can comprise one or more target human body images, the target human body images can be understood as human body images needing to be filed, the human body images can be understood as a part of images in the image to be detected, the image to be detected is a large image, and the target human body images are small images in the large image.
The target human body image can be understood as all human body images in the image to be detected, and can also be appointed human body images in the image to be detected.
102. And carrying out human body image quality detection on the to-be-detected image through the trained human body quality detection model to obtain a quality detection result of the target human body image.
In the embodiment of the invention, the human body image quality can be used for evaluating the human body integrity and definition in the image, the more complete the human body in the image is, the higher the human body image quality is, and similarly, the clearer the human body in the image is, the higher the human body image quality is.
Specifically, the image to be detected can be input into a trained human body quality detection model, the image to be detected is calculated through the human body quality detection model, and a quality detection result of a corresponding target human body image is output.
Further, the human body quality detection model is obtained by training according to the sample human body image, the image quality label corresponding to the sample human body image and the human body segmentation label corresponding to the sample human body image. The human body quality detection model can be a human body quality detection model constructed based on a convolutional neural network.
The image quality label is used for describing the real human body image quality of the sample human body image, and specifically, the image quality label may include labels of types such as truncation, occlusion, and blurring degree. The human body segmentation label is used for describing real human body segmentation information of the sample human body image, and the human body segmentation label can be the position of the human body in the image area occupied by the human body in the sample human body image.
In the embodiment of the invention, an image to be detected is obtained, wherein the image to be detected comprises a target human body image; carrying out human body image quality detection on the to-be-detected image through the trained human body quality detection model to obtain a quality detection result of the target human body image; the human body quality detection model is obtained by training according to a sample human body image, an image quality label corresponding to the sample human body image and a human body segmentation label corresponding to the sample human body image. Through sample human body images and corresponding image quality labels, the human body quality detection model is trained to learn the implicit relation between the human body images and the human body image quality, so that the human body quality detection model can output the quality detection result of a target human body image according to the implicit relation, meanwhile, the training of the human body quality detection model is assisted through the sample human body images and the corresponding human body segmentation labels, so that the human body quality detection model learns the incidence relation between human body segmentation and the human body image quality, and the accuracy of the quality detection result of the human body quality detection model is improved.
Optionally, before the quality of the image to be detected is detected through the trained human body quality detection model to obtain the quality detection result of the target human body image, a sample image and an initial human body quality detection model can be obtained; performing image segmentation on the sample image to obtain a sample human body image, and labeling the sample human body image according to a segmentation result to obtain a human body segmentation label corresponding to the sample human body image; carrying out image quality annotation on the sample image to obtain a corresponding image quality label; constructing a training data set based on the sample human body image, the human body segmentation label and the image quality label; and training the initial human body quality detection model according to the training data set to obtain the trained human body quality detection model.
In the embodiment of the invention, the sample image can be obtained by uploading by a user, or can be obtained by capturing by an image acquisition device, or can be obtained by an image generation method.
The initial human body quality detection model can be a human body quality detection model constructed based on a convolutional neural network. For example, the human body quality detection model may be constructed based on convolutional neural networks such as ResNet, mobileNet, convNeXt, and the like.
Furthermore, the sample image can be segmented through an image segmentation algorithm to obtain a sample human body image. The sample image may include one or more sample human body images, the sample human body image may be understood as a human body image to be segmented, the human body image may be understood as a part of an image in the sample image, the sample image is a large image, and the sample human body image is a small image in the large image. After the sample human body image is obtained, the sample human body image can be labeled according to the position area of the sample human body image, and a human body segmentation label is obtained. For example, if the sample human body image is labeled as a subject person or a non-subject person, the corresponding label is a subject person label (the corresponding label value may be 1) or a non-subject person (the corresponding label value may be 0).
Furthermore, the image quality label can be obtained by adopting the existing quality evaluation algorithm or by manually marking the image quality of the sample image by an expert. The image quality labeling may include labeling the types of the sample human body image such as the truncation ratio, the occlusion ratio, the blur degree, and the like, to obtain labels of the types of the sample image such as the truncation, the occlusion, the blur degree, and the like.
Each sample human body image corresponds to one sample image, the human body segmentation label is associated with the sample image, the image quality label is associated with the sample image, and a training data set is constructed; training an initial human body quality detection model according to a training data set, predicting through an initial human body quality detection model sample image to obtain a prediction result, calculating the loss between the prediction result and a human body segmentation label and an image quality label, training and optimizing the initial human body quality detection model by minimizing the loss, adjusting model parameters of the initial human body quality detection model, and obtaining the trained human body quality detection model when the initial human body quality detection model converges or reaches a preset iteration number.
The initial human body quality detection model is trained through the image quality label, the initial human body quality detection model can learn the quality detection capability of the human body image, the human body segmentation label is added to the sample image, the training of the human body quality detection model can be assisted, the human body quality detection model can learn the incidence relation between the human body segmentation and the human body image quality, meanwhile, human body key point detection is not needed, and therefore the accuracy of the quality detection result of the human body quality detection model is improved.
Optionally, in the step of performing image segmentation on the sample image to obtain a sample human body image, and labeling the sample human body image according to a segmentation result to obtain a human body segmentation label corresponding to the sample human body image, the sample image may be further subjected to image segmentation to obtain a first sample human body image and a second sample human body image, where the first sample human body image is a human body image centered in the sample image and having a largest area; and carrying out classification and labeling on the first sample human body image and the second sample human body image to obtain human body segmentation labels corresponding to the first sample human body image and the second sample human body image.
In the embodiment of the invention, the sample image can be subjected to image segmentation through the existing human body image segmentation algorithm, the sample human body image in the sample image is segmented, and the human body of each person in the sample image corresponds to one sample human body image.
The first sample human body image may be a subject human figure in the sample image or may be referred to as a subject human in the sample image, and the second sample human body image may be a non-subject human figure in the sample image or may be referred to as a non-subject human in the sample image. The first sample human body image may be a human body image centered in the sample image and having a largest area, wherein the human body image in the sample image is the second sample human body image. And marking the first sample human body image and the second sample human body image to obtain a human body segmentation label. For example, the first sample human body image may be labeled as a subject person, and the second sample human body image may be labeled as a non-subject person, so that the label corresponding to the first sample human body image is a subject person label (the corresponding label value may be 1), and the label corresponding to the second sample human body image is a non-subject person (the corresponding label value may be 0).
By adding the human body segmentation labels to the sample images, training of the human body quality detection model can be assisted, so that the human body quality detection model learns the incidence relation between human body segmentation and human body image quality, and meanwhile, human body key point detection is not required, so that the accuracy of the quality detection result of the human body quality detection model is improved.
Optionally, in the step of performing image quality annotation on the sample image to obtain the corresponding image quality label, the first sample body image may be subjected to image processing by a preset image processing method to obtain a third sample image; according to a preset image processing method, carrying out first image quality annotation on the third sample image; calculating the area ratio between the second sample human body image and the first sample human body image; according to the area ratio, carrying out second image quality annotation on the sample image; and obtaining an image quality label corresponding to the sample image based on the first image quality label and the second image quality label.
In the embodiment of the invention, after the first sample human body image and the second sample human body image are segmented in the sample image through the image segmentation algorithm, the first sample human body image can be extracted from the sample image, the first sample human body image is subjected to image processing through a preset image processing method to obtain a third sample image, and then the third sample image is returned to the sample image to form a new sample image.
The image processing method may include image processing methods such as image truncation, image occlusion, and image blurring, and after the first sample human body image is subjected to image processing to obtain the third sample image, the corresponding image processing method may be used as annotation information of the third sample human body image to obtain the first image quality annotation. For example, if the upper body of the first sample body image is truncated at an upper body truncation ratio of 20%, the label information corresponding to the third sample image is at an upper body truncation ratio of 20%, and the image quality label corresponding to the new sample image includes a label at an upper body truncation ratio of 20%.
The area ratio can be a ratio of the sum of the areas of the second sample human body images to the area of the first sample human body image, the distribution relation of the first sample human body image and the second sample human body image can be obtained by representing the degree of multiple persons of the sample images through the area ratio, and the larger the area ratio is, the larger the image area occupied by the second sample human body image is, the smaller the image area occupied by the first sample human body image is, and the poorer the image quality is; the smaller the area ratio is, the smaller the image area occupied by the second sample human body image is, the larger the image area occupied by the first sample human body image is, and the better the image quality is. And taking the area ratio as a second image quality label of the sample image, so that the image quality label corresponding to the sample image can be obtained according to the first image quality label and the second image quality label.
Optionally, in the step of performing image processing on the first sample body image by using a preset image processing method to obtain a third sample image, the first sample body image may be cut according to a preset cutting direction and a cutting proportion to obtain a cut body image, and the cutting direction and the cutting proportion are further used for performing cut image quality labeling on the cut body image; carrying out shielding processing on the first same body image according to a preset shielding direction and a preset shielding proportion to obtain a shielded body image, wherein the shielding direction and the shielding proportion are also used for carrying out shielding image quality marking on the shielded body image; carrying out fuzzy processing on the first sample human body image according to a preset fuzzy parameter to obtain a fuzzy human body image, wherein the fuzzy parameter is also used for carrying out fuzzy image quality annotation on the fuzzy human body image; and obtaining a third sample image based on the truncated human body image, the shielded human body image and the blurred human body image.
In an embodiment of the present invention, the third sample image may include a truncated human body image, an occluded human body image, and a blurred human body image, and the image processing method may include image processing methods such as image truncation, image occlusion, and image blurring. Specifically, the image truncation includes a truncation direction and a truncation ratio, the truncation direction may be a vertical direction, a horizontal direction, or the like, and the truncation direction and the truncation ratio may be randomly selected, and the truncation processing is performed on the first sample human body image by randomly selecting the truncation direction and the truncation ratio, so as to obtain the truncated human body image. The image shielding comprises a shielding direction and a shielding proportion, the shielding direction can be in the upper, lower, left and right directions, the shielding direction and the shielding proportion can be randomly selected, and the shielding direction and the shielding proportion are randomly selected to shield the first sample body image to obtain a shielded human body image. The image blur may include a blur type and a blur degree, the blur type may be motion blur, gaussian blur, or the like, and the first sample body image is blurred by randomly selecting the blur type and the blur degree to obtain a blurred body image.
By carrying out image processing on the first sample body image, the data volume of the sample image can be increased, thereby improving the accuracy of the human body quality detection model.
Optionally, the initial human quality detection model includes a backbone network, a human segmentation branch network and a human quality branch network, and in the step of training the initial human quality detection model according to the training data set to obtain a trained human quality detection model, the common features of the sample images can be extracted through the backbone network; predicting the shared characteristic through a human body segmentation branch network to obtain a human body segmentation prediction, and predicting the shared characteristic through a human body quality branch network to obtain an image quality prediction; calculating a first loss between the human segmentation prediction and the human segmentation label, and calculating a second loss between the image quality prediction and the image quality label; and according to the first loss and the second loss, carrying out iterative adjustment on parameters of the initial human body quality detection model to obtain a trained human body quality detection model.
In the embodiment of the present invention, the backbone network may be a convolutional neural network such as ResNet, mobileNet, convNeXt, or the like, and the backbone network is used for extracting common features of the human body segmentation branch network and the human body quality branch network. The human body segmentation branch network comprises an upper sampling layer and is used for extracting implicit information corresponding to human body segmentation in the common characteristics to obtain human body segmentation prediction of the sample image. The human body quality branch network comprises a pooling layer and a full-connection layer, the pooling layer can be a global average pooling layer, the number of the full-connection layers can be multiple, and the human body quality branch network is used for extracting implicit information corresponding to the human body image in the common characteristics to obtain image quality prediction of the sample image.
Specifically, referring to fig. 2, fig. 2 is a schematic structural diagram of an initial human quality detection model according to an embodiment of the present invention, as shown in fig. 2, an output end of a backbone network is connected to an input end of a human segmentation branch network, an output end of the backbone network is connected to an input end of the human quality branch network, a sample image is processed through the backbone network to obtain a feature map of common features, and the common features are respectively input to the human segmentation branch network and the human quality branch network, where the human segmentation branch network outputs a human segmentation prediction of the sample image, and the human quality branch network outputs an image quality prediction of the sample image. The image quality prediction may include a truncation ratio, a blocking ratio, a blurring degree, and a multi-person degree, and specifically, the image quality prediction may include a truncated ratio of the upper body, a truncated ratio of the lower body, a truncated ratio of the left side, a truncated ratio of the right side, a blocking ratio of the upper body, a blocking ratio of the lower body, a blocking ratio of the left side, a blocking ratio of the right side, a blurring degree, and a multi-person degree.
The first loss is used for representing the degree of the difference between the human body segmentation prediction and the human body segmentation label, and if the first loss is smaller, the smaller the degree of the difference between the human body segmentation prediction and the human body segmentation label is, the more accurate the model prediction is; if the first loss is larger, the larger the difference degree between the human body segmentation prediction and the human body segmentation label is, the more inaccurate the model prediction is. The second loss is used for representing the degree of the phase difference between the image quality prediction and the image quality label, and if the second loss is smaller, the degree of the phase difference between the image quality prediction and the image quality label is smaller, and the model prediction is more accurate; if the second loss is larger, the larger the difference degree between the image quality prediction and the image quality label is, the more inaccurate the model prediction is.
In the training process, iterative training is carried out on the initial human body quality detection model by taking the first loss and the second loss as minimum, and the training can be stopped until the initial human body quality detection model converges or reaches a preset iteration number, so that the trained human body quality detection model is obtained. By calculating the first loss between the human body segmentation prediction and the human body segmentation label and training the initial human body quality detection model by combining the second loss between the image quality prediction and the image quality label, the training effect can be improved, and the accuracy of the human body quality detection model is improved.
Optionally, in the step of performing iterative adjustment of parameters on the initial human quality detection model according to the first loss and the second loss to obtain the trained human quality detection model, network parameters corresponding to the backbone network, the human segmentation branch network, and the human quality branch network may be adjusted by random gradient descent according to the first loss and the second loss; and when the first loss and the second loss are minimum or the iteration times reach preset times, stopping training, and deleting the trained human body segmentation branch network to obtain the trained human body quality detection model.
In an embodiment of the present invention, the first loss may be calculated by a first loss function, and the first loss function may be represented by the following equation:
wherein the Loss is dice For the first loss, the predictor is a human segmentation prediction, and the true is a human segmentation label.
The second loss may be calculated by a second loss function, the second loss function may be an L2 loss function, and the L2 loss function may be referred to as an L2 norm loss function and may be referred to as a least squares error. In general, the sum of the squares of the differences from the image quality labels and the image quality predictions is minimized.
The first loss and the second loss can be added to obtain a total loss, when the total loss is minimum or the iteration times reach preset times, the training is stopped, and the trained human body segmentation branch network is deleted to obtain the trained human body quality detection model.
The human body segmentation branch network is used for assisting the training of the human body quality detection model, so that the detection accuracy of the human body quality detection model can be improved, and when the human body segmentation branch network is deployed, the corresponding quality detection result only needs to be output, and the corresponding human body segmentation prediction result does not need to be output, so that the human body segmentation branch network can be deleted, the model data volume during deployment is reduced, and the deployment speed and the running speed of the human body quality detection model are improved.
It should be noted that the human body image quality detection method provided by the embodiment of the present invention can be applied to devices such as smart phones, computers, servers, and the like, which can perform human body image quality detection.
Optionally, referring to fig. 3, fig. 3 is a schematic structural diagram of a human body image quality detection apparatus according to an embodiment of the present invention, and as shown in fig. 3, the apparatus includes:
the first acquisition module 301 is configured to acquire an image to be detected, where the image to be detected includes a target human body image;
the detection module 302 is configured to perform human body image quality detection on the to-be-detected image through the trained human body quality detection model to obtain a quality detection result of the target human body image;
the human body quality detection model is obtained by training according to a sample human body image, an image quality label corresponding to the sample human body image and a human body segmentation label corresponding to the sample human body image.
Optionally, the apparatus further comprises:
the second acquisition module is used for acquiring a sample image and an initial human body quality detection model;
the first labeling module is used for carrying out image segmentation on the sample image to obtain a sample human body image, and labeling the sample human body image according to a segmentation result to obtain a human body segmentation label corresponding to the sample human body image;
the second labeling module is used for performing image quality labeling on the sample image to obtain a corresponding image quality label;
the construction module is used for constructing a training data set based on the sample human body image, the human body segmentation label and the image quality label;
and the training module is used for training the initial human body quality detection model according to the training data set to obtain the trained human body quality detection model.
Optionally, the sample image includes a plurality of sample human body images, and the first labeling module includes:
the segmentation sub-module is used for carrying out image segmentation on the sample image to obtain a first sample human body image and a second sample human body image, wherein the first sample human body image is a human body image which is centered in the sample image and has the largest area;
and the first labeling submodule is used for classifying and labeling the first sample human body image and the second sample human body image to obtain human body segmentation labels corresponding to the first sample human body image and the second sample human body image.
Optionally, the second labeling module includes:
the first processing submodule is used for carrying out image processing on the first sample body image through a preset image processing method to obtain a third sample image;
the second labeling submodule is used for performing first image quality labeling on the third sample image according to the preset image processing method;
the first calculation submodule is used for calculating the area ratio between the second sample human body image and the first sample human body image;
the third labeling submodule is used for performing second image quality labeling on the sample image according to the area ratio;
and the second processing submodule is used for obtaining an image quality label corresponding to the sample image based on the first image quality label and the second image quality label.
Optionally, the first processing sub-module includes:
the first processing unit is used for carrying out truncation processing on the first sample human body image according to a preset truncation direction and a truncation ratio to obtain a truncated human body image, and the truncation direction and the truncation ratio are also used for carrying out truncation image quality marking on the truncated human body image;
the second processing unit is used for carrying out shielding processing on the first same human body image according to a preset shielding direction and a shielding proportion to obtain a shielded human body image, and the shielding direction and the shielding proportion are also used for carrying out shielding image quality marking on the shielded human body image;
the third processing unit is used for carrying out fuzzy processing on the first sample human body image according to preset fuzzy parameters to obtain a fuzzy human body image, and the fuzzy parameters are also used for carrying out fuzzy image quality marking on the fuzzy human body image;
and the fourth processing unit is used for obtaining a third sample image based on the truncated human body image, the shielded human body image and the blurred human body image.
Optionally, the initial human quality detection model includes a backbone network, a human body segmentation branch network and a human quality branch network, and the training module includes:
an extraction sub-module for extracting common features of the sample images through the backbone network;
the prediction sub-module is used for predicting the shared characteristic through the human body segmentation branch network to obtain a human body segmentation prediction and predicting the shared characteristic through the human body quality branch network to obtain an image quality prediction;
a second calculation sub-module for calculating a first loss between the human segmentation prediction and the human segmentation label and calculating a second loss between the image quality prediction and the image quality label;
and the adjusting submodule is used for carrying out parameter iterative adjustment on the initial human body quality detection model according to the first loss and the second loss to obtain the trained human body quality detection model.
Optionally, the adjusting sub-module includes:
an adjusting unit, configured to adjust, according to the first loss and the second loss, network parameters corresponding to the backbone network, the human body segmentation branch network, and the human body quality branch network through random gradient descent;
and the deleting unit is used for stopping training when the first loss and the second loss are minimum or the iteration times reach preset times, and deleting the trained human body segmentation branch network to obtain the trained human body quality detection model.
It should be noted that the human body image quality detection device provided by the embodiment of the present invention may be applied to devices such as a smart phone, a computer, and a server that can perform human body image quality detection.
The human body image quality detection device provided by the embodiment of the invention can realize each process realized by the human body image quality detection method in the method embodiment, and can achieve the same beneficial effect. To avoid repetition, further description is omitted here.
Referring to fig. 4, fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, as shown in fig. 4, including: a memory 402, a processor 401 and a computer program of a human image quality detection method stored on said memory 402 and executable on said processor 401, wherein:
the processor 401 is configured to call the computer program stored in the memory 402, and execute the following steps:
acquiring an image to be detected, wherein the image to be detected comprises a target human body image;
carrying out human body image quality detection on the to-be-detected image through the trained human body quality detection model to obtain a quality detection result of the target human body image;
the human body quality detection model is obtained by training according to a sample human body image, an image quality label corresponding to the sample human body image and a human body segmentation label corresponding to the sample human body image.
Optionally, before the human body image quality detection is performed on the image to be detected through the trained human body quality detection model to obtain the quality detection result of the target human body image, the method executed by the processor 401 further includes:
acquiring a sample image and an initial human body quality detection model;
carrying out image segmentation on the sample image to obtain a sample human body image, and labeling the sample human body image according to a segmentation result to obtain a human body segmentation label corresponding to the sample human body image;
carrying out image quality annotation on the sample image to obtain a corresponding image quality label;
constructing a training data set based on the sample human body image, the human body segmentation label and the image quality label;
and training the initial human body quality detection model according to the training data set to obtain the trained human body quality detection model.
Optionally, the performing, by the processor 401, image segmentation on the sample image to obtain a sample human body image, and labeling the sample human body image according to a segmentation result to obtain a human body segmentation label corresponding to the sample human body image includes:
performing image segmentation on the sample image to obtain a first sample human body image and a second sample human body image, wherein the first sample human body image is a human body image which is centered in the sample image and has the largest area;
and carrying out classification and labeling on the first sample human body image and the second sample human body image to obtain human body segmentation labels corresponding to the first sample human body image and the second sample human body image.
Optionally, the performing, by the processor 401, the image quality labeling on the sample image to obtain a corresponding image quality label includes:
performing image processing on the first sample body image by a preset image processing method to obtain a third sample image;
according to the preset image processing method, carrying out first image quality annotation on the third sample image;
calculating an area ratio between the second sample human body image and the first sample human body image;
according to the area ratio, carrying out second image quality annotation on the sample image;
and obtaining an image quality label corresponding to the sample image based on the first image quality label and the second image quality label.
Optionally, the performing, by the processor 401, the image processing on the first sample body image by using a preset image processing method to obtain a third sample image includes:
performing truncation processing on the first same human body image according to a preset truncation direction and a preset truncation ratio to obtain a truncated human body image, wherein the truncation direction and the truncation ratio are also used for performing truncation image quality annotation on the truncated human body image;
carrying out shielding processing on the first same body image according to a preset shielding direction and a shielding proportion to obtain a shielded body image, wherein the shielding direction and the shielding proportion are also used for carrying out shielding image quality marking on the shielded body image;
carrying out fuzzy processing on the first sample human body image according to a preset fuzzy parameter to obtain a fuzzy human body image, wherein the fuzzy parameter is also used for carrying out fuzzy image quality marking on the fuzzy human body image;
and obtaining a third sample image based on the truncated human body image, the shielded human body image and the blurred human body image.
Optionally, the initial human quality detection model includes a backbone network, a human segmentation branch network and a human quality branch network, and the processor 401 executes the training of the initial human quality detection model according to the training data set to obtain the trained human quality detection model, including:
extracting common features of the sample images through the backbone network;
predicting the common characteristics through the human body segmentation branch network to obtain human body segmentation prediction, and predicting the common characteristics through the human body quality branch network to obtain image quality prediction;
calculating a first loss between the human segmentation prediction and the human segmentation label, and calculating a second loss between the image quality prediction and the image quality label;
and performing iterative adjustment on parameters of the initial human body quality detection model according to the first loss and the second loss to obtain the trained human body quality detection model.
Optionally, the performing, by the processor 401, iterative adjustment of parameters on the initial human quality detection model according to the first loss and the second loss to obtain the trained human quality detection model includes:
according to the first loss and the second loss, network parameters corresponding to the backbone network, the human body segmentation branch network and the human body quality branch network are adjusted through random gradient descent;
and when the first loss and the second loss are minimum or the iteration times reach preset times, stopping training, and deleting the trained human body segmentation branch network to obtain the trained human body quality detection model.
The electronic equipment provided by the embodiment of the invention can realize each process realized by the human body image quality detection method in the method embodiment, and can achieve the same beneficial effect. To avoid repetition, further description is omitted here.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the human image quality detection method or the application-side human image quality detection method provided in the embodiment of the present invention, and can achieve the same technical effect, and is not described herein again to avoid repetition.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.
Claims (10)
1. A human body image quality detection method is characterized by comprising the following steps:
acquiring an image to be detected, wherein the image to be detected comprises a target human body image;
carrying out human body image quality detection on the image to be detected through a trained human body quality detection model to obtain a quality detection result of the target human body image;
the human body quality detection model is obtained by training according to a sample human body image, an image quality label corresponding to the sample human body image and a human body segmentation label corresponding to the sample human body image.
2. The human body image quality detection method according to claim 1, wherein before the human body image quality detection is performed on the image to be detected through the trained human body quality detection model to obtain the quality detection result of the target human body image, the method further comprises:
acquiring a sample image and an initial human body quality detection model;
carrying out image segmentation on the sample image to obtain a sample human body image, and labeling the sample human body image according to a segmentation result to obtain a human body segmentation label corresponding to the sample human body image;
carrying out image quality annotation on the sample image to obtain a corresponding image quality label;
constructing a training data set based on the sample human body image, the human body segmentation label and the image quality label;
and training the initial human body quality detection model according to the training data set to obtain the trained human body quality detection model.
3. The human body image quality detection method according to claim 2, wherein the sample image includes a plurality of sample human body images, and the image segmentation is performed on the sample image to obtain a sample human body image, and the sample human body image is labeled according to a segmentation result to obtain a human body segmentation label corresponding to the sample human body image, including:
performing image segmentation on the sample image to obtain a first sample human body image and a second sample human body image, wherein the first sample human body image is a human body image which is centered in the sample image and has the largest area;
and carrying out classification and labeling on the first sample human body image and the second sample human body image to obtain human body segmentation labels corresponding to the first sample human body image and the second sample human body image.
4. The human image quality detection method of claim 3, wherein the image quality labeling of the sample image to obtain a corresponding image quality label comprises:
performing image processing on the first sample body image by a preset image processing method to obtain a third sample image;
according to the preset image processing method, carrying out first image quality annotation on the third sample image;
calculating an area ratio between the second sample human body image and the first sample human body image;
according to the area ratio, carrying out second image quality annotation on the sample image;
and obtaining an image quality label corresponding to the sample image based on the first image quality label and the second image quality label.
5. The human image quality detection method of claim 4, wherein the image processing the first sample human image by a preset image processing method to obtain a third sample image comprises:
performing truncation processing on the first human body image according to a preset truncation direction and a preset truncation ratio to obtain a truncated human body image, wherein the truncation direction and the truncation ratio are also used for performing truncation image quality labeling on the truncated human body image;
carrying out shielding processing on the first same body image according to a preset shielding direction and a shielding proportion to obtain a shielded body image, wherein the shielding direction and the shielding proportion are also used for carrying out shielding image quality marking on the shielded body image;
carrying out fuzzy processing on the first sample human body image according to a preset fuzzy parameter to obtain a fuzzy human body image, wherein the fuzzy parameter is also used for carrying out fuzzy image quality annotation on the fuzzy human body image;
and obtaining a third sample image based on the truncated human body image, the shielded human body image and the blurred human body image.
6. The human image quality detection method according to any one of claims 2 to 5, wherein the initial human quality detection model comprises a backbone network, a human segmentation leg network and a human quality leg network, and the training of the initial human quality detection model according to the training data set to obtain the trained human quality detection model comprises:
extracting common features of the sample images through the backbone network;
predicting the common characteristics through the human body segmentation branch network to obtain human body segmentation prediction, and predicting the common characteristics through the human body quality branch network to obtain image quality prediction;
calculating a first loss between the human segmentation prediction and the human segmentation label, and calculating a second loss between the image quality prediction and the image quality label;
and performing iterative adjustment on parameters of the initial human body quality detection model according to the first loss and the second loss to obtain the trained human body quality detection model.
7. The human image quality detection method of claim 6, wherein the iteratively adjusting parameters of the initial human quality detection model according to the first loss and the second loss to obtain the trained human quality detection model comprises:
according to the first loss and the second loss, network parameters corresponding to the backbone network, the human body segmentation branch network and the human body quality branch network are adjusted through random gradient descent;
and when the first loss and the second loss are minimum or the iteration times reach preset times, stopping training, and deleting the trained human body segmentation branch network to obtain the trained human body quality detection model.
8. An apparatus for detecting image quality of a human body, the apparatus comprising:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring an image to be detected, and the image to be detected comprises a target human body image;
the detection module is used for detecting the quality of the human body image of the image to be detected through the trained human body quality detection model to obtain the quality detection result of the target human body image;
the human body quality detection model is obtained by training according to a sample human body image, an image quality label corresponding to the sample human body image and a human body segmentation label corresponding to the sample human body image.
9. An electronic device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the human image quality detection method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps in the human image quality detection method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210819601.XA CN115311680A (en) | 2022-07-12 | 2022-07-12 | Human body image quality detection method and device, electronic equipment and storage medium |
| PCT/CN2022/141457 WO2024011853A1 (en) | 2022-07-12 | 2022-12-23 | Human body image quality measurement method and apparatus, electronic device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210819601.XA CN115311680A (en) | 2022-07-12 | 2022-07-12 | Human body image quality detection method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115311680A true CN115311680A (en) | 2022-11-08 |
Family
ID=83857664
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210819601.XA Pending CN115311680A (en) | 2022-07-12 | 2022-07-12 | Human body image quality detection method and device, electronic equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115311680A (en) |
| WO (1) | WO2024011853A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024011853A1 (en) * | 2022-07-12 | 2024-01-18 | 青岛云天励飞科技有限公司 | Human body image quality measurement method and apparatus, electronic device, and storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111753702A (en) * | 2020-06-18 | 2020-10-09 | 上海高德威智能交通系统有限公司 | Target detection method, device and equipment |
| CN113011345B (en) * | 2021-03-11 | 2024-03-05 | 百度在线网络技术（北京）有限公司 | Image quality detection method, image quality detection device, electronic equipment and readable storage medium |
| CN113158860B (en) * | 2021-04-12 | 2022-06-24 | 烽火通信科技股份有限公司 | Deep learning-based multi-dimensional output face quality evaluation method and electronic equipment |
| CN113962965B (en) * | 2021-10-26 | 2023-06-09 | 腾讯科技（深圳）有限公司 | Image quality evaluation method, device, equipment and storage medium |
| CN115311680A (en) * | 2022-07-12 | 2022-11-08 | 青岛云天励飞科技有限公司 | Human body image quality detection method and device, electronic equipment and storage medium |
-
2022
- 2022-07-12 CN CN202210819601.XA patent/CN115311680A/en active Pending
- 2022-12-23 WO PCT/CN2022/141457 patent/WO2024011853A1/en unknown
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024011853A1 (en) * | 2022-07-12 | 2024-01-18 | 青岛云天励飞科技有限公司 | Human body image quality measurement method and apparatus, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024011853A1 (en) | 2024-01-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111178183B (en) | Face detection method and related device | |
| CN111222500B (en) | Label extraction method and device | |
| CN108229418B (en) | Human body key point detection method and apparatus, electronic device, storage medium, and program | |
| CN110176024B (en) | Method, device, equipment and storage medium for detecting target in video | |
| CN112488071A (en) | Method, device, electronic equipment and storage medium for extracting pedestrian features | |
| CN110610123A (en) | Multi-target vehicle detection method and device, electronic equipment and storage medium | |
| CN111368634B (en) | Human head detection method, system and storage medium based on neural network | |
| CN111814690A (en) | Target re-identification method and device and computer readable storage medium | |
| Feng et al. | A novel saliency detection method for wild animal monitoring images with WMSN | |
| CN110599514B (en) | Image segmentation method and device, electronic equipment and storage medium | |
| CN115311680A (en) | Human body image quality detection method and device, electronic equipment and storage medium | |
| CN115690545A (en) | Training target tracking model and target tracking method and device | |
| CN117475253A (en) | Model training method and device, electronic equipment and storage medium | |
| CN113065379B (en) | Image detection method and device integrating image quality and electronic equipment | |
| KR102230559B1 (en) | Method and Apparatus for Creating Labeling Model with Data Programming | |
| CN116091781B (en) | Data processing method and device for image recognition | |
| CN112329550A (en) | Weak supervision learning-based disaster-stricken building rapid positioning evaluation method and device | |
| CN115908831B (en) | Image detection method and device | |
| CN112070035A (en) | Target tracking method and device based on video stream and storage medium | |
| CN115719428A (en) | Face image clustering method, device, equipment and medium based on classification model | |
| CN113255549B (en) | Intelligent recognition method and system for behavior state of wolf-swarm hunting | |
| CN114638304A (en) | Training method of image recognition model, image recognition method and device | |
| CN110751034B (en) | Pedestrian behavior recognition method and terminal equipment | |
| CN111127327B (en) | Picture inclination detection method and device | |
| Rani et al. | Object Detection in Natural Scene Images Using Thresholding Techniques |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |