CN108229288B

CN108229288B - Neural network training and clothes color detection method and device, storage medium and electronic equipment

Info

Publication number: CN108229288B
Application number: CN201710487723.2A
Authority: CN
Inventors: 邵婧; 刘希慧; 闫俊杰
Original assignee: Beijing Sensetime Technology Development Co Ltd
Current assignee: Beijing Sensetime Technology Development Co Ltd
Priority date: 2017-06-23
Filing date: 2017-06-23
Publication date: 2020-08-11
Anticipated expiration: 2037-06-23
Also published as: CN108229288A

Abstract

The embodiment of the invention provides a neural network training and clothes color detection method, a neural network training and clothes color detection device, a storage medium and electronic equipment. The training method of the neural network comprises the following steps: extracting a clothing region image from the sample image based on the key points of the human body, wherein the clothing region image contains color feature calibration data of the pixel points of the human body clothing; acquiring color feature detection data of the clothing region image from the clothing region image through a clothing color extraction network; and training the clothing color extraction network according to the color feature calibration data and the color feature detection data. By the embodiment of the invention, the clothing color extraction network obtained by training can be accurately positioned to the area image associated with the clothing of the human body, the interference of the background area image is reduced, and the accuracy of detecting the clothing color is improved.

Description

Neural network training and clothes color detection method and device, storage medium and electronic equipment

Technical Field

The embodiment of the invention relates to an artificial intelligence technology, in particular to a training method, a training device, a storage medium and electronic equipment for a neural network, and a detection method, a detection device, a storage medium and electronic equipment for human body clothes color.

Background

Human body clothes color detection has wide application in network image search, such as image search, clothes retrieval, style analysis and the like, and in monitoring image retrieval, such as human body re-identification and the like, and particularly in monitoring cross-camera scenes, the accuracy of pedestrian retrieval can be improved by accurately detecting the color of the pedestrian clothes.

However, pedestrian garment color detection is not a simple task and faces a number of challenges. Due to the fact that the definition of the monitoring camera is low, the angle change range of the camera is wide, the scene illumination is variable, the scene complexity is high and the like, the detection difficulty of colors of clothes of pedestrians in a monitoring scene is increased seriously, even the colors of the clothes of the same pedestrian in each frame of the same monitoring video can be changed obviously, the styles of the clothes of the pedestrians are varied, the color categories of the coats of the pedestrians are not only one, but also the coats of the pedestrians comprise different color categories such as spliced sleeves, grid shirts, striped shirts and various patterns (such as round dots, cartoon patterns or trademarks). In addition, pedestrian tops sometimes include multiple pieces of clothing, such as shirts and coats. Unlike other rigid objects (e.g., vehicles), clothing is non-rigid, and clothing wrinkles and creases can affect the accuracy of clothing color detection.

The existing detection method for the colors of the clothes of the pedestrians comprises a method for directly judging the colors of the clothes without training and learning and a method based on statistical characteristic information and machine learning, is not robust in the problem of illumination or scene inconsistency caused by cross-camera, and can only describe partial color characteristics of the clothes in a limited way. In recent years, the deep convolutional neural network has excellent performance in extracting image features and training models, so that good effects are obtained in tasks such as image classification and object detection. Recently, there are also some methods based on convolutional neural networks to detect pedestrian clothing color. However, in these methods, generally, a classification method is adopted for detection, and in order to obtain high accuracy, a full convolution neural network is adopted for pixel-level classification, so that time consumption from labeling to training is long, and accurate manual labeling is required for clothes color detection by adopting the classification method, and in view of that the color types and styles of the clothes of pedestrians in a monitoring scene are complex, the manual labeling cost is high, and the reliability is low. In addition, the existing clothes color detection method based on the convolutional neural network still utilizes the whole pedestrian image to learn the characteristics and train, and sometimes the clothes color detection is wrong due to the interference of complex background colors, sundry colors and the like.

Disclosure of Invention

The embodiment of the invention aims to provide a technical scheme for neural network training and a technical scheme for human body clothes color detection.

According to a first aspect of embodiments of the present invention, a method for training a neural network is provided. The method comprises the following steps: extracting a clothing region image from the sample image based on the key points of the human body, wherein the clothing region image contains color feature calibration data of the pixel points of the human body clothing; acquiring color feature detection data of the clothing region image from the clothing region image through a clothing color extraction network; and training the clothing color extraction network according to the color feature calibration data and the color feature detection data.

Optionally, the garment region image comprises: a first region image, and/or a second region image; the first area image contains first color feature calibration data of the human body coat pixel points, and the second area image contains second color feature calibration data of the human body coat pixel points.

Optionally, the obtaining, by the clothing color extraction network, color feature detection data of the clothing region image from the clothing region image includes: and acquiring first color feature detection data corresponding to the first area image and/or second color feature detection data corresponding to the second area image from the first area image and/or the second area image through the clothing color extraction network.

Optionally, the training the clothing color extraction network according to the color feature calibration data and the color feature detection data includes: training the clothing color extraction network according to the first color feature calibration data and the first color feature detection data; and/or training the clothing color extraction network according to the second color feature calibration data and the second color feature detection data.

Optionally, before extracting the clothing region image from the sample image based on the key points of the human body, the method further includes: acquiring color feature data of a plurality of first pixel points between at least two human body key points of the upper half of the human body in the sample image, and calibrating the color feature data as first color feature calibration data in the first region image; and/or acquiring color feature data of a plurality of second pixel points between at least two human key points of the lower half of the human body in the sample image, and calibrating the second color feature data in the second region image.

Optionally, the obtaining color feature data of a plurality of first pixel points between at least two human key points of the upper half of the human body in the sample image and calibrating the first color feature calibration data in the first region image includes: determining a first color median value of the human body jacket according to the color feature data of each pixel point in the first region image; based on the first color median value, selecting a pixel point which meets a first set condition between at least two human key points of the upper half of the human body as the first pixel point, and determining the color feature data of the selected first pixel point as first color feature calibration data in the first region image; wherein the first setting condition includes: and a first distance between the color characteristic data of the pixel point and the median value of the first color does not exceed a first preset threshold value.

Optionally, the obtaining color feature data of a plurality of second pixel points between at least two human key points of the lower half of the human body in the sample image and calibrating the second color feature calibration data in the second region image includes: determining a second color median value of the human body lower garment according to the color feature data of each pixel point in the second region image; based on the second color median value, selecting a pixel point which meets a second setting condition between at least two human key points of the lower half of the human body as the second pixel point, and determining the color feature data of the selected second pixel point as second color feature calibration data in the second region image; wherein the second setting condition includes: and a second distance between the color characteristic data of the pixel point and the median value of the second color does not exceed a second preset threshold value.

Optionally, the extracting a clothing region map from the sample image based on the key points of the human body includes: determining second coordinate data of a region of the first region image in the sample image based on first coordinate data of a human key point of the upper half of the human body, and extracting the first region image from the sample image according to the second coordinate data; and/or determining fourth coordinate data of the region of the second region image in the sample image based on third coordinate data of the human key points of the lower half of the human body, and extracting the second region image from the sample image according to the fourth coordinate data.

According to a second aspect of the embodiments of the present invention, there is provided a method for detecting a color of a human body garment. The method comprises the following steps: extracting a clothing region image from a human body image to be detected based on the human body key points; acquiring color feature prediction data of the clothing region image from the clothing region image through a clothing color extraction network, wherein the clothing color extraction network is obtained by training according to the method of the first aspect of the embodiment of the invention.

Optionally, the extracting a clothing region image from a human body image to be detected based on the human body key points includes: and extracting a third area image associated with the human body upper garment and/or a fourth area image associated with the human body lower garment from the human body image to be detected based on the human body key points.

Optionally, the obtaining, by the clothing color extraction network, color feature prediction data of the clothing region image from the clothing region image includes: and acquiring color feature prediction data of the human body upper garment and/or color feature prediction data of the human body lower garment from the third area image and/or the fourth area image through the garment color extraction network.

Optionally, the extracting, based on the human body key point, a third region image associated with a human body upper garment and/or a fourth region image associated with a human body lower garment from the human body image to be detected includes: determining sixth coordinate data of a region of the third region image in the human body image to be detected based on fifth coordinate data of human body key points of the upper half of the human body, and extracting the third region image from the human body image to be detected according to the sixth coordinate data; and/or determining eighth coordinate data of a region of the fourth region image in the human body image to be detected based on seventh coordinate data of human body key points of the lower half of the human body, and extracting the fourth region image from the human body image to be detected according to the eighth coordinate data.

Optionally, the human body image to be detected is a static image, or a video image in a video frame sequence.

According to a third aspect of the embodiments of the present invention, there is provided a training apparatus for a neural network. The device comprises: the first extraction module is used for extracting a clothing region image from the sample image based on the key points of the human body, wherein the clothing region image contains color characteristic calibration data of the pixel points of the human body clothing; the first acquisition module is used for acquiring color feature detection data of the clothing region image from the clothing region image through a clothing color extraction network; and the training module is used for training the clothing color extraction network according to the color feature calibration data and the color feature detection data.

Optionally, the first obtaining module includes: the first obtaining sub-module is used for obtaining first color feature detection data corresponding to the first area image and/or second color feature detection data corresponding to the second area image from the first area image and/or the second area image through the clothing color extraction network.

Optionally, the training module comprises: the first training submodule is used for training the clothing color extraction network according to the first color feature calibration data and the first color feature detection data; and/or a second training submodule, configured to train the clothing color extraction network according to the second color feature calibration data and the second color feature detection data.

Optionally, the apparatus further comprises: the second acquisition module is used for acquiring color feature data of a plurality of first pixel points between at least two human key points of the upper half of the human body in the sample image and calibrating the color feature data as first color feature calibration data in the first region image; and/or a third obtaining module, configured to obtain color feature data of multiple second pixel points between at least two human key points of the lower half of the human body in the sample image, and calibrate the data for the second color feature in the second region image.

Optionally, the second obtaining module is specifically configured to: determining a first color median value of the human body jacket according to the color feature data of each pixel point in the first region image; based on the first color median value, selecting a pixel point which meets a first set condition between at least two human key points of the upper half of the human body as the first pixel point, and determining the color feature data of the selected first pixel point as first color feature calibration data in the first region image; wherein the first setting condition includes: and a first distance between the color characteristic data of the pixel point and the median value of the first color does not exceed a first preset threshold value.

Optionally, the third obtaining module is specifically configured to: determining a second color median value of the human body lower garment according to the color feature data of each pixel point in the second region image; based on the second color median value, selecting a pixel point which meets a second setting condition between at least two human key points of the lower half of the human body as the second pixel point, and determining the color feature data of the selected second pixel point as second color feature calibration data in the second region image; wherein the second setting condition includes: and a second distance between the color characteristic data of the pixel point and the median value of the second color does not exceed a second preset threshold value.

Optionally, the first extraction module includes: the first extraction submodule is used for determining second coordinate data of a region of the first region image in the sample image based on first coordinate data of a human key point of the upper half of the human body, and extracting the first region image from the sample image according to the second coordinate data; and/or the second extraction submodule is used for determining fourth coordinate data of a region of the second region image in the sample image based on third coordinate data of a human key point of the lower half of the human body, and extracting the second region image from the sample image according to the fourth coordinate data.

According to a fourth aspect of the embodiments of the present invention, there is provided a device for detecting a color of a human body garment. The device comprises: the second extraction module is used for extracting a clothing region image from a human body image to be detected based on the human body key points; a fourth obtaining module, configured to obtain color feature prediction data of the clothing region image from the clothing region image through a clothing color extraction network, where the clothing color extraction network is obtained by training the apparatus according to the third aspect of the embodiment of the present invention.

Optionally, the second extraction module includes: and the third extraction submodule is used for extracting a third area image related to the human body upper garment and/or a fourth area image related to the human body lower garment from the human body image to be detected based on the human body key points.

Optionally, the fourth obtaining module includes: and the second obtaining submodule is used for obtaining the color feature prediction data of the human body upper garment and/or the color feature prediction data of the human body lower garment from the third area image and/or the fourth area image through the garment color extraction network.

Optionally, the third extraction sub-module is specifically configured to: determining sixth coordinate data of a region of the third region image in the human body image to be detected based on fifth coordinate data of human body key points of the upper half of the human body, and extracting the third region image from the human body image to be detected according to the sixth coordinate data; and/or determining eighth coordinate data of a region of the fourth region image in the human body image to be detected based on seventh coordinate data of human body key points of the lower half of the human body, and extracting the fourth region image from the human body image to be detected according to the eighth coordinate data.

According to a fifth aspect of embodiments of the present invention, there is provided a computer readable storage medium having stored thereon computer program instructions, wherein the program instructions, when executed by a processor, implement the steps of the training method of a neural network according to the first aspect of embodiments of the present invention.

According to a sixth aspect of the embodiments of the present invention, there is provided a computer readable storage medium having stored thereon computer program instructions, wherein the program instructions, when executed by a processor, implement the steps of the method for detecting human body clothing color according to the second aspect of the embodiments of the present invention.

According to a seventh aspect of the embodiments of the present invention, there is provided an electronic apparatus including: the device comprises a first processor, a first memory, a first communication element and a first communication bus, wherein the first processor, the first memory and the first communication element are communicated with each other through the first communication bus; the first memory is used for storing at least one executable instruction, and the executable instruction causes the first processor to execute the steps of the training method of the neural network according to the first aspect of the embodiment of the invention.

According to an eighth aspect of the embodiments of the present invention, there is provided an electronic apparatus including: the second processor, the second memory, the second communication element and the second communication bus are communicated with each other through the second communication bus; the second memory is used for storing at least one executable instruction, and the executable instruction causes the second processor to execute the steps of the method for detecting the color of the human body clothes according to the second aspect of the embodiment of the invention.

According to the neural network training scheme provided by the embodiment of the invention, based on key points of a human body, a clothing region image containing color feature calibration data of pixel points of clothing of the human body is extracted from a sample image, the color feature detection data of the clothing region image is obtained from the clothing region image through a clothing color extraction network, and then the clothing color extraction network is trained according to the color feature calibration data and the color feature detection data.

Drawings

Fig. 1 is a flowchart of a training method of a neural network according to a first embodiment of the present invention;

FIG. 2 is a flow chart of a neural network training method according to a second embodiment of the present invention;

FIG. 3 is a flow chart of a method for detecting human body clothes color according to a third embodiment of the present invention;

FIG. 4 is a flowchart of a method for detecting human body clothes color according to a fourth embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a training apparatus for neural networks according to a fifth embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a training apparatus for a neural network according to a sixth embodiment of the present invention;

fig. 7 is a block diagram of a human body clothes color detecting apparatus according to a seventh embodiment of the present invention;

fig. 8 is a block diagram of a human body clothes color detecting apparatus according to an eighth embodiment of the present invention;

fig. 9 is a schematic structural diagram of an electronic device according to a ninth embodiment of the present invention;

fig. 10 is a schematic structural diagram of an electronic device according to a tenth embodiment of the present invention.

Detailed Description

The following detailed description of embodiments of the invention is provided in conjunction with the accompanying drawings (like numerals indicate like elements throughout the several views) and examples. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.

It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present invention are used merely to distinguish one element, step, device, module, or the like from another element, and do not denote any particular technical or logical order therebetween.

Example one

Fig. 1 is a flowchart of a training method of a neural network according to a first embodiment of the present invention.

Referring to fig. 1, in step S101, a clothing region image is extracted from a sample image based on a key point of a human body.

In this embodiment, the human body key points may include at least one of the following: a chin, a left shoulder joint point, a right elbow joint point, a left elbow joint point, a right wrist joint point, a left wrist joint point, a right hip joint point, a left hip joint point, a right knee joint point, a left knee joint point, a right ankle joint point, and a left ankle joint point. The sample image may be an image including an upper half of a human body, an image including a lower half of a human body, an image including the entire human body, or the like. The clothing region image is an image of a region related to the human clothing in the sample image and contains color feature calibration data of human clothing pixels. The color feature calibration data may be RGB values.

In step S102, color feature detection data of the clothing region image is acquired from the clothing region image through a clothing color extraction network.

In embodiments of the present invention, the clothing color extraction network may be any suitable neural network that can implement feature extraction or target object detection, including but not limited to a convolutional neural network, an reinforcement learning neural network, a generation network in a countering neural network, and so on. The specific configuration of the neural network may be set by those skilled in the art according to actual requirements, such as the number of convolutional layers, the size of convolutional core, the number of channels, and the like, which is not limited in this embodiment of the present invention. Wherein the color feature detection data may be RGB values.

In step S103, the clothing color extraction network is trained according to the color feature calibration data and the color feature detection data.

In particular embodiments, this step may comprise: and determining color feature difference according to the color feature calibration data and the color feature detection data, and adjusting network parameters of the clothing color extraction network according to the color feature difference. And evaluating the currently obtained color feature detection data by calculating the color feature difference of the human body garment to be used as a basis for extracting a network for the color of the subsequent training garment.

Specifically, the color feature differences may be transmitted back to the garment color extraction network, thereby iteratively training the garment color extraction network. The training of the clothing color extraction network is an iterative process, and the embodiment of the invention only describes one training process, but it should be understood by those skilled in the art that the training mode can be adopted for each training of the clothing color extraction network until the training of the clothing color extraction network is completed.

The invention provides a training method of a neural network, which is characterized in that based on key points of a human body, a clothing region image containing color feature calibration data of clothing pixel points of the human body is extracted from a sample image, through a clothing color extraction network, color feature detection data of the clothing region image is obtained from the clothing region image, and then the clothing color extraction network is trained according to the color feature calibration data and the color feature detection data.

The training method of the neural network of the present embodiment may be performed by any suitable device having data processing capabilities, including but not limited to: terminal equipment, a server and the like.

Example two

Fig. 2 is a flowchart of a neural network training method according to a second embodiment of the present invention.

Referring to fig. 2, in step S201, a clothing region image is extracted from a sample image based on a key point of a human body.

Wherein the garment region image comprises: the image processing method comprises the steps of obtaining a first area image and/or a second area image, wherein the first area image comprises first color feature calibration data of human body coat pixel points, the second area image comprises second color feature calibration data of human body coat pixel points, and the first color feature calibration data and the second color feature calibration data can be RGB values. Of course, in the embodiment of the present invention, the area images included in the clothing area image are not limited to one area image or two area images, and may be any number. For example, an image of a hat area, an image of a hairstyle area (detecting hair color), an image of a shoe area, an image of a backpack area, and the like. Specifically, the step may include: and extracting a first region image associated with the human body upper garment and/or a second region image associated with the human body lower garment from the sample image based on the human body key points.

In this embodiment, before executing step S201, the method further includes: extracting the human key points from the sample image through a neural network for extracting the human key points. The neural network for extracting the key points of the human body includes, but is not limited to, a full convolution neural network, an reinforcement learning neural network, a generation network in an antagonistic neural network, and the like.

Specifically, before this step S201, and after the human body key point is extracted from the sample image, the method further includes: acquiring color feature data of a plurality of first pixel points between at least two human body key points of the upper half of the human body in the sample image, and calibrating the color feature data as first color feature calibration data in the first region image; and/or acquiring color feature data of a plurality of second pixel points between at least two human key points of the lower half of the human body in the sample image, and calibrating the second color feature data in the second region image.

In a specific embodiment, the device with data processing capability selects N pixel points on a connection line between key points as first pixel points according to key points of the upper half of the human body in the sample image, and uses RGB values of the N pixel points as color feature calibration data in the first region image. Similarly, the device with data processing capability selects M pixel points on a connecting line between the key points as second pixel points according to the key points of the lower half of the human body in the sample image, and uses RGB values of the M pixel points as color feature calibration data in the second region image. Wherein N and M each represent a natural number of 5 or less. The device with data processing capability can obtain the first color feature calibration data in the first region image associated with the human jacket according to the human key points of the upper half of the human body in the sample image, and can also obtain the second color feature calibration data in the second region image associated with the human lower garment according to the human key points of the lower half of the human body in the sample image. Therefore, the workload of manual marking is reduced, and the cost of manual marking and the unreliable intervention of manual marking are reduced.

Optionally, the obtaining color feature data of a plurality of first pixel points between at least two human key points of the upper half of the human body in the sample image and calibrating the first color feature calibration data in the first region image includes: determining a first color median value of the human body jacket according to the color feature data of each pixel point in the first region image; based on the first color median value, selecting a pixel point which meets a first set condition between at least two human key points of the upper half of the human body as the first pixel point, and determining the color feature data of the selected first pixel point as first color feature calibration data in the first region image. Wherein the first setting condition includes: and a first distance between the color characteristic data of the pixel point and the median value of the first color does not exceed a first preset threshold value. Thereby, the first pixel can be prevented from being clicked on the background image having a large difference in color.

In a specific embodiment, the device with data processing capability calculates an average RGB value of the human body jacket, that is, a median value of colors of the human body jacket, according to the RGB value of each pixel point in the first region image. When the Euclidean distance between the RGB value of the pixel point on the connecting line between the key points of the upper half of the human body and the RGB average value of the human body jacket is smaller than or equal to a first preset threshold value, the equipment can determine the pixel point as a first pixel point, and determine the RGB value of the pixel point as first color feature calibration data in the first region image. Wherein, the first preset threshold value can be obtained by testing in actual operation by a person skilled in the art. The device with data processing capability may automatically obtain first color feature calibration data in the first region image associated with the human jacket based on human keypoints of the upper body of the human in the sample image. Therefore, the workload of manual marking is reduced, and the cost of manual marking and the unreliable intervention of manual marking are reduced.

Optionally, the obtaining color feature data of a plurality of second pixel points between at least two human key points of the lower half of the human body in the sample image and calibrating the second color feature calibration data in the second region image includes: determining a second color median value of the human body lower garment according to the color feature data of each pixel point in the second region image; and selecting pixel points which accord with a second set condition between at least two human key points of the lower half of the human body as second pixel points based on the second color median value, and determining the color feature data of the selected second pixel points as second color feature calibration data in the second region image. Wherein the second setting condition includes: and a second distance between the color characteristic data of the pixel point and the median value of the second color does not exceed a second preset threshold value. Thereby, the second pixel can be prevented from being clicked on the background image having a large difference in color.

In a specific embodiment, the device with data processing capability calculates an RGB average value of the human lower garment according to the RGB value of each pixel point in the second region image, that is, a color median value of the human lower garment. When the Euclidean distance between the RGB value of the pixel point on the connecting line between the key points of the lower half of the human body and the RGB average value of the lower garment of the human body is smaller than or equal to a second preset threshold, the device can determine the pixel point as a second pixel point, and determine the RGB value of the pixel point as second color feature calibration data in the second region image. Wherein the second preset threshold value can be obtained by testing in actual operation by a person skilled in the art. The device having data processing capabilities may automatically obtain second color feature calibration data in a second region image associated with the person's lower torso in the sample image based on the person's key points of the person's lower torso. Therefore, the workload of manual marking is reduced, and the cost of manual marking and the unreliable intervention of manual marking are reduced.

Compared with the prior art, the method and the device have the advantage that the first color feature calibration data of the first region image associated with the human body jacket and the second color feature calibration data of the second region image associated with the human body lower jacket are automatically obtained through the device with the data processing capacity, so that the processes of pixel point classification, manual determination of marked pixel points, manual marking of the determined marked pixel points and the like are saved.

In practical application, the first area image and the second area image can be sent to a marker, and at the moment, the marker only needs to check whether N pixel points selected from the first area image are all on the coat and whether M pixel points selected from the second area image are all on the coat. And if some pixel points are not on the coat, reselecting one coat pixel point. And if some pixel points are not on the lower garment, reselecting one pixel point of the lower garment.

Specifically, the step S201 includes: determining second coordinate data of a region of the first region image in the sample image based on first coordinate data of a human key point of the upper half of the human body, and extracting the first region image from the sample image according to the second coordinate data; and/or determining fourth coordinate data of the region of the second region image in the sample image based on third coordinate data of the human key points of the lower half of the human body, and extracting the second region image from the sample image according to the fourth coordinate data.

Taking a human body jacket as an example, let K be 4 key points { (x _ K, y _ K), K be 1,.. 4}, and the manner of obtaining the region image associated with the human body jacket be { [ min (x _1, x _2), min (y _1, y _2) ], [ min (x _1, x _2), max (y _1, y _2) ], [ max (x _3, x _4), min (y _3, y _4) ], [ max (x _3, x _4), max (y _3, y _4) ] }. The region image associated with the human body jacket obtained in this way is a regular rectangle.

In step S202, first color feature detection data corresponding to the first area image and/or second color feature detection data corresponding to the second area image are acquired from the first area image and/or the second area image through the clothing color extraction network.

In this embodiment, the clothing color extraction network may be a deep neural network based on a regression mechanism, the deep neural network includes a convolution layer, a pooling layer, a batch normalization layer (batch normalization), and the like to learn color features of the sample human body image, and finally, three RGB color channel values are obtained through regression. For example, the garment color extraction network may be google net, VGG, or ResNet, among others.

Specifically, the clothing color extraction network may be a neural network specially trained for a human body upper garment, may be a neural network specially trained for a human body lower garment, and may also be a neural network trained for a human body upper garment and a human body lower garment. When the clothing color extraction network is a neural network trained for a human body coat and a human body lower coat, the clothing color extraction network is provided with two input ends and two output ends, wherein one input end is used for inputting a first area image associated with the human body coat, the other input end is used for inputting a second area image associated with the human body lower coat, one output end is used for outputting first color feature detection data of the first area image, and the other output end is used for outputting second color feature detection data of the second area image. Wherein the first color feature detection data and the second color feature detection data may both be RGB values.

In step S203, training the clothing color extraction network according to the first color feature calibration data and the first color feature detection data; and/or training the clothing color extraction network according to the second color feature calibration data and the second color feature detection data.

Specifically, the step S203 includes: determining a first color feature difference of the human body coat according to the first color feature calibration data and the first color feature detection data, and/or determining a second color feature difference of the human body coat according to the second color feature calibration data and the second color feature detection data; and adjusting network parameters of the clothing color extraction network according to the first color feature difference and/or the second color feature difference.

In a specific embodiment, the determining a first color feature difference of the human body upper garment according to the first color feature calibration data and the first color feature detection data, and/or determining a second color feature difference of the human body lower garment according to the second color feature calibration data and the second color feature detection data includes: determining first Euclidean distances between first color feature calibration data of each human body jacket pixel point contained in the first region image and the first color feature detection data respectively, calculating an average value of the first Euclidean distances, and determining the average value of the first Euclidean distances as a first color feature difference of the human body jacket; and/or determining second Euclidean distances between second color feature calibration data of each human body underclothes pixel point contained in the second region image and the second color feature detection data, calculating an average value of the second Euclidean distances, and determining the average value of the second Euclidean distances as a second color feature difference of the human body underclothes.

An exemplary embodiment of the present invention is directed to a method for training a neural network, which includes extracting a first area image associated with a human upper garment and/or a second area image associated with a human lower garment from a sample image based on a human key point, obtaining first color feature detection data corresponding to the first area image and/or second color feature detection data corresponding to the second area image from the first area image and/or the second area image through a garment color extraction network, determining a first color feature difference of the human upper garment according to the first color feature calibration data and the first color feature detection data, and/or determining a second color feature difference of the human lower garment according to the second color feature calibration data and the second color feature detection data, and adjusting a garment color according to the first color feature difference and/or the second color feature difference Compared with the prior art, the method has the advantages that the network parameters of the network are extracted, so that the trained clothes color extraction network can be accurately positioned to the area image associated with the human body upper garment and/or the area image associated with the human body lower garment, the interference of the background area image is reduced, and the accuracy of detecting the clothes color is improved.

EXAMPLE III

Fig. 3 is a flowchart of a method for detecting colors of human clothes according to a third embodiment of the present invention.

Referring to fig. 3, in step S301, a clothing region image is extracted from a human body image to be detected based on human body key points.

The human body image to be detected may be an image including the upper body of the human body, an image including the lower body of the human body, an image including the entire human body, or the like, in terms of the content included in the image. From the category of images, the human body image to be detected can be a shot static image, or a video image in a video frame sequence, or a synthesized image, etc. In particular, the human body image to be detected may be a video image in a sequence of video frames in a live broadcast. The clothing region image may be an image of a region associated with the clothing of the human body in the human body image to be detected.

In step S302, color feature prediction data of the clothing region image is acquired from the clothing region image through a clothing color extraction network.

The clothing color extraction network is obtained by training according to the neural network training method provided by the first embodiment of the application or the second embodiment of the application. The color feature prediction data of the clothing region image may be RGB values.

According to the method for detecting the color of the human body clothing, the clothing region image is extracted from the human body image to be detected based on the key points of the human body, the color feature prediction data of the clothing region image is obtained from the clothing region image through the clothing color extraction network, the region image related to the human body clothing can be accurately positioned, the interference of the background region image is reduced, and the accuracy of detecting the color of the clothing is improved.

The method for detecting the color of the human body clothing of the embodiment can be performed by any suitable device with data processing capability, including but not limited to: terminal equipment, a server and the like.

Example four

Fig. 4 is a flowchart of a method for detecting colors of human clothes according to a fourth embodiment of the present invention.

Referring to fig. 4, in step S401, based on the key points of the human body, a third region image associated with a human body top and/or a fourth region image associated with a human body bottom is extracted from the human body image to be detected.

In this embodiment, before executing step S401, the method further includes: and extracting the human key points from the human image to be detected through a neural network for extracting the human key points. The neural network for extracting the key points of the human body includes, but is not limited to, a full convolution neural network, an reinforcement learning neural network, a generation network in an antagonistic neural network, and the like.

Specifically, this step S401 includes: determining sixth coordinate data of a region of the third region image in the human body image to be detected based on fifth coordinate data of human body key points of the upper half of the human body, and extracting the third region image from the human body image to be detected according to the sixth coordinate data; and/or determining eighth coordinate data of a region of the fourth region image in the human body image to be detected based on seventh coordinate data of human body key points of the lower half of the human body, and extracting the fourth region image from the human body image to be detected according to the eighth coordinate data.

In step S402, the color feature prediction data of the human body upper garment and/or the color feature prediction data of the human body lower garment are obtained from the third area image and/or the fourth area image through the garment color extraction network.

The clothing color extraction network is obtained by training according to the neural network training method provided by the first embodiment of the application or the second embodiment of the application. The color feature prediction data of the human body upper garment and the color feature prediction data of the human body lower garment can be RGB values.

The method provided by the embodiment can be used for searching and tracking evasion in the surveillance video, for example, if a suspect wears red jacket blue trousers in the surveillance video, the suspect can be quickly searched in other surveillance videos according to the clothes color, and the search range of the suspect is narrowed.

According to the method for detecting the color of the human body clothes, the third area image related to the human body upper garment and/or the fourth area image related to the human body lower garment are extracted from the human body image to be detected based on the human body key points, the color feature prediction data of the human body upper garment and/or the color feature prediction data of the human body lower garment are obtained from the third area image and/or the fourth area image through the garment color extraction network, the area image related to the human body upper garment and/or the area image related to the human body lower garment can be accurately positioned, interference of the background area image is reduced, and accuracy of detecting the color of the clothes is improved.

EXAMPLE five

Based on the same technical concept, fig. 5 is a schematic structural diagram illustrating a training apparatus of a neural network according to a fifth embodiment of the present invention. The method of training a neural network according to the first embodiment can be performed.

Referring to fig. 5, the training apparatus of the neural network includes a first extraction module 501, a first acquisition module 502, and a training module 503.

A first extraction module 501, configured to extract a clothing region image from a sample image based on a human body key point, where the clothing region image includes color feature calibration data of a human body clothing pixel point;

a first obtaining module 502, configured to obtain, from the clothing region image, color feature detection data of the clothing region image through a clothing color extraction network;

a training module 503, configured to train the clothing color extraction network according to the color feature calibration data and the color feature detection data.

According to the training device of the neural network, based on key points of a human body, a clothing region image containing color feature calibration data of clothing pixel points of the human body is extracted from a sample image, through a clothing color extraction network, color feature detection data of the clothing region image is obtained from the clothing region image, and then the clothing color extraction network is trained according to the color feature calibration data and the color feature detection data.

EXAMPLE six

Based on the same technical concept, fig. 6 is a schematic structural diagram illustrating a training apparatus of a neural network according to a sixth embodiment of the present invention. The method can be used to perform the training method procedure of the neural network as described in the second embodiment.

Referring to fig. 6, the training apparatus of the neural network includes a first extraction module 603, a first acquisition module 604, and a training module 605. The first extraction module 603 is configured to extract a clothing region image from the sample image based on the key points of the human body, where the clothing region image includes color feature calibration data of the clothing pixel points of the human body; a first obtaining module 604, configured to obtain, from the clothing region image, color feature detection data of the clothing region image through a clothing color extraction network; a training module 605, configured to train the clothing color extraction network according to the color feature calibration data and the color feature detection data.

Specifically, the clothing region image includes: a first region image, and/or a second region image; the first area image contains first color feature calibration data of the human body coat pixel points, and the second area image contains second color feature calibration data of the human body coat pixel points.

Optionally, the first obtaining module 604 includes: a first obtaining sub-module 6041, configured to obtain, through the clothing color extraction network, first color feature detection data corresponding to the first area image and/or second color feature detection data corresponding to the second area image from the first area image and/or the second area image.

Optionally, the training module 605 includes: a first training submodule 6051 configured to train the clothing color extraction network according to the first color feature calibration data and the first color feature detection data; and/or a second training submodule 6052 configured to train the clothing color extraction network according to the second color feature calibration data and the second color feature detection data.

Optionally, the apparatus further comprises: a second obtaining module 601, configured to obtain color feature data of a plurality of first pixel points between at least two human key points of the upper half of the human body in the sample image, and obtain first color feature calibration data in the first region image; and/or a third obtaining module 602, configured to obtain color feature data of a plurality of second pixel points between at least two human key points of the lower half of the human body in the sample image, and calibrate data for a second color feature in the second region image.

Optionally, the second obtaining module 601 is specifically configured to: determining a first color median value of the human body jacket according to the color feature data of each pixel point in the first region image; based on the first color median value, selecting a pixel point which meets a first set condition between at least two human key points of the upper half of the human body as the first pixel point, and determining the color feature data of the selected first pixel point as first color feature calibration data in the first region image; wherein the first setting condition includes: and a first distance between the color characteristic data of the pixel point and the median value of the first color does not exceed a first preset threshold value.

Optionally, the third obtaining module 602 is specifically configured to: determining a second color median value of the human body lower garment according to the color feature data of each pixel point in the second region image; based on the second color median value, selecting a pixel point which meets a second setting condition between at least two human key points of the lower half of the human body as the second pixel point, and determining the color feature data of the selected second pixel point as second color feature calibration data in the second region image; wherein the second setting condition includes: and a second distance between the color characteristic data of the pixel point and the median value of the second color does not exceed a second preset threshold value.

Optionally, the first extracting module 603 includes: a first extraction sub-module 6031 configured to determine, based on first coordinate data of a key point of a human body on the upper half of the human body, second coordinate data of a region in which the first region image is located in the sample image, and extract the first region image from the sample image according to the second coordinate data; and/or the second extraction sub-module 6032 is configured to determine fourth coordinate data of a region where the second region image is located in the sample image based on third coordinate data of a human key point of the lower half of the human body, and extract the second region image from the sample image according to the fourth coordinate data.

It should be noted that, specific details related to the training apparatus for a neural network provided in the embodiment of the present invention have been described in detail in the training method for a neural network provided in the embodiment of the present invention, and are not described herein again.

EXAMPLE seven

Based on the same technical concept, fig. 7 is a block diagram illustrating a structure of a human body clothes color detecting apparatus according to a seventh embodiment of the present invention. The method can be used for executing the human body clothes color detection method flow as described in the third embodiment.

Referring to fig. 7, the apparatus for detecting a color of human clothing includes a second extraction module 701 and a fourth acquisition module 702.

A second extraction module 701, configured to extract a clothing region image from a human body image to be detected based on a human body key point;

a fourth obtaining module 702, configured to obtain color feature prediction data of the clothing region image from the clothing region image through a clothing color extraction network,

wherein the garment color extraction network is trained according to the device of embodiment five or embodiment six.

According to the detection device for the color of the human body clothes, the clothing region image is extracted from the human body image to be detected based on the key points of the human body, the color feature prediction data of the clothing region image is obtained from the clothing region image through the clothing color extraction network, the region image related to the human body clothes can be accurately positioned, the interference of the background region image is reduced, and the accuracy of detecting the color of the clothes is improved.

Example eight

Based on the same technical concept, fig. 8 is a block diagram illustrating a human body clothes color detecting apparatus according to an eighth embodiment of the present invention. The method can be used for executing the human body clothes color detection method flow as described in the fourth embodiment.

Referring to fig. 8, the apparatus for detecting human clothing color includes a second extraction module 801 and a fourth acquisition module 802. A second extraction module 801, configured to extract a clothing region image from a human body image to be detected based on a human body key point; a fourth obtaining module 802, configured to obtain color feature prediction data of the clothing region image from the clothing region image through a clothing color extraction network, where the clothing color extraction network is obtained by training according to the apparatus in the fifth embodiment or the sixth embodiment.

Optionally, the second extraction module 801 includes: the third extracting sub-module 8011 is configured to extract, based on the key points of the human body, a third area image associated with a human body upper garment and/or a fourth area image associated with a human body lower garment from the human body image to be detected.

Optionally, the fourth obtaining module 802 includes: a second obtaining sub-module 8021, configured to obtain, through the clothing color extraction network, the color feature prediction data of the human body upper garment and/or the color feature prediction data of the human body lower garment from the third area image and/or the fourth area image.

Optionally, the third extraction sub-module 8011 is specifically configured to: determining sixth coordinate data of a region of the third region image in the human body image to be detected based on fifth coordinate data of human body key points of the upper half of the human body, and extracting the third region image from the human body image to be detected according to the sixth coordinate data; and/or determining eighth coordinate data of a region of the fourth region image in the human body image to be detected based on seventh coordinate data of human body key points of the lower half of the human body, and extracting the fourth region image from the human body image to be detected according to the eighth coordinate data.

It should be noted that, specific details related to the device for detecting colors of human body clothes provided in the embodiment of the present invention have been described in detail in the method for detecting colors of human body clothes provided in the embodiment of the present invention, and are not described herein again.

Example nine

The embodiment of the invention also provides electronic equipment, which can be a mobile terminal, a Personal Computer (PC), a tablet computer, a server and the like. Referring now to fig. 9, shown is a schematic diagram of an electronic device 900 suitable for use as a terminal device or server for implementing embodiments of the present invention. As shown in fig. 9, the electronic device 900 includes one or more first processors, such as: one or more Central Processing Units (CPUs) 901, and/or one or more image processors (GPUs) 913 and the like, the first processor may perform various appropriate actions and processes according to executable instructions stored in a Read Only Memory (ROM)902 or executable instructions loaded from a storage section 908 into a Random Access Memory (RAM) 903. In this embodiment, the first read only memory 902 and the random access memory 903 are collectively referred to as a first memory. The first communication element includes a communication component 912 and/or a communication interface 909. Among them, the communication component 912 may include, but is not limited to, a network card, which may include, but is not limited to, an ib (infiniband) network card, the communication interface 909 includes a communication interface of a network interface card such as a LAN card, a modem, or the like, and the communication interface 909 performs communication processing via a network such as the internet.

The first processor may communicate with the read-only memory 902 and/or the random access memory 903 to execute executable instructions, connect with the communication component 912 through the first communication bus 904, and communicate with other target devices through the communication component 912, thereby completing operations corresponding to any neural network training method provided by the embodiments of the present invention, for example, extracting a clothing region image from a sample image based on key points of a human body, where the clothing region image contains color feature calibration data of pixel points of the clothing of the human body; acquiring color feature detection data of the clothing region image from the clothing region image through a clothing color extraction network; and training the clothing color extraction network according to the color feature calibration data and the color feature detection data.

In addition, in the RAM903, various programs and data necessary for the operation of the device can also be stored. The CPU901 or GPU913, ROM902, and RAM903 are connected to each other via a first communication bus 904. The ROM902 is an optional module in case of the RAM 903. The RAM903 stores or writes executable instructions into the ROM902 at runtime, and the executable instructions cause the first processor to perform operations corresponding to the above-described communication method. An input/output (I/O) interface 905 is also connected to the first communication bus 904. The communication component 912 may be integrated or may be configured with multiple sub-modules (e.g., IB cards) and linked over a communication bus.

The following components are connected to the I/O interface 905: an input portion 906 including a keyboard, a mouse, and the like; an output section 907 including components such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 908 including a hard disk and the like; and a communication interface 909 including a network interface card such as a LAN card, a modem, or the like. The drive 910 is also connected to the I/O interface 905 as necessary. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 910 as necessary, so that a computer program read out therefrom is mounted into the storage section 908 as necessary.

It should be noted that the architecture shown in fig. 9 is only an optional implementation manner, and in a specific practical process, the number and types of the components in fig. 9 may be selected, deleted, added or replaced according to actual needs; in different functional component settings, separate settings or integrated settings may also be used, for example, the GPU and the CPU may be separately set or the GPU may be integrated on the CPU, the communication element may be separately set, or the GPU and the CPU may be integrated, and so on. These alternative embodiments are all within the scope of the present invention.

In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method illustrated in the flowchart, the program code may include instructions corresponding to performing the steps of the method provided by embodiments of the present invention, for example, extracting a clothing region image from a sample image based on key points of a human body, the clothing region image containing color feature calibration data of pixel points of clothing of the human body; acquiring color feature detection data of the clothing region image from the clothing region image through a clothing color extraction network; and training the clothing color extraction network according to the color feature calibration data and the color feature detection data. In such an embodiment, the computer program may be downloaded and installed from a network via the communication element, and/or installed from the removable medium 911. The computer program, when executed by the first processor, performs the above-described functions defined in the method of an embodiment of the invention.

Example ten

The embodiment of the invention also provides electronic equipment, which can be a mobile terminal, a Personal Computer (PC), a tablet computer, a server and the like. Referring now to fig. 10, shown is a schematic diagram of an electronic device 1000 suitable for use as a terminal device or server for implementing embodiments of the present invention. As shown in fig. 10, the electronic device 1000 includes one or more second processors, such as: one or more Central Processing Units (CPUs) 1001, and/or one or more image processors (GPUs) 1013, etc., the second processor may perform various appropriate actions and processes according to executable instructions stored in a Read Only Memory (ROM)1002 or executable instructions loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. In this embodiment, the second read only memory 1002 and the random access memory 1003 are collectively referred to as a second memory. The second communication element includes a communication component 1012 and/or a communication interface 1009. Among other things, the communication component 1012 may include, but is not limited to, a network card, which may include, but is not limited to, an ib (infiniband) network card, the communication interface 1009 includes a communication interface such as a network interface card of a LAN card, a modem, or the like, and the communication interface 1009 performs communication processing via a network such as the internet.

The second processor may communicate with the read-only memory 1002 and/or the random access memory 1003 to execute executable instructions, connect with the communication component 1012 through the second communication bus 1004, and communicate with other target devices through the communication component 1012, so as to complete operations corresponding to any method for detecting human body clothing color provided by the embodiment of the present invention, for example, extracting clothing region images from human body images to be detected based on human body key points; and acquiring color feature prediction data of the clothing region image from the clothing region image through a clothing color extraction network, wherein the clothing color extraction network is obtained by training according to the method of the first embodiment or the second embodiment.

In addition, in the RAM1003, various programs and data necessary for the operation of the device can be stored. The CPU1001 or GPU1013, the ROM1002, and the RAM1003 are connected to each other by a second communication bus 1004. The ROM1002 is an optional module in the case of the RAM 1003. The RAM1003 stores executable instructions or writes executable instructions into the ROM1002 at runtime, and the executable instructions cause the second processor to execute operations corresponding to the communication method described above. An input/output (I/O) interface 1005 is also connected to the second communication bus 1004. The communication component 1012 may be integrated or configured with multiple sub-modules (e.g., IB cards) and linked over a communication bus.

The following components are connected to the I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 1008 including a hard disk and the like; and a communication interface 1009 including a network interface card such as a LAN card, a modem, or the like. The driver 1010 is also connected to the I/O interface 1005 as necessary. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1010 as necessary, so that a computer program read out therefrom is mounted into the storage section 1008 as necessary.

It should be noted that the architecture shown in fig. 10 is only an optional implementation manner, and in a specific practical process, the number and types of the components in fig. 10 may be selected, deleted, added or replaced according to actual needs; in different functional component settings, separate settings or integrated settings may also be used, for example, the GPU and the CPU may be separately set or the GPU may be integrated on the CPU, the communication element may be separately set, or the GPU and the CPU may be integrated, and so on. These alternative embodiments are all within the scope of the present invention.

In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method illustrated in the flowchart, the program code may include instructions corresponding to performing the steps of the method provided by embodiments of the present invention, e.g., extracting a clothing region image from a human body image to be detected based on human body key points; and acquiring color feature prediction data of the clothing region image from the clothing region image through a clothing color extraction network, wherein the clothing color extraction network is obtained by training according to the method of the first embodiment or the second embodiment. In such an embodiment, the computer program may be downloaded and installed from a network via the communication element, and/or installed from the removable medium 1011. The computer program, when executed by the second processor, performs the above-described functions defined in the method of an embodiment of the invention.

It should be noted that, according to the implementation requirement, each component/step described in the present application may be divided into more components/steps, and two or more components/steps or partial operations of the components/steps may also be combined into a new component/step to achieve the purpose of the embodiment of the present invention.

The method and apparatus, device of the present invention may be implemented in a number of ways. For example, the method, apparatus and device of the embodiments of the present invention may be implemented by software, hardware, firmware or any combination of software, hardware and firmware. The above-described order for the steps of the method is for illustrative purposes only, and the steps of the method of the embodiments of the present invention are not limited to the order specifically described above unless specifically stated otherwise. Furthermore, in some embodiments, the present invention may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing methods according to embodiments of the present invention. Thus, the present invention also covers a recording medium storing a program for executing the method according to an embodiment of the present invention.

The description of the present embodiments has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed, and many modifications and variations will be apparent to those skilled in the art. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A method of training a neural network, the method comprising:

generating color characteristic calibration data of a clothing region image for the sample image according to the key points of the human body, wherein the clothing region image comprises: the first region image comprises first color characteristic calibration data of human body coat pixel points, and/or the second region image comprises second color characteristic calibration data of human body coat pixel points;

extracting a clothing region image from the sample image based on the key points of the human body;

acquiring color feature detection data of the clothing region image from the clothing region image through a clothing color extraction network;

training the clothing color extraction network according to the color feature calibration data and the color feature detection data

The method for generating color characteristic calibration data of the clothing region pixel points for the sample image according to the key points of the human body comprises the following steps:

acquiring color characteristic data of a plurality of first pixel points between at least two human body key points of the upper half of the human body in a sample image, and determining a first color median value of the human body jacket according to the color characteristic data of the first pixel points;

based on the first color median value, selecting a pixel point which meets a first setting condition between at least two human key points of the upper half of the human body as the first pixel point, and determining color feature data of the selected first pixel point as first color feature calibration data of a first region image in the sample image, wherein the first setting condition comprises: a first distance between the color feature data of the pixel point and the median value of the first color does not exceed a first preset threshold;

and/or the like, and/or,

acquiring color characteristic data of a plurality of second pixel points between at least two human body key points of the lower half of the human body in the sample image, and determining a second color median value of the human body lower garment according to the color characteristic data of the plurality of second pixel points;

based on the second color median value, selecting a pixel point which meets a second setting condition between at least two human key points of the lower half of the human body as the second pixel point, and determining color feature data of the selected second pixel point as second color feature calibration data of a second region image in the sample image, wherein the second setting condition comprises: and a second distance between the color characteristic data of the pixel point and the median value of the second color does not exceed a second preset threshold value.

2. The method of claim 1, wherein the obtaining color feature detection data of the clothing region image from the clothing region image through a clothing color extraction network comprises:

and acquiring first color feature detection data corresponding to the first area image and/or second color feature detection data corresponding to the second area image from the first area image and/or the second area image through the clothing color extraction network.

3. The method of claim 2, wherein training the garment color extraction network based on the color feature calibration data and the color feature detection data comprises:

training the clothing color extraction network according to the first color feature calibration data and the first color feature detection data; and/or

And training the clothing color extraction network according to the second color feature calibration data and the second color feature detection data.

4. The method according to any one of claims 1 to 3, wherein the extracting of the clothing region map from the sample image based on the human body key points comprises:

determining second coordinate data of a region of the first region image in the sample image based on first coordinate data of a human key point of the upper half of the human body, and extracting the first region image from the sample image according to the second coordinate data; and/or the presence of a gas in the gas,

and determining fourth coordinate data of the region of the second region image in the sample image based on the third coordinate data of the key points of the lower half of the human body, and extracting the second region image from the sample image according to the fourth coordinate data.

5. A method for detecting the color of human clothes is characterized by comprising the following steps:

extracting a clothing region image from a human body image to be detected based on the human body key points;

acquiring color feature prediction data of the clothing region image from the clothing region image through a clothing color extraction network,

the clothing color extraction network is obtained by training according to the method of any one of claims 1 to 4.

6. The method according to claim 5, wherein the extracting the clothing region image from the human body image to be detected based on the human body key point comprises:

and extracting a third area image associated with the human body upper garment and/or a fourth area image associated with the human body lower garment from the human body image to be detected based on the human body key points.

7. The method of claim 6, wherein the obtaining color feature prediction data of the clothing region image from the clothing region image through a clothing color extraction network comprises:

and acquiring color feature prediction data of the human body upper garment and/or color feature prediction data of the human body lower garment from the third area image and/or the fourth area image through the garment color extraction network.

8. The method according to claim 6 or 7, wherein the extracting, from the human body images to be detected, a third region image associated with a human body upper garment and/or a fourth region image associated with a human body lower garment based on the human body key points comprises:

determining sixth coordinate data of a region of the third region image in the human body image to be detected based on fifth coordinate data of human body key points of the upper half of the human body, and extracting the third region image from the human body image to be detected according to the sixth coordinate data; and/or the presence of a gas in the gas,

and determining eighth coordinate data of the region of the fourth region image in the human body image to be detected based on seventh coordinate data of the human body key point of the lower half of the human body, and extracting the fourth region image from the human body image to be detected according to the eighth coordinate data.

9. The method according to claim 5, wherein the human body image to be detected is a still image or a video image in a sequence of video frames.

10. An apparatus for training a neural network, the apparatus comprising:

the calibration module is used for generating color characteristic calibration data of a clothing region image for the sample image according to the key points of the human body, wherein the clothing region image comprises: the first region image comprises first color characteristic calibration data of human body coat pixel points, and/or the second region image comprises second color characteristic calibration data of human body coat pixel points;

the first extraction module is used for extracting a clothing region image from the sample image based on the key points of the human body, wherein the clothing region image contains color characteristic calibration data of the pixel points of the human body clothing;

the first acquisition module is used for acquiring color feature detection data of the clothing region image from the clothing region image through a clothing color extraction network;

a training module for training the clothing color extraction network according to the color feature calibration data and the color feature detection data,

wherein the calibration module comprises:

the second acquisition module is used for acquiring color feature data of a plurality of first pixel points between at least two human key points of the upper half of the human body in the sample image and determining a first color median value of the human body jacket according to the color feature data of the first pixel points; based on the first color median value, selecting a pixel point which meets a first setting condition between at least two human key points of the upper half of the human body as the first pixel point, and determining color feature data of the selected first pixel point as first color feature calibration data of a first region image in the sample image, wherein the first setting condition comprises: a first distance between the color feature data of the pixel point and the median value of the first color does not exceed a first preset threshold;

and/or the like, and/or,

the third acquisition module is used for acquiring color characteristic data of a plurality of second pixel points between at least two human key points of the lower half of the human body in the sample image and determining a second color median value of the human lower garment according to the color characteristic data of the plurality of second pixel points; based on the second color median value, selecting a pixel point which meets a second setting condition between at least two human key points of the lower half of the human body as the second pixel point, and determining color feature data of the selected second pixel point as second color feature calibration data of a second region image in the sample image, wherein the second setting condition comprises: and a second distance between the color characteristic data of the pixel point and the median value of the second color does not exceed a second preset threshold value.

11. The apparatus of claim 10, wherein the first obtaining module comprises:

the first obtaining sub-module is used for obtaining first color feature detection data corresponding to the first area image and/or second color feature detection data corresponding to the second area image from the first area image and/or the second area image through the clothing color extraction network.

12. The apparatus of claim 11, wherein the training module comprises:

the first training submodule is used for training the clothing color extraction network according to the first color feature calibration data and the first color feature detection data; and/or

And the second training submodule is used for training the clothing color extraction network according to the second color feature calibration data and the second color feature detection data.

13. The apparatus according to any one of claims 10 to 12, wherein the first extraction module comprises:

the first extraction submodule is used for determining second coordinate data of a region of the first region image in the sample image based on first coordinate data of a human key point of the upper half of the human body, and extracting the first region image from the sample image according to the second coordinate data; and/or the presence of a gas in the gas,

and the second extraction submodule is used for determining fourth coordinate data of the region of the second region image in the sample image based on the third coordinate data of the human key points of the lower half of the human body, and extracting the second region image from the sample image according to the fourth coordinate data.

14. A device for detecting the color of human clothing, said device comprising:

the second extraction module is used for extracting a clothing region image from a human body image to be detected based on the human body key points;

a fourth obtaining module, configured to obtain color feature prediction data of the clothing region image from the clothing region image through a clothing color extraction network,

wherein the clothing color extraction network is obtained by training the device according to any one of claims 10 to 13.

15. The apparatus of claim 14, wherein the second extraction module comprises:

and the third extraction submodule is used for extracting a third area image related to the human body upper garment and/or a fourth area image related to the human body lower garment from the human body image to be detected based on the human body key points.

16. The apparatus of claim 15, wherein the fourth obtaining module comprises:

and the second obtaining submodule is used for obtaining the color feature prediction data of the human body upper garment and/or the color feature prediction data of the human body lower garment from the third area image and/or the fourth area image through the garment color extraction network.

17. The apparatus according to claim 15 or 16, wherein the third extraction submodule is specifically configured to:

18. The apparatus according to claim 14, wherein the human body image to be detected is a still image or a video image in a video frame sequence.

19. A computer readable storage medium having stored thereon computer program instructions, wherein the program instructions, when executed by a processor, implement the steps of the method of training a neural network of any one of claims 1-4.

20. A computer readable storage medium having stored thereon computer program instructions, wherein the program instructions, when executed by a processor, implement the steps of the method for detecting human clothing color of any one of claims 5 to 9.

21. An electronic device, comprising: the device comprises a first processor, a first memory, a first communication element and a first communication bus, wherein the first processor, the first memory and the first communication element are communicated with each other through the first communication bus;

the first memory is used for storing at least one executable instruction, and the executable instruction causes the first processor to execute the steps of the training method of the neural network as claimed in any one of claims 1 to 4.

22. An electronic device, comprising: the second processor, the second memory, the second communication element and the second communication bus are communicated with each other through the second communication bus;

the second memory is used for storing at least one executable instruction, and the executable instruction causes the second processor to execute the steps of the human body clothes color detection method according to any one of claims 5-9.