WO2021043112A1 - 图像分类方法以及装置 - Google Patents

图像分类方法以及装置 Download PDF

Info

Publication number
WO2021043112A1
WO2021043112A1 PCT/CN2020/112694 CN2020112694W WO2021043112A1 WO 2021043112 A1 WO2021043112 A1 WO 2021043112A1 CN 2020112694 W CN2020112694 W CN 2020112694W WO 2021043112 A1 WO2021043112 A1 WO 2021043112A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
sample image
feature map
loss function
target area
Prior art date
Application number
PCT/CN2020/112694
Other languages
English (en)
French (fr)
Inventor
刘一衡
刘健庄
田奇
岳俊
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2021043112A1 publication Critical patent/WO2021043112A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person

Definitions

  • This application relates to the field of artificial intelligence, and more specifically, to image classification methods and devices in the field of computer vision.
  • Computer vision is an inseparable part of various intelligent/autonomous systems in various application fields, such as manufacturing, inspection, document analysis, medical diagnosis, and military. It is about how to use cameras/video cameras and computers to obtain What we need is the knowledge of the data and information of the subject. Vividly speaking, it is to install eyes (camera/camcorder) and brain (algorithm) on the computer to replace the human eye to identify, track and measure the target, so that the computer can perceive the environment. Because perception can be seen as extracting information from sensory signals, computer vision can also be seen as a science that studies how to make artificial systems "perceive" from images or multi-dimensional data.
  • computer vision is to use various imaging systems to replace the visual organs to obtain input information, and then the computer replaces the brain to complete the processing and interpretation of the input information.
  • the ultimate research goal of computer vision is to enable computers to observe and understand the world through vision like humans, and have the ability to adapt to the environment autonomously.
  • pedestrian re-identification technology can be a technology that uses computer vision technology to determine whether there is a specific pedestrian in an image or video sequence. For example, given a pedestrian image, the pedestrian re-recognition technology can be used to retrieve the image of the pedestrian under the cross-device and generate the pedestrian's behavior trajectory. Therefore, recognition technology can be widely used in fields such as smart video surveillance and smart security.
  • the recognition technology is mainly based on the global feature map of the input image for classification.
  • the global feature map includes the feature map of the target area to be recognized and the feature map of the background area. Because the feature map of the target area to be recognized is vulnerable to the features of the background area during classification. The influence of image noise reduces the accuracy of the classification result of the target area. Therefore, how to improve the accuracy of the classification results of the target area has become an urgent problem to be solved.
  • the present application provides an image classification method and device, which can realize the separation of the target area feature and the background area feature in the image to be processed, and improve the accuracy of the target area classification result by suppressing the influence of background area noise on the target area feature map.
  • an image classification method including: acquiring a to-be-processed image, the to-be-processed image including a target area to be recognized and a background area; and classifying the to-be-processed image according to the target neural network to obtain the to-be-processed image The classification result of the target area and the classification result of the background area;
  • the target neural network is trained based on the target area classification loss function, the background area classification loss function, and the cooperative loss function.
  • the target area classification loss function is used to represent the predicted classification results and the target corresponding to the target area feature map of the sample image.
  • the loss value between the classification results of the area label, the background area classification loss function is used to represent the loss value between the predicted classification result corresponding to the background area feature map of the sample image and the background area label classification result
  • the cooperative loss function is used for Separate the target area feature map of the sample image and the background area feature map of the sample image.
  • the above-mentioned target area to be recognized can be regarded as a foreground area containing the target object, and the target object may be an object with a motion attribute, or the above-mentioned target object may also be other objects.
  • the target area may refer to the area where any one of pedestrians, animals, vehicles, etc. is located, that is, the target area may be any of human images, animal images, and vehicle images.
  • the classification result of the target area may refer to the identification of a pedestrian, where the identification of the pedestrian may refer to the name of the pedestrian, the number of the pedestrian, or the ID number of the pedestrian;
  • the classification result of the background area may refer to the identification of the camera that obtains the image to be processed, for example, the number of the camera and the location information of the camera.
  • the above is an example of the pedestrian identification and the camera identification.
  • the pedestrian identification is used to distinguish different behaviors.
  • the background area image obtained by it can be approximated as unchanged, that is, the image of the background area obtained by the camera can be
  • the identification determines the classification result of the background area image, and the specific implementation form of the identification of the pedestrian and the identification of the camera is not limited in this application.
  • the target neural network is obtained by training the target region classification loss function, the background region loss function, and the cooperative loss function, so that the pre-trained target neural network can accurately separate the target region feature map and the background region in the image to be processed Characteristic map.
  • the target neural network classifies the image to be processed, by separating the feature map of the target area and the feature map of the background area in the image to be processed, it can not only obtain the classification result of the target area and the classification result of the background area, but also obtain the target area.
  • the feature map can also avoid the influence of the background area, and improve the accuracy of the classification result of the target area in the image to be processed.
  • the above-mentioned image to be processed may refer to an image captured by the electronic device through a camera, or the above-mentioned image to be processed may also be an image obtained from the inside of the electronic device (for example, the image stored in the photo album of the electronic device). , Or an image obtained by an electronic device from the cloud).
  • the above-mentioned image to be processed may be a picture in the acquired monitoring data.
  • the target area mask of the sample image can be complementary to the background area mask through the aforementioned cooperative loss function.
  • the target area mask is used to indicate the probability that the location of each pixel in the target area feature map belongs to the target area;
  • the background area mask is used to indicate that the location of each pixel in the background area feature map belongs to the background area The probability value.
  • the aforementioned collaborative loss function can make the overlap area of the target area image corresponding to the target area feature map in the sample image and the background area image corresponding to the background area feature map in the sample image nearly zero. That is, the cooperative loss function can make the target area feature map and the background area feature map in the sample image extracted by the target neural network obvious demarcation, so as to avoid the influence of the noise of the background area on the target area feature map.
  • the synergy loss function is obtained by weighting the first synergy loss function and the second synergy loss function, wherein the first synergy loss function is based on the sample
  • the target area feature map of the image is obtained by multiplying the background area mask of the sample image; the second collaborative loss function is obtained based on the background area feature map of the sample image multiplied by the target area mask of the sample image.
  • the target area mask is obtained by training of the extracted target area feature map;
  • the background area mask is obtained by training of the extracted background area feature map.
  • the above-mentioned target region mask is equivalent to the target region attention map; the above-mentioned background region mask is equivalent to the background region attention map.
  • the scales of any two of the target area feature map, the background area feature map, the target area mask, and the background area mask of the image are the same.
  • the first loss function obtained by multiplying the target area feature map with the corresponding pixels in the background area mask can be used to represent the overlap between the extracted target area feature and the background area, that is, the target area feature map is affected by The size of the range affected by noise in the background area;
  • the second loss function obtained by multiplying the background area feature map with the corresponding pixels in the target area mask can be used to represent the overlap between the extracted background area feature map and the target area, that is, the background The area feature map is affected by the current area noise.
  • the first cooperative loss function and the second loss function are weighted to obtain the cooperative loss function, so that when training the target neural network, the target area feature is separated from the background area feature to achieve the target area
  • the accuracy rate of the feature map and the accuracy rate of the background area feature map promote each other to improve synergistically.
  • the target neural network includes a first branch and a second branch, wherein the first branch is used to process the target area of the sample image to obtain the sample The target area feature map of the image and the target area mask of the sample image;
  • the second branch is used to process the background area of the sample image to obtain the background area feature map of the sample image and the background area mask of the sample image.
  • the target neural network can include two branches.
  • the first branch can be used to process the target area of the sample image
  • the second branch can be used to process the background area of the sample image.
  • the first branch can be used to cooperate with the first branch through the collaborative loss function.
  • the second branch performs training, so that the accuracy of the extracted feature map of the target area and the accuracy of the feature map of the background area promote each other to improve synergistically.
  • the cooperative loss function is obtained by the following equation:
  • L t represents the cooperative loss function
  • N represents the number of pixels in the sample image
  • F i represents the i-th pixel in the target area of the sample image
  • B i represents the i-th pixel in the background area of the sample image
  • N is an integer greater than 1
  • i is An integer greater than or equal to 1 and less than or equal to N.
  • the overall loss function includes the above classification loss function, the above background region classification loss function, and the above cooperative loss function, and the overall loss function is obtained by the following equation:
  • L f can represent the target area classification loss function
  • L b can represent the background area classification loss function
  • L t can represent the cooperative loss function
  • classifying the image to be processed according to the target neural network to obtain the classification result of the target area of the image to be processed and the classification result of the background area includes: passing the target neural network The first branch of the network processes the target area of the image to be processed to obtain an enhanced feature map of the target area of the image to be processed, wherein the enhanced feature map of the target area is obtained through the target area feature map of the image to be processed and the The target area mask of the image to be processed is multiplied; the background area of the image to be processed is processed through the second branch of the target neural network to obtain an enhanced feature map of the background area of the image to be processed, wherein the background The area enhancement feature map is obtained by multiplying the background area feature map of the image to be processed and the background area mask of the image to be processed; classification is performed based on the target area enhancement feature map and the background area enhancement feature map of the image to be processed, The classification result of the target area and the classification result of the background area of the image to be processed are
  • the classification result of the background area of the image to be processed refers to the identification of the camera that has acquired the image to be processed.
  • a method for training an image classification model including: obtaining training data, where the training data includes a sample image, a target region labeling classification result of the sample image, and a background region labeling classification result of the sample image; According to the target area feature map of the sample image extracted by the neural network, the target area prediction classification result of the sample image is obtained; according to the background area feature map of the sample image extracted by the neural network, the background area prediction classification result of the sample image is obtained ; Determine the parameters of the neural network based on the target area classification loss function, the background area classification loss function, and the cooperative loss function, where the target area classification loss function is used to represent the target area prediction classification result and target area label classification of the sample image The loss value between the results, the background area classification loss function is used to represent the loss value between the background area prediction classification result of the sample image and the background area label classification result, the cooperative loss function is used to separate the target area of the sample image The feature map and the background area feature map of the sample image.
  • the above-mentioned target area label classification result may refer to the label classification result of the area where the target object is located, and the target object may be an object with motion attributes.
  • the target object may be any of pedestrians, animals, and vehicles.
  • the above-mentioned target object may also be other objects.
  • the predicted classification result of the target area of the sample image refers to the identification of the camera that obtains the sample image.
  • the classification result of the target area may refer to the identification of a pedestrian, where the identification of the pedestrian may refer to the name of the pedestrian, the number of the pedestrian, or the ID number of the pedestrian;
  • the classification result of the background area may refer to the identification of the camera that obtains the image to be processed, for example, the number of the camera and the location information of the camera.
  • the above is an example of the pedestrian identification and the camera identification.
  • the pedestrian identification is used to distinguish different behaviors.
  • the background area image obtained by it can be approximated as unchanged, that is, the image of the background area obtained by the camera can be
  • the identification determines the classification result of the background area image, and the specific implementation form of the identification of the pedestrian and the identification of the camera is not limited in this application.
  • the collaborative loss function can improve the segmentation accuracy of the feature map of the target region and the feature map of the background region extracted by the target neural network, that is, it can avoid being affected by the background region when acquiring the feature map of the target region. That is, through the overall training of the target area classification loss function, the background area loss function, and the cooperative loss function, the parameters of the network are continuously adjusted, so that the target neural network can be used to classify the image to be processed, so that the accuracy of the extracted target area feature map is as good as The accuracy of the background area feature map promotes each other and improves synergistically.
  • the target area mask of the sample image can be complementary to the background area mask through the aforementioned cooperative loss function.
  • the target area mask is used to indicate the probability that the location of each pixel in the target area feature map belongs to the target area;
  • the background area mask is used to indicate that the location of each pixel in the background area feature map belongs to the background area The probability value.
  • the aforementioned collaborative loss function can make the overlap area of the target area image corresponding to the target area feature map in the sample image and the background area image corresponding to the background area feature map in the sample image nearly zero. That is, the cooperative loss function can make the target area feature map and the background area feature map in the sample image extracted by the target neural network obvious demarcation, so as to avoid the influence of the noise of the background area on the target area feature map.
  • the synergy loss function is obtained by weighting the first synergy loss function and the second synergy loss function, wherein the first synergy loss function is based on the sample
  • the target area feature map of the image is obtained by multiplying the background area mask of the sample image; the second collaborative loss function is obtained based on the background area feature map of the sample image multiplied by the target area mask of the sample image.
  • the target area mask is obtained by training of the extracted target area feature map;
  • the background area mask is obtained by training of the extracted background area feature map.
  • the above-mentioned target region mask is equivalent to the target region attention map; the above-mentioned background region mask is equivalent to the background region attention map.
  • the scales of any two of the target area feature map, the background area feature map, the target area mask, and the background area mask of the image are the same.
  • the first loss function obtained by multiplying the target area feature map with the corresponding pixels in the background area mask can be used to represent the overlap between the extracted target area feature and the background area, that is, the target area feature map is affected by The size of the range affected by noise in the background area;
  • the second loss function obtained by multiplying the background area feature map with the corresponding pixels in the target area mask can be used to represent the overlap between the extracted background area feature map and the target area, that is, the background The area feature map is affected by the current area noise.
  • the first cooperative loss function and the second loss function are weighted to obtain the cooperative loss function, so that when training the target neural network, the target area feature is separated from the background area feature to achieve the target area
  • the accuracy rate of the feature map and the accuracy rate of the background area feature map promote each other to improve synergistically.
  • the neural network includes a first branch and a second branch, and the first branch is used to process the target area of the sample image to obtain the target of the sample image The area feature map and the target area mask of the sample image; the second branch is used to process the background area of the sample image to obtain the background area feature map of the sample image and the background area mask of the sample image.
  • the target neural network can include two branches.
  • the first branch can be used to process the target area of the sample image
  • the second branch can be used to process the background area of the sample image.
  • the first branch can be used to cooperate with the first branch through the collaborative loss function.
  • the second branch performs training, so that the accuracy of the extracted feature map of the target area and the accuracy of the feature map of the background area promote each other to improve synergistically.
  • obtaining the target region prediction classification result of the sample image according to the target region feature map of the sample image extracted by the neural network includes: using the first branch of the target neural network to compare the sample image The target area of the sample image is processed to obtain a target area enhanced feature map of the sample image, where the target area enhanced feature map is obtained by multiplying the target area feature map of the sample image and the target area mask of the sample image; The feature map of the target area of the sample image is enhanced, and the predicted classification result of the target area of the sample image is obtained.
  • obtaining the background area prediction classification result of the sample image according to the background area feature map of the sample image extracted by the neural network includes: using the second branch of the target neural network to compare the sample image Processing the background area of the sample image to obtain a background area enhancement feature map of the sample image, where the background area enhancement feature map is obtained by multiplying the background area feature map of the sample image and the background area mask of the sample image;
  • the feature map is enhanced based on the background area of the sample image, and the predicted classification result of the background area of the sample image is obtained.
  • the cooperative loss function is obtained by the following equation:
  • L t represents the cooperative loss function
  • N represents the number of pixels in the sample image
  • F i represents the i-th pixel in the target area of the sample image
  • B i represents the i-th pixel in the background area of the sample image
  • N is an integer greater than 1
  • i is an integer greater than or equal to 1 and less than or equal to N.
  • the overall loss function includes the above classification loss function, the above background region classification loss function, and the above cooperative loss function, and the overall loss function is obtained by the following equation:
  • L f can represent the target area classification loss function
  • L b can represent the background area classification loss function
  • L t can represent the cooperative loss function
  • an image classification method including: acquiring an image to be processed, the image to be processed includes a target area to be recognized and a background area; feature extraction is performed on the image to be processed through a target neural network to obtain features of the target area Map and the background area feature map; classify the target area feature map and the background area feature map through the target neural network to obtain the classification result of the target area and the classification result of the background area;
  • the target neural network is trained based on the target area classification loss function, the background area classification loss function, and the cooperative loss function.
  • the target area classification loss function is used to represent the predicted classification results and the target corresponding to the target area feature map of the sample image.
  • the loss value between the region labeling classification results, the background region classification loss function is used to represent the loss value between the predicted classification result corresponding to the background region feature map of the sample image and the background region label classification result, the cooperative loss function is used for separation The target area feature map of the sample image and the background area feature map of the sample image.
  • the target neural network is obtained by training the target region classification loss function, the background region loss function, and the cooperative loss function, so that the pre-trained target neural network can accurately separate the target region feature map and the background region in the image to be processed Characteristic map. Therefore, when the target neural network classifies the image to be processed, by separating the feature map of the target area and the feature map of the background area in the image to be processed, it can not only obtain the classification result of the target area and the classification result of the background area, but also obtain the target area.
  • the feature map can also avoid the influence of the background area, and improve the accuracy of the classification result of the target area in the image to be processed.
  • the target neural network includes a first branch and a second branch, and feature extraction is performed on the image to be processed through the target neural network to obtain the target area feature map and the Features map of the background area, including:
  • Feature extraction is performed on the target region of the image to be processed through the first branch of the target neural network to obtain an enhanced feature map of the target region of the image to be processed, wherein the enhanced feature map of the target region passes through the target of the image to be processed
  • the area feature map is obtained by multiplying the target area mask of the image to be processed
  • the target area feature map and the background area feature map are classified by the target neural network to obtain the classification result of the target area and the classification result of the background area ,include:
  • the classification is performed based on the enhanced feature map of the target area of the image to be processed and the enhanced feature map of the background area to obtain the classification result of the target area of the image to be processed and the classification result of the background area.
  • the first branch and the second branch in the above-mentioned target neural network can classify images to be processed at the same time, or, in the target neural network, the images to be processed can also be processed in the order of the first branch and then the second branch. For classification, or, in the target neural network, the image to be processed may be classified in the order of the second branch and then the first branch.
  • the classification result of the background area of the image to be processed refers to the identification of the camera that has obtained the image to be processed.
  • the synergy loss function is obtained by weighting the first synergy loss function and the second synergy loss function, wherein the first synergy loss function is based on the sample
  • the target area feature map of the image is obtained by multiplying the background area mask of the sample image; the second collaborative loss function is obtained based on the background area feature map of the sample image multiplied by the target area mask of the sample image.
  • the target neural network includes a first branch and a second branch, wherein the first branch is used to process the target area of the sample image to obtain the sample The target area feature map of the image and the target area mask of the sample image; the second branch is used to process the background area of the sample image to obtain the background area feature map of the sample image and the background area mask of the sample image .
  • the cooperative loss function is obtained by the following equation:
  • L t represents the cooperative loss function
  • N represents the number of pixels in the sample image
  • F i represents the i-th pixel in the target area of the sample image
  • B i represents the i-th pixel in the background area of the sample image
  • N is an integer greater than 1
  • i is an integer greater than or equal to 1 and less than or equal to N.
  • the overall loss function includes the above classification loss function, the above background region classification loss function, and the above cooperative loss function, and the overall loss function is obtained by the following equation:
  • L f can represent the target area classification loss function
  • L b can represent the background area classification loss function
  • L t can represent the cooperative loss function
  • the classification result of the background area of the image to be processed refers to the identification of the camera that has obtained the image to be processed.
  • an image classification device which includes: an acquisition unit configured to acquire a to-be-processed image, the to-be-processed image includes a target area to be recognized and a background area; The processed image is classified, and the classification result of the target area and the classification result of the background area of the image to be processed are obtained;
  • the target neural network is trained based on the target area classification loss function, the background area classification loss function, and the cooperative loss function.
  • the target area classification loss function is used to represent the predicted classification results and the target corresponding to the target area feature map of the sample image.
  • the loss value between the classification results of the area label, the background area classification loss function is used to represent the loss value between the predicted classification result corresponding to the background area feature map of the sample image and the background area label classification result
  • the cooperative loss function is used for Separate the target area feature map of the sample image and the background area feature map of the sample image.
  • the above-mentioned image classification device includes a functional unit/module that is further configured to execute the first aspect and the method in any one of the first aspect.
  • a training device for an image classification model including: an acquisition unit for acquiring training data, wherein the training data includes a sample image, a target area labeled classification result of the sample image, and a background of the sample image Region annotation classification result; a processing unit for obtaining the predicted classification result of the target region of the sample image according to the target region feature map of the sample image extracted by the neural network; the background region feature map of the sample image extracted according to the neural network, Obtain the background area prediction classification result of the sample image; determine the parameters of the neural network based on the target area classification loss function, the background area classification loss function, and the cooperative loss function;
  • the target area classification loss function is used to represent the loss value between the target area prediction classification result of the sample image and the target area label classification result
  • the background area classification loss function is used to represent the background of the sample image
  • the loss value between the area prediction classification result and the background area annotation classification result, and the cooperative loss function is used to separate the target area feature map of the sample image and the background area feature map of the sample image.
  • the above-mentioned training device includes a functional unit/module that is also used to execute the second aspect and the training method in any one of the second aspect.
  • an image classification device including: an acquisition unit for acquiring an image to be processed, the image to be processed including a target area to be recognized and a background area; Performing feature extraction on the image to be processed to obtain a feature map of the target region and a feature map of the background region; classifying the feature map of the target region and the feature map of the background region through the target neural network, Obtaining a classification result of the target area and a classification result of the background area;
  • the target neural network is trained based on the target area classification loss function, the background area classification loss function, and the collaborative loss function.
  • the target area classification loss function is used to represent the predicted classification result corresponding to the target area feature map of the sample image
  • the loss value between the label classification result of the target region and the background region classification loss function is used to represent the loss value between the predicted classification result corresponding to the background region feature map of the sample image and the background region label classification result
  • the The collaborative loss function is used to separate the target area feature map of the sample image and the background area feature map of the sample image.
  • the above-mentioned training device includes functional units/modules that are also used to execute the third aspect and the method in any one of the third aspects.
  • an image classification device includes: a memory for storing a program; a processor for executing the program stored in the memory, and when the program stored in the memory is executed, the processor is configured to execute : Obtain a to-be-processed image, the to-be-processed image includes a target area to be recognized and a background area; classify the to-be-processed image according to the target neural network to obtain the classification result of the target area of the to-be-processed image and the The classification result of the background area;
  • the target neural network is trained based on the target area classification loss function, the background area classification loss function, and the collaborative loss function.
  • the target area classification loss function is used to represent the predicted classification result corresponding to the target area feature map of the sample image
  • the loss value between the label classification result of the target region and the background region classification loss function is used to represent the loss value between the predicted classification result corresponding to the background region feature map of the sample image and the background region label classification result
  • the The collaborative loss function is used to separate the target area feature map of the sample image and the background area feature map of the sample image.
  • the processor included in the foregoing image classification apparatus is further configured to execute the method in any one of the implementation manners in the first aspect.
  • an image classification model training device includes: a memory for storing a program; a processor for executing the program stored in the memory.
  • the processing The device is used to execute: obtain training data, where the training data includes a sample image, a target area labeling classification result of the sample image, and a background area labeling classification result of the sample image;
  • the target area feature map is used to obtain the target area prediction classification result of the sample image;
  • the background area feature map of the sample image extracted by the neural network is used to obtain the background area prediction classification result of the sample image; based on the target area classification A loss function, a background area classification loss function, and a cooperative loss function to determine the parameters of the neural network;
  • the target area classification loss function is used to represent the loss value between the target area prediction classification result of the sample image and the target area label classification result
  • the background area classification loss function is used to represent the background of the sample image
  • the loss value between the area prediction classification result and the background area annotation classification result, and the cooperative loss function is used to separate the target area feature map of the sample image and the background area feature map of the sample image.
  • the processor included in the foregoing training device is further configured to execute the training method in any one of the implementation manners of the second aspect.
  • an image classification device includes: a memory for storing a program; a processor for executing the program stored in the memory, and when the program stored in the memory is executed, the processor is configured to execute : Obtain a to-be-processed image, which includes the target area to be recognized and a background area; perform feature extraction on the to-be-processed image through the target neural network to obtain a feature map of the target area and a feature map of the background area; pass the The target neural network classifies the feature map of the target area and the feature map of the background area, and obtains the classification result of the target area and the classification result of the background area,
  • the target neural network is trained based on the target area classification loss function, the background area classification loss function, and the cooperative loss function.
  • the target area classification loss function is used to represent the predicted classification results and the target corresponding to the target area feature map of the sample image.
  • the loss value between the classification results of the area label, the background area classification loss function is used to represent the loss value between the predicted classification result corresponding to the background area feature map of the sample image and the background area label classification result
  • the cooperative loss function is used for Separate the target area feature map of the sample image and the background area feature map of the sample image.
  • the processor included in the foregoing image classification apparatus is further configured to execute the method in any one of the implementation manners of the third aspect.
  • a computer-readable medium stores program code for device execution.
  • the program code includes the program code for executing the first aspect to the third aspect and the first aspect to the third aspect.
  • the method in any one of the implementation modes.
  • a computer program product containing instructions is provided.
  • the computer program product runs on a computer, the computer executes any one of the first aspect to the third aspect and the first aspect to the third aspect.
  • a method in a way of implementation.
  • a chip in a twelfth aspect, includes a processor and a data interface.
  • the processor reads instructions stored in a memory through the data interface, and executes the first to third aspects and the first aspect. Aspect to the method in any one of the implementation manners of the third aspect.
  • the chip may further include a memory in which instructions are stored, and the processor is configured to execute instructions stored on the memory.
  • the processor is configured to execute the method in any one of the foregoing first aspect to the third aspect and the first aspect to the third aspect.
  • FIG. 1 is a schematic diagram of an artificial intelligence main body framework provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • Fig. 3 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a system architecture provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a convolutional neural network structure provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a chip hardware structure provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a system architecture provided by an embodiment of the present application.
  • FIG. 9 is a schematic flowchart of an image classification method provided by an embodiment of the present application.
  • FIG. 10 is a schematic flowchart of a method for training an image classification model provided by an embodiment of the present application.
  • FIG. 11 is a schematic flowchart of a method for training an image classification model provided by an embodiment of the present application.
  • FIG. 12 is a schematic diagram of a training method of an image classification model provided by an embodiment of the present application.
  • FIG. 13 is a schematic block diagram of an image classification device provided by an embodiment of the present application.
  • FIG. 14 is a schematic block diagram of an image classification model training device provided by an embodiment of the present application.
  • FIG. 15 is a schematic block diagram of an image classification device provided by an embodiment of the present application.
  • FIG. 16 is a schematic block diagram of an image classification model training device provided by an embodiment of the present application.
  • Figure 1 shows a schematic diagram of an artificial intelligence main framework, which describes the overall workflow of the artificial intelligence system and is suitable for general artificial intelligence field requirements.
  • Intelligent Information Chain reflects a series of processes from data acquisition to processing. For example, it can be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, the data has gone through the condensing process of "data-information-knowledge-wisdom".
  • the "IT value chain” is the industrial ecological process from the underlying infrastructure and information (providing and processing technology realization) of human intelligence to the system, reflecting the value that artificial intelligence brings to the information technology industry.
  • the infrastructure provides computing power support for the artificial intelligence system, realizes communication with the outside world, and realizes support through the basic platform.
  • the infrastructure can communicate with the outside through sensors, and the computing power of the infrastructure can be provided by smart chips.
  • the smart chip here can be a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), and an application specific integrated circuit (application specific).
  • Hardware acceleration chips such as integrated circuit (ASIC) and field programmable gate array (FPGA).
  • the basic platform of infrastructure can include distributed computing framework and network and other related platform guarantees and support, and can include cloud storage and computing, interconnection networks, etc.
  • data can be obtained through sensors and external communication, and then these data can be provided to the smart chip in the distributed computing system provided by the basic platform for calculation.
  • the data in the upper layer of the infrastructure is used to represent the data source in the field of artificial intelligence.
  • the data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and sensory data such as force, displacement, liquid level, temperature, and humidity.
  • the above-mentioned data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other processing methods.
  • machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, training, etc.
  • Reasoning refers to the process of simulating human intelligent reasoning in a computer or intelligent system, using formal information to conduct machine thinking and solving problems based on reasoning control strategies.
  • the typical function is search and matching.
  • Decision-making refers to the process of making decisions based on intelligent information after reasoning, and usually provides functions such as classification, sorting, and prediction.
  • some general capabilities can be formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image Recognition and so on.
  • Intelligent products and industry applications refer to the products and applications of artificial intelligence systems in various fields. It is an encapsulation of the overall solution of artificial intelligence, productizing intelligent information decision-making and realizing landing applications. Its application fields mainly include: intelligent manufacturing, intelligent transportation, Smart home, smart medical, smart security, autonomous driving, safe city, smart terminal, etc.
  • the image classification method of the embodiment of the present application can be applied to the security field.
  • pictures (or videos) collected by monitoring equipment in public places can be classified and processed through the target neural network to identify pedestrian information in the pictures, and provide important clues for case detection.
  • Pedestrian re-recognition technology uses the photos of specific people to match the video segments of specific people in the surveillance video library. In this way, the trajectory of a specific person on each camera can be connected in series. Once this trajectory is connected, it will be very helpful to the police in solving criminal cases.
  • a given monitored pedestrian image can be realized, the image of the pedestrian under the cross-device can be retrieved and the behavior trajectory of the pedestrian can be formed, as shown in Figure 2, which can be passed through the target nerve in this application.
  • the network retrieves the behavior trajectory of the specific person in background 1 to background 4, where background 1 to background 4 may be images obtained through different cameras.
  • an image of a specific person for example, a picture or photo of a suspect
  • the image of the specific person includes a specific portrait area and a street view area
  • the neural network classifies the images of specific people, and obtains the classification results of the specific portraits and the classification results of the street scene area.
  • the pre-trained target neural network can extract the portrait feature map and street view feature map from the image of a specific person, and classify based on the portrait feature map and the street view feature map. For example, classify the portrait feature map to obtain the identification of the specific person, and the street view The feature map is classified to obtain the identification of the camera that has acquired the street view. The location of the street view can be determined by the identification of the camera, and then the behavior trajectory of a specific person can be determined.
  • the training data used to train the target neural network may include sample images of different personnel in the public security system, the identification of the personnel, and the identification of each camera that obtains the sample image.
  • the images of the background area of the captured screen can be regarded as approximately the same, that is, the images of different background areas can be distinguished by the camera ID.
  • the target neural network applied to the field of intelligent security provided by the embodiments of this application is also applicable to the expansion, limitation, explanation and description of the related content of the target neural network in the related embodiments in FIG. 5 to FIG. 12 below. I won't repeat it here.
  • the target neural network in this embodiment of the application can realize real-time monitoring in all areas.
  • the camera looks for an image that matches the child's photo, so as to realize pedestrian recognition technology in public places with relatively high crowd density.
  • an image of a specific person for example, a picture or photo of a lost person
  • the image of the specific person includes a specific portrait area and a street view area
  • the target neural network classifies the image of a specific person, and obtains the classification result of the specific portrait and the classification result of the street scene area.
  • the pre-trained target neural network can extract the portrait feature map and street view feature map in the image of a specific person, and classify based on the portrait feature map and street view feature map.
  • the identification of the portrait can be obtained through the classification of the portrait feature map, and the street view feature
  • the image is classified to obtain the identification of the camera that has acquired the street view, and the location of the street view can be determined through the identification of the camera, and the behavior trajectory of the lost person can be determined.
  • the training data used to train the target neural network may include sample images of the missing person, the identification of the missing person, and the identification of each camera that obtained the sample image. .
  • the images of the background area of the shooting frame acquired by it can be regarded as approximately the same, that is, the images of different background areas can be distinguished by the camera ID.
  • the target neural network using the intelligent tracing system provided in the embodiments of this application is also applicable to the expansion, limitation, explanation and description of the related content of the target neural network in the following related embodiments in FIG. 5 to FIG. 12. I won't repeat it here.
  • the intelligent tracing system can refer to a crowded public area, such as an airport, station, amusement park and other public areas to find specific people.
  • the target neural network in this embodiment of the application can be called The surveillance video library realizes the quick search and trajectory construction of lost children.
  • the intelligent tracing system can also be applied to intelligent home robots.
  • the pedestrian re-identification technology is realized through the target neural network, so that the intelligent home robot can better identify family members. .
  • the intelligent tracing system can also be applied in an unmanned supermarket.
  • the target neural network can dynamically track the user's trajectory in real time based on the photos of the appearance of pedestrians, and convert the trajectory into information that the administrator can understand, so as to help operators obtain effective customer trajectories and further dig into business value.
  • the user's interest can be understood through the behavior trajectory in order to optimize the user experience.
  • the target neural network of the embodiment of the application can be used to classify pictures, so that pictures of different categories (for example, classification based on pictures of different users) can be labeled, which is convenient for users View and find.
  • the classification tags of these pictures can also be provided to the album management system for classification management, saving users management time, improving the efficiency of album management, and enhancing user experience.
  • the target neural network of the embodiment of the present application can classify and label images of different users.
  • the pictures in the smart terminal can be classified by persons, that is, pictures that include user A can be marked as tags of user A; for pictures that include user B The picture can be tagged with user B's label, so that the album can be classified based on different users.
  • an image to be processed for example, a picture or photo of user A
  • the image to be processed includes a portrait area and a street view area; according to a pre-trained target neural network Classify the portrait to obtain the classification result of the portrait and the classification result of the street view area.
  • the pre-trained target neural network can extract the portrait feature map and street view feature map of user A in the image to be processed, and classify based on the portrait feature map and street view feature map.
  • the identification of the portrait can be obtained through the classification of the portrait feature map.
  • the street view feature map is classified to obtain the identification of the camera that has acquired the street view, and further images with the same identification of the portrait can be stored in the same album folder, thereby realizing album classification of different user pictures.
  • the training data used to train the target neural network may be sample images of different users, identifications of different users, and identifications of cameras that have acquired the sample images.
  • the images of the background area of the shooting frame acquired by it can be regarded as approximately the same, that is, the images of different background areas can be distinguished by the camera ID.
  • target neural network using album classification provided in the embodiments of the present application is also applicable to the expansion, limitation, explanation and description of the related content of the target neural network in the related embodiments in FIG. 5 to FIG. 12 below. Go into details again.
  • a neural network can be composed of neural units.
  • a neural unit can refer to an arithmetic unit that takes x s and intercept 1 as inputs.
  • the output of the arithmetic unit can be:
  • s 1, 2,...n, n is a natural number greater than 1
  • W s is the weight of x s
  • b is the bias of the neural unit.
  • f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal.
  • the output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function.
  • a neural network is a network formed by connecting multiple above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit.
  • the input of each neural unit can be connected with the local receptive field of the previous layer to extract the characteristics of the local receptive field.
  • the local receptive field can be a region composed of several neural units.
  • Deep neural network also known as multi-layer neural network
  • the DNN is divided according to the positions of different layers.
  • the neural network inside the DNN can be divided into three categories: input layer, hidden layer, and output layer.
  • the first layer is the input layer
  • the last layer is the output layer
  • the number of layers in the middle are all hidden layers.
  • the layers are fully connected, that is to say, any neuron in the i-th layer must be connected to any neuron in the i+1th layer.
  • DNN looks complicated, it is not complicated as far as the work of each layer is concerned. Simply put, it is the following linear relationship expression: among them, Is the input vector, Is the output vector, Is the offset vector, W is the weight matrix (also called coefficient), and ⁇ () is the activation function.
  • Each layer is just the input vector After such a simple operation, the output vector is obtained Due to the large number of DNN layers, the coefficient W and the offset vector The number is also relatively large.
  • DNN The definition of these parameters in DNN is as follows: Take coefficient W as an example: Suppose in a three-layer DNN, the linear coefficients from the fourth neuron in the second layer to the second neuron in the third layer are defined as The superscript 3 represents the number of layers where the coefficient W is located, and the subscript corresponds to the output third-level index 2 and the input second-level index 4.
  • the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as
  • Convolutional neural network (convolutional neuron network, CNN) is a deep neural network with a convolutional structure.
  • the convolutional neural network contains a feature extractor composed of a convolutional layer and a sub-sampling layer.
  • the feature extractor can be regarded as a filter.
  • the convolutional layer refers to the neuron layer that performs convolution processing on the input signal in the convolutional neural network.
  • a neuron can only be connected to a part of the neighboring neurons.
  • a convolutional layer usually contains several feature planes, and each feature plane can be composed of some rectangularly arranged neural units. Neural units in the same feature plane share weights, and the shared weights here are the convolution kernels.
  • Sharing weight can be understood as the way of extracting image information has nothing to do with location.
  • the convolution kernel can be initialized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning during the training process of the convolutional neural network.
  • the direct benefit of sharing weights is to reduce the connections between the layers of the convolutional neural network, and at the same time reduce the risk of overfitting.
  • Important equation taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, then the training of the deep neural network becomes a process of reducing this loss as much as possible.
  • the neural network can use an error back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during the training process, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, forwarding the input signal until the output will cause error loss, and the parameters in the initial neural network model are updated by backpropagating the error loss information, so that the error loss is converged.
  • the backpropagation algorithm is a backpropagation motion dominated by error loss, and aims to obtain the optimal parameters of the neural network model, such as the weight matrix.
  • Fig. 5 shows a system architecture 200 provided by an embodiment of the present application.
  • a data collection device 260 is used to collect training data.
  • the neural network used for image classification can be further trained through training images, that is, the training data collected by the data collection device 260 can be training images.
  • the training data of the training target neural network in the embodiment of the present application may include a sample image, a pedestrian identification included in the sample image, and a background identification included in the sample image.
  • the pedestrian identification may be the ID of the pedestrian or the identity of the pedestrian.
  • the background identification can be the identification information of the camera corresponding to the image to be processed, for example, it can be the ID information of the camera that obtains the image to be processed.
  • the information of the background area of the captured image obtained by it may be unchanged, that is, the information of different background areas can be distinguished by the camera ID.
  • the data collection device 260 stores the training data in the database 230, and the training device 220 trains to obtain the target model/rule 201 based on the training data maintained in the database 230.
  • the training device 220 processes the input original image and compares the output image with the original image until the output image of the training device 220 differs from the original image. The difference is less than a certain threshold, thereby completing the training of the target model/rule 201.
  • the target neural network for image classification provided in this application can be obtained by training the classification loss function between the predicted classification result of the sample image and the sample classification result.
  • the trained neural network makes the sample image input to The difference between the predicted classification result obtained by the target neural network and the sample classification result is less than a certain threshold, so as to complete the training of the neural network to obtain the parameters of the target neural network.
  • the above-mentioned target model/rule 201 can be used to implement the image classification method of the embodiment of the present application.
  • the target model/rule 201 in the embodiment of the present application may specifically be a neural network.
  • the training data maintained in the database 230 may not all come from the collection of the data collection device 260, and may also be received from other devices.
  • the training device 220 does not necessarily perform the training of the target model/rule 201 completely based on the training data maintained by the database 230. It may also obtain training data from the cloud or other places for model training. The above description should not be used as a reference to this application. Limitations of the embodiment.
  • the target model/rule 201 trained according to the training device 220 can be applied to different systems or devices, such as the execution device 210 shown in FIG. 5, which can be a terminal, such as a mobile phone terminal, a tablet computer, notebook computers, augmented reality (AR)/virtual reality (VR), vehicle-mounted terminals, etc., can also be servers, or cloud, etc.
  • the execution device 210 is configured with an input/output (input/output, I/O) interface 212 for data interaction with external devices.
  • the user can input data to the I/O interface 212 through the client device 240.
  • the input data in this embodiment of the present application may include: a to-be-processed image input by the client device.
  • the preprocessing module 213 and the preprocessing module 214 are used to perform preprocessing according to the input data (such as the image to be processed) received by the I/O interface 212.
  • the preprocessing module 213 and the preprocessing module may not be provided. 214 (there may only be one of the preprocessing modules), and the calculation module 211 is directly used to process the input data.
  • the execution device 210 When the execution device 210 preprocesses input data, or when the calculation module 211 of the execution device 210 performs calculations and other related processing, the execution device 210 can call data, codes, etc. in the data storage system 250 for corresponding processing. , The data, instructions, etc. obtained by corresponding processing may also be stored in the data storage system 250.
  • the I/O interface 212 returns the processing result, as described above, the classification result of the target area to be recognized in the image to be processed and the classification result of the background area to the client device 240, so as to provide the user.
  • the training device 220 can generate corresponding target models/rules 201 based on different training data for different goals or tasks, and the corresponding target models/rules 201 can be used to achieve the above goals or complete The above tasks provide users with the desired results.
  • the user can manually set input data, and the manual setting can be operated through the interface provided by the I/O interface 212.
  • the client device 240 can automatically send input data to the I/O interface 212. If the client device 240 is required to automatically send the input data and the user's authorization is required, the user can set the corresponding authority in the client device 240. The user can view the result output by the execution device 210 on the client device 240, and the specific presentation form may be a specific manner such as display, sound, and action.
  • the client device 240 can also be used as a data collection terminal to collect the input data of the input I/O interface 212 and the output result of the output I/O interface 212 as new sample data, and store it in the database 230 as shown in the figure. Of course, it is also possible not to collect through the client device 240. Instead, the I/O interface 212 directly uses the input data input to the I/O interface 212 and the output result of the output I/O interface 212 as a new sample as shown in the figure. The data is stored in the database 230.
  • FIG. 5 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, devices, modules, etc. shown in the figure does not constitute any limitation.
  • the data The storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 may also be placed in the execution device 210.
  • the target model/rule 201 is obtained by training according to the training device 220.
  • the target model/rule 201 may be the target neural network used for image classification in the embodiment of this application.
  • the target neural network provided in the example may be a convolutional neural network (CNN), or may be a deep convolutional neural network (DCNN), etc.
  • a convolutional neural network is a deep neural network with a convolutional structure. It is a deep learning architecture.
  • the deep learning architecture refers to the algorithm of machine learning. Multi-level learning is carried out on the abstract level of the system.
  • a convolutional neural network is a feed-forward artificial neural network, and each neuron in the feed-forward artificial neural network can respond to the input image.
  • the convolutional neural network 300 may include an input layer 310, a convolutional layer/pooling layer 320 (wherein the pooling layer is optional), and a neural network layer 330.
  • the input layer 310 can obtain the image to be processed, and pass the obtained image to be processed to the convolutional layer/pooling layer 320 and the subsequent neural network layer 330 for processing, and the processing result of the image can be obtained.
  • the convolutional layer/pooling layer 320 may include layers 321-326, for example: in one implementation, layer 321 is a convolutional layer, layer 322 is a pooling layer, and layer 323 is a convolutional layer. Layers, 324 is a pooling layer, 325 is a convolutional layer, and 326 is a pooling layer; in another implementation, 321 and 322 are convolutional layers, 323 is a pooling layer, and 324 and 325 are convolutional layers. Layer, 326 is the pooling layer, that is, the output of the convolutional layer can be used as the input of the subsequent pooling layer, or can be used as the input of another convolutional layer to continue the convolution operation.
  • the convolution layer 321 can include many convolution operators.
  • the convolution operator is also called a kernel. Its role in image processing is equivalent to a filter that extracts specific information from the input image matrix.
  • the convolution operator is essentially It can be a weight matrix. This weight matrix is usually pre-defined. In the process of convolution on the image, the weight matrix is usually one pixel after one pixel (or two pixels after two pixels) along the horizontal direction on the input image. And so on, it depends on the value of stride) to complete the process of extracting specific features from the image.
  • the size of the weight matrix should be related to the size of the image. It should be noted that the depth dimension of the weight matrix and the depth dimension of the input image are the same.
  • the weight matrix will extend to Enter the entire depth of the image. Therefore, convolution with a single weight matrix will produce a single depth dimension convolution output, but in most cases, a single weight matrix is not used, but multiple weight matrices of the same size (row ⁇ column) are applied. That is, multiple homogeneous matrices.
  • the output of each weight matrix is stacked to form the depth dimension of the convolutional image, where the dimension can be understood as determined by the "multiple" mentioned above.
  • Different weight matrices can be used to extract different features in the image. For example, one weight matrix is used to extract the edge information of the image, another weight matrix is used to extract the specific color of the image, and the other weight matrix is used to correct the unwanted images in the image. The noise is blurred and so on.
  • the multiple weight matrices have the same size (row ⁇ column), the size of the convolution feature maps extracted by the multiple weight matrices of the same size are also the same, and then the multiple extracted convolution feature maps of the same size are merged to form The output of the convolution operation.
  • weight values in these weight matrices need to be obtained through a lot of training in practical applications.
  • Each weight matrix formed by the weight values obtained through training can be used to extract information from the input image, so that the convolutional neural network 300 can make correct predictions. .
  • the initial convolutional layer (such as 321) often extracts more general features, and the general features can also be called low-level features; with the convolutional neural network With the 300-depth deepening, the features extracted by the subsequent convolutional layer (for example, 326) become more and more complex, for example, features such as high-level semantics, and features with higher semantics are more suitable for the problem to be solved.
  • the pooling layer can be a convolutional layer followed by a layer.
  • the pooling layer can also be a multi-layer convolutional layer followed by one or more pooling layers.
  • the purpose of the pooling layer is to reduce the size of the image space.
  • the pooling layer may include an average pooling operator and/or a maximum pooling operator for sampling the input image to obtain an image with a smaller size.
  • the average pooling operator can calculate the pixel values in the image within a specific range to generate an average value as the result of the average pooling.
  • the maximum pooling operator can take the pixel with the largest value within a specific range as the result of the maximum pooling.
  • the operators in the pooling layer should also be related to the image size.
  • the size of the image output after processing by the pooling layer can be smaller than the size of the image of the input pooling layer, and each pixel in the image output by the pooling layer represents the average value or the maximum value of the corresponding sub-region of the image input to the pooling layer.
  • Neural network layer 330
  • the convolutional neural network 300 After processing by the convolutional layer/pooling layer 320, the convolutional neural network 300 is not enough to output the required output information. Because as mentioned above, the convolutional layer/pooling layer 320 only extracts features and reduces the parameters brought by the input image. However, in order to generate the final output information (required class information or other related information), the convolutional neural network 300 needs to use the neural network layer 330 to generate one or a group of required classes of output. Therefore, the neural network layer 330 may include multiple hidden layers (331, 332 to 33n as shown in FIG. 6) and an output layer 340. The parameters contained in the multiple hidden layers can be based on specific task types. Relevant training data of, for example, the task type can include image recognition, image classification, image detection, and image super-resolution reconstruction, etc.
  • the output layer 340 has a loss function similar to the classification cross entropy, which is specifically used to calculate the prediction error.
  • the convolutional neural network shown in FIG. 6 is only used as an example of the structure of the target neural network in the embodiment of the present application.
  • the convolutional neural network used in the image classification method of the embodiment of the present application The network can also exist in the form of other network models.
  • FIG. 7 is a hardware structure of a chip provided by an embodiment of the application.
  • the chip includes a neural network processor 400 (neural-network processing unit, NPU).
  • the chip can be set in the execution device 210 as shown in FIG. 4 to complete the calculation work of the calculation module 211.
  • the chip may also be set in the training device 220 as shown in FIG. 4 to complete the training work of the training device 220 and output the target model/rule 201.
  • the algorithms of each layer in the convolutional neural network as shown in FIG. 6 can be implemented in the chip as shown in FIG. 7.
  • the NPU 400 is mounted on a main central processing unit (CPU) as a coprocessor, and the main CPU allocates tasks.
  • the core part of the NPU 400 is the arithmetic circuit 403, and the controller 404 controls the arithmetic circuit 403 to extract data from the memory (weight memory or input memory) and perform calculations.
  • the arithmetic circuit 403 includes multiple processing units (process engines, PE). In some implementations, the arithmetic circuit 403 is a two-dimensional systolic array. The arithmetic circuit 403 may also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 403 is a general-purpose matrix processor.
  • the arithmetic circuit 403 fetches the data corresponding to matrix B from the weight memory 402 and caches it on each PE in the arithmetic circuit 403.
  • the arithmetic circuit 403 fetches the matrix A data and matrix B from the input memory 401 to perform matrix operations, and the partial result or final result of the obtained matrix is stored in an accumulator 408 (accumulator).
  • the vector calculation unit 407 can perform further processing on the output of the arithmetic circuit 403, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, and so on.
  • the vector calculation unit 407 can be used for network calculations in the non-convolutional/non-FC layer of the neural network, such as pooling, batch normalization, local response normalization, etc. .
  • the vector calculation unit 407 can store the processed output vector to the unified memory 406.
  • the vector calculation unit 407 may apply a nonlinear function to the output of the arithmetic circuit 403, such as a vector of accumulated values, to generate the activation value.
  • the vector calculation unit 407 generates a normalized value, a combined value, or both.
  • the processed output vector can be used as an activation input to the arithmetic circuit 403, for example for use in a subsequent layer in a neural network.
  • the unified memory 406 is used to store input data and output data.
  • the weight data directly passes through the storage unit access controller 405 (direct memory access controller, DMAC) to store the input data in the external memory into the input memory 401 and/or unified memory 406, and the weight data in the external memory into the weight memory 402 , And store the data in the unified memory 406 into the external memory.
  • DMAC direct memory access controller
  • the bus interface unit 410 (bus interface unit, BIU) is used to implement interaction between the main CPU, the DMAC, and the fetch memory 409 through the bus.
  • An instruction fetch buffer 409 (instruction fetch buffer) connected to the controller 404 is used to store instructions used by the controller 404.
  • the controller 404 is used to call the instructions cached in the instruction fetch memory 409 to control the working process of the computing accelerator.
  • the unified memory 406, the input memory 401, the weight memory 402, and the instruction fetch memory 409 are all on-chip (On-Chip) memories.
  • the external memory is a memory external to the NPU.
  • the external memory can be a double data rate synchronous dynamic random access memory.
  • Memory double data rate synchronous dynamic random access memory, DDR SDRAM), high bandwidth memory (HBM) or other readable and writable memory.
  • each layer in the convolutional neural network shown in FIG. 6 may be executed by the arithmetic circuit 403 or the vector calculation unit 407.
  • the execution device 210 in FIG. 4 introduced above can execute each step of the image classification method of the embodiment of the present application.
  • the CNN model shown in FIG. 6 and the chip shown in FIG. 7 can also be used to execute the image of the embodiment of the present application.
  • the various steps of the classification method can be executed.
  • FIG. 8 shows a system architecture 500 provided by an embodiment of the present application.
  • the system architecture includes a local device 520, a local device 530, an execution device 510, and a data storage system 550.
  • the local device 520 and the local device 530 are connected to the execution device 510 through a communication network.
  • the execution device 510 may be implemented by one or more servers.
  • the execution device 510 can be used in conjunction with other computing devices, such as data storage, routers, load balancers, and other devices.
  • the execution device 510 may be arranged on one physical site or distributed on multiple physical sites.
  • the execution device 510 may use the data in the data storage system 550 or call the program code in the data storage system 550 to implement the image classification method of the embodiment of the present application.
  • execution device 510 may also be referred to as a cloud device, and in this case, the execution device 510 may be deployed in the cloud.
  • the user can operate respective user devices (for example, the local device 520 and the local device 530) to interact with the execution device 510.
  • Each local device can represent any computing device, for example, a personal computer, a computer workstation, a smart phone, a tablet computer, a smart camera, a smart car or other types of cellular phones, a media consumption device, a wearable device, a set-top box, a game console, etc.
  • the local device of each user can interact with the execution device 510 through a communication network of any communication mechanism/communication standard.
  • the communication network can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof.
  • the local device 520 and the local device 530 can obtain the relevant parameters of the target neural network from the execution device 510, deploy the target neural network on the local device 520 and the local device 530, and use the target neural network to perform image processing. Classification processing, etc.
  • the target neural network can be directly deployed on the execution device 510.
  • the execution device 510 obtains the image to be processed from the local device 520 and the local device 530, and performs image classification processing on the image to be processed according to the target neural network.
  • the above-mentioned target neural network may be the target neural network used for image classification in the embodiment of the present application.
  • the classification of images is the basis of various image processing applications, and computer vision often involves the problem of how to classify the acquired images.
  • the network model usually used for image classification is based on the global feature map of the image for classification. Since the target area to be recognized may be a part of the global feature map, the classification process based on the global feature map will make the target area correspond to The feature map of is affected by the noise in the background area, which leads to a decrease in the accuracy of the network model for image classification.
  • the embodiment of the application proposes an image classification method.
  • the target neural network is trained through the target region classification loss function, the background region classification loss function, and the cooperative loss function, which can realize the separation of the feature map and the background region of the target region in the image to be processed.
  • the target region classification loss function the background region classification loss function
  • the cooperative loss function which can realize the separation of the feature map and the background region of the target region in the image to be processed.
  • FIG. 9 shows a schematic flowchart of an image classification method 600 provided by an embodiment of the present application.
  • the method may be executed by a device capable of image classification.
  • the method may be executed by the execution device 510 in FIG. It can be executed by the local device 520.
  • the method 600 includes steps 610 to 620, and these steps are respectively described in detail below.
  • the image to be processed may include the foreground area and the background area where the target object is located, and the target object may be any item such as pedestrians, animals, and vehicles.
  • the above-mentioned target area to be recognized may be regarded as a foreground area containing the target object, and the target object may be an object with a motion attribute.
  • the above-mentioned image to be processed may be an image captured by an electronic device through a camera, or the above-mentioned image to be processed may also be an image obtained from the inside of the electronic device (for example, an image stored in an album of an electronic device, or an electronic device).
  • the electronic device may be any one of the local device or the execution device shown in FIG. 8.
  • S620 Classify the image to be processed according to the target neural network to obtain the classification result of the target region and the classification result of the background region; wherein the target region classification loss function is used to represent the target region predicted classification result and the target region of the sample image Label the loss value between the classification results, the background region classification loss function is used to represent the loss value between the background region prediction classification result of the sample image and the background region label classification result, and the cooperative loss function is used to separate all The target area feature map of the sample image and the background area feature map of the sample image.
  • the target region classification loss function is used to represent the target region predicted classification result and the target region of the sample image Label the loss value between the classification results
  • the background region classification loss function is used to represent the loss value between the background region prediction classification result of the sample image and the background region label classification result
  • the cooperative loss function is used to separate all The target area feature map of the sample image and the background area feature map of the sample image.
  • the target region feature map (or the background region feature map) of the above-mentioned sample image may be a feature map obtained after processing of other layers in the convolutional neural network.
  • the other layers in the convolutional neural network mentioned here refer to a layer in the convolutional neural network.
  • the other layers can be the input layer, convolutional layer, pooling layer or full-scale convolutional neural network in the convolutional neural network.
  • One of the connection layers is one of the connection layers.
  • the classification result of the target area may refer to the identification of a pedestrian, where the identification of the pedestrian may refer to the name of the pedestrian, the number of the pedestrian, or the ID number of the pedestrian;
  • the classification result of the background area may refer to the identification of the camera that obtains the image to be processed, for example, the number of the camera and the location information of the camera.
  • the above is an example of the pedestrian identification and the camera identification.
  • the pedestrian identification is used to distinguish different behaviors.
  • the background area image obtained by it can be approximated as unchanged, that is, the image of the background area obtained by the camera can be
  • the identification determines the classification result of the background area image, and the specific implementation form of the identification of the pedestrian and the identification of the camera is not limited in this application.
  • the classification result of the output target area may be the identification information of pedestrian A, for example, it may be the ID number of pedestrian A or the ID of pedestrian A, as described above
  • the identification information can be used to identify pedestrian A.
  • the classification result of the output target area may be animal-cat.
  • the classification result of the target area and the classification result of the background area may refer to the identification information of pedestrian A and the identification information of camera A, where pedestrian A
  • the identification information of is the ID number of pedestrian A or the ID of pedestrian A.
  • the above identification information can be used to identify pedestrian A; the identification information of camera A can be the ID of the camera corresponding to the street view 1 area in the captured input image.
  • the target area mask of the sample image can be complementary to the background area mask through the aforementioned cooperative loss function.
  • the target area mask is used to indicate the probability that the location of each pixel in the target area feature map belongs to the target area; the background area mask is used to indicate that the location of each pixel in the background area feature map belongs to the background area The probability value.
  • the aforementioned collaborative loss function can make the overlap area of the target area image corresponding to the target area feature map in the sample image and the background area image corresponding to the background area feature map in the sample image nearly zero. That is to say, the cooperative loss function can make the target area feature map and the background area feature map in the target neural network extracted from the sample image clearly demarcate, so as to avoid the influence of the noise of the background area on the target area feature map.
  • the segmentation accuracy of the feature map of the target region and the feature map of the background region extracted by the target neural network can be improved by the collaborative loss function, that is, the feature map of the target region can be prevented from being affected by the background region. influences.
  • the parameters of the network are continuously adjusted, so that the target neural network can make the extracted target area feature map when the target neural network classifies the image to be processed.
  • the accuracy rate of the background area feature map and the accuracy rate of the background area feature map mutually promote the collaborative improvement.
  • the collaborative loss function may be obtained by weighting the first collaborative loss function and the second collaborative loss function, where the first collaborative loss function may be based on the target region feature map of the sample image and the background region mask of the sample image.
  • the second cooperative loss function may be obtained by multiplying the background area feature map of the sample image and the target area mask of the sample image.
  • the background area mask of the sample image can represent the probability value of the location of each pixel in the background area feature map of the acquired sample image belonging to the background area of the sample image;
  • the target area mask of the sample image can represent the acquired sample image The probability value of the location of each pixel in the target area feature map belonging to the target area of the sample image.
  • the value corresponding to each pixel in the target area mask represents the probability that the location of the pixel belongs to the target area
  • the value range of the probability value can be [0,1]; 0 can indicate the location of the pixel
  • the probability that the location belongs to the target area is 0; 1 can mean that the probability that the location of the pixel belongs to the target area is 1.
  • the value corresponding to each pixel in the background area mask represents the probability that the location of the pixel belongs to the background area.
  • the value range of the probability value can be [0,1]; 0 can indicate the location of the pixel The probability that the location of the pixel belongs to the background area is 0; 1 can indicate that the probability that the location of the pixel is located in the background area is 1.
  • the above-mentioned target region mask is equivalent to the target region attention map; the above-mentioned background region mask is equivalent to the background region attention map.
  • the scales of any two of the target area feature map, the background area feature map, the target area mask, and the background area mask of the image are the same.
  • the synergy loss can be obtained by the following equation:
  • L t represents the cooperative loss function
  • N represents the number of pixels in the sample image
  • F i represents the i-th pixel in the target area of the sample image
  • B i represents the i-th pixel in the background area of the sample image
  • ⁇ can be used to represent the corresponding elements to be multiplied.
  • N is an integer greater than 1
  • i is an integer greater than or equal to 1 and less than or equal to N.
  • the target neural network for image classification is obtained.
  • it can include two branches.
  • the first branch can be used to process the target area of the sample image
  • the second branch can be used to process the background area of the sample image.
  • the first branch can be used to cooperate with the first branch through the collaborative loss function.
  • the second branch performs training, so that the accuracy of the extracted feature map of the target area and the accuracy of the feature map of the background area promote each other to improve synergistically.
  • the target neural network may include a first branch and a second branch, where the first branch may be used to process the target area of the sample image to obtain the target area feature map of the sample image and the target area mask of the sample image ;
  • the second branch can be used to process the background area of the sample image to obtain the background area feature map of the sample image and the background area mask of the sample image.
  • the target area mask is obtained by training of the extracted target area feature map;
  • the background area mask is obtained by training of the extracted background area feature map.
  • the aforementioned target area classification loss function is used to represent the loss value between the predicted classification result corresponding to the target area feature map of the sample image and the target area label classification result, where the target area feature map of the sample image can also be Refers to the target area enhancement feature map of the sample image.
  • the target area enhancement feature map is obtained by multiplying the extracted low-level feature map of the target area with the corresponding pixels in the target area mask; similarly, the background area classification loss function is used Represents the loss value between the predicted classification result corresponding to the background area feature map of the sample image and the background area label classification result, where the background area feature map may also refer to the enhanced feature map of the background area, such as the enhanced feature map of the background area It is obtained by multiplying the extracted low-level feature map of the background area with the corresponding pixels in the background area mask.
  • the image to be processed can be obtained, and the image to be processed includes the target area to be recognized and the background area; the target neural network is used to perform feature extraction on the image to be processed to obtain the target The region feature map and the background region feature map; the target region feature map and the background region feature map are classified by the target neural network to obtain the classification result of the target region and the classification result of the background region.
  • the target neural network may include a first branch and a second branch, the target area feature map may be a target area enhanced feature map, and the background area feature map may be a background area enhanced feature map; for example, the first branch of the target neural network may be used.
  • One branch processes the target area of the image to be processed to obtain an enhanced feature map of the target area of the image to be processed, where the target area enhanced feature map may be the extracted target area layer feature map and the corresponding pixel points in the target area mask. Multiply obtained; classify based on the enhanced feature map of the target area, and obtain the predicted classification result of the target area in the image to be processed.
  • the background area of the image to be processed can be processed through the second branch of the target neural network to obtain the enhanced feature map of the background area of the image to be processed, where the enhanced feature map of the background area can be the extracted background area feature map and the background It is obtained by multiplying the corresponding pixels in the area mask; classification is performed based on the enhanced feature map of the background area, and the predicted classification result of the background area in the image to be processed is obtained.
  • first branch and the second branch in the target neural network can perform classification processing on the image to be processed at the same time, or, in the target neural network, the image to be processed can also be processed in the order of the first branch and then the second branch.
  • the classification processing, or, in the target neural network, the image to be processed may be classified in the order of the second branch and then the first branch.
  • the target area classification loss function can be obtained by the deviation between the predicted classification result of the target area and the target area label classification result
  • the background area classification can be obtained by the deviation between the predicted classification result of the background area and the background area label classification result Loss function.
  • the target neural network is obtained by training the target region classification loss function, the background region loss function, and the cooperative loss function, so that the pre-trained target neural network can accurately separate the target region feature map and the background region in the image to be processed Characteristic map. Therefore, when the target neural network classifies the image to be processed, by separating the feature map of the target area and the feature map of the background area in the image to be processed, it can not only obtain the classification result of the target area and the classification result of the background area, but also obtain the target area.
  • the feature map can also avoid the influence of the background area, and improve the accuracy of the classification result of the target area in the image to be processed.
  • FIG. 10 shows a schematic flowchart of a method 700 for training an image classification model provided by an embodiment of the present application.
  • the method may be executed by a device capable of image classification.
  • the method may be executed by the execution device 510 in FIG. 8.
  • it can also be executed by the local device 520.
  • the method 700 includes steps 710 to 740, and these steps are respectively described in detail below.
  • Step 710 Obtain training data, where the training data includes a sample image, a target area labeling classification result of the sample image, and a background area labeling classification result of the sample image.
  • the target area is the area where the target object is located in the sample image
  • the target object can be any of pedestrians, animals, and vehicles.
  • the training data may refer to a sample image containing pedestrian A, an identification of pedestrian A, and an identification of a background area image in the sample image.
  • it may be an identification of a camera that obtains the sample image.
  • Step 720 Obtain a target area prediction classification result of the sample image according to the target area feature map of the sample image extracted by the neural network.
  • a convolution operation may be performed on the sample image to obtain the low-level global feature of the sample image; the convolution operation may be performed on the low-level global feature of the sample image to obtain a target region feature map of the sample image.
  • a convolution operation may be performed on the sample image to directly obtain the target area feature map of the sample image.
  • the target area mask can be obtained according to the extracted target area feature map.
  • a full convolutional network can be used to perform convolution operations on the target area feature map to obtain a target area mask specific to the target area.
  • the target area mask is used to indicate that the location of each pixel in the target area feature map belongs to
  • the probability value of the target area for example, the value range of the target area mask can be [0,1], 0 can indicate that the location of the pixel is located in the target area, and the probability of belonging to the target area is 0; 1 can represent the location of the pixel The probability of belonging to the target area is 1.
  • obtaining the target region prediction classification result of the sample image includes:
  • the target area of the sample image can be processed by the target neural network to obtain the target area enhanced feature map of the sample image, where the target area enhanced feature map can be the extracted target area low-level feature map and the corresponding pixels in the target area mask It is obtained by multiplying the points; the classification is performed based on the enhanced feature map of the target area, and the predicted classification result of the target area in the sample image is obtained.
  • the above-mentioned classification based on the enhanced feature map of the target region can be classified using the softmax algorithm in the prior art, which will not be repeated here.
  • the softmax algorithm can convert a set of scores of (- ⁇ , + ⁇ ) into a set of probabilities, and make the sum of this set of probabilities a normalized function of 1.
  • this function has Order preservation, that is, the probability value after conversion with the original high score is larger, and the probability value corresponding to the small score is smaller.
  • the classification result of the output target area may be the identification information of pedestrian A, for example, the identification number of pedestrian A or the ID of pedestrian A.
  • the above identification information can be used for Identify pedestrian A.
  • the classification result of the output target area may be animal-cat.
  • Step 730 Obtain the background area prediction classification result of the sample image according to the background area feature map of the sample image extracted by the neural network.
  • a convolution operation may be performed on the sample image to obtain the low-level global feature of the sample image; the convolution operation may be performed on the low-level global feature of the sample image to obtain a background area feature map of the sample image.
  • a convolution operation may be performed on the sample image to directly obtain the background area feature map of the sample image.
  • the background area mask can be obtained according to the extracted feature map of the target area.
  • a full convolutional network can be used to perform convolution operations on the background area feature map to obtain the background area mask corresponding to the background area.
  • the background area mask is used to indicate that the location of each pixel in the background area feature map belongs to
  • the probability value of the background area for example, the value range of the background area mask can be [0,1], 0 can indicate that the location of the pixel is located in the background area, and the probability is 0; 1 can represent the location of the pixel The probability of belonging to the background area is 1.
  • obtaining the background area prediction classification result of the sample image includes:
  • the background area of the sample image can be processed by the target neural network to obtain the background area enhancement feature map of the sample image, where the background area enhancement feature map can be the extracted background area low-level feature map and the corresponding pixels in the background area mask It is obtained by multiplying the points; the classification is performed based on the enhanced feature map of the background area, and the predicted classification result of the background area in the sample image is obtained.
  • the foregoing classification based on the enhanced feature map based on the background area can be classified using the softmax algorithm in the prior art, which will not be repeated here.
  • the classification result of the background area may refer to the identification information of line camera A, and the identification information of camera A may be the ID of the camera corresponding to the street view 1 area in the captured input image .
  • the classification result of the background area of the sample image refers to the identification of the camera that has obtained the sample image.
  • Step 740 Determine the parameters of the neural network based on the target area classification loss function, the background area classification loss function, and the collaborative loss function, where the target area classification loss function is used to represent the target area prediction classification result and the target area label classification result of the sample image
  • the background area classification loss function is used to represent the loss value between the background area prediction classification result of the sample image and the background area annotation classification result
  • the cooperative loss function is used to separate the target area feature map of the sample image and The background area feature map of the sample image.
  • the target area mask of the sample image can be complementary to the background area mask through the aforementioned cooperative loss function.
  • the aforementioned collaborative loss function can make the overlap area of the target area image corresponding to the target area feature map in the sample image and the background area image corresponding to the background area feature map in the sample image nearly zero. That is to say, the cooperative loss function can make the target area feature map and the background area feature map in the target neural network extracted from the sample image clearly demarcate, so as to avoid the influence of the noise of the background area on the target area feature map.
  • the collaborative loss function may be obtained by weighting the first collaborative loss function and the second collaborative loss function, where the first collaborative loss function may be based on the target region feature map of the sample image and the background region mask of the sample image. The corresponding pixel points in the film are multiplied; the second cooperative loss function may be obtained based on the multiplication of the background area feature map of the sample image and the corresponding pixel points in the target area mask of the sample image.
  • the scales of any two of the target area feature map, the background area feature map, the target area mask, and the background area mask of the image are the same.
  • the cooperative loss function can be obtained by the following equation:
  • L t represents the cooperative loss function
  • N represents the number of pixels in the sample image
  • F i represents the i-th pixel in the target area of the sample image
  • B i represents the i-th pixel in the background area of the sample image
  • ⁇ is used to represent the corresponding elements to be multiplied.
  • N is an integer greater than 1
  • i is an integer greater than or equal to 1 and less than or equal to N.
  • the final loss function obtained based on the target area classification loss function, the background area classification loss function, and the cooperative loss function can be:
  • L f can represent the target area classification loss function
  • L b can represent the background area classification loss function
  • L t can represent the cooperative loss function.
  • the segmentation accuracy of the feature map of the target region and the feature map of the background region extracted by the target neural network can be improved by the collaborative loss function, that is, the feature map of the target region can be prevented from being affected by the background region. influences.
  • the parameters of the network are continuously adjusted, so that the target neural network can make the accuracy of the extracted target area feature map when the target neural network classifies the image to be processed.
  • the accuracy of the feature map of the background area and the background area promote each other to improve the synergy.
  • the target neural network may include a first branch and a second branch, where the first branch may be used to process the target area of the sample image to obtain the target area feature map of the sample image and the target area mask of the sample image ;
  • the second branch can be used to process the background area of the sample image to obtain the background area feature map of the sample image and the background area mask of the sample image.
  • the target neural network for image classification is obtained.
  • it can include two branches.
  • the first branch can be used to process the target area of the sample image
  • the second branch can be used to process the background area of the sample image.
  • the first branch can be used to cooperate with the first branch through the collaborative loss function.
  • the second branch performs training, so that the accuracy of the extracted feature map of the target area and the accuracy of the feature map of the background area promote each other to improve synergistically.
  • FIG. 11 shows a schematic flowchart of a method 800 for training an image classification model provided by an embodiment of this application.
  • the method 800 includes steps 801 to 809, and these steps are respectively described in detail below.
  • Step 801 Input an image.
  • the input image may be training data used to train an image classification model, and the training data includes a sample image, a target area labeling classification result of the sample image, and a background area labeling classification result of the sample image.
  • the target area is the area where the target object is located in the sample image
  • the target object can be any of pedestrians, animals, and vehicles.
  • Step 802 Extract low-level features.
  • the low-level features of the input image can be extracted through the low-level feature extraction module, where the low-level feature extraction module can adopt a variety of architectures, such as VGG16, TSN network, and so on.
  • the aforementioned low-level feature of the input image may refer to the low-level texture feature of the input image learned by the low-level feature extraction module.
  • Step 803 Extract the features of the foreground area.
  • the foreground area feature can be extracted by the foreground extraction module, and the foreground feature extraction module can be a multi-layer convolutional neural network.
  • the output feature of the foreground area can be obtained.
  • the aforementioned foreground area feature may refer to the area where the target object is located, for example, the target object may refer to pedestrians, animals, vehicles, and so on.
  • Step 804 Generate a foreground attention mask.
  • the foreground attention mask can be obtained through the foreground attention mask generation module, where the foreground attention mask generation module can be a fully convolutional neural network, and the fully convolutional neural network can include two convolution modules, each A convolution module can contain three consecutive operations: convolution layer, batch normalization layer (bach normalization, BN), linear rectification function (rectified linear unit, ReLU).
  • the input of the foreground attention mask generation module is the foreground feature F, and the output is the foreground attention mask Z f .
  • the size of the foreground attention mask is the same as the size of the foreground feature.
  • the value corresponding to each position in the foreground attention mask represents the probability that the position belongs to the foreground area.
  • the value range of the probability value can be [0,1 ], 0 means that the probability that the position belongs to the foreground is 0, and 1 means that the probability that the position belongs to the foreground is 1.
  • Step 805 Obtain foreground enhancement features
  • the foreground enhancement feature is obtained by enhancing the foreground feature F through the foreground attention mask Z f , thereby suppressing background noise.
  • the enhanced foreground feature can be obtained by multiplying the foreground attention mask Z f and the foreground feature F pixel by pixel, where the foreground feature F and the foreground attention mask Z f have the same size, and each of the foreground attention masks The value of each position represents the probability that it belongs to the foreground area.
  • the foreground enhancement feature can be input into the foreground classifier for classification, and the predicted classification result of the foreground region can be obtained.
  • the classification result is which pedestrian ID the input image belongs to.
  • the pedestrian ID may refer to the name of the pedestrian, the number of the pedestrian, or the ID number of the pedestrian, etc., and the pedestrian ID may be used to distinguish different pedestrians.
  • Step 806 Extract background features.
  • the background area feature can be extracted by the background extraction module, and the background feature extraction module can be a multi-layer convolutional neural network.
  • the background feature extraction module can be a multi-layer convolutional neural network.
  • Step 807 Generate a background attention mask.
  • the background attention mask can be obtained through the background attention mask generation module, where the background attention mask generation module can be a fully convolutional neural network, and the fully convolutional neural network can include two convolution modules, each A convolution module can contain three consecutive operations: convolution layer, batch normalization layer (bach normalization, BN), linear rectification function (rectified linear unit, ReLU).
  • the input of the background attention mask generation module is the background feature B, and the output is the background attention mask Z b .
  • the size of the background attention mask is the same as the size of the background feature.
  • the value corresponding to each position in the background attention mask represents the probability that the position belongs to the background area.
  • the value range of the probability value can be [0,1 ], 0 means that the probability that the position belongs to the background is 0, and 1 means that the probability that the position belongs to the background is 1.
  • Step 808 Obtain background enhancement features
  • the foreground enhancement feature is obtained by enhancing the background feature B through the background attention mask Z b , thereby suppressing foreground noise.
  • the enhanced background feature can be obtained by multiplying the background attention mask Z b and the background feature B pixel by pixel, where the background feature B and the background attention mask Z b have the same size, and each of the background attention masks The value of each position represents the probability that it belongs to the background area.
  • the background enhancement feature can be input into the foreground classifier for classification, and the predicted classification result of the background area can be obtained.
  • the acquired background area can be approximately regarded as unchanged, and the classification result corresponding to the background area image can be determined by the camera ID.
  • the camera ID may refer to the number of the camera, the location information of the camera, and so on.
  • Step 809 Cooperative training.
  • the foreground feature map can be correspondingly multiplied with the background attention mask, and the background feature map can be correspondingly multiplied with the foreground attention mask to generate the co-training loss function.
  • the input image, the pedestrian ID corresponding to the input image, and the camera ID can be included.
  • the softmax loss function of the foreground branch can be obtained through the pedestrian ID, and the softmax loss function of the background branch can be obtained through the camera ID;
  • the co-training loss function, the softmax loss function of the foreground branch and the softmax loss function of the background branch are weighted to obtain the final loss function of the cooperative two-way network, and the co-training loss function is minimized through the back propagation algorithm, thereby generating each of the two-way network The parameter value.
  • the flow chart of the training method of the above-mentioned target neural network for image classification is shown in FIG. 12.
  • the input image is a pedestrian image for example.
  • the output of the classification result of the foreground area It can be the ID of a pedestrian;
  • the classification result of the background area can be the camera ID, that is, it can be the identification of the camera that has acquired the input image.
  • the process of the training method of the target neural neural network may include the following steps:
  • Step 1 Input the training image to a feature extraction module (for example, a low-level feature extraction module) to obtain a global feature map of the training image;
  • a feature extraction module for example, a low-level feature extraction module
  • Step 2 Input the global feature map of the training image into the foreground feature extraction module and the background feature extraction module, respectively, to obtain the foreground feature map and the background feature map of the training image.
  • the global feature map of the training image can be simultaneously input to the foreground feature extraction module and the background feature extraction module; or, the global feature map of the training image can be input to the foreground feature extraction module first, and then the global feature of the training image The image is input to the background feature extraction module.
  • the aforementioned foreground feature extraction module and background feature extraction module can process the feature map of the training image in parallel, and the input sequence is not limited.
  • Step 3 Input the foreground feature map into the foreground attention mask generation module to generate the foreground attention mask
  • Step 4 Input the background feature map into the background attention mask generation module to generate the background attention mask
  • steps three and four may be performed simultaneously, or step four may be performed first and then step three may be performed, and the order of performing steps three and four is not limited in any way.
  • Step 5 Multiply the foreground feature map and the foreground attention mask pixel by pixel to generate an enhanced foreground feature map
  • an enhanced foreground feature map can be obtained by inputting the feature map of the foreground area into a target enhancement module (TEM).
  • TEM target enhancement module
  • Step 6 Multiply the background feature map and the background attention mask pixel by pixel to generate an enhanced background feature map
  • the background area feature map is input into a target enhancement module (target enhancement module, TEM) to obtain an enhanced background feature map.
  • target enhancement module target enhancement module, TEM
  • the TEM of the first branch and the TEM of the second branch in FIG. 12 may be two different modules.
  • the TEM in the first branch is used to enhance the feature map of the foreground area; the TEM in the second branch is used to enhance the feature map of the background area.
  • steps 5 and 6 can be executed simultaneously, or step 5 can be executed first and then step 6 can be executed, and the order of execution of step 5 and step 6 is not limited in any way.
  • Step 7 Down-sample the enhanced foreground feature map and input it into the foreground classifier for classification, and the classified category is pedestrian ID;
  • the pedestrian ID may refer to any item such as the name of the pedestrian, the number of the pedestrian, or the ID number of the pedestrian.
  • Step 8 Down-sample the enhanced background feature map and input it into the background classifier for classification, and the classification obtained is the camera ID.
  • the camera ID may refer to the number of the camera or the location information of the camera.
  • Pedestrian ID is used to distinguish different behaviors.
  • the background area image obtained by it can be approximated as unchanged, that is, the background area can be determined by the camera ID.
  • the classification result of the image, the specific implementation form of the pedestrian ID and the camera ID are not limited in this application.
  • step seven and step eight may be performed simultaneously, or step eight may be performed first and then step seven may be performed, and the execution sequence of step seven and step eight is not limited.
  • Step 9 Multiply the foreground feature map and the background attention mask correspondingly, and at the same time multiply the background feature map and the foreground attention mask to generate the cooperative training Loss, and minimize the cooperative training Loss through the backpropagation algorithm.
  • the cooperative two-way network shown in FIG. 12 uses pedestrian ID and camera ID to supervisely train the cooperative two-way network, so that the cooperative two-way network can simultaneously extract pedestrian features and background features.
  • the accuracy of the foreground feature map, the foreground attention mask, the background feature map, and the background attention mask can be improved synergistically; at the same time, because the camera ID is used as supervision information, the network is extracting attention No additional annotation data set is required for masking.
  • Fig. 13 is a schematic block diagram of an image classification device provided by an embodiment of the present application. It should be understood that the image classification device 900 can execute the image classification method shown in FIG. 9.
  • the image classification device 900 includes: an acquiring unit 910 and a processing unit 920.
  • the acquiring unit 910 is configured to acquire a to-be-processed image, which includes a target area to be recognized and a background area;
  • the processing unit 920 is configured to classify the to-be-processed image according to the target neural network to obtain the The classification result of the target area and the classification result of the background area of the image to be processed;
  • the target neural network is trained based on the target area classification loss function, the background area classification loss function, and the collaborative loss function.
  • the target area classification loss function is used to represent the predicted classification result corresponding to the target area feature map of the sample image
  • the loss value between the label classification result of the target region and the background region classification loss function is used to represent the loss value between the predicted classification result corresponding to the background region feature map of the sample image and the background region label classification result
  • the The collaborative loss function is used to separate the target area feature map of the sample image and the background area feature map of the sample image.
  • the cooperative loss function is obtained by weighting a first cooperative loss function and a second cooperative loss function, wherein the first cooperative loss function is based on the target area of the sample image
  • the feature map is obtained by multiplying corresponding pixels in the background area mask of the sample image
  • the second cooperative loss function is obtained based on the multiplication of the background area feature map of the sample image and the corresponding pixel points in the target area mask of the sample image.
  • the target neural network includes a first branch and a second branch, wherein the first branch is used to process the target area of the sample image to obtain the target of the sample image An area feature map and a target area mask of the sample image;
  • the second branch is used to process the background area of the sample image to obtain a background area feature map of the sample image and a background area mask of the sample image.
  • the cooperative loss function is obtained by the following equation:
  • L t represents the cooperative loss function
  • N represents the number of pixels in the sample image
  • F i represents the i-th pixel in the target area of the sample image
  • B i represents the i-th pixel in the background area of the sample image
  • ⁇ is used to represent the corresponding elements to be multiplied
  • N is an integer greater than 1
  • i is an integer greater than or equal to 1 and less than or equal to N .
  • the processing unit 920 is specifically configured to:
  • the classification is performed based on the target region enhanced feature map and the background region enhanced feature map of the image to be processed, and the classification result of the target region and the classification result of the background region of the image to be processed are obtained.
  • the classification result of the background area of the to-be-processed image refers to the identification of the camera that has acquired the to-be-processed image.
  • Fig. 14 is a schematic block diagram of an image classification model training device provided by an embodiment of the present application. It should be understood that the training device 1000 can execute the training method of the image classification model shown in FIG. 10 or FIG. 11.
  • the training device 1000 includes: an acquisition unit 1010 and a processing unit 1020.
  • the acquiring unit 1010 is configured to acquire training data, wherein the training data includes a sample image, a target area labeling classification result of the sample image, and a background area labeling classification result of the sample image; the processing unit 1020, configured to obtain the target area prediction classification result of the sample image according to the target area feature map of the sample image extracted by the neural network; obtain the background area feature map of the sample image extracted by the neural network Predicting the classification result of the background area of the sample image; determining the parameters of the neural network based on the target area classification loss function, the background area classification loss function, and the cooperative loss function of the sample image;
  • the target area classification loss function is used to represent the loss value between the target area prediction classification result of the sample image and the target area label classification result
  • the background area classification loss function is used to represent the background of the sample image
  • the loss value between the area prediction classification result and the background area annotation classification result, and the cooperative loss function is used to separate the target area feature map of the sample image and the background area feature map of the sample image.
  • the cooperative loss function is obtained by weighting a first cooperative loss function and a second cooperative loss function, wherein the first cooperative loss function is based on the target area of the sample image
  • the feature map is obtained by multiplying corresponding pixels in the background area mask of the sample image
  • the second cooperative loss function is obtained based on the multiplication of the background area feature map of the sample image and the corresponding pixel points in the target area mask of the sample image.
  • the neural network includes a first branch and a second branch, and the first branch is used to process a target area of the sample image to obtain a target area feature map of the sample image And the target area mask of the sample image;
  • the second branch is used to process the background area of the sample image to obtain a background area feature map of the sample image and a background area mask of the sample image.
  • the cooperative loss function is obtained by the following equation:
  • L t represents the cooperative loss function
  • N represents the number of pixels in the sample image
  • F i represents the i-th pixel in the target area of the sample image
  • B i represents the i-th pixel in the background area of the sample image
  • ⁇ is used to represent the corresponding elements to be multiplied
  • N is an integer greater than 1
  • i is an integer greater than or equal to 1 and less than or equal to N .
  • the predicted classification result of the target area of the sample image refers to the identification of the camera that has obtained the sample image.
  • image classification device 900 and image classification model training device 1000 are embodied in the form of functional units.
  • unit herein can be implemented in the form of software and/or hardware, which is not specifically limited.
  • a "unit” may be a software program, a hardware circuit, or a combination of the two that realize the above-mentioned functions.
  • the hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, and a processor for executing one or more software or firmware programs (such as a shared processor, a dedicated processor, or a group processor). Etc.) and memory, combined logic circuits and/or other suitable components that support the described functions.
  • the units of the examples described in the embodiments of the present application can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.
  • FIG. 15 is a schematic diagram of the hardware structure of an image classification device provided by an embodiment of the present application.
  • the image classification apparatus 1100 shown in FIG. 15 includes a memory 1101, a processor 1102, a communication interface 1103, and a bus 1104.
  • the memory 1101, the processor 1102, and the communication interface 1103 implement communication connections between each other through the bus 1104.
  • the memory 1101 may be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).
  • the memory 1101 may store a program.
  • the processor 1102 is configured to execute each step of the image classification method of the embodiment of the present application, for example, execute each step shown in FIG. 9.
  • the image classification device shown in the embodiment of the present application may be a server, for example, it may be a server in the cloud, or may also be a chip configured in a server in the cloud.
  • the processor 1102 may adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more
  • the integrated circuit is used to execute related programs to implement the image classification method in the method embodiment of the present application.
  • the processor 1102 may also be an integrated circuit chip with signal processing capabilities.
  • each step of the image classification method of the present application can be completed by an integrated logic circuit of hardware in the processor 1102 or instructions in the form of software.
  • the above-mentioned processor 1102 may also be a general-purpose processor, a digital signal processing (digital signal processing, DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, Discrete gates or transistor logic devices, discrete hardware components.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory 1101, and the processor 1102 reads the information in the memory 1101, and combines its hardware to complete the functions required by the units included in the image classification device shown in FIG. 13 in the implementation of this application, or execute the method of this application The image classification method shown in FIG. 9 of the embodiment.
  • the communication interface 1103 uses a transceiving device such as but not limited to a transceiver to implement communication between the device 1100 and other devices or a communication network.
  • a transceiving device such as but not limited to a transceiver to implement communication between the device 1100 and other devices or a communication network.
  • the bus 1104 may include a path for transferring information between various components of the device 1100 (for example, the memory 1101, the processor 1102, and the communication interface 1103).
  • FIG. 16 is a schematic diagram of the hardware structure of a training device for an image classification model provided by an embodiment of the present application.
  • the training apparatus 1200 shown in FIG. 16 includes a memory 1201, a processor 1202, a communication interface 1203, and a bus 1204.
  • the memory 1201, the processor 1202, and the communication interface 1203 implement communication connections between each other through the bus 1204.
  • the memory 1201 may be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).
  • the memory 1201 may store a program.
  • the processor 1202 is configured to execute each step of the image classification model training method of the embodiment of the present application, for example, execute the steps shown in FIG. 10 or FIG. 11 The various steps shown.
  • the training device shown in the embodiment of the present application may be a server, for example, it may be a server in the cloud, or may also be a chip configured in a server in the cloud.
  • the processor 1202 may adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more
  • the integrated circuit is used to execute related programs to realize the training method of the image classification model in the method embodiment of the present application.
  • the processor 1202 may also be an integrated circuit chip with signal processing capability.
  • each step of the training method of the image classification model of the present application can be completed by the integrated logic circuit of the hardware in the processor 1202 or the instructions in the form of software.
  • the above-mentioned processor 1202 may also be a general-purpose processor, a digital signal processing (digital signal processing, DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, Discrete gates or transistor logic devices, discrete hardware components.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory 1201, and the processor 1202 reads the information in the memory 1201, and combines its hardware to complete the functions required by the units included in the image classification model training device shown in FIG. 14, or execute the method implementation of this application Example of the training method of the image classification model shown in Fig. 10 or Fig. 11.
  • the communication interface 1203 uses a transceiver device such as but not limited to a transceiver to implement communication between the device 1200 and other devices or a communication network.
  • a transceiver device such as but not limited to a transceiver to implement communication between the device 1200 and other devices or a communication network.
  • the bus 1204 may include a path for transferring information between various components of the device 1200 (for example, the memory 1201, the processor 1202, and the communication interface 1203).
  • the device 1100 and training device 1200 may also include the implementation of normal operation. Other necessary devices.
  • the aforementioned device 1100 and training device 1200 may also include hardware devices that implement other additional functions.
  • the above-mentioned device 1100 and training device 1200 may also only include the components necessary to implement the embodiments of the present application, and not necessarily include all the components shown in FIG. 15 or FIG. 16.
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of the present application essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

本申请公开了人工智能领域中计算机视觉领域的一种图像分类方法以及装置，该图像分类方法包括：获取待处理图像，该待处理图像包括待识别的目标区域以及背景区域；根据目标神经网络对该待处理图像进行分类，得到该待处理图像的该目标区域的分类结果与该背景区域的分类结果；其中，该目标神经网络是基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数训练得到的，该协同损失函数用于分离样本图像的目标区域特征图与该样本图像的背景区域特征图。本申请的技术方案能够抑制背景区域噪声对目标区域特征图的影响，从而提高目标区域分类结果的准确性。

Description

图像分类方法以及装置
本申请要求于2019年09月02日提交中国专利局、申请号为201910824602.1、申请名称为“图像分类方法以及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉人工智能领域，更具体地，涉及计算机视觉领域中的图像分类方法以及装置。
背景技术
计算机视觉是各个应用领域，如制造业、检验、文档分析、医疗诊断，和军事等领域中各种智能/自主系统中不可分割的一部分，它是一门关于如何运用照相机/摄像机和计算机来获取我们所需的，被拍摄对象的数据与信息的学问。形象地说，就是给计算机安装上眼睛(照相机/摄像机)和大脑(算法)用来代替人眼对目标进行识别、跟踪和测量等，从而使计算机能够感知环境。因为感知可以看作是从感官信号中提取信息，所以计算机视觉也可以看作是研究如何使人工系统从图像或多维数据中“感知”的科学。总的来说，计算机视觉就是用各种成像系统代替视觉器官获取输入信息，再由计算机来代替大脑对这些输入信息完成处理和解释。计算机视觉的最终研究目标就是使计算机能像人那样通过视觉观察和理解世界，具有自主适应环境的能力。
识别技术是计算机视觉中的经典问题之一。例如，行人重识别技术可以是通过利用计算机视觉技术判断图像或者视频序列中是否存在特定行人的技术。例如，给定一张行人图像，通过行人重识别技术可以实现检索跨设备下的该行人的图像并生成该行人的行为轨迹。因此，识别技术可以广泛应用于智能视频监控、智能安保等领域。
目前，识别技术主要是基于输入图像的全局特征图进行分类，全局特征图中包括待识别的目标区域特征图以及背景区域特征图，由于进行分类时待识别的目标区域特征图容易受到背景区域特征图噪声的影响，导致目标区域分类结果的准确性降低。因此，如何提高目标区域分类结果的准确性成为一个亟需解决的问题。
发明内容
本申请提供一种图像分类方法以及装置，能够实现待处理图像中目标区域特征与背景区域特征的分离，通过抑制背景区域噪声对目标区域特征图的影响，从而提高目标区域分类结果的准确性。
第一方面，提供了一种图像分类方法，包括：获取待处理图像，该待处理图像包括待识别的目标区域以及背景区域；根据目标神经网络对该待处理图像进行分类，得到该待处理图像的该目标区域的分类结果与该背景区域的分类结果；
其中，该目标神经网络是基于目标区域分类损失函数、背景区域分类损失函数以及协 同损失函数训练得到的，该目标区域分类损失函数用于表示样本图像的目标区域特征图对应的预测分类结果与目标区域标注分类结果之间的损失值，该背景区域分类损失函数用于表示该样本图像的背景区域特征图对应的预测分类结果与背景区域标注分类结果之间的损失值，该协同损失函数用于分离该样本图像的目标区域特征图与该样本图像的背景区域特征图。
其中，上述待识别的目标区域即可以看作是包含目标对象所在的前景区域，目标对象可以是具有运动属性的对象，或者，上述目标对象还可以是其他对象。
例如，目标区域可以是指行人、动物以及车辆等任意一项所在的区域，即目标区域可以是人像、动物图像以及车辆图像等任意一项。
示例性地，当待处理图像中的目标区域为人像时，目标区域的分类结果可以是指行人的标识，其中，行人的标识可以是指行人的姓名、行人的编号或者行人的身份证号码；背景区域的分类结果可以是指获取到待处理图像的摄像头的标识，比如，摄像头的编号、摄像头所在的位置信息。
需要说明的是，上述为对行人的标识与摄像头的标识的举例说明，行人的标识用于区分不同的行为，对于同一摄像头可以近似看作其获取的背景区域图像不变，即可以通过摄像头的标识确定背景区域图像的分类结果，行人的标识与摄像头的标识的具体实现形式本申请不作任何限定。在本申请中，目标神经网络是通过目标区域分类损失函数、背景区域损失函数以及协同损失函数训练得到的，使得预先训练的目标神经网络能够准确地分离待处理图像中目标区域特征图与背景区域的特征图。因此，目标神经网络在对待处理图像进行分类时，通过分离待处理图像中目标区域特征图以及背景区域特征图，不仅可以得到目标区域的分类结果以及背景区域的分类结果，同时在获取目标区域的特征图时还能避免受到背景区域的影响，提高待处理图像中目标区域分类结果的准确性。
在一种可能的实现方式中，上述待处理图像可以是指电子设备通过摄像头拍摄到的图像，或者，上述待处理图像还可以是从电子设备内部获得的图像(例如，电子设备的相册中存储的图像，或者，电子设备从云端获取的图片)。比如，上述待处理图像可以是获取的监控数据中的图片。
在一种可能的实现方式中，通过上述协同损失函数可以使得样本图像的目标区域掩膜与背景区域掩膜互补。
例如，当掩膜的取值范围为[0,1]时，目标区域掩膜与背景区域掩膜的总和可以为1，即背景区域掩膜＝1-目标区域掩膜。其中，目标区域掩膜用于表示目标区域特征图中每个像素点所在的位置属于目标区域的概率值；背景区域掩膜用于表示背景区域特征图中每个像素点所在的位置属于背景区域的概率值。
在另一种可能的实现方式中，通过上述协同损失函数可以使得样本图像中的目标区域特征图对应的目标区域图像与样本图像中的背景区域特征图对应的背景区域图像的重叠区域近乎为零，即通过协同损失函数可以使得目标神经网络提取样本图像中目标区域特征图与背景区域特征图的划界明显，从而避免背景区域的噪声对目标区域特征图的影响。
结合第一方面，在第一方面的某些实现方式中，该协同损失函数是通过第一协同损失函数和第二协同损失函数加权处理得到的，其中，该第一协同损失函数是基于该样本图像的目标区域特征图与该样本图像的背景区域掩膜相乘得到的；该第二协同损失函数是基于 该样本图像的背景区域特征图与该样本图像的目标区域掩膜相乘得到的。
在一种可能的实现方式中，目标区域掩膜是通过提取的目标区域特征图训练得到的；背景区域掩膜是通过提取的背景区域特征图训练得到的。
上述目标区域掩膜等同于目标区域注意力图；上述背景区域掩膜等同于背景区域注意力图。
需要说明的是，在本申请中图像的目标区域特征图、背景区域特征图、目标区域掩膜以及背景区域掩膜中任意两个的尺度大小相同。
在本申请中，通过目标区域特征图与背景区域掩膜中对应的像素点相乘得到的第一损失函数可以用于表示提取的目标区域特征与背景区域的重叠部分，即目标区域特征图受背景区域噪声影响的范围大小；通过背景区域特征图与目标区域掩膜中对应的像素点相乘得到的第二损失函数可以用于表示提取的背景区域特征图与目标区域的重叠部分，即背景区域特征图受目前区域噪声影响的范围大小，通过第一协同损失函数与第二损失函数加权处理得到协同损失函数，使得训练目标神经网络时使得目标区域特征与背景区域特征进行分离，实现目标区域特征图的准确率和背景区域特征图的准确率相互促进协同提高。
结合第一方面，在第一方面的某些实现方式中，该目标神经网络包括第一分支和第二分支，其中，该第一分支用于对该样本图像的目标区域进行处理，得到该样本图像的目标区域特征图和该样本图像的目标区域掩膜；
该第二分支用于对该样本图像的背景区域进行处理，得到该样本图像的背景区域特征图和该样本图像的背景区域掩膜。
在本申请中，为了便于通过前景区域分类损失函数、背景区域分类损失函数以及协同损失函数训练神经网络得到用于图像分类的目标神经网络。目标神经网络中可以包括两个分支，第一分支可以用于对样本图像的目标区域进行处理，第二分支可以用于对样本图像的背景区域进行处理，通过协同损失函数可以协同第一分支与第二分支进行训练，使得提取的目标区域特征图的准确率和背景区域特征图的准确率相互促进协同提高。
结合第一方面，在第一方面的某些实现方式中，该协同损失函数是通过以下等式得到的：
Figure PCTCN2020112694-appb-000001
其中，L t表示该协同损失函数，N表示该样本图像中像素点的数量，F i表示该样本图像的目标区域中第i个像素点，表示该样本图像的目标区域掩膜中第i个像素点的概率值，B i表示该样本图像的背景区域中第i个像素点，表示该样本图像的背景区域掩膜中第i个像素点的概率值，N为大于1的整数，i为大于或等于1且小于或等于N的整数。
在一种可能的实现方式中，整体损失函数包括上述分类损失函数、上述背景区域分类损失函数以及上述协同损失函数，整体损失函数是通过以下等式得到的：
Figure PCTCN2020112694-appb-000002
其中，L f可以表示目标区域分类损失函数，L b可以表示背景区域分类损失函数，L t可以表示协同损失函数。
结合第一方面，在第一方面的某些实现方式中，根据目标神经网络该待处理图像进行 分类，得到待处理图像的目标区域的分类结果与背景区域的分类结果，包括：通过该目标神经网络的该第一分支对该待处理图像的目标区域进行处理，得到该待处理图像的目标区域增强特征图，其中，该目标区域增强特征图是通过该待处理图像的目标区域特征图和该待处理图像的目标区域掩膜相乘得到的；通过该目标神经网络的该第二分支对该待处理图像的背景区域进行处理，得到该待处理图像的背景区域增强特征图，其中，该背景区域增强特征图是通过该待处理图像的背景区域特征图和该待处理图像的背景区域掩膜相乘得到的；基于该待处理图像的目标区域增强特征图与背景区域增强特征图进行分类，得到该待处理图像的目标区域的分类结果与背景区域的分类结果。
结合第一方面，在第一方面的某些实现方式中，待处理图像的背景区域的分类结果是指获取到该待处理图像的摄像头的标识。
应理解，对于同一摄像头可以近似看作其获取的背景区域不变，则可以通过摄像头的标识确定背景区域图像对应的标识信息。
第二方面，提供了一种图像分类模型的训练方法，包括：获取训练数据，其中，该训练数据包括样本图像、该样本图像的目标区域标注分类结果、该样本图像的背景区域标注分类结果；根据神经网络提取的该样本图像的目标区域特征图，得到该样本图像的目标区域预测分类结果；根据该神经网络提取的该样本图像的背景区域特征图，得到该样本图像的背景区域预测分类结果；基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数，确定该神经网络的参数，其中，该目标区域分类损失函数用于表示该样本图像的目标区域预测分类结果与目标区域标注分类结果之间的损失值，该背景区域分类损失函数用于表示该样本图像的背景区域预测分类结果与背景区域标注分类结果之间的损失值，该协同损失函数用于分离该样本图像的目标区域特征图与该样本图像的背景区域特征图。
例如，上述目标区域标注分类结果可以是指目标对象所在区域的标注分类结果，目标对象可以是具有运动属性的对象，比如，目标对象可以是行人、动物以及车辆等任意一项。或者，上述目标对象还可以是其他对象。
在一种可能的实现方式中，该样本图像的目标区域预测分类结果是指获取该样本图像的摄像头的标识。
示例性地，当待处理图像中的目标区域为人像时，目标区域的分类结果可以是指行人的标识，其中，行人的标识可以是指行人的姓名、行人的编号或者行人的身份证号码；背景区域的分类结果可以是指获取到待处理图像的摄像头的标识，比如，摄像头的编号、摄像头所在的位置信息。
需要说明的是，上述为对行人的标识与摄像头的标识的举例说明，行人的标识用于区分不同的行为，对于同一摄像头可以近似看作其获取的背景区域图像不变，即可以通过摄像头的标识确定背景区域图像的分类结果，行人的标识与摄像头的标识的具体实现形式本申请不作任何限定。
在本申请中，通过协同损失函数可以使得目标神经网络提取的目标区域特征图与背景区域的特征图的分割准确性提高，即能够在获取目标区域的特征图时避免受到背景区域的影响。即通过目标区域分类损失函数、背景区域损失函数以及协同损失函数整体训练不断调整网络的参数，使得可以使得目标神经网络在对待处理图像进行分类处理时，使得提取的目标区域特征图的准确率和背景区域特征图的准确率相互促进协同提高。
在一种可能的实现方式中，通过上述协同损失函数可以使得样本图像的目标区域掩膜与背景区域掩膜互补。
例如，当掩膜的取值范围为[0,1]时，目标区域掩膜与背景区域掩膜的总和可以为1，即背景区域掩膜＝1-目标区域掩膜。其中，目标区域掩膜用于表示目标区域特征图中每个像素点所在的位置属于目标区域的概率值；背景区域掩膜用于表示背景区域特征图中每个像素点所在的位置属于背景区域的概率值。
在另一种可能的实现方式中，通过上述协同损失函数可以使得样本图像中的目标区域特征图对应的目标区域图像与样本图像中的背景区域特征图对应的背景区域图像的重叠区域近乎为零，即通过协同损失函数可以使得目标神经网络提取样本图像中目标区域特征图与背景区域特征图的划界明显，从而避免背景区域的噪声对目标区域特征图的影响。
结合第二方面，在第二方面的某些实现方式中，该协同损失函数是通过第一协同损失函数和第二协同损失函数加权处理得到的，其中，该第一协同损失函数是基于该样本图像的目标区域特征图与该样本图像的背景区域掩膜相乘得到的；第二协同损失函数是基于该样本图像的背景区域特征图与该样本图像的目标区域掩膜相乘得到的。
在一种可能的实现方式中，目标区域掩膜是通过提取的目标区域特征图训练得到的；背景区域掩膜是通过提取的背景区域特征图训练得到的。
上述目标区域掩膜等同于目标区域注意力图；上述背景区域掩膜等同于背景区域注意力图。
需要说明的是，在本申请中图像的目标区域特征图、背景区域特征图、目标区域掩膜以及背景区域掩膜中任意两个的尺度大小相同。
在本申请中，通过目标区域特征图与背景区域掩膜中对应的像素点相乘得到的第一损失函数可以用于表示提取的目标区域特征与背景区域的重叠部分，即目标区域特征图受背景区域噪声影响的范围大小；通过背景区域特征图与目标区域掩膜中对应的像素点相乘得到的第二损失函数可以用于表示提取的背景区域特征图与目标区域的重叠部分，即背景区域特征图受目前区域噪声影响的范围大小，通过第一协同损失函数与第二损失函数加权处理得到协同损失函数，使得训练目标神经网络时使得目标区域特征与背景区域特征进行分离，实现目标区域特征图的准确率和背景区域特征图的准确率相互促进协同提高。
结合第二方面，在第二方面的某些实现方式中，该神经网络包括第一分支和第二分支，该第一分支用于对该样本图像的目标区域进行处理，得到该样本图像的目标区域特征图和该样本图像的目标区域掩膜；该第二分支用于对该样本图像的背景区域进行处理，得到该样本图像的背景区域特征图和该样本图像的背景区域掩膜。
在本申请中，为了便于通过前景区域分类损失函数、背景区域分类损失函数以及协同损失函数训练神经网络得到用于图像分类的目标神经网络。目标神经网络中可以包括两个分支，第一分支可以用于对样本图像的目标区域进行处理，第二分支可以用于对样本图像的背景区域进行处理，通过协同损失函数可以协同第一分支与第二分支进行训练，使得提取的目标区域特征图的准确率和背景区域特征图的准确率相互促进协同提高。
在一种可能的实现方式中，根据神经网络提取的该样本图像的目标区域特征图，得到该样本图像的目标区域预测分类结果，包括：通过该目标神经网络的该第一分支对该样本图像的目标区域进行处理，得到该样本图像的目标区域增强特征图，其中，该目标区域增 强特征图是通过该样本图像的目标区域特征图和该样本图像的目标区域掩膜相乘得到的；基于该样本图像的目标区域增强特征图，得到该样本图像的目标区域的预测分类结果。
在一种可能的实现方式中，根据神经网络提取的该样本图像的背景区域特征图，得到该样本图像的背景区域预测分类结果，包括：通过该目标神经网络的该第二分支对该样本图像的背景区域进行处理，得到该样本图像的背景区域增强特征图，其中，背景区域增强特征图是通过该样本图像的背景区域特征图和该样本图像的背景区域掩膜相乘得到的；
基于该样本图像的背景区域增强特征图，得到该样本图像的背景区域的预测分类结果。
结合第二方面，在第二方面的某些实现方式中，该协同损失函数是通过以下等式得到的：
Figure PCTCN2020112694-appb-000003
其中，L t表示该协同损失函数，N表示该样本图像中像素点的数量，F i表示该样本图像的目标区域中第i个像素点，
Figure PCTCN2020112694-appb-000004
表示该样本图像的目标区域掩膜中第i个像素点的概率值，B i表示该样本图像的背景区域中第i个像素点，
Figure PCTCN2020112694-appb-000005
表示该样本图像的背景区域掩膜中第i个像素点的概率值，N为大于1的整数，i为大于或等于1且小于或等于N的整数。
在一种可能的实现方式中，整体损失函数包括上述分类损失函数、上述背景区域分类损失函数以及上述协同损失函数，整体损失函数是通过以下等式得到的：
Figure PCTCN2020112694-appb-000006
其中，L f可以表示目标区域分类损失函数，L b可以表示背景区域分类损失函数，L t可以表示协同损失函数。
第三方面，提供了一种图像分类方法，包括：获取待处理图像，待处理图像包括待识别的目标区域以及背景区域；通过目标神经网络对该待处理图像进行特征提取，得到该目标区域特征图以及该背景区域特征图；通过该目标神经网络对该目标区域特征图以及该背景区域特征图进行分类，得到该目标区域的分类结果以及该背景区域的分类结果；
其中，该目标神经网络是基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数训练得到的，该目标区域分类损失函数用于表示样本图像的目标区域特征图对应的预测分类结果与目标区域标注分类结果之间的损失值，背景区域分类损失函数用于表示该样本图像的背景区域特征图对应的预测分类结果与背景区域标注分类结果之间的损失值，该协同损失函数用于分离该样本图像的目标区域特征图与该样本图像的背景区域特征图。
在本申请中，目标神经网络是通过目标区域分类损失函数、背景区域损失函数以及协同损失函数训练得到的，使得预先训练的目标神经网络能够准确地分离待处理图像中目标区域特征图与背景区域的特征图。因此，目标神经网络在对待处理图像进行分类时，通过分离待处理图像中目标区域特征图以及背景区域特征图，不仅可以得到目标区域的分类结果以及背景区域的分类结果，同时在获取目标区域的特征图时还能避免受到背景区域的影响，提高待处理图像中目标区域分类结果的准确性。
结合第三方面，在第三方面的某些实现方式中，该目标神经网络包括第一分支与第二 分支，通过目标神经网络对该待处理图像进行特征提取，得到该目标区域特征图以及该背景区域特征图，包括：
通过该目标神经网络的该第一分支对该待处理图像的目标区域进行特征提取，得到该待处理图像的目标区域增强特征图，其中，该目标区域增强特征图是通过该待处理图像的目标区域特征图和该待处理图像的目标区域掩膜相乘得到的；
通过该目标神经网络的该第二分支对该待处理图像的背景区域进行特征提取，得到该待处理图像的背景区域增强特征图，其中，该背景区域增强特征图是通过该待处理图像的背景区域特征图和该待处理图像的背景区域掩膜相乘得到的。
结合第三方面，在第三方面的某些实现方式中，通过该目标神经网络对该目标区域特征图以及该背景区域特征图进行分类，得到该目标区域的分类结果以及该背景区域的分类结果，包括：
基于该待处理图像的目标区域增强特征图与该背景区域增强特征图进行分类，得到该待处理图像的目标区域的分类结果与背景区域的分类结果。
需要说明的是，上述目标神经网络中的第一分支与第二分支可以同时对待处理图像进行分类的，或者，目标神经网络中也可以是以先第一分支再第二分支的顺序对待处理图像进行分类，或者，目标神经网络中也可以是以先第二分支再第一分支的顺序对待处理图像进行分类。
结合第三方面，在第三方面的某些实现方式中，该待处理图像的背景区域的分类结果是指获取到该待处理图像的摄像头的标识。
结合第三方面，在第三方面的某些实现方式中，该协同损失函数是通过第一协同损失函数和第二协同损失函数加权处理得到的，其中，该第一协同损失函数是基于该样本图像的目标区域特征图与该样本图像的背景区域掩膜相乘得到的；该第二协同损失函数是基于该样本图像的背景区域特征图与该样本图像的目标区域掩膜相乘得到的。
结合第三方面，在第三方面的某些实现方式中，该目标神经网络包括第一分支和第二分支，其中，该第一分支用于对该样本图像的目标区域进行处理，得到该样本图像的目标区域特征图和该样本图像的目标区域掩膜；该第二分支用于对该样本图像的背景区域进行处理，得到该样本图像的背景区域特征图和该样本图像的背景区域掩膜。
结合第三方面，在第三方面的某些实现方式中，该协同损失函数是通过以下等式得到的：
Figure PCTCN2020112694-appb-000007
其中，L t表示所述协同损失函数，N表示所述样本图像中像素点的数量，F i表示所述样本图像的目标区域中第i个像素点，
Figure PCTCN2020112694-appb-000008
表示所述样本图像的目标区域掩膜中第i个像素点的概率值，B i表示所述样本图像的背景区域中第i个像素点，
Figure PCTCN2020112694-appb-000009
表示所述样本图像的背景区域掩膜中第i个像素点的概率值，N为大于1的整数，i为大于或等于1且小于或等于N的整数。
在一种可能的实现方式中，整体损失函数包括上述分类损失函数、上述背景区域分类损失函数以及上述协同损失函数，整体损失函数是通过以下等式得到的：
Figure PCTCN2020112694-appb-000010
其中，L f可以表示目标区域分类损失函数，L b可以表示背景区域分类损失函数，L t可以表示协同损失函数。
结合第三方面，在第三方面的某些实现方式中，该待处理图像的背景区域的分类结果是指获取到该待处理图像的摄像头的标识。
第四方面，提供了一种图像分类装置，包括：获取单元，用于获取待处理图像，该待处理图像包括待识别的目标区域以及背景区域；处理单元，用于根据目标神经网络对该待处理图像进行分类，得到该待处理图像的该目标区域的分类结果与该背景区域的分类结果；
其中，该目标神经网络是基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数训练得到的，该目标区域分类损失函数用于表示样本图像的目标区域特征图对应的预测分类结果与目标区域标注分类结果之间的损失值，该背景区域分类损失函数用于表示该样本图像的背景区域特征图对应的预测分类结果与背景区域标注分类结果之间的损失值，该协同损失函数用于分离该样本图像的目标区域特征图与该样本图像的背景区域特征图。
在一种可能的实现方式中，上述图像分类装置中包括功能单元/模块还用于执行第一方面以及第一方面中的任意一种实现方式中的方法。
应理解，在上述第一方面中对相关内容的扩展、限定、解释和说明也适用于第四方面中相同的内容。
第五方面，提供了一种图像分类模型的训练装置，包括：获取单元，用于获取训练数据，其中，该训练数据包括样本图像、该样本图像的目标区域标注分类结果、该样本图像的背景区域标注分类结果；处理单元，用于根据神经网络提取的该样本图像的目标区域特征图，得到该样本图像的目标区域预测分类结果；根据该神经网络提取的该样本图像的背景区域特征图，得到该样本图像的背景区域预测分类结果；基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数，确定该神经网络的参数；
其中，所述目标区域分类损失函数用于表示所述样本图像的目标区域预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函数用于表示所述样本图像的背景区域预测分类结果与背景区域标注分类结果之间的损失值，所述协同损失函数用于分离所述样本图像的目标区域特征图与所述样本图像的背景区域特征图。
在一种可能的实现方式中，上述训练装置中包括功能单元/模块还用于执行第二方面以及第二方面中的任意一种实现方式中的训练方法。
应理解，在上述第二方面中对相关内容的扩展、限定、解释和说明也适用于第五方面中相同的内容。
第六方面，提供了一种图像分类装置，包括：获取单元，用于获取待处理图像，所述待处理图像包括待识别的目标区域以及背景区域；处理单元，用于通过目标神经网络对所述待处理图像进行特征提取，得到所述目标区域的特征图以及所述背景区域的特征图；通过所述目标神经网络对所述目标区域的特征图以及所述背景区域的特征图进行分类，得到所述目标区域的分类结果以及所述背景区域的分类结果；
其中，所述目标神经网络是基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数训练得到的，所述目标区域分类损失函数用于表示样本图像的目标区域特征图对应的预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函数用于表示所述样本图像的背景区域特征图对应的预测分类结果与背景区域标注分类结果之间的损失值，所述协同损失函数用于分离所述样本图像的目标区域特征图与所述样本图像的背景区域特征图。
在一种可能的实现方式中，上述训练装置中包括功能单元/模块还用于执行第三方面以及第三方面中的任意一种实现方式中的方法。
应理解，在上述第三方面中对相关内容的扩展、限定、解释和说明也适用于第六方面中相同的内容。
第七方面，提供了一种图像分类装置，该装置包括：存储器，用于存储程序；处理器，用于执行该存储器存储的程序，当该存储器存储的程序被执行时，该处理器用于执行：获取待处理图像，所述待处理图像包括待识别的目标区域以及背景区域；根据目标神经网络对所述待处理图像进行分类，得到所述待处理图像的所述目标区域的分类结果与所述背景区域的分类结果；
其中，所述目标神经网络是基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数训练得到的，所述目标区域分类损失函数用于表示样本图像的目标区域特征图对应的预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函数用于表示所述样本图像的背景区域特征图对应的预测分类结果与背景区域标注分类结果之间的损失值，所述协同损失函数用于分离所述样本图像的目标区域特征图与所述样本图像的背景区域特征图。
在一种可能的实现方式中，上述图像分类装置中包括的处理器还用于执行第一方面中的任意一种实现方式中方法。
应理解，在上述第一方面中对相关内容的扩展、限定、解释和说明也适用于第七方面中相同的内容。
第八方面，提供了一种图像分类模型的训练装置，该装置包括：存储器，用于存储程序；处理器，用于执行该存储器存储的程序，当该存储器存储的程序被执行时，该处理器用于执行：获取训练数据，其中，所述训练数据包括样本图像、所述样本图像的目标区域标注分类结果、所述样本图像的背景区域标注分类结果；根据神经网络提取的所述样本图像的目标区域特征图，得到所述样本图像的目标区域预测分类结果；根据所述神经网络提取的所述样本图像的背景区域特征图，得到所述样本图像的背景区域预测分类结果；基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数，确定所述神经网络的参数；
其中，所述目标区域分类损失函数用于表示所述样本图像的目标区域预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函数用于表示所述样本图像的背景区域预测分类结果与背景区域标注分类结果之间的损失值，所述协同损失函数用于分离所述样本图像的目标区域特征图与所述样本图像的背景区域特征图。
在一种可能的实现方式中，上述训练装置中包括的处理器还用于执行第二方面中的任意一种实现方式中的训练方法。
应理解，在上述第二方面中对相关内容的扩展、限定、解释和说明也适用于第八方面中相同的内容。
第九方面，提供了一种图像分类装置，该装置包括：存储器，用于存储程序；处理器，用于执行该存储器存储的程序，当该存储器存储的程序被执行时，该处理器用于执行：获取待处理图像，该待处理图像包括待识别的目标区域以及背景区域；通过目标神经网络对该待处理图像进行特征提取，得到该目标区域的特征图以及该背景区域的特征图；通过该目标神经网络对该目标区域的特征图以及该背景区域的特征图进行分类，得到该目标区域的分类结果以及该背景区域的分类结果，
其中，该目标神经网络是基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数训练得到的，该目标区域分类损失函数用于表示样本图像的目标区域特征图对应的预测分类结果与目标区域标注分类结果之间的损失值，该背景区域分类损失函数用于表示该样本图像的背景区域特征图对应的预测分类结果与背景区域标注分类结果之间的损失值，该协同损失函数用于分离该样本图像的目标区域特征图与该样本图像的背景区域特征图。
在一种可能的实现方式中，上述图像分类装置中包括的处理器还用于执行第三方面中的任意一种实现方式中方法。
应理解，在上述第三方面中对相关内容的扩展、限定、解释和说明也适用于第九方面中相同的内容。
第十方面，提供了一种计算机可读介质，该计算机可读介质存储用于设备执行的程序代码，该程序代码包括用于执行上述第一方面至第三方面以及第一方面至第三方面中的任意一种实现方式中的方法。
第十一方面，提供了一种包含指令的计算机程序产品，当该计算机程序产品在计算机上运行时，使得计算机执行上述第一方面至第三方面以及第一方面至第三方面中的任意一种实现方式中的方法。
第十二方面，提供了一种芯片，所述芯片包括处理器与数据接口，所述处理器通过所述数据接口读取存储器上存储的指令，执行上述第一方面至第三方面以及第一方面至第三方面中的任意一种实现方式中的方法。
可选地，作为一种实现方式，所述芯片还可以包括存储器，所述存储器中存储有指令，所述处理器用于执行所述存储器上存储的指令，当所述指令被执行时，所述处理器用于执行上述第一方面至第三方面以及第一方面至第三方面中的任意一种实现方式中的方法。
附图说明
图1是本申请实施例提供的一种人工智能主体框架示意图；
图2是本申请实施例提供的一种应用场景的示意图；
图3是本申请实施例提供的一种应用场景的示意图；
图4是本申请实施例提供的一种应用场景的示意图；
图5是本申请实施例提供的系统架构的结构示意图；
图6是本申请实施例提供的一种卷积神经网络结构示意图；
图7是本申请实施例提供的一种芯片硬件结构示意图；
图8是本申请实施例提供了一种系统架构的示意图；
图9是本申请实施例提供的图像分类方法的示意性流程图；
图10是本申请实施例提供的图像分类模型的训练方法的示意性流程图；
图11是本申请实施例提供的图像分类模型的训练方法的示意性流程图；
图12是本申请实施例提供的图像分类模型的训练方法的示意图；
图13是本申请实施例提供的图像分类装置的示意性框图；
图14是本申请实施例提供的图像分类模型的训练装置的示意性框图；
图15是本申请实施例提供的图像分类装置的示意性框图；
图16是本申请实施例提供的图像分类模型的训练装置的示意性框图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
图1示出一种人工智能主体框架示意图，该主体框架描述了人工智能系统总体工作流程，适用于通用的人工智能领域需求。
下面从“智能信息链”(水平轴)和“信息技术(information technology，IT)价值链”(垂直轴)两个维度对上述人工智能主题框架100进行详细的阐述。
“智能信息链”反映从数据的获取到处理的一列过程。举例来说，可以是智能信息感知、智能信息表示与形成、智能推理、智能决策、智能执行与输出的一般过程。在这个过程中，数据经历了“数据—信息—知识—智慧”的凝练过程。
“IT价值链”从人智能的底层基础设施、信息(提供和处理技术实现)到系统的产业生态过程，反映人工智能为信息技术产业带来的价值。
(1)基础设施110
基础设施为人工智能系统提供计算能力支持，实现与外部世界的沟通，并通过基础平台实现支撑。
基础设施可以通过传感器与外部沟通，基础设施的计算能力可以由智能芯片提供。
这里的智能芯片可以是中央处理器(central processing unit，CPU)、神经网络处理器(neural-network processing unit，NPU)、图形处理器(graphics processing unit，GPU)、专门应用的集成电路(application specific integrated circuit，ASIC)以及现场可编程门阵列(field programmable gate array，FPGA)等硬件加速芯片。
基础设施的基础平台可以包括分布式计算框架及网络等相关的平台保障和支持，可以包括云存储和计算、互联互通网络等。
例如，对于基础设施来说，可以通过传感器和外部沟通获取数据，然后将这些数据提供给基础平台提供的分布式计算系统中的智能芯片进行计算。
(2)数据120
基础设施的上一层的数据用于表示人工智能领域的数据来源。该数据涉及到图形、图像、语音、文本，还涉及到传统设备的物联网数据，包括已有系统的业务数据以及力、位 移、液位、温度、湿度等感知数据。
(3)数据处理130
上述数据处理通常包括数据训练，机器学习，深度学习，搜索，推理，决策等处理方式。
其中，机器学习和深度学习可以对数据进行符号化和形式化的智能信息建模、抽取、预处理、训练等。
推理是指在计算机或智能系统中，模拟人类的智能推理方式，依据推理控制策略，利用形式化的信息进行机器思维和求解问题的过程，典型的功能是搜索与匹配。
决策是指智能信息经过推理后进行决策的过程，通常提供分类、排序、预测等功能。
(4)通用能力140
对数据经过上面提到的数据处理后，进一步基于数据处理的结果可以形成一些通用的能力，比如可以是算法或者一个通用系统，例如，翻译，文本的分析，计算机视觉的处理，语音识别，图像的识别等等。
(5)智能产品及行业应用150
智能产品及行业应用指人工智能系统在各领域的产品和应用，是对人工智能整体解决方案的封装，将智能信息决策产品化、实现落地应用，其应用领域主要包括：智能制造、智能交通、智能家居、智能医疗、智能安防、自动驾驶，平安城市，智能终端等。
应用场景一：智能安防
在一个实施例中，如图2所示，本申请实施例的图像分类方法可以应用于安防领域。例如，公共场合的监控设备采集到的图片(或者，视频)，通过目标神经网络可以对采集到的图片进行分类处理识别图片中的行人的信息，为案件侦破提供重要的线索信息。
例如，可以通过某个特定人员的照片，通过监控视频获得其行动轨迹。行人重识别技术根据特定人员的照片，去监控视频库里去匹配特定人员出现的视频段。这样可以把特定人员在各个摄像头的轨迹串连起来，这个轨迹一旦串连起来之后，对警察的破案刑侦有非常大的帮助。具体地，可以根据提供的特定人员照片，实现给定一个监控行人图像，检索跨设备下的该行人的图像并形成该行人的行为轨迹，如图2所示，可以通过本申请中的目标神经网络检索到背景1至背景4中该特定人员的行为轨迹，其中，背景1至背景4可以是通过不同的摄像头获取的图像。
示例性地，在本申请的图像分类方法应用在安防领域时，获取特定人员图像(例如，嫌疑人员的图片或者照片)，该特定人员图像中包括特定人像区域与街景区域；根据预先训练的目标神经网络对特定人员图像进行分类，得到特定人像的分类结果与街景区域的分类结果。
例如，通过预先训练的目标神经网络可以提取特定人员图像中人像特征图与街景特征图，基于人像特征图与街景特征图进行分类，比如，将人像特征图进行分类得到特定人员的标识，将街景特征图进行分类得到获取到该街景的摄像头的标识。通过摄像头的标识可以确定街景的位置进而能够确定特定人员的行为轨迹。例如，当本申请的图像分类方法应用在安防领域时，用于训练目标神经网络的训练数据可以包括公安系统中包含不同人员的样本图像、人员的标识、获取到样本图像的各个摄像头的标识。
应理解，对于同一摄像头而言，其获取的拍摄画面的背景区域的图像可以看作是近似 相同的，即通过摄像头ID可以区分不同的背景区域的图像。
需要说明的是，本申请实施例提供的应用于智能安防领域的目标神经网络同样适用于后面图5至图12中相关实施例中对目标神经网络相关内容的扩展、限定、解释和说明，此处不再赘述。
应用场景二：智能寻人系统
在一个实施例中，如图3所示，通过向智能寻人系统中输入一张行人照片(例如，走散儿童的照片)，通过本申请实施例的目标神经网络可以实现实时的在所有监控摄像头寻找与该儿童照片匹配的图像，从而实现在人流密度比较大的公共场所，进行行人识别技术。
示例性地，在本申请的图像分类方法应用在智能寻人系统时，获取特定人员图像(例如，走失人员的图片或者照片)，该特定人员图像中包括特定人像区域与街景区域；根据预先训练的目标神经网络对特定人员图像进行分类，得到特定人像的分类结果与街景区域的分类结果。
例如，通过预先训练的目标神经网络可以提取特定人员图像中人像特征图与街景特征图，基于人像特征图与街景特征图进行分类，比如，通过人像特征图进行分类得到人像的标识，将街景特征图进行分类得到获取到该街景的摄像头的标识，通过摄像头的标识可以确定街景的位置进而能够确定走失人员的行为轨迹。
例如，当本申请的图像分类方法应用在智能寻人系统时，用于训练目标神经网络的训练数据可以包括的包含走失人员的样本图像、走失人员的标识、获取到样本图像的各个摄像头的标识。
应理解，对于同一摄像头而言，其获取的拍摄画面的背景区域的图像可以看作是近似相同的，即通过摄像头ID可以区分不同的背景区域的图像。
需要说明的是，本申请实施例提供的应用智能寻人系统的目标神经网络同样适用于后面图5至图12中相关实施例中对目标神经网络相关内容的扩展、限定、解释和说明，此处不再赘述。
在一种可能的实现方式中，智能寻人系统可以是指在人流拥挤的公共区域，比如在机场、车站、游乐场等公共区域寻找特定人员，通过本申请实施例的目标神经网络可以通过调用监控视频库实现走失儿童的快速查找和轨迹构建。
在一种可能的实现方式中，智能寻人系统也可以是应用在智能家庭机器人中，在智能家庭机器人中通过目标神经网络实现行人重识别技术，从而使得帮助智能家庭机器人更好的识别家庭成员。
在一种可能的实现方式中，智能寻人系统也可以是应用在无人超市中。例如，通过本申请实施例的目标神经网络可以根据行人外观的照片，实时动态跟踪用户轨迹，把轨迹转化成管理员能够理解的信息，从而能够帮助经营者获取有效的顾客轨迹，进而深入挖掘商业价值。比如，可以通过行为轨迹了解用户的兴趣，以便优化用户体验。
应用场景三：相册分类
在一个实施例中，如图4所示，可以利用本申请实施例的目标神经网络实现对图片进行分类，从而为不同的类别(例如，基于不同用户的图片分类)的图片打上标签，便于用户查看和查找。另外，这些图片的分类标签也可以提供给相册管理系统进行分类管理，节省用户的管理时间，提高相册管理的效率，提升用户体验。
例如，通过本申请实施例目标神经网络对于不同用户的图像可以进行分类打标签。比如，当本申请实施例的目标神经网络应用在智能终端中，可以对于智能终端中的相册实现图片按人分类，即对于包括用户A的图片可以标记为用户A的标签；对于包括用户B的图片可以标记用户B的标签，实现相册基于不同用户的分类。
示例性地，在本申请的图像分类方法应用在相册分类时，获取待处理图像(例如，用户A的图片或者照片)，待处理图像中包括人像区域与街景区域；根据预先训练的目标神经网络对人像进行分类，得到人像的分类结果与街景区域的分类结果。
例如，通过预先训练的目标神经网络可以提取待处理图像中用户A的人像特征图与街景特征图，基于人像特征图与街景特征图进行分类，比如，通过人像特征图进行分类得到人像的标识，将街景特征图进行分类得到获取到该街景的摄像头的标识，进一步可以将人像的标识相同的图像存入同一相册文件夹中，从而实现不同用户图片的相册分类。
例如，当本申请的图像分类方法应用在相册分类时，用于训练目标神经网络的训练数据可以是不同用户的样本图像、不同用户的标识以及获取到样本图像的摄像头的标识。
应理解，对于同一摄像头而言，其获取的拍摄画面的背景区域的图像可以看作是近似相同的，即通过摄像头ID可以区分不同的背景区域的图像。
需要说明的是，本申请实施例提供的应用相册分类的目标神经网络同样适用于后面图5至图12中相关实施例中对目标神经网络相关内容的扩展、限定、解释和说明，此处不再赘述。
应理解，上述为对应用场景的举例说明，并不对本申请的应用场景作任何限定。
由于本申请实施例涉及大量神经网络的应用，为了便于理解，下面先对本申请实施例可能涉及的神经网络的相关术语和概念进行介绍。
(1)神经网络
神经网络可以是由神经单元组成的，神经单元可以是指以x s和截距1为输入的运算单元，该运算单元的输出可以为：
Figure PCTCN2020112694-appb-000011
其中，s＝1、2、……n，n为大于1的自然数，W s为x s的权重，b为神经单元的偏置。f为神经单元的激活函数(activation functions)，用于将非线性特性引入神经网络中，来将神经单元中的输入信号转换为输出信号。该激活函数的输出信号可以作为下一层卷积层的输入，激活函数可以是sigmoid函数。神经网络是将多个上述单一的神经单元联结在一起形成的网络，即一个神经单元的输出可以是另一个神经单元的输入。每个神经单元的输入可以与前一层的局部接受域相连，来提取局部接受域的特征，局部接受域可以是由若干个神经单元组成的区域。
(2)深度神经网络
深度神经网络(deep neural network，DNN)，也称多层神经网络，可以理解为具有多层隐含层的神经网络。按照不同层的位置对DNN进行划分，DNN内部的神经网络可以分为三类：输入层，隐含层，输出层。一般来说第一层是输入层，最后一层是输出层，中间的层数都是隐含层。层与层之间是全连接的，也就是说，第i层的任意一个神经元一定与第i+1层的任意一个神经元相连。
虽然DNN看起来很复杂，但是就每一层的工作来说，其实并不复杂，简单来说就是 如下线性关系表达式：
Figure PCTCN2020112694-appb-000012
其中，
Figure PCTCN2020112694-appb-000013
是输入向量，
Figure PCTCN2020112694-appb-000014
是输出向量，
Figure PCTCN2020112694-appb-000015
是偏移向量，W是权重矩阵(也称系数)，α()是激活函数。每一层仅仅是对输入向量
Figure PCTCN2020112694-appb-000016
经过如此简单的操作得到输出向量
Figure PCTCN2020112694-appb-000017
由于DNN层数多，系数W和偏移向量
Figure PCTCN2020112694-appb-000018
的数量也比较多。这些参数在DNN中的定义如下所述：以系数W为例：假设在一个三层的DNN中，第二层的第4个神经元到第三层的第2个神经元的线性系数定义为
Figure PCTCN2020112694-appb-000019
上标3代表系数W所在的层数，而下标对应的是输出的第三层索引2和输入的第二层索引4。
综上，第L-1层的第k个神经元到第L层的第j个神经元的系数定义为
Figure PCTCN2020112694-appb-000020
需要注意的是，输入层是没有W参数的。在深度神经网络中，更多的隐含层让网络更能够刻画现实世界中的复杂情形。理论上而言，参数越多的模型复杂度越高，“容量”也就越大，也就意味着它能完成更复杂的学习任务。训练深度神经网络的也就是学习权重矩阵的过程，其最终目的是得到训练好的深度神经网络的所有层的权重矩阵(由很多层的向量W形成的权重矩阵)。
(3)卷积神经网络
卷积神经网络(convolutional neuron network，CNN)是一种带有卷积结构的深度神经网络。卷积神经网络包含了一个由卷积层和子采样层构成的特征抽取器，该特征抽取器可以看作是滤波器。卷积层是指卷积神经网络中对输入信号进行卷积处理的神经元层。在卷积神经网络的卷积层中，一个神经元可以只与部分邻层神经元连接。一个卷积层中，通常包含若干个特征平面，每个特征平面可以由一些矩形排列的神经单元组成。同一特征平面的神经单元共享权重，这里共享的权重就是卷积核。共享权重可以理解为提取图像信息的方式与位置无关。卷积核可以以随机大小的矩阵的形式初始化，在卷积神经网络的训练过程中卷积核可以通过学习得到合理的权重。另外，共享权重带来的直接好处是减少卷积神经网络各层之间的连接，同时又降低了过拟合的风险。
(4)损失函数
在训练深度神经网络的过程中，因为希望深度神经网络的输出尽可能的接近真正想要预测的值，所以可以通过比较当前网络的预测值和真正想要的目标值，再根据两者之间的差异情况来更新每一层神经网络的权重向量(当然，在第一次更新之前通常会有初始化的过程，即为深度神经网络中的各层预先配置参数)，比如，如果网络的预测值高了，就调整权重向量让它预测低一些，不断地调整，直到深度神经网络能够预测出真正想要的目标值或与真正想要的目标值非常接近的值。因此，就需要预先定义“如何比较预测值和目标值之间的差异”，这便是损失函数(loss function)或目标函数(objective function)，它们是用于衡量预测值和目标值的差异的重要方程。其中，以损失函数举例，损失函数的输出值(loss)越高表示差异越大，那么深度神经网络的训练就变成了尽可能缩小这个loss的过程。
(5)反向传播算法
神经网络可以采用误差反向传播(back propagation，BP)算法在训练过程中修正初始的神经网络模型中参数的大小，使得神经网络模型的重建误差损失越来越小。具体地，前向传递输入信号直至输出会产生误差损失，通过反向传播误差损失信息来更新初始的神经网络模型中参数，从而使误差损失收敛。反向传播算法是以误差损失为主导的反向传播运动，旨在得到最优的神经网络模型的参数，例如权重矩阵。
图5示出了本申请实施例提供的一种系统架构200。
在图5中，数据采集设备260用于采集训练数据。针对本申请实施例的图像分类方法来说，当可以通过训练图像对用于图像分类的神经网络进行进一步训练，即数据采集设备260采集的训练数据可以是训练图像。
例如，在本申请实施例中训练目标神经网络的训练数据可以包括样本图像、样本图像中包括的行人标识以及样本图像中包括的背景标识，其中，例如行人标识可以是行人的ID、行人的身份证信息等；背景标识可以是待处理图像对应的摄像头的标识信息，比如，可以是获取待处理图像的摄像头的ID信息。
应理解，对于同一摄像头而言，其获取的拍摄画面的背景区域的信息可以是不变的，即通过摄像头ID可以区分不同的背景区域的信息。
在采集到训练数据之后，数据采集设备260将这些训练数据存入数据库230，训练设备220基于数据库230中维护的训练数据训练得到目标模型/规则201。
下面对训练设备220基于训练数据得到目标模型/规则201进行描述，训练设备220对输入的原始图像进行处理，将输出的图像与原始图像进行对比，直到训练设备220输出的图像与原始图像的差值小于一定的阈值，从而完成目标模型/规则201的训练。
例如，在本申请提供的用于图像分类的目标神经网络可以是通过样本图像的预测分类结果与样本分类结果之间的分类损失函数进行训练得到的，训练后的神经网络使得将样本图像输入至目标神经网络得到的预测分类结果与样本分类结果的差值小于一定的阈值，从而完成神经网络的训练得到目标神经网络的参数。
上述目标模型/规则201能够用于实现本申请实施例的图像分类方法。本申请实施例中的目标模型/规则201具体可以为神经网络。
需要说明的是，在实际的应用中，所述数据库230中维护的训练数据不一定都来自于数据采集设备260的采集，也有可能是从其他设备接收得到的。另外需要说明的是，训练设备220也不一定完全基于数据库230维护的训练数据进行目标模型/规则201的训练，也有可能从云端或其他地方获取训练数据进行模型训练，上述描述不应该作为对本申请实施例的限定。
根据训练设备220训练得到的目标模型/规则201可以应用于不同的系统或设备中，如应用于图5所示的执行设备210，所述执行设备210可以是终端，如手机终端，平板电脑，笔记本电脑，增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)，车载终端等，还可以是服务器，或者，云端等。在图5中，执行设备210配置输入/输出(input/output，I/O)接口212，用于与外部设备进行数据交互，用户可以通过客户设备240向I/O接口212输入数据，所述输入数据在本申请实施例中可以包括：客户设备输入的待处理图像。
预处理模块213和预处理模块214用于根据I/O接口212接收到的输入数据(如待处理图像)进行预处理，在本申请实施例中，也可以没有预处理模块213和预处理模块214(也可以只有其中的一个预处理模块)，而直接采用计算模块211对输入数据进行处理。
在执行设备210对输入数据进行预处理，或者在执行设备210的计算模块211执行计算等相关的处理过程中，执行设备210可以调用数据存储系统250中的数据、代码等以用于相应的处理，也可以将相应处理得到的数据、指令等存入数据存储系统250中。
最后，I/O接口212将处理结果，如上述得到待处理图像中待识别的目标区域的分类结果与背景区域的分类结果返回给客户设备240，从而提供给用户。
值得说明的是，训练设备220可以针对不同的目标或称不同的任务，基于不同的训练数据生成相应的目标模型/规则201，该相应的目标模型/规则201即可以用于实现上述目标或完成上述任务，从而为用户提供所需的结果。
在图5中所示情况下，在一种情况下，用户可以手动给定输入数据，该手动给定可以通过I/O接口212提供的界面进行操作。
另一种情况下，客户设备240可以自动地向I/O接口212发送输入数据，如果要求客户设备240自动发送输入数据需要获得用户的授权，则用户可以在客户设备240中设置相应权限。用户可以在客户设备240查看执行设备210输出的结果，具体的呈现形式可以是显示、声音、动作等具体方式。客户设备240也可以作为数据采集端，采集如图所示输入I/O接口212的输入数据及输出I/O接口212的输出结果作为新的样本数据，并存入数据库230。当然，也可以不经过客户设备240进行采集，而是由I/O接口212直接将如图所示输入I/O接口212的输入数据及输出I/O接口212的输出结果，作为新的样本数据存入数据库230。
值得注意的是，图5仅是本申请实施例提供的一种系统架构的示意图，图中所示设备、器件、模块等之间的位置关系不构成任何限制，例如，在图5中，数据存储系统250相对执行设备210是外部存储器，在其它情况下，也可以将数据存储系统250置于执行设备210中。
如图5所示，根据训练设备220训练得到目标模型/规则201，该目标模型/规则201在本申请实施例中可以是本申请中用于图像分类的目标神经网络，具体的，本申请实施例提供的目标神经网络可以是卷积神经网络(convolutional neuron network，CNN)，或者，可以是深度卷积神经网络(deep convolutional neural networks，DCNN)等。
下面结合图6重点对卷积神经网络的结构进行详细的介绍。如上文的基础概念介绍所述，卷积神经网络是一种带有卷积结构的深度神经网络，是一种深度学习(deep learning)架构，深度学习架构是指通过机器学习的算法，在不同的抽象层级上进行多个层次的学习。作为一种深度学习架构，卷积神经网络是一种前馈(feed-forward)人工神经网络，该前馈人工神经网络中的各个神经元可以对输入其中的图像作出响应。
本申请实施例中用于图像分类的神经网络的结构可以如图6所示。在图6中，卷积神经网络300可以包括输入层310，卷积层/池化层320(其中，池化层为可选的)，以及神经网络层330。其中，输入层310可以获取待处理图像，并将获取到的待处理图像交由卷积层/池化层320以及后面的神经网络层330进行处理，可以得到图像的处理结果。下面对图6中的CNN 300中内部的层结构进行详细的介绍。
卷积层/池化层320：
如图6所示卷积层/池化层320可以包括如示例321-326层，举例来说：在一种实现中，321层为卷积层，322层为池化层，323层为卷积层，324层为池化层，325为卷积层，326为池化层；在另一种实现方式中，321、322为卷积层，323为池化层，324、325为卷积层，326为池化层，即卷积层的输出可以作为随后的池化层的输入，也可以作为另一个卷积层的输入以继续进行卷积操作。
下面将以卷积层321为例，介绍一层卷积层的内部工作原理。
卷积层321可以包括很多个卷积算子，卷积算子也称为核，其在图像处理中的作用相当于一个从输入图像矩阵中提取特定信息的过滤器，卷积算子本质上可以是一个权重矩阵，这个权重矩阵通常被预先定义，在对图像进行卷积操作的过程中，权重矩阵通常在输入图像上沿着水平方向一个像素接着一个像素(或两个像素接着两个像素等，这取决于步长stride的取值)的进行处理，从而完成从图像中提取特定特征的工作。该权重矩阵的大小应该与图像的大小相关，需要注意的是，权重矩阵的纵深维度(depth dimension)和输入图像的纵深维度是相同的，在进行卷积运算的过程中，权重矩阵会延伸到输入图像的整个深度。因此，和一个单一的权重矩阵进行卷积会产生一个单一纵深维度的卷积化输出，但是大多数情况下不使用单一权重矩阵，而是应用多个尺寸(行×列)相同的权重矩阵，即多个同型矩阵。每个权重矩阵的输出被堆叠起来形成卷积图像的纵深维度，这里的维度可以理解为由上面所述的“多个”来决定。
不同的权重矩阵可以用来提取图像中不同的特征，例如，一个权重矩阵用来提取图像边缘信息，另一个权重矩阵用来提取图像的特定颜色，又一个权重矩阵用来对图像中不需要的噪点进行模糊化等。该多个权重矩阵尺寸(行×列)相同，经过该多个尺寸相同的权重矩阵提取后的卷积特征图的尺寸也相同，再将提取到的多个尺寸相同的卷积特征图合并形成卷积运算的输出。
这些权重矩阵中的权重值在实际应用中需要经过大量的训练得到，通过训练得到的权重值形成的各个权重矩阵可以用来从输入图像中提取信息，从而使得卷积神经网络300进行正确的预测。
当卷积神经网络300有多个卷积层的时候，初始的卷积层(例如321)往往提取较多的一般特征，一般特征也可以称之为低级别的特征；随着卷积神经网络300深度的加深，越往后的卷积层(例如326)提取到的特征越来越复杂，比如，高级别的语义之类的特征，语义越高的特征越适用于待解决的问题。
池化层：
由于常常需要减少训练参数的数量，因此卷积层之后常常需要周期性的引入池化层，在如图6中320所示例的321-326各层，可以是一层卷积层后面跟一层池化层，也可以是多层卷积层后面接一层或多层池化层。在图像处理过程中，池化层的目的就是减少图像的空间大小。池化层可以包括平均池化算子和/或最大池化算子，以用于对输入图像进行采样得到较小尺寸的图像。平均池化算子可以在特定范围内对图像中的像素值进行计算产生平均值作为平均池化的结果。最大池化算子可以在特定范围内取该范围内值最大的像素作为最大池化的结果。另外，就像卷积层中用权重矩阵的大小应该与图像尺寸相关一样，池化层中的运算符也应该与图像的大小相关。通过池化层处理后输出的图像尺寸可以小于输入池化层的图像的尺寸，池化层输出的图像中每个像素点表示输入池化层的图像的对应子区域的平均值或最大值。
神经网络层330：
在经过卷积层/池化层320的处理后，卷积神经网络300还不足以输出所需要的输出信息。因为如前所述，卷积层/池化层320只会提取特征，并减少输入图像带来的参数。然而为了生成最终的输出信息(所需要的类信息或其他相关信息)，卷积神经网络300需 要利用神经网络层330来生成一个或者一组所需要的类的数量的输出。因此，在神经网络层330中可以包括多层隐含层(如图6所示的331、332至33n)以及输出层340，该多层隐含层中所包含的参数可以根据具体的任务类型的相关训练数据进行预先训练得到，例如该任务类型可以包括图像识别，图像分类，图像检测以及图像超分辨率重建等等。
在神经网络层330中的多层隐含层之后，也就是整个卷积神经网络300的最后层为输出层340，该输出层340具有类似分类交叉熵的损失函数，具体用于计算预测误差，一旦整个卷积神经网络300的前向传播(如图6由310至340方向的传播为前向传播)完成，反向传播(如图6由340至310方向的传播为反向传播)就会开始更新前面提到的各层的权重值以及偏差，以减少卷积神经网络300的损失，及卷积神经网络300通过输出层输出的结果和理想结果之间的误差。
需要说明的是，图6所示的卷积神经网络仅作为一种本申请实施例的目标神经网络的结构示例，在具体的应用中，本申请实施例的图像分类方法所采用的卷积神经网络还可以以其他网络模型的形式存在。
图7为本申请实施例提供的一种芯片的硬件结构，该芯片包括神经网络处理器400(neural-network processing unit，NPU)。该芯片可以被设置在如图4所示的执行设备210中，用以完成计算模块211的计算工作。该芯片也可以被设置在如图4所示的训练设备220中，用以完成训练设备220的训练工作并输出目标模型/规则201。如图6所示的卷积神经网络中各层的算法均可在如图7所示的芯片中得以实现。
NPU 400作为协处理器挂载到主中央处理器(central processing unit，CPU)上，由主CPU分配任务。NPU 400的核心部分为运算电路403，控制器404控制运算电路403提取存储器(权重存储器或输入存储器)中的数据并进行运算。
在一些实现中，运算电路403内部包括多个处理单元(process engine,PE)。在一些实现中，运算电路403是二维脉动阵列。运算电路403还可以是一维脉动阵列或者能够执行例如乘法和加法这样的数学运算的其它电子线路。在一些实现中，运算电路403是通用的矩阵处理器。
举例来说，假设有输入矩阵A，权重矩阵B，输出矩阵C。运算电路403从权重存储器402中取矩阵B相应的数据，并缓存在运算电路403中每一个PE上。运算电路403从输入存储器401中取矩阵A数据与矩阵B进行矩阵运算，得到的矩阵的部分结果或最终结果，保存在累加器408(accumulator)中。
向量计算单元407可以对运算电路403的输出做进一步处理，如向量乘，向量加，指数运算，对数运算，大小比较等等。例如，向量计算单元407可以用于神经网络中非卷积/非FC层的网络计算，如池化(pooling)，批归一化(batch normalization)，局部响应归一化(local response normalization)等。
在一些实现种，向量计算单元能407将经处理的输出的向量存储到统一存储器406。例如，向量计算单元407可以将非线性函数应用到运算电路403的输出，例如累加值的向量，用以生成激活值。在一些实现中，向量计算单元407生成归一化的值、合并值，或二者均有。
在一些实现中，处理过的输出的向量能够用作到运算电路403的激活输入，例如用于在神经网络中的后续层中的使用。
统一存储器406用于存放输入数据以及输出数据。
权重数据直接通过存储单元访问控制器405(direct memory access controller，DMAC)将外部存储器中的输入数据存入到输入存储器401和/或统一存储器406、将外部存储器中的权重数据存入权重存储器402，以及将统一存储器406中的数据存入外部存储器。
总线接口单元410(bus interface unit，BIU)，用于通过总线实现主CPU、DMAC和取指存储器409之间进行交互。
与控制器404连接的取指存储器409(instruction fetch buffer)，用于存储控制器404使用的指令。
控制器404，用于调用取指存储器409中缓存的指令，实现控制该运算加速器的工作过程。
一般地，统一存储器406，输入存储器401，权重存储器402以及取指存储器409均为片上(On-Chip)存储器，外部存储器为该NPU外部的存储器，该外部存储器可以为双倍数据率同步动态随机存储器(double data rate synchronous dynamic random access memory，DDR SDRAM)、高带宽存储器(high bandwidth memory，HBM)或其他可读可写的存储器。
其中，图6所示的卷积神经网络中各层的运算可以由运算电路403或向量计算单元407执行。
上文中介绍的图4中的执行设备210能够执行本申请实施例的图像分类方法的各个步骤，图6所示的CNN模型和图7所示的芯片也可以用于执行本申请实施例的图像分类方法的各个步骤。
图8所示是本申请实施例提供了一种系统架构500。该系统架构包括本地设备520、本地设备530以及执行设备510和数据存储系统550，其中，本地设备520和本地设备530通过通信网络与执行设备510连接。
执行设备510可以由一个或多个服务器实现。可选的，执行设备510可以与其它计算设备配合使用，例如：数据存储器、路由器、负载均衡器等设备。执行设备510可以布置在一个物理站点上，或者分布在多个物理站点上。执行设备510可以使用数据存储系统550中的数据，或者调用数据存储系统550中的程序代码来实现本申请实施例的图像分类方法。
需要说明的是，上述执行设备510也可以称为云端设备，此时执行设备510可以部署在云端。
用户可以操作各自的用户设备(例如，本地设备520和本地设备530)与执行设备510进行交互。每个本地设备可以表示任何计算设备，例如，个人计算机、计算机工作站、智能手机、平板电脑、智能摄像头、智能汽车或其他类型蜂窝电话、媒体消费设备、可穿戴设备、机顶盒、游戏机等。
每个用户的本地设备可以通过任何通信机制/通信标准的通信网络与执行设备510进行交互，通信网络可以是广域网、局域网、点对点连接等方式，或它们的任意组合。
在一种实现方式中，本地设备520、本地设备530可以从执行设备510获取到目标神经网络的相关参数，将目标神经网络部署在本地设备520、本地设备530上，利用该目标 神经网络进行图像分类处理等。
在另一种实现中，执行设备510上可以直接部署目标神经网络，执行设备510通过从本地设备520和本地设备530获取待处理图像，并根据目标神经网络对待处理图像进行图像分类处理等。
例如，上述目标神经网络可以是本申请实施例中的用于图像分类的目标神经网络。
图像(或图片)的分类是各类图像处理应用的基础，计算机视觉常常会涉及到如何对获取到的图像进行分类的问题。但是，通常用于图像分类的网络模型是基于图像的全局特征图进行分类，由于待识别的目标区域可能为全局特征图中的部分区域，因此，基于全局特征图进行分类处理会使得目标区域对应的特征图受到背景区域的噪声的影响，从而导致用于图像分类的网络模型的准确性降低。
本申请实施例提出了一种图像分类方法，通过目标区域分类损失函数、背景区域分类损失函数以及协同损失函数训练得到目标神经网络，能够实现分离待处理图像中待目标区域的特征图以及背景区域的特征图，通过抑制背景区域的噪声对待识别的目标区域的特征图的影响，提高待处理图像中待识别的目标区域分类结果的准确性。
图9示出了本申请实施例提供的图像分类方法600的示意性流程图，该方法可以由能够进行图像分类装置执行，例如，该方法可以由图8中的执行设备510执行，或者，也可以由本地设备520执行。其中，方法600包括步骤610至步骤620，下面分别对这些步骤进行详细的描述。
S610、获取待处理图像，所述待处理图像包括待识别的目标区域以及背景区域。
例如，待处理图像中可以是指包含目标对象所在的前景区域以及背景区域，目标对象可以是行人、动物以及车辆等任意一项。
应理解，上述待识别的目标区域即可以看作是包含目标对象所在的前景区域，目标对象可以是具有运动属性的对象。
示例性地，上述待处理图像可以是电子设备通过摄像头拍摄到的图像，或者，上述待处理图像还可以是从电子设备内部获得的图像(例如，电子设备的相册中存储的图像，或者，电子设备从云端获取的图片)。例如，电子设备可以是图8所示的本地设备或者执行设备中的任意一个。
S620、根据目标神经网络对待处理图像进行分类，得到目标区域的分类结果以及背景区域的分类结果；其中，所述目标区域分类损失函数用于表示所述样本图像的目标区域预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函数用于表示所述样本图像的背景区域预测分类结果与背景区域标注分类结果之间的损失值，所述协同损失函数用于分离所述样本图像的目标区域特征图与所述样本图像的背景区域特征图。
具体地，上述样本图像的目标区域特征图(或者，背景区域特征图)可以是卷积神经网络中的其他层处理后得到的特征图。应理解，这里所说的卷积神经网络中的其他层是指卷积神经网络中的一个层，例如，其他层可以是卷积神经网络中的输入层、卷积层、池化层或全连接层中的一个。
示例性地，当待处理图像中的目标区域为人像时，目标区域的分类结果可以是指行人的标识，其中，行人的标识可以是指行人的姓名、行人的编号或者行人的身份证号码；背景区域的分类结果可以是指获取到待处理图像的摄像头的标识，比如，摄像头的编号、摄 像头所在的位置信息。
需要说明的是，上述为对行人的标识与摄像头的标识的举例说明，行人的标识用于区分不同的行为，对于同一摄像头可以近似看作其获取的背景区域图像不变，即可以通过摄像头的标识确定背景区域图像的分类结果，行人的标识与摄像头的标识的具体实现形式本申请不作任何限定。
例如，当输入目标神经网络的待处理图像中包括行人A时，则输出目标区域的分类结果可以是行人A的标识信息，比如，可以是行人A的身份证号码、或者行人A的ID，上述标识信息可以用于识别行人A。
例如，当输入目标神经网络的待处理图像中包括一只猫，则输出目标区域的分类结果可以是动物-猫。
例如，当输入目标神经网络的待处理图像中包括行人A以及街景1，则目标区域的分类结果以及背景区域的分类结果可以是指行人A的标识信息以及摄像头A的标识信息，其中，行人A的标识信息是行人A的身份证号码、或者行人A的ID，上述标识信息可以用于识别行人A；摄像头A的标识信息可以是拍摄输入图像中街景1区域对应的摄像头的ID。
在一种可能的实现方式中，通过上述协同损失函数可以使得样本图像的目标区域掩膜与背景区域掩膜互补。
例如，当掩膜的取值范围为[0,1]时，目标区域掩膜与背景区域掩膜的总和为1，即背景区域掩膜＝1-目标区域掩膜。其中，目标区域掩膜用于表示目标区域特征图中每个像素点所在的位置属于目标区域的概率值；背景区域掩膜用于表示背景区域特征图中每个像素点所在的位置属于背景区域的概率值。
例如，当掩膜的取值范围为[-1,1]时，目标区域掩膜与背景区域掩膜的总和为0，即背景区域掩膜＝0-目标区域掩膜。
应理解，上述举例为了说明目标区域掩膜与背景区域掩膜互补，并不对目标区域掩膜以及背景区域掩膜的取值范围作任何限定。
在一种可能的实现方式中，通过上述协同损失函数可以使得样本图像中的目标区域特征图对应的目标区域图像与样本图像中的背景区域特征图对应的背景区域图像的重叠区域近乎为零，即通过协同损失函数可以使得目标神经网络提取样本图像中目标区域特征图与背景区域特征图的划界明显，从而避免背景区域的噪声对目标区域特征图的影响。
在本申请的实施例中，通过协同损失函数可以使得目标神经网络提取的目标区域特征图与背景区域的特征图的分割准确性提高，即能够在获取目标区域的特征图时避免受到背景区域的影响。换而言之，通过目标区域分类损失函数、背景区域损失函数以及协同损失函数整体训练不断调整网络的参数，使得可以使得目标神经网络在对待处理图像进行分类处理时，使得提取的目标区域特征图的准确率和背景区域特征图的准确率相互促进协同提高。
可选地，协同损失函数可以是通过第一协同损失函数和第二协同损失函数加权处理得到的，其中，第一协同损失函数可以是基于样本图像的目标区域特征图与样本图像的背景区域掩膜相乘得到的；第二协同损失函数可以是基于样本图像的背景区域特征图与样本图像的目标区域掩膜相乘得到的。
其中，样本图像的背景区域掩膜可以表示获取的样本图像的背景区域特征图中每个像素点所在位置属于样本图像的背景区域的概率值；样本图像的目标区域掩膜可以表示获取的样本图像的目标区域特征图中每个像素点所在位置属于所述样本图像的目标区域的概率值。
例如，目标区域掩膜中每个像素点对应的值表示该像素点所在的位置属于目标区域的概率值，概率值的取值范围可以为[0,1]；0可以表示该像素点所在的位置属于目标区域的概率为0；1可以表示该像素点所在的位置属于目标区域的概率为1。
同理，背景区域掩膜中每个像素点对应的值表示该像素点所在的位置属于背景区域的概率值，概率值的取值范围可以为[0,1]；0可以表示该像素点所在的位置属于背景区域的概率为0；1可以表示该像素点所在的位置属于背景区域的概率为1。
上述目标区域掩膜等同于目标区域注意力图；上述背景区域掩膜等同于背景区域注意力图。
需要说明的是，在本申请中图像的目标区域特征图、背景区域特征图、目标区域掩膜以及背景区域掩膜中任意两个的尺度大小相同。
例如，协同损失可以通过以下等式得到的：
Figure PCTCN2020112694-appb-000021
其中，L t表示协同损失函数，N表示样本图像中像素点的数量，F i表示样本图像的目标区域中第i个像素点，
Figure PCTCN2020112694-appb-000022
表示样本图像的目标区域掩膜中第i个像素点的概率值，B i表示样本图像的背景区域中第i个像素点，
Figure PCTCN2020112694-appb-000023
表示样本图像的背景区域掩膜中第i个像素点的概率值，⊙可以用于表示对应元素进行相乘，N为大于1的整数，i为大于或等于1且小于或等于N的整数。
在本申请的实施例中，为了便于通过前景区域分类损失函数、背景区域分类损失函数以及协同损失函数训练神经网络得到用于图像分类的目标神经网络。设计神经网络时可以包括两个分支，第一分支可以用于对样本图像的目标区域进行处理，第二分支可以用于对样本图像的背景区域进行处理，通过协同损失函数可以协同第一分支与第二分支进行训练，使得提取的目标区域特征图的准确率和背景区域特征图的准确率相互促进协同提高。
示例性地，目标神经网络可以包括第一分支和第二分支，其中，第一分支可以用于对样本图像的目标区域进行处理，得到样本图像的目标区域特征图和样本图像的目标区域掩膜；第二分支可以用于对样本图像的背景区域进行处理，得到样本图像的背景区域特征图和样本图像的背景区域掩膜。
在一种可能的实现方式中，目标区域掩膜是通过提取的目标区域特征图训练得到的；背景区域掩膜是通过提取的背景区域特征图训练得到的。
需要说明的是，上述目标区域分类损失函数用于表示样本图像的目标区域特征图对应的预测分类结果与目标区域标注分类结果之间的损失值，其中，样本图像的目标区域特征图也可以是指样本图像的目标区域增强特征图，比如目标区域增强特征图是通过提取的目标区域的低层特征图与目标区域掩膜中对应像素点相乘得到的；同理，背景区域分类损失函数用于表示所述样本图像的背景区域特征图对应的预测分类结果与背景区域标注分类结果之间的损失值，其中，背景区域特征图也可以是指背景区域的增强特征图，比如背景 区域增强特征图是通过提取的背景区域的低层特征图与背景区域掩膜中对应像素点相乘得到的。
在一个可能的实现方式中，在本申请的实施例中可以获取待处理图像，待处理图像包括待识别的目标区域以及背景区域；通过目标神经网络对该待处理图像进行特征提取，得到该目标区域特征图以及该背景区域特征图；通过该目标神经网络对该目标区域特征图以及该背景区域特征图进行分类，得到该目标区域的分类结果以及该背景区域的分类结果。
可选地，目标神经网络可以包括第一分支与第二分支，目标区域特征图可以是目标区域增强特征图，背景区域特征图可以是背景区域增强特征图；比如，可以通过目标神经网络的第一分支对待处理图像的目标区域进行处理，得到待处理图像的目标区域增强特征图，其中，目标区域增强特征图可以是提取的目标区域层特征图与和目标区域掩膜中对应的像素点相乘得到的；基于目标区域增强特征图进行分类，得到待处理图像中目标区域的预测分类结果。
同理，可以通过目标神经网络的第二分支对待处理图像的背景区域进行处理，得到待处理图像的背景区域增强特征图，其中，背景区域增强特征图可以是提取的背景区域特征图与和背景区域掩膜中对应的像素点相乘得到的；基于背景区域增强特征图进行分类，得到待处理图像中背景区域的预测分类结果。
需要说明的是，目标神经网络中的第一分支与第二分支可以同时对待处理图像进行分类处理，或者，目标神经网络中也可以是以先第一分支再第二分支的顺序对待处理图像进行分类处理，或者，目标神经网络中也可以是以先第二分支再第一分支的顺序对待处理图像进行分类处理。
进一步地，通过目标区域的预测分类结果与目标区域标注分类结果之间的偏差可以得到目标区域分类损失函数，通过背景区域的预测分类结果与背景区域标注分类结果之间的偏差可以得到背景区域分类损失函数。
在本申请中，目标神经网络是通过目标区域分类损失函数、背景区域损失函数以及协同损失函数训练得到的，使得预先训练的目标神经网络能够准确地分离待处理图像中目标区域特征图与背景区域的特征图。因此，目标神经网络在对待处理图像进行分类时，通过分离待处理图像中目标区域特征图以及背景区域特征图，不仅可以得到目标区域的分类结果以及背景区域的分类结果，同时在获取目标区域的特征图时还能避免受到背景区域的影响，提高待处理图像中目标区域分类结果的准确性。
图10示出了本申请实施例提供的图像分类模型的训练方法700的示意性流程图，该方法可以由能够进行图像分类装置执行，例如，该方法可以由图8中的执行设备510执行，或者，也可以由本地设备520执行。其中，方法700包括步骤710至步骤740，下面分别对这些步骤进行详细的描述。
步骤710、获取训练数据，其中，训练数据包括样本图像、样本图像的目标区域标注分类结果、样本图像的背景区域标注分类结果。
例如，目标区域为样本图像中目标对象所在的区域，目标对象可以是行人、动物以及车辆等任意一项。
示例性地，训练数据可以是指包含行人A的样本图像、行人A的标识以及样本图像中背景区域图像的标识，比如，可以是获取样本图像的摄像头的标识。
应理解，对于同一摄像头可以近似看作其获取的背景区域不变，则可以通过摄像头的标识确定背景区域图像对应的标识信息。
步骤720、根据神经网络提取的样本图像的目标区域特征图，得到样本图像的目标区域预测分类结果。
在一种可能的实现方式中，可以是对样本图像进行卷积运算，得到样本图像的低层全局特征；对样本图像的低层全局特征进行卷积运算，得到样本图像的目标区域特征图。
在另一种可能的实现方式中，可以是对样本图像进行卷积运算，直接得到样本图像的目标区域特征图。
进一步地，可以根据提取的目标区域特征图，得到目标区域掩膜。
例如，可以利用全卷积网络对目标区域特征图进行卷积运算，得到目标区域特的目标区域掩膜，其中，目标区域掩膜用于表示目标区域特征图中每个像素点所在的位置属于目标区域的概率值，比如，目标区域掩膜的取值范围可以为[0,1]，0可以表示该像素点所在的位置属于目标区域的概率为0；1可以表示该像素点所在的位置属于目标区域的概率为1。
可选地，根据神经网络提取的样本图像的目标区域特征图，得到样本图像的目标区域预测分类结果，包括：
可以通过目标神经网络对样本图像的目标区域进行处理，得到样本图像的目标区域增强特征图，其中，目标区域增强特征图可以是提取的目标区域低层特征图与和目标区域掩膜中对应的像素点相乘得到的；基于目标区域增强特征图进行分类，得到样本图像中目标区域的预测分类结果。
例如，上述根据基于目标区域增强特征图进行分类可以采用现有技术中的softmax算法进行分类，此处不再赘述。
例如，对于一个输入X，若需要确定X是N个类别中的哪一类，则假设通过一个模型可以对输入X输出N个类别的评分，评分越高说明X属于这个类别的可能性越大，评分最高的被认为是X正确的类别。然而，这样的评分范围很广，softmax算法可以将(-∞,+∞)的一组评分转化为一组概率，并使得这组概率的和为1的归一化的函数；此外这个函数具有保序性，即原来评分高的转换后概率值较大，评分小的对应的概率值较小。
示例性地，当样本图像中包括行人A时，则输出目标区域的分类结果可以是行人A的标识信息，比如，可以是行人A的身份号码、或者行人A的ID，上述标识信息可以用于识别行人A。
示例性地，当样本图像中包括一只猫，则输出目标区域的分类结果可以是动物-猫。
步骤730、根据神经网络提取的样本图像的背景区域特征图，得到样本图像的背景区域预测分类结果。
在一种可能的实现方式中，可以是对样本图像进行卷积运算，得到样本图像的低层全局特征；对样本图像的低层全局特征进行卷积运算，得到样本图像的背景区域特征图。
在另一种可能的实现方式中，可以是对样本图像进行卷积运算，直接得到样本图像的背景区域特征图。
进一步地，可以根据提取的目标区域特征图，得到背景区域掩膜。
例如，可以利用全卷积网络对背景区域特征图进行卷积运算，得到背景区域对应的背 景区域掩膜，其中，背景区域掩膜用于表示背景区域特征图中每个像素点所在的位置属于背景区域的概率值，比如，背景区域掩膜的取值范围可以是[0,1]，0可以表示该像素点所在的位置属于背景区域的概率为0；1可以表示该像素点所在的位置属于背景区域的概率为1。
可选地，根据神经网络提取的样本图像的背景区域特征图，得到样本图像的背景区域预测分类结果，包括：
可以通过目标神经网络对样本图像的背景区域进行处理，得到样本图像的背景区域增强特征图，其中，背景区域增强特征图可以是提取的背景区域低层特征图与和背景区域掩膜中对应的像素点相乘得到的；基于背景区域增强特征图进行分类，得到样本图像中背景区域的预测分类结果。
例如，上述根据基于背景区域增强特征图进行分类可以采用现有技术中的softmax算法进行分类，此处不再赘述。
示例性地，当样本图像中包括行人A以及街景1，则背景区域的分类结果可以是指行摄像头A的标识信息，摄像头A的标识信息可以是拍摄输入图像中街景1区域对应的摄像头的ID。
可选地，样本图像的背景区域的分类结果是指获取到样本图像的摄像头的标识。
应理解，对于同一摄像头可以近似看作其获取的背景区域不变，则可以通过摄像头的标识确定背景区域图像对应的标识信息。
步骤740、基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数，确定神经网络的参数，其中，目标区域分类损失函数用于表示样本图像的目标区域预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函数用于表示样本图像的背景区域预测分类结果与背景区域标注分类结果之间的损失值，协同损失函数用于分离样本图像的目标区域特征图与样本图像的背景区域特征图。
在一种可能的实现方式中，通过上述协同损失函数可以使得样本图像的目标区域掩膜与背景区域掩膜互补。
在一种可能的实现方式中，通过上述协同损失函数可以使得样本图像中的目标区域特征图对应的目标区域图像与样本图像中的背景区域特征图对应的背景区域图像的重叠区域近乎为零，即通过协同损失函数可以使得目标神经网络提取样本图像中目标区域特征图与背景区域特征图的划界明显，从而避免背景区域的噪声对目标区域特征图的影响。
可选地，协同损失函数可以是通过第一协同损失函数和第二协同损失函数加权处理得到的，其中，第一协同损失函数可以是基于样本图像的目标区域特征图与样本图像的背景区域掩膜中对应的像素点相乘得到的；第二协同损失函数可以是基于样本图像的背景区域特征图与样本图像的目标区域掩膜中对应的像素点相乘得到的。
需要说明的是，在本申请中图像的目标区域特征图、背景区域特征图、目标区域掩膜以及背景区域掩膜中任意两个的尺度大小相同。
例如，协同损失函数可以通过以下等式得到的：
Figure PCTCN2020112694-appb-000024
其中，L t表示协同损失函数，N表示样本图像中像素点的数量，F i表示样本图像的目 标区域中第i个像素点，
Figure PCTCN2020112694-appb-000025
表示样本图像的目标区域掩膜中第i个像素点的概率值，B i表示样本图像的背景区域中第i个像素点，
Figure PCTCN2020112694-appb-000026
表示样本图像的背景区域掩膜中第i个像素点的概率值，⊙用于表示对应元素进行相乘，N为大于1的整数，i为大于或等于1且小于或等于N的整数。
例如，基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数得到的最终损失函数可以是：
Figure PCTCN2020112694-appb-000027
其中，L f可以表示目标区域分类损失函数，L b可以表示背景区域分类损失函数，L t可以表示协同损失函数。
在本申请的实施例中，通过协同损失函数可以使得目标神经网络提取的目标区域特征图与背景区域的特征图的分割准确性提高，即能够在获取目标区域的特征图时避免受到背景区域的影响。换而言之，通过目标区域分类损失、背景区域损失以及协同损失整体训练不断调整网络的参数，使得可以使得目标神经网络在对待处理图像进行分类处理时，使得提取的目标区域特征图的准确率和背景区域特征图的准确率相互促进协同提高。
可选地，目标神经网络可以包括第一分支和第二分支，其中，第一分支可以用于对样本图像的目标区域进行处理，得到样本图像的目标区域特征图和样本图像的目标区域掩膜；第二分支可以用于对样本图像的背景区域进行处理，得到样本图像的背景区域特征图和样本图像的背景区域掩膜。
在本申请的实施例中，为了便于通过前景区域分类损失函数、背景区域分类损失函数以及协同损失函数训练神经网络得到用于图像分类的目标神经网络。设计神经网络时可以包括两个分支，第一分支可以用于对样本图像的目标区域进行处理，第二分支可以用于对样本图像的背景区域进行处理，通过协同损失函数可以协同第一分支与第二分支进行训练，使得提取的目标区域特征图的准确率和背景区域特征图的准确率相互促进协同提高。
例如，图11所示的为本申请实施例提供的图像分类模型的训练方法800的示意性流程图。其中，方法800包括步骤801至809，下面分别对这些步骤进行详细的描述。
步骤801、输入图像。
其中，输入图像可以是用于训练图像分类模型的训练数据，训练数据包括样本图像、样本图像的目标区域标注分类结果、样本图像的背景区域标注分类结果。
例如，目标区域为样本图像中目标对象所在的区域，目标对象可以是行人、动物以及车辆等任意一项。
步骤802、提取低层特征。
例如，可以通过低层特征提取模块提取输入图像的低层特征，其中，低层特征提取模块可以采用多种构架，比如VGG16、TSN网络等。
示例性地，上述输入图像的低层特征可以是指低层特征提取模块学到的输入图像的低层纹理特征。
步骤803、提取前景区域特征。
例如，可以通过前景提取模块提取前景区域特征，前景特征提取模块可以是一个多层卷积神经网络。通过向前景特征提取模块输入低层特征，可以得到输出的前景区域特征。
需要说明的是，上述前景区域特征可以是指目标对象所在的区域，例如，目标对象可以是指行人、动物以及车辆等。
步骤804、生成前景注意力掩膜。
例如，可以通过前景注意力掩膜生成模块得到前景注意力掩膜，其中，前景注意力掩膜生成模块可以是全卷积神经网络，该全卷积神经网络可以包含两个卷积模块，每一个卷积模块可以包含三个连续的操作：卷积层、批归一化层(bach normalization,BN)、线性整流函数(rectified linear unit,ReLU)。前景注意力掩膜生成模块的输入是前景特征F，输出为前景注意力掩膜Z f。
应理解，前景注意力掩膜的大小和前景特征的大小一致，前景注意力掩膜中每个位置对应的值表示该位置属于前景区域的概率，概率值的取值范围可以为[0,1]，0表示该位置属于前景的概率为0，1表示该位置属于前景的概率为1。
步骤805、得到前景增强特征
其中，前景增强特征是通过前景注意力掩膜Z f增强前景特征F得到的，从而抑制背景噪声。
例如，可以通过将前景注意力掩膜Z f与前景特征F逐像素相乘，获得增强前景特征，其中，前景特征F与前景注意力掩膜Z f的大小相同，前景注意力掩膜中每个位置的值表示其属于前景区域的概率。
进一步地，可以将前景增强特征输入至前景分类器中进行分类，得到前景区域的预测分类结果。
例如，输入图像中包括行人时，其分类的结果为该输入图像属于哪一个行人ID。
示例性地，行人ID可以指行人的姓名、行人的编号或者行人的身份证号码等，行人ID可以用于区分不同的行人。
步骤806、提取背景特征。
例如，可以通过背景提取模块提取背景区域特征，背景特征提取模块可以是一个多层卷积神经网络。通过向背景特征提取模块输入低层特征，可以得到输出的背景区域特征。
步骤807、生成背景注意力掩膜。
例如，可以通过背景注意力掩膜生成模块得到背景注意力掩膜，其中，背景注意力掩膜生成模块可以是全卷积神经网络，该全卷积神经网络可以包含两个卷积模块，每一个卷积模块可以包含三个连续的操作：卷积层、批归一化层(bach normalization,BN)、线性整流函数(rectified linear unit,ReLU)。背景注意力掩膜生成模块的输入是背景特征B，输出为背景注意力掩膜Z b。
应理解，背景注意力掩膜的大小和背景特征的大小一致，背景注意力掩膜中每个位置对应的值表示该位置属于背景区域的概率，概率值的取值范围可以为[0,1]，0表示该位置属于背景的概率为0，1表示该位置属于背景的概率为1。
步骤808、得到背景增强特征
其中，前景增强特征是通过背景注意力掩膜Z b增强背景特征B得到的，从而抑制前景噪声。
例如，可以通过将背景注意力掩膜Z b与背景特征B逐像素相乘，获得增强背景特征，其中，背景特征B与背景注意力掩膜Z b的大小相同，背景注意力掩膜中每个位置的值表示其属于背景区域的概率。
进一步地，可以将背景增强特征输入至前景分类器中进行分类，得到背景区域的预测 分类结果。
示例性地，对于同一摄像头可以近似看作其获取的背景区域不变，则可以通过摄像头ID确定背景区域图像对应的分类结果。
例如，摄像头ID可以是指摄像头的编号、摄像头所在的位置信息等。
步骤809、协同训练。
例如，可以将前景特征图与背景注意力掩膜对应相乘，同时将背景特征图与前景注意力掩膜对应相乘，生成协同训练损失函数。
在一个实施例中，对于一个训练数据可以包含输入图像、输入图像对应的行人ID以及摄像头ID，通过行人ID可以获得前景分支的softmax损失函数，通过摄像头ID可以获得背景分支的softmax损失函数；将协同训练损失函数、前景分支的softmax损失函数和背景分支的softmax损失函数加权，获得协同双路网络的最终损失函数，通过反向传播算法使得协同训练损失函数最小化，从而生成双路网络的各个参数值。
例如，上述用于图像分类的目标神经网络的训练方法的流程图如图12所示，在图12中以输入图像为行人图像进行举例说明，输入图像为行人图像时输出的前景区域的分类结果可以是行人的ID；背景区域的分类结果可以是摄像头ID，即可以是获取到输入图像的摄像头的标识。
如图12所示，目标神经神经网络的训练方法的流程可以包括以下步骤：
步骤一：将训练图像输入至特征提取模块(例如，低层特征提取模块)，得到训练图像的全局特征图；
步骤二：将训练图像的全局特征图分别输入前景特征提取模块与背景特征提取模块，得到训练图像的前景特征图与背景特征图。
例如，可以同时将训练图像的全局特征图分别输入至前景特征提取模块与背景特征提取模块；或者，也可以是先将训练图像的全局特征图输入前景特征提取模块，再将训练图像的全局特征图输入至背景特征提取模块。
需要说明的是，上述前景特征提取模块与背景特征提取模块可以并行地对待训练图像的特征图进行处理，不限定输入的先后顺序。
步骤三：将前景特征图输入前景注意力掩膜生成模块，生成前景注意力掩膜；
步骤四：将背景特征图输入背景注意力掩膜生成模块，生成背景注意力掩膜；
需要说明的是，上述步骤三与步骤四可以是同时执行的，或者也可以是先执行步骤四再执行步骤三，并不对步骤三与步骤四的执行先后顺序进行任何限定。
步骤五：将前景特征图与前景注意力掩膜逐像素对应相乘，生成增强前景特征图；
例如，前景区域特征图输入至目标增强模块(target enhancement module，TEM)中可以得到增强前景特征图。
步骤六：将背景特征图与背景注意力掩膜逐像素对应相乘，生成增强背景特征图；
例如，背景区域特征图输入至目标增强模块(target enhancement module，TEM)中可以得到增强背景特征图。
应理解，图12中第一分支的TEM与第二分支的TEM可以是两个不同的模块。例如，第一分支中的TEM用于对前景区域特征图进行增强处理；第二分支的TEM用于对背景区域特征图进行增强处理。
需要说明的是，上述步骤五与步骤六可以是同时执行的，或者也可以是先执行步骤五再执行步骤六，并不对步骤五与步骤六的执行先后顺序进行任何限定。
步骤七：将增强前景特征图进行下采样，并输入前景分类器中进行分类，得到分类的类别为行人ID；
示例性地，行人ID可以是指行人的姓名、行人的编号或者行人的身份证号码等任意一项。
步骤八：将增强背景特征图进行下采样，并输入背景分类器中进行分类，得到分类的类别为摄像头ID。
示例性地，摄像头ID可以是指摄像头的编号、或者摄像头所在的位置信息等。
需要说明的是，上述为对行人ID与摄像头ID的举例说明，行人ID用于区分不同的行为，对于同一摄像头可以近似看作其获取的背景区域图像不变，即可以通过摄像头ID确定背景区域图像的分类结果，行人ID与摄像头ID的具体实现形式本申请不作任何限定。
应理解，上述步骤七与步骤八可以是同时执行的，或者也可以是先执行步骤八再执行步骤七，并不对步骤七与步骤八的执行先后顺序进行限定。
步骤九：将前景特征图与背景注意力掩膜对应相乘，同时将背景特征图与前景注意力掩膜对应相乘，生成协同训练Loss，通过反向传播算法使得协同训练Loss最小化，生成双路网络的各个参数值。
在本申请的实施例中，如图12所示的协同双路网络，通过使用行人ID和摄像头ID有监督地训练协同双路网络，使得协同双路网络能够同时提取行人特征和背景特征。通过协同训练双路网络，使得前景特征图、前景注意力掩膜、背景特征图、背景注意力掩膜的精度可以协同提高；同时，由于使用摄像头ID作为监督信息，使得本网络在提取注意力掩膜时不需要额外的标注数据集。
应理解，上述举例说明是为了帮助本领域技术人员理解本申请实施例，而非要将本申请实施例限于所例示的具体数值或具体场景。本领域技术人员根据所给出的上述举例说明，显然可以进行各种等价的修改或变化，这样的修改或变化也落入本申请实施例的范围内。
上文结合图1至图12，详细描述了本申请实施例图像分类方法，下面将结合图13和图16，详细描述本申请的装置实施例。应理解，本申请实施例中的图像分类装置可以执行前述本申请实施例的各种图像分类方法，即以下各种产品的具体工作过程，可以参考前述方法实施例中的对应过程。
图13是本申请实施例提供的图像分类装置的示意性框图。应理解，图像分类装置900可以执行图9所示的图像分类方法。该图像分类装置900包括：获取单元910和处理单元920。
其中，获取单元910用于获取待处理图像，所述待处理图像包括待识别的目标区域以及背景区域；所述处理单元920用于根据目标神经网络对所述待处理图像进行分类，得到所述待处理图像的所述目标区域的分类结果与所述背景区域的分类结果；
其中，所述目标神经网络是基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数训练得到的，所述目标区域分类损失函数用于表示样本图像的目标区域特征图对应的预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函 数用于表示所述样本图像的背景区域特征图对应的预测分类结果与背景区域标注分类结果之间的损失值，所述协同损失函数用于分离所述样本图像的目标区域特征图与所述样本图像的背景区域特征图。
可选地，作为一个实施例，所述协同损失函数是通过第一协同损失函数和第二协同损失函数加权处理得到的，其中，所述第一协同损失函数是基于所述样本图像的目标区域特征图与所述样本图像的背景区域掩膜中对应的像素点相乘得到的；
所述第二协同损失函数是基于所述样本图像的背景区域特征图与所述样本图像的目标区域掩膜中对应的像素点相乘得到的。
可选地，作为一个实施例，所述目标神经网络包括第一分支和第二分支，其中，所述第一分支用于对所述样本图像的目标区域进行处理，得到所述样本图像的目标区域特征图和所述样本图像的目标区域掩膜；
所述第二分支用于对所述样本图像的背景区域进行处理，得到所述样本图像的背景区域特征图和所述样本图像的背景区域掩膜。
可选地，作为一个实施例，所述协同损失函数是通过以下等式得到的：
Figure PCTCN2020112694-appb-000028
其中，L t表示所述协同损失函数，N表示所述样本图像中像素点的数量，F i表示所述样本图像的目标区域中第i个像素点，
Figure PCTCN2020112694-appb-000029
表示所述样本图像的目标区域掩膜中第i个像素点的概率值，B i表示所述样本图像的背景区域中第i个像素点，
Figure PCTCN2020112694-appb-000030
表示所述样本图像的背景区域掩膜中第i个像素点的概率值，⊙用于表示对应元素进行相乘，N为大于1的整数，i为大于或等于1且小于或等于N的整数。
可选地，作为一个实施例，所述处理单元920具体用于：
通过所述目标神经网络的所述第一分支对所述待处理图像的目标区域进行处理，得到所述待处理图像的目标区域增强特征图，其中，所述目标区域增强特征图是通过所述待处理图像的目标区域特征图和所述待处理图像的目标区域掩膜相乘得到的；通过所述目标神经网络的所述第二分支对所述待处理图像的背景区域进行处理，得到所述待处理图像的背景区域增强特征图，其中，所述背景区域增强特征图是通过所述待处理图像的背景区域特征图和所述待处理图像的背景区域掩膜相乘得到的；
基于所述待处理图像的目标区域增强特征图与背景区域增强特征图进行分类，得到所述待处理图像的目标区域的分类结果与背景区域的分类结果。
可选地，作为一个实施例，所述待处理图像的背景区域的分类结果是指获取到所述待处理图像的摄像头的标识。
图14是本申请实施例提供的图像分类模型的训练装置的示意性框图。应理解，训练装置1000可以执行图10或图11所示的图像分类模型的训练方法。该训练装置1000包括：获取单元1010和处理单元1020。
其中，所述获取单元1010，用于获取训练数据，其中，所述训练数据包括样本图像、所述样本图像的目标区域标注分类结果、所述样本图像的背景区域标注分类结果；所述处理单元1020，用于根据神经网络提取的所述样本图像的目标区域特征图，得到所述样本图像的目标区域预测分类结果；根据所述神经网络提取的所述样本图像的背景区域特征 图，得到所述样本图像的背景区域预测分类结果；基于所述样本图像的目标区域分类损失函数、背景区域分类损失函数以及协同损失函数，确定所述神经网络的参数；
其中，所述目标区域分类损失函数用于表示所述样本图像的目标区域预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函数用于表示所述样本图像的背景区域预测分类结果与背景区域标注分类结果之间的损失值，所述协同损失函数用于分离所述样本图像的目标区域特征图与所述样本图像的背景区域特征图。
可选地，作为一个实施例，所述协同损失函数是通过第一协同损失函数和第二协同损失函数加权处理得到的，其中，所述第一协同损失函数是基于所述样本图像的目标区域特征图与所述样本图像的背景区域掩膜中对应的像素点相乘得到的；
所述第二协同损失函数是基于所述样本图像的背景区域特征图与所述样本图像的目标区域掩膜中对应的像素点相乘得到的。
可选地，作为一个实施例，所述神经网络包括第一分支和第二分支，所述第一分支用于对所述样本图像的目标区域进行处理，得到所述样本图像的目标区域特征图和所述样本图像的目标区域掩膜；
所述第二分支用于对所述样本图像的背景区域进行处理，得到所述样本图像的背景区域特征图和所述样本图像的背景区域掩膜。
可选地，作为一个实施例，所述协同损失函数是通过以下等式得到的：
Figure PCTCN2020112694-appb-000031
其中，L t表示所述协同损失函数，N表示所述样本图像中像素点的数量，F i表示所述样本图像的目标区域中第i个像素点，
Figure PCTCN2020112694-appb-000032
表示所述样本图像的目标区域掩膜中第i个像素点的概率值，B i表示所述样本图像的背景区域中第i个像素点，
Figure PCTCN2020112694-appb-000033
表示所述样本图像的背景区域掩膜中第i个像素点的概率值，⊙用于表示对应元素进行相乘，N为大于1的整数，i为大于或等于1且小于或等于N的整数。
可选地，作为一个实施例，所述样本图像的目标区域预测分类结果是指获取到所述样本图像的摄像头的标识。
需要说明的是，上述图像分类装置900以及图像分类模型的训练装置1000以功能单元的形式体现。这里的术语“单元”可以通过软件和/或硬件形式实现，对此不作具体限定。
例如，“单元”可以是实现上述功能的软件程序、硬件电路或二者结合。所述硬件电路可能包括应用特有集成电路(application specific integrated circuit，ASIC)、电子电路、用于执行一个或多个软件或固件程序的处理器(例如共享处理器、专有处理器或组处理器等)和存储器、合并逻辑电路和/或其它支持所描述的功能的合适组件。
因此，在本申请的实施例中描述的各示例的单元，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
图15是本申请实施例提供的图像分类装置的硬件结构示意图。图15所示的图像分类装置1100(该装置1100具体可以是一种计算机设备)包括存储器1101、处理器1102、 通信接口1103以及总线1104。其中，存储器1101、处理器1102、通信接口1103通过总线1104实现彼此之间的通信连接。
存储器1101可以是只读存储器(read only memory，ROM)，静态存储设备，动态存储设备或者随机存取存储器(random access memory，RAM)。存储器1101可以存储程序，当存储器1101中存储的程序被处理器1102执行时，处理器1102用于执行本申请实施例的图像分类方法的各个步骤，例如，执行图9所示的各个步骤。
应理解，本申请实施例所示的图像分类装置可以是服务器，例如，可以是云端的服务器，或者，也可以是配置于云端的服务器中的芯片。
处理器1102可以采用通用的中央处理器(central processing unit，CPU)，微处理器，应用专用集成电路(application specific integrated circuit，ASIC)，图形处理器(graphics processing unit，GPU)或者一个或多个集成电路，用于执行相关程序以实现本申请方法实施例的图像分类方法。
处理器1102还可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，本申请的图像分类方法的各个步骤可以通过处理器1102中的硬件的集成逻辑电路或者软件形式的指令完成。
上述处理器1102还可以是通用处理器、数字信号处理器(digital signal processing，DSP)、专用集成电路(ASIC)、现成可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1101，处理器1102读取存储器1101中的信息，结合其硬件完成本申请实施中图13所示的图像分类装置中包括的单元所需执行的功能，或者，执行本申请方法实施例的图9所示的图像分类方法。
通信接口1103使用例如但不限于收发器一类的收发装置，来实现装置1100与其他设备或通信网络之间的通信。
总线1104可包括在装置1100各个部件(例如，存储器1101、处理器1102、通信接口1103)之间传送信息的通路。
图16是本申请实施例提供的图像分类模型的训练装置的硬件结构示意图。图16所示的训练装置1200(该训练装置1200具体可以是一种计算机设备)包括存储器1201、处理器1202、通信接口1203以及总线1204。其中，存储器1201、处理器1202、通信接口1203通过总线1204实现彼此之间的通信连接。
存储器1201可以是只读存储器(read only memory，ROM)，静态存储设备，动态存储设备或者随机存取存储器(random access memory，RAM)。存储器1201可以存储程序，当存储器1201中存储的程序被处理器1202执行时，处理器1202用于执行本申请实施例的图像分类模型的训练方法的各个步骤，例如，执行图10或图11所示的各个步骤。
应理解，本申请实施例所示的训练装置可以是服务器，例如，可以是云端的服务器，或者，也可以是配置于云端的服务器中的芯片。
处理器1202可以采用通用的中央处理器(central processing unit，CPU)，微处理器，应用专用集成电路(application specific integrated circuit，ASIC)，图形处理器(graphics processing unit，GPU)或者一个或多个集成电路，用于执行相关程序，以实现本申请方法实施例的图像分类模型的训练方法。
处理器1202还可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，本申请的图像分类模型的训练方法的各个步骤可以通过处理器1202中的硬件的集成逻辑电路或者软件形式的指令完成。
上述处理器1202还可以是通用处理器、数字信号处理器(digital signal processing，DSP)、专用集成电路(ASIC)、现成可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1201，处理器1202读取存储器1201中的信息，结合其硬件完成图14所示的图像分类模型的训练装置中包括的单元所需执行的功能，或者，执行本申请方法实施例的图10或者图11所示的图像分类模型的训练方法。
通信接口1203使用例如但不限于收发器一类的收发装置，来实现装置1200与其他设备或通信网络之间的通信。
总线1204可包括在装置1200各个部件(例如，存储器1201、处理器1202、通信接口1203)之间传送信息的通路。
应注意，尽管上述装置1100和训练装置1200仅仅示出了存储器、处理器、通信接口，但是在具体实现过程中，本领域的技术人员应当理解，装置1100和训练装置1200还可以包括实现正常运行所必须的其他器件。同时，根据具体需要本领域的技术人员应当理解，上述装置1100和训练装置1200还可包括实现其他附加功能的硬件器件。此外，本领域的技术人员应当理解，上述装置1100和训练装置1200也可仅仅包括实现本申请实施例所必须的器件，而不必包括图15或图16中所示的全部器件。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显 示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (28)

  1. 一种图像分类方法，其特征在于，包括：
    获取待处理图像，所述待处理图像包括待识别的目标区域以及背景区域；
    根据目标神经网络对所述待处理图像进行分类，得到所述待处理图像的所述目标区域的分类结果与所述背景区域的分类结果；
    其中，所述目标神经网络是基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数训练得到的，所述目标区域分类损失函数用于表示样本图像的目标区域特征图对应的预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函数用于表示所述样本图像的背景区域特征图对应的预测分类结果与背景区域标注分类结果之间的损失值，所述协同损失函数用于分离所述样本图像的目标区域特征图与所述样本图像的背景区域特征图。
  2. 如权利要求1所述的图像分类方法，其特征在于，所述协同损失函数是通过第一协同损失函数和第二协同损失函数加权处理得到的，其中，所述第一协同损失函数是基于所述样本图像的目标区域特征图与所述样本图像的背景区域掩膜相乘得到的；
    所述第二协同损失函数是基于所述样本图像的背景区域特征图与所述样本图像的目标区域掩膜相乘得到的。
  3. 如权利要求1或2所述的图像分类方法，其特征在于，所述目标神经网络包括第一分支和第二分支，其中，所述第一分支用于对所述样本图像的目标区域进行处理，得到所述样本图像的目标区域特征图和所述样本图像的目标区域掩膜；
    所述第二分支用于对所述样本图像的背景区域进行处理，得到所述样本图像的背景区域特征图和所述样本图像的背景区域掩膜。
  4. 如权利要求1至3中任一项所述的图像分类方法，其特征在于，所述协同损失函数是通过以下等式得到的：
    Figure PCTCN2020112694-appb-100001
    其中，L t表示所述协同损失函数，N表示所述样本图像中像素点的数量，F i表示所述样本图像的目标区域中第i个像素点，
    Figure PCTCN2020112694-appb-100002
    表示所述样本图像的目标区域掩膜中第i个像素点的概率值，B i表示所述样本图像的背景区域中第i个像素点，
    Figure PCTCN2020112694-appb-100003
    表示所述样本图像的背景区域掩膜中第i个像素点的概率值，N为大于1的整数，i为大于或等于1且小于或等于N的整数。
  5. 如权利要求3或4所述的图像分类方法，其特征在于，所述根据目标神经网络所述待处理图像进行分类，得到所述待处理图像的目标区域的分类结果与所述背景区域的分类结果，包括：
    通过所述目标神经网络的所述第一分支对所述待处理图像的目标区域进行处理，得到所述待处理图像的目标区域增强特征图，其中，所述目标区域增强特征图是通过所述待处理图像的目标区域特征图和所述待处理图像的目标区域掩膜相乘得到的；
    通过所述目标神经网络的所述第二分支对所述待处理图像的背景区域进行处理，得到 所述待处理图像的背景区域增强特征图，其中，所述背景区域增强特征图是通过所述待处理图像的背景区域特征图和所述待处理图像的背景区域掩膜相乘得到的；
    基于所述待处理图像的目标区域增强特征图与背景区域增强特征图进行分类，得到所述待处理图像的目标区域的分类结果与背景区域的分类结果。
  6. 如权利要求1至5中任一项所述的图像分类方法，其特征在于，所述待处理图像的背景区域的分类结果是指获取到所述待处理图像的摄像头的标识。
  7. 一种图像分类模型的训练方法，其特征在于，包括：
    获取训练数据，其中，所述训练数据包括样本图像、所述样本图像的目标区域标注分类结果、所述样本图像的背景区域标注分类结果；
    根据神经网络提取的所述样本图像的目标区域特征图，得到所述样本图像的目标区域预测分类结果；
    根据所述神经网络提取的所述样本图像的背景区域特征图，得到所述样本图像的背景区域预测分类结果；
    基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数，确定所述神经网络的参数，
    其中，所述目标区域分类损失函数用于表示所述样本图像的目标区域预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函数用于表示所述样本图像的背景区域预测分类结果与背景区域标注分类结果之间的损失值，所述协同损失函数用于分离所述样本图像的目标区域特征图与所述样本图像的背景区域特征图。
  8. 如权利要求7所述的训练方法，其特征在于，所述协同损失函数是通过第一协同损失函数和第二协同损失函数加权处理得到的，其中，所述第一协同损失函数是基于所述样本图像的目标区域特征图与所述样本图像的背景区域掩膜相乘得到的；
    所述第二协同损失函数是基于所述样本图像的背景区域特征图与所述样本图像的目标区域掩膜相乘得到的。
  9. 如权利要求7或8所述的训练方法，其特征在于，所述神经网络包括第一分支和第二分支，所述第一分支用于对所述样本图像的目标区域进行处理，得到所述样本图像的目标区域特征图和所述样本图像的目标区域掩膜；
    所述第二分支用于对所述样本图像的背景区域进行处理，得到所述样本图像的背景区域特征图和所述样本图像的背景区域掩膜。
  10. 如权利要求7至9中任一项所述的训练方法，其特征在于，所述协同损失函数是通过以下等式得到的：
    Figure PCTCN2020112694-appb-100004
    其中，L t表示所述协同损失函数，N表示所述样本图像中像素点的数量，F i表示所述样本图像的目标区域中第i个像素点，
    Figure PCTCN2020112694-appb-100005
    表示所述样本图像的目标区域掩膜中第i个像素点的概率值，B i表示所述样本图像的背景区域中第i个像素点，
    Figure PCTCN2020112694-appb-100006
    表示所述样本图像的背景区域掩膜中第i个像素点的概率值，N为大于1的整数，i为大于或等于1且小于或等于N的整数。
  11. 如权利要求7至10中任一项所述的训练方法，其特征在于，所述样本图像的目 标区域预测分类结果是指获取到所述样本图像的摄像头的标识。
  12. 一种图像分类装置，其特征在于，包括：
    获取单元，用于获取待处理图像，所述待处理图像包括待识别的目标区域以及背景区域；
    处理单元，用于根据目标神经网络对所述待处理图像进行分类，得到所述待处理图像的所述目标区域的分类结果与所述背景区域的分类结果；
    其中，所述目标神经网络是基于目标区域分类损失函数、背景区域分类损失函数以及协同损失函数训练得到的，所述目标区域分类损失函数用于表示样本图像的目标区域特征图对应的预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函数用于表示所述样本图像的背景区域特征图对应的预测分类结果与背景区域标注分类结果之间的损失值，所述协同损失函数用于分离所述样本图像的目标区域特征图与所述样本图像的背景区域特征图。
  13. 如权利要求12所述的图像分类装置，其特征在于，所述协同损失函数是通过第一协同损失函数和第二协同损失函数加权处理得到的，其中，所述第一协同损失函数是基于所述样本图像的目标区域特征图与所述样本图像的背景区域掩膜相乘得到的；
    所述第二协同损失函数是基于所述样本图像的背景区域特征图与所述样本图像的目标区域掩膜相乘得到的。
  14. 如权利要求12或13所述的图像分类装置，其特征在于，所述目标神经网络包括第一分支和第二分支，其中，所述第一分支用于对所述样本图像的目标区域进行处理，得到所述样本图像的目标区域特征图和所述样本图像的目标区域掩膜；
    所述第二分支用于对所述样本图像的背景区域进行处理，得到所述样本图像的背景区域特征图和所述样本图像的背景区域掩膜。
  15. 如权利要求12至14中任一项所述的图像分类装置，其特征在于，所述协同损失函数是通过以下等式得到的：
    Figure PCTCN2020112694-appb-100007
    其中，L t表示所述协同损失函数，N表示所述样本图像中像素点的数量，F i表示所述样本图像的目标区域中第i个像素点，
    Figure PCTCN2020112694-appb-100008
    表示所述样本图像的目标区域掩膜中第i个像素点的概率值，B i表示所述样本图像的背景区域中第i个像素点，
    Figure PCTCN2020112694-appb-100009
    表示所述样本图像的背景区域掩膜中第i个像素点的概率值，N为大于1的整数，i为大于或等于1且小于或等于N的整数。
  16. 如权利要求14或15所述的图像分类装置，其特征在于，所述处理单元具体用于：
    通过所述目标神经网络的所述第一分支对所述待处理图像的目标区域进行处理，得到所述待处理图像的目标区域增强特征图，其中，所述目标区域增强特征图是通过所述待处理图像的目标区域特征图和所述待处理图像的目标区域掩膜相乘得到的；
    通过所述目标神经网络的所述第二分支对所述待处理图像的背景区域进行处理，得到所述待处理图像的背景区域增强特征图，其中，所述背景区域增强特征图是通过所述待处理图像的背景区域特征图和所述待处理图像的背景区域掩膜相乘得到的；
    基于所述待处理图像的目标区域增强特征图与背景区域增强特征图进行分类，得到所 述待处理图像的目标区域的分类结果与背景区域的分类结果。
  17. 如权利要求12至16中任一项所述的图像分类装置，其特征在于，所述待处理图像的背景区域的分类结果是指获取到所述待处理图像的摄像头的标识。
  18. 一种图像分类模型的训练装置，其特征在于，包括：
    获取单元，用于获取训练数据，其中，所述训练数据包括样本图像、所述样本图像的目标区域标注分类结果、所述样本图像的背景区域标注分类结果；
    处理单元，用于根据神经网络提取的所述样本图像的目标区域特征图，得到所述样本图像的目标区域预测分类结果；根据所述神经网络提取的所述样本图像的背景区域特征图，得到所述样本图像的背景区域预测分类结果；基于所述样本图像的目标区域分类损失函数、背景区域分类损失函数以及协同损失函数，确定所述神经网络的参数，
    其中，所述目标区域分类损失函数用于表示所述样本图像的目标区域预测分类结果与目标区域标注分类结果之间的损失值，所述背景区域分类损失函数用于表示所述样本图像的背景区域预测分类结果与背景区域标注分类结果之间的损失值，所述协同损失函数用于分离所述样本图像的目标区域特征图与所述样本图像的背景区域特征图。
  19. 如权利要求18所述的训练装置，其特征在于，所述协同损失函数是通过第一协同损失函数和第二协同损失函数加权处理得到的，其中，所述第一协同损失函数是基于所述样本图像的目标区域特征图与所述样本图像的背景区域掩膜相乘得到的；
    所述第二协同损失函数是基于所述样本图像的背景区域特征图与所述样本图像的目标区域掩膜相乘得到的。
  20. 如权利要求18或19所述的训练装置，其特征在于，所述神经网络包括第一分支和第二分支，所述第一分支用于对所述样本图像的目标区域进行处理，得到所述样本图像的目标区域特征图和所述样本图像的目标区域掩膜；
    所述第二分支用于对所述样本图像的背景区域进行处理，得到所述样本图像的背景区域特征图和所述样本图像的背景区域掩膜。
  21. 如权利要求18至20中任一项所述的训练装置，其特征在于，所述协同损失函数是通过以下等式得到的：
    Figure PCTCN2020112694-appb-100010
    其中，L t表示所述协同损失函数，N表示所述样本图像中像素点的数量，F i表示所述样本图像的目标区域中第i个像素点，
    Figure PCTCN2020112694-appb-100011
    表示所述样本图像的目标区域掩膜中第i个像素点的概率值，B i表示所述样本图像的背景区域中第i个像素点，
    Figure PCTCN2020112694-appb-100012
    表示所述样本图像的背景区域掩膜中第i个像素点的概率值，N为大于1的整数，i为大于或等于1且小于或等于N的整数。
  22. 如权利要求18至21中任一项所述的训练装置，其特征在于，所述样本图像的目标区域预测分类结果是指获取到所述样本图像的摄像头的标识。
  23. 一种图像分类装置，其特征在于，包括处理器和存储器，所述存储器用于存储程序指令，所述处理器用于调用所述程序指令来执行权利要求1至6中任一项所述的图像分类方法。
  24. 一种图像分类模型的训练装置，其特征在于，包括处理器和存储器，所述存储器 用于存储程序指令，所述处理器用于调用所述程序指令来执行权利要求7至11中任一项所述的训练方法。
  25. 一种计算机可读存储介质，其特征在于，所述计算机可读介质存储用于设备执行的程序代码，该程序代码包括用于执行如权利要求1至6中任一项所述的图像分类方法。
  26. 一种计算机可读存储介质，其特征在于，所述计算机可读介质存储用于设备执行的程序代码，该程序代码包括用于执行如权利要求7至11中任一项所述的训练方法。
  27. 一种芯片，其特征在于，所述芯片包括处理器与数据接口，所述处理器通过所述数据接口读取存储器上存储的指令，以执行如权利要求1至6中任一项所述的图像分类方法。
  28. 一种芯片，其特征在于，所述芯片包括处理器与数据接口，所述处理器通过所述数据接口读取存储器上存储的指令，以执行如权利要求7至11中任一项所述的训练方法。
PCT/CN2020/112694 2019-09-02 2020-08-31 图像分类方法以及装置 WO2021043112A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910824602.1 2019-09-02
CN201910824602.1A CN112446398B (zh) 2019-09-02 2019-09-02 图像分类方法以及装置

Publications (1)

Publication Number Publication Date
WO2021043112A1 true WO2021043112A1 (zh) 2021-03-11

Family

ID=74734189

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/112694 WO2021043112A1 (zh) 2019-09-02 2020-08-31 图像分类方法以及装置

Country Status (2)

Country Link
CN (1) CN112446398B (zh)
WO (1) WO2021043112A1 (zh)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113222899A (zh) * 2021-04-15 2021-08-06 浙江大学 一种基于深度学习通过ct的检测分割分类肝脏肿瘤的方法
CN113313729A (zh) * 2021-05-26 2021-08-27 惠州中国科学院遥感与数字地球研究所空间信息技术研究院 单极对象图像成像方法、装置、计算机设备和存储介质
CN113408662A (zh) * 2021-07-19 2021-09-17 北京百度网讯科技有限公司 图像识别、图像识别模型的训练方法和装置
CN113516201A (zh) * 2021-08-09 2021-10-19 中国农业大学 一种基于深度神经网络的肉兔料盒中余料量的估算方法
CN113537248A (zh) * 2021-08-13 2021-10-22 珠海格力电器股份有限公司 图像识别方法和装置、电子设备和存储介质
CN113591736A (zh) * 2021-08-03 2021-11-02 北京百度网讯科技有限公司 特征提取网络、活体检测模型的训练方法和活体检测方法
CN113610026A (zh) * 2021-08-13 2021-11-05 广联达科技股份有限公司 基于掩膜注意力的行人重识别方法及重识别装置
CN113642515A (zh) * 2021-08-30 2021-11-12 北京航空航天大学 基于姿态关联的行人识别方法与装置、电子设备和介质
CN113673332A (zh) * 2021-07-15 2021-11-19 浙江大华技术股份有限公司 对象识别方法、装置以及计算机可读存储介质
CN113837205A (zh) * 2021-09-28 2021-12-24 北京有竹居网络技术有限公司 用于图像特征表示生成的方法、设备、装置和介质
CN113989857A (zh) * 2021-12-27 2022-01-28 四川新网银行股份有限公司 一种基于深度学习的人像照片内容解析方法及系统
CN114119976A (zh) * 2021-11-30 2022-03-01 广州文远知行科技有限公司 语义分割模型训练、语义分割的方法、装置及相关设备
CN114661904A (zh) * 2022-03-10 2022-06-24 北京百度网讯科技有限公司 文档处理模型的训练方法、装置、设备、存储介质及程序
WO2024125380A1 (zh) * 2022-12-13 2024-06-20 广电运通集团股份有限公司 分类识别方法、计算机设备和存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115187824A (zh) * 2021-03-22 2022-10-14 华为技术有限公司 一种模型训练方法、场景识别方法及相关设备
CN114943909B (zh) * 2021-03-31 2023-04-18 华为技术有限公司 运动区域识别的方法、装置、设备及系统
CN113222055B (zh) * 2021-05-28 2023-01-10 新疆爱华盈通信息技术有限公司 一种图像分类方法、装置、电子设备及存储介质
CN113836790B (zh) * 2021-08-25 2024-02-02 成都鲁易科技有限公司 电动单车智能化等级的测评方法、装置及计算机设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108229543A (zh) * 2017-12-22 2018-06-29 中国科学院深圳先进技术研究院 图像分类模型设计方法及装置
CN108764370A (zh) * 2018-06-08 2018-11-06 Oppo广东移动通信有限公司 图像处理方法、装置、计算机可读存储介质和计算机设备
US10127475B1 (en) * 2013-05-31 2018-11-13 Google Llc Classifying images
CN109829849A (zh) * 2019-01-29 2019-05-31 深圳前海达闼云端智能科技有限公司 一种训练数据的生成方法、装置和终端

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106920243B (zh) * 2017-03-09 2019-12-17 桂林电子科技大学 改进的全卷积神经网络的陶瓷材质件序列图像分割方法
CN108230354B (zh) * 2017-05-18 2022-05-10 深圳市商汤科技有限公司 目标跟踪、网络训练方法、装置、电子设备和存储介质
WO2019095118A1 (zh) * 2017-11-14 2019-05-23 深圳和而泰智能控制股份有限公司 一种皮肤瑕疵点分类方法及电子设备
CN109145979B (zh) * 2018-08-15 2022-06-21 上海嵩恒网络科技股份有限公司 敏感图像鉴定方法及终端系统
CN110096960B (zh) * 2019-04-03 2021-06-08 罗克佳华科技集团股份有限公司 目标检测方法及装置
CN110188835B (zh) * 2019-06-05 2021-03-16 国家广播电视总局广播电视科学研究院 基于生成式对抗网络模型的数据增强行人再识别方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10127475B1 (en) * 2013-05-31 2018-11-13 Google Llc Classifying images
CN108229543A (zh) * 2017-12-22 2018-06-29 中国科学院深圳先进技术研究院 图像分类模型设计方法及装置
CN108764370A (zh) * 2018-06-08 2018-11-06 Oppo广东移动通信有限公司 图像处理方法、装置、计算机可读存储介质和计算机设备
CN109829849A (zh) * 2019-01-29 2019-05-31 深圳前海达闼云端智能科技有限公司 一种训练数据的生成方法、装置和终端

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113222899A (zh) * 2021-04-15 2021-08-06 浙江大学 一种基于深度学习通过ct的检测分割分类肝脏肿瘤的方法
CN113313729A (zh) * 2021-05-26 2021-08-27 惠州中国科学院遥感与数字地球研究所空间信息技术研究院 单极对象图像成像方法、装置、计算机设备和存储介质
CN113673332A (zh) * 2021-07-15 2021-11-19 浙江大华技术股份有限公司 对象识别方法、装置以及计算机可读存储介质
CN113408662A (zh) * 2021-07-19 2021-09-17 北京百度网讯科技有限公司 图像识别、图像识别模型的训练方法和装置
CN113591736A (zh) * 2021-08-03 2021-11-02 北京百度网讯科技有限公司 特征提取网络、活体检测模型的训练方法和活体检测方法
CN113516201A (zh) * 2021-08-09 2021-10-19 中国农业大学 一种基于深度神经网络的肉兔料盒中余料量的估算方法
CN113516201B (zh) * 2021-08-09 2023-10-31 中国农业大学 一种基于深度神经网络的肉兔料盒中余料量的估算方法
CN113537248A (zh) * 2021-08-13 2021-10-22 珠海格力电器股份有限公司 图像识别方法和装置、电子设备和存储介质
CN113610026A (zh) * 2021-08-13 2021-11-05 广联达科技股份有限公司 基于掩膜注意力的行人重识别方法及重识别装置
CN113537248B (zh) * 2021-08-13 2024-06-07 珠海格力电器股份有限公司 图像识别方法和装置、电子设备和存储介质
CN113642515A (zh) * 2021-08-30 2021-11-12 北京航空航天大学 基于姿态关联的行人识别方法与装置、电子设备和介质
CN113642515B (zh) * 2021-08-30 2023-11-24 北京航空航天大学 基于姿态关联的行人识别方法与装置、电子设备和介质
CN113837205A (zh) * 2021-09-28 2021-12-24 北京有竹居网络技术有限公司 用于图像特征表示生成的方法、设备、装置和介质
CN113837205B (zh) * 2021-09-28 2023-04-28 北京有竹居网络技术有限公司 用于图像特征表示生成的方法、设备、装置和介质
CN114119976A (zh) * 2021-11-30 2022-03-01 广州文远知行科技有限公司 语义分割模型训练、语义分割的方法、装置及相关设备
CN114119976B (zh) * 2021-11-30 2024-05-14 广州文远知行科技有限公司 语义分割模型训练、语义分割的方法、装置及相关设备
CN113989857A (zh) * 2021-12-27 2022-01-28 四川新网银行股份有限公司 一种基于深度学习的人像照片内容解析方法及系统
CN114661904A (zh) * 2022-03-10 2022-06-24 北京百度网讯科技有限公司 文档处理模型的训练方法、装置、设备、存储介质及程序
WO2024125380A1 (zh) * 2022-12-13 2024-06-20 广电运通集团股份有限公司 分类识别方法、计算机设备和存储介质

Also Published As

Publication number Publication date
CN112446398B (zh) 2024-09-10
CN112446398A (zh) 2021-03-05

Similar Documents

Publication Publication Date Title
WO2021043112A1 (zh) 图像分类方法以及装置
WO2020253416A1 (zh) 物体检测方法、装置和计算机存储介质
WO2021043168A1 (zh) 行人再识别网络的训练方法、行人再识别方法和装置
CN110084281B (zh) 图像生成方法、神经网络的压缩方法及相关装置、设备
US20220092351A1 (en) Image classification method, neural network training method, and apparatus
WO2021120719A1 (zh) 神经网络模型更新方法、图像处理方法及装置
WO2021147325A1 (zh) 一种物体检测方法、装置以及存储介质
WO2021155792A1 (zh) 一种处理装置、方法及存储介质
WO2022042713A1 (zh) 一种用于计算设备的深度学习训练方法和装置
WO2021022521A1 (zh) 数据处理的方法、训练神经网络模型的方法及设备
WO2021164750A1 (zh) 一种卷积层量化方法及其装置
CN113011562B (zh) 一种模型训练方法及装置
US12039440B2 (en) Image classification method and apparatus, and image classification model training method and apparatus
WO2022001805A1 (zh) 一种神经网络蒸馏方法及装置
WO2021164751A1 (zh) 一种感知网络结构搜索方法及其装置
CN110222718B (zh) 图像处理的方法及装置
WO2021073311A1 (zh) 图像识别方法、装置、计算机可读存储介质及芯片
CN111797881B (zh) 图像分类方法及装置
WO2021175278A1 (zh) 一种模型更新方法以及相关装置
WO2022179606A1 (zh) 一种图像处理方法及相关装置
WO2022217434A1 (zh) 感知网络、感知网络的训练方法、物体识别方法及装置
CN112464930A (zh) 目标检测网络构建方法、目标检测方法、装置和存储介质
WO2022179599A1 (zh) 一种感知网络及数据处理方法
CN113449550A (zh) 人体重识别数据处理的方法、人体重识别的方法和装置
CN114693986A (zh) 主动学习模型的训练方法、图像处理方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20860249

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20860249

Country of ref document: EP

Kind code of ref document: A1