WO2024251089A1 - 图像处理方法、装置及电子设备 - Google Patents

图像处理方法、装置及电子设备 Download PDF

Info

Publication number
WO2024251089A1
WO2024251089A1 PCT/CN2024/097120 CN2024097120W WO2024251089A1 WO 2024251089 A1 WO2024251089 A1 WO 2024251089A1 CN 2024097120 W CN2024097120 W CN 2024097120W WO 2024251089 A1 WO2024251089 A1 WO 2024251089A1
Authority
WO
WIPO (PCT)
Prior art keywords
key point
image
probability distribution
prediction model
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/097120
Other languages
English (en)
French (fr)
Inventor
胡青文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2024251089A1 publication Critical patent/WO2024251089A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the embodiments of the present disclosure relate to the field of image processing technology, and in particular, to an image processing method, device and electronic device.
  • the electronic device can recognize the object in the image by detecting the key points in the image. For example, the electronic device can extract the key points in the image, determine the outline of the object based on the key points, and then recognize the object in the image based on the outline.
  • the present disclosure provides an image processing method, the method comprising: acquiring an image including an object, the object being associated with a plurality of key points; processing the image according to a key point prediction model to obtain a probability distribution map corresponding to each key point, the key point prediction model being trained based on a preset probability distribution and an error probability distribution determined by a normalized flow; determining the position of each key point in the image according to the probability distribution map corresponding to each key point.
  • the present disclosure provides an image processing device, which includes an acquisition module, a processing module and a determination module, wherein: the acquisition module is used to acquire an image including an object, wherein the object is associated with a plurality of key points; the processing module is used to process the image according to a key point prediction model to obtain a probability distribution map corresponding to each key point, wherein the key point prediction model is obtained by training based on a preset probability distribution and an error probability distribution determined by a normalized flow; the determination module is used to determine a probability distribution map corresponding to each key point according to the key point prediction model. The probability distribution map corresponding to the point determines the position of each key point in the image.
  • an embodiment of the present disclosure provides an electronic device comprising: a processor and a memory; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the image processing method as described in the first aspect above and various possible aspects involved in the first aspect.
  • an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored.
  • a processor executes the computer execution instructions, the image processing method as described in the first aspect and various possible aspects of the first aspect are implemented.
  • the present disclosure provides an image processing method, device and electronic device.
  • the electronic device can obtain an image including an object, wherein the object is associated with multiple key points.
  • the image is processed according to a key point prediction model to obtain a probability distribution map corresponding to each key point, wherein the key point prediction model is trained based on a preset probability distribution and an error probability distribution determined by a standardized flow.
  • the probability distribution map corresponding to each key point the position of each key point in the image is determined.
  • the error probability distribution determined by the standardized flow can supplement the preset probability distribution.
  • the key point prediction model can learn the complex distribution corresponding to the key point, and the accuracy of the probability distribution map corresponding to each key point output by the key point prediction model is high, which improves the accuracy of determining the position of the key point.
  • the electronic device can accurately calculate the position coordinates of the key point based on the probability distribution map, there is no need to render a high-resolution Gaussian heat map. Therefore, the amount of calculation for determining the key point can be reduced and the efficiency of determining the key point can be improved.
  • FIG1 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure.
  • FIG2 is a schematic diagram of a flow chart of an image processing method provided by an embodiment of the present disclosure
  • FIG3 is a schematic diagram of a key point provided by an embodiment of the present disclosure.
  • FIG4 is a schematic diagram of the structure of a key point prediction model provided by an embodiment of the present disclosure.
  • FIG5 is a schematic diagram of a probability distribution diagram provided by an embodiment of the present disclosure.
  • FIG6 is a schematic diagram of a process for determining a probability distribution graph provided by an embodiment of the present disclosure
  • FIG7 is a schematic diagram of a process for determining target coordinates of key points provided by an embodiment of the present disclosure
  • FIG8 is a schematic diagram of a process of an image processing method provided by an embodiment of the present disclosure.
  • FIG9 is a schematic diagram of a training method for a key point prediction model provided by an embodiment of the present disclosure.
  • FIG10 is a schematic diagram of a key point prediction model provided by an embodiment of the present disclosure.
  • FIG11 is a schematic diagram of a training framework of a key point prediction model provided by an embodiment of the present disclosure.
  • FIG12 is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure.
  • FIG13 is a schematic diagram of the structure of another image processing device provided by an embodiment of the present disclosure.
  • FIG. 14 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.
  • the present disclosure provides an image processing method, device and electronic device, which are used to solve the technical problem of low efficiency in determining key points in the related art.
  • Electronic device a device with wireless transceiver function.
  • Electronic devices can be deployed on land, including indoors or outdoors, handheld, wearable or vehicle-mounted.
  • the electronic device can be a mobile phone, a tablet computer, a computer with wireless transceiver function, a virtual Virtual reality (VR) electronic devices, augmented reality (AR) electronic devices, wireless terminals in industrial control, vehicle-mounted electronic devices, wireless terminals in self-driving, wireless electronic devices in remote medical, wireless electronic devices in smart grid, wireless electronic devices in transportation safety, wireless electronic devices in smart city, wireless electronic devices in smart home, wearable electronic devices, etc.
  • VR virtual Virtual reality
  • AR augmented reality
  • the electronic devices involved in the embodiments of the present disclosure may also be referred to as terminals, user equipment (UE), access electronic devices, vehicle-mounted terminals, industrial control terminals, UE units, UE stations, mobile stations, mobile stations, remote stations, remote electronic devices, mobile devices, UE electronic devices, wireless communication devices, UE agents or UE devices, etc.
  • the electronic devices may also be fixed or mobile.
  • FIG1 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure.
  • the display screen of the electronic device includes image A, and image A may include an object.
  • the electronic device may perform key point extraction processing on image A to obtain image B.
  • Image B may include an object, and image B may include multiple key points associated with the object.
  • image B may include 14 key points, and the 14 key points may be the head key points, hand key points, leg key points, etc. of the object. In this way, the electronic device can accurately determine information such as the object and the outline of the object in image B through the 14 key points.
  • an electronic device can detect key points in an image and identify objects in the image through the key points. For example, an electronic device can detect multiple key points in a face image, and then identify the location of the face in the face image. At present, an electronic device can determine the key points in an image based on a Gaussian heat map. For example, an electronic device can process an image based on a pre-trained model to obtain a Gaussian heat map corresponding to each key point in the image, and determine the maximum point in the Gaussian heat map as the position estimation result of the key point. For example, in the embodiment shown in FIG. 1, the electronic device can generate 14 Gaussian heat maps, and the position of the maximum point in each Gaussian heat map is different.
  • the embodiment of the present disclosure provides an image processing method, and an electronic device can obtain an image including an object, and the object in the image is associated with multiple key points.
  • the electronic device processes the image according to the key point prediction model to obtain multiple feature maps associated with the key points, and normalizes each feature map to obtain a probability distribution map corresponding to each key point.
  • the probability distribution map corresponding to each key point the position of each key point in the image is determined.
  • the key point prediction model can learn the accurate probability distribution map corresponding to the key point based on the error probability distribution during the training stage, thereby improving the accuracy of determining the key point position.
  • the electronic device can calculate the position of the key point in the image based on the probability distribution map, there is no need to generate a Gaussian heat map with a higher resolution. In this way, the amount of calculation for determining the position of the key point can be reduced, and the efficiency of determining the key point can be improved.
  • FIG2 is a flow chart of an image processing method provided by an embodiment of the present disclosure. Referring to FIG2 , the method may include:
  • S201 Acquire an image including an object.
  • the execution subject of the embodiment of the present disclosure may be an electronic device, or an image processing device disposed in the electronic device.
  • the image processing device may be implemented based on software, or the image processing device may be implemented based on a combination of software and hardware, which is not limited in the embodiment of the present disclosure.
  • the image may include an object.
  • the image may include any object such as a user, a face, or a fruit, which is not limited in the embodiments of the present disclosure.
  • the electronic device may receive an image sent by another device.
  • the electronic device may receive an image sent by a server, which may include a face image.
  • the electronic device may receive a A request for key point detection, which may include an image to be detected.
  • the electronic device can acquire an image including an object according to any feasible implementation method, and the embodiments of the present disclosure are not limited to this.
  • an object in an image may be associated with multiple key points.
  • an object in an image may have multiple key points.
  • a face image may have multiple key points
  • a user image may also have multiple key points. It should be noted that in the actual application process, the image obtained by the electronic device is not marked with multiple key points, and the electronic device can obtain multiple key points in the image by processing the image.
  • different objects in the image may have different key points associated with the objects.
  • the face image may be associated with 30 key points (e.g., top of head key points, ear key points, etc.).
  • the image may be associated with 63 key points (e.g., right shoulder key points, right leg key points, etc.).
  • each part may be associated with multiple key points.
  • the top of the user's head may be associated with 1 key point
  • the user's left arm may be associated with 2 key points
  • the user's right leg may be associated with 3 key points, and so on.
  • the multiple key points associated with the object can be predetermined key points.
  • the electronic device can pre-set that the face image can include up to 30 key points, and can pre-set the position of the face indicated by each key point.
  • the first key point in the face image is the key point at the top of the face
  • the 30th key point in the face image is the key point at the bottom of the face.
  • the electronic device can pre-set that the image including the human body can include up to 63 key points, and the outline of the human body can be determined by the 63 key points.
  • the electronic device can pre-set the position of the human body indicated by each key point.
  • the 20th key point in the image including the human body is the key point of the left hand
  • the 40th key point is the key point of the right hand, and so on, which is not limited in the embodiments of the present disclosure.
  • FIG3 is a schematic diagram of a key point provided by an embodiment of the present disclosure.
  • a human body image and a face image are included.
  • the object in the human body image is a human body
  • the object in the face image is a face.
  • the human body image may include multiple key points associated with the human body
  • the face image may include multiple key points associated with the human body.
  • the image also includes a plurality of key points associated with a human face.
  • different objects in the image have different key points associated with the objects.
  • the application scenario of a human body image may be human body contour detection.
  • a plurality of key points in a human body image may be located at the edge of a human body in the image, and the application scenario of a face image may be face recognition. Therefore, a plurality of key points in a face image may be located at a position having a face feature in the image.
  • S202 Process the image according to the key point prediction model to obtain a probability distribution map corresponding to each key point.
  • the key point prediction model can obtain key points in the image.
  • the key point prediction model can process the image and output the image after the key points are marked.
  • Different key point prediction models are targeted at different objects.
  • key point prediction model A can process an image including a human body and obtain multiple key points in the image.
  • Key point prediction model B can process an image including an animal and obtain multiple key points in the image.
  • the key point prediction model can process images including human bodies, then any image including human bodies is input to the key point prediction model, and the key point prediction model can output an image with 63 key points marked.
  • the key point prediction model can process images including faces, then any image including faces is input to the key point prediction model, and the key point prediction model can output an image with 30 key points marked.
  • the key point prediction model can process images including human bodies
  • the key point prediction model can also process images including human faces and limbs.
  • the key point prediction model can determine 63 key points in an image including human bodies.
  • the human face can include 20 key points
  • the key point prediction model processes the face image, the face image including 20 key points can be annotated.
  • the key point prediction model is trained based on a preset probability distribution and an error probability distribution determined by a standardized flow.
  • the preset probability distribution may be a Gaussian distribution or a Laplace distribution.
  • the key point prediction model may learn the Gaussian distribution or the Laplace distribution corresponding to the key points. Since the actual distribution of the key points is relatively complex, an error probability distribution may be determined by the standardized flow, and the Gaussian distribution or the Laplace distribution may be supplemented based on the error probability distribution, so that the key point prediction model can be trained based on the preset probability distribution and the Gaussian distribution or the Laplace distribution may be supplemented based on the error probability distribution.
  • the model can fit the real distribution with a high degree of match with the key points.
  • the structure of the key point prediction model is described below in conjunction with FIG. 4 .
  • Figure 4 is a schematic diagram of the structure of a key point prediction model provided by an embodiment of the present disclosure.
  • the key point prediction model may include a convolutional neural network (CNN), a 1*1 convolutional layer (1*1conv) and a normalization layer (softmax layer).
  • CNN convolutional neural network
  • the convolutional neural network CNN can downsample the input image to obtain multiple feature maps. If the number of feature maps output by the convolutional neural network CNN (that is, the number of channels of the feature map) is greater than the number of key points associated with the objects in the image, the number of feature maps can be made the same as the number of key points based on the 1*1 convolutional layer.
  • the normalization layer can process the feature map, and then the probability distribution map associated with each feature map can be obtained.
  • the probability distribution graph can be used to indicate the distribution probability of key points in the image.
  • multiple key points associated with an object are distributed at different positions in the image.
  • the closer a certain position in the image is to the actual position of the key point the higher the probability value of the position.
  • the probability distribution diagram is described below in conjunction with FIG5 .
  • FIG5 is a schematic diagram of a probability distribution diagram provided by an embodiment of the present disclosure. Please refer to FIG5, which includes: a probability distribution diagram.
  • the probability distribution diagram may include 25 points, each of which may correspond to a probability value, which may be the probability that the key point appears at that position.
  • the sum of the probability values corresponding to the 25 points is 1. The closer the point is to the key point position, the higher the probability value. The farther the point is from the key point position, the lower the probability value. It can be seen from the embodiment shown in FIG5 that the position of the key point corresponding to the probability distribution diagram is close to the position of the coordinates (4, 3).
  • the electronic device may obtain the probability distribution map corresponding to each key point according to the following feasible implementation method: the image is processed according to the key point prediction model to obtain multiple feature maps associated with the key point, and each feature map is normalized to obtain the probability distribution map corresponding to each key point. For example, the electronic device performs downsampling processing on the image based on the CNN network in the key point prediction model, and the CNN network can output multiple feature maps corresponding to the image.
  • the key point prediction model is a pre-trained model.
  • the number of key points and the types of key points (e.g., head key points, hand key points, or leg key points, etc.) predicted by the prediction model are also determined during pre-training.
  • FIG6 is a schematic diagram of a process for determining a probability distribution diagram provided by an embodiment of the present disclosure.
  • the key point prediction model may include a convolutional neural network, a 1*1 convolutional layer, and a normalization layer (not shown in FIG6).
  • the size of the image may be 192*144, and the downsampling multiple of the convolutional neural network in the key point prediction model is 16 times.
  • An electronic device (not shown in FIG6 ) can input an image to a key point prediction model. After the convolutional neural network in the key point prediction model processes the image, multiple feature maps can be obtained. The size of the feature map is 12*9. After multiple feature maps are input to a 1*1 convolutional layer, 63 feature maps can be obtained, and the number of key points can be 63.
  • probability distribution map 1, probability distribution map 2, ..., probability distribution map 63 can be obtained.
  • Each probability distribution map can correspond to the probability distribution of a key point. Since the key point prediction model can learn a complex distribution that is closer to the true probability distribution of the key point during the training phase, the probability distribution map of the key point obtained based on the key point prediction model has a higher accuracy.
  • S203 Determine the position of each key point in the image according to the probability distribution graph corresponding to each key point.
  • the electronic device can determine the position of each key point in the image according to the following feasible implementation method: for any key point, obtain the first coordinate of the point corresponding to each probability value in the probability distribution map corresponding to the key point, and determine the position of the key point in the image according to each probability value and the first coordinate corresponding to each probability value.
  • the probability distribution graph may include multiple points corresponding to the probability values, and the first coordinate may be the coordinate of the point in the probability distribution graph.
  • the probability distribution graph includes 25 probability values, and the probability distribution graph may include 25 points. Therefore, the range of the horizontal coordinate of the point in the probability distribution graph may be 1-5, and the range of the vertical coordinate may be 1-5.
  • the first coordinate corresponding to the point with a probability value of 0.6 may be (4, 3)
  • the first coordinate corresponding to the point with a probability value of 0.1 may be (4, 2), (3, 3), (5, 3) and (4, 4) respectively.
  • the electronic device may obtain the first coordinates of the point corresponding to each pixel value in the probability distribution diagram, and the electronic device may also determine the first coordinates according to any other feasible implementation method, which is not limited in the embodiments of the present disclosure.
  • the position of the key point in the image is determined according to each probability value and the first coordinate corresponding to each probability value. Specifically, each probability value is multiplied by the first coordinate corresponding to each probability value to obtain the second coordinate corresponding to each probability value, multiple second coordinates are added to obtain the target coordinates of the key point, and the position of the key point in the image is determined based on the target coordinates.
  • the second coordinate corresponding to the probability value can be the coordinate after the probability value is multiplied by the first coordinate.
  • the first coordinate corresponding to the probability value is (4, 3). Therefore, after multiplying the first coordinate by the probability value, the coordinate (2.4, 1.8) can be obtained, and the electronic device can determine the coordinate (2.4, 1.8) as the second coordinate corresponding to the probability value.
  • the first coordinates corresponding to the four probability values of 0.1 can be (4, 2), (3, 3), (5, 3) and (4, 4) respectively. Therefore, the second coordinates corresponding to the four probability values of 0.1 can be (0.4, 0.2), (0.3, 0.3), (0.5, 0.3) and (0.4, 0.4) respectively.
  • the target coordinate of the key point may be the sum of multiple second coordinates corresponding to the key point.
  • the electronic device may add the horizontal coordinates of the multiple second coordinates corresponding to the key point to obtain the horizontal coordinate of the target coordinate corresponding to the key point, and add the vertical coordinates of the multiple second coordinates corresponding to the key point to obtain the vertical coordinate of the target coordinate corresponding to the key point.
  • FIG7 is a schematic diagram of a process for determining the target coordinates of a key point provided by an embodiment of the present disclosure.
  • the probability distribution diagram includes 16 probability values, and each probability value corresponds to a point in the probability distribution diagram.
  • An electronic device (not shown in FIG7) can determine a second coordinate based on each probability value and the first coordinate of the point corresponding to the probability value.
  • multiple second coordinates may include (0.2, 0.2), (0.09, 0.09), (0.4, 0.3), (0.06, 0.06), (0.3, 0.2), (2.4, 1.2), (0.06, 0), (0.28, 0).
  • the electronic device can add the abscissas of the plurality of second coordinates to obtain 3.79, and add the ordinates of the plurality of second coordinates to obtain 2.05, and thus determine that the key point corresponds to the target point.
  • the coordinates are (3.79, 2.05).
  • the electronic device can determine the accurate coordinates corresponding to each key point.
  • the decimal point of the coordinates can be retained for multiple digits (such as the decimal point of the probability value in the probability distribution diagram can be retained for multiple digits), the electronic device can obtain more accurate target coordinates, thereby improving the accuracy of determining the position of the key point.
  • the electronic device determines the position of the key point in the image based on the target coordinates.
  • the method may include obtaining a downsampling multiple when the key point prediction model processes the image, and determining the position of the key point in the image based on the downsampling multiple and the target coordinates.
  • the downsampling multiple may be a downsampling multiple of a convolutional neural network in the key point prediction model.
  • the CNN network in the key point prediction model may downsample an image. If the downsampling multiple of the CNN network is 16 (4*4) times and the image size is 192*144, the size of the downsampled image is 12*9.
  • the electronic device can determine the position of the key point in the image based on the target coordinates and the downsampling multiple. For example, if the target coordinates corresponding to the key point are (1, 1) and the downsampling multiple is 16 (4*4), the position coordinates of the key point in the image can be (4, 4). That is, the horizontal coordinate of the target coordinate is multiplied by 4, and the vertical coordinate of the target coordinate is multiplied by 4. If the coordinates corresponding to the key point are (2, 3) and the downsampling multiple is 25 (5*5), the position coordinates of the key point in the image can be (10, 15). That is, the horizontal coordinate of the target coordinate is multiplied by 5, and the vertical coordinate of the target coordinate is multiplied by 5.
  • the disclosed embodiment provides an image processing method, in which an electronic device can obtain an image including an object, and the electronic device processes the image according to a key point prediction model to obtain a plurality of feature maps associated with the key points, and normalizes each feature map to obtain a probability distribution map corresponding to each key point, and determines the position of each key point in the image according to the probability distribution map corresponding to each key point.
  • a key point prediction model since the key point prediction model is trained based on a preset probability distribution and an error probability distribution determined by a normalized flow. Therefore, the key point prediction model can learn the complex distribution associated with the key point, thereby improving the accuracy of the probability distribution map corresponding to the key point, and improving the accuracy of determining the position of the key point.
  • the electronic device can calculate the position of the key point in the image based on the probability distribution map, there is no need to generate a Gaussian heat map with a higher resolution, thereby reducing the amount of calculation for determining the position of the key point and improving the efficiency of determining the key point.
  • FIG8 is a process diagram of an image processing method provided by an embodiment of the present disclosure. Please refer to FIG8, including: including image A and a key point prediction model, image A can be a human body image.
  • An electronic device (not shown in FIG8) can input image A to the key point prediction model, and the CNN network in the key point prediction model can downsample image A by 4*4 times, and the key point prediction model can output probability distribution graph 1, probability distribution graph 2, ..., probability distribution graph 14, wherein the 14 probability distribution graphs correspond to the key points one by one.
  • the electronic device can determine the second coordinate corresponding to each probability value of the probability distribution diagram, and determine the target coordinate associated with the key point based on the multiple second coordinates. For example, the electronic device can obtain the target coordinate 1 corresponding to the key point 1 based on the probability distribution diagram 1, and the electronic device can obtain the target coordinate 2 corresponding to the key point 2 based on the probability distribution diagram 2, etc.
  • the electronic device can multiply the horizontal coordinate and the vertical coordinate of each target coordinate by 4 respectively, and can obtain 14 positions.
  • the electronic device can determine the 14 positions in image A and mark key points at each position, thereby obtaining image B.
  • Image B may include a human body and 14 key points associated with the human body.
  • the electronic device can accurately determine the probability distribution map associated with each key point based on the key point prediction model.
  • the target coordinates are determined based on all parameters (probability values, coordinates of probability values) in the probability distribution map, the accuracy of the target coordinates is high, which can improve the accuracy of determining the key points.
  • the key points are determined based on the position coordinates, the accuracy of the key points is high, and there is no need to render a Gaussian heat map. The computational complexity can be reduced and the efficiency of determining the key points can be improved.
  • the embodiments of the present disclosure further include a training method for a key point prediction model.
  • the training method for the key point prediction model is described below in conjunction with FIG. 9 .
  • FIG9 is a schematic diagram of a training method for a key point prediction model provided by an embodiment of the present disclosure. Referring to FIG9 , the method flow includes:
  • the sample image may include a sample object.
  • the sample image may include Human body images, human face images, etc.
  • the sample objects in the sample images are the same as the objects in the images during the key point prediction model use process.
  • the key point prediction model can detect key points in human face images
  • the sample objects in the sample images can be human faces.
  • the key point prediction model can detect key points in the human body, then during the key point prediction model training process, the sample objects in the sample images can be human bodies.
  • the electronic device can obtain the sample image according to any feasible implementation method, and the embodiments of the present disclosure are not limited to this.
  • the sample key points may be key points pre-set for the sample object. For example, if the sample object is a face, the sample object may be associated with 30 key points. If the sample object is a human image, the sample object may be associated with 63 key points.
  • sample key points associated with the sample objects in the sample images may be pre-set key points, or may be sample key points determined by the electronic device according to any other feasible implementation method, and the embodiments of the present disclosure do not limit this. Furthermore, the embodiments of the present disclosure do not limit information such as the number of key points associated with the sample objects, the positions of the key points, and the types of the key points.
  • the sample position may be the position of the sample key point in the sample image.
  • the user may accurately mark each sample key point in the sample image, and then determine the sample position of the sample key point in the sample image.
  • the sample position may be represented based on coordinates (such as a coordinate system in pixels), or may be represented based on any other feasible implementation method, which is not limited in the embodiments of the present disclosure.
  • S903 Update the model parameters of the key point prediction model according to the sample image and the sample position.
  • the electronic device may update the model parameters of the key point prediction model according to the following feasible implementation method: the sample image is processed according to the key point prediction model to obtain the predicted position corresponding to each sample key point and the variance associated with the predicted position.
  • the error parameter is determined according to the predicted position, variance and sample position corresponding to each sample key point.
  • the model parameters of the key point prediction model are updated according to the error parameter.
  • the predicted position can be the position of the sample key point predicted by the key point prediction model based on the sample image.
  • the electronic device can process the sample image to obtain a sample probability distribution map associated with each sample key point. And based on the sample probability distribution map, determine the position of each sample key point in the sample image. It should be noted that the method by which the electronic device determines the predicted position of each sample key point is the same as the method by which the electronic device in the embodiment shown in FIG. 2 determines the position of each key point in the image, and the embodiments of the present disclosure will not be repeated here.
  • the variance associated with the predicted position may be the variance of the predicted distribution of the key points.
  • the variance associated with the predicted position may indicate the degree of dispersion of the predicted value.
  • the electronic device may process the sample image based on the key point prediction model, and then obtain the variance associated with the predicted position.
  • the key point prediction model may set a branch for training variance during the training phase, and the structure of the branch may include an adaptive average pooling layer and a fully connected layer, so that the variance associated with the predicted position can be obtained based on the branch.
  • the structure of the key point prediction model in the training phase is described below in conjunction with FIG. 10 .
  • FIG10 is a schematic diagram of a key point prediction model provided by an embodiment of the present disclosure.
  • the key point prediction model includes a convolutional neural network CNN.
  • Two branches can be connected after the convolutional neural network, one of which includes a 1*1 convolution layer and a normalization layer, and the branch can output a probability distribution map associated with each key point.
  • the other branch includes an adaptive average pooling layer and a fully connected layer, and the branch can output the variance associated with the predicted position of each key point.
  • the output result can be transformed into a three-dimensional vector of (N, K, 2).
  • the three-dimensional vector can be the variance of the key point prediction distribution
  • N represents the batch size (data size) of the convolutional neural network
  • K represents the number of key points
  • 2 can be expressed as the variance of the x coordinate and the variance of the y coordinate.
  • the error parameter may be a variable parameter in the key point prediction model training process.
  • the error parameter may be determined based on the predicted position, variance and sample position corresponding to the sample key point, specifically, the position difference between the sample position and the predicted position may be determined, and the ratio of the position difference to the variance may be determined as the error parameter.
  • the electronic device may determine the error parameter based on the following formula:
  • A can be an error parameter
  • B can be a sample position
  • C can be a predicted position
  • ⁇ can be a variance
  • the electronic device updates the model parameters of the key point prediction model according to the error parameters, which can be specifically: processing the sample image according to the key point prediction model to obtain the preset probability distribution corresponding to the sample key point. Transforming the preset probability distribution according to the normalized flow model to obtain the error probability distribution corresponding to the sample key point. Constructing a loss function according to the preset probability distribution, the error probability distribution, the error parameter and the variance. And updating the model parameters of the key point prediction model according to the loss function.
  • the preset probability distribution may be a simple distribution such as a Gaussian distribution or a Laplace distribution, which is not limited in the embodiments of the present disclosure.
  • the standardized flow model can perform a reversible transformation on the probability distribution.
  • the standardized flow model can have the function of standardized flow, which can transform any complex distribution into a simple distribution (e.g., transform a complex distribution into a Gaussian distribution), and the standardized flow can also transform any simple distribution into a complex distribution.
  • the transformation process of the probability distribution by the standardized flow is reversible.
  • a normalized flow model may be introduced during the training of the key point prediction model.
  • the key point prediction model is used to learn a preset probability distribution corresponding to the key point, which has a difference with the true distribution corresponding to the key point.
  • the normalized flow model is used to learn an error probability distribution, which is a supplement to the preset probability distribution, so that the preset probability distribution is close to the true distribution of the key point.
  • FIG11 is a schematic diagram of a training framework of a key point prediction model provided by an embodiment of the present disclosure.
  • the sample image is processed according to the key point prediction model to obtain the predicted position and variance corresponding to the key point.
  • the preset probability distribution is transformed according to the standardized flow model to obtain the error probability distribution.
  • the probability distribution of the key points is obtained by predicting the position, variance and error probability distribution. For example, the predicted position is ⁇ , the variance is ⁇ , and the error probability distribution of the normalized flow model fitting is Then the probability distribution corresponding to the key point can be:
  • L may be a loss function
  • A may be an error parameter
  • Q may be a preset probability distribution (such as Gaussian distribution or Laplace distribution, etc.)
  • G may be a normalized flow
  • s may be a constant (s is used for correction and may be set arbitrarily, and the embodiments of the present disclosure do not limit this)
  • ⁇ may be a variance.
  • the loss function may be determined according to the above formula, and the model parameters may be updated according to the loss function.
  • the disclosed embodiment provides a method for training a key point prediction model, which obtains a sample image and obtains the sample position of each sample key point in the sample image.
  • the sample image is processed according to the key point prediction model to obtain the predicted position corresponding to each sample key point and the variance associated with the predicted position.
  • the error parameter is determined according to the predicted position, variance and sample position corresponding to each sample key point.
  • the model parameters of the key point prediction model are updated according to the error parameter.
  • the key point prediction model can learn the complex distribution associated with the key points. Therefore, the training accuracy of the key point prediction model is high, which improves the accuracy of the probability distribution map corresponding to the key points generated by the key point prediction model, and thus can improve the accuracy of determining the key point positions.
  • FIG12 is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure.
  • the image processing device 120 includes an acquisition module 121, a processing module 122 and a determination module 123, wherein:
  • the acquisition module 121 is used to acquire an image including an object, wherein the object is associated with a plurality of key points;
  • the processing module 122 is used to process the image according to the key point prediction model to obtain a probability distribution map corresponding to each key point, wherein the key point prediction model is obtained by training based on a preset probability distribution and an error probability distribution determined by a standardized flow;
  • the determination module 123 is used to determine the position of each key point in the image according to the probability distribution graph corresponding to each key point.
  • the processing module 122 is specifically used for:
  • Each feature map is normalized to obtain a probability distribution map corresponding to each key point.
  • the determining module 123 is specifically configured to:
  • the position of the key point in the image is determined according to each probability value and the first coordinate corresponding to each probability value.
  • the determining module 123 is specifically configured to:
  • the position of the key point in the image is determined.
  • the image processing device provided in the embodiment of the present disclosure may be used to execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.
  • FIG13 is a schematic diagram of the structure of another image processing device provided by an embodiment of the present disclosure. Based on the embodiment shown in FIG12 , referring to FIG13 , the image processing device 120 further includes a training module 124, wherein the training module 124 is used to:
  • the model parameters of the key point prediction model are updated according to the sample image and the sample position.
  • the training module 124 is specifically used for:
  • the model parameters of the key point prediction model are updated according to the error parameter and the variance.
  • the training module 124 is specifically used for:
  • a loss function is constructed according to the preset probability distribution, the error probability distribution, the error parameter and the variance, and the model parameters of the key point prediction model are updated according to the loss function.
  • the image processing device provided in the embodiment of the present disclosure may be used to execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.
  • FIG14 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Please refer to FIG14, which shows a schematic diagram of the structure of an electronic device 1400 suitable for implementing an embodiment of the present disclosure, and the electronic device 1400 can be any device with terminal computing capabilities.
  • the electronic device shown in FIG14 is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present disclosure.
  • the electronic device 1400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1401, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1402 or a program loaded from a storage device 1408 to a random access memory (RAM) 1403.
  • a processing device 1401 e.g., a central processing unit, a graphics processing unit, etc.
  • RAM random access memory
  • Various programs and data required for the operation of the electronic device 1400 are also stored in the RAM 1403.
  • the processing device 1401, the ROM 1402, and the RAM 1403 are connected to each other via a bus 1404.
  • An input/output (I/O) interface 1405 is also connected to the bus 1404.
  • the following devices may be connected to the I/O interface 1405: input devices 1406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 1407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1408 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1409.
  • the communication devices 1409 may allow the electronic device 1400 to communicate with other devices.
  • FIG. 14 shows an electronic device 1400 having various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program can be downloaded and installed from the network through the communication device 1409, or installed from the storage device 1408, or installed from the ROM 1402.
  • the processing device 1401 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried.
  • This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above.
  • the computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device.
  • the program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
  • the computer-readable medium carries one or more programs.
  • the electronic device executes the method shown in the above embodiment.
  • An embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored.
  • a processor executes the computer-executable instructions, the image processing method as described in the first aspect and various possible aspects of the first aspect are implemented.
  • An embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and various image processing methods that may be involved in the first aspect.
  • Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages.
  • the program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
  • LAN Local Area Network
  • WAN Wide Area Network
  • each box in the flowchart or block diagram may represent a module, a program segment, or a portion of a code, which contains one or more executable instructions for implementing a specified logical function.
  • the functions marked in the boxes may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession may actually be executed substantially in parallel, and they may sometimes be executed in the opposite order, depending on the functions involved.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or hardware.
  • the name of a unit does not limit the unit itself in some cases.
  • the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".
  • exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOCs systems on chips
  • CPLDs complex programmable logic devices
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing.
  • a more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • CD-ROM portable compact disk read-only memory
  • CD-ROM compact disk read-only memory
  • magnetic storage device or any suitable combination of the foregoing.
  • a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information.
  • the user can autonomously choose whether to provide personal information to software or hardware such as a terminal device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
  • the prompt information in response to receiving an active request from the user, may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form.
  • the pop-up window may also carry a selection control for the user to choose "agree” or “disagree” to provide personal information to the terminal device.
  • the data involved in this technical solution shall comply with the requirements of the relevant laws and regulations.
  • the data may include information, parameters and messages, such as flow switching indication information.
  • one or more embodiments of the present disclosure provide an image processing method, the method comprising:
  • the position of each key point in the image is determined.
  • the image is processed according to the key point prediction model to obtain a probability distribution map corresponding to each key point, including:
  • Each feature map is normalized to obtain a probability distribution map corresponding to each key point.
  • determining the position of each key point in the image according to the probability distribution corresponding to each key point includes:
  • the position of the key point in the image is determined according to each probability value and the first coordinate corresponding to each probability value.
  • determining the position of the key point in the image according to each probability value and the first coordinate corresponding to each probability value includes:
  • the position of the key point in the image is determined.
  • the key point prediction model is trained according to the following steps:
  • the model parameters of the key point prediction model are updated according to the sample image and the sample position.
  • updating the model parameters of the key point prediction model according to the sample image and the sample position includes:
  • the model parameters of the key point prediction model are updated according to the error parameter and the variance.
  • updating the model parameters of the key point prediction model based on the error parameter and the variance includes:
  • a loss function is constructed according to the preset probability distribution, the error probability distribution, the error parameter and the variance, and the model parameters of the key point prediction model are updated according to the loss function.
  • one or more embodiments of the present disclosure provide an image processing device, the image processing device comprising an acquisition module, a processing module and a determination module, wherein:
  • the acquisition module is used to acquire an image including an object, wherein the object is associated with a plurality of key points;
  • the processing module is used to process the image according to the key point prediction model to obtain a probability distribution map corresponding to each key point.
  • the key point prediction model is based on a preset probability
  • the error probability distribution determined by the distribution and normalization flow is obtained through training;
  • the determination module is used to determine the position of each key point in the image according to the probability distribution graph corresponding to each key point.
  • the processing module is specifically used to:
  • Each feature map is normalized to obtain a probability distribution map corresponding to each key point.
  • the determination module is specifically used to:
  • the position of the key point in the image is determined according to each probability value and the first coordinate corresponding to each probability value.
  • the determination module is specifically used to:
  • the position of the key point in the image is determined.
  • the image processing device also includes a training module 1, wherein the training module is used to: obtain a sample image, wherein the sample image includes a sample object; obtain the sample position of each sample key point in the sample object; and update the model parameters of the key point prediction model according to the sample image and the sample position.
  • the training module is used to: obtain a sample image, wherein the sample image includes a sample object; obtain the sample position of each sample key point in the sample object; and update the model parameters of the key point prediction model according to the sample image and the sample position.
  • the training module is specifically used for:
  • An error parameter is determined according to the predicted position corresponding to the sample key point, the variance and the sample position; and a model parameter of the key point prediction model is updated according to the error parameter and the variance.
  • the training module is specifically used for:
  • a loss function is constructed according to the preset probability distribution, the error probability distribution, the error parameter and the variance, and the model parameters of the key point prediction model are updated according to the loss function.
  • an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored.
  • a processor executes the computer execution instructions, the image processing method as described in the first aspect and various possible aspects of the first aspect are implemented.
  • an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the image processing method as described in the first aspect and various possible aspects of the first aspect.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Data Mining & Analysis (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)

Abstract

本公开提供一种图像处理方法、装置及电子设备，该方法包括：获取包括对象的图像，所述对象与多个关键点相关联；根据关键点预测模型对所述图像进行处理，得到每个关键点对应的概率分布图，所述关键点预测模型是基于预设概率分布和标准化流确定的误差概率分布训练得到的；根据所述每个关键点对应的概率分布图，确定所述每个关键点在所述图像中的位置。

Description

图像处理方法、装置及电子设备
本申请要求2023年6月5日递交的、标题为“图像处理方法、装置及电子设备”、申请号为202310659452X的中国发明专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开实施例涉及图像处理技术领域，尤其涉及一种图像处理方法、装置及电子设备。
背景技术
电子设备可以通过检测图像中的关键点，对图像中的对象进行识别。例如，电子设备可以提取图像中的关键点，并基于关键点确定对象的轮廓，进而基于该轮廓识别图像中的对象。
发明内容
第一方面，本公开提供一种图像处理方法，该方法包括：获取包括对象的图像，所述对象与多个关键点相关联；根据关键点预测模型对所述图像进行处理，得到每个关键点对应的概率分布图，所述关键点预测模型是基于预设概率分布和标准化流确定的误差概率分布训练得到的；根据所述每个关键点对应的概率分布图，确定所述每个关键点在所述图像中的位置。
第二方面，本公开提供一种图像处理装置，该图像处理装置包括获取模块、处理模块和确定模块，其中：所述获取模块用于，获取包括对象的图像，所述对象与多个关键点相关联；所述处理模块用于，根据关键点预测模型对所述图像进行处理，得到每个关键点对应的概率分布图，所述关键点预测模型是基于预设概率分布和标准化流确定的误差概率分布训练得到的；所述确定模块用于，根据所述每个关键 点对应的概率分布图，确定所述每个关键点在所述图像中的位置。
第三方面，本公开实施例提供一种电子设备包括：处理器和存储器；所述存储器存储计算机执行指令；所述处理器执行所述存储器存储的计算机执行指令，使得所述至少一个处理器执行如上第一方面以及第一方面各种可能涉及的所述图像处理方法。
第四方面，本公开实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如上第一方面以及第一方面各种可能涉及的所述图像处理方法。
本公开提供一种图像处理方法、装置及电子设备。电子设备可以获取包括对象的图像，其中，对象与多个关键点相关联。根据关键点预测模型对图像进行处理，得到每个关键点对应的概率分布图，其中，关键点预测模型是基于预设概率分布和标准化流确定的误差概率分布训练得到的。根据每个关键点对应的概率分布图，确定每个关键点在图像中的位置。在上述方法中，由于标准化流确定的误差概率分布可以对预设概率分布进行补充。因此，关键点预测模型可以学习到关键点对应的复杂分布，关键点预测模型输出的每个关键点对应的概率分布图的准确度较高，提高确定关键点位置的准确度。并且，由于电子设备可以基于概率分布图准确的计算关键点的位置坐标，无需渲染高分辨率的高斯热力图。因此，可以降低确定关键点的计算量，提高确定关键点的效率。
附图说明
为了更清楚地说明本公开实施例或相关技术中的技术方案，下面将对实施例或相关技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的一种应用场景示意图；
图2为本公开实施例提供的一种图像处理方法的流程示意图；
图3为本公开实施例提供的一种关键点示意图；
图4为本公开实施例提供的一种关键点预测模型的结构示意图；
图5为本公开实施例提供的一种概率分布图的示意图；
图6为本公开实施例提供的一种确定概率分布图的过程示意图；
图7为本公开实施例提供的一种确定关键点的目标坐标的过程示意图；
图8为本公开实施例提供的一种图像处理方法的过程示意图；
图9为本公开实施例提供的一种关键点预测模型的训练方法示意图；
图10为本公开实施例提供的一种关键点预测模型的示意图；
图11为本公开实施例提供的一种关键点预测模型的训练框架示意图；
图12为本公开实施例提供的一种图像处理装置的结构示意图；
图13为本公开实施例提供的另一种图像处理装置的结构示意图；
图14为本公开实施例提供的一种电子设备的结构示意图。
具体实施方式
本公开提供一种图像处理方法、装置及电子设备，用于解决相关技术中确定关键点的效率较低的技术问题。
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。
为了便于理解，下面，对本公开实施例涉及的概念进行说明。
电子设备：是一种具有无线收发功能的设备。电子设备可以部署在陆地上，包括室内或室外、手持、穿戴或车载。所述电子设备可以是手机(mobile phone)、平板电脑(Pad)、带无线收发功能的电脑、虚拟 现实(virtual reality，VR)电子设备、增强现实(augmented reality，AR)电子设备、工业控制(industrial control)中的无线终端、车载电子设备、无人驾驶(self driving)中的无线终端、远程医疗(remote medical)中的无线电子设备、智能电网(smart grid)中的无线电子设备、运输安全(transportation safety)中的无线电子设备、智慧城市(smart city)中的无线电子设备、智慧家庭(smart home)中的无线电子设备、可穿戴电子设备等。本公开实施例所涉及的电子设备还可以称为终端、用户设备(user equipment，UE)、接入电子设备、车载终端、工业控制终端、UE单元、UE站、移动站、移动台、远方站、远程电子设备、移动设备、UE电子设备、无线通信设备、UE代理或UE装置等。电子设备也可以是固定的或者移动的。
下面，结合图1，对本公开实施例的应用场景进行说明。
图1为本公开实施例提供的一种应用场景示意图。请参见图1，包括：电子设备。其中，电子设备的显示屏幕中包括图像A，图像A中可以包括对象。电子设备可以对该图像A进行关键点提取的处理，得到图像B。其中，图像B中可以包括对象，并且图像B中可以包括对象相关联的多个关键点。例如，在电子设备对图像A进行关键点提取处理之后，图像B中可以包括14个关键点，该14个关键点可以为对象的头部关键点、手部关键点、腿部关键点等。这样，电子设备通过14个关键点，可以准确的确定图像B中的对象、对象的轮廓等信息。
在相关技术中，电子设备可以检测图像中的关键点，并通过关键点对图像中的对象进行识别。例如，电子设备可以检测人脸图像中的多个关键点，进而可以识别人脸图像中的人脸所在的位置。目前，电子设备可以基于高斯热力图确定图像中的关键点。例如，电子设备可以基于预先训练的模型对图像进行处理，得到图像中的每个关键点对应的高斯热力图，并将高斯热力图中的最值点确定为关键点的位置估计结果。例如，如图1所示的实施例中，电子设备可以生成14张高斯热力图，每个高斯热力图中的最值点的位置都不同。这样，通过14张高斯热力图可以确定与对象相关联的14个关键点。但是，由于高斯热力图可以直接显示关键 点的位置，因此，预先训练的模型输出的高斯热力图需要具备较高的分辨率。使得电子设备确定关键点的计算量较大，进而导致确定关键点的效率较低。
为了解决相关技术中的技术问题，本公开实施例提供一种图像处理方法，电子设备可以获取包括对象的图像，图像中的对象与多个关键点相关联。电子设备根据关键点预测模型对图像进行处理，得到多个与关键点相关联的特征图，并对每个特征图进行归一化处理，得到每个关键点对应的概率分布图。根据每个关键点对应的概率分布图，确定每个关键点在图像中的位置。在上述方法中，由于关键点预测模型是基于预设概率分布和标准化流确定的误差概率分布训练得到的，而关键点对应的概率分布较为复杂。因此，关键点预测模型在训练阶段可以基于误差概率分布学习到关键点对应的准确的概率分布图，进而提高确定关键点位置的准确度。并且，由于电子设备可以基于概率分布图计算关键点在图像中的位置，无需生成分辨率较高的高斯热力图。进而可以降低确定关键点位置的计算量，提高确定关键点的效率。
下面以具体地实施例对本公开的技术方案以及本公开的技术方案如何解决上述技术问题进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例中不再赘述。下面将结合附图，对本公开的实施例进行描述。
图2为本公开实施例提供的一种图像处理方法的流程示意图。请参见图2，该方法可以包括：
S201、获取包括对象的图像。
本公开实施例的执行主体可以为电子设备，也可以为设置在电子设备中的图像处理装置。其中，图像处理装置可以基于软件实现，图像处理装置也可以基于软件和硬件的结合实现，本公开实施例对此不作限定。
其中，图像中可以包括对象。例如，图像中可以包括用户、人脸、水果等任意对象，本公开实施例对此不作限定。可选的，电子设备可以接收其它设备发送的图像。例如，电子设备可以接收服务器发送的图像，该图像中可以包括人脸图像。例如，电子设备可以接收到服务器发送的 关键点检测的请求，该请求中可以包括待检测的图像。
需要说明的是，电子设备可以根据任意可行的实现方式，获取包括对象的图像，本公开实施例对此不作限定。
其中，图像中的对象可以与多个关键点相关联。例如，图像中的对象可以存在多个关键点。例如，人脸图像中可以存在多个关键点，用户图像中也可以存在多个关键点。需要说明的是，在实际应用的过程中，电子设备获取的图像中并未标注多个关键点，电子设备可以通过对图像的处理，得到图像中的多个关键点。
可选的，图像中的对象不同，对象相关联的多个关键点也不同。例如，若图像中包括人脸图像，则该人脸图像可以关联30个关键点(如，头顶关键点、耳朵关键点等)。若图像中包括用户。则该图像可以关联63个关键点(如，右肩关键点、右腿关键点等)。
可选的，每个部位可以关联多个关键点。例如，若图像中的对象包括用户，则用户的头顶可以关联1个关键点，用户的左胳膊可以关键2个关键点，用户的右腿可以关联3个关键点等。
需要说明的是，对象相关联的多个关键点可以为预先确定的关键点。例如，针对于人脸图像，电子设备可以预先设定人脸图像中最多可以包括30个关键点，并且可以预先设定每个关键点指示的人脸的位置。例如，人脸图像中的第1个关键点为人脸顶部的关键点，人脸图像中的第30个关键点为人脸底部的关键点。例如，针对于包括人体的图像，电子设备可以预先设定包括人体的图像中最多可以包括63个关键点，通过该63个关键点可以确定人体的轮廓。并且，电子设备可以预先设定每个关键点指示的人体的位置。例如，包括人体的图像中的第20个关键点为左手的关键点，第40个关键点为右手的关键点等，本公开实施例对此不作限定。
下面结合图3对与对象关联的关键点进行说明。
图3为本公开实施例提供的一种关键点示意图。请参见图3，包括人体图像和人脸图像。其中，人体图像中的对象为人体，人脸图像中的对象为人脸。人体图像中可以包括多个与人体相关联的关键点，人脸图像 中也包括多个与人脸相关联的关键点。由图3所示的实施例可知，图像中的对象不同，对象相关联的关键点也不同。例如，如图3所示，人体图像的应用场景可以为人体的轮廓检测。因此，人体图像中的多个关键点可以位于图像中的人体的边缘，人脸图像的应用场景可以为人脸识别。因此，人脸图像中的多个关键点可以位于图像中具有人脸特征的位置。
S202、根据关键点预测模型对所述图像进行处理，得到每个关键点对应的概率分布图。
可选的，关键点预测模型可以获取图像中的关键点。例如，关键点预测模型可以对图像进行处理，进而输出标注关键点之后的图像。其中，不同的关键点预测模型针对的对象不同。例如，关键点预测模型A可以对包括人体的图像进行处理，进而得到该图像中的多个关键点。关键点预测模型B可以对包括动物的图像进行处理，进而得到该图像中的多个关键点。
例如，若关键点预测模型可以对包括人体的图像进行处理，则向该关键点预测模型输入任意一个包括人体的图像，关键点预测模型可以输出标注63个关键点的图像。例如，若关键点预测模型可以对包括人脸的图像进行处理，则向该关键点预测模型输入任意一个包括人脸的图像，关键点预测模型可以输出标注30个关键点的图像。
需要说明的是，若关键点预测模型可以对包括人体的图像进行处理，则该关键点预测模型也可以对包括人脸的图像、肢体的图像进行处理。例如，关键点预测模型可以在包括人体的图像中确定63个关键点。其中，若人脸的部位可以包括20个关键点，则该关键点预测模型对人脸图像进行处理时，可以标注包括20个关键点的人脸图像。
可选的，所述关键点预测模型是基于预设概率分布和标准化流确定的误差概率分布训练得到的。其中，预设概率分布可以为高斯分布或拉普拉斯分布。例如，关键点预测模型在训练阶段时，关键点预测模型可以学习关键点对应的高斯分布或拉普拉斯分布，由于关键点实际的分布较为复杂。因此，可以通过标准化流确定一个误差概率分布，基于该误差概率分布对高斯分布或拉普拉斯分布进行补充，进而使得关键点预测 模型可以拟合出与关键点匹配度较高的真实分布。
下面结合图4对关键点预测模型的结构进行说明。
图4为本公开实施例提供的一种关键点预测模型的结构示意图。请参见图4，包括：关键点预测模型。其中，关键点预测模型中可以包括卷积神经网络(CNN)、1*1卷积层(1*1conv)和归一化层(softmax层)。其中，卷积神经网络CNN可以对输入的图像进行下采样，得到多个特征图。若卷积神经网络CNN输出特征图数量(也即特征图的通道数)大于图像中的对象关联的关键点的数量，则可以基于1*1卷积层使得特征图的数量与关键点的数量相同。归一化层可以对特征图进行处理，进而可以得到每个特征图相关联的概率分布图。
可选的，概率分布图可以用于指示关键点在图像中的分布概率。例如，对象关联的多个关键点在图像中的分布位置不同，针对于图像中任意一个关键点，若图像中的某一位置与该关键点的实际位置越近，则该位置的概率值越高。若图像中的某一位置与该关键点的实际位置越远，则该位置的概率值越低。
下面结合图5对概率分布图进行说明。
图5为本公开实施例提供的一种概率分布图的示意图。请参见图5，包括：概率分布图。其中，概率分布图中可以包括25个点，每个点可以对应一个概率值，该概率值可以为关键点出现在该位置的概率。在图5所示的实施例中，25个点对应的概率值的和为1。其中，与关键点位置越接近的点，概率值越高。与关键点位置越远的点，概率值越低。由图5所示的实施例可知，该概率分布图对应的关键点的位置与坐标(4，3)的位置相近。
其中，电子设备可以根据如下可行的实现方式，得到每个关键点对应的概率分布图：根据关键点预测模型对图像进行处理，得到多个与关键点相关联的特征图，对每个特征图进行归一化处理，得到每个关键点对应的概率分布图。例如，电子设备基于关键点预测模型中的CNN网络对图像进行下采样处理，CNN网络可以输出图像对应的多个特征图。
需要说明的是，关键点预测模型为预先训练的模型。并且，关键点 预测模型预测的关键点的数量和关键点的类型(如，头部关键点、手部关键点或腿部关键点等)也是预先训练时确定的。
下面结合图6对确定每个关键点对应的概率分布图的过程进行说明。
图6为本公开实施例提供的一种确定概率分布图的过程示意图。请参见图6，包括关键点预测模型和图像。其中，关键点预测模型可以包括卷积神经网络、1*1卷积层和归一化层(图6中未示出)。图像的尺寸可以为192*144，关键点预测模型中的卷积神经网络的下采样倍数为16倍。
请参见图6，电子设备(图6中未示出)可以向关键点预测模型输入图像，关键点预测模型中的卷积神经网络对图像进行处理之后，可以得到多个特征图。其中，特征图的尺寸为12*9。向1*1卷积层输入多个特征图之后，可以得到63个特征图，其中，关键点的数量可以为63。
请参见图6，通过归一化层对每个特征图进行归一化处理，可以得到概率分布图1、概率分布图2、......、概率分布图63。其中，每个概率分布图都可以对应一个关键点的概率分布。由于关键点预测模型在训练阶段可以学习到与关键点的真实概率分布更接近的复杂分布，因此，基于关键点预测模型得到的关键点的概率分布图的准确度较高。
S203、根据每个关键点对应的概率分布图，确定每个关键点在图像中的位置。
其中，电子设备可以根据如下可行的实现方式，确定每个关键点在图像中的位置：针对于任意一个关键点，获取关键点对应的概率分布图中的每个概率值对应的点在概率分布图中的第一坐标，根据每个概率值和每个概率值对应的第一坐标，确定关键点在图像中的位置。
其中，概率分布图中可以包括多个与概率值对应的点，第一坐标可以为该点在概率分布图中的坐标。例如，如图5所示的实施例中，概率分布图包括25个概率值，概率分布图中可以包括25个点。因此，概率分布图中的点的横坐标的范围可以为1-5，纵坐标的范围可以为1-5。例如，如图5所示的实施例中，若概率分布图的左下角为坐标原点，则概率值为0.6的点对应的第一坐标可以为(4，3)，概率值为0.1的点对应的第一坐标分别可以为(4，2)、(3，3)、(5，3)和(4，4)。
可选的，电子设备可以在概率分布图中获取每个像素值对应的点的第一坐标，电子设备也可以根据其它任意可行的实现方式确定第一坐标，本公开实施例对此不作限定。
其中，根据每个概率值和每个概率值对应的第一坐标，确定关键点在图像中的位置，具体可以为：将每个概率值与每个概率值对应的第一坐标相乘，得到每个概率值对应的第二坐标，将多个第二坐标相加，得到所述关键点的目标坐标，基于目标坐标，确定关键点在图像中的位置。
其中，概率值对应的第二坐标可以为概率值与第一坐标相乘之后的坐标。例如，在图5所示的实施例中，以概率值为0.6为例，该概率值对应的第一坐标为(4，3)。因此，将第一坐标与概率值相乘之后可以得到坐标(2.4，1.8)，电子设备可以将坐标(2.4，1.8)确定为该概率值对应的第二坐标。以概率值为0.1为例，4个0.1的概率值对应的第一坐标分别可以为(4，2)、(3，3)、(5，3)和(4，4)。因此，该4个0.1的概率值对应的第二坐标分别可以为(0.4，0.2)、(0.3，0.3)、(0.5，0.3)和(0.4，0.4)。
可选的，关键点的目标坐标可以为关键点对应的多个第二坐标的和。例如，电子设备可以将关键点对应的多个第二坐标的横坐标相加，得到关键点对应的目标坐标的横坐标，将关键点对应的多个第二坐标的纵坐标相加，得到关键点对应的目标坐标的纵坐标。
下面结合图7对确定关键点的目标坐标的过程进行说明。
图7为本公开实施例提供的一种确定关键点的目标坐标的过程示意图。请参见图7，包括：关键点对应的概率分布图。其中，概率分布图中包括16个概率值，每个概率值对应概率分布图中的一个点。电子设备(图7中未示出)可以根据每个概率值和概率值对应的点的第一坐标，确定第二坐标。其中，多个第二坐标可以包括(0.2，0.2)、(0.09，0.09)、(0.4，0.3)、(0.06，0.06)、(0.3，0.2)、(2.4，1.2)、(0.06，0)、(0.28，0)。
请参见图7，电子设备可以将多个第二坐标的横坐标相加得到3.79，将多个第二坐标的纵坐标相加得到2.05，进而可以确定该关键点对应目 标坐标为(3.79，2.05)。这样，电子设备可以确定每个关键点对应的准确的坐标。并且，由于坐标的小数点后可以保留多位(如，概率分布图中的概率值的小数点后可以保留多位)，因此，电子设备可以得到较为精确的目标坐标，进而可以提高确定关键点的位置的准确度。
可选的，电子设备基于目标坐标，确定关键点在图像中的位置。具体可以为：获取关键点预测模型对图像进行处理时的下采样倍数，基于下采样倍数和目标坐标，确定关键点在图像中的位置。
其中，下采样倍数可以为关键点预测模型中的卷积神经网络的下采样倍数。例如，关键点预测模型中的CNN网络可以对图像进行下采样，若CNN网络的下采样倍数为16(4*4)倍，图像的尺寸为192*144，则下采样后的图像尺寸为12*9。
可选的，电子设备可以基于目标坐标和下采样倍数，确定关键点在图像中的位置。例如，若关键点对应的目标坐标为(1，1)，下采样倍数为16(4*4)倍，则该关键点在图像中的位置坐标可以为(4，4)。即，目标坐标的横坐标乘以4，目标坐标的纵坐标乘以4。若关键点对应的坐标为(2，3)，下采样倍数为25(5*5)倍，则该关键点在图像中的位置坐标可以为(10，15)。即，目标坐标的横坐标乘以5，目标坐标的纵坐标乘以5。
本公开实施例提供一种图像处理方法，电子设备可以获取包括对象的图像，电子设备根据关键点预测模型对图像进行处理，得到多个与关键点相关联的特征图，并对每个特征图进行归一化处理，得到每个关键点对应的概率分布图，根据每个关键点对应的概率分布图，确定每个关键点在图像中的位置。在上述方法中，由于关键点预测模型是基于预设概率分布和标准化流确定的误差概率分布训练得到的。因此，关键点预测模型可以学习到与关键点相关联的复杂分布，进而提高关键点对应的概率分布图的准确度，提高确定关键点位置的准确度。并且，由于电子设备可以基于概率分布图计算关键点在图像中的位置，无需生成分辨率较高的高斯热力图，进而可以降低确定关键点位置的计算量，提高确定关键点的效率。
在图2所示的实施例的基础上，下面结合图8，对上述图像处理方法的过程进行说明。
图8为本公开实施例提供的一种图像处理方法的过程示意图。请参见图8，包括：包括图像A和关键点预测模型，图像A可以为人体图像。电子设备(图8中未示出)可以向关键点预测模型输入图像A，关键点预测模型中的CNN网络可以对图像A下采样4*4倍，关键点预测模型可以输出概率分布图1、概率分布图2、......、概率分布图14，其中，14个概率分布图与关键点一一对应。
请参见图8，电子设备可以确定概率分布图的每个概率值对应的第二坐标，并根据多个第二坐标，确定关键点相关联的目标坐标。例如，电子设备根据概率分布图1可以得到关键点1对应的目标坐标1，电子设备根据概率分布图2可以得到关键点2对应的目标坐标2等。电子设备可以将每个目标坐标的横坐标和纵坐标分别乘以4，可以得到14个位置。
请参见图8，电子设备可以在图像A中确定该14个位置，并在每个位置上标记关键点，进而可以得到图像B，其中，图像B中可以包括人体、以及与人体相关联的14个关键点。
这样，电子设备可以基于关键点预测模型准确的确定每个关键点相关联的概率分布图。并且，由于目标坐标是基于概率分布图中的所有参数(概率值、概率值的坐标)确定的，因此，目标坐标的准确度较高，进而可以提高确定关键点的准确度。并且，由于关键点是基于位置坐标确定的，关键点的精度较高，无需渲染高斯热力图。可以降低计算复杂度，提高确定关键点的效率。
在上述任意一个实施例的基础上，本公开实施例还包括一种关键点预测模型的训练方法，下面，结合图9，对上述关键点预测模型的训练方法进行说明。
图9为本公开实施例提供的一种关键点预测模型的训练方法示意图。请参见图9，该方法流程包括：
S901、获取样本图像。
其中，样本图像中可以包括样本对象。例如，样本图像中可以包括 人体图像、人脸图像等。需要说明的是，在关键点预测模型训练过程中，样本图像中的样本对象与关键点预测模型使用过程中的图像中的对象相同。例如，若关键点预测模型可以检测人脸图像中的关键点，则在关键点预测模型训练时，样本图像中的样本对象可以为人脸。若关键点预测模型可以检测人体中的关键点，则在关键点预测模型训练时，样本图像中的样本对象可以为人体。
需要说明的是，电子设备可以根据任意可行的实现方式获取样本图像，本公开实施例对此不作限定。
S902、获取样本图像中每个样本关键点的样本位置。
可选的，样本关键点可以为对样本对象预先设置的关键点。例如，若样本对象为人脸，则该样本对象可以相关联30个关键点。若样本对象为人体图像，则该样本对象可以相关联63个关键点。
需要说明的是，样本图像中的样本对象相关联的样本关键点可以为预先设置的关键点，也可以为电子设备根据其它任意可行的实现方式确定的样本关键点，本公开实施例对此不作限定。并且，本公开实施例也不限定样本对象相关联的关键点的数量、关键点的位置和关键点的类型等信息。
可选的，样本位置可以为样本关键点在样本图像中的位置。例如，在制作训练的样本图像时，用户可以准确的在样本图像中标注每个样本关键点，进而可以确定样本关键点在样本图像中的样本位置。该样本位置可以基于坐标表示(如以像素为单位的坐标系)，也可以基于其它任意可行的实现方式进行表示，本公开实施例对此不作限定。
S903、根据样本图像、样本位置，对关键点预测模型的模型参数进行更新。
具体的，电子设备可以根据如下可行的实现方式，对关键点预测模型的模型参数进行更新：根据关键点预测模型对样本图像进行处理，得到每个样本关键点对应的预测位置和预测位置相关联的方差。根据每个样本关键点对应的预测位置、方差和样本位置，确定误差参数。根据误差参数，对关键点预测模型的模型参数进行更新。
可选的，预测位置可以为关键点预测模型基于样本图像预测的样本关键点的位置。例如，电子设备可以对样本图像进行处理，得到每个样本关键点相关联的样本概率分布图。并根据样本概率分布图，确定每个样本关键点在样本图像中的位置。需要说明的是，电子设备确定每个样本关键点的预测位置的方法与图2所示的实施例中的电子设备确定每个关键点在图像中的位置的方法相同，本公开实施例在此不再进行赘述。
其中，预测位置相关联的方差可以为关键点预测分布的方差。例如，预测位置相关联的方差可以指示预测值的离散程度。
可选的，电子设备可以基于关键点预测模型对样本图像进行处理，进而可以得到预测位置相关联的方差。例如，关键点预测模型在训练阶段可以设置一个训练方差的分支，该分支的结构可以包括自适应平均池化层和全连接层，这样，基于该分支可以得到预测位置相关联的方差。
下面结合图10对训练阶段的关键点预测模型的结构进行说明。
图10为本公开实施例提供的一种关键点预测模型的示意图。请参见图10，包括关键点预测模型。其中，关键点预测模型中包括卷积神经网络CNN。在卷积神经网络之后可以连接2个分支，其中一个分支包括1*1卷积层和归一化层，该分支可以输出每个关键点相关联的概率分布图。另一个分支包括自适应平均池化层和全连接层，该分支可以输出每个关键点的预测位置相关联的方差。
例如，图10所示的实施例中，关键点预测模型对中的卷积神经网络对图像处理之后，可以得到多个特征图。通过自适应平均池化层和全连接层之后，可以将输出的结果变换为(N，K，2)的三维向量。其中，该三维向量可以为关键点预测分布的方差，N表示卷积神经网络的batch size(数据大小)，K表示关键点的数量，2可以表示为x坐标的方差和y坐标的方差。
可选的，误差参数可以为关键点预测模型训练过程中的变量参数。可选的，根据样本关键点对应的预测位置、方差和样本位置，确定误差参数，具体可以为：确定样本位置与预测位置之间的位置差值，并将位置差值与方差的比值，确定为误差参数。
例如，电子设备可以基于如下公式确定误差参数：
其中，A可以为误差参数，B可以为样本位置，C可以为预测位置，σ可以为方差。这样，关键点预测模型可以通过误差参数，对关键点预测模型中的参数进行更新。避免训练样本的分散，提高关键点预测模型训练的准确度。
其中，电子设备根据误差参数，对关键点预测模型的模型参数进行更新，具体可以为：根据关键点预测模型对样本图像进行处理，得到样本关键点对应的预设概率分布。根据标准化流模型对预设概率分布进行变换，得到样本关键点对应的误差概率分布。根据预设概率分布、误差概率分布、误差参数和方差，构造损失函数。并根据损失函数对关键点预测模型的模型参数进行更新。
其中，预设概率分布可以为高斯分布、拉普拉斯分布等简单分布，本公开实施例对此不作限定。
其中，标准化流模型可以对概率分布进行可逆的变换。例如，标准化流模型可以具备标准化流的功能，标准化流可以将任意一个复杂分布变换成一个简单分布(如，将复杂分布变化为高斯分布)，标准化流也可以将任意一个简单分布变换成一个复杂分布。并且，标准化流对概率分布的变换过程是可逆的。
可选的，在关键点预测模型训练的过程中，可以引入标准化流模型。其中，关键点预测模型用于学习关键点对应的预设概率分布，该预设概率分布与关键点对应的真实分布具有差值。标准化流模型用于学习误差概率分布，该误差概率分布是对预设概率分布的补充，使得预设概率分布接近关键点的真实分布。
下面结合图11对关键点预测模型的训练框架进行说明。
图11为本公开实施例提供的一种关键点预测模型的训练框架示意图。请参见图11，包括：样本图像、关键点预测模型和标准化流模型。根据关键点预测模型对样本图像进行处理，得到关键点对应的预测位置和方差。根据标准化流模型对预设概率分布进行变换，得到误差概率分 布，并通过预测位置、方差和误差概率分布，得到关键点对应的概率分布。例如，预测位置为μ，方差为σ，标准化流模型拟合的误差概率分布为则关键点对应的概率分布可以为：
可选的，根据预设概率分布、误差概率分布、误差参数和方差，构造损失函数，可以为如下公式：
L＝-log Q(A)-log Gθ(A)-log(s)+log σ
其中，L可以为损失函数，A可以为误差参数，Q可以为预设概率分布(如，高斯分布或拉普拉斯分布等)，G可以为标准化流；s可以为常数(s用于修正，可以任意设置，本公开实施例对此不作限定)，σ可以为方差。根据上述公式可以确定损失函数，并根据损失函数对模型参数进行更新。
本公开实施例提供一种关键点预测模型的训练方法，获取样本图像，获取样本图像中每个样本关键点的样本位置。根据关键点预测模型对样本图像进行处理，得到每个样本关键点对应的预测位置和预测位置相关联的方差。根据每个样本关键点对应的预测位置、方差和样本位置，确定误差参数。根据误差参数，对关键点预测模型的模型参数进行更新。这样，关键点预测模型可以学习到关键点相关联的复杂分布，因此，关键点预测模型的训练准确度较高，提高关键点预测模型生成的关键点对应的概率分布图的准确度，进而可以提高确定关键点位置的准确度。
图12为本公开实施例提供的一种图像处理装置的结构示意图。请参见图12，该图像处理装置120包括获取模块121、处理模块122和确定模块123，其中：
所述获取模块121用于，获取包括对象的图像，所述对象与多个关键点相关联；
所述处理模块122用于，根据关键点预测模型对所述图像进行处理，得到每个关键点对应的概率分布图，所述关键点预测模型是基于预设概率分布和标准化流确定的误差概率分布训练得到的；
所述确定模块123用于，根据所述每个关键点对应的概率分布图，确定所述每个关键点在所述图像中的位置。
根据本公开一个或多个实施例，所述处理模块122具体用于：
根据关键点预测模型对所述图像进行处理，得到多个与关键点相关联的特征图；
对每个特征图进行归一化处理，得到所述每个关键点对应的概率分布图。
根据本公开一个或多个实施例，所述确定模块123具体用于：
针对于任意一个关键点，获取所述关键点对应的概率分布图中的每个概率值对应的点在所述概率分布图中的第一坐标；
根据所述每个概率值和每个概率值对应的第一坐标，确定所述关键点在所述图像中的位置。
根据本公开一个或多个实施例，所述确定模块123具体用于：
将所述每个概率值与所每个概率值对应的第一坐标相乘，得到每个概率值对应的第二坐标；
将所述多个第二坐标相加，得到所述关键点的目标坐标；
基于所述目标坐标，确定所述关键点在所述图像中的位置。
本公开实施例提供的图像处理装置，可用于执行上述方法实施例的技术方案，其实现原理和技术效果类似，本实施例此处不再赘述。
图13为本公开实施例提供的另一种图像处理装置的结构示意图。在图12所示的实施例的基础上，请参见图13，该图像处理装置120还包括训练模块124，其中，训练模块124用于：
获取样本图像，所述样本图像中包括样本对象；
获取所述样本对象中每个样本关键点的样本位置；
根据所述样本图像、所述样本位置，对所述关键点预测模型的模型参数进行更新。
根据本公开一个或多个实施例，所述训练模块124具体用于：
根据所述关键点预测模型对所述样本图像进行处理，得到所述每个样本关键点对应的预测位置和所述预测位置相关联的方差；
根据所述样本关键点对应的预测位置、所述方差和所述样本位置，确定误差参数；
根据所述误差参数和所述方差，对所述关键点预测模型的模型参数进行更新。
根据本公开一个或多个实施例，所述训练模块124具体用于：
根据所述关键点预测模型对所述样本图像进行处理，得到所述样本关键点对应的预设概率分布；
根据标准化流模型对所述预设概率分布进行变换，得到所述样本关键点对应的误差概率分布；
根据所述预设概率分布、所述误差概率分布、所述误差参数和所述方差，构造损失函数，并根据所述损失函数对所述关键点预测模型的模型参数进行更新。
本公开实施例提供的图像处理装置，可用于执行上述方法实施例的技术方案，其实现原理和技术效果类似，本实施例此处不再赘述。
图14为本公开实施例提供的一种电子设备的结构示意图。请参见图14，其示出了适于用来实现本公开实施例的电子设备1400的结构示意图，该电子设备1400可以为任意具备端上计算能力的设备。图14示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图14所示，电子设备1400可以包括处理装置(例如中央处理器、图形处理器等)1401，其可以根据存储在只读存储器(Read Only Memory，简称ROM)1402中的程序或者从存储装置1408加载到随机访问存储器(Random Access Memory，简称RAM)1403中的程序而执行各种适当的动作和处理。在RAM 1403中，还存储有电子设备1400操作所需的各种程序和数据。处理装置1401、ROM 1402以及RAM 1403通过总线1404彼此相连。输入/输出(I/O)接口1405也连接至总线1404。
通常，以下装置可以连接至I/O接口1405：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置1406；包括例如液晶显示器(Liquid Crystal Display，简称LCD)、扬声器、振动器等的输出装置1407；包括例如磁带、硬盘等的存储装置1408；以及通信装置1409。通信装置1409可以允许电子设备1400与其他设备进 行无线或有线通信以交换数据。虽然图14示出了具有各种装置的电子设备1400，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置1409从网络上被下载和安装，或者从存储装置1408被安装，或者从ROM 1402被安装。在该计算机程序被处理装置1401执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备执行上述实施例所示的方法。
本公开实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如上第一方面以及第一方面各种可能涉及的所述图像处理方法。
本公开实施例提供一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现如上第一方面以及第一方面各种可能涉及的所述图像处理方法。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言-诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言-诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(Local Area Network，简称LAN)或广域网(Wide Area Network，简称WAN)——连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图 中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定，例如，第一获取单元还可以被描述为“获取至少两个网际协议地址的单元”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
需要注意，本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
可以理解的是，在使用本公开各实施例公开的技术方案之前，均应当依据相关法律法规通过恰当的方式对本公开所涉及个人信息的类型、使用范围、使用场景等告知用户并获得用户的授权。
例如，在响应于接收到用户的主动请求时，向用户发送提示信息，以明确地提示用户，其请求执行的操作将需要获取和使用到用户的个人信息。从而，使得用户可以根据提示信息来自主地选择是否向执行本公开技术方案的操作的终端设备、应用程序、服务器或存储介质等软件或硬件提供个人信息。
作为一种可选的但非限定性的实现方式，响应于接收到用户的主动请求，向用户发送提示信息的方式例如可以是弹窗的方式，弹窗中可以以文字的方式呈现提示信息。此外，弹窗中还可以承载供用户选择“同意”或者“不同意”向终端设备提供个人信息的选择控件。
可以理解的是，上述通知和获取用户授权过程仅是示意性的，不对本公开的实现方式构成限定，其它满足相关法律法规的方式也可应用于本公开的实现方式中。
可以理解的是，本技术方案所涉及的数据(包括但不限于数据本身、数据的获取或使用)应当遵循相应法律法规及相关规定的要求。数据可以包括信息、参数和消息等，如切流指示信息。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主 题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。
第一方面，本公开一个或多个实施例，提供一种图像处理方法，该方法包括：
获取包括对象的图像，所述对象与多个关键点相关联；
根据关键点预测模型对所述图像进行处理，得到每个关键点对应的概率分布图，所述关键点预测模型是基于预设概率分布和标准化流确定的误差概率分布训练得到的；
根据所述每个关键点对应的概率分布图，确定所述每个关键点在所述图像中的位置。
根据本公开一个或多个实施例，根据关键点预测模型对所述图像进行处理，得到每个关键点对应的概率分布图，包括：
根据关键点预测模型对所述图像进行处理，得到多个与关键点相关联的特征图；
对每个特征图进行归一化处理，得到所述每个关键点对应的概率分布图。
根据本公开一个或多个实施例，根据所述每个关键点对应的概率分布，确定所述每个关键点在所述图像中的位置，包括：
针对于任意一个关键点，获取所述关键点对应的概率分布图中的每个概率值对应的点在所述概率分布图中的第一坐标；
根据所述每个概率值和每个概率值对应的第一坐标，确定所述关键点在所述图像中的位置。
根据本公开一个或多个实施例，根据所述每个概率值和每个概率值对应的第一坐标，确定所述关键点在所述图像中的位置，包括：
将所述每个概率值与所每个概率值对应的第一坐标相乘，得到每个概率值对应的第二坐标；
将所述多个第二坐标相加，得到所述关键点的目标坐标；
基于所述目标坐标，确定所述关键点在所述图像中的位置。
根据本公开一个或多个实施例，所述关键点预测模型是根据如下步骤训练得到的：
获取样本图像，所述样本图像中包括样本对象；
获取所述样本对象中每个样本关键点的样本位置；
根据所述样本图像、所述样本位置，对所述关键点预测模型的模型参数进行更新。
根据本公开一个或多个实施例，根据所述样本图像、所述样本位置，对所述关键点预测模型的模型参数进行更新，包括：
根据所述关键点预测模型对所述样本图像进行处理，得到所述每个样本关键点对应的预测位置和所述预测位置相关联的方差；
根据所述样本关键点对应的预测位置、所述方差和所述样本位置，确定误差参数；
根据所述误差参数和所述方差，对所述关键点预测模型的模型参数进行更新。
根据本公开一个或多个实施例，基于所述误差参数和所述方差，对所述关键点预测模型的模型参数进行更新，包括：
根据所述关键点预测模型对所述样本图像进行处理，得到所述样本关键点对应的预设概率分布；
根据标准化流模型对所述预设概率分布进行变换，得到所述样本关键点对应的误差概率分布；
根据所述预设概率分布、所述误差概率分布、所述误差参数和所述方差，构造损失函数，并根据所述损失函数对所述关键点预测模型的模型参数进行更新。
第二方面，本公开一个或多个实施例，提供一种图像处理装置，该图像处理装置包括获取模块、处理模块和确定模块，其中：
所述获取模块用于，获取包括对象的图像，所述对象与多个关键点相关联；
所述处理模块用于，根据关键点预测模型对所述图像进行处理，得到每个关键点对应的概率分布图，所述关键点预测模型是基于预设概率 分布和标准化流确定的误差概率分布训练得到的；
所述确定模块用于，根据所述每个关键点对应的概率分布图，确定所述每个关键点在所述图像中的位置。
根据本公开一个或多个实施例，所述处理模块具体用于：
根据关键点预测模型对所述图像进行处理，得到多个与关键点相关联的特征图；
对每个特征图进行归一化处理，得到所述每个关键点对应的概率分布图。
根据本公开一个或多个实施例，所述确定模块具体用于：
针对于任意一个关键点，获取所述关键点对应的概率分布图中的每个概率值对应的点在所述概率分布图中的第一坐标；
根据所述每个概率值和每个概率值对应的第一坐标，确定所述关键点在所述图像中的位置。
根据本公开一个或多个实施例，所述确定模块具体用于：
将所述每个概率值与所每个概率值对应的第一坐标相乘，得到每个概率值对应的第二坐标；
将所述多个第二坐标相加，得到所述关键点的目标坐标；
基于所述目标坐标，确定所述关键点在所述图像中的位置。
根据本公开一个或多个实施例，该图像处理装置还包括训练模块1其中，训练模块用于：获取样本图像，所述样本图像中包括样本对象；获取所述样本对象中每个样本关键点的样本位置；根据所述样本图像、所述样本位置，对所述关键点预测模型的模型参数进行更新。
根据本公开一个或多个实施例，所述训练模块具体用于：
根据所述关键点预测模型对所述样本图像进行处理，得到所述每个样本关键点对应的预测位置和所述预测位置相关联的方差；
根据所述样本关键点对应的预测位置、所述方差和所述样本位置，确定误差参数；根据所述误差参数和所述方差，对所述关键点预测模型的模型参数进行更新。
根据本公开一个或多个实施例，所述训练模块具体用于：
根据所述关键点预测模型对所述样本图像进行处理，得到所述样本关键点对应的预设概率分布；
根据标准化流模型对所述预设概率分布进行变换，得到所述样本关键点对应的误差概率分布；
根据所述预设概率分布、所述误差概率分布、所述误差参数和所述方差，构造损失函数，并根据所述损失函数对所述关键点预测模型的模型参数进行更新。
第四方面，本公开实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如上第一方面以及第一方面各种可能涉及的所述图像处理方法。
第五方面，本公开实施例提供一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现如上第一方面以及第一方面各种可能涉及的所述图像处理方法。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述 的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。

Claims (10)

  1. 一种图像处理方法，其中包括：
    获取包括对象的图像，所述对象与多个关键点相关联；
    根据关键点预测模型对所述图像进行处理，得到每个关键点对应的概率分布图，所述关键点预测模型是基于预设概率分布和标准化流确定的误差概率分布训练得到的；
    根据所述每个关键点对应的概率分布图，确定所述每个关键点在所述图像中的位置。
  2. 根据权利要求1所述的方法，根据关键点预测模型对所述图像进行处理，得到每个关键点对应的概率分布图，包括：
    根据关键点预测模型对所述图像进行处理，得到多个与关键点相关联的特征图；
    对每个特征图进行归一化处理，得到所述每个关键点对应的概率分布图。
  3. 根据权利要求1所述的方法，根据所述每个关键点对应的概率分布，确定所述每个关键点在所述图像中的位置，包括：
    针对于任意一个关键点，获取所述关键点对应的概率分布图中的每个概率值对应的点在所述概率分布图中的第一坐标；
    根据所述每个概率值和每个概率值对应的第一坐标，确定所述关键点在所述图像中的位置。
  4. 根据权利要求3所述的方法，根据所述每个概率值和每个概率值对应的第一坐标，确定所述关键点在所述图像中的位置，包括：
    将所述每个概率值与所每个概率值对应的第一坐标相乘，得到每个概率值对应的第二坐标；
    将所述多个第二坐标相加，得到所述关键点的目标坐标；
    基于所述目标坐标，确定所述关键点在所述图像中的位置。
  5. 根据权利要求2-4任一项所述的方法，所述关键点预测模型是 根据如下步骤训练得到的：
    获取样本图像，所述样本图像中包括样本对象；
    获取所述样本对象中每个样本关键点的样本位置；
    根据所述样本图像、所述样本位置，对所述关键点预测模型的模型参数进行更新。
  6. 根据权利要求5所述的方法，根据所述样本图像、所述样本位置，对所述关键点预测模型的模型参数进行更新，包括：
    根据所述关键点预测模型对所述样本图像进行处理，得到所述每个样本关键点对应的预测位置和所述预测位置相关联的方差；
    根据所述样本关键点对应的预测位置、所述方差和所述样本位置，确定误差参数；
    根据所述误差参数和所述方差，对所述关键点预测模型的模型参数进行更新。
  7. 根据权利要求6所述的方法，基于所述误差参数和所述方差，对所述关键点预测模型的模型参数进行更新，包括：
    根据所述关键点预测模型对所述样本图像进行处理，得到所述样本关键点对应的预设概率分布；
    根据标准化流模型对所述预设概率分布进行变换，得到所述样本关键点对应的误差概率分布；
    根据所述预设概率分布、所述误差概率分布、所述误差参数和所述方差，构造损失函数，并根据所述损失函数对所述关键点预测模型的模型参数进行更新。
  8. 一种图像处理装置，包括获取模块、处理模块和确定模块，其中：
    所述获取模块用于，获取包括对象的图像，所述对象与多个关键点相关联；
    所述处理模块用于，根据关键点预测模型对所述图像进行处理，得到每个关键点对应的概率分布图，所述关键点预测模型是基于预设概率分布和标准化流确定的误差概率分布训练得到的；
    所述确定模块用于，根据所述每个关键点对应的概率分布图，确定所述每个关键点在所述图像中的位置。
  9. 一种电子设备，包括：处理器和存储器；
    所述存储器存储计算机执行指令；
    所述处理器执行所述存储器存储的计算机执行指令，使得所述处理器执行如权利要求1-7任一项所述的图像处理方法。
  10. 一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如权利要求1-7任一项所述的图像处理方法。
PCT/CN2024/097120 2023-06-05 2024-06-03 图像处理方法、装置及电子设备 Ceased WO2024251089A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310659452.X 2023-06-05
CN202310659452.XA CN119090952A (zh) 2023-06-05 2023-06-05 图像处理方法、装置及电子设备

Publications (1)

Publication Number Publication Date
WO2024251089A1 true WO2024251089A1 (zh) 2024-12-12

Family

ID=93661035

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/097120 Ceased WO2024251089A1 (zh) 2023-06-05 2024-06-03 图像处理方法、装置及电子设备

Country Status (2)

Country Link
CN (1) CN119090952A (zh)
WO (1) WO2024251089A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107767419A (zh) * 2017-11-07 2018-03-06 广州深域信息科技有限公司 一种人体骨骼关键点检测方法及装置
CN109948441A (zh) * 2019-02-14 2019-06-28 北京奇艺世纪科技有限公司 模型训练、图像处理方法、装置、电子设备及计算机可读存储介质
US20220156970A1 (en) * 2020-11-13 2022-05-19 Ford Global Technologies, Llc Systems and methods for image-based component detection
CN115240267A (zh) * 2022-06-22 2022-10-25 阿里巴巴（中国）有限公司 关键点、关节关键点检测、模型训练、设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107767419A (zh) * 2017-11-07 2018-03-06 广州深域信息科技有限公司 一种人体骨骼关键点检测方法及装置
CN109948441A (zh) * 2019-02-14 2019-06-28 北京奇艺世纪科技有限公司 模型训练、图像处理方法、装置、电子设备及计算机可读存储介质
US20220156970A1 (en) * 2020-11-13 2022-05-19 Ford Global Technologies, Llc Systems and methods for image-based component detection
CN115240267A (zh) * 2022-06-22 2022-10-25 阿里巴巴（中国）有限公司 关键点、关节关键点检测、模型训练、设备及存储介质

Also Published As

Publication number Publication date
CN119090952A (zh) 2024-12-06

Similar Documents

Publication Publication Date Title
US20250191299A1 (en) Rendering method and apparatus for 3d material, and device and storage medium
CN110457407B (zh) 用于处理点云数据的方法和装置
CN109754464B (zh) 用于生成信息的方法和装置
CN118097157B (zh) 基于模糊聚类算法的图像分割方法及系统
CN114419564B (zh) 车辆位姿检测方法、装置、设备、介质及自动驾驶车辆
CN111368668B (zh) 三维手部识别方法、装置、电子设备及存储介质
CN114993328A (zh) 车辆定位评估方法、装置、设备和计算机可读介质
US20260057483A1 (en) Method and apparatus for generating image with effects, device and storage medium
CN115511779B (zh) 图像检测方法、装置、电子设备和存储介质
CN117894038A (zh) 一种图像中对象姿态生成方法和装置
CN110555798A (zh) 图像变形方法、装置、电子设备及计算机可读存储介质
WO2026020755A1 (zh) 占有网络预测方法、装置、设备、存储介质及产品
CN114399588A (zh) 三维车道线生成方法、装置、电子设备和计算机可读介质
CN115086541A (zh) 一种拍摄位置确定方法、装置、设备及介质
CN118936459A (zh) 视觉惯性融合定位方法及设备
CN115100536B (zh) 建筑物识别方法、装置、电子设备和计算机可读介质
CN116563818B (zh) 障碍物信息生成方法、装置、电子设备和计算机可读介质
WO2024251089A1 (zh) 图像处理方法、装置及电子设备
WO2025021169A1 (zh) 图像处理方法、设备、存储介质及程序产品
CN115797455B (zh) 目标检测方法、装置、电子设备和存储介质
WO2024245228A1 (zh) 一种姿态估计方法及其相关设备
WO2024060708A1 (zh) 目标检测方法和装置
CN115761412A (zh) 检测框处理方法、装置、电子设备和计算机可读介质
CN115393472B (zh) 画布处理方法、装置、电子设备、可读介质和程序产品
CN116563817B (zh) 障碍物信息生成方法、装置、电子设备和计算机可读介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24818621

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE