WO2020134010A1 - Training of image key point extraction model and image key point extraction - Google Patents

Training of image key point extraction model and image key point extraction Download PDF

Info

Publication number
WO2020134010A1
WO2020134010A1 PCT/CN2019/094740 CN2019094740W WO2020134010A1 WO 2020134010 A1 WO2020134010 A1 WO 2020134010A1 CN 2019094740 W CN2019094740 W CN 2019094740W WO 2020134010 A1 WO2020134010 A1 WO 2020134010A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
key point
model
point extraction
sub
Prior art date
Application number
PCT/CN2019/094740
Other languages
French (fr)
Chinese (zh)
Inventor
喻冬东
王长虎
Original Assignee
北京字节跳动网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字节跳动网络技术有限公司 filed Critical 北京字节跳动网络技术有限公司
Publication of WO2020134010A1 publication Critical patent/WO2020134010A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features

Definitions

  • the present disclosure relates to the field of image processing, and in particular, to training of image key point extraction models and image key point extraction.
  • the key points of the image are usually extracted through a convolutional neural network, and the labeled image is used to uniformly train the image key point extraction model.
  • the difference in image clarity or the shooting environment may result in difficulty in extracting key points in the image. Therefore, when using such images for unified training, the obtained network has less applicability and lower accuracy.
  • a training method for an image key point extraction model includes a plurality of cascaded sub-models, the method includes:
  • For each sub-model determine the difference between the key points output by the sub-model and the key points in the training image corresponding to the degree identifier of the sub-model, where the degree identifier is used to characterize the difficulty of key point extraction degree;
  • the sum of the differences corresponding to the sub-models is determined as the target difference of the image key point extraction model, and when the training times of the image key point extraction model does not reach the preset number, the The image key point extraction model is described.
  • the input of the first sub-model in the image key point extraction model is a feature map of the human body image portion in the training image, and the image key-point extraction model except the first sub-model
  • the input of the external sub-model is the key points output by the previous sub-model and the feature map of the human image part in the training image.
  • the feature map of the human image part in the training image is determined in the following manner:
  • the resolution corresponding to the first image is adjusted to a preset resolution, a second image is obtained, and the feature map of the human body image part in the training image is determined according to the second image.
  • an image key point extraction method including:
  • the extraction model includes multiple cascaded sub-models, and the image key point extraction model is obtained by training according to the method of the first aspect.
  • a training device for extracting a model of an image key point the image key point extraction model includes a plurality of cascaded sub-models, and the device includes:
  • the processing module is used to input the training image into the image key point extraction model to obtain the key points output by each sub-model as a training for the image key point extraction model;
  • the first determining module is used to determine, for each sub-model, the difference between the key points output by the sub-model and the key points in the training image corresponding to the degree identifier of the sub-model, wherein the degree identifier is used to Characterize the difficulty of key point extraction;
  • the update module is used to determine the sum of the differences corresponding to the respective sub-models as the target difference of the image key point extraction model.
  • the The target difference updates the image key point extraction model.
  • the processing module inputs the training image into the updated image key point extraction model to obtain the key points output by each sub-model until the image The training times of the key point extraction model reach the preset times.
  • the input of the first sub-model in the image key point extraction model is a feature map of the human body image portion in the training image, and the image key-point extraction model except the first sub-model
  • the input of the external sub-model is the key points output by the previous sub-model and the feature map of the human image part in the training image.
  • the device further includes a feature extraction module for obtaining a feature map of a human image part in the training image, the feature extraction module includes:
  • the adjustment sub-module is used to adjust the resolution corresponding to the first image to a preset resolution, obtain a second image, and determine the feature map of the human body image portion in the training image according to the second image.
  • an image key point extraction device comprising:
  • the receiving module is used to receive a target image, the target image includes a human body image portion;
  • a second determining module configured to input the target image into an image key point extraction model, and determine the key point output by the last sub-model of the image key point extraction model as the key point of the human body image part in the target image,
  • the image key point extraction model includes multiple cascaded sub-models, and the image key point extraction model is obtained by training according to the method of the first aspect.
  • a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the method of the first aspect described above.
  • a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the method of the second aspect described above.
  • an electronic device including:
  • a processor is configured to execute the computer program in the memory to implement the method of the first aspect.
  • an electronic device including:
  • a processor is configured to execute the computer program in the memory to implement the method of the second aspect.
  • each sub-model of the image key point extraction model outputs key points, and the difference is calculated separately for each sub-model, so that each sub-model in the image key point extraction model can focus on corresponding to its degree identification Key points to facilitate the extraction of key points with different degrees of difficulty.
  • each sub-model to determine the target difference of the image key point extraction model, to achieve the update of the image key point extraction model, it can effectively ensure the accuracy of the image key point extraction model, by targeting key points of different degrees of difficulty Separate processing, so as to improve the application range of the image key point extraction model and enhance the user experience.
  • FIG. 1 is a flowchart of a training method for an image keypoint extraction model according to an exemplary embodiment of the present disclosure
  • FIG. 2 is a flowchart of a method of acquiring a feature map of a human image portion in a training image according to an exemplary embodiment of the present disclosure
  • FIG. 3 is a flowchart of an image key point extraction method according to an exemplary embodiment of the present disclosure
  • FIG. 4 is a block diagram of a training device for an image keypoint extraction model according to an exemplary embodiment of the present disclosure
  • FIG. 5 is a block diagram of an image keypoint extraction device according to an exemplary embodiment of the present disclosure.
  • FIG. 6 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
  • FIG. 7 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
  • FIG. 1 is a flowchart of a training method for an image keypoint extraction model according to an exemplary embodiment of the present disclosure, the image keypoint extraction model including multiple cascaded sub-models.
  • step S11 the training image is input to the image key point extraction model, and the key points output by each sub-model are obtained as a training for the image key point extraction model.
  • a large number of images can be obtained from a database or the Internet. After that, the key points in the image are marked to determine the training image.
  • step S12 for each sub-model, the difference between the key points output by the sub-model and the key points corresponding to the degree identifier of the sub-model in the training image is determined, where the degree identifier is used to characterize the key point extraction Degree of difficulty.
  • the difficulty of extracting each key point may be marked.
  • One-level identification, the first-level identification is used to characterize the extraction of the key point is relatively simple. It is difficult to extract the key points of the human body image part in the blurred, low-resolution training image.
  • the key points in the training image can be marked with a second degree identifier, which is used to characterize the extraction and comparison of the key points difficult.
  • different key points in the training image may be directly labeled with degree identifiers.
  • the key points that are more difficult to extract from the training image are labeled with the second degree identification
  • the key points that are easier to extract from the training image are labeled with the first degree identification.
  • the degree indicator corresponding to the first sub-model is a first degree indicator
  • the degree indicator corresponding to the next sub-model is a second degree indicator.
  • the difference between the two key points is determined according to the key point output by the next sub-model and the key point corresponding to the second degree identifier in the training image. Therefore, when determining the difference corresponding to each sub-model, the sub-model can only focus on the key points corresponding to the degree identification in the sub-model.
  • step S13 the sum of the differences corresponding to the sub-models is determined as the target difference of the image key point extraction model.
  • the image key point extraction is updated according to the target difference model.
  • the difference corresponding to each sub-model can be used to characterize the accuracy of extracting the key points identified by the corresponding degree of the sub-model. The smaller the difference, the more accurate the extraction of characterizing key points.
  • the sum of the differences corresponding to each sub-model can be determined as the target difference of the image key point extraction model, then the differences of the key point extraction model of the image can be comprehensively characterized according to the differences corresponding to each sub-model , So that the key point extraction model of the image can be updated according to the target difference.
  • the preset number of times may be set according to actual usage scenarios. For example, in a scene with higher accuracy requirements, the preset number of times may be set to be larger; in a scene with general accuracy requirements, the preset number of times may be set to be smaller.
  • each sub-model of the model extracts key points through image key point extraction, and the difference is calculated separately for each sub-model, so that each sub-model in the image key point extraction model can focus on its degree Identify the corresponding key points, so as to facilitate the extraction of key points with different degrees of difficulty.
  • the difference of each sub-model to determine the target difference of the image key point extraction model to achieve the update of the image key point extraction model, it can effectively ensure the accuracy of the image key point extraction model, by targeting key points of different degrees of difficulty Separate processing, so as to improve the application range of the image key point extraction model and enhance the user experience.
  • the image key point extraction model after the image key point extraction model is updated, it may return to step S11 until the training times of the image key point extraction model reaches a preset number of times.
  • updating the image key point extraction model refers to adjusting the weight parameters in the image key point extraction model according to the target difference, which can be implemented through the existing neural network feedback update method, which will not be repeated here.
  • the training image used when returning to the step of inputting the training image into the key point extraction model of the image to obtain the key points output by each sub-model, the training image used may be the training image used before or a new one
  • the training image is not limited in this disclosure.
  • the training process of the image key point extraction model is completed to obtain an accurate image key point extraction model, thereby providing support for the extraction of the image key point.
  • the input of the first sub-model in the image key point extraction model is the feature map of the human body image part in the training image
  • the sub-models in the image key point extraction model other than the first sub-model The input of is the key points output by the previous sub-model and the feature map of the human image part in the training image.
  • the sub-models other than the first sub-model are extracted from the model, and the inputs are the key points output by the previous sub-model and the feature map of the human image part in the training image. Therefore, when the current sub-model performs key point extraction, it can be determined based on the key points output by the previous sub-model, which can effectively simplify the image key point extraction process, avoid repeated data processing and calculation, and improve the image key point extraction model s efficiency.
  • the feature map of the human image part in the training image is determined in the following manner, as shown in FIG. 2:
  • step S21 the first image corresponding to the human body image part of the training image is extracted, wherein the first image can be extracted by an existing human body recognition extraction algorithm.
  • the human body image in the training image may be extracted through the faster-rcnn algorithm or the maskrcnn algorithm.
  • step S22 the resolution corresponding to the first image is adjusted to a preset resolution, a second image is obtained, and the feature map of the human image portion in the training image is determined according to the second image.
  • the corresponding proportions of human image parts in different training images may be the same or different.
  • the training images are obtained by the same user through continuous shooting, where the proportions corresponding to the human body image parts are generally similar, and for images taken by different users, the proportions corresponding to the human body image parts are generally different. Therefore, in order to facilitate uniform processing of the human body image portion in the training image.
  • the resolution of the first image may be adjusted to a preset resolution to obtain the second image.
  • the preset resolution may be 400*600.
  • the resolution of the first image can be made 400*600 by enlarging the image; when the resolution of the extracted first image is greater than the preset For resolution, the resolution of the first image can be reduced to 400*600 by reducing the image.
  • the way to enlarge or reduce the image is the prior art, and will not be repeated here.
  • feature maps with the same resolution can be extracted from different training images, which facilitates uniform processing of the feature maps, effectively simplifies the processing flow, and increases the processing speed. At the same time, it meets the user's needs and is convenient for users.
  • An embodiment of the present disclosure also provides an image key point extraction method. As shown in FIG. 3, the method includes:
  • step S31 a target image is received, and the target image contains a human body image part, wherein the human body image in the target image can be detected by a faster-rcnn algorithm or a maskrcnn algorithm.
  • step S32 the target image is input to the image key point extraction model, and the key points output by the last sub-model of the image key point extraction model are determined as the key points of the human body image part in the target image, where the image key point extraction model includes For multiple cascaded sub-models, the image key point extraction model is trained according to any of the above training methods for the image key point extraction model.
  • the image key point extraction model by inputting the target image to the image key point extraction model, key points in the target image can be extracted.
  • the key point extraction model based on the image can accurately extract key points of different degrees of difficulty in the target image. On the one hand, it can ensure the comprehensiveness and completeness of key point extraction, on the other hand, it can also effectively ensure the extraction of key points.
  • the accuracy of the system provides accurate data support for subsequent processing based on this key point, and further improves the user experience.
  • the key points of the human body image part are the bone key points corresponding to the human body image part.
  • the key points in the target image may be determined according to the bone key points Posture estimation is performed on the part of the human body image. Therefore, the prediction accuracy of the bone key points corresponding to the human body image part can be improved, thereby ensuring the accuracy of the pose estimation of the human body image part in the target image.
  • An embodiment of the present disclosure also provides a training device for extracting a model of an image key point.
  • the image key point extraction model includes multiple cascaded sub-models.
  • the device 10 may include:
  • the processing module 100 is used to input a training image into an image key point extraction model to obtain key points output by each sub-model as a training for the image key point extraction model;
  • the first determination module 200 is used to determine, for each sub-model, the difference between the key points output by the sub-model and the key points in the training image corresponding to the degree identification of the sub-model, where the degree identification is used to characterize the key point Difficulty of extraction;
  • the update module 300 is used to determine the sum of the differences corresponding to the sub-models as the target difference of the image key point extraction model. When the training times of the image key point extraction model do not reach the preset number, update the image key point according to the target difference Extract the model.
  • the processing module may input the training image into the updated image key point extraction model to obtain the key points output by each sub-model until the image key points The training times of the extracted model have reached the preset times.
  • the input of the first sub-model in the image key point extraction model is the feature map of the human body image part in the training image
  • the sub-models in the image key point extraction model other than the first sub-model The input of is the key points output by the previous sub-model and the feature map of the human image part in the training image.
  • the apparatus may further include a feature extraction module for obtaining a feature map of the human body image part in the training image.
  • the feature extraction module may include:
  • the adjustment submodule is used to adjust the resolution corresponding to the first image to a preset resolution, obtain a second image, and determine the feature map of the human image portion in the training image according to the second image.
  • the device 20 may include:
  • the receiving module 400 is used to receive a target image, and the target image includes a human body image part;
  • the second determination module 500 is used to input the target image into the image keypoint extraction model, and determine the keypoint output by the last sub-model of the image keypoint extraction model as the keypoint of the human image part in the target image, where the image keypoint
  • the extraction model includes multiple cascaded sub-models.
  • the image key point extraction model is obtained by training according to any of the above training methods for the image key point extraction model.
  • FIG. 6 is a block diagram of an electronic device 700 according to an embodiment of the present disclosure.
  • the electronic device 700 may include a processor 701 and a memory 702.
  • the electronic device 700 may further include one or more of a multimedia component 703, an input/output (I/O) interface 704, and a communication component 705.
  • a multimedia component 703 an input/output (I/O) interface 704
  • the processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the training method of the image key point extraction model or the image key point extraction method.
  • the memory 702 is used to store various types of data to support operation on the electronic device 700, and the data may include, for example, instructions for any application or method for operating on the electronic device 700, and application-related data, For example, contact data, messages sent and received, pictures, audio, video, etc.
  • the memory 702 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (Read -Only Memory (ROM for short), magnetic memory, flash memory, magnetic disk or optical disk.
  • SRAM static random access memory
  • EEPROM Electrically erasable programmable read-only memory
  • EPROM Erasable Programmable Read-Only Memory
  • PROM Programmable Read-Only Memory
  • Read -Only Memory Read-Only Memory
  • the multimedia component 703 may include a screen and an audio component.
  • the screen may be, for example, a touch screen, and the audio component is used to output and/or input audio signals.
  • the audio component may include a microphone for receiving
  • the received audio signal may be further stored in the memory 702 or transmitted through the communication component 705.
  • the audio component also includes at least one speaker for outputting audio signals.
  • the I/O interface 704 provides an interface between the processor 701 and other interface modules.
  • the other interface modules may be a keyboard, a mouse, a button, and so on. These buttons can be virtual buttons or physical buttons.
  • the communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or 5G, etc., or a combination of one or more of them, in This is not limited. Therefore, the corresponding communication component 707 may include a Wi-Fi module, a Bluetooth module, an NFC module, and so on.
  • the electronic device 700 may be one or more application specific integrated circuits (Application Specific Integrated Circuit (ASIC), digital signal processor (Digital Signal Processor, DSP), digital signal processing device (Digital Signal Processing Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field Programmable Gate Array, FPGA), controller, microcontroller, microprocessor, or other electronic components for implementation
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processor
  • DSPD digital signal processing device
  • PLD programmable logic device
  • FPGA field programmable gate array
  • controller microcontroller, microprocessor, or other electronic components for implementation
  • microcontroller microprocessor
  • a computer-readable storage medium including program instructions is also provided.
  • the program instructions are executed by a processor, the above-mentioned image key point extraction model training method or image key point extraction method is implemented.
  • the computer-readable storage medium may be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions may be executed by the processor 701 of the electronic device 700 to implement the above training method or image key point extraction method for the image key point extraction model.
  • the electronic device 1900 may be provided as a server. 7, the electronic device 1900 may include: a processor 1922, the number of which may be one or more; and a memory 1932 for storing a computer program executable by the processor 1922.
  • the computer program stored in the memory 1932 may include one or more modules each corresponding to a set of instructions.
  • the processor 1922 may be configured to execute the computer program to perform the above-mentioned training method of the image key point extraction model or image key point extraction method.
  • the electronic device 1900 may further include a power supply component 1926 and a communication component 1950, which may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to implement communication of the electronic device 1900, for example, wired Or wireless communication.
  • the electronic device 1900 may also include an input/output (I/O) interface 1958.
  • the electronic device 1900 can operate an operating system based on the memory 1932, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM, and so on.
  • a computer-readable storage medium including program instructions is also provided.
  • the program instructions are executed by a processor, the above method for training an image keypoint extraction model or image keypoint extraction method is implemented .
  • the computer-readable storage medium may be the above-mentioned memory 1932 including program instructions, and the above-mentioned program instructions may be executed by the processor 1922 of the electronic device 1900 to complete the above training method or image key point extraction method for the image key point extraction model.

Abstract

A method for training an image key point extraction model. The image key point extraction model comprises multiple cascaded sub models. The method comprises: inputting a training image into the image key point extraction model, obtaining a key point outputted by each sub model, and taking the key points as the primary training of the image key point extraction model (S11); for each sub model, determining a difference between the key point outputted by the sub model and the key point corresponding to the degree identifier of the sub model in the training image (S12), wherein the degree identifier is used for characterizing the difficulty level of key point extraction; and determining the sum of the differences corresponding to all the sub models to be the target difference of the image key point extraction model, and when the number of times of training of the image key point extraction model does not reach a preset number of times, updating the image key point extraction model according to the target difference (S13). By respectively processing the key points having different difficulty levels, the present invention can improve the accuracy and the applicable range of the image key point extraction model.

Description

对图像关键点提取模型的训练及图像关键点提取Training of image key point extraction model and image key point extraction
相关申请的交叉引用Cross-reference of related applications
本申请要求2018年12月27日在中国知识产权局提交的中国专利申请No.201811615301.X的优先权,通过引用将该中国专利申请公开的全部内容并入本文。This application requires the priority of China Patent Application No. 201811615301.X filed at the China Intellectual Property Office on December 27, 2018. The entire contents of the disclosure of this Chinese patent application are incorporated herein by reference.
技术领域Technical field
本公开涉及图像处理领域,具体地,涉及对图像关键点提取模型的训练及图像关键点提取。The present disclosure relates to the field of image processing, and in particular, to training of image key point extraction models and image key point extraction.
背景技术Background technique
现有技术中,在进行图像关键点提取时,通常通过卷积神经网络提取图像的关键点,使用标记的图像对图像关键点提取模型进行统一训练。然而,图像清晰度的不同或是拍摄环境的不同可能导致提取图像中的关键点的难易程度不同。因此,在使用这样的图像进行统一训练时,获得的网络的适用性较小,准确度较低。In the prior art, when performing image key point extraction, the key points of the image are usually extracted through a convolutional neural network, and the labeled image is used to uniformly train the image key point extraction model. However, the difference in image clarity or the shooting environment may result in difficulty in extracting key points in the image. Therefore, when using such images for unified training, the obtained network has less applicability and lower accuracy.
发明内容Summary of the invention
根据本公开的第一方面,提供一种对图像关键点提取模型的训练方法,所述图像关键点提取模型包括多个级联的子模型,所述方法包括:According to a first aspect of the present disclosure, there is provided a training method for an image key point extraction model, the image key point extraction model includes a plurality of cascaded sub-models, the method includes:
将训练图像输入图像关键点提取模型,获得各个子模型输出的关键点,作为对所述图像关键点提取模型的一次训练;Input the training image into the image key point extraction model to obtain the key points output by each sub-model as a training for the image key point extraction model;
针对每个子模型,确定该子模型输出的关键点与所述训练图像中与该子模型的程度标识对应的关键点之间的差异,其中,所述程度标识用于表征关键点提取的难易程度;For each sub-model, determine the difference between the key points output by the sub-model and the key points in the training image corresponding to the degree identifier of the sub-model, where the degree identifier is used to characterize the difficulty of key point extraction degree;
将所述各个子模型对应的差异之和确定为所述图像关键点提取模型的目 标差异,在对所述图像关键点提取模型的训练次数未达到预设次数时,根据所述目标差异更新所述图像关键点提取模型。The sum of the differences corresponding to the sub-models is determined as the target difference of the image key point extraction model, and when the training times of the image key point extraction model does not reach the preset number, the The image key point extraction model is described.
可选地,在更新所述图像关键点提取模型之后,返回所述将训练图像输入图像关键点提取模型,获得各个子模型输出的关键点的步骤,直到对所述图像关键点提取模型的训练次数达到所述预设次数为止。Optionally, after updating the image key point extraction model, return to the step of inputting the training image into the image key point extraction model to obtain the key points output by each sub-model until the training of the image key point extraction model The number of times reaches the preset number of times.
可选地,所述图像关键点提取模型中的第一个子模型的输入为所述训练图像中人体图像部分的特征图,所述图像关键点提取模型中的除所述第一个子模型之外的子模型的输入为上一子模型输出的关键点和所述训练图像中人体图像部分的特征图。Optionally, the input of the first sub-model in the image key point extraction model is a feature map of the human body image portion in the training image, and the image key-point extraction model except the first sub-model The input of the external sub-model is the key points output by the previous sub-model and the feature map of the human image part in the training image.
可选地,所述训练图像中人体图像部分的特征图通过以下方式确定:Optionally, the feature map of the human image part in the training image is determined in the following manner:
提取所述训练图像的人体图像部分对应的第一图像;Extract the first image corresponding to the human image part of the training image;
将所述第一图像对应的分辨率调整至预设分辨率,获得第二图像,并根据所述第二图像确定所述训练图像中人体图像部分的特征图。The resolution corresponding to the first image is adjusted to a preset resolution, a second image is obtained, and the feature map of the human body image part in the training image is determined according to the second image.
根据本公开的第二方面,提供一种图像关键点提取方法,所述方法包括:According to a second aspect of the present disclosure, an image key point extraction method is provided, the method including:
接收目标图像,所述目标图像中包含人体图像部分;Receiving a target image, the target image containing a human body image portion;
将所述目标图像输入图像关键点提取模型,将所述图像关键点提取模型的最后一个子模型输出的关键点确定为所述目标图像中人体图像部分的关键点,其中,所述图像关键点提取模型包括多个级联的子模型,所述图像关键点提取模型为根据上述第一方面的方法训练得到的。Input the target image into an image key point extraction model, and determine the key points output by the last sub-model of the image key point extraction model as key points of the human body image part in the target image, wherein the image key points The extraction model includes multiple cascaded sub-models, and the image key point extraction model is obtained by training according to the method of the first aspect.
根据本公开的第三方面,提供一种对图像关键点提取模型的训练装置,所述图像关键点提取模型包括多个级联的子模型,所述装置包括:According to a third aspect of the present disclosure, there is provided a training device for extracting a model of an image key point, the image key point extraction model includes a plurality of cascaded sub-models, and the device includes:
处理模块,用于将训练图像输入图像关键点提取模型,获得各个子模型输出的关键点,作为对所述图像关键点提取模型的一次训练;The processing module is used to input the training image into the image key point extraction model to obtain the key points output by each sub-model as a training for the image key point extraction model;
第一确定模块,用于针对每个子模型,确定该子模型输出的关键点与所述训练图像中与该子模型的程度标识对应的关键点之间的差异,其中,所述程度标识用于表征关键点提取的难易程度;The first determining module is used to determine, for each sub-model, the difference between the key points output by the sub-model and the key points in the training image corresponding to the degree identifier of the sub-model, wherein the degree identifier is used to Characterize the difficulty of key point extraction;
更新模块,用于将所述各个子模型对应的差异之和确定为所述图像关键点提取模型的目标差异,在对所述图像关键点提取模型的训练次数未达到预设次数时,根据所述目标差异更新所述图像关键点提取模型。The update module is used to determine the sum of the differences corresponding to the respective sub-models as the target difference of the image key point extraction model. When the training times of the image key point extraction model does not reach the preset number, the The target difference updates the image key point extraction model.
可选地,在所述更新模块更新所述图像关键点提取模型之后,所述处理模块将训练图像输入更新后的图像关键点提取模型,获得各个子模型输出的关键点,直到对所述图像关键点提取模型的训练次数达到所述预设次数为止。Optionally, after the update module updates the image key point extraction model, the processing module inputs the training image into the updated image key point extraction model to obtain the key points output by each sub-model until the image The training times of the key point extraction model reach the preset times.
可选地,所述图像关键点提取模型中的第一个子模型的输入为所述训练图像中人体图像部分的特征图,所述图像关键点提取模型中的除所述第一个子模型之外的子模型的输入为上一子模型输出的关键点和所述训练图像中人体图像部分的特征图。Optionally, the input of the first sub-model in the image key point extraction model is a feature map of the human body image portion in the training image, and the image key-point extraction model except the first sub-model The input of the external sub-model is the key points output by the previous sub-model and the feature map of the human image part in the training image.
可选地,所述装置包括还包括特征提取模块,用于获得所述训练图像中人体图像部分的特征图,所述特征提取模块包括:Optionally, the device further includes a feature extraction module for obtaining a feature map of a human image part in the training image, the feature extraction module includes:
提取子模块,用于提取所述训练图像的人体图像部分对应的第一图像;An extraction sub-module for extracting the first image corresponding to the human body image part of the training image;
调整子模块,用于将所述第一图像对应的分辨率调整至预设分辨率,获得第二图像,并根据所述第二图像确定所述训练图像中人体图像部分的特征图。The adjustment sub-module is used to adjust the resolution corresponding to the first image to a preset resolution, obtain a second image, and determine the feature map of the human body image portion in the training image according to the second image.
根据本公开的第四方面,提供一种图像关键点提取装置,所述装置包括:According to a fourth aspect of the present disclosure, there is provided an image key point extraction device, the device comprising:
接收模块,用于接收目标图像,所述目标图像中包含人体图像部分;The receiving module is used to receive a target image, the target image includes a human body image portion;
第二确定模块,用于将所述目标图像输入图像关键点提取模型,将所述图像关键点提取模型的最后一个子模型输出的关键点确定为所述目标图像中人体图像部分的关键点,其中,所述图像关键点提取模型包括多个级联的子模型,所述图像关键点提取模型为根据上述第一方面的方法训练得到的。A second determining module, configured to input the target image into an image key point extraction model, and determine the key point output by the last sub-model of the image key point extraction model as the key point of the human body image part in the target image, Wherein, the image key point extraction model includes multiple cascaded sub-models, and the image key point extraction model is obtained by training according to the method of the first aspect.
根据本公开的第五方面,提供一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现上述第一方面的方法。According to a fifth aspect of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the method of the first aspect described above.
根据本公开的第六方面,提供一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现上述第二方面的方法。According to a sixth aspect of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the method of the second aspect described above.
根据本公开的第七方面,提供一种电子设备,包括:According to a seventh aspect of the present disclosure, an electronic device is provided, including:
存储器,其上存储有计算机程序;Memory, on which computer programs are stored;
处理器,用于执行所述存储器中的所述计算机程序,以实现第一方面的方法。A processor is configured to execute the computer program in the memory to implement the method of the first aspect.
根据本公开的第八方面,提供一种电子设备,包括:According to an eighth aspect of the present disclosure, an electronic device is provided, including:
存储器,其上存储有计算机程序;Memory, on which computer programs are stored;
处理器,用于执行所述存储器中的所述计算机程序,以实现上述第二方面 的方法。A processor is configured to execute the computer program in the memory to implement the method of the second aspect.
根据本公开的实施例,通过图像关键点提取模型的各个子模型输出关键点,并且针对每个子模型单独计算差异,从而可以使得图像关键点提取模型中的每个子模型可以关注到与其程度标识对应的关键点,从而便于对难易程度不同的关键点进行分别提取。并且,通过各个子模型的差异确定图像关键点提取模型的目标差异,实现对图像关键点提取模型的更新,可以有效保证图像关键点提取模型的准确度,通过针对不同难易程度的关键点进行分别处理,从而提高图像关键点提取模型的适用范围,提升用户使用体验。According to an embodiment of the present disclosure, each sub-model of the image key point extraction model outputs key points, and the difference is calculated separately for each sub-model, so that each sub-model in the image key point extraction model can focus on corresponding to its degree identification Key points to facilitate the extraction of key points with different degrees of difficulty. In addition, through the difference of each sub-model to determine the target difference of the image key point extraction model, to achieve the update of the image key point extraction model, it can effectively ensure the accuracy of the image key point extraction model, by targeting key points of different degrees of difficulty Separate processing, so as to improve the application range of the image key point extraction model and enhance the user experience.
附图说明BRIEF DESCRIPTION
附图是用来帮助对本公开的进一步理解,并且构成说明书的一部分,与下面的详细描述一起用于解释本公开,但并不构成对本公开的限制。在附图中:The drawings are used to help further understanding of the present disclosure, and constitute a part of the specification, together with the following detailed description to explain the present disclosure, but do not constitute a limitation of the present disclosure. In the drawings:
图1是根据本公开的示例性实施例的对图像关键点提取模型的训练方法的流程图;1 is a flowchart of a training method for an image keypoint extraction model according to an exemplary embodiment of the present disclosure;
图2是根据本公开的示例性实施例的获取训练图像中人体图像部分的特征图的方法的流程图;2 is a flowchart of a method of acquiring a feature map of a human image portion in a training image according to an exemplary embodiment of the present disclosure;
图3是根据本公开的示例性实施例的图像关键点提取方法的流程图;3 is a flowchart of an image key point extraction method according to an exemplary embodiment of the present disclosure;
图4是根据本公开的示例性实施例的对图像关键点提取模型的训练装置的框图;4 is a block diagram of a training device for an image keypoint extraction model according to an exemplary embodiment of the present disclosure;
图5是根据本公开的示例性实施例的图像关键点提取装置的框图;5 is a block diagram of an image keypoint extraction device according to an exemplary embodiment of the present disclosure;
图6是根据本公开的示例性实施例的一种电子设备的框图;6 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure;
图7是根据本公开的示例性实施例的一种电子设备的框图。7 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
具体实施方式detailed description
以下结合附图对本公开的具体实施例进行详细说明。应当理解的是,此处所描述的具体实施例仅用于说明和解释本公开,并不用于限制本公开。The specific embodiments of the present disclosure will be described in detail below with reference to the drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure, and are not intended to limit the present disclosure.
图1为根据本公开的示例性实施例的对图像关键点提取模型的训练方法的流程图,该图像关键点提取模型包括多个级联的子模型。FIG. 1 is a flowchart of a training method for an image keypoint extraction model according to an exemplary embodiment of the present disclosure, the image keypoint extraction model including multiple cascaded sub-models.
如图1所示,在步骤S11中,将训练图像输入图像关键点提取模型,获 得各个子模型输出的关键点,作为对图像关键点提取模型的一次训练。As shown in FIG. 1, in step S11, the training image is input to the image key point extraction model, and the key points output by each sub-model are obtained as a training for the image key point extraction model.
其中,可以从数据库或者互联网上获取大量的图像。之后,对该图像中的关键点进行标记以确定训练图像。Among them, a large number of images can be obtained from a database or the Internet. After that, the key points in the image are marked to determine the training image.
在步骤S12中,针对每个子模型,确定该子模型输出的关键点与训练图像中与该子模型的程度标识对应的关键点之间的差异,其中,该程度标识用于表征关键点提取的难易程度。In step S12, for each sub-model, the difference between the key points output by the sub-model and the key points corresponding to the degree identifier of the sub-model in the training image is determined, where the degree identifier is used to characterize the key point extraction Degree of difficulty.
根据本公开的实施例,在对训练图像中的关键点信息进行标记时,可以对各个关键点提取的难易程度进行标记。作为示例,可以根据训练图像的属性进行标记,如,在高清晰度、高分辨率的训练图像中人体图像部分的关键点的提取较为容易,此时可以对该训练图像中的关键点标记第一程度标识,该第一程度标识用于表征该关键点的提取比较简单。在模糊、低分辨率的训练图像中人体图像部分的关键点的提取较为困难,可以对该训练图像中的关键点标记第二程度标识,该第二程度标识用于表征该关键点的提取比较困难。According to an embodiment of the present disclosure, when the key point information in the training image is marked, the difficulty of extracting each key point may be marked. As an example, you can mark according to the attributes of the training image. For example, in the high-definition and high-resolution training image, it is easier to extract the key points of the human image part. At this time, you can mark the key points in the training image. One-level identification, the first-level identification is used to characterize the extraction of the key point is relatively simple. It is difficult to extract the key points of the human body image part in the blurred, low-resolution training image. The key points in the training image can be marked with a second degree identifier, which is used to characterize the extraction and comparison of the key points difficult.
作为另一示例,可以直接对训练图像中的不同关键点标记程度标识。例如,对该训练图像中提取较难的关键点标记第二程度标识,对该训练图像中提取较为容易的关键点标记第一程度标识。以上为标记程度标识的示例性实现方式,不对本公开进行限定。As another example, different key points in the training image may be directly labeled with degree identifiers. For example, the key points that are more difficult to extract from the training image are labeled with the second degree identification, and the key points that are easier to extract from the training image are labeled with the first degree identification. The above is an exemplary implementation of the mark degree identification, and does not limit the present disclosure.
因此,在对图像关键点提取模型进行训练时,可以指定子模型对应的程度标识,其中,针对图像关键点提取模型中各个级联的子模型,按照其级联顺序各个子模型对应的关键点提取的难易程度为由易到难。根据本公开的实施例,第一个子模型对应的程度标识为第一程度标识,下一子模型对应的程度标识为第二程度标识。在确定子模型的差异时,针对第一个子模型,根据第一个子模型输出的关键点、与训练图像中的第一程度标识对应的关键点确定两个关键点之间的差异。针对下一子模型,根据该下一子模型输出的关键点、与训练图像中的第二程度标识对应的关键点确定两个关键点之间的差异。因此,在确定各个子模型对应的差异时,可以使得该子模型只关注该子模型中的程度标识所对应的关键点。Therefore, when training the image keypoint extraction model, you can specify the degree identifier corresponding to the submodel. Among them, for the image keypoint extraction model, each cascaded submodel in the model is extracted, and the keypoints corresponding to each submodel in the cascade order The difficulty of extraction is from easy to difficult. According to an embodiment of the present disclosure, the degree indicator corresponding to the first sub-model is a first degree indicator, and the degree indicator corresponding to the next sub-model is a second degree indicator. When determining the difference of the sub-models, for the first sub-model, the difference between the two key points is determined according to the key points output by the first sub-model and the key points corresponding to the first degree identification in the training image. For the next sub-model, the difference between the two key points is determined according to the key point output by the next sub-model and the key point corresponding to the second degree identifier in the training image. Therefore, when determining the difference corresponding to each sub-model, the sub-model can only focus on the key points corresponding to the degree identification in the sub-model.
在步骤S13中,将各个子模型对应的差异之和确定为图像关键点提取模型的目标差异,在对图像关键点提取模型的训练次数未达到预设次数时,根据 目标差异更新图像关键点提取模型。In step S13, the sum of the differences corresponding to the sub-models is determined as the target difference of the image key point extraction model. When the training times of the image key point extraction model do not reach the preset number, the image key point extraction is updated according to the target difference model.
其中,各个子模型对应的差异可以用于表征该子模型提取其对应程度标识的关键点的准确度。该差异越小,表征关键点的提取越准确。在确定出各个子模型对应的差异之后,可以将各个子模型对应的差异之和确定为图像关键点提取模型的目标差异,则可以根据各个子模型对应的差异综合表征图像关键点提取模型的差异,从而可以根据该目标差异对该图像关键点提取模型进行更新。Among them, the difference corresponding to each sub-model can be used to characterize the accuracy of extracting the key points identified by the corresponding degree of the sub-model. The smaller the difference, the more accurate the extraction of characterizing key points. After the differences corresponding to each sub-model are determined, the sum of the differences corresponding to each sub-model can be determined as the target difference of the image key point extraction model, then the differences of the key point extraction model of the image can be comprehensively characterized according to the differences corresponding to each sub-model , So that the key point extraction model of the image can be updated according to the target difference.
根据本公开的实施例,预设次数可以根据实际使用场景进行设置。例如,在精准度要求较高的场景下,该预设次数可以设置为较大;在精准度要求一般的场景下,预设次数可以设置为较小。According to an embodiment of the present disclosure, the preset number of times may be set according to actual usage scenarios. For example, in a scene with higher accuracy requirements, the preset number of times may be set to be larger; in a scene with general accuracy requirements, the preset number of times may be set to be smaller.
因此,根据本公开的实施例,通过图像关键点提取模型的各个子模型输出关键点,并且针对每个子模型单独计算差异,从而可以使得图像关键点提取模型中的每个子模型可以关注到与其程度标识对应的关键点,从而便于对难易程度不同的关键点进行分别提取。并且,通过各个子模型的差异确定图像关键点提取模型的目标差异,实现对图像关键点提取模型的更新,可以有效保证图像关键点提取模型的准确度,通过针对不同难易程度的关键点进行分别处理,从而提高图像关键点提取模型的适用范围,提升用户使用体验。Therefore, according to an embodiment of the present disclosure, each sub-model of the model extracts key points through image key point extraction, and the difference is calculated separately for each sub-model, so that each sub-model in the image key point extraction model can focus on its degree Identify the corresponding key points, so as to facilitate the extraction of key points with different degrees of difficulty. In addition, through the difference of each sub-model to determine the target difference of the image key point extraction model, to achieve the update of the image key point extraction model, it can effectively ensure the accuracy of the image key point extraction model, by targeting key points of different degrees of difficulty Separate processing, so as to improve the application range of the image key point extraction model and enhance the user experience.
根据本公开的实施例,在更新图像关键点提取模型之后,可以返回步骤S11,直到对图像关键点提取模型的训练次数达到预设次数为止。According to an embodiment of the present disclosure, after the image key point extraction model is updated, it may return to step S11 until the training times of the image key point extraction model reaches a preset number of times.
其中,更新图像关键点提取模型即根据目标差异对图像关键点提取模型中的权重参数进行调整,其可以通过现有的神经网络反馈更新方式实现,在此不再赘述。Among them, updating the image key point extraction model refers to adjusting the weight parameters in the image key point extraction model according to the target difference, which can be implemented through the existing neural network feedback update method, which will not be repeated here.
根据本公开的实施例,在重新返回将训练图像输入图像关键点提取模型,获得各个子模型输出的关键点的步骤时,所用的训练图像可以是之前使用过的训练图像,也可以使用新的训练图像,本公开对此不进行限定。在对图像关键点提取模型的训练次数达到预设次数时,完成对图像关键点提取模型的训练过程,获得准确的图像关键点提取模型,从而为图像关键点的提取提供支持。According to an embodiment of the present disclosure, when returning to the step of inputting the training image into the key point extraction model of the image to obtain the key points output by each sub-model, the training image used may be the training image used before or a new one The training image is not limited in this disclosure. When the training times of the image key point extraction model reaches a preset number, the training process of the image key point extraction model is completed to obtain an accurate image key point extraction model, thereby providing support for the extraction of the image key point.
根据本公开的实施例,图像关键点提取模型中的第一个子模型的输入为训练图像中人体图像部分的特征图,图像关键点提取模型中的除第一个子模 型之外的子模型的输入为上一子模型输出的关键点和训练图像中人体图像部分的特征图。According to an embodiment of the present disclosure, the input of the first sub-model in the image key point extraction model is the feature map of the human body image part in the training image, and the sub-models in the image key point extraction model other than the first sub-model The input of is the key points output by the previous sub-model and the feature map of the human image part in the training image.
在该实施例中,针对图像关键点提取模型中的除第一个子模型之外的子模型,其输入为上一子模型输出的关键点和训练图像中人体图像部分的特征图。因此,在当前子模型进行关键点提取时,可以基于上一子模型输出的关键点进行确定,从而可以有效简化图像关键点提取的流程,避免重复的数据处理与计算,提高图像关键点提取模型的效率。In this embodiment, for the key points of the image, the sub-models other than the first sub-model are extracted from the model, and the inputs are the key points output by the previous sub-model and the feature map of the human image part in the training image. Therefore, when the current sub-model performs key point extraction, it can be determined based on the key points output by the previous sub-model, which can effectively simplify the image key point extraction process, avoid repeated data processing and calculation, and improve the image key point extraction model s efficiency.
根据本公开的实施例,训练图像中人体图像部分的特征图通过以下方式确定,如图2所示:According to an embodiment of the present disclosure, the feature map of the human image part in the training image is determined in the following manner, as shown in FIG. 2:
在步骤S21中,提取训练图像的人体图像部分对应的第一图像,其中,可以通过现有的人体识别提取算法提取出第一图像。根据本公开的实施例,可以通过faster-rcnn算法或者maskrcnn算法对训练图像中的人体图像进行提取。In step S21, the first image corresponding to the human body image part of the training image is extracted, wherein the first image can be extracted by an existing human body recognition extraction algorithm. According to an embodiment of the present disclosure, the human body image in the training image may be extracted through the faster-rcnn algorithm or the maskrcnn algorithm.
在步骤S22中,将第一图像对应的分辨率调整至预设分辨率,获得第二图像,并根据第二图像确定训练图像中人体图像部分的特征图。In step S22, the resolution corresponding to the first image is adjusted to a preset resolution, a second image is obtained, and the feature map of the human image portion in the training image is determined according to the second image.
不同的训练图像中人体图像部分对应的占比可能相同也可能不同。例如,训练图像是同一用户通过连拍获得的,其中人体图像部分对应的占比一般类似,而对于不同用户拍摄的图像而言,其中人体图像部分对应的占比一般不同。因此,为了便于对训练图像中的人体图像部分进行统一的处理。在本实施例中,可以在提取出训练图像中的人体图像部分对应的第一图像之后,将该第一图像的分辨率调整到预设分辨率以获得第二图像。例如,预设分辨率可以是400*600。当提取出的第一图像的分辨率小于该预设分辨率时,可以通过放大图像的方式使得第一图像的分辨率为400*600;当提取出的第一图像的分辨率大于该预设分辨率时,可以通过缩小图像的方式使得第一图像的分辨率为400*600。其中,对图像进行放大或缩小的方式为现有技术,在此不再赘述。The corresponding proportions of human image parts in different training images may be the same or different. For example, the training images are obtained by the same user through continuous shooting, where the proportions corresponding to the human body image parts are generally similar, and for images taken by different users, the proportions corresponding to the human body image parts are generally different. Therefore, in order to facilitate uniform processing of the human body image portion in the training image. In this embodiment, after the first image corresponding to the human body image portion in the training image is extracted, the resolution of the first image may be adjusted to a preset resolution to obtain the second image. For example, the preset resolution may be 400*600. When the resolution of the extracted first image is less than the preset resolution, the resolution of the first image can be made 400*600 by enlarging the image; when the resolution of the extracted first image is greater than the preset For resolution, the resolution of the first image can be reduced to 400*600 by reducing the image. Among them, the way to enlarge or reduce the image is the prior art, and will not be repeated here.
因此,根据本公开的实施例,可以根据不同的训练图像提取出分辨率一致的特征图,便于对特征图进行统一的处理,有效简化处理流程,提高处理速度。同时,贴合用户的使用需求,便于用户使用。Therefore, according to the embodiments of the present disclosure, feature maps with the same resolution can be extracted from different training images, which facilitates uniform processing of the feature maps, effectively simplifies the processing flow, and increases the processing speed. At the same time, it meets the user's needs and is convenient for users.
本公开的实施例还提供一种图像关键点提取方法,如图3所示,该方法包括:An embodiment of the present disclosure also provides an image key point extraction method. As shown in FIG. 3, the method includes:
在步骤S31中,接收目标图像,目标图像中包含人体图像部分,其中,可以通过faster-rcnn算法或者maskrcnn算法对目标图像中的人体图像进行检测。In step S31, a target image is received, and the target image contains a human body image part, wherein the human body image in the target image can be detected by a faster-rcnn algorithm or a maskrcnn algorithm.
在步骤S32中,将目标图像输入图像关键点提取模型,将图像关键点提取模型的最后一个子模型输出的关键点确定为目标图像中人体图像部分的关键点,其中,图像关键点提取模型包括多个级联的子模型,图像关键点提取模型为根据上述任一对图像关键点提取模型的训练方法训练得到的。In step S32, the target image is input to the image key point extraction model, and the key points output by the last sub-model of the image key point extraction model are determined as the key points of the human body image part in the target image, where the image key point extraction model includes For multiple cascaded sub-models, the image key point extraction model is trained according to any of the above training methods for the image key point extraction model.
在本实施例中,通过将目标图像输入至图像关键点提取模型,可以通过该提取目标图像中的关键点进行提取。基于该图像关键点提取模型可以对目标图像中的不同难易程度的关键点进行准确提取,一方面,可以保证关键点提取的全面性和完整性,另一方面,也可以有效保证关键点提取的准确度,为基于该关键点进行后续处理提供准确的数据支持,进一步提升用户使用体验。In this embodiment, by inputting the target image to the image key point extraction model, key points in the target image can be extracted. The key point extraction model based on the image can accurately extract key points of different degrees of difficulty in the target image. On the one hand, it can ensure the comprehensiveness and completeness of key point extraction, on the other hand, it can also effectively ensure the extraction of key points The accuracy of the system provides accurate data support for subsequent processing based on this key point, and further improves the user experience.
根据本公开的实施例,人体图像部分的关键点为人体图像部分对应的骨骼关键点,在确定出目标图像中的人体图像部分的骨骼关键点后,可以根据该骨骼关键点对目标图像中的人体图像部分进行姿态估计。由此可以提高人体图像部分对应的骨骼关键点的预测准确度,从而保证对目标图像中人体图像部分进行姿态估计的准确性。According to an embodiment of the present disclosure, the key points of the human body image part are the bone key points corresponding to the human body image part. After the bone key points of the human body image part in the target image are determined, the key points in the target image may be determined according to the bone key points Posture estimation is performed on the part of the human body image. Therefore, the prediction accuracy of the bone key points corresponding to the human body image part can be improved, thereby ensuring the accuracy of the pose estimation of the human body image part in the target image.
本公开的实施例还提供一种对图像关键点提取模型的训练装置,图像关键点提取模型包括多个级联的子模型,如图4所示,该装置10可以包括:An embodiment of the present disclosure also provides a training device for extracting a model of an image key point. The image key point extraction model includes multiple cascaded sub-models. As shown in FIG. 4, the device 10 may include:
处理模块100,用于将训练图像输入图像关键点提取模型,获得各个子模型输出的关键点,作为对图像关键点提取模型的一次训练;The processing module 100 is used to input a training image into an image key point extraction model to obtain key points output by each sub-model as a training for the image key point extraction model;
第一确定模块200,用于针对每个子模型,确定该子模型输出的关键点与训练图像中与该子模型的程度标识对应的关键点之间的差异,其中,程度标识用于表征关键点提取的难易程度;The first determination module 200 is used to determine, for each sub-model, the difference between the key points output by the sub-model and the key points in the training image corresponding to the degree identification of the sub-model, where the degree identification is used to characterize the key point Difficulty of extraction;
更新模块300,用于将各个子模型对应的差异之和确定为图像关键点提取模型的目标差异,在对图像关键点提取模型的训练次数未达到预设次数时,根据目标差异更新图像关键点提取模型。The update module 300 is used to determine the sum of the differences corresponding to the sub-models as the target difference of the image key point extraction model. When the training times of the image key point extraction model do not reach the preset number, update the image key point according to the target difference Extract the model.
可选地,在所述更新模块300更新所述图像关键点提取模型之后,处理模块可以将训练图像输入更新后的图像关键点提取模型,获得各个子模型输出的关键点,直到对图像关键点提取模型的训练次数达到预设次数为止。Optionally, after the update module 300 updates the image key point extraction model, the processing module may input the training image into the updated image key point extraction model to obtain the key points output by each sub-model until the image key points The training times of the extracted model have reached the preset times.
根据本公开的实施例,图像关键点提取模型中的第一个子模型的输入为训练图像中人体图像部分的特征图,图像关键点提取模型中的除第一个子模型之外的子模型的输入为上一子模型输出的关键点和训练图像中人体图像部分的特征图。According to an embodiment of the present disclosure, the input of the first sub-model in the image key point extraction model is the feature map of the human body image part in the training image, and the sub-models in the image key point extraction model other than the first sub-model The input of is the key points output by the previous sub-model and the feature map of the human image part in the training image.
根据本公开的实施例,所述装置还可以包括特征提取模块,用于获得训练图像中人体图像部分的特征图,特征提取模块可以包括:According to an embodiment of the present disclosure, the apparatus may further include a feature extraction module for obtaining a feature map of the human body image part in the training image. The feature extraction module may include:
提取子模块,用于提取训练图像的人体图像部分对应的第一图像;An extraction sub-module for extracting the first image corresponding to the body image part of the training image;
调整子模块,用于将第一图像对应的分辨率调整至预设分辨率,获得第二图像,并根据第二图像确定训练图像中人体图像部分的特征图。The adjustment submodule is used to adjust the resolution corresponding to the first image to a preset resolution, obtain a second image, and determine the feature map of the human image portion in the training image according to the second image.
本公开还提供一种图像关键点提取装置,如图5所示,该装置20可以包括:The present disclosure also provides an image key point extraction device. As shown in FIG. 5, the device 20 may include:
接收模块400,用于接收目标图像,目标图像中包含人体图像部分;The receiving module 400 is used to receive a target image, and the target image includes a human body image part;
第二确定模块500,用于将目标图像输入图像关键点提取模型,将图像关键点提取模型的最后一个子模型输出的关键点确定为目标图像中人体图像部分的关键点,其中,图像关键点提取模型包括多个级联的子模型,图像关键点提取模型为根据上述任一对图像关键点提取模型的训练方法训练得到的。The second determination module 500 is used to input the target image into the image keypoint extraction model, and determine the keypoint output by the last sub-model of the image keypoint extraction model as the keypoint of the human image part in the target image, where the image keypoint The extraction model includes multiple cascaded sub-models. The image key point extraction model is obtained by training according to any of the above training methods for the image key point extraction model.
关于上述实施例中的装置,其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述,此处将不做详细阐述说明。Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment related to the method, and will not be elaborated here.
图6是根据本公开的实施例的一种电子设备700的框图。如图6所示,该电子设备700可以包括:处理器701,存储器702。该电子设备700还可以包括多媒体组件703,输入/输出(I/O)接口704,以及通信组件705中的一者或多者。FIG. 6 is a block diagram of an electronic device 700 according to an embodiment of the present disclosure. As shown in FIG. 6, the electronic device 700 may include a processor 701 and a memory 702. The electronic device 700 may further include one or more of a multimedia component 703, an input/output (I/O) interface 704, and a communication component 705.
其中,处理器701用于控制该电子设备700的整体操作,以完成上述的图像关键点提取模型的训练方法或图像关键点提取方法中的全部或部分步骤。存储器702用于存储各种类型的数据以支持在该电子设备700的操作,这些数据例如可以包括用于在该电子设备700上操作的任何应用程序或方法的指令,以及应用程序相关的数据,例如联系人数据、收发的消息、图片、音频、视频等等。该存储器702可以由任何类型的易失性或非易失性存储设备或者它们的组合实现,例如静态随机存取存储器(Static Random Access Memory, SRAM),电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory,EEPROM),可擦除可编程只读存储器(Erasable Programmable Read-Only Memory,EPROM),可编程只读存储器(Programmable Read-Only Memory,简称PROM),只读存储器(Read-Only Memory,简称ROM),磁存储器,闪存,磁盘或光盘。多媒体组件703可以包括屏幕和音频组件。其中屏幕例如可以是触摸屏,音频组件用于输出和/或输入音频信号。例如,音频组件可以包括一个麦克风,麦克风用于接收外部音频信号。所接收的音频信号可以被进一步存储在存储器702或通过通信组件705发送。音频组件还包括至少一个扬声器,用于输出音频信号。I/O接口704为处理器701和其他接口模块之间提供接口,上述其他接口模块可以是键盘,鼠标,按钮等。这些按钮可以是虚拟按钮或者实体按钮。通信组件705用于该电子设备700与其他设备之间进行有线或无线通信。无线通信,例如Wi-Fi,蓝牙,近场通信(Near Field Communication,NFC),2G、3G、4G、NB-IOT、eMTC、或5G等,或它们中的一种或几种的组合,在此不做限定。因此相应的该通信组件707可以包括:Wi-Fi模块,蓝牙模块,NFC模块等等。The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the training method of the image key point extraction model or the image key point extraction method. The memory 702 is used to store various types of data to support operation on the electronic device 700, and the data may include, for example, instructions for any application or method for operating on the electronic device 700, and application-related data, For example, contact data, messages sent and received, pictures, audio, video, etc. The memory 702 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (Read -Only Memory (ROM for short), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and/or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or transmitted through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I/O interface 704 provides an interface between the processor 701 and other interface modules. The other interface modules may be a keyboard, a mouse, a button, and so on. These buttons can be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or 5G, etc., or a combination of one or more of them, in This is not limited. Therefore, the corresponding communication component 707 may include a Wi-Fi module, a Bluetooth module, an NFC module, and so on.
在一示例性实施例中,电子设备700可以被一个或多个应用专用集成电路(Application Specific Integrated Circuit,ASIC)、数字信号处理器(Digital Signal Processor,DSP)、数字信号处理设备(Digital Signal Processing Device,DSPD)、可编程逻辑器件(Programmable Logic Device,PLD)、现场可编程门阵列(Field Programmable Gate Array,FPGA)、控制器、微控制器、微处理器或其他电子元件实现,用于执行上述的对图像关键点提取模型的训练方法或图像关键点提取方法。In an exemplary embodiment, the electronic device 700 may be one or more application specific integrated circuits (Application Specific Integrated Circuit (ASIC), digital signal processor (Digital Signal Processor, DSP), digital signal processing device (Digital Signal Processing Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field Programmable Gate Array, FPGA), controller, microcontroller, microprocessor, or other electronic components for implementation The above training method for image key point extraction model or image key point extraction method.
在另一示例性实施例中,还提供了一种包括程序指令的计算机可读存储介质,该程序指令被处理器执行时实现上述的图像关键点提取模型的训练方法或图像关键点提取方法。例如,该计算机可读存储介质可以为上述包括程序指令的存储器702,上述程序指令可由电子设备700的处理器701执行以实现上述的对图像关键点提取模型的训练方法或图像关键点提取方法。In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When the program instructions are executed by a processor, the above-mentioned image key point extraction model training method or image key point extraction method is implemented. For example, the computer-readable storage medium may be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions may be executed by the processor 701 of the electronic device 700 to implement the above training method or image key point extraction method for the image key point extraction model.
图7是根据本公开的实施例的一种电子设备1900的框图。例如,电子设备1900可以被提供为服务器。参照图7,电子设备1900可以包括:处理器 1922,其数量可以为一个或多个;以及存储器1932,用于存储可由处理器1922执行的计算机程序。存储器1932中存储的计算机程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外,处理器1922可以被配置为执行该计算机程序,以执行上述的对图像关键点提取模型的训练方法或图像关键点提取方法。7 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. 7, the electronic device 1900 may include: a processor 1922, the number of which may be one or more; and a memory 1932 for storing a computer program executable by the processor 1922. The computer program stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processor 1922 may be configured to execute the computer program to perform the above-mentioned training method of the image key point extraction model or image key point extraction method.
另外,电子设备1900还可以包括电源组件1926和通信组件1950,该电源组件1926可以被配置为执行电子设备1900的电源管理,该通信组件1950可以被配置为实现电子设备1900的通信,例如,有线或无线通信。此外,该电子设备1900还可以包括输入/输出(I/O)接口1958。电子设备1900可以操作基于存储在存储器1932的操作系统,例如Windows Server TM、Mac OS X TM、Unix TM、Linux TM等等。 In addition, the electronic device 1900 may further include a power supply component 1926 and a communication component 1950, which may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to implement communication of the electronic device 1900, for example, wired Or wireless communication. In addition, the electronic device 1900 may also include an input/output (I/O) interface 1958. The electronic device 1900 can operate an operating system based on the memory 1932, such as Windows Server , Mac OS X , Unix , Linux ™, and so on.
在另一示例性实施例中,还提供了一种包括程序指令的计算机可读存储介质,该程序指令被处理器执行时实现上述的对图像关键点提取模型的训练方法或图像关键点提取方法。例如,该计算机可读存储介质可以为上述包括程序指令的存储器1932,上述程序指令可由电子设备1900的处理器1922执行以完成上述的对图像关键点提取模型的训练方法或图像关键点提取方法。In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When the program instructions are executed by a processor, the above method for training an image keypoint extraction model or image keypoint extraction method is implemented . For example, the computer-readable storage medium may be the above-mentioned memory 1932 including program instructions, and the above-mentioned program instructions may be executed by the processor 1922 of the electronic device 1900 to complete the above training method or image key point extraction method for the image key point extraction model.
以上结合附图详细描述了本公开的示例性实施例,但是,本公开并不限于上述实施例中的具体细节。在本公开的技术构思范围内,可以对本公开的实施例进行多种简单变型,这些简单变型均属于本公开的保护范围。The exemplary embodiments of the present disclosure have been described in detail above with reference to the drawings, however, the present disclosure is not limited to the specific details in the above-described embodiments. Within the scope of the technical concept of the present disclosure, various simple modifications can be made to the embodiments of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.
另外需要说明的是,在上述具体实施例中所描述的各个具体技术特征,在不矛盾的情况下,可以通过任何合适的方式进行组合。为了避免不必要的重复,本公开对各种可能的组合方式不再另行说明。In addition, it should be noted that the specific technical features described in the foregoing specific embodiments can be combined in any suitable manner without contradictions. In order to avoid unnecessary repetition, the present disclosure does not describe various possible combinations.
此外,本公开的各种不同的实施例之间也可以进行任意组合,只要其不违背本公开的构思,就应被视为在本公开的范围内。In addition, any combination of various embodiments of the present disclosure can also be arbitrarily combined, as long as it does not violate the concept of the present disclosure, it should be regarded as within the scope of the present disclosure.

Claims (11)

  1. 一种对图像关键点提取模型的训练方法,所述图像关键点提取模型包括多个级联的子模型,所述方法包括:A training method for an image key point extraction model. The image key point extraction model includes multiple cascaded sub-models. The method includes:
    将训练图像输入图像关键点提取模型,获得各个子模型输出的关键点,作为对所述图像关键点提取模型的一次训练;Input the training image into the image key point extraction model to obtain the key points output by each sub-model as a training for the image key point extraction model;
    针对每个子模型,确定该子模型输出的关键点与所述训练图像中与该子模型的程度标识对应的关键点之间的差异,其中,所述程度标识用于表征关键点提取的难易程度;For each sub-model, determine the difference between the key points output by the sub-model and the key points in the training image corresponding to the degree identifier of the sub-model, where the degree identifier is used to characterize the difficulty of key point extraction degree;
    将所述各个子模型对应的差异之和确定为所述图像关键点提取模型的目标差异,在对所述图像关键点提取模型的训练次数未达到预设次数时,根据所述目标差异更新所述图像关键点提取模型。The sum of the differences corresponding to the sub-models is determined as the target difference of the image key point extraction model, and when the training times of the image key point extraction model does not reach the preset number, the The image key point extraction model is described.
  2. 根据权利要求1所述的方法,其中,在更新所述图像关键点提取模型之后,返回所述将训练图像输入图像关键点提取模型,获得各个子模型输出的关键点的步骤,直到对所述图像关键点提取模型的训练次数达到所述预设次数为止。The method according to claim 1, wherein after updating the image keypoint extraction model, returning to the step of inputting the training image into the image keypoint extraction model to obtain the keypoints output by each sub-model until the The training times of the image key point extraction model reach the preset times.
  3. 根据权利要求1所述的方法,其中,所述图像关键点提取模型中的第一个子模型的输入为所述训练图像中人体图像部分的特征图,所述图像关键点提取模型中的除所述第一个子模型之外的子模型的输入为上一子模型输出的关键点和所述训练图像中人体图像部分的特征图。The method according to claim 1, wherein the input of the first sub-model in the image key point extraction model is a feature map of the human image part in the training image, and the image key point extraction model is divided by The inputs of the sub-models other than the first sub-model are the key points output by the previous sub-model and the feature map of the human image part in the training image.
  4. 根据权利要求3所述的方法,其中,所述训练图像中人体图像部分的特征图通过以下方式确定:The method according to claim 3, wherein the feature map of the human image part in the training image is determined in the following manner:
    提取所述训练图像的人体图像部分对应的第一图像;Extract the first image corresponding to the human image part of the training image;
    将所述第一图像对应的分辨率调整至预设分辨率,获得第二图像,并根据所述第二图像确定所述训练图像中人体图像部分的特征图。The resolution corresponding to the first image is adjusted to a preset resolution, a second image is obtained, and the feature map of the human body image part in the training image is determined according to the second image.
  5. 一种图像关键点提取方法,所述方法包括:An image key point extraction method, the method includes:
    接收目标图像,所述目标图像中包含人体图像部分;Receiving a target image, the target image containing a human body image portion;
    将所述目标图像输入图像关键点提取模型,将所述图像关键点提取模型的最后一个子模型输出的关键点确定为所述目标图像中人体图像部分的关键点,其中,所述图像关键点提取模型包括多个级联的子模型,所述图像关键点提取模型为根据权利要求1-4中任一项所述的方法训练得到的。Input the target image into an image key point extraction model, and determine the key points output by the last sub-model of the image key point extraction model as key points of the human body image part in the target image, wherein the image key points The extraction model includes multiple cascaded sub-models, and the image key point extraction model is obtained by training according to any one of claims 1-4.
  6. 一种对图像关键点提取模型的训练装置,所述图像关键点提取模型包括多个级联的子模型,所述装置包括:A training device for extracting a model of an image key point. The image key point extraction model includes a plurality of cascaded sub-models. The device includes:
    处理模块,用于将训练图像输入图像关键点提取模型,获得各个子模型输出的关键点,作为对所述图像关键点提取模型的一次训练;The processing module is used to input the training image into the image key point extraction model to obtain the key points output by each sub-model as a training for the image key point extraction model;
    第一确定模块,用于针对每个子模型,确定该子模型输出的关键点与所述训练图像中与该子模型的程度标识对应的关键点之间的差异,其中,所述程度标识用于表征关键点提取的难易程度;The first determining module is used to determine, for each sub-model, the difference between the key points output by the sub-model and the key points in the training image corresponding to the degree identifier of the sub-model, wherein the degree identifier is used to Characterize the difficulty of key point extraction;
    更新模块,用于将所述各个子模型对应的差异之和确定为所述图像关键点提取模型的目标差异,在对所述图像关键点提取模型的训练次数未达到预设次数时,根据所述目标差异更新所述图像关键点提取模型。The update module is used to determine the sum of the differences corresponding to the respective sub-models as the target difference of the image key point extraction model. When the training times of the image key point extraction model does not reach the preset number, the The target difference updates the image key point extraction model.
  7. 一种图像关键点提取装置,所述装置包括:An image key point extraction device, the device includes:
    接收模块,用于接收目标图像,所述目标图像中包含人体图像部分;The receiving module is used to receive a target image, the target image includes a human body image portion;
    第二确定模块,用于将所述目标图像输入所述图像关键点提取模型,将所述图像关键点提取模型的最后一个子模型输出的关键点确定为所述目标图像中人体图像部分的关键点,其中,所述图像关键点提取模型包括多个级联的子模型,所述图像关键点提取模型为根据权利要求1-4中任一项所述的方法训练得到的。The second determination module is used for inputting the target image into the image key point extraction model, and determining the key point output by the last sub-model of the image key point extraction model as the key of the human body image part in the target image Point, wherein the image key point extraction model includes multiple cascaded sub-models, and the image key point extraction model is obtained by training according to any one of claims 1-4.
  8. 一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现权利要求1-4中任一项所述的方法。A computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the method of any one of claims 1-4.
  9. 一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现权利要求5所述的方法。A computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the method of claim 5.
  10. 一种电子设备,包括:An electronic device, including:
    存储器,其上存储有计算机程序;Memory, on which computer programs are stored;
    处理器,用于执行所述存储器中的所述计算机程序,以实现权利要求1-4中任一项所述的方法。A processor, configured to execute the computer program in the memory, to implement the method according to any one of claims 1-4.
  11. 一种电子设备,包括:An electronic device, including:
    存储器,其上存储有计算机程序;Memory, on which computer programs are stored;
    处理器,用于执行所述存储器中的所述计算机程序,以实现权利要求5所述的方法。A processor, configured to execute the computer program in the memory, to implement the method of claim 5.
PCT/CN2019/094740 2018-12-27 2019-07-04 Training of image key point extraction model and image key point extraction WO2020134010A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811615301.X 2018-12-27
CN201811615301.XA CN109753910B (en) 2018-12-27 2018-12-27 Key point extraction method, model training method, device, medium and equipment

Publications (1)

Publication Number Publication Date
WO2020134010A1 true WO2020134010A1 (en) 2020-07-02

Family

ID=66404087

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/094740 WO2020134010A1 (en) 2018-12-27 2019-07-04 Training of image key point extraction model and image key point extraction

Country Status (2)

Country Link
CN (1) CN109753910B (en)
WO (1) WO2020134010A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112053360A (en) * 2020-10-10 2020-12-08 腾讯科技(深圳)有限公司 Image segmentation method and device, computer equipment and storage medium
CN112270669A (en) * 2020-11-09 2021-01-26 北京百度网讯科技有限公司 Human body 3D key point detection method, model training method and related device
CN112614568A (en) * 2020-12-28 2021-04-06 东软集团股份有限公司 Inspection image processing method and device, storage medium and electronic equipment
CN114518801A (en) * 2022-02-18 2022-05-20 美的集团(上海)有限公司 Device control method, computer program product, control device, and storage medium
CN117079242A (en) * 2023-09-28 2023-11-17 比亚迪股份有限公司 Deceleration strip determining method and device, storage medium, electronic equipment and vehicle

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109753910B (en) * 2018-12-27 2020-02-21 北京字节跳动网络技术有限公司 Key point extraction method, model training method, device, medium and equipment
CN113468924A (en) * 2020-03-31 2021-10-01 北京沃东天骏信息技术有限公司 Key point detection model training method and device and key point detection method and device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104077613A (en) * 2014-07-16 2014-10-01 电子科技大学 Crowd density estimation method based on cascaded multilevel convolution neural network
CN107665351A (en) * 2017-05-06 2018-02-06 北京航空航天大学 The airfield detection method excavated based on difficult sample
WO2018052587A1 (en) * 2016-09-14 2018-03-22 Konica Minolta Laboratory U.S.A., Inc. Method and system for cell image segmentation using multi-stage convolutional neural networks
CN107909053A (en) * 2017-11-30 2018-04-13 济南浪潮高新科技投资发展有限公司 A kind of method for detecting human face based on grade study concatenated convolutional neutral net
CN109753910A (en) * 2018-12-27 2019-05-14 北京字节跳动网络技术有限公司 Crucial point extracting method, the training method of model, device, medium and equipment

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105404861B (en) * 2015-11-13 2018-11-02 中国科学院重庆绿色智能技术研究院 Training, detection method and the system of face key feature points detection model
CN106295567B (en) * 2016-08-10 2019-04-12 腾讯科技(深圳)有限公司 A kind of localization method and terminal of key point
CN106845398B (en) * 2017-01-19 2020-03-03 北京小米移动软件有限公司 Face key point positioning method and device
KR101993729B1 (en) * 2017-02-15 2019-06-27 동명대학교산학협력단 FACE RECOGNITION Technique using Multi-channel Gabor Filter and Center-symmetry Local Binary Pattern
CN106951840A (en) * 2017-03-09 2017-07-14 北京工业大学 A kind of facial feature points detection method
CN108230390B (en) * 2017-06-23 2021-01-01 北京市商汤科技开发有限公司 Training method, key point detection method, device, storage medium and electronic equipment
CN108960232A (en) * 2018-06-08 2018-12-07 Oppo广东移动通信有限公司 Model training method, device, electronic equipment and computer readable storage medium
CN109063584B (en) * 2018-07-11 2022-02-22 深圳大学 Facial feature point positioning method, device, equipment and medium based on cascade regression

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104077613A (en) * 2014-07-16 2014-10-01 电子科技大学 Crowd density estimation method based on cascaded multilevel convolution neural network
WO2018052587A1 (en) * 2016-09-14 2018-03-22 Konica Minolta Laboratory U.S.A., Inc. Method and system for cell image segmentation using multi-stage convolutional neural networks
CN107665351A (en) * 2017-05-06 2018-02-06 北京航空航天大学 The airfield detection method excavated based on difficult sample
CN107909053A (en) * 2017-11-30 2018-04-13 济南浪潮高新科技投资发展有限公司 A kind of method for detecting human face based on grade study concatenated convolutional neutral net
CN109753910A (en) * 2018-12-27 2019-05-14 北京字节跳动网络技术有限公司 Crucial point extracting method, the training method of model, device, medium and equipment

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112053360A (en) * 2020-10-10 2020-12-08 腾讯科技(深圳)有限公司 Image segmentation method and device, computer equipment and storage medium
CN112053360B (en) * 2020-10-10 2023-07-25 腾讯科技(深圳)有限公司 Image segmentation method, device, computer equipment and storage medium
CN112270669A (en) * 2020-11-09 2021-01-26 北京百度网讯科技有限公司 Human body 3D key point detection method, model training method and related device
CN112270669B (en) * 2020-11-09 2024-03-01 北京百度网讯科技有限公司 Human body 3D key point detection method, model training method and related devices
CN112614568A (en) * 2020-12-28 2021-04-06 东软集团股份有限公司 Inspection image processing method and device, storage medium and electronic equipment
CN114518801A (en) * 2022-02-18 2022-05-20 美的集团(上海)有限公司 Device control method, computer program product, control device, and storage medium
CN114518801B (en) * 2022-02-18 2023-10-27 美的集团(上海)有限公司 Device control method, control device, and storage medium
CN117079242A (en) * 2023-09-28 2023-11-17 比亚迪股份有限公司 Deceleration strip determining method and device, storage medium, electronic equipment and vehicle
CN117079242B (en) * 2023-09-28 2024-01-26 比亚迪股份有限公司 Deceleration strip determining method and device, storage medium, electronic equipment and vehicle

Also Published As

Publication number Publication date
CN109753910B (en) 2020-02-21
CN109753910A (en) 2019-05-14

Similar Documents

Publication Publication Date Title
WO2020134010A1 (en) Training of image key point extraction model and image key point extraction
CN109961780B (en) A man-machine interaction method a device(s) Server and storage medium
US20200043471A1 (en) Voice data processing method, voice interaction device, and storage medium
BR112016021658B1 (en) METHOD OF PRESENTATION OF CONTENT, METHOD OF PROMOTING MODE OF PRESENTATION OF CONTENT, AND INTELLIGENT TERMINAL
CN109711273B (en) Image key point extraction method and device, readable storage medium and electronic equipment
CN106778820B (en) Identification model determining method and device
CN108885787B (en) Method for training image restoration model, image restoration method, device, medium, and apparatus
CN109657539B (en) Face value evaluation method and device, readable storage medium and electronic equipment
WO2019019396A1 (en) Push result prediction method and apparatus, computer device and storage medium
US10291838B2 (en) Focusing point determining method and apparatus
WO2020207024A1 (en) Authority management method and related product
JP6309539B2 (en) Method and apparatus for realizing speech input
CN109658346B (en) Image restoration method and device, computer-readable storage medium and electronic equipment
CN109492531B (en) Face image key point extraction method and device, storage medium and electronic equipment
CN109697446B (en) Image key point extraction method and device, readable storage medium and electronic equipment
JP7024255B2 (en) Information processing equipment and programs
US11301669B2 (en) Face recognition system and method for enhancing face recognition
CN110427849B (en) Face pose determination method and device, storage medium and electronic equipment
WO2020103606A1 (en) Model processing method and device, terminal, and storage medium
WO2019144710A1 (en) Method and apparatus for determining position of pupil
CN108108102B (en) Image recommendation method and electronic equipment
JP5430636B2 (en) Data acquisition apparatus, method and program
CN111652382B (en) Data processing method, device and equipment based on block chain and storage medium
CN110288668B (en) Image generation method, device, computer equipment and storage medium
KR101647911B1 (en) Mobile authentication by image inpainting

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19901616

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19901616

Country of ref document: EP

Kind code of ref document: A1