WO2024007496A1 - 图像处理方法、装置、电子设备及存储介质 - Google Patents

图像处理方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2024007496A1
WO2024007496A1 PCT/CN2022/129443 CN2022129443W WO2024007496A1 WO 2024007496 A1 WO2024007496 A1 WO 2024007496A1 CN 2022129443 W CN2022129443 W CN 2022129443W WO 2024007496 A1 WO2024007496 A1 WO 2024007496A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
finger
bone
wrist
registered
Prior art date
Application number
PCT/CN2022/129443
Other languages
English (en)
French (fr)
Inventor
王思元
李亦彤
徐力有
Original Assignee
北京字跳网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字跳网络技术有限公司 filed Critical 北京字跳网络技术有限公司
Priority to EP22854407.8A priority Critical patent/EP4325435A1/en
Publication of WO2024007496A1 publication Critical patent/WO2024007496A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person

Definitions

  • Embodiments of the present disclosure relate to image processing technology, such as an image processing method, device, electronic device, and storage medium.
  • the present disclosure provides an image processing method, device, electronic device, and storage medium to improve the authenticity of the display of target parts in a display interface without passing through the mold, thereby improving user experience.
  • an embodiment of the present disclosure provides an image processing method, which method includes:
  • the skeleton chain to be registered includes wrist bone points to be registered and finger bone points to be registered, and the preset skeleton chain is Assume that the bone chain includes preset wrist bone points and preset finger bone points;
  • Wrist status information includes wrist rotation information and wrist posture information
  • the finger status information includes finger rotation information and finger posture information
  • the target part is displayed in the video frame to be processed.
  • embodiments of the present disclosure also provide an image processing device, which includes:
  • the skeleton chain determination module is configured to determine the skeleton chain to be registered and the preset skeleton chain of the target part in the video frame to be processed; wherein the skeleton chain to be registered includes a wrist bone point to be registered and a finger to be registered.
  • the preset skeletal chain includes preset wrist skeletal points and preset finger skeletal points;
  • the wrist determination module is configured to determine the wrist of the wrist bone point to be registered based on the first posture information of the wrist bone point to be registered and the second posture information of the preset wrist bone point. Status information; wherein the wrist status information includes wrist rotation information and wrist posture information;
  • the finger determination module is configured to determine the finger skeletal point to be registered based on the wrist status information, the third pose information of the finger skeletal point to be registered, and the fourth pose information of the preset finger skeletal point. Finger status information of finger bone points; wherein the finger status information includes finger rotation information and finger posture information;
  • a display module is configured to display the target part in the video frame to be processed based on the wrist status information and finger status information.
  • embodiments of the present disclosure also provide an electronic device, where the electronic device includes:
  • processors one or more processors
  • a storage device configured to store one or more programs
  • the one or more processors are caused to implement the image processing method as described in any one of the embodiments of the present disclosure.
  • embodiments of the present disclosure also provide a storage medium containing computer-executable instructions, characterized in that, when executed by a computer processor, the computer-executable instructions are used to perform any of the embodiments of the present disclosure.
  • Figure 1 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure
  • Figure 2 is a schematic diagram of a skeleton chain to be registered according to an embodiment of the present disclosure
  • Figure 3 is a schematic diagram of a preset skeleton chain provided by an embodiment of the present disclosure.
  • Figure 4 is a schematic diagram of an arm bone chain adjustment provided by an embodiment of the present disclosure.
  • FIG. 5 is a schematic flowchart of another image processing method provided by an embodiment of the present disclosure.
  • Figure 6 is a schematic diagram of a wrist bone point to be registered and at least one finger bone point to be registered that is directly related to the wrist bone point to be registered according to an embodiment of the present disclosure
  • Figure 7 is a schematic diagram of a preset wrist bone point and at least one preset finger bone point directly related to the preset wrist bone point provided by an embodiment of the present disclosure
  • Figure 8 is a schematic flowchart of another image processing method provided by an embodiment of the present disclosure.
  • Figure 9 is a schematic diagram of a finger bone point to be registered before rotation provided by an embodiment of the present disclosure.
  • Figure 10 is a schematic diagram of a finger bone point to be registered after rotation according to an embodiment of the present disclosure
  • Figure 11 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure.
  • FIG. 12 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • the term “include” and its variations are open-ended, ie, “including but not limited to.”
  • the term “based on” means “based at least in part on.”
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment”; and the term “some embodiments” means “at least some embodiments”. Relevant definitions of other terms will be given in the description below.
  • a prompt message is sent to the user to clearly remind the user that the operation requested will require the acquisition and use of the user's personal information. Therefore, users can autonomously choose whether to provide personal information to software or hardware such as electronic devices, applications, servers or storage media that perform the operations of the technical solution of the present disclosure based on the prompt information.
  • the method of sending prompt information to the user may be, for example, a pop-up window, and the prompt information may be presented in the form of text in the pop-up window.
  • the pop-up window can also contain a selection control for the user to choose "agree” or "disagree” to provide personal information to the electronic device.
  • the technical solution of the present disclosure can be applied to scenes where body animation exists in the display interface, such as animation redirection, artificial intelligence body recognition, etc.
  • Body animation can be triggered by buttons displayed on the display interface, or by a preset continuous motion process of the target part of the target object. For example: in the game, the user controls the target character in the display interface to release skills by pressing buttons or touching, which can cause the target part of the target character to move to show the special effects of the skills. At this time, when the body animation is displayed in the display interface, you can The solution provided by the embodiments of the present disclosure is used to achieve the effect of relocating bone parts.
  • the method provided by the embodiments of the present disclosure can be used.
  • Technical solutions are used for rendering and display to improve the display effect. If there is a target part with mutually dependent skeletal points in the skeletal model, for example, the target part may be a hand or a foot, etc., the technical solution of the embodiment of the present disclosure can be used to migrate the animation data.
  • the device for executing the image processing method provided by the embodiments of the present disclosure can be integrated into application software with image processing functions, and the software can be installed in an electronic device.
  • the electronic device can be a mobile terminal or a PC.
  • the application software may be a type of software for image/video processing. The specific application software will not be described in detail here, as long as it can realize image/video processing.
  • Figure 1 is a schematic flow chart of an image processing method provided by an embodiment of the disclosure.
  • the embodiment of the disclosure is suitable for processing and displaying target parts in a video frame to be processed.
  • the method can be executed by an image processing device.
  • the device can be implemented in the form of software and/or hardware, for example, through an electronic device, and the electronic device can be a mobile terminal, a PC or a server.
  • the method includes:
  • the video frame to be processed may be a video frame in a video containing the target part.
  • the target part may be a part including the wrist and fingers, for example, it may be a hand or a foot.
  • the bone chain to be registered may be the initial bone model of the target part to be registered, as shown in Figure 2.
  • the skeleton chain to be registered includes the wrist bone points to be registered and the finger bone points to be registered.
  • the wrist bone points to be registered can be the bone points corresponding to the wrist in the bone chain to be registered, as shown in Figure 2 Root'
  • the bone point of the finger to be registered can be the bone point corresponding to the finger in the bone chain to be registered, such as the bone points other than Root' in Figure 2.
  • the preset bone chain can be the bone model of the target part to be registered, as shown in Figure 3.
  • the preset bone chain includes preset wrist bone points and preset finger bone points.
  • the preset wrist bone point can be the bone point corresponding to the wrist in the preset bone chain, such as Root in Figure 3.
  • Preset The finger bone points can be bone points corresponding to the fingers in the preset bone chain, such as the bone points except Root in Figure 3.
  • the video frame to be processed can be obtained, and the target part in the video frame to be processed can be determined. Accordingly, the skeleton chain to be registered can be extracted according to the target part in the video frame to be processed, and the preset skeleton chain corresponding to the target part can be determined according to the action corresponding to the target part.
  • the preset skeletal chain may be a preset skeletal chain corresponding to the movements of different target parts.
  • the skeletal chain to be registered can be registered to the posture corresponding to the preset skeletal chain, that is, Render and display according to the preset bone chain to complete the movement of the target part.
  • the first posture information may be the position information and posture information of the wrist bone points to be registered.
  • the second posture information may be the position information and posture information of the preset wrist skeletal points.
  • the wrist status information may be information used to represent the wrist bone point registration result.
  • the wrist status information includes wrist rotation information and wrist posture information.
  • the wrist rotation information may be used to represent coordinate rotation transformation.
  • the wrist status information may be used to represent the coordinate rotation transformation.
  • the rotation matrix indicates that the wrist posture information can include position information and posture information.
  • the two need to be aligned.
  • Alignment processing can be performed based on the first posture information of the wrist bone points to be registered and the second posture information of the preset wrist bone points to determine the wrist status information required for registration of the wrist bone points to be registered. That is, the information required to convert the pose of the wrist to be registered.
  • a local coordinate system to be registered can be established based on the wrist bone points to be registered, a preset local coordinate system can be established based on the preset wrist bone points, and the wrist rotation information corresponding to the coordinate system transformation is determined. It can also determine the wrist posture information required to convert the wrist bone points to be registered to the preset wrist bone points, and then use the wrist rotation information and wrist posture information as wrist status information.
  • the third posture information may be the position information and posture information of the finger bone points to be registered.
  • the fourth posture information may be position information and posture information of preset finger bone points.
  • the finger status information may be information used to represent the finger bone point registration results.
  • the finger status information includes finger rotation information and finger posture information.
  • the finger rotation information may be represented by a finger rotation matrix.
  • the finger posture information Can include location information and attitude information.
  • the finger bone points are dependent on the wrist bone points and/or the finger bone points, and each finger bone point will be dependent on the wrist bone point, it may be a direct dependence or an indirect dependence. Therefore, alignment processing can be performed based on the wrist status information, the third pose information of the finger bone point to be registered, and the fourth pose information of the preset finger bone point to determine the finger of the finger bone point to be registered.
  • Status information that is, the information required to convert the pose of the finger to be registered.
  • the skeleton chain to be registered can be processed based on the wrist status information and finger status information to process the skeleton chain to be registered as the target part, and render it in the video frame to be processed, so that the target part is set according to the preset skeleton.
  • the pose information of the chain is accurately displayed in the video frame to be processed.
  • the display position and display posture of multiple skeletal points in the target part in the display interface can be adjusted based on the wrist status information and finger status information. , obtain the target video frame corresponding to the video frame to be processed.
  • the display position may be used to describe the position information, for example, it may be the position coordinate information of the skeletal point in the display interface, etc.
  • the display posture may be information used to describe the posture, for example, it may be the angle information between the skeleton point and the coordinate axis in the display page, etc.
  • the target video frame may be a video frame obtained after processing the target part in the video frame to be processed.
  • the wrist skeletal points of the target part in the video frame to be processed can be adjusted based on the wrist status information, and the wrist skeletal points of the target part in the video frame to be processed can be adjusted based on the finger status information.
  • the finger bone points are adjusted to determine the display position and display posture of multiple bone points in the target part, and based on the display position and display posture, each bone point of the target part is rendered to the display interface, and the video frame is used as the target video frame.
  • a skinning effect can be added to the bone chain of the target part to make the display of the target part closer to the skin and improve display effect.
  • the arm bone chain and/or the arm bone chain to which the wrist bone point to be registered belongs is determined. Or the length information of the leg bone chain; based on the length information and position offset, adjust the display position of at least one bone point in the arm bone chain and/or the leg bone chain.
  • the position offset may be the distance between the position information of the wrist bone point to be registered and the position information of the preset wrist bone point.
  • the preset offset threshold may be the maximum distance that the wrist bone point to be registered can offset.
  • the arm bone chain can be the corresponding bone chain when the target part is the arm.
  • the leg bone chain can be the corresponding bone chain when the target part is the leg.
  • the length information may be the length value of the arm bone chain and/or the leg bone chain.
  • the position offset between the wrist bone point to be registered and the preset wrist bone point is greater than the preset offset threshold, it means that it is impossible to directly adjust the wrist bone point to be registered.
  • Register to preset wrist bone points can be achieved by adjusting at least one bone point in the arm bone chain and/or the leg bone chain to which the wrist bone point to be registered belongs.
  • the length information of the arm bone chain and/or leg bone chain to which the wrist bone point to be registered can be determined, and at least one bone point in the arm bone chain and/or leg bone chain can be adjusted based on the length information and position offset.
  • display position For example, the display positions of multiple skeletal points can be determined by stretching in equal proportions, so that the wrist skeletal points to be registered can be registered according to the wrist status information.
  • the schematic diagram of arm bone chain adjustment is shown in Figure 4.
  • W is the wrist bone point to be registered
  • T is the preset wrist bone point.
  • the wrist bone point to be registered is The position offset from the preset wrist bone point is greater than the preset offset threshold, that is, the wrist bone point to be registered cannot reach the preset wrist bone point. Therefore, to lengthen the arm bone chain U-L-W, you can lengthen it proportionally based on the length information of the upper arm U-L and the length information of the lower arm L-W to correct the display positions of L and W, as shown in the lower figure of Figure 4 , so that the wrist bone point W to be registered can be registered with the preset wrist bone point T.
  • the wrist bone points to be registered in the bone chain to be registered in the animation can be combined with the preset wrist bones in the preset bone chain.
  • Point registration is used to obtain the wrist status information of the wrist bone points to be registered.
  • the fingers can be registered based on multiple preset finger bone points in the preset bone chain.
  • the pose information of the skeletal points determines the finger status information of multiple finger skeletal points to be registered, and then based on the wrist status information and finger status information, the target part and the display information in the video frame to be processed are adjusted to avoid In situations where the animation data of the target part is easy to wear through the mold when migrating, and when the display effect of the target part is poor and the authenticity is poor, without wearing the mold, the authenticity of the display of the target part in the display interface can be improved, and then Improve user experience.
  • FIG. 5 is a schematic flowchart of another image processing method provided by an embodiment of the present disclosure. Based on the foregoing embodiments, the wrist status information of the wrist bone points to be registered can be determined first. The specific implementation method can be found in Detailed elaboration of this technical solution. The explanations of terms that are the same as or corresponding to the above-mentioned embodiments will not be repeated here.
  • the method includes:
  • S220 Determine the first rotation information based on the first posture information of the wrist bone point to be registered and the third posture information of at least one finger bone point to be registered that is directly related to the wrist bone point to be registered.
  • the third posture information may be the position information and posture information of at least one finger bone point to be registered that is directly related to the wrist bone point to be registered.
  • the first rotation information may be the rotation information corresponding to the wrist bone point to be registered, and may be in the form of a rotation matrix or a quaternion.
  • At least one finger bone point to be registered that is directly related to the wrist bone point to be registered can be determined, which can be understood as extending from the wrist bone point to be registered to multiple fingertip directions and multiple extension directions.
  • the first finger bone point to be registered is the directly related finger bone point to be registered.
  • Root' in Figure 2 is the wrist bone point to be registered
  • U', P', K', F' and A' are all finger bone points to be registered that are directly related to the wrist bone points to be registered.
  • the wrist bone point to be registered can be determined based on the first posture information of the wrist bone point to be registered and the third posture information of at least one finger bone point to be registered that is directly related to the wrist bone point to be registered.
  • the corresponding rotation matrix and determine the first rotation information according to the rotation matrix.
  • At least one finger bone point to be registered includes a first finger bone point to be registered and a second finger bone point to be registered, it can be determined more quickly and accurately in the following manner First rotation information:
  • the first vector is determined; based on the first posture information and the posture information of the second finger bone point to be registered, the second vector is determined Vector; based on the first vector and the second vector, determine the first rotation information.
  • the first vector may be a vector from which the wrist bone point to be registered points to the first finger bone point to be registered.
  • the second vector may be a vector from which the wrist bone point to be registered points to the second finger bone point to be registered. It should be noted that the first finger bone point to be registered and the second finger bone point to be registered are any two of the finger bone points to be registered that are directly related to the wrist bone point to be registered. There is no specific limitation in this embodiment.
  • the wrist bone point to be registered points to the first finger bone point to be registered.
  • the vector of is the first vector.
  • the vector from the wrist bone point to be registered to the second finger bone point to be registered can be determined is the second vector.
  • a rotation matrix is constructed based on the first vertical vector, the second vertical vector, and the vector corresponding to the second vertical vector (the first vector or the second vector).
  • the rotation matrix can be used as the first rotation information, or the rotation can be
  • the matrix is transformed into a quaternion, and the quaternion is used as the first rotation information.
  • a schematic diagram of a wrist bone point to be registered and at least one finger bone point to be registered is directly related to the wrist bone point to be registered.
  • the wrist bone point to be registered is O’
  • the finger bone points to be registered that are directly related to the wrist bone point to be registered are A’, B’, C’, D’ and E’.
  • the fourth posture information may be the position information and posture information of at least one preset finger bone point directly associated with the preset wrist bone point.
  • the second rotation information may be rotation information corresponding to the preset wrist bone points, and may be in the form of a rotation matrix or a quaternion.
  • At least one preset finger bone point directly related to the preset wrist bone point can be determined, which can be understood as extending in multiple fingertip directions starting from the preset wrist bone point, and the first one in the multiple extension directions.
  • a default finger bone point is a directly related default finger bone point.
  • Root is the preset wrist bone point
  • V, Q, L, G, and A are all preset finger bone points directly related to the preset wrist bone point.
  • the rotation matrix corresponding to the preset wrist bone point may be determined based on the second posture information of the preset wrist bone point and the fourth posture information of at least one preset finger bone point directly related to the preset wrist bone point. , and determine the second rotation information according to the rotation matrix.
  • the second rotation can be determined more quickly and accurately in the following manner information:
  • the third vector may be a vector from which the preset wrist bone point points to the first preset finger bone point.
  • the fourth vector may be a vector from the preset wrist bone point to the second preset finger bone point. It should be noted that the first preset finger bone point and the second preset finger bone point are any two of the preset finger bone points directly related to the preset wrist bone point. In this embodiment, No specific restrictions are made.
  • the vector from the preset wrist bone point to the first preset finger bone point is the first preset finger bone point.
  • Three vectors According to the position information in the second pose information and the position information in the pose information of the second preset finger bone point, it can be determined that the vector from the preset wrist bone point to the second preset finger bone point is the fourth vector.
  • a rotation matrix is constructed based on the third vertical vector, the fourth vertical vector, and the vector corresponding to the fourth vertical vector (the third vector or the fourth vector).
  • the rotation matrix can be used as the second rotation information, or the rotation can be The matrix is transformed into a quaternion, and the quaternion is used as the second rotation information.
  • FIG. 7 a schematic diagram of a preset wrist bone point and at least one preset finger bone point directly associated with the preset wrist bone point is shown in FIG. 7 .
  • the preset wrist bone point is O
  • the preset finger bone points directly related to the preset wrist bone point are A, B, C, D and E.
  • the third vector is OC
  • the fourth vector is OB.
  • the rotation matrix can be determined according to left, up and forward, and the rotation matrix is converted to obtain the quaternion q, which is the second rotation information.
  • the wrist bone points to be registered can be registered according to the preset wrist bone points, and the registered information can be used as the wrist state of the wrist bone points to be registered. information.
  • an alignment transformation matrix can be introduced for processing, which can be:
  • the wrist status information is determined.
  • the alignment transformation matrix may be a rotation transformation matrix that aligns the wrist bone points to be registered with the preset wrist bone points.
  • the alignment transformation matrix is obtained by multiplying the second rotation information and the inverse of the first rotation information.
  • the position information and posture information in the wrist posture information can be obtained.
  • the following method can be used to calculate the position of the wrist bone points to be registered based on the alignment transformation matrix and the third wrist bone point to be registered.
  • a piece of posture information to determine the wrist posture information in the wrist status information :
  • the position information in the wrist pose information is determined; based on the alignment transformation matrix and the position information in the first pose information, the pose information in the wrist pose information is determined.
  • the position information in the second pose information is determined as the position information in the wrist pose information.
  • the position information in the first posture information is multiplied by the alignment transformation matrix to obtain the posture information in the wrist posture information.
  • the position information in the second pose information is Op
  • the position information O'p in the wrist pose information Op
  • the position information in the first posture information is Or and the alignment transformation matrix is r
  • the wrist posture information O'r in the wrist status information Or*r.
  • the first rotation information and the second rotation information can be determined accurately and quickly to unify the wrist bone points to be registered and the preset wrist bone points, improving the registration efficiency of the wrist bone points .
  • S250 Determine the finger status information of the finger bone point to be registered based on the wrist state information, the third pose information of the finger bone point to be registered, and the fourth pose information of the preset finger bone point.
  • the technical solution of the embodiment of the present disclosure is to determine the skeleton chain to be registered and the preset skeleton chain of the target part in the video frame to be processed, and based on the first posture information of the wrist bone point to be registered and the wrist to be registered
  • the first rotation information is determined based on the third posture information of at least one finger bone point to be registered that is directly related to the bone point
  • the second posture information of the preset wrist bone point is directly related to the preset wrist bone point.
  • the registered wrist bone points are registered according to the preset wrist bone points.
  • the third pose information of the finger bone points to be registered and the fourth pose information of the preset finger bone points, Determine the finger status information of the finger bone points to be registered.
  • display the target part in the video frame to be processed avoiding the mold-crossing phenomenon when the wrist animation data in the target part is migrated.
  • the resulting poor display effect and poor authenticity can improve the display effect and authenticity of the wrist in the target part without wearing the mold, thereby improving the user experience.
  • FIG. 8 is a schematic flowchart of another image processing method provided by an embodiment of the present disclosure.
  • the finger state information of multiple finger skeletal points to be registered can be determined in sequence.
  • the wrist state information of the wrist skeletal points to be registered can be determined in sequence.
  • the method includes:
  • the dependency information may be information used to describe the finger bone points or wrist bone points on which each finger bone point directly depends.
  • the dependency information includes finger bone points or wrist bone points on which at least one finger bone point located on the same branch in the bone chain directly depends.
  • the first dependency information corresponds to the skeleton chain to be registered, and the second dependence information corresponds to the preset skeleton chain.
  • connection information of the wrist bone points to be registered and multiple finger bone points to be registered in the bone chain to be registered the wrist bone points to be registered and the finger bone points to be registered can be set.
  • First rely on information.
  • Second dependency information of the preset wrist bone points and the preset finger bone points can also be set according to the connection information of the preset wrist bone points and the multiple preset finger bone points in the preset bone chain.
  • the first dependency information and the second dependence information can be retrieved.
  • the dependency information corresponding to the finger bone point extends from the finger bone point to the wrist, and the finger bone point or wrist bone point adjacent to the finger bone point in the extension direction is used as the dependency information. .
  • the bone chain to be registered takes the branch Root'-U'-V'-W'-X'-Y' as an example.
  • the finger bone point U' to be registered directly depends on is the wrist bone point Root' to be registered, that is, the first dependent information of U' is Root'; the finger bone point V' to be registered directly depends on the finger bone point U' to be registered, that is, the The first dependence information is U'; in the same way, the first dependence information corresponding to other finger bone points to be registered can be determined to obtain the first dependence information of the wrist bone points to be registered and the finger bone points to be registered.
  • the preset bone chain takes the branch Root-V-W-X-Y as an example.
  • the preset finger bone point V directly depends on the preset wrist bone point Root, that is, the second dependency information of V is Root; the preset The finger bone point W directly depends on the preset finger bone point V, that is, the second dependency information of W is V; in the same way, the second dependency information corresponding to other preset finger bone points can be determined to obtain the preset Secondary dependency information of wrist bone points and preset finger bone points.
  • S304 Determine the processing priorities of the finger bone points to be registered and the preset finger bone points according to the first dependency information and the second dependence information, so as to determine the fingers of the finger bone points to be registered in sequence based on the processing priorities. status information.
  • the processing priority may be a priority determined according to the dependency information. Taking any bone point as an example, the priority of the bone point is lower than the priority of the dependency information corresponding to the bone point.
  • the processing priority of multiple finger bone points to be registered may be determined according to the first dependency information, and the processing priority of the multiple preset finger bone points may be determined according to the second dependence information.
  • the processing priorities of multiple finger bone points to be registered and the processing priorities of multiple preset finger bone points can be determined, so that the finger bones to be registered are subsequently ordered in an orderly manner according to the processing priorities. Click to align.
  • the finger bone point group may be a combination of the bone chain to be registered and the corresponding finger bone points in the preset bone chain.
  • the finger bone point group includes preset finger bone points and finger bone points to be registered that are located on corresponding bone chain branches and have the same processing priority.
  • the branch Root'-A'-B'-C'-D'-E' in the skeleton chain to be registered is related to the branch Root-A-B-C-D-E in the preset skeleton chain.
  • the preset finger bone point A and the finger bone point to be registered A' have the same processing priority.
  • the preset finger bone point A and the finger bone point to be registered A' can be regarded as one finger bone point. group, other finger bone point groups can be determined in the same way as above.
  • the former dependent bone point group may be a wrist bone point group or a finger bone point group that the current finger bone point group depends on.
  • the subsequent dependent bone point group may be a finger bone point group that depends on the current finger bone point group.
  • the front-dependent bone point group includes the wrist bone point group or the finger bone point group.
  • the post-dependent bone point group includes the finger bone point group. It can be understood that the back-dependent bone point group directly depends on the current finger bone point group, and the current finger bone point group directly depends on the front-dependent bone point group.
  • the finger bone point group For example, for each finger bone point group, you can use the finger bone point group as the current finger bone point group, and use the same method to determine the front-dependent bone point group and back-dependency that the current finger bone point group depends on.
  • Bone point group According to the processing priority, determine the finger bone point group that directly depends on the current finger bone point group as the post-dependent bone point group, and determine the finger bone point group or wrist bone point group that is directly dependent on the current finger bone point group as Pre-dependent bone point group.
  • the finger bone point group consisting of the preset finger bone point A and the finger bone point to be registered A' is used as the current finger bone point group, according to the processing priority , it can be determined that the pre-dependent bone point group on which the current finger bone point group depends is the wrist bone point group composed of the preset wrist bone point Root and the wrist bone point to be registered Root', and the current finger bone point can be determined
  • the post-dependent bone point group on which the group depends is the finger bone point group composed of the preset wrist bone point B and the wrist bone point to be registered B'.
  • the front-dependent bone point group and the rear-dependent bone point group on which other finger bone point groups depend can be determined in the same manner as above.
  • S307. Determine whether the former dependent bone point group is the wrist bone point group or the finger bone point group. Based on the judgment result that the former dependent bone point group is the wrist bone point group, execute S308; based on the former dependent bone point group, it is the finger bone point group. Click the group judgment result and execute S309.
  • the rotation matrix of the bone points to be registered in the current finger bone point group can be determined.
  • the current finger rotation information of the bone points to be registered in the current finger bone point group can be determined.
  • the current finger posture information of the bone points to be registered in the current finger bone point group can be determined.
  • the current finger rotation information and the current finger posture information are determined as the finger state information of the finger bone points to be registered in the current finger bone point group.
  • S309 Determine based on the target finger rotation information in the finger status information of the skeleton points to be registered in the previous dependent skeleton point group, the pose information in the current finger skeleton point group, and the pose information of the posterior dependent skeleton point group. Finger status information of the finger bone point to be registered in the current finger bone point group, and execute S310.
  • the rotation matrix of the bone points to be registered in the current finger bone point group can be determined.
  • the current finger rotation information of the bone points to be registered in the current finger bone point group can be determined.
  • the current finger posture information of the bone points to be registered in the current finger bone point group can be determined.
  • the current finger rotation information and the current finger posture information are determined as the finger state information of the finger bone points to be registered in the current finger bone point group.
  • the finger status information of multiple finger bone points to be registered in each branch of the bone chain to be registered can be determined sequentially to ensure that the multiple finger bone points to be registered are
  • the finger rotation information and finger posture information are accurately registered with multiple preset finger bone points, improving the accuracy and efficiency of registration.
  • the current finger rotation information and the current finger posture information of the skeletal points to be registered in the current finger skeletal point group can be determined in the following manner, so as to determine the current finger skeletal point group to be Register the finger status information of the finger bone points to reduce the complexity of determining the finger status information and improve the accuracy of determining the finger status information:
  • the current finger is determined based on the rotation information of the target finger, the third pose information and the fourth pose information in the current finger bone point group, and the third pose information and the fourth pose information in the subsequent dependent bone point group.
  • the direction of the finger bone to be currently registered can be determined based on the third pose information in the current finger bone point group and the third pose information in the subsequent dependent bone point group.
  • the direction of the current preset finger bone can be determined based on the fourth posture information in the current finger bone point group and the fourth posture information in the subsequent dependent bone point group.
  • the rotation matrix required to rotate from the direction of the current finger bone to be registered to the direction of the current preset finger bone can be determined.
  • the current finger rotation information of the bone points to be registered in the current finger bone point group can be determined based on the target finger rotation information and the determined rotation matrix.
  • the current finger posture information of the bone points to be registered in the current finger bone point group is determined.
  • the position information of the current finger pose information of the skeletal points to be registered in the current finger skeletal point group can be determined.
  • the posture information of the bone points to be registered in the current finger bone point group can be determined. The position information and posture information determined above are used as the current finger posture information of the bone points to be registered in the current finger bone point group.
  • the finger status information of the skeletal point to be registered is determined.
  • the current finger rotation information and the current finger posture information are used as the finger state information of the skeletal points to be registered.
  • the current finger rotation information of the bone points to be registered in the current finger bone point group can be determined in the following way, so that the current finger rotation information can be smoothly transitioned, so that the subsequent finger rotation information can be determined according to the current finger rotation information.
  • the display effect when displayed is more realistic.
  • the third rotation vector may be a vector in which the finger bone point to be registered in the current finger bone point group points to the finger bone point to be registered in the dependent bone point group.
  • the fourth rotation vector may be a vector in which the preset finger bone point in the current finger bone point group points to the preset finger bone point in the dependent bone point group.
  • the position information in the third pose information in the current finger bone point group and the position information in the third pose information in the subsequent dependent bone point group it is possible to determine the position information to be matched in the current finger bone point group.
  • the vector of the finger bone point to be registered after pointing to the dependent bone point group is the third rotation vector.
  • the preset finger in the current finger skeleton point group can be determined
  • the vector that the bone point points to depends on the preset finger bone point in the bone point group is the fourth rotation vector.
  • the rotation quaternion that rotates from the direction of the third rotation vector to the direction of the fourth rotation vector can be obtained, which is the rotation matrix.
  • the current finger rotation information of the bone points to be registered in the current finger bone point group can be obtained.
  • FIG. 9 An exemplary schematic diagram of the finger bone point to be registered before rotation is shown in Figure 9, where R-S-T is the preset bone chain, R'-S'-T' is the bone chain to be registered, and S' and S represent The current finger bone point group, T' and T represent the posterior dependent bone point group, R' and R represent the anterior dependent bone point group.
  • the position information in the third pose information in the current finger bone point group is Sp'
  • the position information in the third pose information in the subsequent dependent bone point group is Tp'
  • the third rotation vector O T'p -S'p
  • the position information in the fourth pose information in the current finger bone point group is Sp
  • the position information in the fourth pose information in the subsequent dependent bone point group is Tp
  • the fourth rotation vector C Tp-Sp.
  • the target finger rotation information is R’r
  • the schematic diagram after rotating the finger bone points to be registered is shown in Figure 10.
  • the technical solution of the embodiment of the present disclosure determines the skeleton chain to be registered and the preset skeleton chain of the target part in the video frame to be processed, and based on the first posture information of the wrist skeleton point to be registered and the preset wrist skeleton point
  • the second pose information determine the wrist status information of the wrist bone points to be registered, retrieve the preset first dependency information of the wrist bone points to be registered and the finger bone points to be registered, and the preset The second dependency information of the wrist bone point and the preset finger bone point, based on the first dependence information and the second dependence information, determines the processing priority of the finger bone point to be registered and the preset finger bone point, based on The processing priority determines the finger status information of the finger bone point to be registered in turn, determines at least one finger bone point group, and for at least one finger bone point group, determines the current finger bone point group depends on according to the processing priority.
  • the front-dependent bone point group and the rear-dependent bone point group determine whether the front-dependent bone point group is the wrist bone point group or the finger bone point group, so as to determine the current finger bone point group to be selected based on different judgment results.
  • Register the finger status information of the finger bone points and display the target part in the video frame to be processed based on the wrist status information and finger status information, avoiding the display due to the mold-crossing phenomenon when the finger animation data in the target part is migrated
  • the display effect and authenticity of the middle finger of the target part can be improved without wearing the mold, thereby improving the user experience.
  • Figure 11 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure. As shown in Figure 11, the device includes: a skeleton chain determination module 410, a wrist determination module 420, a finger determination module 430 and a display module. 440.
  • the skeleton chain determination module 410 is configured to determine the skeleton chain to be registered and the preset skeleton chain of the target part in the video frame to be processed; wherein the skeleton chain to be registered includes a wrist bone point to be registered and a preset skeleton chain to be processed.
  • the preset bone chain includes preset wrist bone points and preset finger bone points;
  • the wrist determination module 420 is configured to first determine the wrist bone points according to the to-be-registered wrist bone points.
  • the pose information and the second pose information of the preset wrist skeletal points are used to determine the wrist status information of the wrist skeletal points to be registered; wherein the wrist status information includes wrist rotation information and wrist posture.
  • the finger determination module 430 is configured to determine the said wrist status information, the third pose information of the finger bone points to be registered, and the fourth pose information of the preset finger bone points. Finger status information of the finger bone points to be registered; wherein the finger status information includes finger rotation information and finger posture information; the display module 440 is configured to be based on the wrist status information and finger status information , display the target part in the video frame to be processed.
  • the wrist determination module 420 is further configured to be based on the first posture information of the wrist bone point to be registered and at least one finger bone point to be registered that is directly associated with the wrist bone point to be registered.
  • the third pose information determines the first rotation information; according to the second pose information of the preset wrist skeletal point and the at least one preset finger skeletal point directly associated with the preset wrist skeletal point.
  • the fourth pose information determines the second rotation information; based on the first rotation information and the second rotation information, determines the wrist state information of the wrist bone point to be registered.
  • the wrist determination module 420 is further configured to determine an alignment transformation matrix based on the second rotation information and the inverse of the first rotation information; based on the alignment transformation matrix and the wrist bone point to be registered
  • the first posture information determines the wrist status information.
  • the wrist determination module 420 is further configured to determine the position information in the wrist posture information based on the position information in the second posture information; based on the alignment transformation matrix and the first posture information.
  • Position information determines the posture information in the wrist posture information.
  • the device further includes: a dependency processing module configured to retrieve the preset first dependency information of the wrist bone points to be registered and the finger bone points to be registered, as well as the preset wrist bone points and the preset Second dependency information of finger bone points; wherein the dependence information includes finger bone points or wrist bone points on which at least one finger bone point on the same branch in the bone chain directly depends; according to the first dependency Information and the second dependency information, determine the processing priority of the finger bone points to be registered and the preset finger bone points, so as to determine the finger to be registered in sequence based on the processing priority. Finger status information of bone points.
  • a dependency processing module configured to retrieve the preset first dependency information of the wrist bone points to be registered and the finger bone points to be registered, as well as the preset wrist bone points and the preset Second dependency information of finger bone points; wherein the dependence information includes finger bone points or wrist bone points on which at least one finger bone point on the same branch in the bone chain directly depends; according to the first dependency Information and the second dependency information, determine the processing priority of the finger bone points to
  • the finger determination module 430 is further configured to determine at least one finger bone point group; wherein the finger bone point group includes preset finger bone points located in corresponding bone chain branches and with the same processing priority. and the finger bone points to be registered; for the at least one finger bone point group, determine the front-dependent bone point group and the back-dependent bone point group that the current finger bone point group depends on according to the processing priority; wherein , the front-dependent bone point group includes a wrist bone point group or a finger bone point group; if the front-dependent bone point group is a wrist bone point group, then according to the wrist rotation information in the wrist status information , the pose information in the current finger skeletal point group and the pose information in the post-dependent skeletal point group, determine the finger status information of the finger skeletal point to be registered in the current finger skeletal point group .
  • the finger determination module 430 is further configured to: if the front-dependent bone point group is a finger bone point group, the target finger in the finger state information of the bone points to be registered in the front-dependent bone point group is determined. Using the rotation information, the pose information in the current finger skeletal point group, and the pose information of the subsequent dependent skeletal point group, determine the finger to be registered with the finger skeletal point in the current finger skeletal point group. status information.
  • the finger determination module 430 is further configured to use the target finger rotation information, the third pose information and the fourth pose information in the current finger bone point group, and the posterior dependent bone point group. Based on the third pose information and the fourth pose information in the current finger skeletal point group, the current finger rotation information of the skeletal points to be registered is determined; based on the target finger rotation information and the current finger rotation information , determine the current finger posture information of the bone point to be registered in the current finger bone point group; determine the finger state of the bone point to be registered based on the current finger rotation information and the current finger posture information information.
  • the finger determination module 430 is further configured to determine a third rotation vector based on the third pose information in the current finger bone point group and the third pose information in the subsequent dependent bone point group; and , determine the fourth rotation vector according to the fourth pose information in the current finger skeletal point group and the fourth pose information in the later dependent skeletal point group; according to the third rotation vector and the fourth rotation Vector, determine the rotation matrix of the finger bone point to be registered in the current finger bone point group to the corresponding preset finger bone point; determine the current finger based on the rotation matrix and the target finger rotation information.
  • the current finger rotation information of the bone points to be registered in the head bone point group is further configured to determine a third rotation vector based on the third pose information in the current finger bone point group and the third pose information in the subsequent dependent bone point group; and , determine the fourth rotation vector according to the fourth pose information in the current finger skeletal point group and the fourth pose information in the later dependent skeletal point group; according to the third rotation vector and the fourth rotation Vector, determine the rotation matrix of the finger bone point to be registered
  • the display module 440 is also configured to adjust the display position and display posture of multiple skeletal points in the target part in the display interface based on the wrist state information and finger state information, and obtain the display position corresponding to the video frame to be processed. Target video frame.
  • the device further includes: an adjustment module configured to determine, based on the wrist status information, that the position offset between the wrist bone point to be registered and the preset wrist bone point is greater than the preset offset. threshold, then determine the length information of the arm bone chain and/or leg bone chain to which the wrist bone point to be registered belongs; based on the length information and the position offset, adjust the arm bone chain and/or The display position of at least one bone point in the leg bone chain.
  • an adjustment module configured to determine, based on the wrist status information, that the position offset between the wrist bone point to be registered and the preset wrist bone point is greater than the preset offset. threshold, then determine the length information of the arm bone chain and/or leg bone chain to which the wrist bone point to be registered belongs; based on the length information and the position offset, adjust the arm bone chain and/or The display position of at least one bone point in the leg bone chain.
  • the wrist bone points to be registered in the bone chain to be registered in the animation can be combined with the preset wrist bones in the preset bone chain.
  • Point registration is used to obtain the wrist status information of the wrist bone points to be registered.
  • the fingers can be registered based on multiple preset finger bone points in the preset bone chain.
  • the pose information of the skeletal points determines the finger status information of multiple finger skeletal points to be registered, and then based on the wrist status information and finger status information, the target part and the display information in the video frame to be processed are adjusted to avoid In situations where the animation data of the target part is easy to wear through the mold when migrating, and when the display effect of the target part is poor and the authenticity is poor, without wearing the mold, the authenticity of the display of the target part in the display interface can be improved, and then Improve user experience.
  • the image processing device provided by the embodiments of the present disclosure can execute the image processing method provided by any embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.
  • FIG. 12 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablets), PMPs (Portable Multimedia Players), vehicle-mounted terminals (such as Mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG. 12 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.
  • the electronic device 500 may include a processing device (eg, central processing unit, graphics processor, etc.) 501, which may be loaded into a random access device according to a program stored in a read-only memory (ROM) 502 or from a storage device 508.
  • the program in the memory (RAM) 503 executes various appropriate actions and processes.
  • various programs and data required for the operation of the electronic device 500 are also stored.
  • the processing device 501, ROM 502 and RAM 503 are connected to each other via a bus 504.
  • An editing/output (I/O) interface 505 is also connected to bus 504.
  • input devices 506 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration
  • An output device 507 such as a computer
  • a storage device 508 including a magnetic tape, a hard disk, etc.
  • Communication device 509 may allow electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data.
  • FIG. 12 illustrates electronic device 500 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product including a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart.
  • the computer program may be downloaded and installed from the network via communication device 509, or from storage device 508, or from ROM 502.
  • the processing device 501 When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.
  • the electronic device provided by the embodiments of the present disclosure and the image processing method provided by the above embodiments belong to the same concept.
  • Technical details that are not described in detail in this embodiment can be referred to the above embodiments, and this embodiment has the same benefits as the above embodiments. Effect.
  • Embodiments of the present disclosure provide a computer storage medium on which a computer program is stored.
  • the program is executed by a processor, the image processing method provided by the above embodiments is implemented.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmd read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .
  • Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.
  • the client and server can communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can communicate with digital data in any form or medium.
  • Communications e.g., communications network
  • communications networks include local area networks (“LAN”), wide area networks (“WAN”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or developed in the future network of.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs.
  • the electronic device executes the above-mentioned one or more programs.
  • the computer-readable medium carries one or more programs.
  • the electronic device determines the skeleton chain to be registered and the preset skeleton of the target part in the video frame to be processed.
  • Chain wherein, the skeleton chain to be registered includes wrist bone points to be registered and finger bone points to be registered, and the preset skeleton chain includes preset wrist bone points and preset finger bone points.
  • the wrist status information includes wrist rotation information and wrist posture information; based on the wrist status information, the third posture information of the finger bone points to be registered and the fourth position of the preset finger bone points posture information to determine the finger state information of the finger bone points to be registered; wherein the finger state information includes finger rotation information and finger posture information; based on the wrist state information and finger state information , display the target part in the video frame to be processed.
  • Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages—such as "C” or similar programming languages.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider through Internet connection).
  • LAN local area network
  • WAN wide area network
  • Internet service provider such as an Internet service provider through Internet connection
  • each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved.
  • each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.
  • the units involved in the embodiments of the present disclosure can be implemented in software or hardware. Among them, the name of the unit does not constitute a limitation on the unit itself under certain circumstances.
  • the bone chain determination module can also be described as "a module for obtaining the bone chain to be registered and the preset bone chain.”
  • FPGAs Field Programmable Gate Arrays
  • ASICs Application Specific Integrated Circuits
  • ASSPs Application Specific Standard Products
  • SOCs Systems on Chips
  • CPLD Complex Programmable Logical device
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
  • the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • CD-ROM portable compact disk read-only memory
  • magnetic storage device or any suitable combination of the above.
  • Example 1 provides an image processing method, which includes:
  • the skeleton chain to be registered includes wrist bone points to be registered and finger bone points to be registered, and the preset skeleton chain is Assume that the bone chain includes preset wrist bone points and preset finger bone points;
  • Wrist status information includes wrist rotation information and wrist posture information
  • the finger status information includes finger rotation information and finger posture information
  • the target part is displayed in the video frame to be processed.
  • Example 2 provides an image processing method, further including:
  • determining the wrist status information of the wrist bone point to be registered based on the first posture information of the wrist bone point to be registered and the second posture information of the preset wrist bone point include:
  • the first rotation is determined based on the first posture information of the wrist bone point to be registered and the third posture information of at least one finger bone point to be registered that is directly related to the wrist bone point to be registered. information;
  • the wrist state information of the wrist bone point to be registered is determined.
  • Example 3 provides an image processing method, further including:
  • the first bone point based on the wrist bone point to be registered is The pose information and the third pose information of at least one finger bone point to be registered that are directly related to the wrist bone point to be registered, determine the first rotation information, including:
  • first rotation information is determined.
  • Example 4 provides an image processing method, further including:
  • the second pose information based on the preset wrist bone point and the fourth pose information of at least one preset finger skeletal point directly related to the preset wrist skeletal point determines the second rotation information, including:
  • Second rotation information is determined.
  • Example 5 provides an image processing method, further including:
  • determining the wrist state information of the wrist bone point to be registered based on the first rotation information and the second rotation information includes:
  • the wrist status information is determined based on the alignment transformation matrix and the first posture information of the wrist bone point to be registered.
  • Example 6 provides an image processing method, further including:
  • determining the wrist posture information in the wrist status information includes:
  • the posture information in the wrist posture information is determined.
  • Example 7 provides an image processing method, further including:
  • the finger to be registered is Before the finger status information of the bone point, it also includes:
  • the Dependency information includes finger bone points or wrist bone points on which at least one finger bone point located on the same branch in the bone chain directly depends;
  • the processing priority of the finger bone point to be registered and the preset finger bone point determine the processing priorities in sequence based on the processing priority. Describes the finger status information of the finger bone points to be registered.
  • Example 8 provides an image processing method, further comprising:
  • the finger bones to be registered are Point finger status information, including:
  • the finger bone point group includes preset finger bone points and to-be-registered finger bone points that are located in corresponding bone chain branches and have the same processing priority;
  • For the at least one finger bone point group determine the front-dependent bone point group and the back-dependent bone point group that the current finger bone point group depends on according to the processing priority; wherein the front-dependent bone point group includes Wrist bone point group or finger bone point group;
  • the pose information in the current finger bone point group determines the finger status information of the finger bone point to be registered in the current finger bone point group.
  • Example 9 provides an image processing method, further comprising:
  • it also includes:
  • the front-dependent bone point group is a finger bone point group
  • the current finger bone point The pose information in the group and the pose information of the subsequent dependent skeletal point group are used to determine the finger status information of the finger skeletal point to be registered in the current finger skeletal point group.
  • Example 10 provides an image processing method, further comprising:
  • the target finger rotation information in the finger state information of the skeleton points to be registered in the front dependent skeleton point group, the pose information in the current finger skeleton point group and the posterior dependent skeleton determines the finger status information of the finger bone point to be registered in the current finger bone point group, including:
  • the target finger rotation information the third pose information and the fourth pose information in the current finger bone point group, and the third pose information and the fourth position information in the subsequent dependent bone point group pose information to determine the current finger rotation information of the bone points to be registered in the current finger bone point group;
  • the finger state information of the skeletal point to be registered is determined.
  • Example 11 provides an image processing method, further comprising:
  • the method is based on the target finger rotation information, the third pose information and the fourth pose information in the current finger bone point group, and the third pose information in the subsequent dependent bone point group.
  • the fourth pose information which determines the current finger rotation information of the bone points to be registered in the current finger bone point group, including:
  • the current finger rotation information of the bone points to be registered in the current finger bone point group is determined.
  • Example 12 provides an image processing method, further comprising:
  • displaying the target part in the video frame to be processed based on the wrist status information and finger status information includes:
  • the display positions and display postures of multiple skeletal points in the target part in the display interface are adjusted to obtain a target video frame corresponding to the video frame to be processed.
  • Example 13 provides an image processing method, further comprising:
  • it also includes:
  • the wrist bone point to be registered is The length information of the arm bone chain and/or leg bone chain;
  • the display position of at least one bone point in the arm bone chain and/or the leg bone chain is adjusted.
  • Example 14 provides an image processing device, which includes:
  • the skeleton chain determination module is configured to determine the skeleton chain to be registered and the preset skeleton chain of the target part in the video frame to be processed; wherein the skeleton chain to be registered includes a wrist bone point to be registered and a finger to be registered.
  • the preset skeletal chain includes preset wrist skeletal points and preset finger skeletal points;
  • the wrist determination module is configured to determine the wrist of the wrist bone point to be registered based on the first posture information of the wrist bone point to be registered and the second posture information of the preset wrist bone point. Status information; wherein the wrist status information includes wrist rotation information and wrist posture information;
  • the finger determination module is configured to determine the finger skeletal point to be registered based on the wrist status information, the third pose information of the finger skeletal point to be registered, and the fourth pose information of the preset finger skeletal point. Finger status information of finger bone points; wherein the finger status information includes finger rotation information and finger posture information;
  • a display module is configured to display the target part in the video frame to be processed based on the wrist status information and finger status information.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种图像处理方法、装置、电子设备及存储介质，该方法包括：确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链（S110）；根据所述待配准腕部骨骼点的第一位姿信息和预设腕部骨骼点的第二位姿信息，确定所述待配准腕部骨骼点的腕部状态信息（S120）；依据所述腕部状态信息、待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息（S130）；基于所述腕部状态信息和指部状态信息，在所述待处理视频帧中显示所述目标部位（S140）。所述方法实现了在不穿模的情况下，提高目标部位于显示界面中显示的真实性，进而提高用户体验度的效果。

Description

图像处理方法、装置、电子设备及存储介质
本申请要求在2022年7月6日提交中国专利局、申请号为202210797502.6的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开实施例涉及图像处理技术，例如涉及一种图像处理方法、装置、电子设备及存储介质。
背景技术
无论是动画重定向，还是人工智能肢体识别，可能会存在将一组动画数据从一套骨骼模型迁移到另外一套骨骼模型上进行显示。
在两套骨骼模型差异较大的情形下，例如，骨骼模型尺寸差异较大、姿态差异较大，会存在迁移后的显示画面出现明显穿模现象。尤其是对于骨骼点较多的骨骼模型来说，此类问题尤为明显，导致动画显示效果不佳以及真实性较差的问题，甚至会影响用户体验度。
发明内容
本公开提供一种图像处理方法、装置、电子设备及存储介质，以在不穿模的情况下，提高目标部位于显示界面中显示的真实性，进而提高用户体验度。
第一方面，本公开实施例提供了一种图像处理方法，该方法包括：
确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链；其中，所述待配准骨骼链上包括待配准腕部骨骼点和待配准指部骨骼点，所述预设骨骼链上包括预设腕部骨骼点和预设指部骨骼点；
根据所述待配准腕部骨骼点的第一位姿信息和预设腕部骨骼点的第二位姿信息，确定所述待配准腕部骨骼点的腕部状态信息；其中，所述腕部状态信息包括腕部旋转信息和腕部位姿信息；
依据所述腕部状态信息、待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息；其中，所述指部状态信息包括指部旋转信息和指部位姿信息；
基于所述腕部状态信息和指部状态信息，在所述待处理视频帧中显示所述目标部位。
第二方面，本公开实施例还提供了一种图像处理装置，该装置包括：
骨骼链确定模块，设置为确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链；其中，所述待配准骨骼链上包括待配准腕部骨骼点和待配准指部骨骼点，所述预设骨骼链上包括预设腕部骨骼点和预设指部骨骼点；
腕部确定模块，设置为根据所述待配准腕部骨骼点的第一位姿信息和预设腕部骨骼点的第二位姿信息，确定所述待配准腕部骨骼点的腕部状态信息；其中，所述腕部状态信息包括腕部旋转信息和腕部位姿信息；
指部确定模块，设置为依据所述腕部状态信息、待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息；其中，所述指部状态信息包括指部旋转信息和指部位姿信息；
显示模块，设置为基于所述腕部状态信息和指部状态信息，在所述待处理视频帧中显示所述目标部位。
第三方面，本公开实施例还提供了一种电子设备，所述电子设备包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序，
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如本公开实施例中任一所述的图像处理方法。
第四方面，本公开实施例还提供了一种包含计算机可执行指令的存储介质，其特征在于，所述计算机可执行指令在由计算机处理器执行时用于执行如本公开实施例中任一所述的图像处理方法。
附图说明
贯穿附图中，相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的，原件和元素 不一定按照比例绘制。
图1为本公开实施例所提供的一种图像处理方法的流程示意图；
图2为本公开实施例所提供的一种待配准骨骼链的示意图；
图3为本公开实施例所提供的一种预设骨骼链的示意图；
图4为本公开实施例所提供的一种胳膊骨骼链调整示意图；
图5为本公开实施例所提供的另一种图像处理方法的流程示意图；
图6为本公开实施例所提供的一种待配准腕部骨骼点和与待配准腕部骨骼点直接关联的至少一个待配准指部骨骼点的示意图；
图7为本公开实施例所提供的一种预设腕部骨骼点和与预设腕部骨骼点直接关联的至少一个预设指部骨骼点的示意图；
图8为本公开实施例所提供的另一种图像处理方法的流程示意图；
图9为本公开实施例所提供的一种待配准指部骨骼点旋转前的示意图；
图10为本公开实施例所提供的一种待配准指部骨骼点旋转后的示意图；
图11为本公开实施例所提供的一种图像处理装置的结构示意图；
图12为本公开实施例所提供的一种电子设备的结构示意图。
具体实施方式
应当理解，本公开的方法实施方式中记载的多个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
需要注意，本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
需要注意，本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
可以理解的是，在使用本公开多个实施例公开的技术方案之前，均应当依据相关法律法规通过恰当的方式对本公开所涉及个人信息的类型、使用范围、使用场景等告知用户并获得用户的授权。
例如，在响应于接收到用户的主动请求时，向用户发送提示信息，以明确地提示用户，其请求执行的操作将需要获取和使用到用户的个人信息。从而，使得用户可以根据提示信息来自主地选择是否向执行本公开技术方案的操作的电子设备、应用程序、服务器或存储介质等软件或硬件提供个人信息。
作为一种示例的但非限定性的实现方式，响应于接收到用户的主动请求，向用户发送提示信息的方式例如可以是弹窗的方式，弹窗中可以以文字的方式呈现提示信息。此外，弹窗中还可以承载供用户选择“同意”或者“不同意”向电子设备提供个人信息的选择控件。
可以理解的是，上述通知和获取用户授权过程仅是示意性的，不对本公开的实现方式构成限定，其它满足相关法律法规的方式也可应用于本公开的实现方式中。
可以理解的是，本技术方案所涉及的数据(包括但不限于数据本身、数据的获取或使用)应当遵循相应法律法规及相关规定的要求。
在介绍本技术方案之前，可以先对应用场景进行示例性说明。可以将本公开技术方案应用在显示界面中存在肢体动画的场景，例如：动画重定向、人工智能肢体识别等。肢体动画可以通过显示界面上显示的按键来触发，也可以通过目标对象的目标部位预设的连续运动过程来触发。例如：在游戏中，用户通过按键或触摸等方式控制显示界面中的目标角色释放技能，可以令目标角色的目标部位产生运动以展示技能特效，此时，在显示界面中显示肢体动画时，可以采用本公开实施例所提供的方案，以实现骨骼部位重定位的效果。也就是说，只要在显示界面中存在肢体动画的动画数据迁移，如图像显示、视频显示、直播显示等场景中存在手部或脚部的动画数据迁移，均可以采用本公开实施例所提供的技术方案来进行渲染和显示， 以提高显示效果。若骨骼模型中存在骨骼点相互依赖的目标部位，例如：目标部位可以是手部或脚部等，则可以使用本公开实施例的技术方案来进行动画数据迁移。执行本公开实施例提供的图像处理方法的装置，可以集成在具有图像处理功能的应用软件中，且该软件可以安装至电子设备中，例如，电子设备可以是移动终端或者PC端等。应用软件可以是对图像/视频处理的一类软件，其具体的应用软件在此不再一一赘述，只要可以实现图像/视频处理即可。
图1为本公开实施例所提供的一种图像处理方法的流程示意图，本公开实施例适用于对待处理视频帧中的目标部位进行处理和显示的情形，该方法可以由图像处理装置来执行，该装置可以通过软件和/或硬件的形式实现，例如，通过电子设备来实现，该电子设备可以是移动终端、PC端或服务器等。
如图1所示，所述方法包括：
S110、确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链。
其中，待处理视频帧可以是包含目标部位的视频中的视频帧。目标部位可以是包含腕部和指部的部位，例如可以是手部或脚部。待配准骨骼链可以是目标部位的初始的待进行配准的骨骼模型，如图2所示。待配准骨骼链上包括待配准腕部骨骼点和待配准指部骨骼点，待配准腕部骨骼点可以是待配准骨骼链中与腕部对应的骨骼点，如图2中的Root’，待配准指部骨骼点可以是待配准骨骼链中与指部对应的骨骼点，如图2中除Root’之外的骨骼点。预设骨骼链可以是目标部位待配准成为的骨骼模型，如图3所示。预设骨骼链上包括预设腕部骨骼点和预设指部骨骼点，预设腕部骨骼点可以是预设骨骼链中与腕部对应的骨骼点，如图3中的Root，预设指部骨骼点可以是预设骨骼链中与指部对应的骨骼点，如图3中除Root之外的骨骼点。
例如，可以获取待处理视频帧，可以确定待处理视频帧中的目标部位。据此，根据待处理视频帧中的目标部位可以提取待配准骨骼链，并且根据目标部位对应的动作可以确定目标部位对应的预设骨骼链。
需要说明的是，预设骨骼链可以是预先设置的与不同目标部位的动作对应的骨骼链，通过本公开实施例能够将待配准骨骼链配准至预设骨骼链对应的位姿，即按照预设骨骼链进行渲染和显示以完成目标部位的动作。
S120、根据待配准腕部骨骼点的第一位姿信息和预设腕部骨骼点的第二位姿信息，确定待配准腕部骨骼点的腕部状态信息。
其中，第一位姿信息可以是待配准腕部骨骼点的位置信息和姿态信息。第二位姿信息可以是预设腕部骨骼点的位置信息和姿态信息。腕部状态信息可以是用于表示腕部骨骼点配准结果的信息，腕部状态信息包括腕部旋转信息和腕部位姿信息，腕部旋转信息可以用于表示坐标旋转变换，可以用腕部旋转矩阵表示，腕部位姿信息可以包括位置信息和姿态信息。
例如，由于在全局坐标系下，待配准腕部骨骼点的第一位姿信息与预设腕部骨骼点的第二位姿信息不同，因此，需要将二者进行对齐。可以根据待配准腕部骨骼点的第一位姿信息和预设腕部骨骼点的第二位姿信息进行对齐处理，确定待配准腕部骨骼点配准所需的腕部状态信息，即转换待配准腕部的位姿所需的信息。
示例性的，基于待配准腕部骨骼点可以建立一个待配准局部坐标系，基于预设腕部骨骼点可以建立一个预设局部坐标系，确定坐标系转换所对应的腕部旋转信息，并可以确定待配准腕部骨骼点的转换至预设腕部骨骼点所需的腕部位姿信息，进而，将腕部旋转信息以及腕部位姿信息作为腕部状态信息。
S130、依据腕部状态信息、待配准指部骨骼点的第三位姿信息以及预设指部骨骼点的第四位姿信息，确定待配准指部骨骼点的指部状态信息。
其中，第三位姿信息可以是待配准指部骨骼点的位置信息和姿态信息。第四位姿信息可以是预设指部骨骼点的位置信息和姿态信息。指部状态信息可以是用于表示指部骨骼点配准结果的信息，指部状态信息包括指部旋转信息和指部位姿信息，指部旋转信息可以用指部旋转矩阵表示，指部位姿信息可以包括位置信息和姿态信息。
例如，由于指部骨骼点与腕部骨骼点和/或指部骨骼点相依赖，并且，每一指部骨骼点都会与腕部骨骼点相依赖，可以是直接依赖或间接依赖。因此，可以根据腕部状态信息、待配准指部骨骼点的第三位姿信息以及预设指部骨骼点的第四位姿信息进行对齐处理，确定待配准指部骨骼点的指部状态信息，即转换待配准指部的位姿所需的信息。
S140、基于腕部状态信息和指部状态信息，在待处理视频帧中显示目标部位。
例如，根据腕部状态信息和指部状态信息可以对待配准骨骼链进行处理，以将待配准骨骼链处理为目标部位，并渲染在待处理视频帧中，以将目标部位按照预设骨骼链的位姿信息，准确显示在待处理视频帧中。
在上述示例的基础上，为了提高目标部位显示效果的真实性和准确性，可以基于腕部状态信息和指部状态信息，调整目标部位中多个骨骼点在于显示界面中的显示位置和显示姿态，得到与待处理视频帧相对应的目标视频帧。
其中，显示位置可以用于描述位置的信息，例如可以是骨骼点在显示界面中的位置坐标信息等。显示姿态可以是用于描述姿态的信息，例如可以是骨骼点在显示页面中的与坐标轴的夹角信息等。目标视频帧可以是待处理视频帧中目标部位处理后得到的视频帧。
例如，在获取腕部状态信息和指部状态信息之后，可以根据腕部状态信息对待处理视频帧中目标部位的腕部骨骼点进行调整，并根据指部状态信息对待处理视频帧中目标部位的指部骨骼点进行调整，以确定目标部位中多个骨骼点的显示位置和显示姿态，并基于显示位置和显示姿态将目标部位的各骨骼点渲染至显示界面中，将该视频帧作为目标视频帧。
需要说明的是，在调整目标部位中多个骨骼点在于显示界面中的显示位置和显示姿态的过程中，可以目标部位的骨骼链添加蒙皮效果，以使目标部位的显示更接近皮肤，提高显示效果。
在上述示例的基础上，为了提高目标部位的显示效果，使视觉效果提升，避免显示失真，可以通过下述方式来显示：
若基于腕部状态信息，确定待配准腕部骨骼点与预设腕部骨骼点的位置偏移量大于预设偏移量阈值，则确定待配准腕部骨骼点所属胳膊骨骼链和/或腿部骨骼链的长度信息；基于长度信息和位置偏移量，调整胳膊骨骼链和/或腿部骨骼链中至少一个骨骼点的显示位置。
其中，位置偏移量可以是待配准腕部骨骼点的位置信息与预设腕部骨骼点的位置信息之间距离。预设偏移量阈值可以是待配准腕部骨骼点能够偏移的最大距离。胳膊骨骼链可以是目标部位为胳膊时所对应的骨骼链。腿部骨骼链可以是目标部位为腿部时所对应的骨骼链。长度信息可以是胳膊骨骼链和/或腿部骨骼链的长度值。
例如，若基于腕部状态信息，确定待配准腕部骨骼点与预设腕部骨骼点的位置偏移量大于预设偏移量阈值，则表明直接调整待配准腕部骨骼点不能够与预设腕部骨骼点相配准。在这种情况下，可以通过调整待配准腕部骨骼点所属胳膊骨骼链和/或腿部骨骼链中的至少一个骨骼点来实现。可以确定待配准腕部骨骼点所属胳膊骨骼链和/或腿部骨骼链的长度信息，并根据长度信息和位置偏移量调整胳膊骨骼链和/或腿部骨骼链中的至少一个骨骼点的显示位置。例如，可以等比例拉伸确定多个骨骼点的显示位置，以使待配准腕部骨骼点能够按照腕部状态信息进行配准。
示例性的，胳膊骨骼链调整示意图如图4所示，W为待配准腕部骨骼点，T为预设腕部骨骼点，由图4中的上图可知，待配准腕部骨骼点与预设腕部骨骼点的位置偏移量大于预设偏移量阈值，即待配准腕部骨骼点无法达到预设腕部骨骼点。因此，将胳膊骨骼链U-L-W进行拉长，可以根据胳膊上臂U-L的长度信息，胳膊下臂L-W的长度信息，进行等比拉长，以修正L和W的显示位置，如图4下图所示，使得待配准腕部骨骼点W能够与预设腕部骨骼点T进行配准。
本公开实施例的技术方案，在确定需要骨骼重定位时，可以将接入动画中的待配准骨骼链中的待配准腕部骨骼点，与预设骨骼链中的预设腕部骨骼点配准，得到待配准腕部骨骼点的腕部状态信息，在确定腕部状态信息的基础上，可以依次根据预设骨骼链中多个预设指部骨骼点，以及配准指部骨骼点的位姿信息，确定多个待配准指部骨骼点的指部状态信息，进而基于腕部状态信息和指部状态信息，调整目标部位与待处理视频帧中的显示信息，避免了目标部位的动画数据迁移时容易穿模的情况，以及目标部位的显示效果较差、真实性较差的情况，在不穿模的情况下，提高目标部位于显示界面中显示的真实性，进而提高用户体验度。
图5为本公开实施例所提供的另一种图像处理方法的流程示意图，在前述实施例的基础上，可以先确定待配准腕部骨骼点的腕部状态信息，其具体实施方式可以参见本技术方案的详细阐述。其中，与上述多个实施例相同或相应的术语的解释在此不再赘述。
如图5所示，所述方法包括：
S210、确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链。
S220、根据待配准腕部骨骼点的第一位姿信息和与待配准腕部骨骼点直接关联的至少一个待配准指部骨骼点的第三位姿信息，确定第一旋转信息。
其中，第三位姿信息可以是与待配准腕部骨骼点直接关联的至少一个待配准指部骨骼点的位置信息和姿态信息。第一旋转信息可以是待配准腕部骨骼点对应的旋转信息，可以是旋转矩阵或者是四元数等形式。
例如，可以确定与待配准腕部骨骼点直接关联的至少一个待配准指部骨骼点，可以理解为以待配准腕部骨骼点为起点向多个指尖方向延伸，多个延伸方向上的第一个待配准指部骨骼点为直接关联的待配准指部骨骼点，例如图2中Root’为待配准腕部骨骼点，U’、P’、K’、F’以及A’均为与待配准腕部骨骼点直接关联的待配准指部骨骼点。根据待配准腕部骨骼点的第一位姿信息和与待配准腕部骨骼点直接关联的至少一个待配准指部骨骼点的第三位姿信息可以确定待配准腕部骨骼点对应的旋转矩阵，并根据旋转矩阵确定第一旋转信息。
在上述示例的基础上，若至少一个待配准指部骨骼点包括第一待配准指部骨骼点和第二待配准指部骨骼点，可以通过下述方式来更快速且准确的确定第一旋转信息：
基于第一位姿信息和第一待配准指部骨骼点的位姿信息，确定第一向量；基于第一位姿信息和第二待配准指部骨骼点的位姿信息，确定第二向量；基于第一向量和第二向量，确定第一旋转信息。
其中，第一向量可以是待配准腕部骨骼点指向第一待配准指部骨骼点的向量。第二向量可以是待配准腕部骨骼点指向第二待配准指部骨骼点的向量。需要说明的是，第一待配准指部骨骼点和第二待配准指部骨骼点为与待配准腕部骨骼点直接关联的待配准指部骨骼点中的任意两个，在本实施例中不做具体限定。
例如，根据第一位姿信息中的位置信息和第一待配准指部骨骼点的位姿信息中的位置信息，可以确定待配准腕部骨骼点指向第一待配准指部骨骼点的向量为第一向量。根据第一位姿信息中的位置信息和第二待配准指部骨骼点的位姿信息中的位置信息，可以确定待配准腕部骨骼点指向第二待配准指部骨骼点的向量为第二向量。根据第一向量和第二向量进行叉乘，可以得到与第一向量和第二向量都垂直的第一垂直向量，基于第一向量和第二向量中的任意一个和第一垂直向量进行叉乘，得到第二垂直向量。进而，根据第一垂直向量、第二垂直向量以及与第二垂直向量相对应的向量(第一向量或第二向量)，构建旋转矩阵，可以将旋转矩阵作为第一旋转信息，还可以将旋转矩阵变换为四元数，将四元数作为第一旋转信息。
示例性的，如图6所示的待配准腕部骨骼点和与待配准腕部骨骼点直接关联的至少一个待配准指部骨骼点的示意图。其中，待配准腕部骨骼点为O’，与待配准腕部骨骼点直接关联的待配准指部骨骼点为A’、B’、C’、D’以及E’。以待配准指部骨骼点C’为第一待配准指部骨骼点，以待配准指部骨骼点B’为第二待配准指部骨骼点，可以确定第一向量为O’C’，第二向量为O’B’。第一垂直向量up’＝O’C’×O’B’，选择第一向量O’C’来确定第二垂直向量，记forward’＝O’C’，第二垂直向量left’＝up’×forward’。进而，根据left’、up’以及forward’可以确定旋转矩阵，并转换旋转矩阵得到四元数q’，即为第一旋转信息。
S230、根据预设腕部骨骼点的第二位姿信息和与预设腕部骨骼点直接关联的至少一个预设指部骨骼点的第四位姿信息，确定第二旋转信息。
其中，第四位姿信息可以是与预设腕部骨骼点直接关联的至少一个预设指部骨骼点的位置信息和姿态信息。第二旋转信息可以是预设腕部骨骼点对应的旋转信息，可以是旋转矩阵或者是四元数等形式。
例如，可以确定与预设腕部骨骼点直接关联的至少一个预设指部骨骼点，可以理解为以预设腕部骨骼点为起点向多个指尖方向延伸，多个延伸方向上的第一个预设指部骨骼点为直接关联的预设指部骨骼点。例如图3中Root为预设腕部骨骼点，V、Q、L、G以及A均为与预设腕部骨骼点直接关联的预设指部骨骼点。根据预设腕部骨骼点的第二位姿信息和与预设腕部骨骼点直接关联的至少一个预设指部骨骼点的第四位姿信息可以确定预设腕部骨骼点对应的旋转矩阵，并根据旋转矩阵确定第二旋转信息。
在上述示例的基础上，若至少一个预设指部骨骼点包括第一预设指部骨骼点和第二预设指部骨骼点，可以通过下述方式来更快速且准确的确定第二旋转信息：
基于第二位姿信息和第一预设指部骨骼点的位姿信息，确定第三向量；基于第二位姿信息和第二预设指部骨骼点的位姿信息，确定第四向量；基于第三向量和第四向量，确定第二旋转信息。
其中，第三向量可以是预设腕部骨骼点指向第一预设指部骨骼点的向量。第四向量可以是预设腕部骨骼点指向第二预设指部骨骼点的向量。需要说明的是，第一预设指部骨骼点和第二预设指部骨骼点为与预设腕部骨骼点直接关联的预设指部骨骼点中的任意两个，在本实施例中不做具体限定。
例如，根据第二位姿信息中的位置信息和第一与指部骨骼点的位姿信息中的位置信息，可以确定预设 腕部骨骼点指向第一预设指部骨骼点的向量为第三向量。根据第二位姿信息中的位置信息和第二预设指部骨骼点的位姿信息中的位置信息，可以确定预设腕部骨骼点指向第二预设指部骨骼点的向量为第四向量。根据第三向量和第四向量进行叉乘，可以得到与第三向量和第四向量都垂直的第三垂直向量，基于第三向量和第四向量中的任意一个和第三垂直向量进行叉乘，得到第四垂直向量。进而，根据第三垂直向量、第四垂直向量以及与第四垂直向量相对应的向量(第三向量或第四向量)，构建旋转矩阵，可以将旋转矩阵作为第二旋转信息，还可以将旋转矩阵变换为四元数，将四元数作为第二旋转信息。
示例性的，如图7所示的预设腕部骨骼点和与预设腕部骨骼点直接关联的至少一个预设指部骨骼点的示意图。其中，预设腕部骨骼点为O，与预设腕部骨骼点直接关联的预设指部骨骼点为A、B、C、D以及E。以预设指部骨骼点C为第一预设指部骨骼点，以预设指部骨骼点B为第二预设指部骨骼点，可以确定第三向量为OC，第四向量为OB。第三垂直向量up＝OC×OB，选择第三向量OC来确定第四垂直向量，记forward＝OC，第四垂直向量left＝up×forward。进而，根据left、up以及forward可以确定旋转矩阵，并转换旋转矩阵得到四元数q，即为第二旋转信息。
S240、基于第一旋转信息和第二旋转信息，确定待配准腕部骨骼点的腕部状态信息。
例如，根据第一旋转信息和第二旋转信息可以将待配准腕部骨骼点按照预设腕部骨骼点进行配准，将配准后的信息作为待配准腕部骨骼点的腕部状态信息。
在上述示例的基础上，为了更加准确且快速的确定腕部状态信息，可以引入对齐变换矩阵来进行处理，可以是：
根据第二旋转信息和第一旋转信息的逆，确定对齐变换矩阵；
基于对齐变换矩阵和待配准腕部骨骼点的第一位姿信息，确定腕部状态信息。
其中，对齐变换矩阵可以是将待配准腕部骨骼点与预设腕部骨骼点进行对齐的旋转变换矩阵。
例如，将第二旋转信息与第一旋转信息的逆相乘得到对齐变换矩阵。根据对齐变换矩阵对第一位姿信息进行处理，可以得到腕部位姿信息中的位置信息和姿态信息。
示例性的，若第一旋转信息为q’，第二旋转信息为q，则对齐变换矩阵r为r＝q*Inverse(q’)，其中，Inverse(·)表示逆运算。
在上述示例的基础上，为了更准确的将待配准腕部骨骼点与预设腕部骨骼点进行位置对齐，可以通过下述方式来基于对齐变换矩阵和待配准腕部骨骼点的第一位姿信息，确定腕部状态信息中的腕部位姿信息：
基于第二位姿信息中的位置信息，确定腕部位姿信息中的位置信息；基于对齐变换矩阵和第一位姿信息中的位置信息，确定腕部位姿信息中的姿态信息。
例如，将第二位姿信息中的位置信息确定为腕部位姿信息中的位置信息。将第一位姿信息中的位置信息与对齐变换矩阵相乘，得到腕部位姿信息中的姿态信息。
示例性的，若第二位姿信息中的位置信息为Op，则腕部位姿信息中的位置信息O’p＝Op。若第一位姿信息中的位置信息为Or，对齐变换矩阵为r，则腕部状态信息中的腕部位姿信息O’r＝Or*r。
通过S220-S240，可以准确且快速的确定第一旋转信息和第二旋转信息，以将待配准腕部骨骼点和预设腕部骨骼点进行统一，提高了腕部骨骼点的配准效率。
S250、依据腕部状态信息、待配准指部骨骼点的第三位姿信息以及预设指部骨骼点的第四位姿信息，确定待配准指部骨骼点的指部状态信息。
S260、基于腕部状态信息和指部状态信息，在待处理视频帧中显示目标部位。
本公开实施例的技术方案，通过确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链，根据待配准腕部骨骼点的第一位姿信息和与待配准腕部骨骼点直接关联的至少一个待配准指部骨骼点的第三位姿信息，确定第一旋转信息，根据预设腕部骨骼点的第二位姿信息和与预设腕部骨骼点直接关联的至少一个预设指部骨骼点的第四位姿信息，确定第二旋转信息，基于第一旋转信息和第二旋转信息，确定待配准腕部骨骼点的腕部状态信息，以将待配准腕部骨骼点按照预设腕部骨骼点进行配准，依据腕部状态信息、待配准指部骨骼点的第三位姿信息以及预设指部骨骼点的第四位姿信息，确定待配准指部骨骼点的指部状态信息，基于腕部状态信息和指部状态信息，在待处理视频帧中显示目标部位，避免了目标部位中腕部动画数据迁移时由于穿模现象导致的显示效果较差以及真实性较差的情况，在不穿模的情况下提高目标部位中腕部的显示效果以及真实性，进而提高用户体验度。
图8为本公开实施例所提供的另一种图像处理方法的流程示意图。在确定待配准腕部骨骼点的腕部状 态信息的基础上，可以依次确定多个待配准指部骨骼点的指部状态信息，其具体实施方式可以参见本技术方案的详细阐述。其中，与上述多个实施例相同或相应的术语的解释在此不再赘述。
如图8所示，所述方法包括：
S301、确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链。
S302、根据待配准腕部骨骼点的第一位姿信息和预设腕部骨骼点的第二位姿信息，确定待配准腕部骨骼点的腕部状态信息。
S303、调取预先设置的待配准腕部骨骼点和待配准指部骨骼点的第一依赖信息，以及预设腕部骨骼点和预设指部骨骼点的第二依赖信息。
其中，依赖信息可以是用于描述每个指部骨骼点直接依赖的指部骨骼点或腕部骨骼点的信息。依赖信息包括骨骼链中位于同一分支上至少一个指部骨骼点所直接依赖的指部骨骼点或腕部骨骼点。第一依赖信息与待配准骨骼链相对应，第二依赖信息与预设骨骼链相对应。
例如，可以按照待配准骨骼链中的待配准腕部骨骼点以及多个待配准指部骨骼点的连接信息，可以设置待配准腕部骨骼点和待配准指部骨骼点的第一依赖信息。还可以按照预设骨骼链中的预设腕部骨骼点以及多个预设指部骨骼点的连接信息，可以设置预设腕部骨骼点和预设指部骨骼点的第二依赖信息。为了后续骨骼点配准，可以调取第一依赖信息和第二依赖信息。
需要说明的是，指部骨骼点对应的依赖信息为由该指部骨骼点向腕部延伸，将延伸方向上与该指部骨骼点相邻的指部骨骼点或腕部骨骼点作为依赖信息。
示例性的，如图2所示的待配准骨骼链，以分支Root’-U’-V’-W’-X’-Y’为例，待配准指部骨骼点U’直接依赖的是待配准腕部骨骼点Root’，即U’的第一依赖信息为Root’；待配准指部骨骼点V’直接依赖的是待配准指部骨骼点U’，即V’的第一依赖信息为U’；按照同样方式可以确定其他待配准指部骨骼点对应的第一依赖信息，以得到待配准腕部骨骼点和待配准指部骨骼点的第一依赖信息。如图3所示的预设骨骼链，以分支Root-V-W-X-Y为例，预设指部骨骼点V直接依赖的是预设腕部骨骼点Root，即V的第二依赖信息为Root；预设指部骨骼点W直接依赖的是预设指部骨骼点V，即W的第二依赖信息为V；按照同样方式可以确定其他预设指部骨骼点对应的第二依赖信息，以得到预设腕部骨骼点和预设指部骨骼点的第二依赖信息。
S304、依据第一依赖信息和第二依赖信息，确定对待配准指部骨骼点和预设指部骨骼点的处理优先级，以基于处理优先级依次确定待配准指部骨骼点的指部状态信息。
其中，处理优先级可以是按照依赖信息确定的优先级，以任一骨骼点为例，该骨骼点的优先级低于该骨骼点对应的依赖信息的优先级。
例如，可以按照第一依赖信息确定多个待配准指部骨骼点的处理优先级，按照第二依赖信息确定多个预设指部骨骼点的处理优先级。
示例性的，如图2所示，以分支Root’-U’-V’-W’-X’-Y’为例，其中，待配准指部骨骼点的处理优先级由高到低为U’-V’-W’-X’-Y’。如图3所示，以分支Root-V-W-X-Y为例，其中，预设指部骨骼点的处理优先级由高到低为V-W-X-Y。
通过S303-S304可以确定多个待配准指部骨骼点的处理优先级以及多个预设指部骨骼点的处理优先级，以便于后续根据处理优先级来有序的对待配准指部骨骼点进行配准。
S305、确定至少一个指部骨骼点组。
其中，指部骨骼点组可以是待配准骨骼链与预设骨骼链中对应的指部骨骼点的组合。指部骨骼点组中包括位于相应骨骼链分支且处理优先级相同的预设指部骨骼点和待配准指部骨骼点。
示例性的，如图2和图3所示，待配准骨骼链中的分支Root’-A’-B’-C’-D’-E’与预设骨骼链中的分支Root-A-B-C-D-E相对应。其中，预设指部骨骼点A和待配准指部骨骼点A’的处理优先级相同，可以将预设指部骨骼点A和待配准指部骨骼点A’作为一个指部骨骼点组，其他指部骨骼点组可以按照上述相同方式确定。
S306、对于至少一个指部骨骼点组，根据处理优先级，确定当前指部骨骼点组所依赖的前依赖骨骼点组和后依赖骨骼点组。
其中，前依赖骨骼点组可以是被当前指部骨骼点组所依赖的腕部骨骼点组或指部骨骼点组。后依赖骨骼点组可以是依赖当前指部骨骼点组的指部骨骼点组。前依赖骨骼点组包括腕部骨骼点组或指部骨骼点组。 后依赖骨骼点组包括指部骨骼点组。可以理解的是，后依赖骨骼点组直接依赖当前指部骨骼点组，当前指部骨骼点组直接依赖前依赖骨骼点组。
例如，针对每一个指部骨骼点组，可以将该指部骨骼点组作为当前指部骨骼点组，并使用相同的方式确定当前指部骨骼点组所依赖的前依赖骨骼点组和后依赖骨骼点组。根据处理优先级，确定直接依赖当前指部骨骼点组的指部骨骼点组为后依赖骨骼点组，确定被当前指部骨骼点组直接依赖的指部骨骼点组或腕部骨骼点组为前依赖骨骼点组。
示例性的，如图2和图3所示，以预设指部骨骼点A和待配准指部骨骼点A’组成的指部骨骼点组作为当前指部骨骼点组，根据处理优先级，可以确定当前指部骨骼点组所依赖的前依赖骨骼点组为预设腕部骨骼点Root和待配准腕部骨骼点Root’组成的腕部骨骼点组，可以确定当前指部骨骼点组所依赖的后依赖骨骼点组为预设腕部骨骼点B和待配准腕部骨骼点B’组成的指部骨骼点组。其他指部骨骼点组所依赖的前依赖骨骼点组和后依赖骨骼点组可以按照上述相同方式确定。
S307、判断前依赖骨骼点组为腕部骨骼点组还是指部骨骼点组，基于前依赖骨骼点组为腕部骨骼点组的判断结果，执行S308；基于前依赖骨骼点组为指部骨骼点组的判断结果，执行S309。
S308、根据腕部状态信息中的腕部旋转信息、当前指部骨骼点组中的位姿信息以及后依赖骨骼点组中的位姿信息，确定当前指部骨骼点组中待配准指部骨骼点的指部状态信息，并执行S310。
例如，根据当前指部骨骼点组中的位姿信息以及后依赖骨骼点组中的位姿信息，可以确定当前指部骨骼点组中待配准骨骼点的旋转矩阵。根据该旋转矩阵与腕部旋转信息，可以确定当前指部骨骼点组中待配准骨骼点的当前指部旋转信息。根据当前指部旋转信息以及腕部旋转信息，可以确定当前指部骨骼点组中待配准骨骼点的当前指部位姿信息。进而，将当前指部旋转信息和当前指部位姿信息，确定为当前指部骨骼点组中待配准指部骨骼点的指部状态信息。
S309、根据前依赖骨骼点组中待配准骨骼点的指部状态信息中的目标指部旋转信息、当前指部骨骼点组中的位姿信息以及后依赖骨骼点组的位姿信息，确定当前指部骨骼点组中待配准指部骨骼点的指部状态信息，并执行S310。
例如，根据当前指部骨骼点组中的位姿信息以及后依赖骨骼点组中的位姿信息，可以确定当前指部骨骼点组中待配准骨骼点的旋转矩阵。根据该旋转矩阵与目标指部旋转信息，可以确定当前指部骨骼点组中待配准骨骼点的当前指部旋转信息。根据当前指部旋转信息以及目标指部旋转信息，可以确定当前指部骨骼点组中待配准骨骼点的当前指部位姿信息。进而，将当前指部旋转信息和当前指部位姿信息，确定为当前指部骨骼点组中待配准指部骨骼点的指部状态信息。
通过S305-S308或S305-S309，可以依次确定待配准骨骼链中每个分支中的多个待配准指部骨骼点的指部状态信息，以保证多个待配准指部骨骼点的指部旋转信息以及指部位姿信息与多个预设指部骨骼点准确配准，提高配准的准确率和效率。
在上述示例的基础上，可以通过下述方式来分别确定当前指部骨骼点组中待配准骨骼点的当前指部旋转信息和当前指部位姿信息，以确定当前指部骨骼点组中待配准指部骨骼点的指部状态信息，降低确定指部状态信息的复杂度，并提高确定指部状态信息准确度：
根据目标指部旋转信息、当前指部骨骼点组中的第三位姿信息和第四位姿信息、以及后依赖骨骼点组中的第三位姿信息和第四位姿信息，确定当前指部骨骼点组中待配准骨骼点的当前指部旋转信息。
例如，根据当前指部骨骼点组中的第三位姿信息和后依赖骨骼点组中的第三位姿信息可以确定当前待配准指部骨骼的方向。根据当前指部骨骼点组中的第四位姿信息和后依赖骨骼点组中的第四位姿信息可以确定当前预设指部骨骼的方向。根据当前待配准指部骨骼的方向以及当前预设指部骨骼的方向，可以确定由当前待配准指部骨骼的方向旋转至当前预设指部骨骼的方向所需的旋转矩阵。根据目标指部旋转信息和确定出的旋转矩阵可以确定出当前指部骨骼点组中待配准骨骼点的当前指部旋转信息。
基于目标指部旋转信息和当前指部旋转信息，确定当前指部骨骼点组中待配准骨骼点的当前指部位姿信息。
例如，根据当前指部骨骼点组中的第四位姿信息中的位置信息，可以确定当前指部骨骼点组中待配准骨骼点的当前指部位姿信息的位置信息。根据目标指部旋转信息和当前指部旋转信息，可以确定当前指部骨骼点组中待配准骨骼点的姿态信息。将上述确定出的位置信息和姿态信息作为当前指部骨骼点组中待配准骨骼点的当前指部位姿信息。
基于当前指部旋转信息和当前指部位姿信息，确定待配准骨骼点的指部状态信息。
例如，将当前指部旋转信息和当前指部位姿信息作为待配准骨骼点的指部状态信息。
在上述示例的基础上，可以通过下述方式来确定当前指部骨骼点组中待配准骨骼点的当前指部旋转信息，令当前指部旋转信息平滑过渡，以便后续按照当前指部旋转信息显示时的显示效果更具真实性。
根据当前指部骨骼点组中的第三位姿信息和后依赖骨骼点组中的第三位姿信息，确定第三旋转向量；以及，根据当前指部骨骼点组中的第四位姿信息和后依赖骨骼点组中的第四位姿信息，确定第四旋转向量；根据第三旋转向量和第四旋转向量，确定当前指部骨骼点组中的待配准指部骨骼点旋转至相应预设指部骨骼点的旋转矩阵；基于旋转矩阵和目标指部旋转信息，确定当前指部骨骼点组中待配准骨骼点的当前指部旋转信息。
其中，第三旋转向量可以是当前指部骨骼点组中的待配准指部骨骼点指向后依赖骨骼点组中的待配准指部骨骼点的向量。可以是第四旋转向量当前指部骨骼点组中的预设指部骨骼点指向后依赖骨骼点组中的预设指部骨骼点的向量。
例如，根据当前指部骨骼点组中的第三位姿信息中的位置信息和后依赖骨骼点组中的第三位姿信息中的位置信息，可以确定当前指部骨骼点组中的待配准指部骨骼点指向后依赖骨骼点组中的待配准指部骨骼点的向量为第三旋转向量。根据当前指部骨骼点组中的第四位姿信息中的位置信息和后依赖骨骼点组中的第四位姿信息中的位置信息，可以确定当前指部骨骼点组中的预设指部骨骼点指向后依赖骨骼点组中的预设指部骨骼点的向量为第四旋转向量。根据第三旋转向量和第四旋转向量可以求出由第三旋转向量的方向旋转到第四旋转向量的方向的旋转四元数，即为旋转矩阵。将目标指部旋转信息的逆与旋转矩阵相乘，可以得到当前指部骨骼点组中待配准骨骼点的当前指部旋转信息。
示例性的，待配准指部骨骼点旋转前的示意图如图9所示，其中，R-S-T为预设骨骼链，R’-S’-T’为待配准骨骼链，S’和S表示当前指部骨骼点组，T’和T表示后依赖骨骼点组，R’和R表示前依赖骨骼点组。当前指部骨骼点组中的第三位姿信息中的位置信息为Sp’，后依赖骨骼点组中的第三位姿信息中的位置信息为Tp’，第三旋转向量O＝T’p-S’p。当前指部骨骼点组中的第四位姿信息中的位置信息为Sp，后依赖骨骼点组中的第四位姿信息中的位置信息为Tp，第四旋转向量C＝Tp-Sp。当前指部骨骼点组中的待配准指部骨骼点旋转至相应预设指部骨骼点的旋转矩阵Q＝FromToQuaternion(O,C)，其表示求出O旋转到C的旋转四元数。目标指部旋转信息为R’r，当前指部骨骼点组中待配准骨骼点的当前指部旋转信息S’lr＝Inverse(R’r)*Q。待配准指部骨骼点旋转后的示意图如图10所示。
需要说明的是，若当前指部骨骼点组中待配准骨骼点存在初始旋转S’r，则Q＝FromToQuaternion(O,C)*S’r，即根据旋转矩阵乘以当前指部骨骼点组中待配准骨骼点的初始旋转的结果，更新旋转矩阵。
S310、基于腕部状态信息和指部状态信息，在待处理视频帧中显示目标部位。
本公开实施例的技术方案，通过确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链，根据待配准腕部骨骼点的第一位姿信息和预设腕部骨骼点的第二位姿信息，确定待配准腕部骨骼点的腕部状态信息，调取预先设置的待配准腕部骨骼点和待配准指部骨骼点的第一依赖信息，以及预设腕部骨骼点和预设指部骨骼点的第二依赖信息，依据第一依赖信息和第二依赖信息，确定对待配准指部骨骼点和预设指部骨骼点的处理优先级，以基于处理优先级依次确定待配准指部骨骼点的指部状态信息，确定至少一个指部骨骼点组，对于至少一个指部骨骼点组，根据处理优先级，确定当前指部骨骼点组所依赖的前依赖骨骼点组和后依赖骨骼点组，进而，判断前依赖骨骼点组为腕部骨骼点组还是指部骨骼点组，以根据不同的判断结果分别确定当前指部骨骼点组中待配准指部骨骼点的指部状态信息，基于腕部状态信息和指部状态信息，在待处理视频帧中显示目标部位，避免了目标部位中指部动画数据迁移时由于穿模现象导致的显示效果较差以及真实性较差的情况，在不穿模的情况下提高目标部位中指部的显示效果以及真实性，进而提高用户体验度。
图11为本公开实施例所提供的一种图像处理装置的结构示意图，如图11所示，所述装置包括：骨骼链确定模块410、腕部确定模块420、指部确定模块430以及显示模块440。
其中，骨骼链确定模块410，设置为确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链；其中，所述待配准骨骼链上包括待配准腕部骨骼点和待配准指部骨骼点，所述预设骨骼链上包括预设腕部骨骼点和预设指部骨骼点；腕部确定模块420，设置为根据所述待配准腕部骨骼点的第一位姿信息和预设腕 部骨骼点的第二位姿信息，确定所述待配准腕部骨骼点的腕部状态信息；其中，所述腕部状态信息包括腕部旋转信息和腕部位姿信息；指部确定模块430，设置为依据所述腕部状态信息、待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息；其中，所述指部状态信息包括指部旋转信息和指部位姿信息；显示模块440，设置为基于所述腕部状态信息和指部状态信息，在所述待处理视频帧中显示所述目标部位。
例如，腕部确定模块420，还设置为根据所述待配准腕部骨骼点的第一位姿信息和与所述待配准腕部骨骼点直接关联的至少一个待配准指部骨骼点的第三位姿信息，确定第一旋转信息；根据所述预设腕部骨骼点的第二位姿信息和与所述预设腕部骨骼点直接关联的至少一个预设指部骨骼点的第四位姿信息，确定第二旋转信息；基于所述第一旋转信息和所述第二旋转信息，确定所述待配准腕部骨骼点的腕部状态信息。
例如，腕部确定模块420，还设置为根据所述第二旋转信息和所述第一旋转信息的逆，确定对齐变换矩阵；基于所述对齐变换矩阵和所述待配准腕部骨骼点的第一位姿信息，确定所述腕部状态信息。
例如，腕部确定模块420，还设置为基于第二位姿信息中的位置信息，确定所述腕部位姿信息中的位置信息；基于所述对齐变换矩阵和所述第一位姿信息中的位置信息，确定所述腕部位姿信息中的姿态信息。
例如，所述装置还包括：依赖处理模块，设置为调取预先设置的待配准腕部骨骼点和待配准指部骨骼点的第一依赖信息，以及预设腕部骨骼点和预设指部骨骼点的第二依赖信息；其中，所述依赖信息包括骨骼链中位于同一分支上至少一个指部骨骼点所直接依赖的指部骨骼点或腕部骨骼点；依据所述第一依赖信息和所述第二依赖信息，确定对所述待配准指部骨骼点和所述预设指部骨骼点的处理优先级，以基于所述处理优先级依次确定所述待配准指部骨骼点的指部状态信息。
例如，指部确定模块430，还设置为确定至少一个指部骨骼点组；其中，所述指部骨骼点组中包括位于相应骨骼链分支且所述处理优先级相同的预设指部骨骼点和待配准指部骨骼点；对于所述至少一个指部骨骼点组，根据所述处理优先级，确定当前指部骨骼点组所依赖的前依赖骨骼点组和后依赖骨骼点组；其中，所述前依赖骨骼点组包括腕部骨骼点组或指部骨骼点组；若所述前依赖骨骼点组为腕部骨骼点组，则根据所述腕部状态信息中的腕部旋转信息、所述当前指部骨骼点组中的位姿信息以及所述后依赖骨骼点组中的位姿信息，确定所述当前指部骨骼点组中待配准指部骨骼点的指部状态信息。
例如，指部确定模块430，还设置为若所述前依赖骨骼点组为指部骨骼点组，则根据所述前依赖骨骼点组中待配准骨骼点的指部状态信息中的目标指部旋转信息、所述当前指部骨骼点组中的位姿信息以及所述后依赖骨骼点组的位姿信息，确定所述当前指部骨骼点组中待配准指部骨骼点的指部状态信息。
例如，指部确定模块430，还设置为根据所述目标指部旋转信息、所述当前指部骨骼点组中的第三位姿信息和第四位姿信息、以及所述后依赖骨骼点组中的第三位姿信息和第四位姿信息，确定当前指部骨骼点组中待配准骨骼点的当前指部旋转信息；基于所述目标指部旋转信息和所述当前指部旋转信息，确定当前指部骨骼点组中待配准骨骼点的当前指部位姿信息；基于所述当前指部旋转信息和所述当前指部位姿信息，确定所述待配准骨骼点的指部状态信息。
例如，指部确定模块430，还设置为根据所述当前指部骨骼点组中的第三位姿信息和所述后依赖骨骼点组中的第三位姿信息，确定第三旋转向量；以及，根据所述当前指部骨骼点组中的第四位姿信息和所述后依赖骨骼点组中的第四位姿信息，确定第四旋转向量；根据所述第三旋转向量和第四旋转向量，确定所述当前指部骨骼点组中的待配准指部骨骼点旋转至相应预设指部骨骼点的旋转矩阵；基于所述旋转矩阵和所述目标指部旋转信息，确定当前指部骨骼点组中待配准骨骼点的当前指部旋转信息。
例如，显示模块440，还设置为基于所述腕部状态信息和指部状态信息，调整目标部位中多个骨骼点在于显示界面中的显示位置和显示姿态，得到与待处理视频帧相对应的目标视频帧。
例如，所述装置还包括：调整模块，设置为若基于所述腕部状态信息，确定所述待配准腕部骨骼点与预设腕部骨骼点的位置偏移量大于预设偏移量阈值，则确定所述待配准腕部骨骼点所属胳膊骨骼链和/或腿部骨骼链的长度信息；基于所述长度信息和所述位置偏移量，调整所述胳膊骨骼链和/或腿部骨骼链中至少一个骨骼点的显示位置。
本公开实施例的技术方案，在确定需要骨骼重定位时，可以将接入动画中的待配准骨骼链中的待配准腕部骨骼点，与预设骨骼链中的预设腕部骨骼点配准，得到待配准腕部骨骼点的腕部状态信息，在确定腕部状态信息的基础上，可以依次根据预设骨骼链中多个预设指部骨骼点，以及配准指部骨骼点的位姿信息，确定多个待配准指部骨骼点的指部状态信息，进而基于腕部状态信息和指部状态信息，调整目标部位与待 处理视频帧中的显示信息，避免了目标部位的动画数据迁移时容易穿模的情况，以及目标部位的显示效果较差、真实性较差的情况，在不穿模的情况下，提高目标部位于显示界面中显示的真实性，进而提高用户体验度。
本公开实施例所提供的图像处理装置可执行本公开任意实施例所提供的图像处理方法，具备执行方法相应的功能模块和有益效果。
值得注意的是，上述装置所包括的多个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，多个功能单元的具体名称也只是为了便于相互区分，并不用于限制本公开实施例的保护范围。
图12为本公开实施例所提供的一种电子设备的结构示意图。下面参考图12，其示出了适于用来实现本公开实施例的电子设备(例如图12中的终端设备或服务器)500的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图12示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图12所示，电子设备500可以包括处理装置(例如中央处理器、图形处理器等)501，其可以根据存储在只读存储器(ROM)502中的程序或者从存储装置508加载到随机访问存储器(RAM)503中的程序而执行多种适当的动作和处理。在RAM 503中，还存储有电子设备500操作所需的多种程序和数据。处理装置501、ROM 502以及RAM 503通过总线504彼此相连。编辑/输出(I/O)接口505也连接至总线504。
通常，以下装置可以连接至I/O接口505：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置506；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置507；包括例如磁带、硬盘等的存储装置508；以及通信装置509。通信装置509可以允许电子设备500与其他设备进行无线或有线通信以交换数据。虽然图12示出了具有多种装置的电子设备500，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置509从网络上被下载和安装，或者从存储装置508被安装，或者从ROM 502被安装。在该计算机程序被处理装置501执行时，执行本公开实施例的方法中限定的上述功能。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
本公开实施例提供的电子设备与上述实施例提供的图像处理方法属于同一构思，未在本实施例中详尽描述的技术细节可参见上述实施例，并且本实施例与上述实施例具有相同的有益效果。
本公开实施例提供了一种计算机存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述实施例所提供的图像处理方法。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等 等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如HTTP(HyperText Transfer Protocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链；其中，所述待配准骨骼链上包括待配准腕部骨骼点和待配准指部骨骼点，所述预设骨骼链上包括预设腕部骨骼点和预设指部骨骼点；根据所述待配准腕部骨骼点的第一位姿信息和预设腕部骨骼点的第二位姿信息，确定所述待配准腕部骨骼点的腕部状态信息；其中，所述腕部状态信息包括腕部旋转信息和腕部位姿信息；依据所述腕部状态信息、待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息；其中，所述指部状态信息包括指部旋转信息和指部位姿信息；基于所述腕部状态信息和指部状态信息，在所述待处理视频帧中显示所述目标部位。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开多种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定，例如，骨骼链确定模块还可以被描述为“获取待配准骨骼链和预设骨骼链的模块”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
根据本公开的一个或多个实施例，【示例一】提供了一种图像处理方法，该方法包括：
确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链；其中，所述待配准骨骼链上包括待配准腕部骨骼点和待配准指部骨骼点，所述预设骨骼链上包括预设腕部骨骼点和预设指部骨骼点；
根据所述待配准腕部骨骼点的第一位姿信息和预设腕部骨骼点的第二位姿信息，确定所述待配准腕部 骨骼点的腕部状态信息；其中，所述腕部状态信息包括腕部旋转信息和腕部位姿信息；
依据所述腕部状态信息、待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息；其中，所述指部状态信息包括指部旋转信息和指部位姿信息；
基于所述腕部状态信息和指部状态信息，在所述待处理视频帧中显示所述目标部位。
根据本公开的一个或多个实施例，【示例二】提供了一种图像处理方法，还包括：
例如，所述根据所述待配准腕部骨骼点的第一位姿信息和预设腕部骨骼点的第二位姿信息，确定所述待配准腕部骨骼点的腕部状态信息，包括：
根据所述待配准腕部骨骼点的第一位姿信息和与所述待配准腕部骨骼点直接关联的至少一个待配准指部骨骼点的第三位姿信息，确定第一旋转信息；
根据所述预设腕部骨骼点的第二位姿信息和与所述预设腕部骨骼点直接关联的至少一个预设指部骨骼点的第四位姿信息，确定第二旋转信息；
基于所述第一旋转信息和所述第二旋转信息，确定所述待配准腕部骨骼点的腕部状态信息。
根据本公开的一个或多个实施例，【示例三】提供了一种图像处理方法，还包括：
例如，若所述至少一个待配准指部骨骼点包括第一待配准指部骨骼点和第二待配准指部骨骼点，所述根据所述待配准腕部骨骼点的第一位姿信息和与所述待配准腕部骨骼点直接关联的至少一个待配准指部骨骼点的第三位姿信息，确定第一旋转信息，包括：
基于所述第一位姿信息和所述第一待配准指部骨骼点的位姿信息，确定第一向量；
基于所述第一位姿信息和所述第二待配准指部骨骼点的位姿信息，确定第二向量；
基于所述第一向量和所述第二向量，确定第一旋转信息。
根据本公开的一个或多个实施例，【示例四】提供了一种图像处理方法，还包括：
例如，若所述至少一个预设指部骨骼点包括第一预设指部骨骼点和第二预设指部骨骼点，所述根据所述预设腕部骨骼点的第二位姿信息和与所述预设腕部骨骼点直接关联的至少一个预设指部骨骼点的第四位姿信息，确定第二旋转信息，包括：
基于所述第二位姿信息和所述第一预设指部骨骼点的位姿信息，确定第三向量；
基于所述第二位姿信息和所述第二预设指部骨骼点的位姿信息，确定第四向量；
基于所述第三向量和所述第四向量，确定第二旋转信息。
根据本公开的一个或多个实施例，【示例五】提供了一种图像处理方法，还包括：
例如，所述基于所述第一旋转信息和所述第二旋转信息，确定所述待配准腕部骨骼点的腕部状态信息，包括：
根据所述第二旋转信息和所述第一旋转信息的逆，确定对齐变换矩阵；
基于所述对齐变换矩阵和所述待配准腕部骨骼点的第一位姿信息，确定所述腕部状态信息。
根据本公开的一个或多个实施例，【示例六】提供了一种图像处理方法，还包括：
例如，基于所述对齐变换矩阵和所述待配准腕部骨骼点的第一位姿信息，确定所述腕部状态信息中的腕部位姿信息，包括：
基于第二位姿信息中的位置信息，确定所述腕部位姿信息中的位置信息；
基于所述对齐变换矩阵和所述第一位姿信息中的位置信息，确定所述腕部位姿信息中的姿态信息。
根据本公开的一个或多个实施例，【示例七】提供了一种图像处理方法，还包括：
例如，在所述依据所述腕部状态信息、待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息之前，还包括：
调取预先设置的待配准腕部骨骼点和待配准指部骨骼点的第一依赖信息，以及预设腕部骨骼点和预设指部骨骼点的第二依赖信息；其中，所述依赖信息包括骨骼链中位于同一分支上至少一个指部骨骼点所直接依赖的指部骨骼点或腕部骨骼点；
依据所述第一依赖信息和所述第二依赖信息，确定对所述待配准指部骨骼点和所述预设指部骨骼点的处理优先级，以基于所述处理优先级依次确定所述待配准指部骨骼点的指部状态信息。
根据本公开的一个或多个实施例，【示例八】提供了一种图像处理方法，还包括：
例如，所述依据所述腕部状态信息、待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息，包括：
确定至少一个指部骨骼点组；其中，所述指部骨骼点组中包括位于相应骨骼链分支且所述处理优先级相同的预设指部骨骼点和待配准指部骨骼点；
对于所述至少一个指部骨骼点组，根据所述处理优先级，确定当前指部骨骼点组所依赖的前依赖骨骼点组和后依赖骨骼点组；其中，所述前依赖骨骼点组包括腕部骨骼点组或指部骨骼点组；
若所述前依赖骨骼点组为腕部骨骼点组，则根据所述腕部状态信息中的腕部旋转信息、所述当前指部骨骼点组中的位姿信息以及所述后依赖骨骼点组中的位姿信息，确定所述当前指部骨骼点组中待配准指部骨骼点的指部状态信息。
根据本公开的一个或多个实施例，【示例九】提供了一种图像处理方法，还包括：
例如，还包括：
若所述前依赖骨骼点组为指部骨骼点组，则根据所述前依赖骨骼点组中待配准骨骼点的指部状态信息中的目标指部旋转信息、所述当前指部骨骼点组中的位姿信息以及所述后依赖骨骼点组的位姿信息，确定所述当前指部骨骼点组中待配准指部骨骼点的指部状态信息。
根据本公开的一个或多个实施例，【示例十】提供了一种图像处理方法，还包括：
例如，所述根据所述前依赖骨骼点组中待配准骨骼点的指部状态信息中的目标指部旋转信息、所述当前指部骨骼点组中的位姿信息以及所述后依赖骨骼点组的位姿信息，确定所述当前指部骨骼点组中待配准指部骨骼点的指部状态信息，包括：
根据所述目标指部旋转信息、所述当前指部骨骼点组中的第三位姿信息和第四位姿信息、以及所述后依赖骨骼点组中的第三位姿信息和第四位姿信息，确定当前指部骨骼点组中待配准骨骼点的当前指部旋转信息；
基于所述目标指部旋转信息和所述当前指部旋转信息，确定当前指部骨骼点组中待配准骨骼点的当前指部位姿信息；
基于所述当前指部旋转信息和所述当前指部位姿信息，确定所述待配准骨骼点的指部状态信息。
根据本公开的一个或多个实施例，【示例十一】提供了一种图像处理方法，还包括：
例如，所述根据所述目标指部旋转信息、所述当前指部骨骼点组中的第三位姿信息和第四位姿信息、以及所述后依赖骨骼点组中的第三位姿信息和第四位姿信息，确定当前指部骨骼点组中待配准骨骼点的当前指部旋转信息，包括：
根据所述当前指部骨骼点组中的第三位姿信息和所述后依赖骨骼点组中的第三位姿信息，确定第三旋转向量；以及，
根据所述当前指部骨骼点组中的第四位姿信息和所述后依赖骨骼点组中的第四位姿信息，确定第四旋转向量；
根据所述第三旋转向量和第四旋转向量，确定所述当前指部骨骼点组中的待配准指部骨骼点旋转至相应预设指部骨骼点的旋转矩阵；
基于所述旋转矩阵和所述目标指部旋转信息，确定当前指部骨骼点组中待配准骨骼点的当前指部旋转信息。
根据本公开的一个或多个实施例，【示例十二】提供了一种图像处理方法，还包括：
例如，所述基于所述腕部状态信息和指部状态信息，在所述待处理视频帧中显示所述目标部位，包括：
基于所述腕部状态信息和指部状态信息，调整目标部位中多个骨骼点在于显示界面中的显示位置和显示姿态，得到与待处理视频帧相对应的目标视频帧。
根据本公开的一个或多个实施例，【示例十三】提供了一种图像处理方法，还包括：
例如，还包括：
若基于所述腕部状态信息，确定所述待配准腕部骨骼点与预设腕部骨骼点的位置偏移量大于预设偏移量阈值，则确定所述待配准腕部骨骼点所属胳膊骨骼链和/或腿部骨骼链的长度信息；
基于所述长度信息和所述位置偏移量，调整所述胳膊骨骼链和/或腿部骨骼链中至少一个骨骼点的显示位置。
根据本公开的一个或多个实施例，【示例十四】提供了一种图像处理装置，该装置包括：
骨骼链确定模块，设置为确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链；其中，所述待配准骨骼链上包括待配准腕部骨骼点和待配准指部骨骼点，所述预设骨骼链上包括预设腕部骨骼点和预设 指部骨骼点；
腕部确定模块，设置为根据所述待配准腕部骨骼点的第一位姿信息和预设腕部骨骼点的第二位姿信息，确定所述待配准腕部骨骼点的腕部状态信息；其中，所述腕部状态信息包括腕部旋转信息和腕部位姿信息；
指部确定模块，设置为依据所述腕部状态信息、待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息；其中，所述指部状态信息包括指部旋转信息和指部位姿信息；
显示模块，设置为基于所述腕部状态信息和指部状态信息，在所述待处理视频帧中显示所述目标部位。
此外，虽然采用特定次序描绘了多种操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的多种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。

Claims (14)

  1. 一种图像处理方法，包括：
    确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链；其中，所述待配准骨骼链上包括待配准腕部骨骼点和待配准指部骨骼点，所述预设骨骼链上包括预设腕部骨骼点和预设指部骨骼点；
    根据所述待配准腕部骨骼点的第一位姿信息和所述预设腕部骨骼点的第二位姿信息，确定所述待配准腕部骨骼点的腕部状态信息；其中，所述腕部状态信息包括腕部旋转信息和腕部位姿信息；
    依据所述腕部状态信息、所述待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息；其中，所述指部状态信息包括指部旋转信息和指部位姿信息；
    基于所述腕部状态信息和所述指部状态信息，在所述待处理视频帧中显示所述目标部位。
  2. 根据权利要求1所述的方法，其中，所述根据所述待配准腕部骨骼点的第一位姿信息和所述预设腕部骨骼点的第二位姿信息，确定所述待配准腕部骨骼点的腕部状态信息，包括：
    根据所述待配准腕部骨骼点的第一位姿信息和与所述待配准腕部骨骼点直接关联的至少一个待配准指部骨骼点的第三位姿信息，确定第一旋转信息；
    根据所述预设腕部骨骼点的第二位姿信息和与所述预设腕部骨骼点直接关联的至少一个预设指部骨骼点的第四位姿信息，确定第二旋转信息；
    基于所述第一旋转信息和所述第二旋转信息，确定所述待配准腕部骨骼点的腕部状态信息。
  3. 根据权利要求2所述的方法，其中，所述基于所述第一旋转信息和所述第二旋转信息，确定所述待配准腕部骨骼点的腕部状态信息，包括：
    根据所述第二旋转信息和所述第一旋转信息的逆，确定对齐变换矩阵；
    基于所述对齐变换矩阵和所述待配准腕部骨骼点的第一位姿信息，确定所述腕部状态信息。
  4. 根据权利要求3所述的方法，其中，基于所述对齐变换矩阵和所述待配准腕部骨骼点的第一位姿信息，确定所述腕部状态信息中的腕部位姿信息，包括：
    基于所述第二位姿信息中的位置信息，确定所述腕部位姿信息中的位置信息；
    基于所述对齐变换矩阵和所述第一位姿信息中的位置信息，确定所述腕部位姿信息中的姿态信息。
  5. 根据权利要求1所述的方法，在所述依据所述腕部状态信息、所述待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息之前，还包括：
    调取预先设置的待配准腕部骨骼点和待配准指部骨骼点的第一依赖信息，以及预设腕部骨骼点和预设指部骨骼点的第二依赖信息；其中，所述第一依赖信息和所述第二依赖信息分别包括骨骼链中位于同一分支上至少一个指部骨骼点所直接依赖的指部骨骼点或腕部骨骼点；
    依据所述第一依赖信息和所述第二依赖信息，确定对所述待配准指部骨骼点和所述预设指部骨骼点的处理优先级，以基于所述处理优先级依次确定所述待配准指部骨骼点的指部状态信息。
  6. 根据权利要求5所述的方法，其中，所述依据所述腕部状态信息、所述待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息，包括：
    确定至少一个指部骨骼点组；其中，所述指部骨骼点组中包括位于相应骨骼链分支且所述处理优先级相同的预设指部骨骼点和待配准指部骨骼点；
    对于所述至少一个指部骨骼点组，根据所述处理优先级，确定当前指部骨骼点组所依赖的前依赖骨骼点组和后依赖骨骼点组；其中，所述前依赖骨骼点组包括腕部骨骼点组或指部骨骼点组；
    响应于确定所述前依赖骨骼点组为腕部骨骼点组，根据所述腕部状态信息中的腕部旋转信息、所述当前指部骨骼点组中的位姿信息以及所述后依赖骨骼点组中的位姿信息，确定所述当前指部骨骼点组中待配准指部骨骼点的指部状态信息。
  7. 根据权利要求6所述的方法，还包括：
    响应于确定所述前依赖骨骼点组为指部骨骼点组，根据所述前依赖骨骼点组中待配准骨骼点的指部状态信息中的目标指部旋转信息、所述当前指部骨骼点组中的位姿信息以及所述后依赖骨骼点组的位姿信息，确定所述当前指部骨骼点组中待配准指部骨骼点的指部状态信息。
  8. 根据权利要求7所述的方法，其中，所述根据所述前依赖骨骼点组中待配准骨骼点的指部状态信息中的目标指部旋转信息、所述当前指部骨骼点组中的位姿信息以及所述后依赖骨骼点组的位姿信息，确定所述当前指部骨骼点组中待配准指部骨骼点的指部状态信息，包括：
    根据所述目标指部旋转信息、所述当前指部骨骼点组中的第三位姿信息和第四位姿信息、以及所述后依赖骨骼点组中的第三位姿信息和第四位姿信息，确定所述当前指部骨骼点组中待配准骨骼点的当前指部旋转信息；
    基于所述目标指部旋转信息和所述当前指部旋转信息，确定所述当前指部骨骼点组中待配准骨骼点的当前指部位姿信息；
    基于所述当前指部旋转信息和所述当前指部位姿信息，确定所述当前指部骨骼点组中所述待配准骨骼点的指部状态信息。
  9. 根据权利要求8所述的方法，其中，所述根据所述目标指部旋转信息、所述当前指部骨骼点组中的第三位姿信息和第四位姿信息、以及所述后依赖骨骼点组中的第三位姿信息和第四位姿信息，确定所述当前指部骨骼点组中待配准骨骼点的当前指部旋转信息，包括：
    根据所述当前指部骨骼点组中的第三位姿信息和所述后依赖骨骼点组中的第三位姿信息，确定第三旋转向量；
    根据所述当前指部骨骼点组中的第四位姿信息和所述后依赖骨骼点组中的第四位姿信息，确定第四旋转向量；
    根据所述第三旋转向量和所述第四旋转向量，确定所述当前指部骨骼点组中的待配准指部骨骼点旋转至相应预设指部骨骼点的旋转矩阵；
    基于所述旋转矩阵和所述目标指部旋转信息，确定所述当前指部骨骼点组中待配准骨骼点的当前指部旋转信息。
  10. 根据权利要求1所述的方法，其中，所述基于所述腕部状态信息和所述指部状态信息，在所述待处理视频帧中显示所述目标部位，包括：
    基于所述腕部状态信息和所述指部状态信息，调整所述目标部位中多个骨骼点在所述待处理视频帧的显示界面中的显示位置和显示姿态。
  11. 根据权利要求1所述的方法，还包括：
    响应于确定基于所述腕部状态信息确定所述待配准腕部骨骼点与预设腕部骨骼点的位置偏移量大于预设偏移量阈值，确定所述待配准腕部骨骼点所属胳膊骨骼链和腿部骨骼链中至少之一的长度信息；
    基于所述长度信息和所述位置偏移量，调整以下至少之一的骨骼链中的至少一个骨骼点的显示位置：所述胳膊骨骼链和所述腿部骨骼链。
  12. 一种图像处理装置，包括：
    骨骼链确定模块，设置为确定待处理视频帧中目标部位的待配准骨骼链和预设骨骼链；其中，所述待配准骨骼链上包括待配准腕部骨骼点和待配准指部骨骼点，所述预设骨骼链上包括预设腕部骨骼点和预设指部骨骼点；
    腕部确定模块，设置为根据所述待配准腕部骨骼点的第一位姿信息和所述预设腕部骨骼点的第二位姿信息，确定所述待配准腕部骨骼点的腕部状态信息；其中，所述腕部状态信息包括腕部旋转信息和腕部位姿信息；
    指部确定模块，设置为依据所述腕部状态信息、所述待配准指部骨骼点的第三位姿信息以及所述预设指部骨骼点的第四位姿信息，确定所述待配准指部骨骼点的指部状态信息；其中，所述指部状态信息包括指部旋转信息和指部位姿信息；
    显示模块，设置为基于所述腕部状态信息和所述指部状态信息，在所述待处理视频帧中显示所述目标部位。
  13. 一种电子设备，包括：
    一个或多个处理器；
    存储装置，设置为存储一个或多个程序，
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-11中任一所述的图像处理方法。
  14. 一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行如权利要求1-11中任一所述的图像处理方法。
PCT/CN2022/129443 2022-07-06 2022-11-03 图像处理方法、装置、电子设备及存储介质 WO2024007496A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP22854407.8A EP4325435A1 (en) 2022-07-06 2022-11-03 Image processing method and apparatus, and electronic device and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210797502.6A CN115082604A (zh) 2022-07-06 2022-07-06 图像处理方法、装置、电子设备及存储介质
CN202210797502.6 2022-07-06

Publications (1)

Publication Number Publication Date
WO2024007496A1 true WO2024007496A1 (zh) 2024-01-11

Family

ID=83258423

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/129443 WO2024007496A1 (zh) 2022-07-06 2022-11-03 图像处理方法、装置、电子设备及存储介质

Country Status (3)

Country Link
EP (1) EP4325435A1 (zh)
CN (1) CN115082604A (zh)
WO (1) WO2024007496A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115082604A (zh) * 2022-07-06 2022-09-20 北京字跳网络技术有限公司 图像处理方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106709976A (zh) * 2016-12-22 2017-05-24 厦门幻世网络科技有限公司 骨骼动画的生成方法和装置
CN111127606A (zh) * 2019-12-25 2020-05-08 北京像素软件科技股份有限公司 柔体飘动方向控制方法、装置及电子设备
CN111223171A (zh) * 2020-01-14 2020-06-02 腾讯科技（深圳）有限公司 图像处理方法、装置、电子设备及存储介质
US20210295580A1 (en) * 2018-10-03 2021-09-23 Sony Interactive Entertainment Inc. Skeleton model update apparatus, skeleton model update method, and program
CN114299200A (zh) * 2021-12-31 2022-04-08 网易（杭州）网络有限公司 布料动画处理方法及装置、电子设备、存储介质
CN115082604A (zh) * 2022-07-06 2022-09-20 北京字跳网络技术有限公司 图像处理方法、装置、电子设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106709976A (zh) * 2016-12-22 2017-05-24 厦门幻世网络科技有限公司 骨骼动画的生成方法和装置
US20210295580A1 (en) * 2018-10-03 2021-09-23 Sony Interactive Entertainment Inc. Skeleton model update apparatus, skeleton model update method, and program
CN111127606A (zh) * 2019-12-25 2020-05-08 北京像素软件科技股份有限公司 柔体飘动方向控制方法、装置及电子设备
CN111223171A (zh) * 2020-01-14 2020-06-02 腾讯科技（深圳）有限公司 图像处理方法、装置、电子设备及存储介质
CN114299200A (zh) * 2021-12-31 2022-04-08 网易（杭州）网络有限公司 布料动画处理方法及装置、电子设备、存储介质
CN115082604A (zh) * 2022-07-06 2022-09-20 北京字跳网络技术有限公司 图像处理方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN115082604A (zh) 2022-09-20
EP4325435A1 (en) 2024-02-21

Similar Documents

Publication Publication Date Title
US20220241689A1 (en) Game Character Rendering Method And Apparatus, Electronic Device, And Computer-Readable Medium
US20230386137A1 (en) Elastic object rendering method and apparatus, device, and storage medium
WO2023138504A1 (zh) 图像渲染方法、装置、电子设备及存储介质
WO2023179346A1 (zh) 特效图像处理方法、装置、电子设备及存储介质
WO2024011792A1 (zh) 图像处理方法、装置、电子设备及存储介质
CN113289327A (zh) 移动终端的显示控制方法及装置、存储介质及电子设备
WO2020253716A1 (zh) 图像生成方法和装置
WO2024007496A1 (zh) 图像处理方法、装置、电子设备及存储介质
WO2023185391A1 (zh) 交互式分割模型训练方法、标注数据生成方法及设备
WO2023193642A1 (zh) 视频处理方法、装置、设备及介质
WO2020124995A1 (zh) 手掌法向量确定方法、装置、设备及存储介质
WO2023116801A1 (zh) 一种粒子效果渲染方法、装置、设备及介质
WO2023221409A1 (zh) 虚拟现实空间的字幕渲染方法、装置、设备及介质
WO2024027820A1 (zh) 基于图像的动画生成方法、装置、设备及存储介质
WO2023202357A1 (zh) 显示对象的运动控制方法及设备
WO2024016924A1 (zh) 视频处理方法、装置、电子设备及存储介质
WO2023197911A1 (zh) 三维虚拟对象生成方法、装置、设备、介质和程序产品
US11935176B2 (en) Face image displaying method and apparatus, electronic device, and storage medium
US11593983B2 (en) Image processing method and apparatus, electronic device, and storage medium
WO2022218104A1 (zh) 虚拟形象的碰撞处理方法、装置、电子设备和存储介质
WO2022135017A1 (zh) 动态流体显示方法、装置、电子设备和可读介质
US20240103625A1 (en) Interaction method and apparatus, electronic device, storage medium, and computer program product
WO2024036764A1 (zh) 一种图像处理方法、装置、设备及介质
CN112926629B (zh) 超参数确定方法、装置、深度强化学习框架、介质及设备
WO2023030091A1 (zh) 控制运动物体运动的方法、装置、设备及介质

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2022854407

Country of ref document: EP

Effective date: 20230214

WWE Wipo information: entry into national phase

Ref document number: 18022455

Country of ref document: US