WO2025007829A1 - 用于交互和制作特效的方法、装置、设备和存储介质 - Google Patents

用于交互和制作特效的方法、装置、设备和存储介质 Download PDF

Info

Publication number
WO2025007829A1
WO2025007829A1 PCT/CN2024/102777 CN2024102777W WO2025007829A1 WO 2025007829 A1 WO2025007829 A1 WO 2025007829A1 CN 2024102777 W CN2024102777 W CN 2024102777W WO 2025007829 A1 WO2025007829 A1 WO 2025007829A1
Authority
WO
WIPO (PCT)
Prior art keywords
reference key
key points
virtual object
mounting position
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/102777
Other languages
English (en)
French (fr)
Inventor
袁琦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025007829A1 publication Critical patent/WO2025007829A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/04—Context-preserving transformations, e.g. by using an importance map
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features

Definitions

  • Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for interaction and special effect creation.
  • an interaction method comprises: detecting a plurality of reference key points associated with a virtual object to be mounted from a target image; determining a mounting position of the virtual object based on position information of the plurality of reference key points in the target image; and mounting the virtual object at the mounting position in the target image.
  • a method for producing special effects comprises: presenting an editing interface for producing a mounting special effect; determining an expected mounting position of a virtual object to be mounted based on input information received in the editing interface; determining a plurality of reference key points associated with the expected mounting position, the plurality of reference key points being used to determine a target mounting position of the virtual object during the execution of the mounting special effect; and generating a mounting position based on the plurality of reference key points.
  • Special effects file corresponding to the special effects.
  • an interactive device configured to include: a key point determination module configured to detect multiple reference key points associated with a virtual object to be mounted from a target image; a position determination module configured to determine a mounting position of the virtual object based on position information of the multiple reference key points in the target image; and a mounting module configured to mount the virtual object at the mounting position in the target image.
  • a device for making special effects includes: an interface presentation module, configured to present an editing interface for making a mounting special effect; a position marking module, configured to determine an expected mounting position of a virtual object to be mounted based on input information received in the editing interface; a key point marking module, configured to determine a plurality of reference key points associated with the expected mounting position, the plurality of reference key points being used to determine a target mounting position of the virtual object during the execution of the mounting special effect; and a file generation module, configured to generate a special effect file corresponding to the mounting special effect based on the plurality of reference key points.
  • an electronic device in a fifth aspect of the present disclosure, includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect and/or the second aspect.
  • a computer-readable storage medium wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect and/or the second aspect.
  • FIGS. 1A and 1B show schematic diagrams of conventional mounting virtual objects
  • FIG2 shows a schematic diagram of an example environment in which embodiments according to the present disclosure may be implemented
  • 3A to 3D show schematic diagrams of mounting virtual objects according to some embodiments of the present disclosure
  • FIG4 shows a schematic diagram of mounting a virtual object according to some other embodiments of the present disclosure.
  • 5A to 5C are schematic diagrams showing the production of mounting special effects according to some embodiments of the present disclosure.
  • FIG6 shows a flow chart of an example interaction process according to some embodiments of the present disclosure
  • FIG7 shows a flowchart of an example process for producing special effects according to some embodiments of the present disclosure
  • FIG8 shows a schematic structural block diagram of an interaction device according to some embodiments of the present disclosure
  • FIG9 shows a schematic structural block diagram of an apparatus for producing special effects according to some embodiments of the present disclosure.
  • FIG. 10 shows a block diagram of an electronic device capable of implementing various embodiments of the present disclosure.
  • the embodiments of the present disclosure may involve user data, data acquisition and/or use, etc. These aspects are subject to the corresponding laws, regulations and relevant provisions.
  • all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user knows and confirms. Accordingly, when implementing each embodiment of the present disclosure, the type, scope of use, usage scenario, etc. of the data or information that may be involved should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with the relevant laws and regulations.
  • the specific notification and/or authorization method can vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this respect.
  • Some special effects can support mounting some virtual objects at predetermined locations in an image. As shown in FIG. 1A and FIG. 1B , such an example special effect can, for example, support mounting a virtual object 110 at a predetermined location in an image 105 .
  • the orientation information of the virtual object usually only depends on the orientation of the key points. This results in the orientation of the mounted virtual object being Inappropriate, for example, not consistent with the posture of the human body.
  • an embodiment of the present disclosure proposes an interactive solution.
  • multiple reference key points associated with a virtual object to be mounted can be detected from a target image.
  • the mounting position of the virtual object can be determined based on the position information of the multiple reference key points in the target image. Accordingly, the virtual object can be mounted at the mounting position in the target image.
  • the embodiments of the present disclosure can achieve more accurate mounting of virtual objects and improve the user's interactive experience.
  • the embodiments of the present disclosure can also enrich the support for mounting virtual objects at positions without key points (for example, parts such as the chest), thereby improving the interactive experience.
  • an embodiment of the present disclosure proposes a special effect production scheme.
  • an editing interface for producing a mounting special effect can be presented.
  • the expected mounting position of the virtual object to be mounted can be determined based on the input information received in the editing interface.
  • multiple reference key points associated with the expected mounting position can be determined, and the multiple reference key points are used to determine the target mounting position of the virtual object during the execution of the mounting special effect. Accordingly, a special effect file corresponding to the mounting special effect can be generated based on the multiple reference key points.
  • the embodiments of the present disclosure can support users in creating mounting special effects mounted at non-key point locations, thereby improving the flexibility of users in creating mounting special effects.
  • Fig. 2 shows a schematic diagram of an example environment 200 in which embodiments of the present disclosure can be implemented.
  • the example environment 200 may include an interactive device 210.
  • the interactive device 210 may apply a mount special effect to a target image 220 to provide media content 230 with a mount special effect.
  • such a target image 220 may include, for example, a static image (e.g., a photo) or a dynamic image (e.g., a video) acquired by an image acquisition device. Take pictures or videos and apply mount effects to them.
  • a static image e.g., a photo
  • a dynamic image e.g., a video
  • such a target image 220 may also include a generated image file (e.g., a picture file or a video file).
  • a generated image file e.g., a picture file or a video file.
  • the user may, for example, select an existing image file locally in the interactive device 210 to apply the mount special effect.
  • the user may, for example, upload an image file to be processed to the interactive device 210 to apply the mount special effect.
  • such a target image 220 may also be, for example, a static image or a dynamic image with a target style generated based on a generated image file.
  • a user may upload a local image or video to the interactive device 210, and the interactive device 210 and/or other appropriate electronic devices may use a style conversion tool to generate another image or video corresponding to the generated image or video.
  • a user may upload a realistic photo, and the generated image may be, for example, a cartoon-style picture. This disclosure is not intended to limit the image conversion process.
  • the interactive device 210 can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, PDAs, portable game terminals, VR/AR devices, personal communication systems (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio/video players, digital cameras/camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, game devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof.
  • the electronic device 110 can also support any type of interface for the user (such as "wearable" circuits, etc.).
  • the interactive device 210 may be, for example, an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, as well as big data and artificial intelligence platforms.
  • the interactive device 210 can also apply the mounting special effect to the target image 220 based on the published special effect file 250. As shown in FIG. 2, as will be described in detail below Such special effect files may be generated by the generating device 240, for example.
  • the generating device 240 and the interacting device 210 are shown as separate blocks in FIG. 2 , the generating device 240 and the interacting device 210 may be the same or different electronic devices.
  • Fig. 3A shows a schematic diagram of an interaction process according to some embodiments of the present disclosure.
  • the interaction device 210 can obtain a target image 220 and detect a plurality of key points from the target image 220, for example, key points 305-1 to 305-10 (individually or collectively referred to as key points 305).
  • the key points 305 may also be of different types depending on the objects included in the target image 220. Taking the target image 220 including a human object as an example, the key points 305 may include detected human key points (eg, bone key points).
  • any appropriate key point detection technology may be used to identify the key points 305 in the target image 220 , examples of which may include but are not limited to: Skeleton2d, avatar3d, and the like.
  • the interaction device 210 may determine a plurality of reference key points associated with the virtual object to be mounted from the plurality of key points 305 .
  • such multiple reference key points may be determined based on the expected mounting position of the virtual object. As shown in FIG3A , the virtual object corresponds to the expected mounting position 310. Such expected mounting position 310 does not overlap with the detectable multiple key points 305.
  • the expected mounting position 310 may include, for example, a specific point, or may also include a specific area.
  • the expected mounting position 310 may indicate the left chest area of the human subject.
  • the interaction device 210 may determine, from the multiple key points 305, multiple key points whose distances to the expected mounting position 310 are less than a threshold value as reference key points.
  • the determined reference key points may include, for example, reference key point 305-4 (shoulder point), reference key point 305-2 (neck point), reference key point 305-5 (upper arm point), and reference key point 305-6 (spine point).
  • the multiple reference key points may also include those specified in the process of making the special effect file.
  • multiple reference key points to be detected may also be specified in the process of generating the special effect file.
  • the interaction device 210 may also perform detection of multiple desired reference key points, without performing detection of all key points.
  • the interaction device 210 may determine the mounting position of the virtual object based on a plurality of reference key points.
  • the interactive device 210 may determine the mounting position based on two reference key points among the multiple reference key points. As shown in FIG3B , the interactive device 210 may determine the mounting position based on the reference key point 305 - 6 and the reference key point 305 - 5 .
  • the interaction device 210 may construct a first reference line 315 - 1 based on the reference key point 305 - 6 and the reference key point 305 - 5 , and determine the midpoint of the first reference line 315 - 1 as the mounting position.
  • the interactive device 210 may also determine the target position in the reference line as the mounting position based on the position description information.
  • position description information may indicate, for example, the relationship between the first distance from the target position to be determined to the reference key point 305-6 and the second distance from the target position to the reference key point 305-5.
  • the special effect file 250 may maintain position description information, which may indicate that the distance from the target position to be determined to the reference key point 305-6 is equal to the distance from the target position to the reference key point 305-5.
  • the determined target position may be the midpoint of the reference line.
  • the position description information may indicate a ratio (eg, 2:1) between the distance from the target position to be determined to the reference key point 305-6 and the distance from the target position to the reference key point 305-5.
  • the interaction device 210 may determine the first A target position on the reference line 315 - 1 is determined as the mounting position.
  • such position description information may be determined based on interactions during the special effects production process, for example.
  • the interactive device 210 may also determine the mounting position based on a greater number of reference key points. Accordingly, the position description information may also indicate, for example, the influence weight of each reference key point on the mounting position. Further, the interactive device 210 may determine the target position to be mounted based on a weighted interpolation operation of multiple reference key points.
  • the interaction device 210 may also determine the target position to be mounted based on a third reference key point (eg, reference key point 305 - 4 ) among the plurality of reference key points.
  • a third reference key point eg, reference key point 305 - 4
  • the interaction device 210 may construct the second reference line 315 - 2 based on the reference key point 305 - 4 and the target direction information.
  • the reference key point 305 - 4 may be the reference key point closest to the expected mounting position 310 .
  • the target direction information used to construct the second reference key point 315 - 2 may include preset first direction information.
  • the first direction information may include the vertical direction in the target image 220 .
  • the target direction information used to construct the second reference key point 315-2 may be based on the second direction information determined by detecting the posture of the target object (e.g., human body) in the target image 220.
  • the interaction device 210 may analyze the orientation of the human spine in the target image 220 and determine the orientation as the second direction information.
  • the interaction device 210 may determine the mounting position 320 of the virtual object based on the intersection of the first reference line 315 - 1 and the second reference line 315 - 2 . Accordingly, the interaction device 210 may mount the virtual object at the mounting position 320 of the target image 220 .
  • the embodiments of the present disclosure can determine the mounting position of a virtual object through multiple reference key points, thereby supporting mounting the virtual object to an area that cannot be covered by key point detection, thereby improving the flexibility of special effect interaction.
  • the interactive device 210 may further determine the target orientation of the virtual object based on the position information of the multiple reference key points, and may Towards to complete the mounting of the virtual object.
  • the interactive device 210 can determine at least one reference plane based on multiple reference key points, wherein each reference plane is determined based on at least three reference key points among the multiple reference key points. For example, in the example of FIG3C , the interactive device 210 can determine a first plane 325-1 based on a triangle formed by reference key points 305-2, 305-6, and 305-4; and can determine a second plane 325-2 based on a triangle formed by reference key points 305-5, 305-6, and 305-4.
  • the interaction device 210 can determine the orientation information (also referred to as first orientation information) of the first plane 325-1 and the second plane 325-2, and determine the target orientation of the virtual object based on the orientation information.
  • the interaction device 210 can determine the target orientation of the virtual object based on the mean of the normal directions of the first plane 325-1 and the second plane 325-2.
  • the interaction device 210 may also consider orientation information of a target reference key point among multiple reference key points.
  • a target reference key point may be, for example, a reference key point whose distance to the expected mounting position of the virtual object is less than a preset distance, for example, the closest reference key point 305-4.
  • the interactive device 210 may obtain the orientation information (also referred to as the second orientation information) of the reference key point 305-4. It should be understood that such orientation information may be determined based on the key point recognition process, for example. Further, the interactive device 210 may use the first orientation information and the second orientation information to determine the target orientation of the virtual object.
  • orientation information also referred to as the second orientation information
  • the interaction device 210 may convert the rotation quaternion of the reference key point 305 - 4 into a direction vector, and normalize the direction vector with the average of the normal vector of the first plane 325 - 1 and the normal vector of the second plane, and finally determine the target orientation of the virtual object.
  • the embodiments of the present disclosure can enable the mounted virtual object to better match the posture of the target object (eg, human body) in the target image, thereby improving the interactive experience of the special effects.
  • the target object eg, human body
  • the interaction device 210 may mount the virtual object 330 onto the target image 220 based on the determined mounting position 320 and the target orientation, thereby completing the interaction of the mounting special effect.
  • the embodiment of the present disclosure can support mounting in areas without key point coverage on the one hand, and can also improve the mounting authenticity of virtual objects on the other hand, thereby improving the interactive experience of the mounting special effect.
  • the target image acquired by the interactive device 210 may not be able to detect all the reference key points, thus causing mounting failure.
  • the interactive device 210 can, for example, detect a facial region from the target image, that is, a region corresponding to the face of a person in the target image. Further, the interactive device 210 can determine the proportion of the facial region in the target image. Taking FIG. 4 as an example, the interactive device 210 can detect a facial region 410 from the target image 405 and determine the proportion of the facial region 410 in the target image 405.
  • the interaction device 210 may determine that all reference key points can be detected, and may use the process discussed above to complete the mounting of the virtual object.
  • the interaction device 210 may determine that it is difficult to complete the mounting of the virtual object based on multiple reference key points. In this case, the interaction device 210 may, for example, directly determine the position of the predetermined reference key point among the key points identified from the target image 405 as the mounting position of the virtual object.
  • the interaction device 210 may determine the reference key point 305 - 4 shown in FIG. 3A as the mounting position.
  • a reference key point may be, for example, a key point that is closest to the expected mounting position of the virtual object.
  • such a reference key point may also be a specific key point specified in the generation process of the special effect file.
  • the interactive device 210 may also directly use the orientation information of the reference key point 305 - 5 as the target orientation of the virtual object 415 .
  • the interactive device 210 can also adjust the size of the virtual object 415 so that it can be fully presented in the target image 405.
  • the embodiments of the present disclosure can further ensure that Successfully mounted, improving the stability and flexibility of the mounted special effects interaction.
  • Fig. 5A shows an editing interface 500A for making mounting special effects.
  • Such an editing interface 500A may be provided by the generating device 240 shown in Fig. 2, for example.
  • the editing interface 500A can be used to make a reference model 505 of the mounting effect. It should be understood that such a reference model 505 can be used to represent the basic style of the target object to be mounted, and the present disclosure is not intended to limit the specific form of the reference model 505.
  • the editing device 500A can also present multiple key points 510 that can be used to produce mounting special effects in association with the reference model 505.
  • multiple key points 510 can include multiple preset human key points.
  • the generating device 240 may obtain input information received in the editing interface and determine an expected mounting position 515 of the virtual object to be mounted.
  • the user may set the expected mounting position 515 by clicking, or may adjust the expected mounting position by moving the indicator of the expected mounting position 515.
  • the present disclosure is not intended to limit the specific interaction form of setting the expected mounting position 515.
  • the generating device 240 may determine a plurality of reference key points based on the expected mounting position 515.
  • a plurality of reference key points may include, for example, a reference key point 520-1, a reference key point 520-2, a reference key point 520-3, and a reference key point 520-4 as shown in FIG. 5B.
  • the generating device 240 may determine as reference key points a plurality of key points whose distances to the expected mounting position 515 are less than a threshold distance. Additionally, such a threshold distance may be an initially set distance, or may also be a distance determined based on user input.
  • the generating device 240 may indicate the range 525 for determining the reference key point through a graphic style during the process of determining the reference key point.
  • the range 525 is adjusted to add or remove one or more reference key points.
  • the generating device 240 may also highlight a plurality of reference key points (eg, reference key points 520 - 1 to 520 - 4 ) determined in real time, for example, to distinguish them from other key points that are not determined as reference key points.
  • a plurality of reference key points eg, reference key points 520 - 1 to 520 - 4
  • the user may also add or remove one or more reference key points by, for example, selecting the key points.
  • the generating device 240 may determine the mounting position and/or target orientation of the virtual object based on the mounting logic based on multiple reference key points as discussed above with reference to Figures 3A to 3D. This disclosure will not be repeated here.
  • the generating device 240 may also present a real-time mounting effect corresponding to the determined multiple reference key points in the editing interface. As shown in FIG5C , the generating device 240 may mount the predetermined virtual object 535 on the sample image 530 according to the multiple reference key points determined in the editing interface.
  • virtual objects 535 and/or sample images 530 may be added or uploaded by users during the special effects production process.
  • the generating device 240 may generate a special effect file corresponding to the mounted special effect based on the multiple reference key points.
  • such a special effect file may maintain identifiers of multiple reference key points for determining the mounting position and/or target orientation of the virtual object, so as to detect multiple reference key points during the special effect execution process to complete the mounting special effect.
  • the generation device 240 may also determine multiple distances from the expected mounting position 515 to multiple reference key points (e.g., reference key point 520-1 to reference key point 520-4), and may generate distance description information based on the multiple distances.
  • the distance description information may be used to indicate the relationship between the multiple distances, and the distance description information is used to determine the target mounting position of the virtual object during the execution of the mounting special effect.
  • Such distance description information may, for example, indicate the influence weights of the multiple reference key points 520 - 1 to 520 - 4 on the mounting position to be determined, thereby supporting the determination of the mounting position of the virtual object by weight-based interpolation.
  • the embodiments of the present disclosure can support In the special effects application process discussed above, the mounting position is determined based on the positions of multiple reference key points.
  • FIG6 shows a flow chart of an example interaction process 600 according to some embodiments of the present disclosure.
  • the process 600 may be implemented at the interaction device 210.
  • the process 600 is described below with reference to FIG2.
  • the interaction device 210 detects a plurality of reference key points associated with the virtual object to be mounted from the target image.
  • the interaction device 210 determines a mounting position of the virtual object based on position information of the plurality of reference key points in the target image.
  • the interaction device 210 mounts the virtual object at the mount location in the target image.
  • the mounting position is a target mounting position
  • detecting multiple reference key points associated with the virtual object to be mounted from the target image includes: based on the expected mounting position of the virtual object, determining multiple reference key points from the target image, and the distance between the multiple reference key points and the expected mounting position is less than a first threshold.
  • the multiple reference key points include at least a first reference key point and a second reference key point
  • determining the mounting position of the virtual object based on the position information of the multiple reference key points in the target image includes: determining a first reference line based on the first reference key point and the second reference key point; and determining the mounting position of the virtual object based on the first reference line.
  • determining the mounting position of the virtual object based on the first reference line includes: determining the target position on the first reference line as the mounting position according to preset distance description information, wherein the distance description information indicates the relationship between a first distance from the target position to a first reference key point and a second distance from the target position to a second reference key point.
  • the multiple reference key points also include a third reference key point
  • determining the mounting position of the virtual object based on the first reference line includes: determining the second reference line based on the third reference key point and target direction information; and determining the mounting position based on the intersection of the first reference line and the second reference line.
  • the third reference key point is a reference key point that is closest to the expected mounting position of the virtual object among the multiple reference key points.
  • the target direction information includes: preset first direction information; or second direction information determined based on the posture of the target object in the target image.
  • mounting the virtual object at a mounting position in a target image includes: determining a target orientation of the virtual object based on position information of a plurality of reference key points; and mounting the virtual object at the mounting position in the target image according to the target orientation.
  • determining the target orientation of a virtual object based on position information of multiple reference key points includes: determining at least one reference plane based on multiple reference key points, wherein each reference plane is determined based on at least three reference key points among the multiple reference key points; and determining the target orientation of the virtual object based on first orientation information of at least one reference plane.
  • determining the target orientation of a virtual object based on first orientation information of at least one reference plane includes: determining second orientation information of a target reference key point among multiple reference key points, wherein the distance from the target reference key point to the expected mounting position of the virtual object is less than a preset distance; and determining the target orientation of the virtual object based on the first orientation information and the second orientation information.
  • determining the mounting position of the virtual object based on the position information of multiple reference key points in the target image includes: detecting a facial area from the target image; and in response to the proportion of the facial area in the target image being less than or equal to a threshold, determining the mounting position of the virtual object based on the position information of multiple reference key points in the target image.
  • process 600 further includes: in response to a proportion of the facial area in the target image being greater than a threshold, determining a position of a predetermined reference key point among the plurality of reference key points as a mounting position of the virtual object.
  • process 600 further includes: determining an orientation of a predetermined reference key point among the plurality of reference key points as a target orientation of the virtual object.
  • the predetermined reference key point is a reference key point that is closest to the expected mounting position of the virtual object among the multiple reference key points.
  • the target image includes: a static image captured by an image acquisition device; An image or dynamic image; an uploaded image file; or an image corresponding to a target style generated based on an uploaded image file.
  • Process 700 may be implemented at a generating device 240. Process 700 is described below with reference to FIG.
  • the generating device 240 presents an editing interface for producing mounting special effects.
  • the generating device 240 determines an expected mounting position of the virtual object to be mounted based on the input information received in the editing interface.
  • the generation device 240 determines a plurality of reference key points associated with the expected mounting position, and the plurality of reference key points are used to determine a target mounting position of the virtual object during the execution of the mounting special effect.
  • the generating device 240 generates a special effect file corresponding to the mounted special effect based on the plurality of reference key points.
  • the editing interface presents a reference model and multiple key points corresponding to the reference model, and determining multiple reference key points associated with the expected mounting position includes: determining multiple reference key points from the multiple key points whose distance to the expected mounting position is less than a threshold distance.
  • the threshold distance is determined based on user input.
  • process 700 further includes: highlighting multiple reference key points among the multiple key points in the editing interface.
  • process 700 also includes: determining multiple distances from the expected mounting position to multiple reference key points; and generating distance description information based on the multiple distances, the distance description information is used to indicate the relationship between the multiple distances, and the distance description information is used to determine the target mounting position of the virtual object during the execution of the mounting special effect.
  • FIG8 shows a schematic structure of an example interaction device 800 according to some embodiments of the present disclosure.
  • the apparatus 800 may be implemented as or included in the interactive device 210.
  • Each module/component in the apparatus 800 may be implemented by hardware, software, firmware or any combination thereof.
  • the device 800 includes a key point determination module 810, which is configured to detect multiple reference key points associated with a virtual object to be mounted from a target image; a position determination module 820, which is configured to determine the mounting position of the virtual object based on the position information of the multiple reference key points in the target image; and a mounting module 830, which is configured to mount the virtual object to the mounting position in the target image.
  • a key point determination module 810 which is configured to detect multiple reference key points associated with a virtual object to be mounted from a target image
  • a position determination module 820 which is configured to determine the mounting position of the virtual object based on the position information of the multiple reference key points in the target image
  • a mounting module 830 which is configured to mount the virtual object to the mounting position in the target image.
  • the mounting position is a target mounting position
  • the key point determination module 810 is further configured to: determine multiple reference key points from the target image based on the expected mounting position of the virtual object, and the distances from the multiple reference key points to the expected mounting position are less than a first threshold.
  • the multiple reference key points include at least a first reference key point and a second reference key point
  • the position determination module 820 is further configured to: determine a first reference line based on the first reference key point and the second reference key point; and determine the mounting position of the virtual object based on the first reference line.
  • the position determination module 820 is further configured to: determine the target position on the first reference line as the mounting position according to preset distance description information, wherein the distance description information indicates the relationship between a first distance from the target position to the first reference key point and a second distance from the target position to the second reference key point.
  • the multiple reference key points also include a third reference key point
  • the position determination module 820 is further configured to: determine the second reference line based on the third reference key point and the target direction information; and determine the mounting position based on the intersection of the first reference line and the second reference line.
  • the third reference key point is a reference key point that is closest to the expected mounting position of the virtual object among the multiple reference key points.
  • the target direction information includes: preset first direction information; or second direction information determined based on the posture of the target object in the target image.
  • the mounting module 830 is further configured to: point, determine the target orientation of the virtual object; and mount the virtual object to the mounting position in the target image according to the target orientation.
  • the mounting module 830 is further configured to: determine at least one reference plane based on multiple reference key points, wherein each reference plane is determined based on at least three reference key points among the multiple reference key points; and determine the target orientation of the virtual object based on the first orientation information of at least one reference plane.
  • the mounting module 830 is further configured to: determine second orientation information of a target reference key point among multiple reference key points, wherein the distance from the target reference key point to the expected mounting position of the virtual object is less than a preset distance; and determine the target orientation of the virtual object based on the first orientation information and the second orientation information.
  • the position determination module 820 is further configured to: detect a facial area from the target image; and in response to the proportion of the facial area in the target image being less than or equal to a threshold, determine the mounting position of the virtual object based on the position information of multiple reference key points in the target image.
  • the position determination module 820 is further configured to: in response to a proportion of the facial area in the target image being greater than a threshold, determine the position of a predetermined reference key point among the multiple reference key points as the mounting position of the virtual object.
  • the mounting module 830 is further configured to: determine the orientation of a predetermined reference key point among the multiple reference key points as the target orientation of the virtual object.
  • the predetermined reference key point is a reference key point that is closest to the expected mounting position of the virtual object among the multiple reference key points.
  • the target image includes: a static image or a dynamic image captured by an image capture device; an uploaded image file; or an image corresponding to the target style generated based on the uploaded image file.
  • FIG. 9 shows a schematic structural block diagram of an example device 900 for producing special effects according to certain embodiments of the present disclosure.
  • the device 900 can be implemented as or included in the generating device 240.
  • Each module/component in the device 900 can be implemented by hardware, software, firmware, or any combination thereof.
  • the device 900 includes an interface presentation module 910, which is configured to present an editing interface for producing a mounting special effect; a position marking module 920, which is configured to generate a device 240 to determine the expected mounting position of the virtual object to be mounted based on the input information received in the editing interface; a key point marking module 930, which is configured to determine a plurality of reference key points associated with the expected mounting position, and the plurality of reference key points are used to determine the target mounting position of the virtual object during the execution of the mounting special effect; and a file generation module 940, which is configured to generate a special effect file corresponding to the mounting special effect based on the plurality of reference key points.
  • an interface presentation module 910 which is configured to present an editing interface for producing a mounting special effect
  • a position marking module 920 which is configured to generate a device 240 to determine the expected mounting position of the virtual object to be mounted based on the input information received in the editing interface
  • a key point marking module 930 which is configured to determine a plurality of reference key
  • the editing interface presents a reference model and a plurality of key points corresponding to the reference model
  • the key point marking module 930 is further configured to: determine a plurality of reference key points whose distance to the expected mounting position is less than a threshold distance from the plurality of key points.
  • the threshold distance is determined based on user input.
  • the interface presenting module 910 is further configured to: highlight and present multiple reference key points among the multiple key points in the editing interface.
  • the file generation module 940 is further configured to: determine multiple distances from the expected mounting position to multiple reference key points; and generate distance description information based on the multiple distances, wherein the distance description information is used to indicate the relationship between the multiple distances, and the distance description information is used to determine the target mounting position of the virtual object during the execution of the mounting special effect.
  • FIG10 shows a block diagram of an electronic device 1000 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 1000 shown in FIG10 is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. The electronic device 1000 shown in FIG10 may be used to implement the electronic device 110 of FIG1 .
  • the electronic device 1000 is in the form of a general electronic device.
  • the components of the electronic device 1000 may include, but are not limited to, one or more processors or processing units 1010, a memory 1020, a storage device 1030, one or more communication units 1040, one or more input devices 1050, and one or more output devices 1060.
  • the processing unit 1010 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 1020. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 1000.
  • Electronic device 1000 typically includes a plurality of computer storage media. Such media may Any removable medium accessible to the electronic device 1000, including but not limited to volatile and non-volatile media, removable and non-removable media.
  • the memory 1020 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof.
  • the storage device 1030 can be a removable or non-removable medium, and can include a machine-readable medium, such as a flash drive, a disk, or any other medium, which can be used to store information and/or data (e.g., training data for training) and can be accessed within the electronic device 1000.
  • a machine-readable medium such as a flash drive, a disk, or any other medium, which can be used to store information and/or data (e.g., training data for training) and can be accessed within the electronic device 1000.
  • the electronic device 1000 may further include additional removable/non-removable, volatile/non-volatile storage media.
  • a disk drive for reading or writing from a removable, non-volatile disk e.g., a “floppy disk”
  • an optical drive for reading or writing from a removable, non-volatile optical disk may be provided.
  • each drive may be connected to a bus (not shown) by one or more data media interfaces.
  • the memory 1020 may include a computer program product 1025 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.
  • the communication unit 1040 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 1000 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.
  • PC network personal computer
  • the input device 1050 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc.
  • the output device 1060 may be one or more output devices, such as a display, a speaker, a printer, etc.
  • the electronic device 1000 may also communicate with one or more external devices (not shown) through the communication unit 1040 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 1000, or communicate with any device that allows the electronic device 1000 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input/output (I/O) interface (not shown).
  • I/O input/output
  • a computer-readable storage medium on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above.
  • a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.
  • These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions/actions specified in one or more boxes in the flowchart and/or block diagram is generated.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and/or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
  • Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
  • each box in the flowchart or block diagram may represent a module, a program segment, or a portion of an instruction that contains one or more executable instructions for implementing a specified logical function.
  • the functions labeled in the box may be
  • the blocks in the flowchart and/or the flowchart may also occur in a different order than that indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the opposite order, depending on the functions involved.
  • each block in the block diagram and/or flowchart, and combinations of blocks in the block diagram and/or flowchart may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本公开的实施例涉及用于交互和特效制作的方法、装置、设备和存储介质。在此提出的交互方法包括：从目标图像中检测与待挂载的虚拟对象相关联的多个参考关键点；基于多个参考关键点在目标图像中的位置信息，确定虚拟对象的挂载位置；以及将虚拟对象挂载到目标图像中的挂载位置处。基于以上方式，本公开的实施例能够实现虚拟对象的更为准确的挂载，提高用户的交互体验。

Description

用于交互和制作特效的方法、装置、设备和存储介质
本申请要求2023年07月03日递交的申请号为202310807910.X、标题为“用于交互和制作特效的方法、装置、设备和存储介质”的中国发明专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开的示例实施例总体涉及计算机领域，特别地涉及用于交互和制作特效的方法、装置、设备和计算机可读存储介质。
背景技术
随着计算机技术的发展，互联网已经成为创作和分享媒体内容的重要平台。在媒体内容的创作过程中，视频特效和图片特效是用户在创作过程中经常使用的创作手段。这样的特效能够丰富用户所创作的媒体内容，提高内容的趣味程度。
发明内容
在本公开的第一方面，提供了一种交互方法。该方法包括：从目标图像中检测与待挂载的虚拟对象相关联的多个参考关键点；基于多个参考关键点在目标图像中的位置信息，确定虚拟对象的挂载位置；以及将虚拟对象挂载到目标图像中的挂载位置处。
在本公开的第二方面，提供了一种制作特效的方法。该方法包括：呈现用于制作挂载特效的编辑界面；基于编辑界面中接收到的输入信息，确定待挂载的虚拟对象的预期挂载位置；确定与预期挂载位置相关联的多个参考关键点，多个参考关键点用于确定与虚拟对象在挂载特效执行期间的目标挂载位置；以及基于多个参考关键点生成与挂载 特效对应的特效文件。
在本公开的第三方面，提供了一种交互装置。该装置包括：关键点确定模块，被配置为从目标图像中检测与待挂载的虚拟对象相关联的多个参考关键点；位置确定模块，被配置为基于多个参考关键点在目标图像中的位置信息，确定虚拟对象的挂载位置；以及挂载模块，被配置为将虚拟对象挂载到目标图像中的挂载位置处。
在本公开的第四方面，提供了一种用于制作特效的装置。该装置包括：界面呈现模块，被配置为呈现用于制作挂载特效的编辑界面；位置标记模块，被配置为基于编辑界面中接收到的输入信息，确定待挂载的虚拟对象的预期挂载位置；关键点标记模块，被配置为确定与预期挂载位置相关联的多个参考关键点，多个参考关键点用于确定与虚拟对象在挂载特效执行期间的目标挂载位置；以及文件生成模块，被配置为基于多个参考关键点生成与挂载特效对应的特效文件。
在本公开的第五方面，提供了一种电子设备。该设备包括至少一个处理单元；以及至少一个存储器，至少一个存储器被耦合到至少一个处理单元并且存储用于由至少一个处理单元执行的指令。指令在由至少一个处理单元执行时使设备执行第一方面和/或第二方面的方法。
在本公开的第六方面，提供了一种计算机可读存储介质。该计算机可读存储介质上存储有计算机程序，计算机程序可由处理器执行以实现第一方面和/或第二方面的方法。
应当理解，本内容部分中所描述的内容并非旨在限定本公开的实施例的关键特征或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的描述而变得容易理解。
附图说明
结合附图并参考以下详细说明，本公开各实施例的上述和其他特征、优点及方面将变得更加明显。在附图中，相同或相似的附图标记表示相同或相似的元素，其中：
图1A和图1B示出了传统的挂载虚拟对象的示意图；
图2示出了其中可以实施根据本公开的实施例的示例环境的示意图；
图3A至图3D示出了根据本公开的一些实施例的挂载虚拟对象的示意图；
图4示出了根据本公开的另一些实施例的挂载虚拟对象的示意图；
图5A至图5C示出了根据本公开的一些实施例的制作挂载特效的示意图；
图6示出了根据本公开的一些实施例的示例交互过程的流程图；
图7示出了根据本公开的一些实施例中的用于制作特效的示例过程的流程图；
图8示出了根据本公开的一些实施例的交互装置的示意性结构框图；
图9示出了根据本公开的一些实施例的用于制作特效的装置的示意性结构框图；以及
图10示出了能够实施本公开的多个实施例的电子设备的框图。
具体实施方式
下面将参照附图更详细地描述本公开的实施例。虽然附图中示出了本公开的某些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反，提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实施例仅用于示例性作用，并非用于限制本公开的保护范围。
需要注意的是，本文中所提供的任何节/子节的标题并不是限制性的。本文通篇描述了各种实施例，并且任何类型的实施例都可以包括在任何节/子节下。此外，在任一节/子节中描述的实施例可以以任何方式与同一节/子节和/或不同节/子节中描述的任何其他实施例相结合。
在本公开的实施例的描述中，术语“包括”及其类似用语应当理 解为开放性包含，即“包括但不限于”。术语“基于”应当理解为“至少部分地基于”。术语“一个实施例”或“该实施例”应当理解为“至少一个实施例”。术语“一些实施例”应当理解为“至少一些实施例”。下文还可能包括其他明确的和隐含的定义。术语“第一”、“第二”等可以指代不同的或相同的对象。下文还可能包括其他明确的和隐含的定义。
本公开的实施例中可能涉及用户的数据、数据的获取和/或使用等。这些方面均遵循相应的法律法规及相关规定。在本公开的实施例中，所有数据的采集、获取、处理、加工、转发、使用等，都是在用户知晓并且确认的前提下进行的。相应地，在实现本公开的各实施例时，均应根据相关法律法规通过适当的方式，将可能所涉及的数据或信息的类型、使用范围、使用场景等告知用户并获得用户的授权。具体的告知和/或授权方式可以根据实际情况和应用场景而变化，本公开的范围在此方面不受限制。
本说明书及实施例中方案，如涉及个人信息处理，则均会在具备合法性基础(例如征得个人信息主体同意，或者为履行合同所必需等)的前提下进行处理，且仅会在规定或者约定的范围内进行处理。用户拒绝处理基本功能所需必要信息以外的个人信息，不会影响用户使用基本功能。
一些特效能够支持将一些虚拟对象挂载在图像中的预定部位。如图1A和图1B所示，这样的示例特效例如可以支持将虚拟对象110挂载到图像105的预定位置处。
然而，传统的挂载特效依赖于图像中所识别的特定关键点(也称为特征点)。然而，以图1A所示出的人体对象作为示例，人体对象中所能够识别出来的关键点位置可能是受限的。例如，人体的胸部位置可能没有匹配的关键点。这导致人们可能无法将虚拟对象有效地挂载到人体的胸部位置。
此外，在基于特定关键点的挂载方案中，虚拟对象的朝向信息通常也仅依赖于关键点的朝向。这导致所挂载的虚拟对象的朝向可能是 不恰当的，例如，与人体的姿态不符。
由此，这样的挂载特效难以满足人们的交互需求。
一方面，本公开的实施例提出了一种交互方案。根据该方案，可以从目标图像中检测与待挂载的虚拟对象相关联的多个参考关键点。进一步地，可以基于多个参考关键点在目标图像中的位置信息，确定虚拟对象的挂载位置。相应地，可以将虚拟对象挂载到目标图像中的挂载位置处。
基于以上方式，本公开的实施例能够实现虚拟对象的更为准确的挂载，提高用户的交互体验。此外，本公开的实施例还能够丰富支持在没有关键点(例如，胸部等部位)位置处的虚拟对象挂载，从而提升交互体验。
另一方面，本公开的实施例提出了一种特效制作方案。根据该方案，可以呈现用于制作挂载特效的编辑界面。此外，可以基于编辑界面中接收到的输入信息，确定待挂载的虚拟对象的预期挂载位置。进一步地，可以确定与预期挂载位置相关联的多个参考关键点，多个参考关键点用于确定与虚拟对象在挂载特效执行期间的目标挂载位置。相应地，可以基于多个参考关键点生成与挂载特效对应的特效文件。
基于以上方式，本公开的实施例能够支持用户创作挂载在非关键点位置处的挂载特效，从而提升了用户创作挂载特效的灵活性。
以下进一步结合附图来详细描述该方案的各种示例实现。
示例环境
图2示出了本公开的实施例能够在其中实现的示例环境200的示意图。如图2所示，示例环境200可以包括交互设备210。如下文将详细介绍地，交互设备210可以将挂载特效应用于目标图像220，以提供具有挂载特效的媒体内容230。
在一些实施例中，这样的目标图像220例如可以包括利用图像采集设备所获取的静态图像(例如，照片)或动态图像(例如，视频)。示例性地，用户例如可以利用交互设备210所搭载的图像采集设备来 拍摄图片或视频，并可以将挂载特效应用于这样的图片或视频上。
在又一些实施例中，这样的目标图像220例如还可以包括已生成的图像文件(例如，图片文件或视频文件)。用户例如可以选择交互设备210本地已有的图像文件，以应用挂载特效。或者，用户例如还可以向交互设备210上传待处理的图像文件，以应用挂载特效。
在另一些实施例中，这样的目标图像220例如还可以是基于经生成的图像文件所生成的、具有目标风格的静态图像或动态图像。例如，用户可以向交互设备210上传本地的图像或视频，并由交互设备210和/或其它适当的电子设备利用风格转换工具，以生成与生成的图像或视频对应的另一图像或视频。例如，用户上传的可以是写实照片，所生成的图像例如可以是卡通风格的图片。本空开不旨在对图像的转换过程进行限定。
在一些实施例中，交互设备210可以是任意类型的移动终端、固定终端或便携式终端，包括移动手机、台式计算机、膝上型计算机、笔记本计算机、上网本计算机、平板计算机、媒体计算机、多媒体平板、掌上电脑、便携式游戏终端、VR/AR设备、个人通信系统(Personal Communication System，PCS)设备、个人导航设备、个人数字助理(Personal Digital Assistant，PDA)、音频/视频播放器、数码相机/摄像机、定位设备、电视接收器、无线电广播接收器、电子书设备、游戏设备或者前述各项的任意组合，包括这些设备的配件和外设或者其任意组合。在一些实施例中，电子设备110也能够支持任意类型的针对用户的接口(诸如“可佩戴”电路等)。
在又一些示例中，交互设备210例如还可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络、以及大数据和人工智能平台等基础云计算服务的云服务器。
在一些实施例中，交互设备210还可以基于所发布的特效文件250来将挂载特效应用于目标图像220。如图2所示，如下文将详细介绍 地，这样的特效文件例如可以是由生成设备240所生成。
应当理解，虽然生成设备240和交互设备210在图2中被示出为分离的框，生成设备240和交互设备210可以是相同或不同的电子设备。
应当理解，仅出于示例性的目的描述环境200中各个元素的结构和功能，而不暗示对于本公开的范围的任何限制。
以下将继续参考附图描述本公开的一些示例实施例。
示例挂载特效
以下将参考图3A至图3D来描述根据本公开的实施例的特效交互过程。
图3A示出了根据本公开的一些实施例的交互过程的示意图。如图3A所示，交互设备210可以获取目标图像220，并可以从目标图像220中检测多个关键点，例如，关键点305-1至关键点305-10(单独或统一称为关键点305)。
应当理解，根据目标图像220中所包括的对象不同，关键点305也可以具有不同的类型。以目标图像220包括人体对象作为示例，关键点305例如可以包括检测到的人体关键点(例如，骨骼关键点)。
应当理解，可以利用任何适当的关键点检测技术来识别目标图像220中的关键点305，其示例可以包括但不限于：Skeleton2d、avatar3d等等。
进一步地，交互设备210可以从多个关键点305中确定与待挂载的虚拟对象相关联的多个参考关键点。
在一些实施例中，这样的多个参考关键点可以是基于虚拟对象的预期挂载位置所确定的。如图3A所示，虚拟对象对应于预期挂载位置310。这样的预期挂载位置310与可检测的多个关键点305不重叠。
在一些实施例中，预期挂载位置310例如可以包括特定的点，或者也可以包括特定的区域。例如，预期挂载位置310可以指示人体对象的左胸区域。
进一步地，交互设备210可以从多个关键点305中确定到预期挂载位置310的距离小于阈值的多个关键点，以作为参考关键点。以图3A作为示例，所确定的参考关键点例如可以包括参考关键点305-4(肩膀点)、参考关键点305-2(颈部点)、参考关键点305-5(上臂点)和参考关键点305-6(脊柱点)。
备选地或附加地，多个参考关键点还可以包括制作特效文件的过程中所指定的。如下文将参考特效制作过程所详细介绍的，例如还可以在特效文件的生成过程中指定待检测的多个参考关键点。
作为另一实施例，交互设备210例如也可以进执行对多个期望的参考关键点的检测，而无需执行全部关键点的检测。
进一步地，交互设备210可以基于多个参考关键点来确定虚拟对象的挂载位置。
在一些实施例中，交互设备210可以基于多个参考关键点中的两个参考关键点来确定挂载位置。如图3B所示，交互设备210例如可以基于参考关键点305-6和参考关键点305-5来确定挂载位置。
作为一个示例，交互设备210例如可以基于参考关键点305-6和参考关键点305-5来构建第一参考线315-1，并将该第一参考线315-1的中点确定作为挂载位置。
作为另一示例，交互设备210例如还可以基于位置描述信息来将该参考线中的目标位置确定作为挂载位置。这样的位置描述信息例如可以指示待确定的目标位置到参考关键点305-6的第一距离与目标位置到参考关键点305-5的第二距离之间的关系。
例如，特效文件250中可以维护有位置描述信息，其例如可以指示待确定的目标位置到参考关键点305-6的距离与目标位置到参考关键点305-5的距离相等。在这种情况下，所确定的目标位置可以为该参考线的中点。
作为另一示例，位置描述信息例如可以指示待确定的目标位置到参考关键点305-6的距离与目标位置到参考关键点305-5的距离之间的比值(例如，2∶1)。由此，交互设备210可以根据该比值来将第 一参考线315-1上的目标位置确定作为挂载位置。
在一些实施例中，如下文将详细介绍的，这样的位置描述信息例如可以是基于特效制作过程中的交互所确定的。
在一些实施例中，交互设备210还可以基于更多数目的参考关键点来确定挂载位置。相应地，位置描述信息例如还可以指示各参考关键点对挂载位置的影响权重。进一步地，交互设备210可以基于多个参考关键点的加权插值运算来确定待挂载的目标位置。
在一些实施例中，交互设备210例如还可以基于多个参考关键点中的第三参考关键点(例如，参考关键点305-4)来确定待挂载的目标位置。
示例性地，交互设备210可以基于参考关键点305-4和目标方向信息来构建第二参考线315-2。在一些实施例中，参考关键点305-4可以是与预期挂载位置310最近的参考关键点。
在一些实施例中，用于构建第二参考关键点315-2的目标方向信息例如可以包括预设的第一方向信息。例如，考虑到人体上半身在目标图像220中大部分情况是竖直状态，第一方向信息可以包括目标图像220中的竖直方向。
作为另一实施例，用于构建第二参考关键点315-2的目标方向信息例如可以基于对目标图像220中目标对象(例如，人体)的姿态检测所确定的第二方向信息。例如，交互设备210可以分析目标图像220中人体脊柱的朝向，并将该朝向确定作为第二方向信息。
进一步的，交互设备210可以基于第一参考线315-1和第二参考线315-2的交点来确定虚拟对象的挂载位置320。相应地，交互设备210可以将虚拟对象挂载到目标图像220的挂载位置320处。
基于以上所讨论的方式，本公开的实施例能够通过多个参考关键点来确定虚拟对象的挂载位置，从而可以支持将虚拟对象挂载到关键点检测无法覆盖的区域，从而提高了特效交互的灵活性。
在一些实施例中，交互设备210还可以进一步基于多个参考关键点的位置信息来确定虚拟对象的目标朝向，并可以根据所确定的目标 朝向来完成虚拟对象的挂载。
具体地，交互设备210可以基于多个参考关键点来确定至少一个参考平面，其中每个参考平面基于多个参考关键点中的至少三个参考关键点所确定。例如，在图3C的示例中，交互设备210例如可以确定基于由参考关键点305-2、参考关键点305-6和参考关键点305-4所构成的三角形所在的第一平面325-1；并可以确定基于由参考关键点305-5、参考关键点305-6和参考关键点305-4所构成的三角形所在的第二平面325-2。
相应地，交互设备210可以确定第一平面325-1和第二平面325-2的朝向信息(也称为第一朝向信息)，并根据该朝向信息来确定虚拟对象的目标朝向。示例性地，交互设备210可以根据第一平面325-1和第二平面325-2的法向方向的均值来确定虚拟对象的目标朝向。
在一些实施例中，在确定虚拟对象的目标朝向时，交互设备210还可以考虑多个参考关键点中的目标参考关键点的朝向信息。这样的目标参考关键点例如可以到虚拟对象的预期挂载位置的距离小于预设距离的参考关键点，例如，距离最近的参考关键点305-4。
进一步地，交互设备210可以获取参考关键点305-4的朝向信息(也称为第二朝向信息)。应当理解，这样的朝向信息例如可以基于关键点识别过程而被确定。进一步地，交互设备210可以利用第一朝向信息和第二朝向信息来确定虚拟对象的目标朝向。
示例性地，交互设备210可以将参考关键点305-4的旋转四元数转换为方向向量，并将该方向向量同第一平面325-1的法向向量和第二平面的法向向量的均值进行归一化，最终确定虚拟对象的目标朝向。
基于这样的方式，本公开的实施例可以使得所挂载的虚拟对象能够更加匹配目标图像中目标对象(例如，人体)的姿态，从而提高特效的交互体验。
示例性地，如图3D所示，交互设备210可以基于所确定的挂载位置320和目标朝向来将虚拟对象330挂载到目标图像220上，从而完成挂载特效的交互。
能够看到，与图1B所示的挂载特效相比，本公开的实施例能够一方面支持对于无关键点覆盖区域的挂载，另一方面还能够提高虚拟对象的挂载真实性。由此，可以提高挂载特效的交互体验。
在又一些实施例中，对于预期挂载位置为胸部作为示例，交互设备210所获取的目标图像可能无法检测出全部参考关键点，从而导致挂载失效。
由此，在利用多个参考关键点确定挂载位置前，交互设备210例如可以从目标图像检测面部区域，也即，与目标图像中的人的脸部所对应的区域。进一步地，交互设备210可以确定面部区域在目标图像中的占比。以图4作为示例，交互设备210可以从目标图像405中检测面部区域410，并确定面部区域410在目标图像405中的占比。
进一步地，如果该占比小于等于阈值，则交互设备210可以确定能够检测出完整的参考关键点，并可以利用上文所讨论的过程来完成虚拟对象的挂载。
相反，如果该占比大于阈值，则交互设备210可以确定难以根据多个参考关键点来完成虚拟对象的挂载。在这种情况下，交互设备210例如可以将从目标图像405中所识别出的关键点中的预定参考关键点的位置直接确定作为虚拟对象的挂载位置。
例如，交互设备210可以将如图3A所示的参考关键点305-4确定作为挂载位置。这样的参考关键点例如可以到虚拟对象的预期挂载位置距离最近的关键点。
备选地，这样的参考关键点也可以是在特效文件的生成过程所指定的特定关键点。
在一些实施例中，交互设备210还可以直接将该参考关键点305-5的朝向信息作为虚拟对象415的目标朝向。
作为另一示例，为了保证挂载特效的呈现效果，在面部区域的占比大于阈值的情况下，交互设备210例如还可以调整虚拟对象415的尺寸，使得其能够完整地被呈现在目标图像405中。
以此方式，本公开的实施例可以进一步保证在一些特殊场景中的 成功挂载，提高挂载特效交互的稳定性和灵活性。
示例特效制作
以下将进一步描述用于制作如上文所讨论的挂载特效文件的过程。
图5A示出了用于制作挂载特效的编辑界面500A。这样的编辑界面500A例如可以由如图2所示的生成设备240所提供。
如图5A所示，编辑界面500A例如可以用于制作挂载特效的参考模型505。应当理解，这样的参考模型505例如可以用于表征待挂载到的目标对象的基本样式，本公开不旨在对参考模型505的具体形式进行限定。
进一步地，编辑设备500A例如还可以关联于参考模型505来呈现可以用于制作挂载特效的多个关键点510。以人体作为目标对象的示例，这样的多个关键点510可以包括多个预设的人体关键点。
进一步地，如图5B所示，生成设备240可以获取在编辑界面中所接收到的输入信息，并确定待挂载的虚拟对象的预期挂载位置515。
示例性地，用户可以通过点击操作来制定预期挂载位置515，或者可以通过移动预期挂载位置515的指示符来调整预期挂载位置。本公开不旨在对制定预期挂载位置515的具体交互形态进行限定。
进一步地，生成设备240可以基于预期挂载位置515来确定多个参考关键点。这样的多个参考关键点例如可以包括如图5B所示的参考关键点520-1、参考关键点520-2、参考关键点520-3和参考关键点520-4。
在一些实施例中，生成设备240可以将到预期挂载位置515的距离小于阈值距离的多个关键点确定作为参考关键点。附加地，这样的阈值距离可以是初始设置的距离，或者还可以是基于用户的输入所确定的距离。
例如，生成设备240可以在确定参考关键点的过程中通过图形样式来指示用于确定参考关键点的范围525。用户例如还可以通过拖拽 操作来调整范围525，从而新增或移除一个或多个参考关键点。
附加地，生成设备240还可以将实时确定的多个参考关键点(例如，参考关键点520-1至520-4)突出呈现，以例如区别与其他未被确定为参考关键点的关键点。
备选地，除调整范围525外，用户例如还可以通过对关键点的选择操作来新增或移除一个或多个参考关键点。
在确定了这样的多个参考关键点后，生成设备240可以基于如上文参考图3A至图3D所讨论的基于多个参考关键点的挂载逻辑来确定虚拟对象的挂载位置和/或目标朝向。本公开在此不再赘述。
在一些实施例中，生成设备240例如还可以在编辑界面中呈现与所确定的多个参考关键点对应的实时挂载效果。如图5C所示，生成设备240例如可以根据编辑界面中所确定的多个参考关键点来将预定的虚拟对象535挂载到样本图像530上。
应当理解，这样的虚拟对象535和/或样本图像530可以是用户在特效制作过程中所添加或上传的。
进一步地，在接收到关于多个参考关键点的确认后，生成设备240可以生成基于多个参考关键点生成与挂载特效对应的特效文件。
示例性地，这样的特效文件可以维护用于确定虚拟对象的挂载位置和/或目标朝向的多个参考关键点的标识，以用于在特效执行过程中检测多个参考关键点，以完成挂载特效。
在一些实施例中，如上文所讨论的，生成设备240例如还可以确定预期挂载位置515到多个参考关键点(例如，参考关键点520-1至参考关键点520-4)的多个距离，并可以基于多个距离来生成距离描述信息。该距离描述信息可以用于指示多个距离之间的关系，距离描述信息用于在挂载特效执行期间确定虚拟对象的目标挂载位置。
这样的距离描述信息例如可以指示多个参考关键点520-1至520-4对于待确定的挂载位置的影响权重，从而可以支持通过基于权重的插值来确定虚拟对象的挂载位置。
通过在特效文件中维护距离描述信息，本公开的实施例能够支持 在上文讨论的特效应用过程中根据多个参考关键点的位置来确定挂载位置。
示例过程
图6示出了根据本公开的一些实施例的示例交互过程600的流程图。过程600可以被实现在交互设备210处。下面参考图2来描述过程600。
如图6所示，在框610，交互设备210从目标图像中检测与待挂载的虚拟对象相关联的多个参考关键点。
在框620，交互设备210基于多个参考关键点在目标图像中的位置信息，确定虚拟对象的挂载位置。
在框630，交互设备210将虚拟对象挂载到目标图像中的挂载位置处。
在一些实施例中，其中挂载位置为目标挂载位置，并且从目标图像中检测与待挂载的虚拟对象相关联的多个参考关键点包括：基于虚拟对象的预期挂载位置，从目标图像中确定多个参考关键点，多个参考关键点到预期挂载位置的距离小于第一阈值。
在一些实施例中，多个参考关键点至少包括第一参考关键点和第二参考关键点，并且基于多个参考关键点在目标图像中的位置信息确定虚拟对象的挂载位置包括：基于第一参考关键点和第二参考关键点，确定第一参考线；以及基于第一参考线，确定虚拟对象的挂载位置。
在一些实施例中，基于第一参考线确定虚拟对象的挂载位置包括：根据预设的距离描述信息，确定第一参考线上的目标位置，以作为挂载位置，其中距离描述信息指示目标位置到第一参考关键点的第一距离与目标位置到第二参考关键点的第二距离之间的关系。
在一些实施例中，多个参考关键点还包括第三参考关键点，并且基于第一参考线确定虚拟对象的挂载位置包括：基于第三参考关键点和目标方向信息，确定第二参考线；以及基于第一参考线和第二参考线的交点，确定挂载位置。
在一些实施例中，第三参考关键点为多个参考关键点中到虚拟对象的预期挂载位置距离最近的参考关键点。
在一些实施例中，目标方向信息包括：预设的第一方向信息；或基于目标图像中目标对象的姿态所确定的第二方向信息。
在一些实施例中，将虚拟对象挂载到目标图像中的挂载位置处包括：基于多个参考关键点的位置信息，确定虚拟对象的目标朝向；以及根据目标朝向，将虚拟对象挂载到目标图像中的挂载位置处。
在一些实施例中，基于多个参考关键点的位置信息确定虚拟对象的目标朝向包括：基于多个参考关键点，确定至少一个参考平面，其中每个参考平面基于多个参考关键点中的至少三个参考关键点所确定；以及基于至少一个参考平面的第一朝向信息，确定虚拟对象的目标朝向。
在一些实施例中，基于至少一个参考平面的第一朝向信息确定虚拟对象的目标朝向包括：确定多个参考关键点中的目标参考关键点的第二朝向信息，目标参考关键点到虚拟对象的预期挂载位置的距离小于预设距离；以及基于第一朝向信息和第二朝向信息，确定虚拟对象的目标朝向。
在一些实施例中，基于多个参考关键点在目标图像中的位置信息确定虚拟对象的挂载位置包括：从目标图像检测面部区域；以及响应于面部区域在目标图像中的占比小于等于阈值，基于多个参考关键点在目标图像中的位置信息确定虚拟对象的挂载位置。
在一些实施例中，过程600还包括：响应于面部区域在目标图像中的占比大于阈值，将多个参考关键点中的预定参考关键点的位置确定作为虚拟对象的挂载位置。
在一些实施例中，过程600还包括：将多个参考关键点中的预定参考关键点的朝向确定作为虚拟对象的目标朝向。
在一些实施例中，预定参考关键点为多个参考关键点中到虚拟对象的预期挂载位置距离最近的参考关键点。
在一些实施例中，目标图像包括：由图像采集设备所采集的静态 图像或动态图像；经上传的图像文件；或基于经上传的图像文件所生成的、与目标风格对应的图像。
图7示出了根据本公开的一些实施例的用于制作特效的示例过程700的流程图。过程700可以被实现在生成设备240处。下面参考图2来描述过程700。
如图7所示，在框710，生成设备240呈现用于制作挂载特效的编辑界面。
在框720，生成设备240基于编辑界面中接收到的输入信息，确定待挂载的虚拟对象的预期挂载位置。
在框730，生成设备240确定与预期挂载位置相关联的多个参考关键点，多个参考关键点用于确定与虚拟对象在挂载特效执行期间的目标挂载位置。
在框740，生成设备240基于多个参考关键点生成与挂载特效对应的特效文件。
在一些实施例中，编辑界面呈现参考模型和与参考模型对应的多个关键点，并且确定与预期挂载位置相关联的多个参考关键点包括：从多个关键点中确定到预期挂载位置的距离小于阈值距离的多个参考关键点。
在一些实施例中，阈值距离是基于用户输入所确定的。
在一些实施例中，过程700还包括：在编辑界面中，突出呈现多个关键点中的多个参考关键点。
在一些实施例中，过程700还包括：确定预期挂载位置到多个参考关键点的多个距离；以及基于多个距离，生成距离描述信息，距离描述信息用于指示多个距离之间的关系，距离描述信息用于在挂载特效执行期间确定虚拟对象的目标挂载位置。
示例装置和设备
本公开的实施例还提供了用于实现上述方法或过程的相应装置。图8示出了根据本公开的某些实施例的示例交互装置800的示意性结 构框图。装置800可以被实现为或者被包括在交互设备210中。装置800中的各个模块/组件可以由硬件、软件、固件或者它们的任意组合来实现。
如图8所示，装置800包括关键点确定模块810，被配置为从目标图像中检测与待挂载的虚拟对象相关联的多个参考关键点；位置确定模块820，被配置为基于多个参考关键点在目标图像中的位置信息，确定虚拟对象的挂载位置；以及挂载模块830，被配置为将虚拟对象挂载到目标图像中的挂载位置处。
在一些实施例中，其中挂载位置为目标挂载位置，并且关键点确定模块810还被配置为：基于虚拟对象的预期挂载位置，从目标图像中确定多个参考关键点，多个参考关键点到预期挂载位置的距离小于第一阈值。
在一些实施例中，多个参考关键点至少包括第一参考关键点和第二参考关键点，并且位置确定模块820还被配置为：基于第一参考关键点和第二参考关键点，确定第一参考线；以及基于第一参考线，确定虚拟对象的挂载位置。
在一些实施例中，位置确定模块820还被配置为：根据预设的距离描述信息，确定第一参考线上的目标位置，以作为挂载位置，其中距离描述信息指示目标位置到第一参考关键点的第一距离与目标位置到第二参考关键点的第二距离之间的关系。
在一些实施例中，多个参考关键点还包括第三参考关键点，并且位置确定模块820还被配置为：基于第三参考关键点和目标方向信息，确定第二参考线；以及基于第一参考线和第二参考线的交点，确定挂载位置。
在一些实施例中，第三参考关键点为多个参考关键点中到虚拟对象的预期挂载位置距离最近的参考关键点。
在一些实施例中，目标方向信息包括：预设的第一方向信息；或基于目标图像中目标对象的姿态所确定的第二方向信息。
在一些实施例中，挂载模块830还被配置为：基于多个参考关键 点的位置信息，确定虚拟对象的目标朝向；以及根据目标朝向，将虚拟对象挂载到目标图像中的挂载位置处。
在一些实施例中，挂载模块830还被配置为：基于多个参考关键点，确定至少一个参考平面，其中每个参考平面基于多个参考关键点中的至少三个参考关键点所确定；以及基于至少一个参考平面的第一朝向信息，确定虚拟对象的目标朝向。
在一些实施例中，挂载模块830还被配置为：确定多个参考关键点中的目标参考关键点的第二朝向信息，目标参考关键点到虚拟对象的预期挂载位置的距离小于预设距离；以及基于第一朝向信息和第二朝向信息，确定虚拟对象的目标朝向。
在一些实施例中，位置确定模块820还被配置为：从目标图像检测面部区域；以及响应于面部区域在目标图像中的占比小于等于阈值，基于多个参考关键点在目标图像中的位置信息确定虚拟对象的挂载位置。
在一些实施例中，位置确定模块820还被配置为：响应于面部区域在目标图像中的占比大于阈值，将多个参考关键点中的预定参考关键点的位置确定作为虚拟对象的挂载位置。
在一些实施例中，挂载模块830还被配置为：将多个参考关键点中的预定参考关键点的朝向确定作为虚拟对象的目标朝向。
在一些实施例中，预定参考关键点为多个参考关键点中到虚拟对象的预期挂载位置距离最近的参考关键点。
在一些实施例中，目标图像包括：由图像采集设备所采集的静态图像或动态图像；经上传的图像文件；或基于经上传的图像文件所生成的、与目标风格对应的图像。
本公开的实施例还提供了用于实现上述方法或过程的相应装置。图9示出了根据本公开的某些实施例的用于制作特效的示例装置900的示意性结构框图。装置900可以被实现为或者被包括在生成设备240中。装置900中的各个模块/组件可以由硬件、软件、固件或者它们的任意组合来实现。
如图9所示，装置900包括界面呈现模块910，被配置为呈现用于制作挂载特效的编辑界面；位置标记模块920，被配置为生成设备240基于编辑界面中接收到的输入信息，确定待挂载的虚拟对象的预期挂载位置；关键点标记模块930，被配置为确定与预期挂载位置相关联的多个参考关键点，多个参考关键点用于确定与虚拟对象在挂载特效执行期间的目标挂载位置；以及文件生成模块940，被配置为基于多个参考关键点生成与挂载特效对应的特效文件。
在一些实施例中，编辑界面呈现参考模型和与参考模型对应的多个关键点，并且关键点标记模块930还被配置为：从多个关键点中确定到预期挂载位置的距离小于阈值距离的多个参考关键点。
在一些实施例中，阈值距离是基于用户输入所确定的。
在一些实施例中，界面呈现模块910还被配置为：在编辑界面中，突出呈现多个关键点中的多个参考关键点。
在一些实施例中，文件生成模块940还被配置为：确定预期挂载位置到多个参考关键点的多个距离；以及基于多个距离，生成距离描述信息，距离描述信息用于指示多个距离之间的关系，距离描述信息用于在挂载特效执行期间确定虚拟对象的目标挂载位置。
图10示出了其中可以实施本公开的一个或多个实施例的电子设备1000的框图。应当理解，图10所示出的电子设备1000仅仅是示例性的，而不应当构成对本文所描述的实施例的功能和范围的任何限制。图10所示出的电子设备1000可以用于实现图1的电子设备110。
如图10所示，电子设备1000是通用电子设备的形式。电子设备1000的组件可以包括但不限于一个或多个处理器或处理单元1010、存储器1020、存储设备1030、一个或多个通信单元1040、一个或多个输入设备1050以及一个或多个输出设备1060。处理单元1010可以是实际或虚拟处理器并且能够根据存储器1020中存储的程序来执行各种处理。在多处理器系统中，多个处理单元并行执行计算机可执行指令，以提高电子设备1000的并行处理能力。
电子设备1000通常包括多个计算机存储介质。这样的介质可以 是电子设备1000可访问的任何可以获取的介质，包括但不限于易失性和非易失性介质、可拆卸和不可拆卸介质。存储器1020可以是易失性存储器(例如寄存器、高速缓存、随机访问存储器(RAM))、非易失性存储器(例如，只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、闪存)或它们的某种组合。存储设备1030可以是可拆卸或不可拆卸的介质，并且可以包括机器可读介质，诸如闪存驱动、磁盘或者任何其他介质，其可以能够用于存储信息和/或数据(例如用于训练的训练数据)并且可以在电子设备1000内被访问。
电子设备1000可以进一步包括另外的可拆卸/不可拆卸、易失性/非易失性存储介质。尽管未在图10中示出，可以提供用于从可拆卸、非易失性磁盘(例如“软盘”)进行读取或写入的磁盘驱动和用于从可拆卸、非易失性光盘进行读取或写入的光盘驱动。在这些情况中，每个驱动可以由一个或多个数据介质接口被连接至总线(未示出)。存储器1020可以包括计算机程序产品1025，其具有一个或多个程序模块，这些程序模块被配置为执行本公开的各种实施例的各种方法或动作。
通信单元1040实现通过通信介质与其他电子设备进行通信。附加地，电子设备1000的组件的功能可以以单个计算集群或多个计算机器来实现，这些计算机器能够通过通信连接进行通信。因此，电子设备500可以使用与一个或多个其他服务器、网络个人计算机(PC)或者另一个网络节点的逻辑连接来在联网环境中进行操作。
输入设备1050可以是一个或多个输入设备，例如鼠标、键盘、追踪球等。输出设备1060可以是一个或多个输出设备，例如显示器、扬声器、打印机等。电子设备1000还可以根据需要通过通信单元1040与一个或多个外部设备(未示出)进行通信，外部设备诸如存储设备、显示设备等，与一个或多个使得用户与电子设备1000交互的设备进行通信，或者与使得电子设备1000与一个或多个其他电子设备通信的任何设备(例如，网卡、调制解调器等)进行通信。这样的通信可以经由输入/输出(I/O)接口(未示出)来执行。
根据本公开的示例性实现方式，提供了一种计算机可读存储介质，其上存储有计算机可执行指令，其中计算机可执行指令被处理器执行以实现上文描述的方法。根据本公开的示例性实现方式，还提供了一种计算机程序产品，计算机程序产品被有形地存储在非瞬态计算机可读介质上并且包括计算机可执行指令，而计算机可执行指令被处理器执行以实现上文描述的方法。
这里参照根据本公开实现的方法、装置、设备和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理单元，从而生产出一种机器，使得这些指令在通过计算机或其他可编程数据处理装置的处理单元执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
可以把计算机可读程序指令加载到计算机、其他可编程数据处理装置、或其他设备上，使得在计算机、其他可编程数据处理装置或其他设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其他可编程数据处理装置、或其他设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实现的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功 能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
以上已经描述了本公开的各实现，上述说明是示例性的，并非穷尽性的，并且也不限于所公开的各实现。在不偏离所说明的各实现的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实现的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其他普通技术人员能理解本文公开的各个实现方式。

Claims (20)

  1. 一种交互方法，包括：
    从目标图像中检测与待挂载的虚拟对象相关联的多个参考关键点；
    基于所述多个参考关键点在所述目标图像中的位置信息，确定所述虚拟对象的挂载位置；以及
    将所述虚拟对象挂载到所述目标图像中的所述挂载位置处。
  2. 根据权利要求1所述的方法，其中所述挂载位置为目标挂载位置，并且从目标图像中检测与待挂载的虚拟对象相关联的多个参考关键点包括：
    基于所述虚拟对象的预期挂载位置，从所述目标图像中确定所述多个参考关键点，所述多个参考关键点到所述预期挂载位置的距离小于第一阈值。
  3. 根据权利要求1所述的方法，其中所述多个参考关键点至少包括第一参考关键点和第二参考关键点，并且基于所述多个参考关键点在所述目标图像中的位置信息确定所述虚拟对象的挂载位置包括：
    基于所述第一参考关键点和所述第二参考关键点，确定第一参考线；以及
    基于所述第一参考线，确定所述虚拟对象的所述挂载位置。
  4. 根据权利要求3所述的方法，其中基于所述第一参考线确定所述虚拟对象的所述挂载位置包括：
    根据预设的距离描述信息，确定所述第一参考线上的目标位置，以作为所述挂载位置，其中所述距离描述信息指示所述目标位置到所述第一参考关键点的第一距离与所述目标位置到所述第二参考关键点的第二距离之间的关系。
  5. 根据权利要求3所述的方法，其中所述多个参考关键点还包括第三参考关键点，并且基于所述第一参考线确定所述虚拟对象的所述挂载位置包括：
    基于所述第三参考关键点和目标方向信息，确定第二参考线；以及
    基于所述第一参考线和所述第二参考线的交点，确定所述挂载位置。
  6. 根据权利要求5所述的方法，其中所述第三参考关键点为所述多个参考关键点中到所述虚拟对象的预期挂载位置距离最近的参考关键点。
  7. 根据权利要求5所述的方法，其中所述目标方向信息包括：
    预设的第一方向信息；或
    基于所述目标图像中目标对象的姿态所确定的第二方向信息。
  8. 根据权利要求1所述的方法，其中将所述虚拟对象挂载到所述目标图像中的所述挂载位置处包括：
    基于所述多个参考关键点的所述位置信息，确定所述虚拟对象的目标朝向；以及
    根据所述目标朝向，将所述虚拟对象挂载到所述目标图像中的所述挂载位置处。
  9. 根据权利要求8所述的方法，其中基于所述多个参考关键点的所述位置信息确定所述虚拟对象的目标朝向包括：
    基于所述多个参考关键点，确定至少一个参考平面，其中每个参考平面基于所述多个参考关键点中的至少三个参考关键点所确定；以及
    基于所述至少一个参考平面的第一朝向信息，确定所述虚拟对象的所述目标朝向。
  10. 根据权利要求9所述的方法，其中基于所述至少一个参考平面的第一朝向信息确定所述虚拟对象的所述目标朝向包括：
    确定所述多个参考关键点中的目标参考关键点的第二朝向信息，所述目标参考关键点到所述虚拟对象的预期挂载位置的距离小于预设距离；以及
    基于所述第一朝向信息和所述第二朝向信息，确定所述虚拟对象 的所述目标朝向。
  11. 根据权利要求1所述的方法，其中基于所述多个参考关键点在所述目标图像中的位置信息确定所述虚拟对象的挂载位置包括：
    从所述目标图像检测面部区域；以及
    响应于所述面部区域在所述目标图像中的占比小于等于阈值，基于所述多个参考关键点在所述目标图像中的位置信息确定所述虚拟对象的挂载位置。
  12. 根据权利要求11所述的方法，还包括：
    响应于所述面部区域在所述目标图像中的占比大于所述阈值，将所述多个参考关键点中的预定参考关键点的位置确定作为所述虚拟对象的所述挂载位置；以及
    将所述多个参考关键点中的所述预定参考关键点的朝向确定作为所述虚拟对象的目标朝向；其中所述预定参考关键点为所述多个参考关键点中到所述虚拟对象的预期挂载位置距离最近的参考关键点。
  13. 一种生成特效的方法，包括：
    呈现用于生成挂载特效的编辑界面；
    基于所述编辑界面中接收到的输入信息，确定待挂载的虚拟对象的预期挂载位置；
    确定与所述预期挂载位置相关联的一组参考关键点，所述一组参考关键点用于确定与所述虚拟对象在挂载特效执行期间的目标挂载位置；以及
    基于所述一组参考关键点生成与所述挂载特效对应的特效文件。
  14. 根据权利要求13所述的方法，其中所述编辑界面呈现参考模型和与所述参考模型对应的多个关键点，并且确定与所述预期挂载位置相关联的一组参考关键点包括：
    从所述多个关键点中确定到所述预期挂载位置的距离小于阈值距离的所述一组参考关键点，其中所述阈值距离是基于用户输入所确定的。
  15. 根据权利要求14所述的方法，还包括：
    在所述编辑界面中，突出呈现所述多个关键点中的所述一组参考关键点。
  16. 根据权利要求13所述的方法，还包括：
    确定所述预期挂载位置到所述一组参考关键点的一组距离；以及
    基于所述一组距离，生成距离描述信息，以指示所述一组距离之间的关系，所述距离描述信息用于在所述挂载特效执行期间确定所述虚拟对象的所述目标挂载位置。
  17. 一种交互装置，包括：
    关键点确定模块，被配置为从目标图像中检测与待挂载的虚拟对象相关联的多个参考关键点；
    位置确定模块，被配置为基于所述多个参考关键点在所述目标图像中的位置信息，确定所述虚拟对象的挂载位置；以及
    挂载模块，被配置为将所述虚拟对象挂载到所述目标图像中的所述挂载位置处。
  18. 一种用于生成特效的装置，包括：
    界面呈现模块，被配置为呈现用于生成挂载特效的编辑界面；
    位置标记模块，被配置为基于所述编辑界面中接收到的输入信息，确定待挂载的虚拟对象的预期挂载位置；
    关键点标记模块，被配置为确定与所述预期挂载位置相关联的一组参考关键点，所述一组参考关键点用于确定与所述虚拟对象在挂载特效执行期间的目标挂载位置；以及
    特效生成模块，被配置为基于所述一组参考关键点生成与所述挂载特效对应的特效文件。
  19. 一种电子设备，包括：
    至少一个处理单元；以及
    至少一个存储器，所述至少一个存储器被耦合到所述至少一个处理单元并且存储用于由所述至少一个处理单元执行的指令，所述指令在由所述至少一个处理单元执行时使所述电子设备执行根据权利要求1至12或13至16中任一项所述的方法。
  20. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序可由处理器执行以实现根据权利要求1至12或13至16中任一项所述的方法。
PCT/CN2024/102777 2023-07-03 2024-06-30 用于交互和制作特效的方法、装置、设备和存储介质 Ceased WO2025007829A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310807910.X 2023-07-03
CN202310807910.XA CN119273534A (zh) 2023-07-03 2023-07-03 用于交互和制作特效的方法、装置、设备和存储介质

Publications (1)

Publication Number Publication Date
WO2025007829A1 true WO2025007829A1 (zh) 2025-01-09

Family

ID=94106298

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/102777 Ceased WO2025007829A1 (zh) 2023-07-03 2024-06-30 用于交互和制作特效的方法、装置、设备和存储介质

Country Status (2)

Country Link
CN (1) CN119273534A (zh)
WO (1) WO2025007829A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110072046A (zh) * 2018-08-24 2019-07-30 北京微播视界科技有限公司 图像合成方法和装置
CN110941327A (zh) * 2018-09-21 2020-03-31 北京微播视界科技有限公司 虚拟对象的显示方法和装置
CN112348841A (zh) * 2020-10-27 2021-02-09 北京达佳互联信息技术有限公司 虚拟对象的处理方法、装置、电子设备及存储介质
CN114419299A (zh) * 2022-01-21 2022-04-29 北京字跳网络技术有限公司 虚拟物体的生成方法、装置、设备及存储介质
KR20230048821A (ko) * 2021-10-05 2023-04-12 가온미디어 주식회사 클론객체를 이용하는 ar 콘텐츠 렌더링 처리 방법
CN116228952A (zh) * 2023-02-24 2023-06-06 北京字跳网络技术有限公司 一种虚拟对象挂载方法、装置、设备及介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110072046A (zh) * 2018-08-24 2019-07-30 北京微播视界科技有限公司 图像合成方法和装置
CN110941327A (zh) * 2018-09-21 2020-03-31 北京微播视界科技有限公司 虚拟对象的显示方法和装置
CN112348841A (zh) * 2020-10-27 2021-02-09 北京达佳互联信息技术有限公司 虚拟对象的处理方法、装置、电子设备及存储介质
KR20230048821A (ko) * 2021-10-05 2023-04-12 가온미디어 주식회사 클론객체를 이용하는 ar 콘텐츠 렌더링 처리 방법
CN114419299A (zh) * 2022-01-21 2022-04-29 北京字跳网络技术有限公司 虚拟物体的生成方法、装置、设备及存储介质
CN116228952A (zh) * 2023-02-24 2023-06-06 北京字跳网络技术有限公司 一种虚拟对象挂载方法、装置、设备及介质

Also Published As

Publication number Publication date
CN119273534A (zh) 2025-01-07

Similar Documents

Publication Publication Date Title
CN110517214B (zh) 用于生成图像的方法和装置
WO2015200782A1 (en) 3-d model generation
CN110956131B (zh) 单目标追踪方法、装置及系统
WO2025139966A1 (zh) 交互方法、装置、设备和存储介质
CN115209031B (zh) 视频防抖处理方法、装置、电子设备和存储介质
CN113284237A (zh) 一种三维重建方法、系统、电子设备及存储介质
TWI711004B (zh) 圖片處理方法和裝置
CN113705379A (zh) 一种手势估计方法、装置、存储介质及设备
CN108875479A (zh) 人脸图像的获取方法及装置
US10600202B2 (en) Information processing device and method, and program
JP6086491B2 (ja) 画像処理装置およびそのデータベース構築装置
CN111563490B (zh) 一种人脸关键点跟踪方法、装置及电子设备
CN118368491A (zh) 设置封面的方法、装置、设备和存储介质
CN113723146B (zh) 人脸关键点检测模型的训练方法及装置
WO2021159672A1 (zh) 一种人脸图像的识别方法及设备
WO2026026818A1 (zh) 生成虚拟资源的方法、装置、设备和存储介质
WO2025007829A1 (zh) 用于交互和制作特效的方法、装置、设备和存储介质
CN118158340B (zh) 一种运镜控制方法、装置、设备和存储介质
US10652472B2 (en) Enhanced automatic perspective and horizon correction
Chiu et al. Cloud computing based mobile augmented reality interactive system
CN116206049A (zh) 三维场景重建方法、装置和存储介质
Schmid et al. ScreenshotMatcher: Taking Smartphone Photos to Capture Screenshots
CN115937964B (zh) 姿态估计的方法、装置、设备和存储介质
US12608879B2 (en) Generation of a 360-degree object view by leveraging available images on an online platform
CN111309206A (zh) 数据处理方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24835310

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024835310

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2024835310

Country of ref document: EP

Effective date: 20260203