WO2020034786A1 - 三维模型处理方法、装置、电子设备和存储介质 - Google Patents

三维模型处理方法、装置、电子设备和存储介质 Download PDF

Info

Publication number
WO2020034786A1
WO2020034786A1 PCT/CN2019/095287 CN2019095287W WO2020034786A1 WO 2020034786 A1 WO2020034786 A1 WO 2020034786A1 CN 2019095287 W CN2019095287 W CN 2019095287W WO 2020034786 A1 WO2020034786 A1 WO 2020034786A1
Authority
WO
WIPO (PCT)
Prior art keywords
key
area
target
dimensional model
target object
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/095287
Other languages
English (en)
French (fr)
Inventor
阎法典
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Publication of WO2020034786A1 publication Critical patent/WO2020034786A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00Three-dimensional [3D] image rendering
    • G06T15/005General purpose rendering architectures
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/04Context-preserving transformations, e.g. by using an importance map
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/77Retouching; Inpainting; Scratch removal

Definitions

  • the present disclosure relates to the technical field of electronic devices, and in particular, to a method, a device, an electronic device, and a storage medium for processing a three-dimensional model.
  • the present disclosure proposes a three-dimensional model processing method, device, electronic device, and storage medium, which are used to solve the problems of obtaining shaping parameters based on a two-dimensional face image in related technologies, which are complicated and inflexible, and the shaping effect is not real.
  • An embodiment of one aspect of the present disclosure provides a three-dimensional model processing method, including:
  • the three-dimensional model includes a plurality of key points and position information of the plurality of key points;
  • a proportional relationship between multiple key regions of the target object and a position relationship between multiple key points in each key region are determined, where each Each key region is obtained by concatenating multiple key points as vertices;
  • the model adjustment parameters corresponding to the target object are determined according to the proportional relationship between the multiple key regions and the positional relationship between multiple key points in each key region.
  • the three-dimensional model processing method of the embodiment of the present disclosure obtains a three-dimensional model of a target object, where the three-dimensional model includes multiple key points and position information of multiple key points, and according to the positions of the multiple key points in the three-dimensional model of the target object Information to determine the proportional relationship between multiple key areas of the target object, and the positional relationship between multiple key points in each key area, based on the proportional relationship between multiple key areas and between multiple key points in each key area Position relationship, determine the model adjustment parameters corresponding to the target object.
  • the adjustment parameters corresponding to the target object can be automatically determined without the user having to repeatedly adjust the amplitude of each parameter to obtain Appropriate adjustment parameters are flexible and convenient, and the shaping effect based on the three-dimensional model is realistic.
  • An embodiment of another aspect of the present disclosure provides a three-dimensional model processing apparatus, including:
  • An acquisition module for acquiring a three-dimensional model of a target object, wherein the three-dimensional model includes a plurality of key points and position information of the plurality of key points;
  • a first determining module configured to determine a proportional relationship between a plurality of key regions of the target object and a plurality of key points in each key region according to position information of a plurality of key points in the three-dimensional model of the target object; Position relationship, where each key region is obtained by stitching multiple adjacent key points as vertices;
  • a second determining module is configured to determine a model adjustment parameter corresponding to the target object according to a proportional relationship between the multiple key areas and a position relationship between multiple key points in each key area.
  • the three-dimensional model processing device of the embodiment of the present disclosure obtains a three-dimensional model of a target object, where the three-dimensional model includes multiple key points and position information of multiple key points, and according to the positions of the multiple key points in the three-dimensional model of the target object Information to determine the proportional relationship between multiple key areas of the target object, and the positional relationship between multiple key points in each key area, based on the proportional relationship between multiple key areas and between multiple key points in each key area Position relationship, determine the model adjustment parameters corresponding to the target object.
  • the adjustment parameters corresponding to the target object can be automatically determined without the user having to repeatedly adjust the amplitude of each parameter to obtain Appropriate adjustment parameters are flexible and convenient, and the shaping effect based on the three-dimensional model is realistic.
  • An embodiment of another aspect of the present disclosure provides an electronic device including a processor and a memory
  • the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the three-dimensional model processing method according to the embodiment of the above aspect.
  • Another embodiment of the present disclosure provides a non-transitory computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the three-dimensional model processing method according to the foregoing embodiment is implemented.
  • FIG. 1 is a schematic flowchart of a three-dimensional model processing method according to an embodiment of the present disclosure
  • FIG. 2 is a schematic frame diagram 1 of a three-dimensional model of a human face in an embodiment of the present disclosure
  • FIG. 3 is a second schematic frame diagram of a three-dimensional model of a human face in an embodiment of the present disclosure
  • FIG. 4 is a schematic flowchart of another three-dimensional model processing method according to an embodiment of the present disclosure.
  • FIG. 5 is a schematic flowchart of another three-dimensional model processing method according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic structural diagram of a three-dimensional model processing apparatus according to an embodiment of the present disclosure.
  • FIG. 7 illustrates a block diagram of an exemplary electronic device suitable for use in implementing embodiments of the present disclosure.
  • the embodiments of the present disclosure address a problem that the method for obtaining the shaping parameters in the related art is cumbersome, inflexible, and the shaping effect is not real, and a three-dimensional model processing method is proposed.
  • the three-dimensional model processing method of the embodiment of the present disclosure first obtains a three-dimensional model of a target object, where the three-dimensional model includes multiple key points and position information of multiple key points, and then according to the multiple key points in the three-dimensional model of the target object, Position information to determine the proportional relationship between multiple key areas of the target object, and the positional relationship between multiple key points in each key area, and then based on the proportional relationship between multiple key areas and multiple keys in each key area The positional relationship between the points determines the model adjustment parameters corresponding to the target object.
  • the adjustment parameters corresponding to the target object can be automatically determined without the user having to repeatedly adjust the amplitude of each parameter to obtain Appropriate adjustment parameters are flexible and convenient, and the shaping effect based on the three-dimensional model is realistic.
  • FIG. 1 is a schematic flowchart of a three-dimensional model processing method according to an embodiment of the present disclosure.
  • the three-dimensional model processing method provided by the embodiment of the present disclosure may be executed by the three-dimensional model processing device provided by the present disclosure.
  • the above device is configured in an electronic device to achieve a proportional relationship between a plurality of key regions of a target object and each key region. The positional relationship between multiple key points automatically determines the model adjustment parameters corresponding to the target object.
  • the electronic device in this embodiment may be any hardware device having a photographing function, such as a mobile phone, a palm computer, a wearable device, and the like.
  • the three-dimensional model processing method includes:
  • Step 101 Obtain a three-dimensional model of a target object, where the three-dimensional model includes multiple key points and position information of multiple key points.
  • the object in the target object may refer to a person or a face.
  • the three-dimensional model of the target object refers to the three-dimensional model of the face of the target object.
  • the electronic device may include a visible light image sensor, and a two-dimensional face image may be acquired based on the visible light image sensor in the electronic device.
  • the visible light image sensor may include a visible light camera, and the visible light camera may capture visible light reflected by a human face for imaging to obtain a two-dimensional face image.
  • the electronic device may further include a structured light image sensor, and based on the structured light image sensor in the electronic device, depth information corresponding to a human face may be acquired.
  • the structured light image sensor may include a laser light and a laser camera.
  • Pulse Width Modulation PWM
  • a laser camera can capture structured light reflected from a human face for imaging, and obtain a structured light image corresponding to the human face.
  • the depth engine may calculate the depth information corresponding to the face according to the structured light image corresponding to the face, that is, the depth information corresponding to the two-dimensional face image.
  • the two-dimensional face image corresponding to the target object and the depth information corresponding to the two-dimensional face image are pre-stored in the electronic device, so that the two-dimensional face image of the target object and the depth corresponding to the two-dimensional face image can be directly obtained.
  • Information and then use the pre-stored two-dimensional face image of the target object and the depth information corresponding to the two-dimensional face image to construct a three-dimensional model of the target object.
  • At least two two-dimensional face images of the target object and depth information corresponding to the at least two two-dimensional face images may be obtained from at least two angles.
  • a three-dimensional model of the target object can be constructed according to the two-dimensional face images and the corresponding depth information.
  • the construction of the three-dimensional model of the target object is obtained by performing three-dimensional reconstruction based on the depth information and the face image, instead of simply acquiring RGB data and depth data.
  • the depth information and the color information corresponding to the two-dimensional face image can be fused to obtain a three-dimensional model of the target object.
  • the key points of the image are processed for registration and key point fusion.
  • a three-dimensional model of the target object is generated based on the key points after fusion.
  • the key point is a conspicuous point on a human face or a point at a key position, for example, the key point may be a corner of an eye, a tip of a nose, a corner of a mouth, and the like.
  • keypoint recognition can be performed on each two-dimensional face image to obtain each third keypoint, and then in each two-dimensional face image according to each third keypoint And the depth information of each third key point to determine the relative position of each second key point in the three-dimensional space corresponding to each third key point, so that the relative position of each second key point in the three-dimensional space can be determined, A local three-dimensional frame stitched from adjacent second key points is generated.
  • different local three-dimensional frames can be stitched according to the overlapping parts in the local three-dimensional frames to obtain a three-dimensional model of the target object. Specifically, according to the same second key point contained in each local three-dimensional frame, the local three-dimensional frames can be stitched to obtain a three-dimensional model of the target object.
  • the forehead three-dimensional frame and the eye three-dimensional frame are spliced, and according to the same second key point included in each local three-dimensional frame, each local three-dimensional Frames can be stitched to obtain a three-dimensional model of the face.
  • the three-dimensional model includes multiple key points and position information of the multiple key points. For example, multiple key points on the nose, multiple key points on the orbit, and so on.
  • FIG. 2 and FIG. 3 are schematic diagrams of a three-dimensional model of a target object corresponding to two two-dimensional face images obtained from different angles, respectively.
  • step 102 according to the position information of multiple key points in the three-dimensional model of the target object, a proportional relationship between multiple key regions of the target object and a position relationship between multiple key points in each key region are determined.
  • a plurality of adjacent key points can be used as vertices to obtain multiple key regions. For example, after stitching adjacent key points, key areas such as nose, eyes, and mouth are obtained.
  • the key region is obtained by splicing adjacent key points as vertices, it can be understood that the key points in the area surrounded by the adjacent key points are the key points contained in the key region. Therefore, according to the position information of multiple key points in each key area, the position relationship between the multiple key points in each key area can be determined.
  • the positional relationship between the multiple key points in the eye region can be determined, for example, the difference between the two key points of the eye corners is 0.1 mm.
  • the proportional relationship of each key region in the three-dimensional model of the target object can be determined, and thus the proportional relationship between the multiple key regions of the target object can be determined.
  • the proportion of each part in the face can be obtained.
  • the three-dimensional degree of each key area can be determined. For example, according to the positional relationship between the key point corresponding to the nose tip and the key point corresponding to the root of the nose in the nose area, the three-dimensional degree of the nose can be determined.
  • Step 103 Determine model adjustment parameters corresponding to the target object according to a proportional relationship between multiple key areas and a position relationship between multiple key points in each key area.
  • a key region that needs to be adjusted may be determined. After determining the key area that needs to be adjusted, according to the positional relationship between multiple key points in the key area, the model adjustment parameters corresponding to the target object are determined.
  • a key region in a target object's face that does not match the proportion of other regions may be determined. For example, it is determined through analysis that the eye area has a smaller proportion than other areas, so that the eye area can be determined as the area to be adjusted. Furthermore, according to the ratio of the eye area to other areas, the adjustment parameters corresponding to the eye area can be directly determined.
  • the width of the eye area should be 1/3 of the width of the face compared to the width of the face, and the analysis found that the width of the eye area in the model of the target object is 7/24 of the width of the face, so According to the relationship between 7/24 and 1/3, the adjustment parameters corresponding to each key point in the eye area can be determined, such as moving the key points corresponding to the corner of the eye to the sides respectively x, so that the width of the moved eye area occupies 1/3 of the total face width.
  • the model adjustment parameters corresponding to the target object can be automatically determined without the user's manual adjustment to determine the adjustment parameters, which is convenient and fast.
  • the three-dimensional model of the target object may be adjusted according to the adjustment parameters.
  • the key points can be adjusted according to the adjustment parameters of the determined key points, thereby realizing the adjustment of the three-dimensional model.
  • FIG. 4 is a schematic flowchart of another three-dimensional model processing method according to an embodiment of the present disclosure.
  • the three-dimensional model processing method includes:
  • Step 201 Obtain a three-dimensional model of a target object, where the three-dimensional model includes multiple key points and position information of multiple key points.
  • step 202 according to the position information of multiple key points in the three-dimensional model of the target object, a proportional relationship between multiple key regions of the target object and a position relationship between multiple key points in each key region are determined.
  • steps 201 to 202 are similar to steps 101 to 102 in the foregoing embodiment, so details are not described herein again.
  • Step 203 Determine a target key area according to a proportional relationship between the plurality of key areas.
  • the key areas that need to be adjusted can be determined according to the proportional relationship between the multiple key areas.
  • Called the target critical area For example, compared with facial features, the ratio of eyes to other areas of the face is smaller, so you need to adjust the eyes.
  • the preset 3D model can be parsed to determine the reference scale relationship between multiple key areas, and then based on the scale relationship and reference scale between multiple key areas in the 3D model of the target object Relationships to identify key areas of interest.
  • the preset three-dimensional model may be a standard three-dimensional model of a human face.
  • a proportional relationship between multiple key regions may be obtained as a reference proportional relationship.
  • the scale relationship between the key areas in the three-dimensional model of the target object and the reference scale relationship are matched, and the key area where the scale relationship does not match the reference scale relationship is determined as the target key area.
  • the eyes or eyebrows may be the target key area.
  • the ratio of the eye width to the face width is 1/3, and the ratio of the eye width to the face width in the three-dimensional model of the target object is also 1/3, so that the eyebrows can be determined as the target key area.
  • Step 204 Determine an adjustment parameter corresponding to each key point according to the position relationship between multiple key points in the target key area.
  • the key points that need to be adjusted and the extent to be adjusted can be determined according to the positional relationship between multiple key points within the target key area.
  • Step 205 Determine a non-key area adjacent to the target key area in the three-dimensional model.
  • the target key area when adjusted, it may affect other surrounding areas. If only the target key area is adjusted, the adjustment may not be true and natural. For example, according to the key points in the orbit, the eye socket may be deepened, which may make the periphery of the eye socket unnatural. At this time, the area around the eye socket may be adjusted.
  • a non-key area adjacent to the target key area in the three-dimensional model may be determined. For example, if the nose is the target key area, the key area adjacent to the nose wing can be used as the non-key area.
  • Step 206 Determine the adjustment corresponding to each sub-keypoint in the non-key area according to the positional relationship between multiple sub-keypoints that constitute the non-key area and the target key area, and the adjustment parameters corresponding to each key point in the target key area. parameter.
  • the non-key area is composed of multiple sub-key points. After determining the non-critical area, determine the adjustment parameters for the sub-critical points in the non-critical area.
  • the position relationship between the multiple key points and the target key area is determined. Then, according to the positional relationship between the multiple sub-key points and the target key area, and the adjustment parameters corresponding to each key point in the target key area, the adjustment parameters corresponding to each sub-key point in the non-key area are determined.
  • the positional relationship between multiple sub-critical points and the target key area may be the positional relationship between the sub-key points and each key point in the target key area.
  • the amplitude of the adjustment parameters of each key point is smaller than the amplitude of the adjustment parameter corresponding to the key point, so that a smooth transition can be achieved during the adjustment of the 3D model of the target object.
  • the sub-keys associated with the corner of the eye are two points in the orbit, and the distance between the two sub-keys and the corner of the eye is different, then the sub-key closest to the corner of the eye
  • the amplitude of the adjustment parameter of the point is greater than the amplitude of the adjustment parameter of the sub-critical point far from the corner of the eye, so that the 3D model of the target object undergoes a smooth transition during the adjustment process, and the 3D model of the target object is finally obtained More natural.
  • FIG. 5 is a schematic flowchart of another three-dimensional model processing method according to an embodiment of the present disclosure.
  • the three-dimensional model processing method may further include:
  • Step 301 Determine a target area for skin texture adjustment according to the position of each key point in the three-dimensional model.
  • the target area to be adjusted for skin texture can be determined according to the position of each key point in the target object's three-dimensional model of the target key area and the magnitude of the corresponding adjustment parameter.
  • the skin texture area corresponding to the target key area and the skin texture area involved in the adjustment parameter may be used as the to-be-waited Target area for skin texture adjustment.
  • Step 302 Adjust the skin texture of the target area according to the adjustment parameters of key points in the target area.
  • the adjustment range of the skin texture of the target area can be determined according to the adjustment parameters of key points in the target area, and then the skin texture can be adjusted according to the adjustment range of the skin texture of the target area.
  • the key point in the target key area is a point on the bridge of the nose, and the amplitude of the corresponding adjustment parameter is increased by 0.1 centimeter (cm), after adjusting the key point, it may result in
  • the skin is deformed due to being overstretched, so the area to be adjusted for skin texture can be determined on both sides of the nose, based on the position of the key point (nasal bridge) and the amplitude of the corresponding adjustment parameter (increase by 0.1cm).
  • the skin texture on both sides of the nose can be appropriately shrunk, for example, shrinking by 0.05cm, etc., so as to make the final face image more natural.
  • corresponding adjustment thresholds can be set for key points at different positions in the target key area. Only when the adjustment range of the key points exceeds the threshold, the skin texture of the area where the key point needs to be adjusted needs to be adjusted.
  • the adjustment thresholds corresponding to different key points may be different. For example, if the key point is a point in the eye socket, the adjustment threshold corresponding to the position of the key point may be different from the adjustment threshold corresponding to the key point in the bridge of the nose.
  • the skin texture may not change much, and the corresponding adjustment threshold may be large; while when the key points on the bridge of the nose are raised, the skin around the bridge of the nose may be easily seen being pulled Extension, skin breakage occurs, then the corresponding adjustment threshold should be relatively small.
  • skin texture adjustment is performed on a target area by adjusting parameters of key points in the target area, so that a final processing effect can be more real and natural.
  • FIG. 6 is a schematic structural diagram of a three-dimensional model processing apparatus according to an embodiment of the present disclosure.
  • the three-dimensional model processing apparatus includes: an obtaining module 410, a first determining module 420, and a second determining module 430.
  • the obtaining module 410 is configured to obtain a three-dimensional model of a target object, where the three-dimensional model includes a plurality of key points and position information of the plurality of key points.
  • the first determining module 420 is configured to determine the proportional relationship between multiple key regions of the target object and the position relationship between multiple key points in each key region according to the position information of multiple key points in the three-dimensional model of the target object. Among them, each key region is obtained by concatenating a plurality of adjacent key points as vertices.
  • the second determining module 430 is configured to determine a model adjustment parameter corresponding to the target object according to a proportional relationship between a plurality of key areas and a position relationship between a plurality of key points in each key area.
  • the second determining module 430 may include:
  • a first determining unit configured to determine a target key area according to a proportional relationship between a plurality of key areas
  • the second determining unit is configured to determine an adjustment parameter corresponding to each key point according to a position relationship between multiple key points in the target key area.
  • the second determining module 430 further includes:
  • a third determining unit configured to analyze a plurality of preset three-dimensional models and determine a reference proportional relationship between the plurality of key regions before determining the target key region according to the proportion relationship between the plurality of key regions;
  • the second determining unit is further configured to determine a key area where the proportional relationship and the reference proportional relationship do not match as a target key area.
  • the second determining module 430 may further include:
  • a fourth determining unit configured to determine a non-critical area adjacent to the target key area in the three-dimensional model after determining the adjustment parameter corresponding to each key point;
  • a fifth determining unit is configured to determine each sub-key in the non-key area according to the positional relationship between multiple sub-key points that constitute the non-key area and the target key area, and the adjustment parameters corresponding to each key point in the target key area The adjustment parameter corresponding to the point, wherein the amplitude of the adjustment parameter corresponding to the secondary key point is smaller than the amplitude of the adjustment parameter corresponding to the key point.
  • the second determining module 430 may also include:
  • a sixth determining unit configured to determine a target region for skin texture adjustment according to a position of each key point in the three-dimensional model after determining adjustment parameters corresponding to each key point;
  • the adjusting unit is configured to adjust the skin texture of the target area according to the adjustment parameters of key points in the target area.
  • the device may further include:
  • the adjustment module is configured to adjust a three-dimensional model of the target object based on the adjustment parameter after determining a model adjustment parameter corresponding to the target object.
  • the three-dimensional model processing device of the embodiment of the present disclosure obtains a three-dimensional model of a target object, where the three-dimensional model includes multiple key points and position information of multiple key points, and according to the positions of the multiple key points in the three-dimensional model of the target object Information to determine the proportional relationship between multiple key areas of the target object, and the positional relationship between multiple key points in each key area, based on the proportional relationship between multiple key areas and between multiple key points in each key area Position relationship, determine the model adjustment parameters corresponding to the target object.
  • the adjustment parameters corresponding to the target object can be automatically determined without the user having to repeatedly adjust the amplitude of each parameter to obtain Appropriate adjustment parameters are flexible and convenient, and the shaping effect based on the three-dimensional model is realistic.
  • An embodiment of the present disclosure further provides an electronic device including a processor and a memory; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the foregoing embodiment.
  • FIG. 7 illustrates a block diagram of an exemplary electronic device suitable for use in implementing embodiments of the present disclosure.
  • the electronic device 12 shown in FIG. 7 is merely an example, and should not impose any limitation on the functions and the scope of use of the embodiments of the present disclosure.
  • the electronic device 12 is expressed in the form of a general-purpose computing device.
  • the components of the electronic device 12 may include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).
  • the bus 18 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local area bus using any of a variety of bus structures.
  • these architectures include, but are not limited to, Industry Standard Architecture (hereinafter referred to as ISA) bus, Micro Channel Architecture (hereinafter referred to as MAC) bus, enhanced ISA bus, video electronics Standards Association (Video) Standards Association (hereinafter referred to as: VESA) local bus and Peripheral Component Interconnection (hereinafter referred to as: PCI) bus.
  • ISA Industry Standard Architecture
  • MAC Micro Channel Architecture
  • VESA video electronics Standards Association
  • PCI Peripheral Component Interconnection
  • the electronic device 12 typically includes a variety of computer system-readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.
  • the memory 28 may include a computer system readable medium in the form of volatile memory, such as Random Access Memory (hereinafter referred to as RAM) 30 and / or cache memory 32.
  • RAM Random Access Memory
  • the electronic device 12 may further include other removable / non-removable, volatile / nonvolatile computer system storage media.
  • the storage system 34 may be used to read and write non-removable, non-volatile magnetic media (not shown in FIG. 7 and is commonly referred to as a "hard drive").
  • a disk drive for reading and writing to a removable non-volatile disk such as a "floppy disk” and a removable non-volatile optical disk (such as a compact disk read-only memory (Compact Disc ReadRead Only Memory (hereinafter referred to as: CD-ROM), digital multi-function read-only optical disc (Digital Video Disc Read Read Only Memory; hereinafter referred to as: DVD-ROM) or other optical media) read and write optical disc drive.
  • CD-ROM Compact Disc ReadRead Only Memory
  • DVD-ROM digital multi-function read-only optical disc
  • each drive may be connected to the bus 18 through one or more data medium interfaces.
  • the memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of embodiments of the present disclosure.
  • a program / utility tool 40 having a set (at least one) of program modules 42 may be stored in, for example, the memory 28.
  • Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data Each of these examples, or some combination, may include an implementation of a network environment.
  • the program module 42 generally performs functions and / or methods in the embodiments described in the present disclosure.
  • the electronic device 12 may also communicate with one or more external devices 14 (such as a keyboard, pointing device, display 24, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 12, and / or with Any device (such as a network card, modem, etc.) that enables the electronic device 12 to communicate with one or more other computing devices. This communication can be performed through an input / output (I / O) interface 22.
  • the electronic device 12 may also be connected to one or more networks (such as a local area network (hereinafter referred to as LAN), a wide area network (hereinafter referred to as WAN), and / or a public network such as the Internet through the network adapter 20 ) Communication.
  • networks such as a local area network (hereinafter referred to as LAN), a wide area network (hereinafter referred to as WAN), and / or a public network such as the Internet through the network adapter 20 ) Communication.
  • the network adapter 20 communicates with other modules of the electronic device 12 through the bus 18. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage systems.
  • the processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, for example, implementing the methods mentioned in the foregoing embodiments.
  • the embodiment of the present disclosure also proposes a non-transitory computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the three-dimensional model processing method according to the foregoing embodiment is implemented.
  • first and second are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as “first” and “second” may explicitly or implicitly include at least one of the features.
  • any process or method description in a flowchart or otherwise described herein can be understood as representing a module, fragment, or portion of code that includes one or more executable instructions for implementing steps of a custom logic function or process
  • the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present disclosure belong.
  • Logic and / or steps represented in a flowchart or otherwise described herein, for example, a sequenced list of executable instructions that may be considered to implement a logical function, may be embodied in any computer-readable medium, For use by, or in combination with, an instruction execution system, device, or device (such as a computer-based system, a system that includes a processor, or another system that can fetch and execute instructions from an instruction execution system, device, or device) Or equipment.
  • a "computer-readable medium” may be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.
  • computer-readable media include the following: electrical connections (electronic devices) with one or more wirings, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disk read-only memory (CDROM).
  • the computer-readable medium may even be paper or other suitable medium on which the program can be printed, because, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or other suitable Processing to obtain the program electronically and then store it in computer memory.
  • portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.
  • multiple steps or methods may be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.
  • Discrete logic circuits with logic gates for implementing logic functions on data signals Logic circuits, ASICs with suitable combinational logic gate circuits, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.
  • a person of ordinary skill in the art can understand that all or part of the steps carried by the methods in the foregoing embodiments can be implemented by a program instructing related hardware.
  • the program can be stored in a computer-readable storage medium.
  • the program is When executed, one or a combination of the steps of the method embodiment is included.
  • the aforementioned storage medium may be a read-only memory, a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本公开提出一种三维模型处理方法、装置、电子设备和存储介质,其中,方法包括:获取目标对象的三维模型,其中,三维模型中包括多个关键点及多个关键点的位置信息,根据目标对象的三维模型中多个关键点的位置信息,确定目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系,根据多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,确定目标对象对应的模型调整参数。该方法通过根据目标对象的多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,可自动确定目标对象对应的调整参数,无需用户反复调整各参数幅度来得到合适的调整参数,灵活便捷,并且基于三维模型整形效果真实感强。

Description

三维模型处理方法、装置、电子设备和存储介质
相关申请的交叉引用
本公开要求OPPO广东移动通信有限公司于2018年08月16日提交的、发明名称为“三维模型处理方法、装置、电子设备和存储介质”的、中国专利申请号“201810935099.2”的优先权。
技术领域
本公开涉及电子设备技术领域,尤其涉及一种三维模型处理方法、装置、电子设备和存储介质。
背景技术
随着移动终端技术的不断发展,越来越多的用户选择使用移动终端,如手机、掌上电脑等,进行拍照。为了达到较佳的拍摄效果,还可以采用相关的图像处理手段对图像进行处理。例如,对人脸进行微调整处理。
相关技术中,通过一组标准参数对二维的人脸图像各部位进行调整,实现人脸的整形,该方法需要用户不断地调整各部位的参数幅度来得到适合的整形参数。
发明内容
本公开提出一种三维模型处理方法、装置、电子设备和存储介质,用于解决相关技术中基于二维人脸图像获取整形参数的方法,操作繁琐不够灵活,并且整形效果不真实的问题。
本公开一方面实施例提出一种三维模型处理方法,包括:
获取目标对象的三维模型,其中,所述三维模型中包括多个关键点及所述多个关键点的位置信息;
根据所述目标对象的三维模型中多个关键点的位置信息,确定所述目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系,其中,每个关键区域由相邻多个关键点作为顶点拼接得到;
根据所述多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,确定所述目标对象对应的模型调整参数。
本公开实施例的三维模型处理方法,通过获取目标对象的三维模型,其中,三维模型中包括多个关键点及多个关键点的位置信息,根据目标对象的三维模型中多个关键点的位置信息,确定目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系,根据多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,确定目标对象对应的模型调整参数。由此,通过根据目标对象的多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,可自动确定目标对象对应的调整参数,无需用户反复调整各参数幅度来得到合适的调整参数,灵活便捷,并且基于三维模型整形效果真实感强。
本公开另一方面实施例提出一种三维模型处理装置,包括:
获取模块,用于获取目标对象的三维模型,其中,所述三维模型中包括多个关键点及所述多个关键点的位置信息;
第一确定模块,用于根据所述目标对象的三维模型中多个关键点的位置信息,确定所述目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系,其中,每个关键区域由相邻多个关键点作为顶点拼接得到;
第二确定模块,用于根据所述多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,确定所述目标对象对应的模型调整参数。
本公开实施例的三维模型处理装置,通过获取目标对象的三维模型,其中,三维模型中包括多个关键点及多个关键点的位置信息,根据目标对象的三维模型中多个关键点的位置信息,确定目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系,根据多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,确定目标对象对应的模型调整参数。由此,通过根据目标对象的多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,可自动确定目标对象对应的调整参数,无需用户反复调整各参数幅度来得到合适的调整参数,灵活便捷,并且基于三维模型整形效果真实感强。
本公开另一方面实施例提出一种电子设备,包括处理器和存储器;
所述处理器通过读取所述存储器中存储的可执行程序代码来运行与所述可执行程序代码对应的程序,以用于实现如上述一方面实施例所述的三维模型处理方法。
本公开另一方面实施例提出一种非临时性计算机可读存储介质,其上存储有计算机程序,该计算机程序被处理器执行时实现如上述一方面实施例所述的三维模型处理方法。
本公开附加的方面和优点将在下面的描述中部分给出,部分将从下面的描述中变得明显,或通过本公开的实践了解到。
附图说明
为了更清楚地说明本公开实施例中的技术方案,下面将对实施例中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图是本公开的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的一种三维模型处理方法的流程示意图;
图2本公开实施例中人脸三维模型的框架示意图一;
图3本公开实施例中人脸三维模型的框架示意图二;
图4为本公开实施例提供的另一种三维模型处理方法的流程示意图;
图5为本公开实施例提供的另一种三维模型处理方法的流程示意图;
图6为本公开实施例提供的一种三维模型处理装置的结构示意图;
图7示出了适于用来实现本公开实施方式的示例性电子设备的框图。
具体实施方式
下面详细描述本公开的实施例,所述实施例的示例在附图中示出,其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的,旨在用于解释本公开,而不能理解为对本公开的限制。
下面参考附图描述本公开实施例的三维模型处理方法、装置、电子设备和存储介质。
本公开各实施例,针对相关技术中获取整形参数的方法操作繁琐,不够灵活,并且整形效果不真实的问题,提出一种三维模型处理方法。
本公开实施例的三维模型处理方法,首先获取目标对象的三维模型,其中,三维模型中包括多个关键点及多个关键点的位置信息,然后根据目标对象的三维模型中多个关键点的位置信息,确定目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系,再根据多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,确定目标对象对应的模型调整参数。由此,通过根据目标对象的多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,可自动确定目标对象对应的调整参数,无需用户反复调整各参数幅度来得到合适的调整参数,灵活便捷,并且基于三维模型整形效果真实感强。
图1为本公开实施例提供的一种三维模型处理方法的流程示意图。
本公开实施例提供的三维模型处理方法,可以由本公开提供的三维模型处理装置执行,上述装置配置于电子设备中,以实现根据目标对象的多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,自动确定目标对象对应的模型调整参数。
其中,本实施例中的电子设备可以为任意具有拍照功能的硬件设备,如手机、掌上电 脑、穿戴式设备等。
如图1所示,该三维模型处理方法包括:
步骤101,获取目标对象的三维模型,其中,三维模型中包括多个关键点及多个关键点的位置信息。
本实施例中,目标对象中的对象可以指人,也可以指人脸。当对象理解为人时,目标对象的三维模型指目标对象的人脸的三维模型。
本公开实施例中,电子设备可以包括可见光图像传感器,可以基于电子设备中的可见光图像传感器获取二维的人脸图像。具体地,可见光图像传感器可以包括可见光摄像头,可见光摄像头可以捕获由人脸反射的可见光进行成像,得到二维的人脸图像。
本公开实施例中,电子设备还可以包括结构光图像传感器,可以基于电子设备中的结构光图像传感器,获取人脸对应的深度信息。可选地,结构光图像传感器可以包括镭射灯以及激光摄像头。脉冲宽度调制(Pulse Width Modulation,简称PWM)可以调制镭射灯以发出结构光,结构光照射至成人脸,激光摄像头可以捕获由人脸反射的结构光进行成像,得到人脸对应的结构光图像。深度引擎可以根据人脸对应的结构光图像,计算获得人脸对应的深度信息,即二维人脸图像对应的深度信息。
可选地,电子设备中预存有目标对象对应的二维人脸图像和二维人脸图像对应的深度信息,从而可以直接获取目标对象的二维人脸图像及二维人脸图像对应的深度信息,进而利用预存的目标对象的二维人脸图像及二维人脸图像对应的深度信息构建目标对象的三维模型。
本公开实施例中,可以从至少两个角度获取目标对象的至少两张二维人脸图像,及至少两张二维人脸图像分别对应的深度信息。在获取多张二维人脸图像和二维人脸图像对应的深度信息后,可以根据多张二维人脸图像和对应的深度信息,构建目标对象的三维模型。
本公开中,目标对象的三维模型的构建,是根据深度信息和人脸图像,进行三维重构得到的,而不是简单的获取RGB数据和深度数据。
作为一种可能的实现方式,可以将深度信息与二维人脸图像对应的色彩信息进行融合,得到目标对象的三维模型。具体地,可以基于人脸关键点检测技术,从深度信息提取人脸的关键点,以及从色彩信息中提取人脸的关键点,而后将从深度信息中提取的关键点和从色彩信息中提取的关键点,进行配准和关键点融合处理,最终根据融合后的关键点,生成目标对象的三维模型。其中,关键点为人脸上显眼的点,或者为关键位置上的点,例如关键点可以为眼角、鼻尖、嘴角等。
作为另一种可能的实现方式,可以基于人脸关键点检测技术,对每张二维人脸图像进行关键点识别,得到各第三关键点,而后根据各第三关键点在每张二维人脸图像中的位置 及各第三关键点的深度信息,确定与各第三关键点对应的各第二关键点在三维空间中的相对位置,从而可以根据各第二关键点在三维空间中的相对位置,生成由相邻的各第二关键点拼接的局部三维框架。
在确定各局部三维框架后,可以根据各局部三维框架中的重合部分,对不同的局部三维框架进行拼接,得到目标对象的三维模型。具体地,可以根据各局部三维框架中包含的相同第二关键点,对各局部三维框架进行拼接,得到目标对象的三维模型。
比如,根据额头三维框架与眼睛三维框架包含的相同的第二关键点,将额头三维框架与眼睛三维框架进行拼接,据此根据各局部三维框架中包含的相同第二关键点,将各个局部三维框架进行拼接,可以得到人脸三维模型。
其中,三维模型中包括多个关键点及多个关键点的位置信息。例如,鼻子上的多个关键点、眼眶上的多个关键点等。
作为一种示例,参见图2和图3,图2和图3分别为不同的角度获取的两张二维人脸图像所对应的目标对象的三维模型的示意图。
步骤102,根据目标对象的三维模型中多个关键点的位置信息,确定目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系。
在获取目标对象的三维模型后,可根据目标对象的三维模型中多个关键的位置信息,将相邻的多个关键点作为顶点拼接得到多个关键区域。例如,经过相邻关键点拼接,得到鼻子、眼睛、嘴巴等关键区域。
由于关键区域是由相邻的多个关键点作为顶点拼接得到,可以理解的是,相邻的多个关键点包围的区域中的关键点即为关键区域包含的关键点。由此,根据每个关键区域中多个关键点的位置信息,可以确定每个关键区域中多个关键点间的位置关系。
例如,根据眼睛区域内包含的多个关键点的位置信息,可以确定眼睛区域内多个关键点间的位置关系,如眼角的两个关键点相差0.1毫米。
本实施例中,根据得到的多个关键区域,可以确定每个关键区域在目标对象的三维模型中的比例关系,由此可以得到确定目标对象的多个关键区域间的比例关系。也就是说,可以得到人脸中每个部位的比例大小。
并且,根据每个关键区域内各关键点间的位置关系,可以确定各关键区域的立体程度。比如,根据鼻子区域中,鼻尖对应的关键点与鼻根对应的关键点间的位置关系,即可确定鼻子的立体程度。
步骤103,根据多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,确定目标对象对应的模型调整参数。
本实施例中,根据多个关键区域间的比例关系,可以确定需要调整的关键区域。在确 定需要调整的关键区域后,根据该关键区域中多个关键点间的位置关系,确定目标对象对应的模型调整参数。
具体地,根据多个关键区域间的比例关系,可以确定目标对象人脸中,与其他区域的比例不匹配的关键区域。例如,通过分析确定,眼睛区域与其他区域相比,比例较小,从而即可确定眼睛区域为待调整区域。进而,根据眼睛区域与其他区域的比例大小,即可直接确定眼睛区域对应的调整参数。
例如,标准情况下,眼睛区域的宽度与人脸的宽度相比,应为人脸宽度的1/3,而通过分析发现,目标对象的模型中眼睛区域的宽度为人脸宽度的7/24,从而根据7/24与1/3的关系,即可确定眼睛区域中各关键点对应的调整参数,比如将眼角对应的关键点,分别向两侧移动x,以使移动后的眼睛区域的宽度占人脸总宽度的1/3。
本实施例中,通过多个关键区域间的比例关系及每个关键区域中多个关键间的位置关系,可自动确定目标对象对应的模型调整参数,无需用户手动调整确定调整参数,方便快捷。
进一步地,在确定目标对象的模型调整参数后,可根据调整参数对目标对象的三维模型进行调整处理。具体地,可根据确定的关键点的调整参数,对关键点进行调整,从而实现对三维模型的调整。
图4为本公开实施例提供的另一种三维模型处理方法的流程示意图。
如图4所示,该三维模型处理方法包括:
步骤201,获取目标对象的三维模型,其中,三维模型中包括多个关键点及多个关键点的位置信息。
步骤202,根据目标对象的三维模型中多个关键点的位置信息,确定目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系。
本实施例中,步骤201-步骤202与上述实施例中步骤101-步骤102类似,故在此不再赘述。
步骤203,根据多个关键区域间的比例关系,确定目标关键区域。
在实际应用中,目标对象的三维模型中有多个关键区域,可能只需要调整某一个或者几个关键区域,由此可根据多个关键区域间的比例关系,确定需要调整的关键区域,这里称为目标关键区域。比如,人脸五官区域相比,眼睛与人脸其他区域的比值较小,那么就需要调整眼睛。
在确定目标关键区域之前,可先对预设的三维模型进行解析处理,以确定多个关键区域间的参考比例关系,进而根据目标对象的三维模型中多个关键区域间的比例关系和参考比例关系,确定目标关键区域。
其中,预设的三维模型可以是标准人脸三维模型,通过对标准人脸三维模型进行解析,可以得到多个关键区域间的比例关系,作为参考比例关系。在确定参考比例关系后,将目标对象的三维模型中多个关键区域间的比例关系和参考比例关系进行匹配,将比例关系与参考比例关系未匹配的关键区域,确定为目标关键区域。
例如,目标对象的三维模型中眼睛与眉毛比值小于1,而参考比例关系中眼睛与眉毛的比值大于1,即与参考比例不匹配,则眼睛或眉毛可能为目标关键区域。而参考比例关系中,眼宽与脸宽的比例为1/3,目标对象的三维模型中,眼宽与脸宽的比例也为1/3,从而即可确定眉毛为目标关键区域。
步骤204,根据目标关键区域内多个关键点间的位置关系,确定每个关键点对应的调整参数。
在确定目标关键区域后,可根据目标关键区域内多个关键点间的位置关系,确定需要调整的关键点及需要调整的幅度。
例如,确定眼睛太小需要进行调整,即眼睛为目标关键区域,那么再根据眼睛内多个关键点间的位置关系,确定眼睛小是因为眼眶太小,那么将眼眶上的关键点均向外扩展1毫米。
步骤205,确定三维模型中与目标关键区域相邻的非关键区域。
在实际应用中,在调整目标关键区域时,可能会对周围的其他区域造成影响,如果只调整目标关键区域,可能会使调整不够真实自然。比如,根据眼眶中的关键点内推后,眼窝加深,可能会使眼眶周围不够自然,这时可对眼眶周围的区域进行调整。
本实施例中,在确定目标关键区域每个关键点的调整参数后,可确定三维模型中与目标关键区域相邻的非关键区域。例如,鼻子为目标关键区域,则与鼻翼相邻的关键区域可作为非关键区域。
步骤206,根据组成非关键区域的多个次关键点与目标关键区域间的位置关系,及目标关键区域内每个关键点对应的调整参数,确定非关键区域内每个次关键点对应的调整参数。
本实施例中,非关键区域由多个次关键点组成。在确定非关键区域后,确定对非关键区域内次关键点的调整参数。
具体地,根据多个次关键点的位置信息以及目标关键区域内每个关键点的位置信息,确定多个次关键点与目标关键区域间的位置关系。然后,根据多个次关键点与目标关键区域间的位置关系,及目标关键区域内每个关键点对应的调整参数,确定非关键区域内每个次关键点对应的调整参数。其中,多个次关键点与目标关键区域间的位置关系,可以是次关键点与目标关键区域内各关键点之间的位置关系。
为了使得调整后的目标对象的三维模型较为自然,各次关键点的调整参数的幅值小于关键点对应的调整参数的幅值,从而在目标对象的三维模型调整过程中可以实现平滑过渡。
作为一种可能的实现方式,确定多个次关键点与目标关键区域内距离最近的关键点,通过适当减小该距离最近的关键点的调整参数的幅值,得到对应的次关键点的调整参数的幅值。
举例而言,当目标关键区域内包括一个关键点眼角,与眼角关联的次关键点分别为眼眶中的两个点,且两个次关键点与眼角的距离不同,那么距离眼角最近的次关键点的调整参数的幅值,要大于距离眼角较远的次关键点的调整参数的幅值,从而使得目标对象的三维模型在调整的过程中进行了平滑过渡,最终获得的目标对象的三维模型较为自然。
为了避免由于三维模型中目标关键区域内多个关键点的位置改变,引起皮肤纹理的变形,本实施例中,在确定每个关键点对应的调整参数后,还可对覆盖目标三维模型的皮肤纹理进行调整,从而使最终根据目标对象的三维模型得到调整效果更加真实自然。图5为本公开实施例提供的另一种三维模型处理方法的流程示意图。
在确定每个关键点对应的调整参数之后,如图5所示,该三维模型处理方法还可包括:
步骤301,根据每个关键点在三维模型中的位置,确定待进行皮肤纹理调整的目标区域。
为了使皮肤纹理比较自然,可根据目标关键区域内每个关键点在目标对象的三维模型中的位置,及分别对应的调整参数的幅值,确定待进行皮肤纹理调整的目标区域。
具体地,根据目标关键区域内每个关键点在目标对象的三维模型中的位置信息和调整参数,可以将对应覆盖目标关键区域的皮肤纹理区域、及调整参数所涉及的皮肤纹理区域,作为待进行皮肤纹理调整的目标区域。
步骤302,根据目标区域中关键点的调整参数,将目标区域的皮肤纹理进行调整。
本实施例中,可根据目标区域中关键点的调整参数,确定目标区域的皮肤纹理的调整幅度,进而根据目标区域的皮肤纹理的调整幅度调整皮肤纹理。
举例而言,若目标关键区域内的关键点为鼻梁上的点,且对应的调整参数的幅值为增加0.1厘米(cm),那么在将该关键点调整后,可能会导致鼻翼两侧的皮肤由于被过度拉伸而出现变形,从而则可以根据该关键点的位置(鼻梁)及对应的调整参数的幅值(增加0.1cm)确定待进行皮肤纹理调整的区域为鼻翼两侧,进而即可对鼻翼两侧的皮肤纹理进行适当的收缩处理,比如收缩0.05cm等,从而使最终得到的人脸图像更自然。
本实施例中,可以为目标关键区域内不同位置的关键点设置对应的调整阈值,仅当该关键点的调整幅度超过该阈值时,才需要对其所在的区域的皮肤纹理进行调整。
需要说明的是,不同的关键点对应的调整阈值可能不同。比如,关键点为眼眶中的点, 那么将关键点的位置调整时对应的调整阈值,可能与鼻梁中的关键点对应的调整阈值不同。
例如,将眼眶中的关键点内推时,皮肤纹理的变化可能不大,相应的调整阈值可以较大;而将鼻梁上的关键点变高时,鼻梁周围的皮肤可能很容易看出被拉伸,出现皮肤断裂,那么相应的调整阈值要相对小些。
本公开实施例的三维模型处理方法,通过目标区域中关键点的调整参数,对目标区域进行皮肤纹理调整,可以使最终得到处理效果更加真实自然。
本公开实施例还提出一种三维模型处理装置。图6为本公开实施例提供的一种三维模型处理装置的结构示意图。
如图6所示,该三维模型处理装置包括:获取模块410、第一确定模块420、第二确定模块430。
获取模块410用于获取目标对象的三维模型,其中,三维模型中包括多个关键点及所述多个关键点的位置信息。
第一确定模块420用于根据目标对象的三维模型中多个关键点的位置信息,确定目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系,其中,每个关键区域由相邻多个关键点作为顶点拼接得到。
第二确定模块430用于根据多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,确定目标对象对应的模型调整参数。
在本公开实施例一种可能的实现方式中,第二确定模块430可包括:
第一确定单元,用于根据多个关键区域间的比例关系,确定目标关键区域;
第二确定单元,用于根据目标关键区域内多个关键点间的位置关系,确定每个关键点对应的调整参数。
在本公开实施例一种可能的实现方式中,第二确定模块430还包括:
第三确定单元,用于在根据多个关键区域间的比例关系,确定目标关键区域之前,对多个预设的三维模型进行解析处理,确定多个关键区域间的参考比例关系;
第二确定单元,还用于将比例关系与参考比例关系未匹配的关键区域,确定为目标关键区域。
在本公开实施例一种可能的实现方式中,第二确定模块430还可包括:
第四确定单元,用于确定每个关键点对应的调整参数之后,确定三维模型中与目标关键区域相邻的非关键区域;
第五确定单元,用于根据组成非关键区域的多个次关键点与目标关键区域间的位置关系,及目标关键区域内每个关键点对应的调整参数,确定非关键区域内每个次关键点对应的调整参数,其中,次关键点对应的调整参数的幅值小于关键点对应的调整参数的幅值。
在本公开实施例一种可能的实现方式中,第二确定模块430也可包括:
第六确定单元,用于在确定每个关键点对应的调整参数之后,根据每个关键点在三维模型中的位置,确定待进行皮肤纹理调整的目标区域;
调整单元,用于根据目标区域中关键点的调整参数,将目标区域的皮肤纹理进行调整。
在本公开实施例一种可能的实现方式中,该装置还可包括:
调整模块,用于在确定目标对象对应的模型调整参数之后,基于调整参数,将目标对象的三维模型进行调整处理。
需要说明的是,前述对三维模型处理方法实施例的解释说明,也适用于该实施例的三维模型处理装置,在此不再赘述。
本公开实施例的三维模型处理装置,通过获取目标对象的三维模型,其中,三维模型中包括多个关键点及多个关键点的位置信息,根据目标对象的三维模型中多个关键点的位置信息,确定目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系,根据多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,确定目标对象对应的模型调整参数。由此,通过根据目标对象的多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,可自动确定目标对象对应的调整参数,无需用户反复调整各参数幅度来得到合适的调整参数,灵活便捷,并且基于三维模型整形效果真实感强。
本公开实施例还提出了一种电子设备,包括处理器和存储器;处理器通过读取存储器中存储的可执行程序代码来运行与可执行程序代码对应的程序,以用于实现如上述实施例所述的三维模型处理方法。
图7示出了适于用来实现本公开实施方式的示例性电子设备的框图。图7显示的电子设备12仅仅是一个示例,不应对本公开实施例的功能和使用范围带来任何限制。
如图7所示,电子设备12以通用计算设备的形式表现。电子设备12的组件可以包括但不限于:一个或者多个处理器或者处理单元16,系统存储器28,连接不同系统组件(包括系统存储器28和处理单元16)的总线18。
总线18表示几类总线结构中的一种或多种,包括存储器总线或者存储器控制器,外围总线,图形加速端口,处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说,这些体系结构包括但不限于工业标准体系结构(Industry Standard Architecture;以下简称:ISA)总线,微通道体系结构(Micro Channel Architecture;以下简称:MAC)总线,增强型ISA总线、视频电子标准协会(Video Electronics Standards Association;以下简称:VESA)局域总线以及外围组件互连(Peripheral Component Interconnection;以下简称:PCI)总线。
电子设备12典型地包括多种计算机系统可读介质。这些介质可以是任何能够被电子设备12访问的可用介质,包括易失性和非易失性介质,可移动的和不可移动的介质。
存储器28可以包括易失性存储器形式的计算机系统可读介质,例如随机存取存储器(Random Access Memory;以下简称:RAM)30和/或高速缓存存储器32。电子设备12可以进一步包括其它可移动/不可移动的、易失性/非易失性计算机系统存储介质。仅作为举例,存储系统34可以用于读写不可移动的、非易失性磁介质(图7未显示,通常称为“硬盘驱动器”)。尽管图7中未示出,可以提供用于对可移动非易失性磁盘(例如“软盘”)读写的磁盘驱动器,以及对可移动非易失性光盘(例如:光盘只读存储器(Compact Disc Read Only Memory;以下简称:CD-ROM)、数字多功能只读光盘(Digital Video Disc Read Only Memory;以下简称:DVD-ROM)或者其它光介质)读写的光盘驱动器。在这些情况下,每个驱动器可以通过一个或者多个数据介质接口与总线18相连。存储器28可以包括至少一个程序产品,该程序产品具有一组(例如至少一个)程序模块,这些程序模块被配置以执行本公开各实施例的功能。
具有一组(至少一个)程序模块42的程序/实用工具40,可以存储在例如存储器28中,这样的程序模块42包括但不限于操作系统、一个或者多个应用程序、其它程序模块以及程序数据,这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块42通常执行本公开所描述的实施例中的功能和/或方法。
电子设备12也可以与一个或多个外部设备14(例如键盘、指向设备、显示器24等)通信,还可与一个或者多个使得用户能与该电子设备12交互的设备通信,和/或与使得该电子设备12能与一个或多个其它计算设备进行通信的任何设备(例如网卡,调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口22进行。并且,电子设备12还可以通过网络适配器20与一个或者多个网络(例如局域网(Local Area Network;以下简称:LAN),广域网(Wide Area Network;以下简称:WAN)和/或公共网络,例如因特网)通信。如图所示,网络适配器20通过总线18与电子设备12的其它模块通信。应当明白,尽管图中未示出,可以结合电子设备12使用其它硬件和/或软件模块,包括但不限于:微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统等。
处理单元16通过运行存储在系统存储器28中的程序,从而执行各种功能应用以及数据处理,例如实现前述实施例中提及的方法。
本公开实施例还提出了一种非临时性计算机可读存储介质,其上存储有计算机程序,该计算机程序被处理器执行时实现如上述实施例所述的三维模型处理方法。
在本说明书的描述中,术语“第一”、“第二”仅用于描述目的,而不能理解为指示 或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为,表示包括一个或更多个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分,并且本公开的优选实施方式的范围包括另外的实现,其中可以不按所示出或讨论的顺序,包括根据所涉及的功能按基本同时的方式或按相反的顺序,来执行功能,这应被本公开的实施例所属技术领域的技术人员所理解。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤,例如,可以被认为是用于实现逻辑功能的可执行指令的定序列表,可以具体实现在任何计算机可读介质中,以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用,或结合这些指令执行系统、装置或设备而使用。就本说明书而言,"计算机可读介质"可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下:具有一个或多个布线的电连接部(电子装置),便携式计算机盘盒(磁装置),随机存取存储器(RAM),只读存储器(ROM),可擦除可编辑只读存储器(EPROM或闪速存储器),光纤装置,以及便携式光盘只读存储器(CDROM)。另外,计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质,因为可以例如通过对纸或其他介质进行光学扫描,接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序,然后将其存储在计算机存储器中。
应当理解,本公开的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中,多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。如,如果用硬件来实现和在另一实施方式中一样,可用本领域公知的下列技术中的任一项或他们的组合来实现:具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路,具有合适的组合逻辑门电路的专用集成电路,可编程门阵列(PGA),现场可编程门阵列(FPGA)等。
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成,所述的程序可以存储于一种计算机可读存储介质中,该程序在执行时,包括方法实施例的步骤之一或其组合。
上述提到的存储介质可以是只读存储器,磁盘或光盘等。尽管上面已经示出和描述了本公开的实施例,可以理解的是,上述实施例是示例性的,不能理解为对本公开的限制,本领域的普通技术人员在本公开的范围内可以对上述实施例进行变化、修改、替换和变型。

Claims (16)

  1. 一种三维模型处理方法,其特征在于,包括:
    获取目标对象的三维模型,其中,所述三维模型中包括多个关键点及所述多个关键点的位置信息;
    根据所述目标对象的三维模型中多个关键点的位置信息,确定所述目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系,其中,每个关键区域由相邻多个关键点作为顶点拼接得到;
    根据所述多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,确定所述目标对象对应的模型调整参数。
  2. 如权利要求1所述的方法,其特征在于,所述确定所述目标对象对应的模型调整参数,包括:
    根据所述多个关键区域间的比例关系,确定目标关键区域;
    根据所述目标关键区域内多个关键点间的位置关系,确定每个关键点对应的调整参数。
  3. 如权利要求2所述的方法,其特征在于,所述根据所述多个关键区域间的比例关系,确定目标关键区域之前,还包括:
    对多个预设的三维模型进行解析处理,确定多个关键区域间的参考比例关系;
    所述根据所述多个关键区域间的比例关系,确定目标关键区域,包括:
    将比例关系与所述参考比例关系未匹配的关键区域,确定为目标关键区域。
  4. 如权利要求2所述的方法,其特征在于,所述确定每个关键点对应的调整参数之后,还包括:
    确定所述三维模型中与所述目标关键区域相邻的非关键区域;
    根据组成所述非关键区域的多个次关键点与所述目标关键区域间的位置关系,及所述目标关键区域内每个关键点对应的调整参数,确定所述非关键区域内每个次关键点对应的调整参数,其中,次关键点对应的调整参数的幅值小于所述关键点对应的调整参数的幅值。
  5. 如权利要求4所述的方法,其特征在于,所述确定所述非关键区域内每个次关键点对应的调整参数,包括:
    确定多个次关键点与目标关键区域内距离最近的关键点,通过减小所述距离最近的关键点的调整参数的幅值,得到对应的次关键点的调整参数的幅值。
  6. 如权利要求2-5任一所述的方法,其特征在于,所述确定每个关键点对应的调整参数之后,还包括:
    根据每个关键点在三维模型中的位置,确定待进行皮肤纹理调整的目标区域;
    根据所述目标区域中关键点的调整参数,将所述目标区域的皮肤纹理进行调整。
  7. 如权利要求1-5任一所述的方法,其特征在于,所述确定所述目标对象对应的模型调整参数之后,还包括:
    基于所述调整参数,将所述目标对象的三维模型进行调整处理。
  8. 一种三维模型处理装置,其特征在于,包括:
    获取模块,用于获取目标对象的三维模型,其中,所述三维模型中包括多个关键点及所述多个关键点的位置信息;
    第一确定模块,用于根据所述目标对象的三维模型中多个关键点的位置信息,确定所述目标对象的多个关键区域间的比例关系,及每个关键区域中多个关键点间的位置关系,其中,每个关键区域由相邻多个关键点作为顶点拼接得到;
    第二确定模块,用于根据所述多个关键区域间的比例关系及每个关键区域中多个关键点间的位置关系,确定所述目标对象对应的模型调整参数。
  9. 如权利要求8所述的装置,其特征在于,所述第二确定模块包括:
    第一确定单元,用于根据所述多个关键区域间的比例关系,确定目标关键区域;
    第二确定单元,用于根据所述目标关键区域内多个关键点间的位置关系,确定每个关键点对应的调整参数。
  10. 如权利要求9所述的装置,其特征在于,所述第二确定模块还包括:
    第三确定单元,用于在所述根据所述多个关键区域间的比例关系,确定目标关键区域之前,对多个预设的三维模型进行解析处理,确定多个关键区域间的参考比例关系;
    所述第二确定单元,还用于将比例关系与所述参考比例关系未匹配的关键区域,确定为目标关键区域。
  11. 如权利要求9所述的装置,其特征在于,所述第二确定模块还包括:
    第四确定单元,用于确定所述三维模型中与所述目标关键区域相邻的非关键区域;
    第五确定单元,用于根据组成所述非关键区域的多个次关键点与所述目标关键区域间的位置关系,及所述目标关键区域内每个关键点对应的调整参数,确定所述非关键区域内每个次关键点对应的调整参数,其中,次关键点对应的调整参数的幅值小于所述关键点对应的调整参数的幅值。
  12. 如权利要求11所述的装置,其特征在于,所述第五确定单元,具体用于:
    确定多个次关键点与目标关键区域内距离最近的关键点,通过减小所述距离最近的关键点的调整参数的幅值,得到对应的次关键点的调整参数的幅值。
  13. 如权利要求9-12任一所述的装置,其特征在于,所述第二确定模块还包括:
    第六确定单元,用于在所述确定每个关键点对应的调整参数之后,根据每个关键点在 三维模型中的位置,确定待进行皮肤纹理调整的目标区域;
    调整单元,用于根据所述目标区域中关键点的调整参数,将所述目标区域的皮肤纹理进行调整。
  14. 如权利要求8-12任一所述的装置,其特征在于,还包括:
    调整模块,用于在所述确定所述目标对象对应的模型调整参数之后,基于所述调整参数,将所述目标对象的三维模型进行调整处理。
  15. 一种电子设备,其特征在于,包括处理器和存储器;
    所述处理器通过读取所述存储器中存储的可执行程序代码来运行与所述可执行程序代码对应的程序,以用于实现如权利要求1-7中任一所述的三维模型处理方法。
  16. 一种非临时性计算机可读存储介质,其上存储有计算机程序,其特征在于,该计算机程序被处理器执行时实现如权利要求1-7任一所述的三维模型处理方法。
PCT/CN2019/095287 2018-08-16 2019-07-09 三维模型处理方法、装置、电子设备和存储介质 Ceased WO2020034786A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810935099.2A CN109191552A (zh) 2018-08-16 2018-08-16 三维模型处理方法、装置、电子设备和存储介质
CN201810935099.2 2018-08-16

Publications (1)

Publication Number Publication Date
WO2020034786A1 true WO2020034786A1 (zh) 2020-02-20

Family

ID=64918419

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/095287 Ceased WO2020034786A1 (zh) 2018-08-16 2019-07-09 三维模型处理方法、装置、电子设备和存储介质

Country Status (2)

Country Link
CN (1) CN109191552A (zh)
WO (1) WO2020034786A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113377821A (zh) * 2021-06-09 2021-09-10 深圳前海微众银行股份有限公司 组合比例获取方法、设备、介质及程序产品

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109191552A (zh) * 2018-08-16 2019-01-11 Oppo广东移动通信有限公司 三维模型处理方法、装置、电子设备和存储介质
CN109903375B (zh) * 2019-02-21 2023-06-06 Oppo广东移动通信有限公司 模型生成方法、装置、存储介质及电子设备
CN110288554B (zh) * 2019-06-29 2022-09-16 北京字节跳动网络技术有限公司 视频美化方法、装置及电子设备
CN110288551B (zh) * 2019-06-29 2021-11-09 北京字节跳动网络技术有限公司 视频美化方法、装置及电子设备
CN113570634B (zh) * 2020-04-28 2024-07-12 北京达佳互联信息技术有限公司 对象三维重建方法、装置、电子设备及存储介质
CN113570506B (zh) * 2020-04-28 2025-02-18 阿里巴巴集团控股有限公司 数据处理、界面交互方法及设备
CN111710036B (zh) 2020-07-16 2023-10-17 腾讯科技(深圳)有限公司 三维人脸模型的构建方法、装置、设备及存储介质
CN112581518B (zh) * 2020-12-25 2024-09-06 百果园技术(新加坡)有限公司 基于三维卡通模型的眼球配准方法、装置、服务器和介质
CN113657357B (zh) * 2021-10-20 2022-02-25 北京市商汤科技开发有限公司 图像处理方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105227832A (zh) * 2015-09-09 2016-01-06 厦门美图之家科技有限公司 一种基于关键点检测的自拍方法、自拍系统及拍摄终端
CN105844276A (zh) * 2015-01-15 2016-08-10 北京三星通信技术研究有限公司 人脸姿态校正方法和装置
US20170109569A1 (en) * 2015-08-28 2017-04-20 Hongtae KIM Hybrid face recognition based on 3d data
CN107730445A (zh) * 2017-10-31 2018-02-23 广东欧珀移动通信有限公司 图像处理方法、装置、存储介质和电子设备
CN107944435A (zh) * 2017-12-27 2018-04-20 广州图语信息科技有限公司 一种三维人脸识别方法、装置及处理终端
CN109191552A (zh) * 2018-08-16 2019-01-11 Oppo广东移动通信有限公司 三维模型处理方法、装置、电子设备和存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104899575A (zh) * 2015-06-19 2015-09-09 南京大学 一种基于人脸检测和关键点定位的人体组件划分方法
CN106446781A (zh) * 2016-08-29 2017-02-22 厦门美图之家科技有限公司 一种人脸图像处理方法及装置
CN107302662A (zh) * 2017-07-06 2017-10-27 维沃移动通信有限公司 一种拍照的方法、装置及移动终端
CN107507269A (zh) * 2017-07-31 2017-12-22 广东欧珀移动通信有限公司 个性化三维模型生成方法、装置和终端设备
CN107977927A (zh) * 2017-12-14 2018-05-01 北京奇虎科技有限公司 基于图像数据的身材调整方法及装置、计算设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105844276A (zh) * 2015-01-15 2016-08-10 北京三星通信技术研究有限公司 人脸姿态校正方法和装置
US20170109569A1 (en) * 2015-08-28 2017-04-20 Hongtae KIM Hybrid face recognition based on 3d data
CN105227832A (zh) * 2015-09-09 2016-01-06 厦门美图之家科技有限公司 一种基于关键点检测的自拍方法、自拍系统及拍摄终端
CN107730445A (zh) * 2017-10-31 2018-02-23 广东欧珀移动通信有限公司 图像处理方法、装置、存储介质和电子设备
CN107944435A (zh) * 2017-12-27 2018-04-20 广州图语信息科技有限公司 一种三维人脸识别方法、装置及处理终端
CN109191552A (zh) * 2018-08-16 2019-01-11 Oppo广东移动通信有限公司 三维模型处理方法、装置、电子设备和存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113377821A (zh) * 2021-06-09 2021-09-10 深圳前海微众银行股份有限公司 组合比例获取方法、设备、介质及程序产品
CN113377821B (zh) * 2021-06-09 2024-06-07 深圳前海微众银行股份有限公司 组合比例获取方法、设备、介质及程序产品

Also Published As

Publication number Publication date
CN109191552A (zh) 2019-01-11

Similar Documents

Publication Publication Date Title
WO2020034786A1 (zh) 三维模型处理方法、装置、电子设备和存储介质
CN108876708B (zh) 图像处理方法、装置、电子设备和存储介质
EP3992919B1 (en) Three-dimensional facial model generation method and apparatus, device, and medium
CN108765272B (zh) 图像处理方法、装置、电子设备及可读存储介质
CN109272579B (zh) 基于三维模型的美妆方法、装置、电子设备和存储介质
CN111652123B (zh) 图像处理和图像合成方法、装置和存储介质
WO2018201551A1 (zh) 一种人脸图像的融合方法、装置及计算设备
US20180204052A1 (en) A method and apparatus for human face image processing
CN109147037B (zh) 基于三维模型的特效处理方法、装置和电子设备
CN109242760B (zh) 人脸图像的处理方法、装置和电子设备
CN108764180A (zh) 人脸识别方法、装置、电子设备及可读存储介质
WO2020034743A1 (zh) 三维模型处理方法、装置、电子设备及可读存储介质
CN108876709A (zh) 人脸美化方法、装置、电子设备及可读存储介质
CN110163832A (zh) 人脸融合方法、装置和终端
CN108463823A (zh) 一种用户头发模型的重建方法、装置及终端
CN109949237A (zh) 图像处理方法及装置、图像设备及存储介质
CN114049290A (zh) 图像处理方法、装置、设备及存储介质
CN108682050A (zh) 基于三维模型的美颜方法和装置
CN113223128B (zh) 用于生成图像的方法和装置
CN117934339B (zh) 一种图像修复方法、装置、电子设备和存储介质
CN118262078B (zh) 三维人脸模型的摆正处理方法、装置、设备及介质
TWI873036B (zh) 三維數字人生成方法及裝置、電子設備和存儲介質
CN113487489A (zh) 一种脸部修容显示及关键点检测模型训练方法及终端
CN116091687B (zh) 一种三维人脸重建方法、装置、电子设备及存储介质
CN113421197B (zh) 一种美颜图像的处理方法及其处理系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19849567

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19849567

Country of ref document: EP

Kind code of ref document: A1