WO2023207379A1 - 图像处理方法、装置、设备及存储介质 - Google Patents

图像处理方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2023207379A1
WO2023207379A1 PCT/CN2023/081253 CN2023081253W WO2023207379A1 WO 2023207379 A1 WO2023207379 A1 WO 2023207379A1 CN 2023081253 W CN2023081253 W CN 2023081253W WO 2023207379 A1 WO2023207379 A1 WO 2023207379A1
Authority
WO
WIPO (PCT)
Prior art keywords
map
depth
virtual
target
image
Prior art date
Application number
PCT/CN2023/081253
Other languages
English (en)
French (fr)
Inventor
袁琦
Original Assignee
北京字跳网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字跳网络技术有限公司 filed Critical 北京字跳网络技术有限公司
Publication of WO2023207379A1 publication Critical patent/WO2023207379A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the embodiments of the present disclosure relate to the field of image processing technology, such as an image processing method, device, equipment and storage medium.
  • Adding virtual objects to detected target objects is a common application scenario in augmented reality.
  • this application scenario when adding a virtual object, it is necessary to determine the occlusion relationship between the virtual object and the target object, and render the virtual object and the target object based on the occlusion relationship.
  • a standard virtual model is used to determine the occlusion relationship between the virtual object and the target object.
  • the standard virtual model has a fixed size and cannot match various target objects, making the determined occlusion relationship inaccurate, and The virtual object and the target object do not fit well enough, affecting the authenticity of the image.
  • Embodiments of the present disclosure provide an image processing method, device, equipment and storage medium, which can add virtual objects to target objects and improve the authenticity of virtual objects.
  • an embodiment of the present disclosure provides an image processing method, including:
  • the set part map and the virtual object map are superimposed to obtain a target image.
  • embodiments of the present disclosure also provide an image processing device, including:
  • the initial mask image acquisition module is configured to segment the set parts of the target object and obtain the initial mask image
  • a depth map acquisition module configured to acquire a first depth map of the virtual object and a second depth map of the standard virtual model related to the target object;
  • a target mask map acquisition module configured to adjust the initial mask map based on the first depth map and the second depth map to obtain a target mask map
  • a rendering module configured to render the set part based on the target mask image to obtain a set part map; to render the virtual object to obtain a virtual object map;
  • the target image acquisition module is configured to superimpose the set part map and the virtual object map to obtain a target image.
  • embodiments of the present disclosure also provide an electronic device, where the electronic device includes:
  • a storage device configured to store programs
  • the processing device When the program is executed by the processing device, the processing device implements the image processing method as described in the embodiment of the present disclosure.
  • embodiments of the present disclosure also provide a computer-readable medium.
  • a computer program is stored on the computer-readable medium.
  • the image processing as described in the embodiments of the present disclosure is implemented. method.
  • Figure 1 is a flow chart of an image processing method in an embodiment of the present disclosure
  • Figure 2 is an initial mask image after segmenting the face in an embodiment of the present disclosure
  • Figure 3a is a depth map of a virtual object in an embodiment of the present disclosure
  • Figure 3b is a depth map of a standard virtual model in an embodiment of the present disclosure
  • Figure 4 is an example diagram of a two-dimensional diagram in an embodiment of the present disclosure
  • Figure 5 is an example diagram of an adjusted mask image in an embodiment of the present disclosure
  • Figure 6 is an example diagram of processing frame delay in an embodiment of the present disclosure.
  • Figure 7 is an example diagram of adding a virtual object to a target object in an embodiment of the present disclosure.
  • Figure 8 is a schematic structural diagram of an image processing device in an embodiment of the present disclosure.
  • Figure 9 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure.
  • the term “include” and its variations are open-ended, ie, “including but not limited to.”
  • the term “based on” means “based at least in part on.”
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment”; and the term “some embodiments” means “at least some embodiments”. Relevant definitions of other terms will be given in the description below.
  • Figure 1 is a flow chart of an image processing method provided by an embodiment of the present disclosure. This embodiment can be applied to the situation of adding virtual objects to images.
  • the method can be executed by an image processing device, which can be hardware and/or It is composed of software and can generally be integrated into a device with image processing functions.
  • the device can be an electronic device such as a server, mobile terminal or server cluster. As shown in Figure 1, the method may include the following steps:
  • S110 Segment the set parts of the target object to obtain an initial mask image.
  • the target object can be a real object recognized in the current scene, or a real object that needs to be added as a virtual object.
  • the target object can be a human body, plants, vehicles, buildings, etc.
  • the set part may be a part that has an occlusion relationship with the added virtual object, and may be determined based on the added position of the virtual object. For example: if the target object is the human body and the virtual object is added to the top of the head, the setting part can be the hair area. If the virtual object is added to the neck, the setting part can be the facial area, etc.
  • the process of segmenting a set part of the target object and obtaining the initial mask image may be: detecting the set part of the image containing the target object, and obtaining the confidence that the pixels in the image belong to the set part, Use the confidence as the pixel value of the pixel to obtain the initial mask image.
  • Figure 2 is an initial mask image after face segmentation in this embodiment. As shown in Figure 2, the white area represents the facial area, and the black area represents the non-face area. The face can be segmented through this initial mask.
  • S120 Obtain the first depth map of the virtual object and the second depth map of the standard virtual model related to the target object.
  • Virtual objects can be any constructed virtual objects, and virtual objects can be objects with irregular shapes.
  • virtual objects can be virtual animals (such as cats, dogs, etc.), virtual headwear, virtual earrings, and virtual objects that can be added in Virtual objects on the neck (such as virtual necklaces, virtual neck pillows, etc.).
  • the number of virtual objects can be set according to actual needs.
  • the virtual object can be a combination of a virtual animal and a virtual neck pillow, etc.
  • the standard virtual model can be a virtual model related to the target object, or a virtual model that can replace the target object for depth detection.
  • the standard virtual model may be a virtual model of the target object form, or a virtual model associated with the target object form, or a virtual model constructed based on the target object in the current frame.
  • the target object is a human head
  • the standard virtual model is a virtual model in the form of a human head
  • the form associated with the target object form can be a virtual model in the form of a cube or a cylinder.
  • the method of constructing a virtual model based on the target object of the current frame may be: scanning the target object in the current frame in three dimensions (3D), obtaining the 3D data of the target object, and building a standard virtual model based on the 3D data.
  • the amount of calculation can be reduced by using a virtual model associated with the target object form. Real-time construction of standard virtual models based on target objects can improve the accuracy of depth detection.
  • the first depth map of the virtual object may be the depth map after the virtual object is added to the target object;
  • the second depth map of the standard virtual model may be the depth map after the standard virtual model is added to the target object.
  • the method of obtaining the first depth map of the virtual object may be: tracking the added part of the object based on the set tracking algorithm, and obtaining the position information of the added part of the object; adding the virtual object to the added part of the object based on the position information; Obtain the depth information of the added virtual object and obtain the first depth map.
  • the object adding part is the part where the virtual object is added to the target object.
  • the set tracking algorithm may be a tracking algorithm of the related art.
  • Position information can be represented by a transformation matrix.
  • the tracking algorithm is set to obtain the transformation matrix corresponding to the added part of the object, and the matrix corresponding to the virtual object is multiplied by the transformation matrix, thereby realizing the addition of the virtual object to the added part of the object. operation, and finally use a virtual camera to obtain the depth information of the added virtual object, and obtain the A depth map.
  • Figure 3a is a depth map of a virtual object in this embodiment. As shown in Figure 3a, this figure is a depth map of a "virtual neck pillow".
  • adding the virtual object to the target object and obtaining the depth map can improve the accuracy of subsequent depth detection.
  • the method of obtaining the second depth map of the standard virtual model may be: tracking the set part based on the set tracking algorithm to obtain the position information of the set part; adding the standard virtual model to the set based on the position information Part; obtain the depth information of the added standard virtual model and obtain the second depth map.
  • the position information can be represented by a transformation matrix.
  • the setting part can be the face.
  • the set tracking algorithm obtains the transformation matrix corresponding to the set part, and multiplies the matrix corresponding to the standard virtual model with the transformation matrix, thereby realizing the addition of the standard virtual model to the set part.
  • the positioning operation is performed, and finally a virtual camera is used to obtain the depth information of the added standard virtual model to obtain the second depth map.
  • Figure 3b is a depth map of a standard virtual model in this embodiment. As shown in Figure 3b, this figure is a depth map of a "virtual human head".
  • the principle of adjusting the initial mask image based on the first depth map and the second depth map can be understood as: judging the corresponding position in the set location based on the depth values of the corresponding pixels in the first depth map and the second depth map. Whether the pixel is blocked by the virtual object. If it is blocked, adjust the pixel value of the corresponding pixel in the initial mask map so that the adjusted mask map reflects the blocking relationship between the virtual object and the set part of the target object.
  • the initial mask map is adjusted based on the first depth map and the second depth map.
  • the process of obtaining the target mask map may be: obtaining the near plane depth value and the far plane depth value of the virtual camera; according to the near plane depth Linearly transform the first depth map and the second depth map respectively using the first depth map and the far plane depth value; adjust the initial mask map based on the linearly transformed first depth map and second depth map to obtain the target mask map.
  • the near plane depth value and the far plane depth value can be directly obtained from the configuration information of the virtual camera (such as the field of view angle).
  • Performing linear transformation on the first depth map and the second depth map respectively can be understood as: transforming the depth values in the first depth map and the second depth map into the range of the near plane depth value and the far plane depth value.
  • the formula for linearly transforming the first depth map and the second depth map respectively can be expressed as: Among them, L(d) represents the depth value after linear transformation, d represents the depth value before linear transformation, zNear is the near plane depth value, and zFar is the far plane depth value.
  • the accuracy of adjusting the mask map can be improved by linearly transforming the depth values in the first depth map and the second depth map into the range of the near plane depth value and the far plane depth value.
  • the initial mask image is adjusted based on the first depth map and the second depth map to obtain the target
  • the way to mark the mask map may be: if the first depth value in the first depth map is greater than the second depth value in the second depth map, keep the pixel value of the corresponding pixel in the initial mask map unchanged; if If the first depth value is less than or equal to the second depth value, the pixel value of the corresponding pixel in the initial mask image is adjusted to the set value.
  • the first depth value may be characterized by one of the channel values (such as: R channel) in the first depth map;
  • the second depth value may be characterized by one of the channel values (such as: R channel) in the second depth map.
  • the pixel value of the pixel in the initial mask image can be characterized by one of the channel values (such as: R channel). Since the first depth map, the second depth map and the initial mask image are all grayscale images, and the values of the three color channels (red, green, blue, RGB) are equal, therefore, any channel value can be selected arbitrarily.
  • the setting value may be set to 0.
  • the first depth value in the first depth map is greater than the second depth value in the second depth map, it indicates that the virtual object is located behind the set part and the set part is not blocked by the virtual object, then at this time Keep the pixel value of the corresponding pixel in the initial mask map unchanged; if the first depth value is less than or equal to the second depth value, it indicates that the virtual object is in front of the set part and the virtual object will block the set part, then the initial mask will be The pixel value of the corresponding pixel in the film image is adjusted to 0. In this embodiment, if the virtual object blocks the set part, the pixel value of the corresponding pixel in the initial mask image is adjusted to 0, which can increase the speed of adjusting the mask image.
  • the initial mask map is adjusted based on the first depth map and the second depth map.
  • the target mask map may be obtained by: obtaining a two-dimensional map of the virtual object; if the first depth in the first depth map If the value is greater than the second depth value in the second depth map, the pixel value of the corresponding pixel in the initial mask map will be kept unchanged; if the first depth value is less than or equal to the second depth value, the corresponding pixel in the initial mask map will be The pixel value of the pixel is subtracted from the set channel value of the corresponding pixel in the two-dimensional image to obtain the final pixel value.
  • the method of obtaining the two-dimensional map of the virtual object may be to project the 3D points constituting the virtual object onto a two-dimensional plane to obtain the two-dimensional map corresponding to the virtual object.
  • the set channel may be the A channel in the two-dimensional image.
  • the two-dimensional image contains four channels, which are RGBA, where RGB represents three color channels, the A channel is the Alpha channel, and the value of the A channel is a value between 0 and 1, indicating the transparency of the pixel. If the A channel value is 0, it means that the pixel is transparent; if the A channel value is greater than 0, the pixel is not transparent.
  • FIG. 4 is an example diagram of a two-dimensional diagram in this embodiment. As shown in Figure 4, this picture is a two-dimensional diagram corresponding to the "virtual neck pillow". As shown in Figure 4, the black area is a transparent area, that is, the A channel value is 0.
  • first depth value in the first depth map is greater than the second depth value in the second depth map, it indicates that the virtual object is located behind the set part and the set part is not blocked by the virtual object, then it remains The pixel value of the corresponding pixel in the initial mask image remains unchanged. If the first depth value is less than or equal to the second depth value, it indicates that the virtual object is in front of the set part. The virtual object will block the set part by subtracting the pixel value of the corresponding pixel in the initial mask image from the corresponding pixel in the two-dimensional image.
  • the A channel value of the pixel Exemplarily, Figure 5 is an example of the adjusted mask image in this embodiment.
  • S140 Render the set part based on the target mask map to obtain the set part map; render the virtual object to obtain the virtual object map.
  • the target mask map can represent which pixels of the set part are blocked by the virtual object. Therefore, when rendering the set part based on the mask map, only the pixels that are not blocked can be rendered, or the pixels that will be blocked can be rendered. The transparency of the pixels is adjusted to 0.
  • the method of rendering the set part based on the target mask image may be: fusing the image information corresponding to the target mask image and the image information corresponding to the original image of the set part to obtain the fused image information; based on the fusion The image information renders the set parts.
  • the image information can be represented by a matrix with the same size as the image, and each element of the matrix represents the pixel value of the corresponding pixel point.
  • the method of fusing the image information corresponding to the target mask image and the image information corresponding to the original image of the set part can be: multiplying the pixel values of the corresponding pixels in the target mask image and the original image of the set part, Thus fused image information is obtained.
  • the fused image information includes the final pixel value of the pixel
  • the set part is rendered based on the final pixel value of the pixel.
  • the target mask image is fused with the original image and then rendered, so that the rendered set part image accurately reflects the occlusion relationship with the virtual object.
  • the method of rendering the set part based on the target mask map may be: determining the transparency information of the pixels in the original image of the set part based on the target mask map; and rendering the set part based on the transparency information.
  • the pixel value of the pixel in the target mask image represents transparency.
  • the pixel value of the pixel in the target mask map is a value between 0 and 1. If the pixel value is 0, it means that the corresponding pixel in the set part map is transparent. If the pixel value is greater than 0, it means that the pixel is transparent.
  • the pixel corresponding to this pixel in the set part map is non-transparent, and the transparency is determined by the actual pixel value. For example: if the pixel value is 1, the transparency is 100%, and if the pixel value is 0.5, the transparency is 50%.
  • the set part is rendered based on transparency information, which can reduce the amount of calculation.
  • the method of obtaining the first depth map of the virtual object may be: obtaining multiple first depth maps corresponding to the multiple virtual objects; accordingly,
  • the method of rendering virtual objects may be: rendering multiple virtual objects based on multiple first depth maps.
  • the process of rendering multiple virtual objects based on multiple first depth maps may be: comparing the depth values in the multiple depth maps, and rendering the pixels in the virtual object with the smallest depth value.
  • the depth maps are the first depth map a and the first depth map b respectively. For a pixel, if the first depth value a is less than the A depth value b, then render the pixel in virtual object A. In this embodiment, pixels in the virtual object with the smallest depth value are rendered, so that the rendered virtual objects reflect their respective occlusion relationships, thereby improving the authenticity of the image.
  • Obtaining the second depth map of the standard virtual model includes: using a virtual camera to obtain the standard virtual model and setting the depth information of the virtual object to obtain the second depth map.
  • the setting of the virtual object may be selected by the user based on the actual form of the virtual object (for example, earrings).
  • the depth information of the set virtual object and the standard virtual model is placed in the same depth map, so as to better match the set virtual object with the target.
  • Objects merge into one.
  • the same virtual camera is used to obtain the standard virtual model and set the depth information of the virtual object to obtain the second depth map.
  • S150 Overlay the set part map and the virtual object map to obtain the target image.
  • the method of overlaying the set part map and the virtual object map may be: superimposing the set part map onto the virtual object map.
  • the set part map and the virtual object map are rendered using different virtual cameras and are on different layers.
  • the set part map is superimposed on the virtual object map so that the set part map covers the virtual object. image to generate the target image.
  • the pixels in the area in the set part map are either transparent or not rendered, so the virtual object will not be blocked; when the first depth value When the depth value is greater than the second depth value, the pixels in this area have been rendered with color in the set part map, which will block the virtual object, making the target image more realistic.
  • the method further includes the following steps: caching the target mask image.
  • the set part is rendered based on the target mask map to obtain the set part map; the virtual object is rendered to obtain the virtual object map by: for the current frame, based on the target mask corresponding to the forward frame.
  • the film map renders the set part to obtain the set part map; the virtual object corresponding to the set forward frame is rendered to obtain the virtual object map.
  • the set forward frame may be a frame that is separated from the current frame by a set number of timings.
  • N can be determined according to the actual processing speed of each algorithm. For example, N can be 3.
  • the determined target mask image is first cached.
  • the target mask image corresponding to the set forward frame is obtained from the cache to render the set part; the set forward frame corresponding to Virtual objects are rendered. Since the first N frames have no algorithm data, the data determined at frame 0 can be used for rendering.
  • FIG. 6 is an example diagram of processing frame delay in this embodiment. As shown in Figure 6, render0-render3 are rendered using the data in buffer0, and subsequent renderings are rendered using the corresponding data three frames ago. In this embodiment, the problem of frame delay in the rendered picture can be avoided.
  • FIG. 7 is an example diagram of adding a virtual object to a target object in this embodiment, taking adding a virtual object on the neck of a human body as an example.
  • the terminal's camera collects human body images in real time, uses a neck tracking algorithm to track the neck, obtains the neck position, and mounts the virtual object to the neck position.
  • a depth camera is used to obtain the first depth image of the virtual object and the second depth image of the virtual head model.
  • one application scenario in this embodiment is to add a virtual animal (such as a kitten or a puppy) to the shoulders of a character.
  • a virtual animal such as a kitten or a puppy
  • the technical solution of the above embodiment is based on the rendering of virtual animals and human shoulders and human faces, thereby obtaining the special effect of adding virtual animals to human shoulders.
  • the virtual neck pillow can be mounted on the human neck.
  • the technical solutions of the above embodiments need to be used to determine the occlusion relationship between the virtual animal and the human neck and the human face.
  • the virtual neck pillow can be mounted on the human body's neck.
  • the pillow, the human neck and the human face are rendered to obtain the special effect of mounting the virtual neck pillow to the human neck.
  • the technical solution of the embodiment of the present disclosure is to segment the set parts of the target object to obtain an initial mask image; obtain the first depth map of the virtual object and the second depth map of the standard virtual model; based on the first depth map and the second The depth map adjusts the initial mask map to obtain the target mask map; renders the set part based on the target mask map to obtain the set part map; renders the virtual object to obtain the virtual object map; and converts the set part map Superimpose it on the virtual object map to obtain the target image.
  • the image processing method provided by the embodiment of the present disclosure renders the set part based on the target mask map, and superimposes the set part map onto the virtual object map, which can add the virtual object to the target object and improve the realism of the virtual object. sex.
  • FIG 8 is a schematic structural diagram of an image processing device disclosed in an embodiment of the present disclosure. As shown in Figure 8, the device includes:
  • the initial mask image acquisition module 810 is configured to segment the set parts of the target object and obtain the initial mask image
  • the depth map acquisition module 820 is configured to acquire the first depth map of the virtual object and the second depth map of the standard virtual model related to the target object;
  • the target mask map acquisition module 830 is configured to adjust the initial mask map based on the first depth map and the second depth map to obtain the target mask map;
  • the rendering module 840 is configured to render the set part based on the target mask map to obtain the set part map; to render the virtual object to obtain the virtual object map;
  • the target image acquisition module 850 is configured to superimpose the set part map and the virtual object map to obtain the target image.
  • the depth map acquisition module 820 is configured to obtain the first depth map of the virtual object in the following manner:
  • the standard virtual model is a virtual model of the target object form, or a virtual model associated with the target object form, or a virtual model constructed based on the target object of the current frame.
  • the target mask image acquisition module 830 is configured to obtain the target mask image in the following manner:
  • the initial mask image is adjusted based on the linearly transformed first depth map and the second depth map to obtain the target mask image.
  • the target mask image acquisition module 830 is also configured to obtain the target mask image in the following manner:
  • the pixel value of the corresponding pixel in the initial mask map is kept unchanged
  • the pixel value of the corresponding pixel in the initial mask image is adjusted to the set value.
  • the target mask image acquisition module 830 is also configured to obtain the target mask image in the following manner:
  • the pixel value of the corresponding pixel in the initial mask map is kept unchanged
  • the pixel value of the corresponding pixel in the initial mask image is subtracted from the set channel value of the corresponding pixel in the two-dimensional image to obtain the final pixel value.
  • the rendering module 840 is configured to render the set part based on the target mask image in the following manner:
  • the rendering module 840 is configured to render the set part based on the target mask image in the following manner:
  • the virtual object includes multiple depth map acquisition modules 820, which are configured to obtain the first depth map in the following manner:
  • the rendering module 840 is configured to render virtual objects in the following manner:
  • the device further includes: a virtual object determination module configured to: in response to the virtual object including a plurality of virtual objects, determine and set the virtual object from the plurality of virtual objects;
  • the depth map acquisition module 820 is configured to obtain the second depth map in the following manner:
  • a virtual camera is used to obtain the standard virtual model and the depth information of the set virtual object to obtain the second depth map.
  • the device also includes: a cache module, set to:
  • the rendering module 840 is configured to obtain the virtual object map in the following manner:
  • the set part is rendered based on the target mask map corresponding to the set forward frame to obtain the set part map; the virtual object corresponding to the set forward frame is rendered to obtain the virtual object map.
  • the above-mentioned device can execute the methods provided by all the foregoing embodiments of the present disclosure, and has corresponding functional modules and beneficial effects for executing the above-mentioned methods.
  • the methods provided by all the previous embodiments of this disclosure please refer to the methods provided by all the previous embodiments of this disclosure.
  • Electronic devices in embodiments of the present disclosure may include mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA), tablet computers (Portable Android Device, PAD), portable multimedia players (Portable Media Player, PMP), mobile terminals such as vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital televisions (also known as digital TVs), desktop computers, etc., or various forms of servers, such as independent servers or server clusters.
  • PDA Personal Digital Assistant
  • PMP portable multimedia players
  • mobile terminals such as vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital televisions (also known as digital TVs), desktop computers, etc., or various forms of servers, such as independent servers or server clusters.
  • the electronic device shown in FIG. 9 is only an example.
  • the electronic device 300 may include a processing device (such as a central processing unit, a graphics processor, etc.) 301.
  • the processing device 301 may process data according to a program stored in a read-only storage device (Read Only Memory, ROM) 302 or from a program.
  • the storage device 305 loads the program in the random access memory device (Random Access Memory, RAM) 303 to perform various appropriate actions and processes.
  • RAM random access memory
  • various programs and data required for the operation of the electronic device 300 are also stored.
  • the processing device 301, ROM 302 and RAM 303 are connected to each other via a bus 304.
  • An input/output (I/O) interface 305 is also connected to bus 304.
  • the following devices can be connected to the I/O interface 305: input devices 306 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a Liquid Crystal Display (LCD) , an output device 307 such as a speaker, a vibrator, etc.; a storage device 308 including a magnetic tape, a hard disk, etc.; and a communication device 309.
  • the communication device 309 may allow the electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data.
  • FIG. 9 illustrates electronic device 300 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program including program code for performing a recommendation method for words.
  • the computer program may be downloaded and installed from the network via communication device 309, or from storage device 305, or from ROM 302.
  • the processing device 301 When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or a combination of the above two.
  • the computer-readable storage medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination thereof.
  • Examples of computer readable storage media may include: an electrical connection having at least one conductor, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (e.g., electronically removable memory).
  • a computer-readable storage medium may be a tangible medium that contains or stores a program that may be used by or in connection with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take a variety of forms, including electromagnetic signals, optical signals, or a suitable combination of the above.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .
  • Program code contained on a computer-readable medium can be transmitted using any appropriate medium, including: wires, optical cables, radio frequency (Radio Frequency, RF), etc., or appropriate combinations of the above.
  • the client and server can communicate using currently known or future developed network protocols such as HyperText Transfer Protocol (HTTP), and can communicate with digital data in any form or medium (e.g., communications network) interconnection.
  • HTTP HyperText Transfer Protocol
  • Examples of communication networks include local area networks (LAN), wide area networks (WAN), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as currently known or networks for future research and development.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.
  • the computer-readable medium carries at least one program.
  • the electronic device segments the set parts of the target object to obtain an initial mask image; and obtains the first image of the virtual object.
  • Depth map and standard virtual model associated with the target object a second depth map; adjusting the initial mask map based on the first depth map and the second depth map to obtain a target mask map; rendering the set part based on the target mask map , obtain a set part map; render the virtual object to obtain a virtual object map; superimpose the set part map and the virtual object map to obtain a target image.
  • Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedures, or a combination thereof. programming language such as "C" or a similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through the Internet using an Internet service provider) .
  • LAN local area network
  • WAN wide area network
  • Internet service provider such as AT&T, MCI, Sprint, EarthLink, MSN, GTE, etc.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains at least one operable function for implementing the specified logical function.
  • Execute instructions may also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved.
  • each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.
  • the units involved in the embodiments of the present disclosure can be implemented in software or hardware. Among them, the name of a unit does not constitute a limitation on the unit itself under certain circumstances.
  • exemplary types of hardware logic components include: Field-Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP) ), System on Chip (SOC), Complex Programmable Logic Device (CPLD), etc.
  • FPGA Field-Programmable Gate Array
  • ASIC Application Specific Integrated Circuit
  • ASSP Application Specific Standard Parts
  • SOC System on Chip
  • CPLD Complex Programmable Logic Device
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
  • the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • Machine-readable media may include electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or suitable combinations of the foregoing.
  • machine-readable storage media may include Including electrical connections based on at least one wire, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable Compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or a suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • CD-ROM portable Compact disk read-only memory
  • magnetic storage device or a suitable combination of the foregoing.
  • an image processing method including:
  • the set part map and the virtual object map are superimposed to obtain a target image.
  • obtain the first depth map of the virtual object including:
  • the standard virtual model is a virtual model of the target object form, or a virtual model associated with the target object form, or a virtual model constructed based on the target object of the current frame.
  • adjusting the initial mask map based on the first depth map and the second depth map to obtain a target mask map includes:
  • the initial mask map is adjusted based on the linearly transformed first depth map and the second depth map to obtain a target mask map.
  • adjusting the initial mask map based on the first depth map and the second depth map to obtain a target mask map includes:
  • first depth value in the first depth map is greater than the second depth value in the second depth map, keep the pixel value of the corresponding pixel in the initial mask map unchanged;
  • the initial mask image is The pixel value of the corresponding pixel is adjusted to the set value.
  • adjusting the initial mask map based on the first depth map and the second depth map to obtain a target mask map includes:
  • first depth value in the first depth map is greater than the second depth value in the second depth map, keep the pixel value of the corresponding pixel in the initial mask map unchanged;
  • the first depth value is less than or equal to the second depth value, subtract the set channel value of the corresponding pixel in the two-dimensional image from the pixel value of the corresponding pixel in the initial mask image to obtain The final pixel value.
  • rendering the set part based on the target mask includes:
  • the set part is rendered based on the fused image information.
  • rendering the set part based on the target mask includes:
  • the transparency information of the pixels in the original image of the set part is determined according to the target mask map; wherein the pixel values of the pixels in the target mask map represent transparency;
  • the set part is rendered based on the transparency information.
  • the virtual objects include multiple, and obtaining the first depth map of the virtual objects includes:
  • Rendering the virtual object includes:
  • the plurality of virtual objects are rendered based on the plurality of first depth maps.
  • the method also includes:
  • Obtain the second depth map of the standard virtual model related to the target object including:
  • a virtual camera is used to obtain the depth information of the standard virtual model and the set virtual object, and a second depth map is obtained.
  • the method also includes:
  • the set part is rendered based on the target mask map corresponding to the set forward frame, Obtain the set part map; render the virtual object corresponding to the set forward frame to obtain the virtual object map.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Generation (AREA)

Abstract

本公开公开了一种图像处理方法、装置、设备及存储介质。该方法包括：对目标物体的设定部位进行分割，获得初始掩膜图；获取虚拟物体的第一深度图以及与所述目标物体相关的标准虚拟模型的第二深度图；基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图；基于所述目标掩膜图对所述设定部位进行渲染，获得设定部位图；对所述虚拟物体进行渲染，获得虚拟物体图；将所述设定部位图和所述虚拟物体图进行叠加，获得目标图像。

Description

图像处理方法、装置、设备及存储介质
本公开要求在2022年4月26日提交中国专利局、申请号为202210451633.9的中国专利申请的优先权，以上申请的全部内容通过引用结合在本公开中。
技术领域
本公开实施例涉及图像处理技术领域，例如涉及一种图像处理方法、装置、设备及存储介质。
背景技术
通过在检测到目标物体上添加虚拟物体是目前增强现实中常见的应用场景。该应用场景中，在添加虚拟物体时，需要确定虚拟物体与目标物体的遮挡关系，基于该遮挡关系对虚拟物体和目标物体进行渲染。
相关技术中，通过一标准虚拟模型来确定虚拟物体和目标物体的遮挡关系，这种方式由于标准虚拟模型为固定尺寸，无法与各种目标物体相匹配，使得确定出的遮挡关系不准确，且虚拟物体和目标物体不够贴合，影响图像的真实性。
发明内容
本公开实施例提供一种图像处理方法、装置、设备及存储介质，可以实现将虚拟物体添加至目标物体上，提高虚拟物体的真实性。
第一方面，本公开实施例提供了一种图像处理方法，包括：
对目标物体的设定部位进行分割，获得初始掩膜图；
获取虚拟物体的第一深度图以及与所述目标物体相关的标准虚拟模型的第二深度图；
基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图；
基于所述目标掩膜图对所述设定部位进行渲染，获得设定部位图；对所述虚拟物体进行渲染，获得虚拟物体图；
将所述设定部位图与所述虚拟物体图进行叠加，获得目标图像。
第二方面，本公开实施例还提供了一种图像处理装置，包括：
初始掩膜图获取模块，设置为对目标物体的设定部位进行分割，获得初始掩膜图；
深度图获取模块，设置为获取虚拟物体的第一深度图以及与所述目标物体相关的标准虚拟模型的第二深度图；
目标掩膜图获取模块，设置为基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图；
渲染模块，设置为基于所述目标掩膜图对所述设定部位进行渲染，获得设定部位图；对所述虚拟物体进行渲染，获得虚拟物体图；
目标图像获取模块，设置为将所述设定部位图和所述虚拟物体图进行叠加，获得目标图像。
第三方面，本公开实施例还提供了一种电子设备，所述电子设备包括：
处理装置；
存储装置，设置为存储程序；
在所述程序被所述处理装置执行时，所述处理装置实现如本公开实施例所述的图像处理方法。
第四方面，本公开实施例还提供了一种计算机可读介质，所述计算机可读介质上存储有计算机程序，所述计算机程序被处理装置执行时实现如本公开实施例所述的图像处理方法。
附图说明
图1是本公开实施例中一种图像处理方法的流程图；
图2是本公开实施例中的对面部进行分割后初始掩膜图；
图3a是本公开实施例中的一种虚拟物体的深度图；
图3b是本公开实施例中的一种标准虚拟模型的深度图；
图4是本公开实施例中的一种二维图的示例图；
图5是本公开实施例中的调整后的掩膜图的示例图；
图6是本公开实施例中的处理帧延迟的示例图；
图7是本公开实施例中的在目标物体添加虚拟物体的示例图；
图8是本公开实施例中的一种图像处理装置的结构示意图；
图9是本公开实施例中的一种电子设备的结构示意图。
具体实施方式
下面将参照附图描述本公开的实施例。虽然附图中显示了本公开的一些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例。应当理解的是，本公开的附图及实施例仅用于示例性作用。
应当理解，本公开的方法实施方式中记载的各个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
需要注意，本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
需要注意，本公开中提及的“一个”、“多个”的修饰是示意性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
图1为本公开实施例提供的一种图像处理方法的流程图，本实施例可适用于在图像中添加虚拟物体的情况，该方法可以由图像处理装置来执行，该装置可由硬件和/或软件组成，并一般可集成在具有图像处理功能的设备中，该设备可以是服务器、移动终端或服务器集群等电子设备。如图1所示，该方法可以包括如下步骤：
S110，对目标物体的设定部位进行分割，获得初始掩膜图。
目标物体可以是当前场景中识别到的真实物体，或者设置的需要添加虚拟物体的真实物体，例如：目标物体可以是人体、植物、车辆和建筑物等。设定部位可以是与添加的虚拟物体存在遮挡关系的部位，可以根据虚拟物体的添加位置确定。例如：若目标物体是人体，且虚拟物体添加在头顶，则设定部位可以是头发区域，若虚拟物体添加在脖颈，则设定部位可以是面部区域等。
示例性的，对目标物体的设定部位进行分割，获得初始掩膜图的过程可以是：对包含目标物体的图像进行设定部位的检测，获取图像中像素点属于设定部位的置信度，将置信度作为像素点的像素值，从而获得初始掩膜图。示例性的，图2是本实施例中对面部进行分割后初始掩膜图，如图2所述，白色区域代表面部区域，黑色区域为非面部区域。通过该初始掩膜图可以将面部分割出来。
S120，获取虚拟物体的第一深度图以及与目标物体相关的标准虚拟模型的第二深度图。
虚拟物体可以是任意的构建出的虚拟物体，且虚拟物体可以是形状不规则的物体，例如：虚拟物体可以是虚拟动物(如：猫、狗等)、虚拟头饰、虚拟耳饰以及可以添加在脖子上的虚拟物体(如虚拟项链、虚拟脖枕等)等。本实施例中，虚拟物体的数量可以根据实际需求设置，例如：虚拟物体可以是虚拟动物与虚拟脖枕的结合体等。
标准虚拟模型可以是与目标物体相关的虚拟模型，可以代替目标物体做深度检测的虚拟模型。标准虚拟模型可以是目标物体形态的虚拟模型，或者为与目标物体形态相关联的虚拟模型，或者为基于当前帧的目标物体构建的虚拟模型。例如：若目标物体是人体头部，则标准虚拟模型则为人头形态的虚拟模型，与目标物体形态相关联的形态可以是立方体形态、圆柱体形态的虚拟模型。其中，目标物体形态的虚拟模型和与目标物体形态相关联的虚拟模型可以是预先创建好的。基于当前帧的目标物体构建虚拟模型可以理解为：基于目标物体实时构建标准虚拟模型。
基于当前帧的目标物体构建虚拟模型的方式可以是：对当前帧的目标物体进行三度空间(3 Dimensions，3D)扫描，获得目标物体的3D数据，基于3D数据构建标准虚拟模型。本实施例中，采用与目标物体形态相关联的虚拟模型，可以降低计算量。基于目标物体实时构建标准虚拟模型可以提高深度检测的准确性。
本实施例中，虚拟物体的第一深度图可以是虚拟物体添加到目标物体后的深度图；标准虚拟模型的第二深度图可以是标准虚拟模型添加至目标物体后的深度图。
本实施例中，获取虚拟物体的第一深度图的方式可以是：基于设定跟踪算法对物体添加部位进行跟踪，获得物体添加部位的位置信息；基于位置信息将虚拟物体添加至物体添加部位；获取添加后的虚拟物体的深度信息，获得第一深度图。
其中，物体添加部位为虚拟物体添加至目标物体的部位。设定跟踪算法可以是相关技术的跟踪算法。位置信息可以由变换矩阵表征。示例性的，设定跟踪算法在对物体添加部位跟踪的过程中，获得物体添加部位对应的变换矩阵，将虚拟物体对应的矩阵与该变换矩阵进行相乘，实现了虚拟物体添加至物体添加部位的操作，最后采用虚拟相机获取添加后的虚拟物体的深度信息，获得第 一深度图。示例性，图3a是本实施例中一种虚拟物体的深度图，如图3a所示，该图是一个“虚拟脖枕”的深度图。本实施例中，将虚拟物体添加至目标物体上在获取深度图，可以提高后续深度检测的准确性。
本实施例中，获取标准虚拟模型的第二深度图的方式可以是：基于设定跟踪算法对设定部位进行跟踪，获得设定部位的位置信息；基于位置信息将标准虚拟模型添加至设定部位；获取添加后的标准虚拟模型的深度信息，获得第二深度图。
其中，位置信息可以由变换矩阵表征。设定部位可以是面部。示例性的，设定跟踪算法在对设定部位跟踪的过程中，获得设定部位对应的变换矩阵，将标准虚拟模型对应的矩阵与该变换矩阵进行相乘，实现了标准虚拟模型添加至设定部位的操作，最后采用虚拟相机获取添加后的标准虚拟模型的深度信息，获得第二深度图。示例性，图3b是本实施例中一种标准虚拟模型的深度图，如图3b所示，该图是一个“虚拟人头”的深度图。
S130，基于第一深度图及第二深度图对初始掩膜图进行调整，获得目标掩膜图。
本实施例中，基于第一深度图及第二深度图对初始掩膜图进行调整原理可以理解为：根据第一深度图和第二深度图中对应像素点的深度值判断设定部位中对应像素点是否被虚拟物体遮挡，若被遮挡，则将初始掩膜图中对应像素点的像素值进行调整，使得调整后的掩膜图体现虚拟物体与目标物体设定部位的遮挡关系。
可选的，基于第一深度图及第二深度图对初始掩膜图进行调整，获得目标掩膜图的过程可以是：获取虚拟相机的近平面深度值和远平面深度值；根据近平面深度值和远平面深度值对第一深度图及第二深度图分别进行线性变换；基于线性变换后的第一深度图和第二深度图对初始掩膜图进行调整，获得目标掩膜图。
其中，近平面深度值和远平面深度值可以从虚拟相机的配置信息(如视场角)中直接获取到。对第一深度图及第二深度图分别进行线性变换可以理解为：将第一深度图及第二深度图中的深度值变换至近平面深度值和远平面深度值的范围内。
可选的，对第一深度图及第二深度图分别进行线性变换的公式可以表示为：其中，L(d)表示线性变换后的深度值，d表示线性变换前的深度值，zNear为近平面深度值，zFar为远平面深度值。本实施例中，将第一深度图及第二深度图中的深度值线性变换至近平面深度值和远平面深度值的范围内，可以提高调整掩膜图的准确性。
示例性的，基于第一深度图及第二深度图对初始掩膜图进行调整，获得目 标掩膜图的方式可以是：若第一深度图中的第一深度值大于第二深度图中的第二深度值，则保持初始掩膜图中对应像素点的像素值不变；若第一深度值小于或者等于第二深度值，则将初始掩膜图中对应像素点的像素值调整为设定值。
其中，第一深度值可以是由第一深度图中的其中一个通道值(如：R通道)表征；第二深度值可以由第二深度图中的其中一个通道值(如：R通道)表征；初始掩膜图中像素点的像素值可以由其中一个通道值(如：R通道)表征。由于第一深度图、第二深度图及初始掩膜图均为灰度图，三个颜色通道(红绿蓝，RGB)的值相等，因此，可以任意选择一个通道值。
在一实施例中，设定值可以设置为0。本实施例中，若第一深度图中的第一深度值大于第二深度图中的第二深度值，表明虚拟物体位于设定部位的后面，设定部位未被虚拟物体遮挡，则此时保持初始掩膜图中对应像素点的像素值不变；若第一深度值小于或者等于第二深度值，表明虚拟物体位于设定部位的前面，虚拟物体将设定部位遮挡，则将初始掩膜图中对应像素点的像素值调整为0。本实施例中，虚拟物体将设定部位遮挡，则将初始掩膜图中对应像素点的像素值调整为0，可以提高调整掩膜图的速度。
可选的，基于第一深度图及第二深度图对初始掩膜图进行调整，获得目标掩膜图的方式可以是：获取虚拟物体的二维图；若第一深度图中的第一深度值大于第二深度图中的第二深度值，则保持初始掩膜图中对应像素点的像素值不变；若第一深度值小于或者等于第二深度值，则将初始掩膜图中对应像素点的像素值减去二维图中对应像素点的设定通道值，获得最终的像素值。
在一实施例中，获取虚拟物体的二维图的方式可以是：将构成虚拟物体的3D点投影至二维平面，获得虚拟物体对应的二维图。其中，设定通道可以是二维图像中的A通道。本实施例中，二维图包含四个通道，为RGBA，其中，RGB表示三颜色通道，A通道为Alpha通道，A通道的值为0-1之间的值，表示像素点的透明度。若A通道值为0，表明该像素点透明，若A通道值大于0，则该像素点非透明。示例性的，图4是本实施例中一种二维图的示例图。如图4所示，该图为“虚拟脖枕”对应的二维图，如图4所示，黑色区域为透明区域，即A通道值为0。
示例性的，若第一深度图中的第一深度值大于第二深度图中的第二深度值，表明虚拟物体位于设定部位的后面，设定部位未被虚拟物体遮挡，则此时保持初始掩膜图中对应像素点的像素值不变。若第一深度值小于或者等于第二深度值，表明虚拟物体位于设定部位的前面，虚拟物体将设定部位遮挡，将初始掩膜图中对应像素点的像素值减去二维图中对应像素点的A通道值。示例性的，图5是本实施例中调整后的掩膜图的示例图，从图中可以看出，相对于图2中的初始掩膜图，图5中的目标掩膜图中，设定部位被虚拟物体遮挡区域像素点的像素值进行了调整。本实施例中，若虚拟物体将设定部位遮挡，将初始掩膜图中对应像素点的像素值减去二维图中对应像素点的A通道值，可以提高边缘的平滑性， 使得遮挡过渡平滑，使得渲染出的图像更真实。
S140，基于目标掩膜图对设定部位进行渲染，获得设定部位图；对虚拟物体进行渲染，获得虚拟物体图。
本实施例中，目标掩膜图可以表征设定部位哪些像素点被虚拟物体遮挡，因此，在基于掩膜图渲染设定部位时，可以只渲染未被遮挡的像素点，或者，将被遮挡的像素点的透明度调整为0。
可选的，基于目标掩膜图对设定部位进行渲染的方式可以是：将目标掩膜图对应的图像信息与设定部位的原始图对应的图像信息进行融合，获得融合图像信息；基于融合图像信息对设定部位进行渲染。
在一实施例中，图像信息可以由与图像尺寸相同的矩阵表示，矩阵的每个元素表示对应像素点的像素值。将目标掩膜图对应的图像信息与设定部位的原始图对应的图像信息进行融合的方式可以是：将目标掩膜图与设定部位的原始图中对应像素点的像素值进行相乘，从而获得融合图像信息。本实施例中，由于融合图像信息中包含像素点最终的像素值，基于像素点最终的像素值渲染设定部位。本实施例中，将目标掩膜图与原始图融合后再渲染，使得渲染出的设定部位图准确的体现出与虚拟物体的遮挡关系。
可选的，基于目标掩膜图对设定部位进行渲染的方式可以是：根据目标掩膜图确定设定部位的原始图中像素点的透明度信息；基于透明度信息对设定部位进行渲染。
其中，目标掩膜图中像素点的像素值表征透明度。目标掩膜图中像素点的像素值为0-1之间的值，若像素值为0，则表明该像素点在设定部位图中对应的像素点透明，若像素值大于0，则表明该像素点在设定部位图中对应的像素点为非透明，透明度由实际像素值确定。例如：若像素值为1，则透明图为100％，若像素值为0.5，则透明度为50％。本实施例，基于透明度信息对设定部位进行渲染，可以减少计算量。
可选的，若虚拟物体包括多个，则获取虚拟物体的第一深度图的方式可以是：获取多个虚拟物体分别对应的多张第一深度图；相应的，
对虚拟物体进行渲染的方式可以是：基于多张第一深度图对多个虚拟物体进行渲染。
示例性的，基于多张第一深度图对多个虚拟物体进行渲染的过程可以是：比较多张深度图中的深度值，渲染深度值最小的虚拟物体中的像素点。假设：包含有两个虚拟物体，分别为虚拟物体A和虚拟物体B，对于的深度图分别为第一深度图a和第一深度图b，对于一个像素点，若第一深度值a小于第一深度值b，则渲染虚拟物体A中的该像素点。本实施例中，渲染深度值最小的虚拟物体中的像素点，使得渲染出的虚拟物体体现出各自的遮挡关系，提高图像的真实性。
可选的，若虚拟物体包括多个，则从多个虚拟物体中确定设定虚拟物体； 获取标准虚拟模型的第二深度图，包括：采用虚拟相机获取标准虚拟模型和设定虚拟物体的深度信息，获得第二深度图。
其中，设定虚拟物体可以是用户根据虚拟物体的实际形态(例如：耳饰)选择的。为了保证设定虚拟物体与目标物体更贴合，在进行深度检测时，将设定虚拟物体和标准虚拟模型的深度信息放置于同一张深度图中，从而更好的将设定虚拟物体与目标物体融为一体。示例性的，采用同一个虚拟相机获取标准虚拟模型和设定虚拟物体的深度信息，获得第二深度图。
S150，将设定部位图和虚拟物体图进行叠加，获得目标图像。
其中，将设定部位图和虚拟物体图进行叠加的方式可以是：将设定部位图叠加至虚拟物体图上。本实施例中，设定部位图和虚拟物体图是采用不同的虚拟相机渲染处理的，分别处于不同的图层，将设定部位图叠加至虚拟物体图上，使得设定部位图覆盖虚拟物体图，从而生成目标图像。本实施例中，当第一深度值小于或等于第二深度值时，设定部位图中已经将该区域的像素点或为透明或不渲染，因此，虚拟物体不会被遮挡；当第一深度值大于第二深度值时，设定部位图中已经将该区域的像素点渲染出颜色，会将虚拟物体遮挡住，使得目标图像更真实。
可选的，在获得目标掩膜图之后，该方法还包括如下步骤：将目标掩膜图进行缓存。相应的，基于目标掩膜图对设定部位进行渲染，获得设定部位图；对虚拟物体进行渲染，获得虚拟物体图的方式可以是：对于当前帧，基于设定前向帧对应的目标掩膜图对设定部位进行渲染，获得设定部位图；对设定前向帧对应的虚拟物体进行渲染，获得虚拟物体图。
其中，设定前向帧可以是与当前帧间隔设定数量的时序靠前的帧。本实施例中，由于不同算法的处理速度存在差异，存在帧延迟问题，为了保证整体算法结果符合实际显示效果，会将屏幕当前显示的画面替换为N帧前的画面。其中，N可以根据各算法的实际处理速度确定，例如：N可以是3。
本实施例中，首先将确定出的目标掩膜图进行缓存，对于当前帧，从缓存中获取设定前向帧对应的目标掩膜图对设定部位进行渲染；对设定前向帧对应的虚拟物体进行渲染。由于最开始的N帧没有算法数据，可以均采用第0帧确定的数据渲染。示例性的，图6是本实施例中处理帧延迟的示例图。如图6所示，render0-render3均采用buffer0中的数据进行渲染，之后的渲染画面，采用其三帧前对应的数据渲染。本实施例中，可以避免渲染出的画面出现帧延迟的问题。
实例性的，图7是本实施例中在目标物体添加虚拟物体的示例图，以在人体脖子上添加虚拟物体为例。如图7所示，终端的相机实时采集人体图像，采用颈部跟踪算对脖颈进行跟踪，获得脖颈位置，将虚拟物体挂载至脖颈位置处。采用面部跟踪算法对面部进行跟踪，获得面部位置，基于面部位置挂载虚拟人头模型。采用面部分割算法对面部进行分割，获得初始面部掩膜图。采用深度相机获取虚拟物体的第一深度图和虚拟人头模型的第二深度图。比较第一深度图 和第二深度图的深度值，基于比较结果调整初始面部掩膜图中的像素值，获得目标面部掩膜图。在不同的图层分别渲染面部图像和虚拟物体图，将面部图像叠加至虚拟物体图上，获得颈部遮挡效果图。
示例性的，本实施例中的一个应用场景为将虚拟动物(如小猫或者小狗)添加至人物的肩膀上，此时需要采用上述实施例技术方案确定出虚拟动物和人体肩膀及人体面部的遮挡关系，基于该遮挡关系对虚拟动物和人体肩膀及人体面部的渲染，从而获得将虚拟动物添加至人体肩膀的特效。在另一个应用场景中，可以是将虚拟脖枕挂载于人体脖颈处，此时需要采用上述实施例技术方案确定出虚拟动物和人体脖颈及人体面部的遮挡关系，基于该遮挡关系对虚拟脖枕和人体脖颈及人体面部的渲染，从而获得将虚拟脖枕挂载至人体脖颈的特效。
本公开实施例的技术方案，对目标物体的设定部位进行分割，获得初始掩膜图；获取虚拟物体的第一深度图以及标准虚拟模型的第二深度图；基于第一深度图及第二深度图对初始掩膜图进行调整，获得目标掩膜图；基于目标掩膜图对设定部位进行渲染，获得设定部位图；对虚拟物体进行渲染，获得虚拟物体图；将设定部位图叠加至虚拟物体图上，获得目标图像。本公开实施例提供的图像处理方法，基于目标掩膜图对设定部位进行渲染，将设定部位图叠加至虚拟物体图上，可以实现将虚拟物体添加至目标物体上，提高虚拟物体的真实性。
图8是本公开实施例公开的一种图像处理装置的结构示意图，如图8所示，该装置包括：
初始掩膜图获取模块810，设置为对目标物体的设定部位进行分割，获得初始掩膜图；
深度图获取模块820，设置为获取虚拟物体的第一深度图以及与目标物体相关的标准虚拟模型的第二深度图；
目标掩膜图获取模块830，设置为基于第一深度图及第二深度图对初始掩膜图进行调整，获得目标掩膜图；
渲染模块840，设置为基于目标掩膜图对设定部位进行渲染，获得设定部位图；对虚拟物体进行渲染，获得虚拟物体图；
目标图像获取模块850，设置为将设定部位图和虚拟物体图进行叠加，获得目标图像。
可选的，深度图获取模块820，设置为通过以下方式获取虚拟物体的第一深度图：
基于设定跟踪算法对物体添加部位进行跟踪，获得物体添加部位的位置信息；其中，物体添加部位为目标物体的添加虚拟物体添的部位；
基于位置信息将虚拟物体添加至物体添加部位；
获取添加后的虚拟物体的深度信息，获得第一深度图。
可选的，标准虚拟模型为目标物体形态的虚拟模型，或者为与所述目标物体形态相关联的虚拟模型，或者为基于当前帧的目标物体构建的虚拟模型。
可选的，目标掩膜图获取模块830，设置为通过以下方式获得目标掩膜图：
获取虚拟相机的近平面深度值和远平面深度值；
根据近平面深度值和远平面深度值对第一深度图及第二深度图分别进行线性变换；
基于线性变换后的第一深度图和第二深度图对初始掩膜图进行调整，获得目标掩膜图。
可选的，目标掩膜图获取模块830，还设置为通过以下方式获得目标掩膜图：
若第一深度图中的第一深度值大于第二深度图中的第二深度值，则保持初始掩膜图中对应像素点的像素值不变；
若第一深度值小于或者等于第二深度值，则将初始掩膜图中对应像素点的像素值调整为设定值。
可选的，目标掩膜图获取模块830，还设置为通过以下方式获得目标掩膜图：
获取虚拟物体的二维图；
若第一深度图中的第一深度值大于第二深度图中的第二深度值，则保持初始掩膜图中对应像素点的像素值不变；
若第一深度值小于或者等于第二深度值，则将初始掩膜图中对应像素点的像素值减去二维图中对应像素点的设定通道值，获得最终的像素值。
可选的，渲染模块840，设置为通过以下方式基于目标掩膜图对设定部位进行渲染：
将目标掩膜图对应的图像信息与设定部位的原始图对应的图像信息进行融合，获得融合图像信息；
基于融合图像信息对设定部位进行渲染。
可选的，渲染模块840，设置为通过以下方式基于目标掩膜图对设定部位进行渲染：
根据目标掩膜图确定设定部位的原始图中像素点的透明度信息；其中，目标掩膜图中像素点的像素值表征透明度；
基于透明度信息对设定部位进行渲染。
可选的，虚拟物体包括多个，深度图获取模块820，设置为通过以下方式获得第一深度图：
获取多个虚拟物体分别对应的多张第一深度图；
渲染模块840，设置为通过以下方式对虚拟物体进行渲染：
基于多张第一深度图对多个虚拟物体进行渲染。
可选的，该装置还包括：虚拟物体确定模块，设置为：响应于虚拟物体包括多个，从多个虚拟物体中确定设定虚拟物体；
深度图获取模块820，设置为通过以下方式获得第二深度图：
采用虚拟相机获取标准虚拟模型和设定虚拟物体的深度信息，获得第二深度图。
可选的，该装置还包括：缓存模块，设置为：
将目标掩膜图进行缓存；
渲染模块840，设置为通过以下方式获得虚拟物体图：
对于当前帧，基于设定前向帧对应的目标掩膜图对设定部位进行渲染，获得设定部位图；对设定前向帧对应的虚拟物体进行渲染，获得虚拟物体图。
上述装置可执行本公开前述所有实施例所提供的方法，具备执行上述方法相应的功能模块和有益效果。未在本实施例中详尽描述的技术细节，可参见本公开前述所有实施例所提供的方法。
下面参考图9，其示出了适于用来实现本公开实施例的电子设备300的结构示意图。本公开实施例中的电子设备可以包括诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，PDA)、平板电脑(Portable Android Device，PAD)、便携式多媒体播放器(Portable Media Player，PMP)、车载终端(例如车载导航终端)等的移动终端以及诸如数字电视机(也即数字TV)、台式计算机等的固定终端，或者各种形式的服务器，如独立服务器或者服务器集群。图9示出的电子设备仅仅是一个示例。
如图9所示，电子设备300可以包括处理装置(例如中央处理器、图形处理器等)301，处理装置301可以根据存储在只读存储装置(Read Only Memory，ROM)302中的程序或者从存储装置305加载到随机访问存储装置(Random Access Memory，RAM)303中的程序而执行各种适当的动作和处理。在RAM 303中，还存储有电子设备300操作所需的各种程序和数据。处理装置301、ROM 302以及RAM 303通过总线304彼此相连。输入/输出(Input/Output，I/O)接口305也连接至总线304。
通常，以下装置可以连接至I/O接口305：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置306；包括例如液晶显示器(Liquid Crystal Display，LCD)、扬声器、振动器等的输出装置307；包括例如磁带、硬盘等的存储装置308；以及通信装置309。通信装置309可以允许电子设备300与其他设备进行无线或有线通信以交换数据。虽然图9示出了具有各种装置的电子设备300，但是应理解的是，并不要求实施或具备所有示出的装置。 可以替代地实施或具备更多或更少的装置。
在一实施例中，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行词语的推荐方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置309从网络上被下载和安装，或者从存储装置305被安装，或者从ROM 302被安装。在该计算机程序被处理装置301执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的组合。计算机可读存储介质例如可以是电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者以上的组合。计算机可读存储介质的示例可以包括：具有至少一个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(如电子可编程只读存储器(Electronic Programable Read Only Memory，EPROM)或闪存)、光纤、便携式紧凑磁盘只读存储器(Compact Disc-Read Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的合适的组合。在本公开中，计算机可读存储介质可以是包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括电磁信号、光信号或上述的合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括：电线、光缆、射频(Radio Frequency，RF)等，或者上述的合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如超文本传输协议(HyperText Transfer Protocol，HTTP)之类的当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(Local Area Network，LAN)，广域网(Wide Area Network，WAN)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有至少一个程序，当上述至少一个程序被该电子设备执行时，使得该电子设备：对目标物体的设定部位进行分割，获得初始掩膜图；获取虚拟物体的第一深度图以及与所述目标物体相关的标准虚拟模型的 第二深度图；基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图；基于所述目标掩膜图对所述设定部位进行渲染，获得设定部位图；对所述虚拟物体进行渲染，获得虚拟物体图；将所述设定部位图和所述虚拟物体图进行叠加，获得目标图像。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络包括局域网(LAN)或广域网(WAN)连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含至少一个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定。
本文中以上描述的功能可以至少部分地由至少一个硬件逻辑部件来执行。例如，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(Field-Programmable Gate Array，FPGA)、专用集成电路(Application Specific Integrated Circuit，ASIC)、专用标准产品(Application Specific Standard Parts，ASSP)、片上系统(System on Chip，SOC)、复杂可编程逻辑设备(Complex Programmable Logic Device，CPLD)等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的合适组合。机器可读存储介质的示例可以包 括基于至少一个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的合适组合。
根据本公开实施例的一个或多个实施例，本公开实施例公开了一种图像处理方法，包括：
对目标物体的设定部位进行分割，获得初始掩膜图；
获取虚拟物体的第一深度图以及与所述目标物体相关的标准虚拟模型的第二深度图；
基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图；
基于所述目标掩膜图对所述设定部位进行渲染，获得设定部位图；对所述虚拟物体进行渲染，获得虚拟物体图；
将所述设定部位图和所述虚拟物体图进行叠加，获得目标图像。
进一步地，获取虚拟物体的第一深度图，包括：
基于设定跟踪算法对物体添加部位进行跟踪，获得物体添加部位的位置信息；其中，所述物体添加部位为所述目标物体的添加所述虚拟物体的部位；
基于所述位置信息将所述虚拟物体添加至所述物体添加部位；
获取添加后的所述虚拟物体的深度信息，获得第一深度图。
进一步地，所述标准虚拟模型为所述目标物体形态的虚拟模型，或者为与所述目标物体形态相关联的虚拟模型，或者为基于当前帧的目标物体构建的虚拟模型。
进一步地，基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图，包括：
获取虚拟相机的近平面深度值和远平面深度值；
根据所述近平面深度值和所述远平面深度值对第一深度图及所述第二深度图分别进行线性变换；
基于线性变换后的第一深度图和第二深度图对所述初始掩膜图进行调整，获得目标掩膜图。
进一步地，基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图，包括：
若所述第一深度图中的第一深度值大于所述第二深度图中的第二深度值，则保持所述初始掩膜图中对应像素点的像素值不变；
若所述第一深度值小于或者等于所述第二深度值，则将所述初始掩膜图中 对应像素点的像素值调整为设定值。
进一步地，基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图，包括：
获取所述虚拟物体的二维图；
若所述第一深度图中的第一深度值大于所述第二深度图中的第二深度值，则保持所述初始掩膜图中对应像素点的像素值不变；
若所述第一深度值小于或者等于所述第二深度值，则将所述初始掩膜图中对应像素点的像素值减去所述二维图中对应像素点的设定通道值，获得最终的像素值。
进一步地，基于所述目标掩膜图对所述设定部位进行渲染，包括：
将所述目标掩膜图对应的图像信息与所述设定部位的原始图对应的图像信息进行融合，获得融合图像信息；
基于所述融合图像信息对所述设定部位进行渲染。
进一步地，基于所述目标掩膜图对所述设定部位进行渲染，包括：
根据所述目标掩膜图确定所述设定部位的原始图中像素点的透明度信息；其中，所述目标掩膜图中像素点的像素值表征透明度；
基于所述透明度信息对所述设定部位进行渲染。
进一步地，所述虚拟物体包括多个，所述获取虚拟物体的第一深度图，包括：
获取多个虚拟物体分别对应的多张第一深度图；
对所述虚拟物体进行渲染，包括：
基于所述多张第一深度图对所述多个虚拟物体进行渲染。
进一步地，该方法还包括：
响应于所述虚拟物体包括多个，从多个虚拟物体中确定设定虚拟物体；
获取与所述目标物体相关的标准虚拟模型的第二深度图，包括：
采用虚拟相机获取所述标准虚拟模型和所述设定虚拟物体的深度信息，获得第二深度图。
进一步地，在获得目标掩膜图之后，该方法还包括：
将所述目标掩膜图进行缓存；
基于所述目标掩膜图对所述设定部位进行渲染，获得设定部位图；对所述虚拟物体进行渲染，获得虚拟物体图，包括：
对于当前帧，基于设定前向帧对应的目标掩膜图对所述设定部位进行渲染， 获得设定部位图；对设定前向帧对应的虚拟物体进行渲染，获得虚拟物体图。
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本公开中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本公开的技术方案所期望的结果。

Claims (14)

  1. 一种图像处理方法，包括：
    对目标物体的设定部位进行分割，获得初始掩膜图；
    获取虚拟物体的第一深度图以及与所述目标物体相关的标准虚拟模型的第二深度图；
    基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图；
    基于所述目标掩膜图对所述设定部位进行渲染，获得设定部位图；对所述虚拟物体进行渲染，获得虚拟物体图；
    将所述设定部位图和所述虚拟物体图进行叠加，获得目标图像。
  2. 根据权利要求1所述的方法，其中，所述获取虚拟物体的第一深度图，包括：
    基于设定跟踪算法对物体添加部位进行跟踪，获得所述物体添加部位的位置信息；其中，所述物体添加部位为所述目标物体的添加所述虚拟物体的部位；
    基于所述位置信息将所述虚拟物体添加至所述物体添加部位；
    获取添加后的所述虚拟物体的深度信息，获得第一深度图。
  3. 根据权利要求1所述的方法，其中，所述标准虚拟模型为所述目标物体形态的虚拟模型，或者所述标准虚拟模型为与所述目标物体形态相关联的虚拟模型，或者所述标准虚拟模型为基于当前帧的目标物体构建的虚拟模型。
  4. 根据权利要求1所述的方法，其中，所述基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图，包括：
    获取虚拟相机的近平面深度值和远平面深度值；
    根据所述近平面深度值和所述远平面深度值对所述第一深度图及所述第二深度图分别进行线性变换；
    基于线性变换后的第一深度图和第二深度图对所述初始掩膜图进行调整，获得目标掩膜图。
  5. 根据权利要求1或4所述的方法，其中，所述基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图，包括：
    响应于所述第一深度图中的第一深度值大于所述第二深度图中的第二深度值，保持所述初始掩膜图中对应像素点的像素值不变；
    响应于所述第一深度值小于或者等于所述第二深度值，将所述初始掩膜图中对应像素点的像素值调整为设定值。
  6. 根据权利要求1或4所述的方法，其中，所述所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图，包括：
    获取所述虚拟物体的二维图；
    响应于所述第一深度图中的第一深度值大于所述第二深度图中的第二深度值，保持所述初始掩膜图中对应像素点的像素值不变；
    响应于所述第一深度值小于或者等于所述第二深度值，将所述初始掩膜图中对应像素点的像素值减去所述二维图中对应像素点的设定通道值，获得最终的像素值。
  7. 根据权利要求1所述的方法，其中，所述基于所述目标掩膜图对所述设定部位进行渲染，包括：
    将所述目标掩膜图对应的图像信息与所述设定部位的原始图对应的图像信息进行融合，获得融合图像信息；
    基于所述融合图像信息对所述设定部位进行渲染。
  8. 根据权利要求1所述的方法，其中，所述基于所述目标掩膜图对所述设定部位进行渲染，包括：
    根据所述目标掩膜图确定所述设定部位的原始图中像素点的透明度信息；其中，所述目标掩膜图中像素点的像素值表征透明度；
    基于所述透明度信息对所述设定部位进行渲染。
  9. 根据权利要求1所述的方法，其中，所述虚拟物体包括多个，所述获取虚拟物体的第一深度图，包括：
    获取多个虚拟物体分别对应的多张第一深度图；
    对所述虚拟物体进行渲染，包括：
    基于所述多张第一深度图对所述多个虚拟物体进行渲染。
  10. 根据权利要求1或9所述的方法，所述方法还包括：
    响应于所述虚拟物体包括多个，从多个虚拟物体中确定设定虚拟物体；
    所述获取与所述目标物体相关的标准虚拟模型的第二深度图，包括：
    采用虚拟相机获取所述标准虚拟模型和所述设定虚拟物体的深度信息，获得第二深度图。
  11. 根据权利要求1所述的方法，在获得目标掩膜图之后，所述方法还包括：
    将所述目标掩膜图进行缓存；
    所述基于所述目标掩膜图对所述设定部位进行渲染，获得设定部位图；对所述虚拟物体进行渲染，获得虚拟物体图，包括：
    对于当前帧，基于设定前向帧对应的目标掩膜图对所述设定部位进行渲染，获得设定部位图；对设定前向帧对应的虚拟物体进行渲染，获得虚拟物体图。
  12. 一种图像处理装置，包括：
    初始掩膜图获取模块，设置为对目标物体的设定部位进行分割，获得初始掩膜图；
    深度图获取模块，设置为获取虚拟物体的第一深度图以及与所述目标物体相关的标准虚拟模型的第二深度图；
    目标掩膜图获取模块，设置为基于所述第一深度图及所述第二深度图对所述初始掩膜图进行调整，获得目标掩膜图；
    渲染模块，设置为基于所述目标掩膜图对所述设定部位进行渲染，获得设定部位图；对所述虚拟物体进行渲染，获得虚拟物体图；
    目标图像获取模块，设置为将所述设定部位图和所述虚拟物体图进行叠加，获得目标图像。
  13. 一种电子设备，所述电子设备包括：
    处理装置；
    存储装置，设置为存储程序；
    在所述程序被所述处理装置执行时，所述处理装置实现如权利要求1-11中任一所述的图像处理方法。
  14. 一种计算机可读介质，所述计算机可读介质上存储有计算机程序，所述 计算机程序被处理装置执行时实现如权利要求1-11中任一所述的图像处理方法。
PCT/CN2023/081253 2022-04-26 2023-03-14 图像处理方法、装置、设备及存储介质 WO2023207379A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210451633.9 2022-04-26
CN202210451633.9A CN114782659A (zh) 2022-04-26 2022-04-26 图像处理方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023207379A1 true WO2023207379A1 (zh) 2023-11-02

Family

ID=82432620

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/081253 WO2023207379A1 (zh) 2022-04-26 2023-03-14 图像处理方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN114782659A (zh)
WO (1) WO2023207379A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114782659A (zh) * 2022-04-26 2022-07-22 北京字跳网络技术有限公司 图像处理方法、装置、设备及存储介质
CN116681811B (zh) * 2022-09-19 2024-04-19 荣耀终端有限公司 图像渲染方法、电子设备及可读介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110889890A (zh) * 2019-11-29 2020-03-17 深圳市商汤科技有限公司 图像处理方法及装置、处理器、电子设备及存储介质
CN111079545A (zh) * 2019-11-21 2020-04-28 上海工程技术大学 一种基于图像修复的三维目标检测方法和系统
CN112102340A (zh) * 2020-09-25 2020-12-18 Oppo广东移动通信有限公司 图像处理方法、装置、电子设备和计算机可读存储介质
CN113870439A (zh) * 2021-09-29 2021-12-31 北京百度网讯科技有限公司 用于处理图像的方法、装置、设备以及存储介质
CN114782659A (zh) * 2022-04-26 2022-07-22 北京字跳网络技术有限公司 图像处理方法、装置、设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111079545A (zh) * 2019-11-21 2020-04-28 上海工程技术大学 一种基于图像修复的三维目标检测方法和系统
CN110889890A (zh) * 2019-11-29 2020-03-17 深圳市商汤科技有限公司 图像处理方法及装置、处理器、电子设备及存储介质
CN112102340A (zh) * 2020-09-25 2020-12-18 Oppo广东移动通信有限公司 图像处理方法、装置、电子设备和计算机可读存储介质
CN113870439A (zh) * 2021-09-29 2021-12-31 北京百度网讯科技有限公司 用于处理图像的方法、装置、设备以及存储介质
CN114782659A (zh) * 2022-04-26 2022-07-22 北京字跳网络技术有限公司 图像处理方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN114782659A (zh) 2022-07-22

Similar Documents

Publication Publication Date Title
WO2023207379A1 (zh) 图像处理方法、装置、设备及存储介质
WO2021139382A1 (zh) 人脸图像的处理方法、装置、可读介质和电子设备
KR102145220B1 (ko) 딥러닝을 활용하여 2차원 이미지를 3차원 이미지로 변환하는 방법 및 장치
WO2023071707A1 (zh) 视频图像处理方法、装置、电子设备及存储介质
WO2022042290A1 (zh) 一种虚拟模型处理方法、装置、电子设备和存储介质
WO2023207356A1 (zh) 图像渲染方法、装置、设备及存储介质
CN110728622B (zh) 鱼眼图像处理方法、装置、电子设备及计算机可读介质
WO2023207963A1 (zh) 图像处理方法、装置、电子设备及存储介质
WO2024104248A1 (zh) 虚拟全景图的渲染方法、装置、设备及存储介质
WO2024037556A1 (zh) 图像处理方法、装置、设备及存储介质
CN113989173A (zh) 视频融合方法、装置、电子设备及存储介质
CN111340865B (zh) 用于生成图像的方法和装置
WO2022166868A1 (zh) 漫游视图的生成方法、装置、设备和存储介质
WO2024056030A1 (zh) 一种图像深度估计方法、装置、电子设备及存储介质
CN111818265B (zh) 基于增强现实模型的交互方法、装置、电子设备及介质
WO2023193613A1 (zh) 高光渲染方法、装置、介质及电子设备
CN109816791B (zh) 用于生成信息的方法和装置
US20230103814A1 (en) Image Processing Systems and Methods
WO2023109564A1 (zh) 视频图像处理方法、装置、电子设备及存储介质
WO2023035973A1 (zh) 视频处理方法、装置、设备及介质
WO2022227996A1 (zh) 图像处理方法、装置、电子设备以及可读存储介质
CN114049403A (zh) 一种多角度三维人脸重建方法、装置及存储介质
WO2021121291A1 (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
RU2802724C1 (ru) Способ и устройство обработки изображений, электронное устройство и машиночитаемый носитель информации
CN114049417B (zh) 虚拟角色图像的生成方法、装置、可读介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23794815

Country of ref document: EP

Kind code of ref document: A1