WO2023197912A1 - 图像的处理方法、装置、设备、存储介质和程序产品 - Google Patents

图像的处理方法、装置、设备、存储介质和程序产品 Download PDF

Info

Publication number
WO2023197912A1
WO2023197912A1 PCT/CN2023/086211 CN2023086211W WO2023197912A1 WO 2023197912 A1 WO2023197912 A1 WO 2023197912A1 CN 2023086211 W CN2023086211 W CN 2023086211W WO 2023197912 A1 WO2023197912 A1 WO 2023197912A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target object
prop
depth information
mask
Prior art date
Application number
PCT/CN2023/086211
Other languages
English (en)
French (fr)
Inventor
余煜斌
邱达裕
罗孺冲
刘慧琳
Original Assignee
北京字跳网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字跳网络技术有限公司 filed Critical 北京字跳网络技术有限公司
Publication of WO2023197912A1 publication Critical patent/WO2023197912A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images

Definitions

  • the present disclosure relates to the field of computer communication technology, and in particular, to an image processing method, device, equipment, storage medium and program product.
  • embodiments of the present disclosure provide an image processing method, including: obtaining a mask image of a target object, wherein the target object is in a moving or stationary state, and the prop image is in a moving state; and in the prop image When an occlusion relationship occurs with the target object during movement, the target object and the prop image are displayed based on the positional relationship between the target object's mask image and the prop image.
  • embodiments of the present disclosure provide an image processing device, including: a mask image acquisition module, configured to acquire a mask image of a target object, wherein the target object is in a moving or stationary state, and the prop image is in motion. state; and a display module configured to display the target object based on the positional relationship between the mask image of the target object and the prop image when the prop image has an occlusion relationship with the target object during movement. with the props image.
  • a mask image acquisition module configured to acquire a mask image of a target object, wherein the target object is in a moving or stationary state, and the prop image is in motion. state
  • a display module configured to display the target object based on the positional relationship between the mask image of the target object and the prop image when the prop image has an occlusion relationship with the target object during movement. with the props image.
  • embodiments of the present disclosure provide an electronic device, including: one or more processors; and a storage device for storing one or more programs.
  • a storage device for storing one or more programs.
  • the one or more programs are processed by the one or more Execution by multiple processors causes the one or more processors to implement the method described in any one of the first aspects.
  • embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the method as described in any one of the first aspects is implemented.
  • embodiments of the present disclosure provide a computer program product, which when the computer program product is run on a computer, causes the computer to execute the method described in any one of the first aspects.
  • an embodiment of the present disclosure provides a computer program, including: instructions that, when executed by a processor, cause the processor to perform the method according to any one of the first aspects.
  • Figure 1 is a schematic diagram of an interface for adding prop images to images in related technologies
  • FIG. 2 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure
  • Figure 2A is a schematic diagram of an interface for adding a prop image to an image provided by an embodiment of the present disclosure
  • Figure 2B is a schematic diagram of another interface for adding a prop image to an image provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic flowchart of another image processing method provided by an embodiment of the present disclosure.
  • Figure 3A is a schematic diagram of another interface for adding a prop image to an image provided by an embodiment of the present disclosure
  • Figure 4 is a schematic flowchart of yet another image processing method provided by an embodiment of the present disclosure.
  • Figure 4A is a schematic diagram of another interface for adding a prop image to an image provided by an embodiment of the present disclosure
  • Figure 4B is a schematic structural diagram of a constructed target object box provided by an embodiment of the present disclosure.
  • Figure 5 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure.
  • FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • Figure 1 is the effect of adding prop images to images using related technologies.
  • the target object in order to realize that part of the prop image is blocked by the target object, the target object is simulated based on the algorithm-driven model, and then based on the positional relationship between the simulated target object and the prop image, part of the prop image is blocked by the target object.
  • the target object 200' simulated based on the algorithm-driven model can only simulate the structural characteristics of the target object 100', and cannot simulate the additional characteristics of the target object 100'.
  • the algorithm-driven model simulates the structural characteristics of the person, such as the body structure, but the additional characteristics of the person, such as hair and clothes, cannot be simulated.
  • occlusion is simulated based on the 3D (three dimensions, three-dimensional) model of the identified target object.
  • the 3D model may not be able to simulate additional features of the target object, such as hair, skirts and other features, which affects the simulated occlusion effect.
  • image processing methods may not be able to achieve good occlusion effects and consume large amounts of performance.
  • embodiments of the present disclosure provide an image processing method to realize that part of the prop image can be blocked based on the mask image of the target object, thereby improving the authenticity of the props displayed in the image including the target object.
  • This embodiment can be applied to the situation of adding image props to the image.
  • the method can be executed by an image processing device.
  • the device can be implemented in the form of software and/or hardware.
  • the device can be configured in a terminal device, such as a computer. wait.
  • Terminal devices can be tablets, mobile phones, wearable devices, vehicle-mounted devices, augmented reality (AR)/virtual reality (VR), laptops, ultra-mobile personal computers (UMPC) , netbooks, personal digital assistants (PDAs), smart TVs, smart screens, high-definition TVs, 4K TVs, smart speakers, smart projectors, etc. This disclosure does not place any restrictions on the specific types of electronic devices.
  • AR augmented reality
  • VR virtual reality
  • UMPC ultra-mobile personal computers
  • PDAs personal digital assistants
  • smart TVs smart screens, high-definition TVs, 4K TVs, smart speakers, smart projectors, etc. This disclosure does not place any restrictions on the specific types of electronic devices.
  • this disclosure does not limit the type of operating system of the electronic device.
  • Android system Linux system, Windows system, iOS system, etc.
  • FIG. 2 is a schematic flowchart of an image processing method provided by the present disclosure. As shown in Figure 2, the image processing method includes steps S10 to S20.
  • step S10 a mask image of the target object is obtained.
  • the target object is in a moving or static state
  • the prop image is in a moving state
  • the target object can be in motion or at rest.
  • the mask image of the target object can be obtained through real-time acquisition, or it can be acquired only once or at a preset time interval to reduce the amount of calculation, because when the target object is in a stationary state, the mask image of the target object can be acquired in real time.
  • the mask images are all the same and can be used in the processing of special effects on pictures.
  • the mask image of the target object is obtained through real-time acquisition.
  • the real-time acquisition can be every frame of image acquisition, or it can be every few frames of image acquisition. Specifically, the interval of several frames of image acquisition is different from that of the target.
  • the movement speed and amplitude of the object can be determined based on empirical values or actual application scenarios to improve the accuracy of the obtained mask image, which can be applied to the processing of video special effects.
  • Prop images can be applied to special effects scenes that process images, such as circular props, oval props, elf props, etc., where the prop images are in a moving state.
  • the prop image may move around the target object in the image including the target object, or the prop image may not move around the target object in the image including the target object.
  • the embodiments of the present disclosure do not limit the specific motion state of the prop image.
  • the obtained mask image of the target object can be a mask image of the target object in a certain image, or it can be a mask image of the target object in a video, and the video is composed of a series of static image frames that are continuously projected at an extremely fast speed. form.
  • the video can be split into a series of image frames, and editing operations can be performed on the image frames, thereby realizing the editing operation on the video.
  • the image processing method can be a process of processing a certain image, or it can be a process of processing each frame of image in the video, where the video can be a complete video that has been recorded, It can also be a video being recorded in real time.
  • the image is composed of multiple pixels, and pixels at different locations have different pixel values.
  • the pixel corresponding to the target object is determined based on the pixel value of each pixel corresponding to the image or image frame.
  • the mask is the same as the pixel point corresponding to the image or image frame, and the mask value at the position that coincides with the pixel point of the target object in the image or image frame is the same as the pixel value of the target object.
  • the pixel value of each pixel corresponding to a certain image is Among them, pixel points 145, 123, 23, 28, and 201 are the pixel points corresponding to the location of the target object in the image, then the prepared mask is By operating the corresponding pixels of the image and the mask, the mask image of the target object is obtained.
  • the pixel values of the pixels corresponding to the image will be displayed.
  • 0 pixel values or 256 pixels are displayed. At this time, 0 pixel values or 256 pixels in the image correspond to black or red.
  • the pixel value corresponding to the pixel point represents the color displayed at that position.
  • the pixel value at the position of the target object can be set to be different from the pixel value at the position corresponding to the non-target object, so as to facilitate the determination of the position of the target object. information.
  • step S20 when the prop image has an occlusion relationship with the target object during movement, the target object and the prop image are displayed based on the positional relationship between the mask image of the target object and the prop image.
  • the prop image 300 is an elliptical prop, and the prop image 300 moves around the target object 200. At this time, there is an occlusion relationship between the prop image and the target object during the movement.
  • the display attributes of the mask image relative to the target object determine the displayed target object and prop images.
  • FIG. 2A and FIG. 2B exemplarily show that when the prop image moves to the front of the target object or the prop image moves to the back of the target object, an occlusion relationship occurs between the prop image and the target object.
  • the occlusion relationship between the prop image and the target object can also be expressed in other forms. For example, distinguish the left and right of the target object.
  • the prop image is displayed at this time. Another example is to distinguish the target object from top to bottom.
  • the target is displayed at this time.
  • the prop image moves and the prop image is blocked by the upper part of the target object (the prop image is located in front of the target object, or the prop image is located behind the target object)
  • the target is displayed at this time.
  • the prop image is displayed at this time image.
  • the image includes a target object 100.
  • a mask image 200 of the target object of the image is obtained, as shown in FIG. 2B.
  • the oval prop displays the target object and the prop image based on the positional relationship between the mask image 200 of the target object and the prop image 300 in the image.
  • FIG. 2B according to the display attribute of the prop image 300, it is determined whether the image is displayed in the portion where the mask image of the target object overlaps with the prop image.
  • FIG. 2B exemplarily shows that in the overlapping portion between the target object mask image and the prop image 300, the prop image 300 located in front of the target object mask image is displayed, and the prop image 300 located behind the target object mask image is not displayed.
  • the above embodiments exemplify the method of displaying the target object and the prop image in the overlapping portion of the target object's mask image and the prop image.
  • the image may not only include the target object, but also include other images.
  • the other images and prop images are displayed in a normal display mode, which is not specifically limited in the embodiments of the present disclosure.
  • the target object and the prop image are displayed based on the positional relationship between the target object's mask image and the prop image, that is, the acquired target object's mask image can not only represent the target
  • the structural features of the object also represent the additional features of the target object.
  • the target object is a human body
  • the target object not only includes the structural features of the human body, but also includes additional features such as hair and clothes. Therefore, the obtained mask of the target object
  • the image can convert the structural features and additional features of the target object into the corresponding mask image, and then realize part of the prop image when displaying the target object and prop image based on the positional relationship between the target object's mask image and the prop image.
  • Occlusion can be performed based on the mask image of the target object, improving the authenticity of the props displayed in the image including the target object.
  • Occlusion can be performed based on the mask image of the target object, improving the authenticity of the props displayed in the image including the target object.
  • the image processing method provided by the embodiment of the present disclosure obtains a mask image of the target object, and based on the positional relationship between the mask image of the target object and the prop image, displays the target object and the prop image, that is, the image processing method provided by the embodiment of the present disclosure.
  • the method is to obtain a mask image of the target object by processing the image, and then based on the mask image of the target object and setting the display attributes of the prop image relative to the mask image of the target object, display the target object and the prop image, and realize the target object based on the mask image.
  • For the occlusion of the prop image try to ensure the occlusion effect of the target object on the prop image, and improve the accuracy of the target object.
  • the image processing method provided by the embodiment of the present disclosure blocks the prop image based on the mask image of the target object, compared with the related technology that blocks the prop image based on the 3D model of the simulated target object, in the embodiment of the present disclosure, there is no need to The algorithm drives the 3D model to simulate the target object, which can reduce the performance loss during image processing.
  • the prop image includes a two-dimensional plane image and a three-dimensional stereoscopic image.
  • the prop image is a two-dimensional plane image; for example, an elf prop, the prop image is a three-dimensional image.
  • the following will describe the image processing method when the prop image is a two-dimensional plane image or a three-dimensional stereoscopic image through specific embodiments.
  • FIG. 3 is a schematic flowchart of another image processing method provided by an embodiment of the present disclosure.
  • the embodiment of the present disclosure is based on the above embodiment.
  • Figure 3 illustrates an image processing method when the prop image is a two-dimensional plane image.
  • one implementation method of step S20 includes steps S21 to S22. .
  • step S21 the overlapping portion of the prop image and the mask image of the target object is determined.
  • the prop image is a two-dimensional plane image
  • the prop image is divided into at least two parts, and each part has corresponding display attributes
  • the display attributes include: display or not display.
  • the basis for dividing the prop image into at least two parts may be based on custom settings.
  • the prop image is divided into front and back, and another example is divided into left and right, etc.
  • the embodiments of the present disclosure do not specifically limit this.
  • the overlapping portion of the prop image and the mask image of the target object is first determined based on the display position of the prop image. For example, see FIG. 3A.
  • the dotted line area is the area where the prop image has an occlusion relationship with the target object during movement, that is, the overlapping portion of the target object's mask image and the prop image.
  • step S22 it is determined whether to display the overlapping portion of the prop image according to the display attribute corresponding to the overlapping portion.
  • the display attributes of each part of the divided prop image are known. At this time, based on the display attributes of each part of the prop image after the division, it is determined whether to display the overlapping portion of the prop image.
  • the prop image is an oval prop, and the prop image is divided into two parts, the front and rear parts.
  • the solid thick line represents the front part of the prop image
  • the solid thin line represents the rear part of the prop image.
  • the display attribute of the front part is displayed, and the display attribute of the rear part of the prop image is not displayed.
  • the prop image is displayed where the mask image of the target object overlaps the front part of the prop image
  • the target object is displayed where the mask image of the target object overlaps the rear part of the prop image.
  • the coordinate point For each coordinate point in the overlapping part, it is determined that the coordinate point belongs to the target part of the prop image. If the display attribute of the target part is display, then the coordinate point displays the pixel points of the prop image. If the display attribute of the target part is If the attribute is not displayed, the coordinate points display the pixels of the target object.
  • the coordinate points corresponding to the overlapping portion of the prop image and the mask image of the target object can be obtained, and based on each coordinate point, it is determined that the coordinate point belongs to the target part of the prop image. If the display attribute of the target part is display, then The coordinate points display the pixels of the prop image. If the display attribute of the target part is not displayed, the coordinate points display the pixels of the target object.
  • displaying the target object and the prop image based on the positional relationship between the mask image of the target object and the prop image further includes: determining a display attribute of the overlapping portion.
  • the determination of the display attribute of the overlapping portion includes: in the case where the prop image is divided into front and back parts (ie, the front part of the prop image and the back part of the prop image) based on the front and rear division, based on the center point O of the prop image
  • After determining the overlapping portion of the prop image and the mask image of the target object obtain each coordinate point of the overlapping portion. If the coordinate point of the X-axis corresponding to the overlapping portion is greater than 0, it means The display attribute of the overlapping part is display. If the coordinate point of the X-axis corresponding to the overlapping part is less than 0, it means that the display attribute of the overlapping part is not displayed.
  • the above embodiments exemplarily represent a constructed coordinate system that determines the display attributes of prop images based on coordinate points.
  • the embodiments of the present disclosure do not specifically limit the constructed coordinate system.
  • the image processing method provided by the embodiment of the present disclosure when the prop image is a two-dimensional plane image, first obtains the mask image of the target object, and then determines the prop when the prop image has an occlusion relationship with the target object during movement.
  • the overlapping part of the image and the mask image of the target object determines whether to display the overlapping part of the prop image based on the attribute information corresponding to the overlapping part.
  • the prop image is a two-dimensional plane image, based on the occlusion of the prop image by the target object, try to ensure that The occlusion effect of the target object on the prop image is eliminated, and the authenticity of the prop image displayed in the image including the target object is improved.
  • FIG. 4 is a schematic flowchart of yet another image processing method provided by an embodiment of the present disclosure.
  • the embodiment of the present disclosure is based on the above-mentioned embodiment.
  • Figure 4 illustrates an image processing method when the prop is a three-dimensional stereoscopic image.
  • another implementation method of step S20 includes steps S23 to S25.
  • step S23 the overlapping portion of the mask image of the target object and the prop image is obtained.
  • the prop image is a three-dimensional stereoscopic image
  • first the overlapping portion of the mask image of the target object and the prop image is obtained.
  • the prop image is a three-dimensional image, which means that the prop image has a certain thickness.
  • Exemplary table in Figure 4A shows a schematic structural diagram of the prop image 300.
  • step S24 for each coordinate point of the overlapping portion, first depth information of the coordinate point of the mask image is obtained, and second depth information of the coordinate point of the prop image is obtained.
  • the prop image When the prop image is a three-dimensional image, the prop image has a certain thickness, that is, the prop image includes depth information. Therefore, after obtaining the overlapping portion of the mask image of the target object and the prop image, for each coordinate point of the overlapping portion, the first depth information of the coordinate point of the mask image and the second depth of the coordinate point of the prop image are obtained information.
  • obtaining the first depth information of the coordinate point of the mask image includes: constructing a target object box based on the mask image, the target object box having depth information; and determining the depth information of the mask image based on the depth information of the target object box. The first depth information of the coordinate point.
  • the first depth information of the coordinate point of the acquired mask image may be determined based on the acquired texture information of the image. For example, after obtaining the texture information of an image, a mask image is obtained based on the texture information of the image, and then the depth information corresponding to the image is filtered through the mask image, and the depth information corresponding to the image is filtered based on the mask image. , filter based on the characteristics of the target object in the image. For example, when the target object in the image is a person, the first depth information of the target object can be filtered based on the elliptical sieve method. For example, a target object box is constructed based on the mask image.
  • the target object box has depth information, and the middle part of the target object is thicker and the edge part is thinner. Therefore, the first depth information of the corresponding target object is closer to the center point of the person. The smaller the first depth information is, the smaller the first depth information is near the edge of the person.
  • the target object box is constructed based on the mask image as shown in Figure 4B, where the center point of the target object box is the center point of the target object, and the height information of the target object box is the height from the highest point to the lowest point of the target object.
  • the width information of the target object box is the width of the target object from the leftmost to the rightmost.
  • the height information H of the target object box is the distance from point A to point B in Figure 4A
  • the width information W of the target object box is the distance from point C to point D in Figure 4A.
  • obtaining the second depth information of the coordinate point of the prop image includes: determining the second depth of the prop image based on the position information of the prop image relative to the center point of the target object box. information.
  • step S25 based on the values of the first depth information and the second depth information, it is determined that the coordinate point displays the pixel point of the target object or the pixel point of the prop image.
  • the coordinate point is determined to display the pixel point of the target object or the pixel point of the prop image, where, The prop image moves around the center point of the target object box, and based on the position information of the prop image relative to the center point of the target object box, the second depth information of the prop image is determined.
  • the coordinate point is determined to display the pixel point of the prop image; if the first depth information is less than the second depth information, the coordinate point is determined to display the pixel point of the target object.
  • Depth information represents the distance of the target object or prop image relative to the camera. The greater the depth information, the farther the target object or prop image is from the camera. If the first depth information is greater than the second depth information, it means that the distance of the prop image from the camera is compared. If the distance between the target object and the camera is close, then the coordinate point is determined to display the pixel point of the prop image. If the first depth information is less than the second depth information, it means that the distance between the prop image and the camera is farther than the distance between the target object and the camera, and the coordinate point is determined. Displays the pixels of the target object.
  • the image processing method provided by the embodiment of the present disclosure when the prop image is a three-dimensional stereoscopic image, first obtains the overlapping part of the mask image of the target object and the prop image, and then obtains the mask image for each coordinate point of the overlapping part. code the first depth information of the coordinate point of the image, and obtain the second depth information of the coordinate point of the prop image, to determine the pixel point of the coordinate point to display the target object or to display the prop based on the values of the first depth information and the second depth information.
  • the pixels of the image are used to realize that when the prop image is a three-dimensional image, based on the target object's occlusion of the prop image, the occlusion effect of the target object on the prop image is ensured as much as possible, and the authenticity of the props displayed in the image including the target object is improved. .
  • FIG. 5 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure. As shown in FIG. 5 , the image processing device includes: a mask image acquisition module 510 and a display module 520 .
  • the mask image acquisition module 510 is used to acquire the mask image of the target object, where the target object is in a moving or stationary state, and the prop image is in a moving state.
  • the display module 520 is configured to display the target object and the prop image based on the positional relationship between the mask image of the target object and the prop image when the prop image has an occlusion relationship with the target object during movement.
  • the mask image acquisition module obtains the mask image of the target object
  • the display module displays the target object and the prop image based on the positional relationship between the mask image of the target object and the prop image
  • the image processing method provided by the embodiment obtains a mask image of the target object by processing the image, and then based on the mask image of the target object and the display attribute of the set prop image relative to the mask image of the target object, the target object and the mask image are displayed.
  • the prop image implements occlusion of the prop image based on the target object, ensuring the occlusion effect of the target object on the prop image as much as possible, and improving the authenticity of the props displayed in the image including the target object.
  • the image processing method provided by the embodiment of the present disclosure blocks the prop image based on the mask image of the target object, compared with the related technology that blocks the prop image based on the 3D model of the simulated target object, in the embodiment of the present disclosure, there is no need to The algorithm drives the 3D model to simulate the target object, which can reduce the performance loss during image processing.
  • the display module includes: a first overlapping portion determination unit, used to determine the overlapping portion of the prop image and the mask image of the target object; and a first display unit, used to determine the overlapping portion according to the display attribute corresponding to the overlapping portion. Whether to display the overlapping portion of the prop image.
  • the first display unit is configured to: for each coordinate point in the overlapping part, determine that the coordinate point belongs to the target part of the prop image. If the display attribute of the target part is display, then the coordinate point displays the pixel point of the prop image. , if the display attribute of the target part is not displayed, the coordinate point displays the pixels of the target object.
  • the display module includes: a display attribute determining unit configured to determine the display attribute of the overlapping portion.
  • the display attribute determination unit is configured to establish a plane coordinate system based on the center point O of the prop image when the prop image is divided into a front part of the prop image and a rear part of the prop image. After the mask image of the target object overlaps, obtain each coordinate point of the overlapping part. If the coordinate point of the X-axis corresponding to the overlapping part is greater than 0, it means that the display attribute of the overlapping part is display, and if the X-axis corresponding to the overlapping part The coordinate point is less than 0, which means that the display attribute of the overlapping part is not displayed.
  • the display module further includes: a second overlapping portion determination unit, used to obtain the overlapping portion of the mask image of the target object and the prop image; a second depth information acquisition unit, used to determine each coordinate of the overlapping portion point, obtain the first depth information of the coordinate point of the mask image, and obtain the second depth information of the coordinate point of the prop image; and a second display unit for determining based on the values of the first depth information and the second depth information.
  • the coordinate points display the pixels of the target object or the pixels of the prop image.
  • the second display unit is configured to: if the first depth information is greater than the second depth information, determine the coordinate point to display the pixel point of the prop image; if the first depth information is less than the second depth information, determine the coordinate point to display The pixels of the target object.
  • the second depth information acquisition unit is configured to: construct a target object box based on the mask image, the target object box having depth information; and determine the first depth of the coordinate point of the mask image based on the depth information of the target object box. information.
  • the center point of the target object box is the center point of the target object
  • the height information of the target object box is the height from the highest point to the lowest point of the target object
  • the width information of the target object box is the target The width of the object from the far left to the far right.
  • the second depth information acquisition unit is configured to determine second depth information of the prop image based on position information of the prop image relative to a center point of the target object box.
  • the various units and modules included are only arranged according to functional logic.
  • the functional units are divided into rows, but are not limited to the above-mentioned divisions, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present invention.
  • the present disclosure also provides an electronic device, including: a processor, the processor is configured to execute a computer program stored in a memory, and when the computer program is executed by the processor, the method of the above embodiment is implemented.
  • FIG. 6 is a schematic structural diagram of an electronic device provided by the present disclosure.
  • FIG. 6 shows a block diagram of an exemplary electronic device suitable for implementing embodiments of the present invention.
  • the electronic device shown in FIG. 6 is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.
  • electronic device 600 is embodied in the form of a general computing device.
  • the components of electronic device 600 may include, but are not limited to: one or more processors 610, system memory 620, and a bus 630 connecting different system components (including system memory 620 and processors).
  • Bus 630 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics accelerated port, a processor, or a local bus using any of a variety of bus structures.
  • these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect ( PCI) bus.
  • ISA Industry Standard Architecture
  • MAC Micro Channel Architecture
  • VESA Video Electronics Standards Association
  • PCI Peripheral Component Interconnect
  • Electronic device 600 typically includes a variety of computer system readable media. These media can be any media that can be accessed by electronic device 600, including volatile and nonvolatile media, removable and non-removable media.
  • System memory 620 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 640 and/or cache memory 650. Electronic device 600 may further include other removable/non-removable, volatile/non-volatile computer system storage media.
  • storage system 660 may be used to read and write to non-removable, non-volatile magnetic media (commonly referred to as "hard drives").
  • Disk drives may be provided for reading and writing from removable non-volatile disks (e.g., "floppy disks"), and for reading and writing from removable non-volatile optical disks (e.g., CD-ROMs, DVD-ROMs, or other optical media).
  • CD-ROM drive may be connected to bus 630 through one or more data media interfaces.
  • System memory 620 may include at least one program product having a set (eg, at least one) program module configured to perform the functions of various embodiments of the present invention.
  • a program/utility 680 having a set of (at least one) program modules 670 may be stored, for example, in system memory 620. Data, each of these examples or some combination may include an implementation of a network environment.
  • Program modules 670 generally perform functions and/or methods in the described embodiments of the present invention.
  • the processor 610 executes at least one program among a plurality of programs stored in the system memory 620 . Perform various functional applications and information processing, such as implementing the method embodiments provided by the embodiments of the present invention.
  • the electronic device 600 also includes an I/O interface 690 and a network adapter 691.
  • the present disclosure also provides a computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the method of the above embodiments is implemented.
  • the computer-readable storage medium is a non-transitory computer-readable storage medium.
  • the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
  • the computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer readable storage media include: electrical connections having one or more conductors, portable computer disks, hard drives, random access memory (RAM), read only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .
  • Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the foregoing.
  • Computer program code for performing the operations of the present invention may be written in one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional Procedural programming language—such as "C" or a similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or wide area network (WAN) domain, or may be connected to an external computer (e.g., using an Internet service provider). Internet connection).
  • LAN local area network
  • WAN wide area network
  • Internet service provider e.g., using an Internet service provider
  • the present disclosure also provides a computer program product.
  • the computer program product When the computer program product is run on a computer, it causes the computer to execute the method for implementing the above embodiments.
  • the present disclosure also provides a computer program, including: instructions that, when executed by a processor, cause the processor to perform the method as described in the above embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本公开涉及一种图像的处理方法、装置、设备、存储介质和程序产品。图像的处理方法包括：获取目标物体的掩码图像，其中，目标物体处于运动或者静止状态，道具图像处于运动状态；在道具图像在运动过程中与目标物体产生遮挡关系的情况下，基于目标物体的掩码图像与道具图像的位置关系，显示目标物体与道具图像。

Description

图像的处理方法、装置、设备、存储介质和程序产品
相关申请的交叉引用
本申请是以申请号为202210375988.4，申请日为2022年4月11日的中国申请为基础，并主张其优先权，该中国申请的公开内容在此作为整体引入本申请中。
技术领域
本公开涉及计算机通信技术领域，尤其涉及一种图像的处理方法、装置、设备、存储介质和程序产品。
背景技术
随着通信技术和终端设备的发展，各种终端设备例如手机、平板电脑等已经成为了人们工作和生活中不可或缺的一部分，而且随着终端设备的日益普及，图像交互应用成为一种娱乐的主要渠道。
发明内容
第一方面，本公开实施例提供一种图像的处理方法，包括：获取目标物体的掩码图像，其中，所述目标物体处于运动或者静止状态，道具图像处于运动状态；和在所述道具图像在运动过程中与所述目标物体产生遮挡关系的情况下，基于所述目标物体的掩码图像与道具图像的位置关系，显示所述目标物体与所述道具图像。
第二方面，本公开实施例提供一种图像的处理装置，包括：掩码图像获取模块，用于获取目标物体的掩码图像，其中，所述目标物体处于运动或者静止状态，道具图像处于运动状态；和显示模块，用于在所述道具图像在运动过程中与所述目标物体产生遮挡关系的情况下，基于所述目标物体的掩码图像与道具图像的位置关系，显示所述目标物体与所述道具图像。
第三方面，本公开实施例提供一种电子设备，包括：一个或多个处理器；和存储装置，用于存储一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如第一方面中任一所述的方法。
第四方面，本公开实施例提供一种计算机存储介质，其上存储有计算机程序，该程序被处理器执行时实现如第一方面中任一所述的方法。
第五方面，本公开实施例提供一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如第一方面任一项所述的方法。
第六方面，本公开实施例提供一种计算机程序，包括：指令，所述指令当由处理器执行时使所述处理器执行如第一方面任一项所述的方法。
相关技术
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
为了更清楚地说明本公开实施例或相关技术中的技术方案，下面将对实施例或相关技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是相关技术中图像中添加道具图像的界面示意图；
图2是本公开实施例提供的一种图像的处理方法的流程示意图；
图2A是本公开实施例提供的一种在图像中添加道具图像的界面示意图；
图2B是本公开实施例提供的另一种在图像中添加道具图像的界面示意图；
图3是本公开实施例提供的另一种图像的处理方法的流程示意图；
图3A是本公开实施例提供的又一种在图像中添加道具图像的界面示意图；
图4是本公开实施例提供的又一种图像的处理方法的流程示意图；
图4A是本公开实施例提供的又一种在图像中添加道具图像的界面示意图；
图4B是本公开实施例提供的构建的目标对象盒子的结构示意图；
图5是本公开实施例提供的图像的处理装置的结构示意图；
图6是本公开实施例提供的电子设备的结构示意图。
具体实施方式
为了能够更清楚地理解本公开的上述目的、特征和优点，下面将对本公开的方案进行进一步描述。需要说明的是，在不冲突的情况下，本公开的实施例及实施例中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本公开，但本公开还可以采用其他不同于在此描述的方式来实施；显然，说明书中的实施例只是本公开的一部分实施例， 而不是全部的实施例。
通过在图像中添加道具图像添加特效，实现特效，使得图像交互的多样性。
图1是相关技术在图像中添加道具图像的效果。相关技术中，为实现部分道具图像通过目标物体进行遮挡，基于算法驱动模型进而模拟目标物体，然后基于模拟的目标物体与道具图像的位置关系，实现部分道具图像被目标物体遮挡。但是，由图1可知，基于算法驱动模型模拟的目标物体200'仅仅只能模拟目标物体100'的结构特征，对于目标物体100'的附加特征无法模拟。例如，目标物体为人，算法驱动模型模拟人的结构特征，例如身体结构，但是人的附加特征，例如头发、穿的衣服无法模拟。当基于模拟的目标物体与道具图像的位置关系，显示模拟的目标物体200'和道具图像300'的时候，会存在图1出现的问题，即位于模拟的目标物体200'双腿中间的道具图像应该被目标物体遮挡，但是由于算法模型模拟的目标物体无法模拟人的裙子等附加特征，因此，此部分处无法实现基于模拟的目标物体对道具图像的遮挡，降低了在包括目标物体的图像中显示图像道具的真实性。
因此，本公开的发明人发现，在相关技术中，图像中添加特效，在图像中目标对象对特效的遮挡方法中，基于识别的目标对象的3D(three dimensions，三维)模型模拟遮挡，但是由于使用3D模型模拟遮挡的过程中，3D模型可能无法模拟目标对象的附加特征，例如头发、裙子等特征，影响模拟遮挡效果。在相关技术中，图像的处理方法可能无法实现较好的遮挡效果，且性能消耗较大。
鉴于此，本公开的实施例提供一种图像的处理方法，以实现部分道具图像可基于目标物体的掩码图像进行遮挡，提高在包括目标物体的图像中显示的道具的真实性。
本实施例可适用于在图像中添加图像道具的情况，该方法可以由图像的处理装置来执行，该装置可以采用软件和/或硬件的方式实现，该装置可以配置于终端设备中，例如计算机等。
终端设备可以是平板电脑、手机、可穿戴设备、车载设备、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)、智能电视、智慧屏、高清电视、4K电视、智能音箱、智能投影仪等，本公开对电子设备的具体类型不作任何限制。
其中，本公开对电子设备的操作系统的类型不做限定。例如，Android系统、Linux系统、Windows系统、iOS系统等。
下面以几个具体的实施例对本公开的技术方案进行描述。
图2为本公开提供的一种图像的处理方法的流程示意图。如图2所示，图像的处理方法包括步骤S10至S20。
在步骤S10，获取目标物体的掩码图像。
其中，目标物体处于运动或者静止状态，道具图像处于运动状态。
目标物体可以处于运动状态，也可以处于静止状态。当目标物体处于静止状态时，可以通过实时获取的方式获取的目标物体的掩码图像，也可以只获取一次或者间隔预设时间获取一次，以减少计算量，因为目标物体处于静止状态时，获取的掩码图像均相同，可以应用于对图片特效的处理过程中。当目标物体处于运动状态时，通过实时获取的方式获取的目标物体的掩码图像，实时获取可以是每一帧图像获取，也可以是每隔几帧图像获取，具体间隔几帧图像获取与目标物体的运动速度、幅度等有关，可根据经验值或者实际应用场景确定，以提高获取的掩码图像的准确性，可以应用于对视频特效的处理过程中。
道具图像示例性可应用于对图像进行处理的特效场景中，例如圆形道具、椭圆形道具、精灵道具等，其中，道具图像为运动状态。具体的，道具图像可以围绕包括目标物体的图像中的目标物体进行运动，道具图像也可以不围绕包括目标物体的图像中的目标物体进行运动。本公开实施例不对道具图像的具体运动状态进行限定。
获取的目标物体的掩码图像可以为某一个图像中目标物体的掩码图像，也可以为视频中目标物体的掩码图像，而视频是由一系列静态的图像帧以极快的速度连续放映形成。由此，可以将视频拆分成一系列图像帧，并对图像帧进行编辑操作，从而实现对视频的编辑操作。在本公开实施例中，图像的处理方法可以为对某一个图像进行处理的过程，也可以为对视频中的每一帧图像进行处理的过程，其中，视频可以是一个录制完成的完整视频，也可以是正在实时录制的视频。
图像是由多个像素点组成的，不同位置处的像素点的像素值不同。在获取到某个图像或图像帧后，基于该图像或图像帧对应的各像素点的像素值，确定目标物体对应的像素点。通过制备特定的掩膜(该掩膜与图像或图像帧对应的像素点相同，且掩膜中与图像或图像帧中目标物体的像素点重合位置处的掩膜值与目标物体的像素值相同)，通过将图像或图像帧对应的像素值与掩膜进行运算，可以得到目标物体的掩码图像。
示例性的，某一图像对应的各像素点的像素值为其中，像素点145、123、23、28、201为该图像中目标物体所在位置处对应的像素点，则制备的掩膜为 通过将图像对应的像素点与掩膜进行运算，得到目标物体的掩码图像。
需要说明的是，在具体的实施方式中，图像对应的像素点与掩膜进行运算的过程中，图像对应的像素点与掩膜的像素点相同时，则显示图像对应的像素点的像素值，当图像对应的像素点与掩膜的像素点不相同时，则显示0像素值或256像素点，此时图像中0像素值或256像素点对应显示黑色或红色。
此外，上述实施例中，像素点对应的像素值表示该位置处显示的颜色，通过设置目标物体对应的位置处显示图像对应的颜色，而设置非目标物体对应的位置处显示黑色和红色，可以确定图像中的目标物体的位置。
在其它可实施方式中，当确定图像中的目标物体的位置后，可以将目标物体位置处的像素值设置为与非目标物体对应的位置处的像素值不相同，以方便确定目标物体的位置信息。
在步骤S20，在道具图像在运动过程中与目标物体产生遮挡关系的情况下，基于目标物体的掩码图像与道具图像的位置关系，显示目标物体与道具图像。
在道具图像处于运动状态的情况下，若道具图像围绕目标物体进行运动，此时，道具图像在运动过程中会存在道具图像遮挡目标物体，以及道具图像被目标物体遮挡的现象。示例性的，结合图2A和图2B，道具图像300为椭圆形道具，道具图像300围绕目标物体200运动，此时，存在道具图像在运动过程中与目标物体产生遮挡关系，通过设定道具图像相对目标物体的掩码图像的显示属性，确定显示的目标物体以及道具图像。
需要说明的是，图2A和图2B示例性表示道具图像运动到目标物体的前面或道具图像运动到目标物体的后面时，道具图像与目标物体产生遮挡关系。在其它可实施方式中，道具图像与目标物体的遮挡关系也可以为其它表现形式。例如，对目标物体进行左右区分，当道具图像运动后，道具图像与目标物体的左侧部分产生遮挡(道具图像位于目标物体的前面，或道具图像位于目标物体的后面)时，此时显示目标物体，道具图像与目标物体的右侧部分产生遮挡(道具图像位于目标物体的前面，或道具图像位于目标物体的后面)时，此时显示道具图像。又例如，对目标物体进行上下区分，当道具图像运动后，道具图像与目标物体的上部分产生遮挡(道具图像位于目标物体的前面，或道具图像位于目标物体的后面)时，此时显示目标物体，当道具图像运动后，道具图像与目标物体的下部分产生遮挡(道具图像位于目标物体的前面，或道具图像位于目标物体的后面)时，此时显示道具 图像。
例如，参见图2A，图2A中，图像包括目标物体100，通过对图像进行处理，获取的图像的目标物体的掩码图像200，如图2B所示，通过在图像中添加道具图像300，例如椭圆形道具，基于图像中目标物体的掩码图像200与道具图像300的位置关系，显示目标物体和道具图像。图2B中，根据道具图像300的显示属性，确定目标物体的掩码图像与道具图像重叠的部分是否显示图像。图2B示例性表示目标物体的掩码图像与道具图像300重叠部分中，位于目标物体掩码图像前面的道具图像300显示，位于目标物体掩码图像后面的道具图像300不显示。
通过基于目标物体的掩码图像与道具图像的位置关系，显示目标物体与道具图像，实现部分道具图像可基于目标物体的掩码图像进行遮挡，提高在包括目标物体的图像中显示的道具的真实性。
需要说明的是，上述实施例示例性说明在目标物体的掩码图像与道具图像重叠部分显示目标物体与道具图像的方式。在具体的实施方式中，图像中可能不仅仅包括目标物体，还包括其它画面，此时，其它画面与道具图像的显示方式以正常显示方式显示，本公开实施例不对此进行具体限定。
此外，由于本公开实施例提供的图像的处理方法，基于目标物体的掩码图像与道具图像的位置关系，显示目标物体与道具图像，即获取的目标物体的掩码图像不仅仅能表示出目标物体的结构特征，对于目标物体的附加特征也进行表示，例如，目标物体为人体，目标物体不仅仅包括人体的结构特征，还包括头发、衣服等附加特征，因此，获取的目标物体的掩码图像可以将目标物体的结构特征以及附加特征等均转换成对应的掩码图像，进而在基于目标物体的掩码图像与道具图像的位置关系，显示目标物体与道具图像的时候，实现部分道具图像可基于目标物体的掩码图像进行遮挡，提高在包括目标物体的图像中显示的道具的真实性。此外，获取目标物体的掩码图像相比较相关技术中通过模拟目标物体的3D模型对道具图像进行遮挡，本公开实施例中无需通过算法驱动3D模型模拟目标物体，可以减小图像处理过程中的性能损耗。
本公开实施例提供的图像的处理方法，获取目标物体的掩码图像，基于目标物体的掩码图像与道具图像的位置关系，显示目标物体与道具图像，即本公开实施例提供的图像的处理方法，通过对图像进行处理得到目标物体的掩码图像，再基于目标物体的掩码图像与设定道具图像相对目标物体的掩码图像的显示属性，显示目标物体与道具图像，实现基于目标物体对道具图像的遮挡，尽量保证了目标物体对道具图像的遮挡效果，提高在包括目 标物体的图像中显示的道具的真实性。此外，由于本公开实施例提供的图像的处理方法基于目标物体的掩码图像对道具图像进行遮挡，相比较相关技术基于模拟目标物体的3D模型对道具图像进行遮挡，本公开实施例中无需通过算法驱动3D模型模拟目标物体，可以减小图像处理过程中的性能损耗。
在具体的实施方式中，道具图像包括二维平面图像和三维立体图像。示例性的，例如圆形道具或椭圆形道具，该道具图像为二维平面图像；例如精灵道具，该道具为三维立体图像。以下将通过具体的实施例说明道具图像为二维平面图像或三维立体图像时图像的处理方法。
图3是本公开实施例提供的另一种图像的处理方法的流程示意图。本公开实施例是在上述实施例的基础上，图3示例性说明道具图像为二维平面图像时图像的处理方法，如图3所示，步骤S20的一种可实现方式包括步骤S21至S22。
在步骤S21，确定道具图像与目标物体的掩码图像的重叠部分。
例如，在道具图像为二维平面图像的情况下，道具图像被划分为至少两部分，每部分具有对应的显示属性，显示属性包括：显示或者不显示。
其中，道具图像被划分为至少两部分的划分依据可基于自定义设置。例如道具图像按照前后进行划分，又例如按照左右划分等。本公开实施例不对此进行具体限定。
在将道具图像划分为至少两部分后，获取划分后各部分对应的显示属性。
在获取到目标物体的掩码图像后，当道具图像在运动过程中与目标物体产生遮挡关系时，首先基于道具图像的显示位置确定道具图像与目标物体的掩码图像的重叠部分。示例性的，参见图3A，图3A中，虚线部分的区域为道具图像在运动过程中与目标物体产生遮挡关系的区域，即目标物体的掩码图像与道具图像的重叠部分。
在步骤S22，根据重叠部分对应的显示属性，确定是否显示道具图像的重叠部分。
在对道具图像划分后，划分后各部分道具图像的显示属性是已知的。此时，基于划分后各部分道具图像的显示属性，确定是否显示道具图像的重叠部分。
例如，结合图3A，道具图像为椭圆形道具，道具图像划分为前后两部分，其中，图3A中，实粗线部分表示道具图像前部分，实细线部分表示道具图像的后部分，道具图像前部分的显示属性为显示，道具图像后部分的显示属性为不显示。
由于道具图像的前部分显示属性为显示，因此，目标物体的掩码图像与道具图像前部分重叠的地方显示道具图像，目标物体的掩码图像与道具图像后部分重叠的地方显示目标物体。
作为一种可实施方式，针对重叠部分的每个坐标点，确定坐标点属于道具图像的目标部分，若目标部分的显示属性为显示，则坐标点显示道具图像的像素点，若目标部分的显示属性为不显示，则坐标点显示目标对象的像素点。
在一些实施方式中，可以获取道具图像与目标物体的掩码图像重叠部分对应的坐标点，并基于各坐标点确定该坐标点属于道具图像的目标部分，若目标部分的显示属性为显示，则坐标点显示道具图像的像素点，若目标部分的显示属性为不显示，则坐标点显示目标对象的像素点。
在一些实施例中，所述基于所述目标物体的掩码图像与道具图像的位置关系，显示所述目标物体与所述道具图像，还包括：确定所述重叠部分的显示属性。
例如，所述确定所述重叠部分的显示属性包括：在基于前后划分对道具图像划分为前后两部分(即，道具图像前部分和道具图像后部分)的情况下，基于道具图像的中心点O建立平面坐标系，如图3A所示，在确定道具图像与目标物体的掩码图像重叠部分后，获取重叠部分的每个坐标点，若重叠部分对应的X轴的坐标点大于0，则表示重叠部分的显示属性为显示，若重叠部分对应的X轴的坐标点小于0，则表示重叠部分的显示属性为不显示。
需要说明的是，上述实施例示例性表示一种构建坐标系，基于坐标点确定道具图像的显示属性，本公开实施例不对构建的坐标系进行具体限定。
本公开实施例提供的图像的处理方法，在道具图像为二维平面图像的情况下，首先获取目标物体的掩码图像，然后当道具图像在运动过程中与目标物体产生遮挡关系时，确定道具图像与目标物体的掩码图像的重叠部分，基于重叠部分对应的属性信息确定是否显示道具图像的重叠部分，实现当道具图像为二维平面图像时，基于目标物体对道具图像的遮挡，尽量保证了目标物体对道具图像的遮挡效果，提高在包括目标物体的图像中显示的道具图像的真实性。
图4是本公开实施例提供的又一种图像的处理方法的流程示意图。本公开实施例是在上述实施例的基础上，图4示例性说明道具为三维立体图像时图像的处理方法，如图4所示，步骤S20的另一种可实现方式包括步骤S23至S25。
在步骤S23，获取目标物体的掩码图像与道具图像的重叠部分。
在道具图像为三维立体图像的情况下，首先获取目标物体的掩码图像与道具图像的重叠部分。
例如，道具图像为三维立体图像，表示道具图像是有一定厚度的。图4A中示例性表 示道具图像300的结构示意图。
在步骤S24，针对重叠部分的每个坐标点，获取掩码图像的坐标点的第一深度信息，并获取道具图像的坐标点的第二深度信息。
在道具图像为三维立体图像的情况下，道具图像是有一定厚度的，即道具图像包括深度信息。因此，在获取到目标物体的掩码图像与道具图像的重叠部分后，针对重叠部分的每个坐标点，获取掩码图像的坐标点的第一深度信息以及道具图像的坐标点的第二深度信息。
在一些实施例中，获取掩码图像的坐标点的第一深度信息，包括：基于掩码图像构建目标对象盒子，目标对象盒子具有深度信息；和基于目标对象盒子的深度信息确定掩码图像的坐标点的第一深度信息。
其中，获取掩码图像的坐标点的第一深度信息可基于获取的图像的纹理信息确定。例如，在获取到图像的纹理信息后，基于该图像的纹理信息获取到掩码图像，然后通过掩码图像筛选该图像对应的深度信息，基于掩码图像筛选该图像对应的深度信息的过程中，基于图像中目标物体的特征进行筛选。示例性的，当图像中的目标物体为人时，可基于椭圆形筛法筛选目标物体的第一深度信息。例如，基于掩码图像构建目标对象盒子，目标对象盒子具有深度信息，而目标物体的中间部分比较厚，边缘部分比较薄，因此，对应的目标物体的第一深度信息为越靠近人的中心点的地方第一深度信息越小，越靠近人的边缘的地方第一深度信息越大。
示例性的，基于掩码图像构建目标对象盒子如图4B所示，其中，目标对象盒子的中心点为目标对象的中心点，目标对象盒子的高度信息为目标对象最高点至最低点的高度，目标对象盒子的宽度信息为目标对象从最左边到最右边的宽度。例如，结合图4A和图4B，目标对象盒子的高度信息H为图4A中A点到B点的距离，目标对象盒子的宽度信息W为图4A中C点到D点的距离。
在一些实施例中，获取所述道具图像的所述坐标点的第二深度信息包括：基于所述道具图像相对所述目标对象盒子的中心点的位置信息，确定所述道具图像的第二深度信息。
在步骤S25，基于第一深度信息和第二深度信息的值，确定坐标点显示目标对象的像素点或者显示道具图像的像素点。
在道具图像围绕目标对象运动的情况下，基于道具图像的第二深度信息与目标对象的第一深度信息的关系，确定坐标点显示目标对象的像素点或显示道具图像的像素点，其中， 道具图像围绕目标对象盒子的中心点运动，基于道具图像相对目标对象盒子的中心点的位置信息，确定道具图像的第二深度信息。
作为一种可实施方式，若第一深度信息大于第二深度信息，则确定坐标点显示道具图像的像素点，若第一深度信息小于第二深度信息，确定坐标点显示目标对象的像素点。
深度信息表示目标对象或道具图像相对摄像头的距离，深度信息越大，目标对象或道具图像距离摄像头的距离越远，若第一深度信息大于第二深度信息，表示道具图像距离摄像头的距离相比较目标对象距离摄像头的距离近，则确定坐标点显示道具图像的像素点，若第一深度信息小于第二深度信息，表示道具图像距离摄像头的距离相比较目标对象距离摄像头的距离远，确定坐标点显示目标对象的像素点。
本公开实施例提供的图像的处理方法，在道具图像为三维立体图像的情况下，首先获取目标物体的掩码图像与道具图像的重叠部分，然后确针对重叠部分的每个坐标点，获取掩码图像的坐标点的第一深度信息，并获取道具图像的坐标点的第二深度信息，实现基于第一深度信息和第二深度信息的值，确定坐标点显示目标对象的像素点或者显示道具图像的像素点，实现当道具图像为三维立体图像时，基于目标物体对道具图像的遮挡，尽量保证了目标物体对道具图像的遮挡效果，提高在包括目标物体的图像中显示的道具的真实性。
图5是本公开实施例提供的一种图像的处理装置的结构示意图。如图5所示，图像的处理装置包括：掩码图像获取模块510和显示模块520。
掩码图像获取模块510，用于获取目标物体的掩码图像，其中，目标物体处于运动或者静止状态，道具图像处于运动状态。
显示模块520，用于在道具图像在运动过程中与目标物体产生遮挡关系的情况下，基于目标物体的掩码图像与道具图像的位置关系，显示目标物体与道具图像。
本公开实施例提供的图像的处理装置，掩码图像获取模块获取目标物体的掩码图像，显示模块基于目标物体的掩码图像与道具图像的位置关系，显示目标物体与道具图像，即本公开实施例提供的图像的处理方法，通过对图像进行处理得到目标物体的掩码图像，再基于目标物体的掩码图像与设定道具图像相对目标物体的掩码图像的显示属性，显示目标物体与道具图像，实现基于目标物体对道具图像的遮挡，尽量保证了目标物体对道具图像的遮挡效果，提高在包括目标物体的图像中显示的道具的真实性。此外，由于本公开实施例提供的图像的处理方法基于目标物体的掩码图像对道具图像进行遮挡，相比较相关技术基于模拟目标物体的3D模型对道具图像进行遮挡，本公开实施例中无需通过算法驱动3D模型模拟目标物体，可以减小图像处理过程中的性能损耗。
在一些实施例中，显示模块包括：第一重叠部分确定单元，用于确定道具图像与目标物体的掩码图像的重叠部分；和第一显示单元，用于根据重叠部分对应的显示属性，确定是否显示道具图像的重叠部分。
在一些实施例中，第一显示单元用于：针对重叠部分的每个坐标点，确定坐标点属于道具图像的目标部分，若目标部分的显示属性为显示，则坐标点显示道具图像的像素点，若目标部分的显示属性为不显示，则坐标点显示目标对象的像素点。
在一些实施例中，显示模块包括：显示属性确定单元，用于确定所述重叠部分的显示属性。
在一些实施例中，所述显示属性确定单元用于在对道具图像划分为道具图像前部分和道具图像后部分的情况下，基于道具图像的中心点O建立平面坐标系，在确定道具图像与目标物体的掩码图像重叠部分后，获取重叠部分的每个坐标点，若重叠部分对应的X轴的坐标点大于0，则表示重叠部分的显示属性为显示，和若重叠部分对应的X轴的坐标点小于0，则表示重叠部分的显示属性为不显示。
在一些实施例中，显示模块还包括：第二重叠部分确定单元，用于获取目标物体的掩码图像与道具图像的重叠部分；第二深度信息获取单元，用于针对重叠部分的每个坐标点，获取掩码图像的坐标点的第一深度信息，并获取道具图像的坐标点的第二深度信息；和第二显示单元，用于基于第一深度信息和第二深度信息的值，确定坐标点显示目标对象的像素点或者显示道具图像的像素点。
在一些实施例中，第二显示单元用于：若第一深度信息大于第二深度信息，则确定坐标点显示道具图像的像素点，若第一深度信息小于第二深度信息，确定坐标点显示目标对象的像素点。
在一些实施例中，第二深度信息获取单元用于：基于掩码图像构建目标对象盒子，目标对象盒子具有深度信息；和基于目标对象盒子的深度信息确定掩码图像的坐标点的第一深度信息。
在一些实施例中，所述目标对象盒子的中心点为目标对象的中心点，所述目标对象盒子的高度信息为目标对象最高点至最低点的高度，所述目标对象盒子的宽度信息为目标对象从最左边到最右边的宽度。
在一些实施例中，第二深度信息获取单元用于基于所述道具图像相对所述目标对象盒子的中心点的位置信息，确定所述道具图像的第二深度信息。
值得注意的是，上述装置的实施例中，所包括的各个单元和模块只是按照功能逻辑进 行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，各功能单元的具体名称也只是为了便于相互区分，并不用于限制本发明的保护范围。
本公开还提供一种电子设备，包括：处理器，所述处理器用于执行存储于存储器的计算机程序，所述计算机程序被处理器执行时实现上述实施例的方法。
图6为本公开提供的一种电子设备的结构示意图，图6示出了适于用来实现本发明实施例实施方式的示例性电子设备的框图。图6显示的电子设备仅仅是一个示例，不应对本发明实施例的功能和使用范围带来任何限制。
如图6所示，电子设备600以通用计算设备的形式表现。电子设备600的组件可以包括但不限于：一个或者多个处理器610，系统存储器620，连接不同系统组件(包括系统存储器620和处理器)的总线630。
总线630表示几类总线结构中的一种或多种，包括存储器总线或者存储器控制器，外围总线，图形加速端口，处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说，这些体系结构包括但不限于工业标准体系结构(ISA)总线，微通道体系结构(MAC)总线，增强型ISA总线、视频电子标准协会(VESA)局域总线以及外围组件互连(PCI)总线。
电子设备600典型地包括多种计算机系统可读介质。这些介质可以是任何能够被电子设备600访问的介质，包括易失性和非易失性介质，可移动的和不可移动的介质。
系统存储器620可以包括易失性存储器形式的计算机系统可读介质，例如随机存取存储器(RAM)640和/或高速缓存存储器650。电子设备600可以进一步包括其它可移动/不可移动的、易失性/非易失性计算机系统存储介质。仅作为举例，存储系统660可以用于读写不可移动的、非易失性磁介质(通常称为“硬盘驱动器”)。可以提供用于对可移动非易失性磁盘(例如“软盘”)读写的磁盘驱动器，以及对可移动非易失性光盘(例如CD-ROM、DVD-ROM或者其它光介质)读写的光盘驱动器。在这些情况下，每个驱动器可以通过一个或者多个数据介质接口与总线630相连。系统存储器620可以包括至少一个程序产品，该程序产品具有一组(例如至少一个)程序模块，这些程序模块被配置以执行本发明实施例各实施例的功能。
具有一组(至少一个)程序模块670的程序/实用工具680，可以存储在例如系统存储器620中，这样的程序模块670包括但不限于操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块670通常执行本发明实施例所描述的实施例中的功能和/或方法。
处理器610通过运行存储在系统存储器620中的多个程序中的至少一个程序，从而执 行各种功能应用以及信息处理，例如实现本发明实施例所提供的方法实施例。
在一些实施例中，如图6所示，电子设备600还包括I/O接口690和网络适配器691。
本公开还提供一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述实施例的方法。例如，该计算机可读存储介质为非瞬时性计算机可读存储介质。
可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质例如可以是但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本文件中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于无线、电线、光缆、RF等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言或其组合来编写用于执行本发明操作的计算机程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)域连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
本公开还提供一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行实现上述实施例的方法。
本公开还提供一种计算机程序，包括：指令，所述指令当由处理器执行时使所述处理器执行如上述实施例所述的方法。
需要说明的是，在本文中，诸如“第一”和“第二”等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅是本公开的具体实施方式，使本领域技术人员能够理解或实现本公开。对这些实施例的多种修改对本领域的技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本公开的精神或范围的情况下，在其它实施例中实现。因此，本公开将不会被限制于本文所述的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (15)

  1. 一种图像的处理方法，包括：
    获取目标物体的掩码图像，其中，所述目标物体处于运动或者静止状态，道具图像处于运动状态；和
    在所述道具图像在运动过程中与所述目标物体产生遮挡关系的情况下，基于所述目标物体的掩码图像与道具图像的位置关系，显示所述目标物体与所述道具图像。
  2. 根据权利要求1所述的图像的处理方法，其中，所述道具图像为二维平面图像，所述道具图像被划分为至少两部分，每部分具有对应的显示属性，所述显示属性包括：显示或者不显示；
    所述基于所述目标物体的掩码图像与道具图像的位置关系，显示所述目标物体与所述道具图像，包括：
    确定所述道具图像与所述目标物体的掩码图像的重叠部分；和
    根据所述重叠部分对应的显示属性，确定是否显示所述道具图像的所述重叠部分。
  3. 根据权利要求2所述的图像的处理方法，其中，所述根据所述重叠部分对应的显示属性，确定是否显示所述道具图像的所述重叠部分，包括：
    针对所述重叠部分的每个坐标点，确定所述坐标点属于所述道具图像的目标部分，若所述目标部分的显示属性为显示，则所述坐标点显示所述道具图像的像素点，若所述目标部分的显示属性为不显示，则所述坐标点显示所述目标对象的像素点。
  4. 根据权利要求2或3所述的图像的处理方法，其中，所述基于所述目标物体的掩码图像与道具图像的位置关系，显示所述目标物体与所述道具图像，还包括：确定所述重叠部分的显示属性。
  5. 根据权利要求4所述的图像的处理方法，其中，所述确定所述重叠部分的显示属性包括：
    在对道具图像划分为道具图像前部分和道具图像后部分的情况下，基于道具图像的中心点O建立平面坐标系，在确定道具图像与目标物体的掩码图像重叠部分后，获取 重叠部分的每个坐标点，若重叠部分对应的X轴的坐标点大于0，则表示重叠部分的显示属性为显示，和若重叠部分对应的X轴的坐标点小于0，则表示重叠部分的显示属性为不显示。
  6. 根据权利要求1所述的图像的处理方法，其中，所述道具图像为三维立体图像；
    所述基于所述目标物体的掩码图像与道具图像的位置关系，显示所述目标物体与所述道具图像，包括：
    获取所述目标物体的掩码图像与道具图像的重叠部分；
    针对所述重叠部分的每个坐标点，获取所述掩码图像的所述坐标点的第一深度信息，并获取所述道具图像的所述坐标点的第二深度信息；和
    基于所述第一深度信息和所述第二深度信息的值，确定所述坐标点显示所述目标对象的像素点或者显示所述道具图像的像素点。
  7. 根据权利要求6所述的图像的处理方法，其中，所述基于所述第一深度信息和所述第二深度信息的值，确定所述坐标点显示所述掩码图像的像素点或者显示所述道具图像的像素点，包括：
    若所述第一深度信息大于所述第二深度信息，则确定所述坐标点显示所述道具图像的像素点，若所述第一深度信息小于所述第二深度信息，确定所述坐标点显示所述目标对象的像素点。
  8. 根据权利要求6或7所述的图像的处理方法，其中，所述获取所述掩码图像的所述坐标点的第一深度信息，包括：
    基于所述掩码图像构建目标对象盒子，所述目标对象盒子具有深度信息；和
    基于所述目标对象盒子的深度信息确定所述掩码图像的所述坐标点的第一深度信息。
  9. 根据权利要求8所述的图像的处理方法，其中，所述目标对象盒子的中心点为目标对象的中心点，所述目标对象盒子的高度信息为目标对象最高点至最低点的高度，所述目标对象盒子的宽度信息为目标对象从最左边到最右边的宽度。
  10. 根据权利要求8或9所述的图像的处理方法，其中，获取所述道具图像的所述坐标点的第二深度信息包括：
    基于所述道具图像相对所述目标对象盒子的中心点的位置信息，确定所述道具图像的第二深度信息。
  11. 一种图像的处理装置，包括：
    掩码图像获取模块，用于获取目标物体的掩码图像，其中，所述目标物体处于运动或者静止状态，道具图像处于运动状态；和
    显示模块，用于在所述道具图像在运动过程中与所述目标物体产生遮挡关系的情况下，基于所述目标物体的掩码图像与道具图像的位置关系，显示所述目标物体与所述道具图像。
  12. 一种电子设备，包括：
    一个或多个处理器；和
    存储装置，用于存储一个或多个程序，
    当所述一个或多个程序被所述一个或多个处理器执行时，使得所述一个或多个处理器实现如权利要求1至10中任一项所述的图像的处理方法。
  13. 一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如权利要求1至10中任一项所述的图像的处理方法。
  14. 一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如权利要求1至10中任一项所述的图像的处理方法。
  15. 一种计算机程序，包括：
    指令，所述指令当由处理器执行时使所述处理器执行如权利要求1至10中任一项所述的图像的处理方法。
PCT/CN2023/086211 2022-04-11 2023-04-04 图像的处理方法、装置、设备、存储介质和程序产品 WO2023197912A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210375988.4 2022-04-11
CN202210375988.4A CN114693780A (zh) 2022-04-11 2022-04-11 图像的处理方法、装置、设备、存储介质和程序产品

Publications (1)

Publication Number Publication Date
WO2023197912A1 true WO2023197912A1 (zh) 2023-10-19

Family

ID=82142405

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/086211 WO2023197912A1 (zh) 2022-04-11 2023-04-04 图像的处理方法、装置、设备、存储介质和程序产品

Country Status (2)

Country Link
CN (1) CN114693780A (zh)
WO (1) WO2023197912A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114693780A (zh) * 2022-04-11 2022-07-01 北京字跳网络技术有限公司 图像的处理方法、装置、设备、存储介质和程序产品

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110929651A (zh) * 2019-11-25 2020-03-27 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN113222830A (zh) * 2021-03-05 2021-08-06 北京字跳网络技术有限公司 图像处理方法和装置
CN113628132A (zh) * 2021-07-26 2021-11-09 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN114693780A (zh) * 2022-04-11 2022-07-01 北京字跳网络技术有限公司 图像的处理方法、装置、设备、存储介质和程序产品

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110929651A (zh) * 2019-11-25 2020-03-27 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN113222830A (zh) * 2021-03-05 2021-08-06 北京字跳网络技术有限公司 图像处理方法和装置
CN113628132A (zh) * 2021-07-26 2021-11-09 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN114693780A (zh) * 2022-04-11 2022-07-01 北京字跳网络技术有限公司 图像的处理方法、装置、设备、存储介质和程序产品

Also Published As

Publication number Publication date
CN114693780A (zh) 2022-07-01

Similar Documents

Publication Publication Date Title
CN111815755B (zh) 虚拟物体被遮挡的区域确定方法、装置及终端设备
WO2020186935A1 (zh) 虚拟对象的显示方法、装置、电子设备和计算机可读存储介质
WO2021139408A1 (zh) 显示特效的方法、装置、存储介质及电子设备
WO2018188499A1 (zh) 图像、视频处理方法和装置、虚拟现实装置和存储介质
CN109064390B (zh) 一种图像处理方法、图像处理装置及移动终端
CN108347657B (zh) 一种显示弹幕信息的方法和装置
US9135678B2 (en) Methods and apparatus for interfacing panoramic image stitching with post-processors
WO2020248900A1 (zh) 全景视频的处理方法、装置及存储介质
WO2022012085A1 (zh) 人脸图像处理方法、装置、存储介质及电子设备
CN110796664B (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
WO2020143728A1 (zh) 画面渲染方法、装置、终端及对应的存储介质
WO2023197912A1 (zh) 图像的处理方法、装置、设备、存储介质和程序产品
WO2015188666A1 (zh) 三维视频滤波方法和装置
WO2021135683A1 (zh) 一种显示终端调整方法及显示终端
CN110781823A (zh) 录屏检测方法、装置、可读介质及电子设备
WO2022247630A1 (zh) 图像处理方法、装置、电子设备及存储介质
WO2020215789A1 (zh) 虚拟画笔实现方法、装置和计算机可读存储介质
WO2023092950A1 (zh) 虚拟场景的素材处理方法及装置、电子设备、存储介质和计算机程序产品
JP7262530B2 (ja) 位置情報の生成方法、関連装置及びコンピュータプログラム製品
CN112565883A (zh) 一种用于虚拟现实场景的视频渲染处理系统和计算机设备
CN109816791B (zh) 用于生成信息的方法和装置
US20240177409A1 (en) Image processing method and apparatus, electronic device, and readable storage medium
CN110390717B (zh) 3d模型重建方法、装置及电子设备
CN112465692A (zh) 图像处理方法、装置、设备及存储介质
KR102534449B1 (ko) 이미지 처리 방법, 장치, 전자 장치 및 컴퓨터 판독 가능 저장 매체

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23787557

Country of ref document: EP

Kind code of ref document: A1