WO2019047984A1 - Procédé et dispositif de traitement d'images, dispositif électronique et support de stockage lisible par ordinateur - Google Patents

Procédé et dispositif de traitement d'images, dispositif électronique et support de stockage lisible par ordinateur Download PDF

Info

Publication number
WO2019047984A1
WO2019047984A1 PCT/CN2018/105120 CN2018105120W WO2019047984A1 WO 2019047984 A1 WO2019047984 A1 WO 2019047984A1 CN 2018105120 W CN2018105120 W CN 2018105120W WO 2019047984 A1 WO2019047984 A1 WO 2019047984A1
Authority
WO
WIPO (PCT)
Prior art keywords
background
video
image
frame rate
scene
Prior art date
Application number
PCT/CN2018/105120
Other languages
English (en)
Chinese (zh)
Inventor
张学勇
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201710811811.3A external-priority patent/CN107509045A/zh
Priority claimed from CN201710812756.XA external-priority patent/CN107644440A/zh
Priority claimed from CN201710811502.6A external-priority patent/CN107610076A/zh
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2019047984A1 publication Critical patent/WO2019047984A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing

Definitions

  • the present invention relates to image processing technology, and more particularly to an image processing method and apparatus, an electronic device, and a computer readable storage medium.
  • Embodiments of the present invention provide an image processing method, an image processing apparatus, an electronic device, and a computer readable storage medium.
  • the image processing method of the embodiment of the present invention is used for an electronic device, the image processing method includes: collecting a first scene video of a current user; acquiring a plurality of depth images of the current user; and acquiring a frame rate according to the first scene video Matching the second scene video and the dynamic background to be fused; processing each frame scene image of the second scene video according to the plurality of depth images to obtain the current user's character area in each frame scene image Corresponding person region image; merging each of the character region images with a background image of a corresponding frame in the dynamic background to be fused to obtain a merged image.
  • An image processing device is used in an electronic device.
  • the image processing device includes a visible light camera, a depth image acquisition component, and a processor.
  • the visible light camera is configured to collect a first scene video of a current user.
  • the depth image acquisition component is configured to acquire a plurality of depth images of the current user.
  • the processor is configured to: acquire, according to the first scene video, a second scene video that matches a frame rate and a dynamic background to be fused; and process, according to the multiple depth images, each frame image of the second scene video, to Acquiring a character area of the current user in each frame scene image to obtain a corresponding person area image; and merging each of the character area images with a background image of a corresponding frame in the dynamic background to be merged to obtain a merged image.
  • An electronic device of an embodiment of the invention includes one or more processors, a memory, and one or more programs. Wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the program comprising instructions for performing the image processing method described above.
  • a computer readable storage medium in accordance with an embodiment of the present invention includes a computer program for use in conjunction with an electronic device capable of imaging, the computer program being executable by a processor to perform the image processing method described above.
  • the image processing method, the image processing apparatus, the electronic device, and the computer readable storage medium of the embodiments of the present invention extract the character regions in each frame scene image by acquiring a plurality of depth images of the current user. Since the acquisition of the depth image is not easily affected by factors such as illumination and color distribution in the scene, the corresponding person region extracted by the plurality of depth images is more accurate, and in particular, the boundary of the person region can be accurately calibrated. In addition, because the frame rate of the second scene video and the dynamic background to be merged is matched, the more accurate character area image and the background image of the corresponding frame in the dynamic background to be merged can be well integrated.
  • FIG. 1 is a schematic block diagram of an image processing apparatus according to an embodiment of the present invention.
  • FIG. 2 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
  • FIG. 3 is a schematic flowchart diagram of an image processing method according to an embodiment of the present disclosure.
  • FIG. 4 is a schematic flowchart of an image processing method according to Embodiment 2 of the present invention.
  • FIG. 5 is a schematic flowchart diagram of an image processing method according to Embodiment 3 of the present invention.
  • FIG. 6 is a schematic flowchart diagram of an image processing method according to Embodiment 4 of the present invention.
  • FIG. 7 is a schematic flowchart diagram of an image processing method according to Embodiment 5 of the present invention.
  • FIGS. 8(a) to 8(e) are schematic diagrams of scenes of structured light measurement according to an embodiment of the present invention.
  • 9(a) and 9(b) are schematic diagrams showing a scene of structured light measurement according to an embodiment of the present invention.
  • FIG. 10 is a schematic flowchart diagram of an image processing method according to Embodiment 6 of the present invention.
  • FIG. 11 is a schematic flowchart of an image processing method according to Embodiment 7 of the present invention.
  • FIG. 12A is a schematic flowchart of an image processing method according to Embodiment 8 of the present invention.
  • FIG. 12B is a schematic flowchart diagram of an image processing method according to Embodiment 9 of the present invention.
  • FIG. 13 is a schematic block diagram of another image processing apparatus according to an embodiment of the present invention.
  • FIG. 14 is a schematic block diagram of another electronic device according to an embodiment of the present invention.
  • an embodiment of the present invention provides an image processing method.
  • the image processing method of the embodiment of the present invention can be implemented by an image processing apparatus according to an embodiment of the present invention, and the image processing apparatus is used for an electronic device.
  • FIG. 1 is a schematic block diagram of an image processing apparatus according to an embodiment of the present invention
  • FIG. 2 is a schematic structural diagram of an electronic apparatus according to an embodiment of the present invention.
  • the image processing apparatus 100 is used for the electronic apparatus 1000, that is, the electronic apparatus 1000 includes the image processing apparatus 100.
  • the image processing apparatus 100 includes a visible light camera 11, a depth image acquisition component 12, and a processor 20.
  • the visible light camera 11 is used to capture a video taken by a user.
  • the depth image acquisition component 12 is configured to collect a plurality of depth images of the user.
  • the processor 20 is configured to process the collected data.
  • the depth image capturing component 12 may further include: a structured light projector 121 and a structured light camera 122.
  • the structured light projector 121 is configured to project structured light to the current user.
  • the structured light camera 122 captures a structured light image modulated by the user.
  • the electronic device 1000 may include a mobile phone, a tablet computer, a notebook computer, a smart bracelet, a smart watch, a smart helmet, smart glasses, and the like.
  • FIG. 3 is a schematic flowchart of an image processing method according to an embodiment of the present invention.
  • the image processing method of the embodiment is implemented based on the image processing apparatus and the electronic device of the foregoing embodiment, and the image processing method includes:
  • Step S301 collecting a first scene video of the current user.
  • the frame rate of the first scene video may be referred to as a video frame rate
  • the frame rate of the dynamic background may be referred to as a background frame rate
  • the application is installed in the electronic device, and the user opens the application to collect the current video scene image of the user through the visible light camera, which is called a first scene video, where the first scene video includes a plurality of frame scene images.
  • Step S302 Acquire a plurality of depth images of the current user.
  • Structured Light is a specific light that is projected onto the surface of the object. Since the surface of the object is uneven, the surface of the object changes and the possible gap modulates the illuminated light and then emits it.
  • the device for generating structured light is a structured light projector, and the structured light projector emits structured light to the user.
  • the structured light projector emits structured light to the user.
  • the structured light is irradiated onto the human body, since the surface of the user's body is uneven, the body is in the opposite structure. When the light is reflected, it causes distortion of the structured light.
  • the structured light image modulated by the current user is captured by the structured optical camera on the electronic device, and the phase information corresponding to each pixel in the structured light image is demodulated, and the phase information is converted into depth information, and the depth information is obtained according to the depth information.
  • One depth image in the depth image and in turn, all depth images.
  • the depth image is multiple, and each frame in the video corresponds to one depth image.
  • the character in the video is stationary in the multi-frame scene image, there will be a stationary multi-frame scene image corresponding to the same depth image.
  • the depth image collected by the structured light camera and the scene image collected by the visible light camera correspond to the same frame image, and the corresponding scene range is the same, and each pixel in the scene image can find the corresponding pixel in the depth image. Depth information.
  • Step S303 Acquire a second scene video with a frame rate matching and a dynamic background to be merged according to the first scene video.
  • step S303 includes step S3031.
  • step S3031 Select a matching target dynamic background from the plurality of dynamic backgrounds as the dynamic background to be merged according to the video frame rate used by the first scene video, and use the first scene video as the second scene video.
  • a plurality of dynamic backgrounds are stored in the storage unit of the electronic device or in the cloud server. Specifically, the frame rate used by the multiple dynamic backgrounds is different. According to the frame rate adopted by the first scene video and the preset threshold, the target dynamic background is selected from the multiple dynamic backgrounds, and the target dynamic background and the first scene are selected. The difference between the frame rates used by the video is less than a preset threshold.
  • dynamic backgrounds and video should use standard frame rates, such as 30 frames per second, 60 frames per second, 120 frames per second, and so on.
  • standard frame rates such as 30 frames per second, 60 frames per second, 120 frames per second, and so on.
  • a non-zero preset threshold may be set.
  • the frame rate adopted by the first scene video is 30 frames/second
  • the preset threshold is 2 frames
  • the frame rate range of the target dynamic background is 29 frames to 31 frames.
  • the target dynamic background having the smallest frame rate difference from the first scene video is selected, such as the target dynamic background of the frame rate of 30 frames in this example; if there is a frame rate difference between the frame and the first scene video
  • one target dynamic background may be selected, such as the target dynamic background of the frame rate of 29 frames/second or 31 frames/second in this example.
  • the target dynamic background may be presented to the user, and the user selects a target dynamic background according to the degree of preference.
  • one of the processors of the electronic device randomly selects one as the target dynamic background.
  • FIG. 5 is a schematic flowchart of an image processing method according to Embodiment 3 of the present invention. As shown in FIG. 5, step S303 may also include steps S3032 and S3033. Step S3032: Acquire a first dynamic background.
  • the first dynamic background is selected from the plurality of dynamic backgrounds, wherein a background frame rate of the first dynamic background is closest to a video frame rate of the first scene video, or a background frame rate of the first dynamic background is first There is a multiple relationship between the video frame rate of the scene video.
  • Step S3033 Perform frame extraction on the first scene video and/or the first dynamic background to obtain a second scene video and a second dynamic background that match each other, wherein the second dynamic background is used as the dynamic background to be merged.
  • the video frame rate of the first scene video is greater than the background frame rate of the first dynamic background
  • frame extraction is performed on the first scene video, so that the video frame rate of the second scene video obtained by the frame extraction is compared with the first dynamic background.
  • the background frame rate matches and the first dynamic background is used as the second dynamic background.
  • the method for extracting may adopt a frame-by-frame extraction manner on the first scene video, so that the video frame rate of the first scene video is halved.
  • the difference N between the video frame rate of the first scene video and the background frame rate of the first dynamic background is calculated.
  • the partial scene frame extraction may be performed on the first scene video, and the frame frame extraction N frame deletion is performed on the frame included in the second scene, so that the video frame rate of the first scene video matches the background frame rate of the first dynamic background.
  • the video frame rate of the first scene video is smaller than the background frame rate of the first dynamic background, performing frame extraction on the first dynamic background, so that the video frame rate of the first scene video and the background frame of the second dynamic background obtained by the frame extraction The rate matches and the first scene video is used as the second scene video.
  • the background frame rate of the first dynamic background is greater than the video frame rate of the first scene video.
  • the difference N1 is performed by performing frame-by-frame extraction of N1 frames for the number of frames per second, reducing the frame rate of the first dynamic background, so that the background frame rate of the first dynamic background and the video frame rate of the first scene video are completely matched;
  • a random extraction method is adopted, and the number of frames per second is randomly selected and deleted.
  • the difference between the frame rate of the first dynamic background and the first scene video is calculated. For example, if the difference is 7 frames, 7 frames are randomly selected from the multi-frame background image of each second for the second dynamic background. , wherein the 7 frames may be selected from the beginning of the multi-frame background image included in each second 7 frames or 7 frames at the end for deletion, or may be randomly selected 7 frames to delete, so that the obtained second scene video is The video frame rate and the background frame rate of the second dynamic background exactly match.
  • FIG. 6 is a schematic flowchart of an image processing method according to Embodiment 4 of the present invention. As shown in FIG. 6, step S303 may further include steps S3032 and S3034. Step S3034, inserting a supplementary frame in the first scene video and/or the first dynamic background to obtain a second scene video and a second dynamic background that match each other, wherein the second dynamic background is used as the dynamic background to be merged .
  • the video frame rate of the first scene video is greater than the background frame rate of the first dynamic background, a supplementary frame is inserted between adjacent frames of the first dynamic background, so that the video frame rate and the frame insertion of the first scene video are performed.
  • the obtained background frame rate of the second dynamic background is matched, and the first scene video is used as the second scene video.
  • the video frame rate of the first scene video is greater than the background frame rate of the first dynamic background, as a possible implementation manner, according to the video frame rate of the first scene video and the background frame rate of the first dynamic background.
  • the difference N in the frame included in the second dynamic background, randomly inserts a supplementary frame in the adjacent frame, and inserts a total of N supplementary frames to obtain a second dynamic background, so that the second dynamic background and The frame rate of the first scene video is completely matched.
  • a method of inserting the supplementary frame that is, inserting a supplementary frame between each adjacent frame in the first dynamic background Obtaining a second dynamic background such that the background frame rate of the second dynamic background is doubled compared to the background frame rate of the first dynamic background, thereby completely matching the video frame rate of the first scene video.
  • the difference N between the video frame rate of the first scene video and the background frame rate of the first dynamic background is calculated as In another possible implementation manner, according to the difference N between the background frame rate of the first dynamic background and the video frame rate of the first scene video, a random insertion manner is adopted, and the beginning of the frame included in the second dynamic background is included. Or, at the end, a total of N supplementary frames are inserted to increase the background frame rate of the first dynamic background, so that the video frame rate of the first scene video and the obtained background frame rate of the second dynamic background are completely matched. For example, the difference between the frame rate of the first dynamic background and the first scene video is calculated. For example, if the difference is 7 frames, the first dynamic background is inserted at the beginning or end of the multi-frame background image of each second. The frame is such that the video frame rate of the second scene video and the background frame rate of the second dynamic background are exactly matched.
  • the pixels of the inserted 7-frame image may adopt the same pixel value corresponding to the last 7 frames or the first 7 frames adjacent to the inserted 7 frames.
  • the video frame rate of the first scene video is smaller than the background frame rate of the first dynamic background, insert a supplementary frame in the adjacent frame of the first scene video, so that the video frame rate of the second scene video obtained by the frame insertion is the first
  • the background frame rate of the dynamic background matches, and the first dynamic background is used as the second dynamic background.
  • each frame included in the first scene video is used in each frame.
  • a supplementary frame is inserted into an adjacent frame, and the N1 frame is inserted, and the frame rate of the first scene video is increased, so that the background frame rate of the first dynamic background and the video frame rate of the second scene video are completely matched.
  • each pixel of the supplementary frame inserted in the adjacent frame is determined according to the value of the corresponding pixel in the adjacent frame.
  • the value of each pixel of the inserted supplementary frame may be an average of the average of the pixels in the adjacent frame; as another possible implementation, each pixel of the inserted supplementary frame is inserted.
  • the value may be the same as the pixel point of the image of the previous frame relative to the inserted supplementary frame in the adjacent frame, or the pixel point of the image of the next frame relative to the inserted supplementary frame in the adjacent frame.
  • a method for determining a value of a pixel point may be selected by a person skilled in the art according to the actual situation, which is not limited in this embodiment.
  • Step S304 processing each frame scene image of the second scene video according to the plurality of depth images, so as to obtain a current person region in each frame scene image to obtain a corresponding person region image.
  • the face region in the scene image is identified, and according to the depth image corresponding to the frame image, the depth information corresponding to the face region is obtained, and the depth information of the face region is determined according to the depth information of the face region.
  • the depth range of the person region is then determined, and the person region that is connected to the face region and falls within the depth range is determined according to the depth range of the person region, thereby acquiring the person region image, and the person region image can be accurately determined.
  • Step S305 merging each of the character area images with the background image of the corresponding frame in the dynamic background to be merged to obtain a merged image.
  • the second scene video and the dynamic background frame rate to be merged are matched, and the character area image and each frame image in the first video have a corresponding relationship, and the character area image can be matched with the selected corresponding frame of the dynamic background to be merged.
  • the image is fused, and the edge of each character region image obtained is relatively clear and accurate, and the merged image obtained after fusion is better.
  • the frame rate difference between the first scene video and the background video to be fused is not 0, but is a threshold with a small value, for example, 1 frame or 2 frames, when the fusion is performed, due to the difference of the frame rate The value is small, and after fusion, the human eye is hard to perceive that there is a picture without fusion.
  • the first scene video of the current user is collected, and multiple depth images of the current user are acquired, and the second scene video and the dynamic background to be merged are obtained according to the first scene video.
  • the depth image is processed to process each frame image of the second scene video to obtain a person region image in each frame scene image, and each of the character region images is merged with the background image of the corresponding frame in the dynamic background to be merged to obtain a merged image.
  • the existing methods of segmenting characters and background mainly divide the characters and backgrounds according to the similarity and discontinuity of adjacent pixels in pixel values, but this segmentation method is susceptible to environmental factors such as external illumination.
  • the image processing method of the embodiment of the present invention extracts a character region in each scene image in the scene video by acquiring a depth image of the current user. Since the acquisition of the depth image is not easily affected by factors such as illumination and color distribution in the scene, the character region extracted by the depth image is more accurate, and in particular, the boundary of the person region can be accurately calibrated. Further, the dynamic background to be merged is matched with the video frame rate of the first scene captured by the user, so that the more accurate image of the person region can be well integrated with the dynamic background to be merged.
  • FIG. 7 is a schematic flowchart of an image processing method according to Embodiment 5 of the present invention, such as As shown in FIG. 4, based on any of the above embodiments, step S302 may further include the following steps:
  • Step S3021 projecting the structured light to the current user.
  • the application installed on the electronic device calls the device that generates the structured light, that is, the structured light projector, and the structured light projector projects the structured light of the certain mode onto the face and the body of the current user, and on the face of the current user and The surface of the body forms a structured light image modulated by the current user.
  • the pattern of the structured light may be a laser stripe, a Gray code, a sine stripe, a non-uniform speckle, or the like.
  • step S3022 a structured light image modulated by the current user is captured.
  • the structured light camera in the depth image acquisition component captures a structured light image modulated by the user.
  • Step S3023 demodulating phase information corresponding to each pixel in the structured light image.
  • the phase information of the modulated structured light is changed compared to the unmodulated structured light, and the structured light presented in the structured light image is the structured light after the distortion is generated, wherein the changed phase information It can represent the depth information of the object. Therefore, the structured light camera demodulates the phase information corresponding to each pixel in the structured light image.
  • Step S3024 the phase information is converted into depth information, and one of the plurality of depth images is generated according to the depth information.
  • the phase of the image is a function of the spatial position, and the phase relationship between the phase and the spatial position is obtained by the phase information corresponding to each pixel, thereby obtaining the depth information corresponding to each pixel point, according to the position and depth of each pixel point.
  • Information generating one of a plurality of depth images.
  • a corresponding depth image may be generated according to depth information corresponding to each frame scene image, thereby acquiring a plurality of depth images of the current user.
  • the first scene video of the current user is collected, and multiple depth images of the current user are acquired, and the second scene video and the dynamic background to be merged are obtained according to the first scene video.
  • the depth image is processed to process each frame image of the second scene video to obtain a person region image in each frame scene image, and each of the character region images is merged with the background image of the corresponding frame in the dynamic background to be merged to obtain a merged image.
  • the existing methods of segmenting characters and background mainly divide the characters and backgrounds according to the similarity and discontinuity of adjacent pixels in pixel values, but this segmentation method is susceptible to environmental factors such as external illumination.
  • the image processing method of the embodiment of the present invention extracts a character region in each scene image in the scene video by acquiring a depth image of the current user. Since the acquisition of the depth image is not easily affected by factors such as illumination and color distribution in the scene, the character region extracted by the depth image is more accurate, and in particular, the boundary of the person region can be accurately calibrated. Further, the dynamic background to be merged is matched with the video frame rate of the first scene captured by the user, so that the more accurate image of the person region can be well integrated with the dynamic background to be merged.
  • the grating projection technique belongs to the surface structure light in a broad sense.
  • a sinusoidal stripe is generated by computer programming, and the sinusoidal stripe is projected to the object to be measured through the structured light projector, and then the stripe object is photographed by the structured light camera.
  • the degree of bending after modulation, and then demodulating the curved stripe to obtain a phase, and then converting the phase into depth information can obtain a depth image.
  • the depth image acquisition component needs to be calibrated before using the structured light for depth information acquisition.
  • the calibration includes geometric parameters (for example, relative position parameters between the structured light camera and the structured light projector). Calibration, internal parameters of the structured light camera, and calibration of internal parameters of the structured light projector.
  • the first step computer programming produces sinusoidal stripes. Since the phase needs to be acquired by using the distorted stripes, for example, the phase is obtained by the four-step phase shift method, four phase differences are generated here.
  • the stripe is then projected by the structured light projector to the object to be measured (the mask shown in Fig. 8(a)).
  • the structured light camera collects the image on the left side of Fig. 8(b) and reads Take the stripe of the reference plane as shown on the right side of Figure 8(b).
  • the second step is to perform phase recovery.
  • the structured optical camera calculates the modulated phase based on the acquired four modulated fringe patterns (ie, the structured light image), and the phase map obtained at this time is a truncated phase map. Since the result obtained by the four-step phase shifting algorithm is calculated by the inverse tangent function, the phase after the structured light modulation is limited to [- ⁇ , ⁇ ], that is, whenever the modulated phase exceeds [- ⁇ ] , ⁇ ], which will start again. The resulting phase principal value is shown in Figure 8(c).
  • the bounce processing is required, that is, the truncated phase is restored to the continuous phase.
  • the left side is the modulated continuous phase diagram and the right side is the reference continuous phase diagram.
  • the modulated continuous phase and the reference continuous phase are subtracted to obtain a phase difference (ie, phase information), the phase difference is used to represent the depth information of the measured object relative to the reference surface, and then the phase difference is substituted into the phase and depth.
  • phase difference ie, phase information
  • the formula the parameters involved in the formula are calibrated
  • the three-dimensional model of the object to be tested as shown in Fig. 8(e) can be obtained.
  • the structured light used in the embodiments of the present invention may be any other pattern in addition to the above-mentioned grating, depending on the specific application scenario.
  • the present invention can also use the speckle structure light to perform the acquisition of the depth information of the current user.
  • the method of obtaining depth information by the speckle structure light is to use a substantially flat plate diffraction element having an embossed diffraction structure of a specific phase distribution, the cross section being a step relief structure having two or more concavities and convexities.
  • the thickness of the substrate in the diffractive element is approximately 1 micron, and the height of each step is not uniform, and the height may range from 0.7 micron to 0.9 micron.
  • the structure shown in Fig. 9(a) is a partial diffraction structure of the collimating beam splitting element of the present embodiment.
  • Fig. 9(b) is a cross-sectional side view taken along section A-A, and the units of the abscissa and the ordinate are both micrometers.
  • the speckle pattern generated by the speckle structure light has a high degree of randomness and will change the pattern as the distance is different. Therefore, before using the speckle structure light to obtain depth information, it is first necessary to calibrate the speckle pattern in the space. For example, in a range of 0 to 4 meters from the structured optical camera, a reference plane is taken every 1 cm. After the calibration is completed, 400 speckle images are saved. The smaller the calibration interval, the higher the accuracy of the acquired depth information. Subsequently, the structured light projector projects the speckle structure light onto the object to be measured (i.e., the current user), and the difference in height of the surface of the object to be measured causes the speckle pattern of the speckle structure light projected onto the object to be measured to change.
  • the object to be measured i.e., the current user
  • the speckle pattern ie, the structured light image
  • the speckle pattern is cross-correlated with the 400 speckle images saved in the previous calibration, and 400 correlations are obtained. image.
  • the position of the measured object in the space will show the peak on the correlation image, and the above peaks will be superimposed and interpolated to obtain the depth information of the measured object.
  • a collimating beam splitting element is used, which not only has the function of collimating the non-collimated beam, but also has the function of splitting light, that is, the non-collimated light reflected by the mirror passes through the collimating beam splitting element.
  • a plurality of collimated beams are emitted at different angles, and the cross-sectional areas of the plurality of collimated beams are approximately equal, and the energy fluxes are approximately equal, so that the effect of the astigmatism of the diffracted light by the beam is better.
  • the laser light is scattered to each beam, which further reduces the risk of harming the human eye, and the speckle structure light consumes the same collection effect compared to other uniformly arranged structured light.
  • the battery is lower.
  • step S304 is a schematic flowchart diagram of an image processing method according to Embodiment 6 of the present invention, which may be implemented by a processor in an image processing apparatus, by processing a scene image and a depth image to extract a current user area in a scene image.
  • the image of the person area as shown in FIG. 10, on the basis of any of the above embodiments, step S304 may further include the following steps:
  • Step S3041 identifying a face region in the scene image for each frame scene image.
  • the processor in the electronic device can identify the face region in the scene image by using the trained deep learning model for each frame scene image.
  • Step S3042 Obtain depth information corresponding to the face region from the depth image corresponding to the scene image.
  • the depth information of the face region may be determined according to the correspondence between the scene image and the depth image. Since the face region includes features such as a nose, an eye, an ear, and a lip, the depth data corresponding to each feature in the face region is different in the depth image, for example, when the face is facing the depth image capturing component. In the depth image captured by the depth image acquisition component, the depth data corresponding to the nose may be small, and the depth data corresponding to the ear may be large. Therefore, the depth information of the face area described above may be a numerical value or a numerical range. Wherein, when the depth information of the face region is a value, the value may be obtained by averaging the depth data of the face region; or, by taking the median value of the depth data of the face region.
  • Step S3043 determining a depth range of the person region according to the depth information of the face region.
  • the processor may determine the depth information of the face area according to the face area.
  • the depth information sets the depth range of the person area.
  • Step S3044 determining a person area that is connected to the face area and falls within the depth range according to the depth range of the person area to obtain a person area image.
  • a person region that falls within the depth range and is connected to the face region is extracted according to the depth range of the person region to obtain a person region image.
  • the person region image can be extracted from the scene image according to the depth information. Since the acquisition of the depth information is not affected by the image of the illumination, color temperature and the like in the environment, the extracted image of the person region is more accurate.
  • the first scene video of the current user is collected, and multiple depth images of the current user are acquired, and the second scene video and the dynamic background to be merged are obtained according to the first scene video.
  • the depth image is processed to process each frame image of the second scene video to obtain a person region image in each frame scene image, and each of the character region images is merged with the background image of the corresponding frame in the dynamic background to be merged to obtain a merged image.
  • the existing methods of segmenting characters and background mainly divide the characters and backgrounds according to the similarity and discontinuity of adjacent pixels in pixel values, but this segmentation method is susceptible to environmental factors such as external illumination.
  • the image processing method of the embodiment of the present invention extracts a character region in each scene image in the scene video by acquiring a depth image of the current user. Since the acquisition of the depth image is not easily affected by factors such as illumination and color distribution in the scene, the character region extracted by the depth image is more accurate, and in particular, the boundary of the person region can be accurately calibrated. Further, the dynamic background to be merged is matched with the video frame rate of the first scene captured by the user, so that the more accurate image of the person region can be well integrated with the dynamic background to be merged.
  • FIG. 11 is a schematic flowchart diagram of an image processing method according to Embodiment 7 of the present invention.
  • the steps of the method may be implemented by a processor in an image processing apparatus, as shown in FIG.
  • the image processing method further includes the following steps:
  • Step S801 processing the scene image to obtain a full field edge image of the scene image.
  • the processor first performs edge extraction on the scene image to obtain a full-field edge image, wherein the edge line in the full-field edge image includes an edge line of the current user and the background object in the scene where the current user is located.
  • edge extraction of the scene image can be performed by the Canny operator.
  • the core of the algorithm for edge extraction of Canny operator mainly includes the following steps: First, convolving the scene image with 2D Gaussian filter template to eliminate noise; then, using the differential operator to obtain the gradient value of the gray level of each pixel, and Calculating the gradient direction of the gray level of each pixel according to the gradient value, and finding the adjacent pixel of the corresponding pixel along the gradient direction by the gradient direction; then, traversing each pixel, if the gray value of a certain pixel and the two directions of the gradient direction If the gray value of the neighboring pixel is not the largest, then the pixel is considered not to be the edge point. In this way, the pixel points at the edge position in the scene image can be determined, thereby obtaining the full-field edge image after the edge extraction.
  • Step S802 the person area image is corrected according to the full field edge image.
  • the processor corrects the image of the person region according to the full-field edge image.
  • the image of the person region is obtained by merging all the pixels in the scene image that are connected to the face region and fall within the set depth range. In some scenarios, there may be some connections with the face region and falling into the image. Objects in the depth range. Therefore, in order to make the extracted person region image more accurate, the person region image can be corrected using the full field edge map.
  • the processor may also perform secondary correction on the corrected person region image.
  • the corrected person region image may be expanded to enlarge the person region image to preserve the edge details of the person region image.
  • the image of the character area can be merged with the dynamic background to be merged, thereby obtaining a merged image.
  • the target dynamic background may be randomly selected by the processor or selected by the current user.
  • the merged merged image can be displayed on the display of the electronic device or printed by a printer connected to the electronic device.
  • the current user wants to hide the current background during the video process with the other user.
  • the image processing method of the embodiment of the present invention can be used to match the current user image corresponding to the current user to the dynamic background to be merged.
  • the background image of the corresponding frame in the frame is merged, and the merged merged image is displayed to the other party. Since the current user is in a video call with the other party, the visible light camera needs to capture the current user's scene image in real time, and the depth image capturing component also needs to collect the depth image corresponding to the current user in real time, and the scene image and depth captured by the processor in real time.
  • the image is processed so that the other party can see a smooth video frame composed of a plurality of combined images.
  • the first scene video of the current user is collected, and multiple depth images of the current user are acquired, and the second scene video and the dynamic background to be merged are obtained according to the first scene video.
  • the depth image is processed to process each frame image of the second scene video to obtain a person region image in each frame scene image, and each of the character region images is merged with the background image of the corresponding frame in the dynamic background to be merged to obtain a merged image.
  • the existing methods of segmenting characters and background mainly divide the characters and backgrounds according to the similarity and discontinuity of adjacent pixels in pixel values, but this segmentation method is susceptible to environmental factors such as external illumination.
  • the image processing method of the embodiment of the present invention extracts a character region in each scene image in the scene video by acquiring a depth image of the current user. Since the acquisition of the depth image is not easily affected by factors such as illumination and color distribution in the scene, the character region extracted by the depth image is more accurate, and in particular, the boundary of the person region can be accurately calibrated. Further, the dynamic background to be merged is matched with the video frame rate of the first scene captured by the user, so that the more accurate image of the person region can be well integrated with the dynamic background to be merged.
  • the target dynamic background is pre-stored in the cloud server or the mobile phone memory, and the background frame rate and the video frame rate of the first scene video do not necessarily match, when the video frame rate of the first scene video and the background frame rate of the target dynamic background are not When matching, the first scene video and/or the target dynamic background need to be processed, and the processed frame rate of the first scene video and the target dynamic background is matched. For this reason, another embodiment of the present invention provides image processing. Possible implementation of the method.
  • FIG. 12A is a schematic flowchart of an image processing method according to Embodiment 8 of the present invention.
  • the method for extracting a frame is used to match a first scene video with a target dynamic background frame rate. As shown in FIG. 12A, the method includes:
  • Step S901 Collect a first scene video of the current user.
  • Step S902 acquiring a plurality of depth images of the current user.
  • Step S903 processing each frame scene image of the first scene video according to the plurality of depth images, to obtain a current person region in each frame scene image to obtain a corresponding person region image.
  • Step S901 to step S903 reference may be made to step S301, step S302, and step S304 in the corresponding embodiment of FIG. 3, and details are not described herein again.
  • step S904 it is determined whether the video frame rate adopted by the first scene video does not match the background frame rate of the plurality of dynamic backgrounds. If yes, step S906 is performed, and if no, step S905 is performed.
  • the user captures the first scene video, and the frame rate of the plurality of dynamic backgrounds that are stored in advance does not necessarily have a matching relationship, and the video frame rate used by the first scene video does not match the background frame rate of the multiple dynamic backgrounds.
  • the first dynamic background is selected from a plurality of dynamic backgrounds. If a dynamic background matching the frame rate of the first scene video exists in the plurality of dynamic backgrounds, the target dynamic background is selected from the plurality of dynamic backgrounds.
  • Step S905 selecting a matching target dynamic background from a plurality of dynamic backgrounds.
  • a dynamic background in which a difference between a background frame rate and a video frame rate of the first scene video is less than a threshold is selected from a plurality of dynamic backgrounds, wherein the threshold value is small, and the threshold is an integer, for example, the threshold value is 2,
  • the video frame rate of a scene video is 30 frames/second.
  • the frame rate of the dynamic background matching the first scene video is 29 frames/second, or 30 frames/second, or 31 frames/second, which is preferentially selected.
  • the dynamic background corresponding to the frame rate difference of 0 is 30 frames/second as the matching dynamic background.
  • Step S906 selecting a first dynamic background from a plurality of dynamic backgrounds.
  • the first dynamic background is selected from the plurality of dynamic backgrounds, wherein the background frame rate of the first dynamic background and the video frame rate of the video frame rate The rate is closest, or the background frame rate of the first dynamic background has a multiple relationship with the video frame rate of the video frame rate.
  • Step S907 it is determined whether the video frame rate of the first scene video is greater than the background frame rate of the first dynamic background. If yes, step S909 is performed, and if no, step S908 is performed.
  • Step S908 Perform frame extraction on the first dynamic background to obtain a second scene video and a second dynamic background whose frame rates match each other.
  • the video frame rate of the first scene video is smaller than the background frame rate of the first dynamic background
  • frame extraction is performed on the background frame rate of the first dynamic background, so that the video frame rate of the first scene video and the frame extraction are obtained.
  • the background frame rate of the second dynamic background is matched, and the first scene video is used as the second scene video.
  • the difference between the frame rate of the first dynamic background and the first scene video for example, the difference is 7 frames
  • the first dynamic background from the multi-frame of each second A 7-frame deletion is randomly selected in the background image to obtain a second dynamic background, so that the video frame rate of the second scene video and the background frame rate of the second dynamic background are completely matched.
  • Step S909 performing frame extraction on the first scene video to obtain a second scene video and a second dynamic background whose frame rates match each other.
  • the video frame rate of the first scene video is greater than the background frame rate of the first dynamic background
  • frame extraction is performed on the first scene video, so that the video frame rate of the second scene video obtained by the frame extraction is compared with the first dynamic background.
  • the background frame rate matches and the first dynamic background is used as the second dynamic background.
  • Step S910 merging each of the character area images with the background image of the corresponding frame in the obtained dynamic background to obtain a merged image.
  • the frame rate of the obtained second scene video and the second dynamic background is matched, and the image of the person in the second scene video and the image of each frame in the second dynamic background have a corresponding relationship, and the image of the person region can be selected from the target dynamics.
  • the corresponding frame images of the background are fused, and the edges of each of the obtained person region images are relatively clear and accurate, and the merged images obtained after the fusion are better.
  • the first scene video of the current user is collected, and multiple depth images of the current user are acquired, and the second scene video and the dynamic background to be merged are obtained according to the first scene video.
  • the depth image is processed to process each frame image of the second scene video to obtain a person region image in each frame scene image, and each of the character region images is merged with the background image of the corresponding frame in the dynamic background to be merged to obtain a merged image.
  • the existing methods of segmenting characters and background mainly divide the characters and backgrounds according to the similarity and discontinuity of adjacent pixels in pixel values, but this segmentation method is susceptible to environmental factors such as external illumination.
  • the image processing method of the embodiment of the present invention extracts a character region in each scene image in the scene video by acquiring a depth image of the current user. Since the acquisition of the depth image is not easily affected by factors such as illumination and color distribution in the scene, the character region extracted by the depth image is more accurate, and in particular, the boundary of the person region can be accurately calibrated. Further, the dynamic background to be merged that matches the first scene video frame rate captured by the user is obtained, and the more accurate character area image can be well integrated with the dynamic background to be merged.
  • FIG. 12B is a schematic flowchart of an image processing method according to Embodiment 9 of the present invention.
  • the method for inserting a supplementary frame in an adjacent frame is used to match the first scene video and the target dynamic background frame rate, as shown in FIG. 12B. include:
  • Step S911 collecting a first scene video of the current user.
  • Step S912 acquiring a plurality of depth images of the current user.
  • Step S913 processing each frame scene image of the first scene video according to the plurality of depth images, so as to obtain a current person region in each frame scene image to obtain a corresponding person region image.
  • Step S911 to step S913 reference may be made to step S301, step 302, and step S304 in the corresponding embodiment of FIG. 3, and details are not described herein again.
  • step S914 it is determined whether the video frame rate used by the first scene video matches the background frame rate of the plurality of dynamic backgrounds. If yes, step S915 is performed, and if no, step S916 is performed.
  • Step S915 selecting a matching target dynamic background from a plurality of dynamic backgrounds.
  • Step S916 selecting a first dynamic background from a plurality of dynamic backgrounds.
  • Steps S914 to S916 can refer to steps S904 to S906 in the previous embodiment, and details are not described herein again.
  • step S917 it is determined whether the video frame rate of the first scene video is greater than the background frame rate of the first dynamic background. If yes, step S919 is performed, and if no, step S918 is performed.
  • Step S918 inserting a supplementary frame between adjacent frames of the first scene video, and obtaining a second scene video and a second dynamic background whose frame rates match each other.
  • the supplementary frame is inserted in the adjacent frame of the first scene video, so that the video frame rate of the second scene video obtained by the frame insertion is obtained. Matching the background frame rate of the first dynamic background and using the first dynamic background as the second dynamic background.
  • the value of each pixel in the supplementary frame inserted in the first scene video is determined according to the value of the corresponding pixel in the adjacent frame, that is, the inserted supplementary frame and the adjacent frame are completely the same scene picture.
  • Step S919 inserting a supplementary frame between adjacent frames of the first dynamic background, and obtaining a second scene video and a second dynamic background whose frame rates match each other.
  • the video frame rate of the first scene video is greater than the background frame rate of the first dynamic background, a supplementary frame is inserted between adjacent frames of the first dynamic background, so that the video frame rate and the frame insertion of the first scene video are performed.
  • the obtained background frame rate of the second dynamic background is matched, and the first scene video is used as the second scene video.
  • the value of each pixel in the supplementary frame inserted in the first dynamic background is determined according to the value of the corresponding pixel in the adjacent frame, that is, the obtained supplementary frame and the background image of the adjacent frame are exactly the same.
  • Step S920 merging each of the character area images with the background image of the corresponding frame in the obtained dynamic background to obtain a merged image.
  • the frame rate of the obtained second scene video and the second dynamic background is matched, and the image of the person in the second scene video and the image of each frame in the second dynamic background have a corresponding relationship, and the image of the person region can be selected from the target dynamics.
  • the corresponding frame images of the background are fused, and the edges of each of the obtained person region images are relatively clear and accurate, and the merged images obtained after the fusion are better.
  • the first scene video of the current user is collected, and multiple depth images of the current user are acquired, and the second scene video and the dynamic background to be merged are obtained according to the first scene video.
  • the depth image is processed to process each frame image of the second scene video to obtain a person region image in each frame scene image, and each of the character region images is merged with the background image of the corresponding frame in the dynamic background to be merged to obtain a merged image.
  • the existing methods of segmenting characters and background mainly divide the characters and backgrounds according to the similarity and discontinuity of adjacent pixels in pixel values, but this segmentation method is susceptible to environmental factors such as external illumination.
  • the image processing method of the embodiment of the present invention extracts a character region in each scene image in the scene video by acquiring a depth image of the current user. Since the acquisition of the depth image is not easily affected by factors such as illumination and color distribution in the scene, the character region extracted by the depth image is more accurate, and in particular, the boundary of the person region can be accurately calibrated. Further, the dynamic background to be merged is matched with the video frame rate of the first scene captured by the user, so that the more accurate image of the person region can be well integrated with the dynamic background to be merged.
  • FIG. 13 is a schematic diagram of a module of an electronic device according to an embodiment of the present invention.
  • the electronic device 1000 includes an image processing device 100.
  • the image processing apparatus 100 can be implemented using hardware and/or software.
  • the image processing apparatus 100 includes an imaging device 10 and a processor 20.
  • the imaging device 10 includes a visible light camera 11 and a depth image acquisition assembly 12.
  • the visible light camera 11 includes an image sensor 111 and a lens 112, and the visible light camera 11 can be used to capture color information of a current user to obtain a scene video image, wherein the image sensor 111 includes a color filter array (such as a Bayer filter array), and a lens
  • the number of 112s may be one or more.
  • the visible light camera 11 in the process of acquiring the scene video image, each imaging pixel in the image sensor 111 senses light intensity and wavelength information from the shooting scene to generate a set of original image data; the image sensor 111 sends the set of original image data to In the processor 20, the processor 20 obtains a color scene image by performing operations such as denoising, interpolation, and the like on the original image data.
  • the processor 20 can process each image pixel in the original image data one by one in a plurality of formats, for example, each image pixel can have a bit depth of 8, 10, 12 or 14 bits, and the processor 20 can be the same or different.
  • the bit depth is processed for each image pixel.
  • the depth image acquisition component 12 includes a structured light projector 121 and a structured light camera 122 that can be used to capture depth information of the current user to obtain a depth image.
  • the structured light projector 121 is for projecting structured light to a current user, wherein the structured light pattern may be a laser stripe, a Gray code, a sinusoidal stripe, or a randomly arranged speckle pattern or the like.
  • the structured light camera 122 includes an image sensor 1221 and a lens 1222, and the number of the lenses 1222 may be one or more.
  • Image sensor 1221 is used to capture a structured light image that structured light projector 121 projects onto the current user.
  • the structured light image may be transmitted by the depth acquisition component 12 to the processor 20 for processing such as demodulation, phase recovery, phase information calculation, etc. to obtain depth information of the current user.
  • the functions of the visible light camera 11 and the structured light camera 122 can be implemented by a single camera, that is, the imaging device 10 includes only one camera and one structured light projector 121, which can not only capture scene images, It is also possible to take a structured light image.
  • depth images of the current user can be acquired by a binocular vision method, a depth image acquisition method based on Time of Flight (TOF).
  • TOF Time of Flight
  • the processor 20 is further configured to fuse the image of the person region extracted from the scene image and the depth image with the image of the corresponding frame in the matching target dynamic background selected from the plurality of dynamic backgrounds.
  • the processor 20 may extract the two-dimensional image of the person region from the scene image in combination with the depth information in the depth image, or create a three-dimensional image of the person region according to the depth information in the depth image, and then combine the scene The color information in the image is color-filled to the three-dimensional character area to obtain a three-dimensional colored person area image.
  • the fusion processing of each of the character area image and the image of the corresponding frame in the target dynamic background may be to merge the two-dimensional person area image with the two-dimensional background image in the target dynamic background to obtain a merged image, or may be three-dimensional
  • the colored character area image is fused with the two-dimensional background image in the target dynamic background to obtain a merged image.
  • the image processing apparatus 100 further includes an image memory 30.
  • the image memory 30 may be embedded in the electronic device 1000, or may be a memory independent of the electronic device 1000, and may include a direct memory access (DMA) feature.
  • the raw image data acquired by the visible light camera 11 or the structured light image related data collected by the depth image acquisition component 12 may be transferred to the image memory 30 for storage or buffering.
  • the processor 20 can read the raw image data from the image memory 30 for processing to obtain a scene image, and can also read the structured light image related data from the image memory 30 for processing to obtain a depth image.
  • the scene image and the depth image may also be stored in the image memory 30 for the processor 20 to call the processing at any time.
  • the processor 20 calls the scene image and the depth image to perform character region extraction, and the extracted character region is extracted.
  • the image is blended with the target dynamic background image to obtain a merged image.
  • the target dynamic background image and the merged image may also be stored in the image memory 30.
  • the image processing apparatus 100 may also include a display 50.
  • the display 50 can acquire the merged image directly from the processor 20, and can also acquire the merged image from the image memory 30.
  • Display 50 displays the merged image for viewing by the user or for further processing by a graphics engine or graphics processing unit (GPU).
  • the image processing apparatus 100 further includes an encoder/decoder 60 that can encode image data of a scene image, a depth image, a merged image, and the like, and the encoded image data can be saved in the image memory 30 and can be The image is decompressed by the decoder for display before the image is displayed on display 50.
  • Encoder/decoder 60 may be implemented by a Central Processing Unit (CPU), GPU, or coprocessor. In other words, the encoder/decoder 60 may be any one or more of a central processing unit (CPU), a GPU, and a coprocessor.
  • the image processing apparatus 100 also includes a control logic 40.
  • the processor 20 analyzes the data acquired by the imaging device to determine image statistical information of one or more control parameters (eg, exposure time, etc.) of the imaging device 10.
  • Processor 20 sends image statistics to control logic 40, which controls imaging device 10 to determine good control parameters for imaging.
  • Control logic 40 may include a processor and/or a microcontroller that executes one or more routines, such as firmware.
  • One or more routines may determine control parameters of imaging device 10 based on the received image statistics.
  • FIG. 14 is a block diagram of another electronic device according to an embodiment of the present invention.
  • the electronic device 1000 of the embodiment of the present invention includes one or more processors 200 and a memory 300. And one or more programs 310.
  • One or more of the programs 310 are stored in the memory 300 and are configured to be executed by one or more processors 200.
  • the program 310 includes instructions for executing the image processing method of any of the above embodiments.
  • program 310 includes instructions for performing the image processing methods described in the following steps:
  • Step S301 collecting a first scene video of the current user.
  • Step S302 Acquire a plurality of depth images of the current user.
  • Step S303 Acquire a second scene video with a frame rate matching and a dynamic background to be merged according to the first scene video.
  • Step S304 processing each frame scene image of the second scene video according to the plurality of depth images, so as to obtain a current person region in each frame scene image to obtain a corresponding person region image.
  • Step S305 merging each of the character area images with the background image of the corresponding frame in the target dynamic background to obtain a merged image.
  • program 310 further includes instructions for performing the image processing method described in the following steps:
  • Step S3023 demodulating phase information corresponding to each pixel in the structured light image
  • Step S3024 the phase information is converted into depth information, and one of the plurality of depth images is generated according to the depth information.
  • a computer readable storage medium in accordance with an embodiment of the present invention includes a computer program for use in conjunction with an electronic device 1000 capable of imaging.
  • the computer program can be executed by the processor 200 to perform the image processing method of any of the above embodiments.
  • a computer program can be executed by processor 200 to perform the image processing methods described in the following steps:
  • Step S301 collecting a first scene video of the current user.
  • Step S302 Acquire a plurality of depth images of the current user.
  • Step S303 Acquire a second scene video with a frame rate matching and a dynamic background to be merged according to the first scene video.
  • Step S304 processing each frame scene image of the second scene video according to the plurality of depth images to obtain a current person region in the frame image of each frame to obtain a corresponding person region image.
  • Step S305 merging each of the character area images with the background image of the corresponding frame in the target dynamic background to obtain a merged image.
  • a computer program can also be executed by processor 200 to perform the image processing methods described in the following steps:
  • Step S3023 demodulating phase information corresponding to each pixel in the structured light image
  • Step S3024 the phase information is converted into depth information, and one of the plurality of depth images is generated according to the depth information.
  • first and second are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated.
  • features defining “first” and “second” may include at least one of the features, either explicitly or implicitly.
  • the meaning of "a plurality” is at least two, such as two, three, etc., unless specifically defined otherwise.
  • a "computer-readable medium” can be any apparatus that can contain, store, communicate, propagate, or transport a program for use in an instruction execution system, apparatus, or device, or in conjunction with the instruction execution system, apparatus, or device.
  • computer readable media include the following: electrical connections (electronic devices) having one or more wires, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable editable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM).
  • the computer readable medium may even be a paper or other suitable medium on which the program can be printed, as it may be optically scanned, for example by paper or other medium, followed by editing, interpretation or, if appropriate, other suitable The method is processed to obtain the program electronically and then stored in computer memory.
  • portions of the invention may be implemented in hardware, software, firmware or a combination thereof.
  • multiple steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques well known in the art: having logic gates for implementing logic functions on data signals. Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.
  • each functional unit in each embodiment of the present invention may be integrated into one processing module, or each unit may exist physically separately, or two or more units may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of software functional modules.
  • the integrated modules, if implemented in the form of software functional modules and sold or used as stand-alone products, may also be stored in a computer readable storage medium.
  • the above mentioned storage medium may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé de traitement d'images, un dispositif de traitement d'images (100), un dispositif électronique (1000) et un support de stockage lisible par ordinateur. Le procédé de traitement d'images consiste à : capturer une première vidéo de situation d'un utilisateur actuel (S301) ; acquérir de multiples images de profondeur de l'utilisateur (S302) ; acquérir, sur la base de la première vidéo de situation, une seconde vidéo de situation d'une fréquence image correspondante et d'un arrière-plan dynamique à fusionner (S303) ; traiter les images de situation de la seconde vidéo de situation sur la base des multiples images de profondeur pour acquérir des zones de portrait de l'utilisateur actuel dans les images de situation de façon à produire des images de zone de portrait correspondantes (S304) ; et fusionner les images de zone de portrait avec des images d'arrière-plan des trames correspondantes dans l'arrière-plan dynamique à fusionner pour produire des images combinées (S305).
PCT/CN2018/105120 2017-09-11 2018-09-11 Procédé et dispositif de traitement d'images, dispositif électronique et support de stockage lisible par ordinateur WO2019047984A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CN201710811502.6 2017-09-11
CN201710811811.3A CN107509045A (zh) 2017-09-11 2017-09-11 图像处理方法和装置、电子装置和计算机可读存储介质
CN201710811811.3 2017-09-11
CN201710812756.XA CN107644440A (zh) 2017-09-11 2017-09-11 图像处理方法和装置、电子装置和计算机可读存储介质
CN201710811502.6A CN107610076A (zh) 2017-09-11 2017-09-11 图像处理方法和装置、电子装置和计算机可读存储介质
CN201710812756.X 2017-09-11

Publications (1)

Publication Number Publication Date
WO2019047984A1 true WO2019047984A1 (fr) 2019-03-14

Family

ID=65633557

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/105120 WO2019047984A1 (fr) 2017-09-11 2018-09-11 Procédé et dispositif de traitement d'images, dispositif électronique et support de stockage lisible par ordinateur

Country Status (1)

Country Link
WO (1) WO2019047984A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022130379A1 (fr) * 2020-12-15 2022-06-23 Tetavi Ltd. Procédé de fusion de séquences de maillage à l'intérieur d'une vidéo volumétrique

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011068672A1 (fr) * 2009-11-18 2011-06-09 Zoran Corporation Méthode et appareil de traitement de données vidéo et graphiques pour créer une image composite de sortie comportant des couches indépendantes et séparées de plans d'affichage vidéo et graphique
CN102724492A (zh) * 2012-06-28 2012-10-10 广东威创视讯科技股份有限公司 视频图像传输、播放方法和系统
CN104660918A (zh) * 2015-03-11 2015-05-27 南京航空航天大学 一种机载视频实时混合显示系统
CN106909911A (zh) * 2017-03-09 2017-06-30 广东欧珀移动通信有限公司 图像处理方法、图像处理装置和电子装置
CN106954058A (zh) * 2017-03-09 2017-07-14 深圳奥比中光科技有限公司 深度图像获取系统和方法
CN107509045A (zh) * 2017-09-11 2017-12-22 广东欧珀移动通信有限公司 图像处理方法和装置、电子装置和计算机可读存储介质
CN107610076A (zh) * 2017-09-11 2018-01-19 广东欧珀移动通信有限公司 图像处理方法和装置、电子装置和计算机可读存储介质
CN107644440A (zh) * 2017-09-11 2018-01-30 广东欧珀移动通信有限公司 图像处理方法和装置、电子装置和计算机可读存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011068672A1 (fr) * 2009-11-18 2011-06-09 Zoran Corporation Méthode et appareil de traitement de données vidéo et graphiques pour créer une image composite de sortie comportant des couches indépendantes et séparées de plans d'affichage vidéo et graphique
CN102724492A (zh) * 2012-06-28 2012-10-10 广东威创视讯科技股份有限公司 视频图像传输、播放方法和系统
CN104660918A (zh) * 2015-03-11 2015-05-27 南京航空航天大学 一种机载视频实时混合显示系统
CN106909911A (zh) * 2017-03-09 2017-06-30 广东欧珀移动通信有限公司 图像处理方法、图像处理装置和电子装置
CN106954058A (zh) * 2017-03-09 2017-07-14 深圳奥比中光科技有限公司 深度图像获取系统和方法
CN107509045A (zh) * 2017-09-11 2017-12-22 广东欧珀移动通信有限公司 图像处理方法和装置、电子装置和计算机可读存储介质
CN107610076A (zh) * 2017-09-11 2018-01-19 广东欧珀移动通信有限公司 图像处理方法和装置、电子装置和计算机可读存储介质
CN107644440A (zh) * 2017-09-11 2018-01-30 广东欧珀移动通信有限公司 图像处理方法和装置、电子装置和计算机可读存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022130379A1 (fr) * 2020-12-15 2022-06-23 Tetavi Ltd. Procédé de fusion de séquences de maillage à l'intérieur d'une vidéo volumétrique
US11893688B2 (en) 2020-12-15 2024-02-06 Tetavi Ltd. Method of fusing mesh sequences within volumetric video

Similar Documents

Publication Publication Date Title
US11503228B2 (en) Image processing method, image processing apparatus and computer readable storage medium
CN107734267B (zh) 图像处理方法和装置
CN107480613B (zh) 人脸识别方法、装置、移动终端和计算机可读存储介质
CN107610080B (zh) 图像处理方法和装置、电子装置和计算机可读存储介质
CN107734264B (zh) 图像处理方法和装置
CN107623817B (zh) 视频背景处理方法、装置和移动终端
CN107705278B (zh) 动态效果的添加方法和终端设备
CN107610077A (zh) 图像处理方法和装置、电子装置和计算机可读存储介质
CN107509045A (zh) 图像处理方法和装置、电子装置和计算机可读存储介质
CN107452034B (zh) 图像处理方法及其装置
CN107509043B (zh) 图像处理方法、装置、电子装置及计算机可读存储介质
CN107707831A (zh) 图像处理方法和装置、电子装置和计算机可读存储介质
WO2019047982A1 (fr) Procédé et appareil de traitement d'images, dispositif électronique et support de stockage lisible par ordinateur
CN107707835A (zh) 图像处理方法及装置、电子装置和计算机可读存储介质
CN107590828B (zh) 拍摄图像的虚化处理方法和装置
CN107623814A (zh) 拍摄图像的敏感信息屏蔽方法和装置
CN107613239B (zh) 视频通信背景显示方法和装置
CN107592491B (zh) 视频通信背景显示方法和装置
CN107707838A (zh) 图像处理方法和装置
CN107644440A (zh) 图像处理方法和装置、电子装置和计算机可读存储介质
CN107610078A (zh) 图像处理方法和装置
CN107454336B (zh) 图像处理方法及装置、电子装置和计算机可读存储介质
CN107705276B (zh) 图像处理方法和装置、电子装置和计算机可读存储介质
CN107734266B (zh) 图像处理方法和装置、电子装置和计算机可读存储介质
CN107610076A (zh) 图像处理方法和装置、电子装置和计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18853985

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18853985

Country of ref document: EP

Kind code of ref document: A1