WO2019084712A1 - Image processing method and apparatus, and terminal - Google Patents

Image processing method and apparatus, and terminal Download PDF

Info

Publication number
WO2019084712A1
WO2019084712A1 PCT/CN2017/108314 CN2017108314W WO2019084712A1 WO 2019084712 A1 WO2019084712 A1 WO 2019084712A1 CN 2017108314 W CN2017108314 W CN 2017108314W WO 2019084712 A1 WO2019084712 A1 WO 2019084712A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
video data
target video
foreground
frame
Prior art date
Application number
PCT/CN2017/108314
Other languages
French (fr)
Chinese (zh)
Inventor
何展鹏
张立天
吴博
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2017/108314 priority Critical patent/WO2019084712A1/en
Priority to CN201780009967.8A priority patent/CN108702463B/en
Priority to CN202011396682.4A priority patent/CN112541414A/en
Publication of WO2019084712A1 publication Critical patent/WO2019084712A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G06V10/507—Summing image-intensity values; Histogram projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/76—Circuitry for compensating brightness variation in the scene by influencing the image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/57—Control of the dynamic range
    • H04N25/58—Control of the dynamic range involving two or more exposures

Definitions

  • the present application relates to the field of image processing technologies, and in particular, to an image processing method, apparatus, and terminal.
  • Multiple exposure is a shooting technique.
  • the principle of multiple exposure technology is to record the image of a moving object in a time segment on a picture by two or more exposures, which can show magical effects. .
  • the above multiple exposure technique requires a multi-step fine operation, and the shooting is difficult.
  • the embodiment of the invention provides an image processing method, device and terminal, which are convenient to operate, can effectively realize multiple exposures, and reduce shooting difficulty.
  • the first aspect of the embodiment of the present invention discloses an image processing method, including:
  • the foreground sub-image of each frame and the background image of the target video data are image-fused to obtain an exposure image.
  • the second aspect of the embodiment of the present invention discloses an image processing apparatus, including:
  • An image obtaining module configured to select at least two frame key images in the target video data according to an image selection algorithm corresponding to the target video data, where the key images in each frame include a foreground object;
  • a sub-image obtaining module configured to acquire a foreground sub-image including the foreground object in the key image of each frame
  • an image fusion module configured to image fuse the foreground sub-image of each frame and the background image of the target video data to obtain an exposure image.
  • a third aspect of the embodiments of the present invention discloses a terminal, including: a memory and a processor,
  • the memory is configured to store program instructions
  • the processor is configured to invoke the program instruction, and when the program instruction is executed, perform the following operations:
  • the foreground sub-image of each frame and the background image of the target video data are image-fused to obtain an exposure image.
  • the embodiment selects at least two frame key images in the target video data, acquires a foreground sub-image including the foreground object in each frame key image, and displays each frame foreground sub-image and target video.
  • the background image of the data is image-fused to obtain an exposed image.
  • the multi-step fine operation is required, and the shooting is difficult.
  • the embodiment of the present invention is convenient to operate, and can effectively realize multiple exposures and reduce the shooting difficulty.
  • FIG. 1 is a schematic flow chart of an image processing method according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of an interface image of a background image according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of an interface of a key image disclosed in an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of an interface of an exposure image disclosed in an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a terminal according to an embodiment of the present invention.
  • the terminal in the embodiment of the present invention may include a personal computer, a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, a palmtop computer, a mobile Internet device (MID, Mobile Internet Devices), a wearable smart device, an aircraft, or an unmanned person. Ground control station, etc.
  • the target video data may be acquired by the camera device, or may be obtained in the memory of the terminal or in the Internet, and is not limited by the embodiment of the present invention.
  • the camera device can be integrated in the terminal or can be connected to the terminal.
  • the target video data may include at least two frames of images.
  • the content included in the background image may be a background in at least two frames of images included in the target video data.
  • the foreground object may be an object of a relative background in at least two frames of images included in the target video data, such as a pedestrian, an animal, or an item (eg, a skateboard, a ball, etc.).
  • the terminal may select an image including the foreground object in the target video data, and select at least two frame designated images selected from the image including the foreground object as the key image.
  • the foreground sub-image may be an area of the key image containing the foreground object, for example, the edge of the area coincides with the edge of the foreground object, and the distance between the edge of the area and the edge of the foreground object is less than a preset distance threshold.
  • the exposure image may be an image obtained by processing the foreground sub-image and the background image by image fusion technology.
  • FIG. 1 is a schematic flowchart diagram of an image processing method according to an embodiment of the present invention. Specifically, as shown in FIG. 1, the image processing method of the embodiment of the present invention may include the following steps:
  • the terminal may pre-establish an image selection algorithm corresponding to different video data.
  • the terminal may acquire an image selection algorithm corresponding to the target video data, and select at least two frame key images in the target video data.
  • the image selection algorithm is used to select a key image, and the key image may include a foreground object.
  • the terminal may acquire the foreground sub-image included in each image in at least two frames of images included in the target video data, obtain the foreground sub-image located at the center point through the statistical information of the foreground sub-image, and according to the foreground located at the center point
  • the spatial information and the time information of the sub-image determine the image to which the selected foreground sub-image belongs as a key image, and the key image may be as shown in FIG. 3 .
  • the terminal may obtain an application scenario of the target video data, according to a preset application field.
  • the corresponding relationship between the scene and the image selection algorithm is obtained, and an image selection algorithm corresponding to the application scene is obtained, and the target video data is used as an input of the image selection algorithm to obtain at least two key images.
  • the terminal may pre-establish an image selection algorithm corresponding to different application scenarios.
  • the terminal may acquire an application scenario of the target video data, and obtain an image selection algorithm corresponding to the application scenario, and the terminal may The image selection algorithm corresponding to the application scene is used as an image selection algorithm corresponding to the target video data, and the target video data is used as an input of the image selection algorithm, and the image output by the image selection algorithm is used as at least two frame key images.
  • the application scenario may include a motion gesture of the foreground object, such as a jumping gesture, a thousand-hand Guanyin gesture, or a martial arts action gesture.
  • the image selection algorithm may be: acquiring a frame image in the target video data every predetermined number of frames, and using the acquired image as a key image.
  • the preset number of frames may be preset, for example, three frames per interval or five frames per interval.
  • the target video data includes 10 frames of images
  • the terminal may acquire one frame of image in the target video data every two frames, that is, the terminal may use the first frame image, the fourth frame image, the seventh frame image, and the tenth frame image as Key image.
  • the terminal may determine that the application scene of the target video data is the first application scenario, and then acquire an image selection algorithm corresponding to the first application scenario, and the target is obtained.
  • the video data is used as an input of the image selection algorithm, and the terminal may acquire one frame image in the target video data every predetermined number of frames, and use the acquired image as a key image.
  • the terminal may acquire the foreground sub-image in each frame image included in the target video data according to the background image, select the target foreground sub-image according to the spatial information and time information of each foreground sub-image, and select the target foreground sub-image.
  • the image to which it belongs is determined as a key image.
  • the terminal may obtain the foreground sub-image in each frame image included in the target video data according to the background image, and may The spatial information and time information of the image are selected as the foreground sub-image of the foreground object, the jump to the highest point and the foreground sub-image, and the selected foreground sub-image is used as the target foreground sub-image, and then the image of the foreground sub-image belongs.
  • the terminal may obtain the foreground sub-image in each frame image included in the target video data according to the background image, and may The spatial information and time information of the image are selected as the foreground sub-image of the foreground object, the jump to the highest point and the foreground sub-image, and the selected foreground sub-image is used as the target foreground sub-image, and then the image of the foreground sub-image belongs.
  • the key image may be a key image.
  • the terminal can determine the target video.
  • the application scenario of the data is a second application scenario, and then an image selection algorithm corresponding to the second application scenario is obtained, and the target video data is used as an input of the image selection algorithm, and the terminal may use each frame included in the target video data according to the background image.
  • the foreground sub-image is acquired in the image, and the target foreground sub-image is selected according to the spatial information and the time information of each foreground sub-image, and the image to which the target foreground sub-image belongs is determined as the key image.
  • the terminal selects at least two frame key images in the target video data according to the image selection algorithm corresponding to the target video data, and then processes each frame key image to obtain a foreground sub-image including the foreground object in the key image.
  • the foreground sub-image may include a pedestrian with a moving posture as a take-off.
  • the terminal may acquire a foreground sub-image including the foreground object in each frame key image according to the background image.
  • the terminal may compare the background image with at least two frame key images, and acquire a foreground sub-image including the foreground object in each frame key image.
  • the terminal can compare the background image to the key image based on global variation factors.
  • the strategy of switching the branch algorithm is also selected, that is, the practical sub-algorithm is selected according to the application scene of the target video data, and the background image and the key image are processed according to the sub-algorithm to obtain the foreground object.
  • Prospect sub-image wherein, at least one foreground sub-image can be acquired in one key image.
  • the terminal may perform pedestrian recognition on each frame key image according to a pedestrian recognition algorithm to obtain a foreground sub-image including a pedestrian.
  • the terminal may perform face recognition on the key image.
  • the terminal may determine that the foreground object is a pedestrian, and the terminal may perform pedestrian recognition on the key image according to the pedestrian recognition algorithm to obtain a pedestrian-containing Prospect sub-image.
  • the feature extraction may be performed in the key image to obtain a Histogram of Oriented Gradient (HOG) feature, and then the HOG feature is used as an input of the SVM classifier to obtain a pedestrian.
  • HOG Histogram of Oriented Gradient
  • the terminal can use the pedestrian recognition network (R-CNN), the fast convolutional neural network feature Fast-RCNN or the faster convolutional neural network feature Faster-RCNN to identify the key image. Conduct pedestrian identification and get pedestrians The foreground sub-image.
  • R-CNN pedestrian recognition network
  • Fast-RCNN fast convolutional neural network feature
  • Faster-RCNN faster convolutional neural network feature
  • the embodiment of the present invention performs pedestrian recognition on a key image according to a pedestrian recognition algorithm to obtain a foreground sub-image including a pedestrian. Even if the foreground object in the target video data is in a non-moving state (ie, a stationary state), the terminal can obtain the foreground sub-subject by the above method.
  • the image can improve the recognition efficiency of foreground objects and effectively achieve multiple exposures.
  • the terminal may acquire, in the target video data, an image in which the time information is greater than the time information of the foreground sub-image according to the time information of the foreground sub-image, and the foreground sub-image Image fusion is performed with each acquired image to update the acquired image, and the target video data is updated according to the updated image, and the updated target video data includes the updated image.
  • the terminal selects four key images in the target video data, which are the first frame image, the fourth frame image, the seventh frame image, and the tenth frame image in the target video data, and are acquired in the first frame image.
  • the motion image of the foreground object contained in the first foreground sub-image is the run-up
  • the motion pose of the foreground object included in the second foreground sub-image acquired in the fourth frame image is the take-off, in the seventh frame image.
  • the motion posture of the foreground object included in the acquired third foreground sub-image is jumped to the highest point, and the motion posture of the foreground object included in the fourth foreground sub-image acquired in the tenth frame image is landing.
  • the terminal may determine that the time information of the first foreground sub-image is the first frame, and the time information of the target video data is larger than the image of the first frame is the image of the second to the tenth frame, and then the first foreground sub-image and the second - 10 frames of images are image-fused to obtain an updated image of the 2nd-10th frame.
  • the terminal may determine that the time information of the second foreground sub-image is the fourth frame, and the time information of the target video data is greater than the image of the fourth frame is the 5-10th image, and then the second foreground sub-image is respectively The image is blended with the updated 5-10th image to obtain the updated 5-10th image.
  • the terminal may determine that the time information of the third foreground sub-image is the seventh frame, and the image in which the time information in the target video data is larger than the seventh frame is the 8-10th image, and then the third foreground sub-image is respectively Image fusion is performed with the updated image of the 8th-10th frame to obtain an updated image of the 8th-10th frame.
  • the terminal may determine that the time information of the fourth foreground sub-image is the tenth frame, and the target video data does not have the image whose time information is greater than the tenth frame, the terminal may update the target video data, and the updated target video data includes an update.
  • the subsequent image for example, the target video data includes the first frame image, and the updated 2-10th frame image, wherein the updated second frame image is the first foreground sub image and the second frame image
  • the updated fifth frame image is obtained by image fusion of the first foreground sub-image, the second foreground sub-image, and the fifth frame image
  • the updated eighth frame image is A foreground sub-image, a second foreground sub-image, a third foreground sub-image and a fifth frame image are obtained by image fusion
  • the updated tenth frame image is the first foreground sub-image and the second foreground
  • the sub-image, the third foreground sub-image, the fourth foreground sub-image, and the fifth frame image are obtained by image fusion.
  • the terminal may perform image fusion on all foreground sub-images and background images to obtain an exposure image, and the exposure image may be as shown in FIG. 4 .
  • the background image may be obtained by processing the target video data by the terminal, or may be obtained by the terminal by the camera, acquired in a local memory, or obtained through the Internet.
  • the terminal performs image fusion on the foreground image of each frame and the background image of the target video data, and before the obtained image is obtained, the target video data may be processed to obtain a background image, and the background image may be as shown in FIG. 2 .
  • the terminal may obtain a position of each foreground sub-image in the key image to which the foreground sub-image belongs, and combine the foreground sub-image and the background image according to the position to obtain an exposure image.
  • the terminal fuses the first foreground sub-image and the background image according to the position to obtain an exposure image, and the first image in the exposure image
  • the foreground object contained in the foreground sub-image is located on the right side of the exposure image, and the distance between the foreground object and each edge of the exposure image is the same as the distance between the foreground object and the corresponding edge of the first frame image.
  • the embodiment selects a key image in the target video data, acquires a foreground sub-image including the foreground object in the key image, and performs image fusion on the foreground sub-image and the background image of the target video data. , the exposure image is obtained, the operation is convenient, the multiple exposure can be effectively realized, and the shooting difficulty is reduced.
  • the embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores program instructions, and the program may include some or all of the steps of the image processing method in the corresponding embodiment of FIG. 1 .
  • FIG. 5 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.
  • the image processing apparatus described in this embodiment includes:
  • the image obtaining module 501 is configured to select at least two frame key images in the target video data according to an image selection algorithm corresponding to the target video data, where the key images of each frame include a foreground object;
  • a sub-image obtaining module 502 configured to acquire, in the key image of each frame, a foreground sub-image including the foreground object;
  • the image fusion module 503 is configured to image fuse the foreground sub-image of each frame and the background image of the target video data to obtain an exposure image.
  • the sub-image obtaining module 502 is configured to acquire a foreground sub-image including the foreground object in the key image of each frame according to the background image.
  • the foreground object is a pedestrian
  • the sub-image obtaining module 502 is configured to perform pedestrian recognition on the key image of each frame according to a pedestrian recognition algorithm to obtain a foreground sub-image including the pedestrian.
  • the image obtaining module 501 is further configured to: the image fusion module 503 performs image fusion on the foreground sub-image of each frame and the background image of the target video data, and obtains the target image before the exposure image is obtained.
  • the video data is processed to obtain the background image.
  • the image obtaining module 501 is specifically configured to:
  • the at least two frames of key images are obtained.
  • the image obtaining module 501 uses the target video data as an input of the image selection algorithm to obtain the key image, specifically for:
  • the acquired image is taken as the key image.
  • the image obtaining module 501 uses the target video data as an input of the image selection algorithm to obtain the key image, specifically for:
  • the image to which the target foreground sub-image belongs is determined as a key image.
  • the image fusion module 503 is specifically configured to:
  • the foreground sub-image and the background image are image-fused to obtain the exposed image.
  • the image obtaining module 501 is further configured to: after the sub-image obtaining module 502 acquires a foreground sub-image including the foreground object in each key image of each frame, according to time information of the foreground sub-image Obtaining, in the target video data, an image in which time information is greater than time information of the foreground sub-image;
  • the image fusion module 503 is further configured to image fuse the foreground sub-image and each acquired image to update the acquired image;
  • the image processing apparatus further includes:
  • the update module 504 is configured to update the target video data according to the updated image, and the updated target video data includes the updated image.
  • the image obtaining module 501 selects a key image in the target video data according to the image selection algorithm corresponding to the target video data, and the sub-image acquiring module 502 acquires a foreground sub-image including the foreground object in the key image, and the image fusion module 503
  • the foreground image and the background image of the target video data are image-fused to obtain an exposed image, which is convenient to operate, and can effectively achieve multiple exposures and reduce shooting difficulty.
  • FIG. 6 is a schematic structural diagram of a terminal according to an embodiment of the present invention.
  • the terminal described in this embodiment includes: a memory 601 and a processor 602.
  • the above processor 602 and memory 601 are connected by a bus.
  • the processor 602 may be a central processing unit (CPU), and the processor may be another general-purpose processor, a digital signal processor (DSP), or an application specific integrated circuit (ASIC). ), a Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, and the like.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the above memory 601 can include read only memory and random access memory and provides instructions and data to the processor 602.
  • a portion of the memory 601 may also include a non-volatile random access memory. among them:
  • the memory 601 is configured to store program instructions
  • the processor 602 is configured to invoke the program instruction, and when the program instruction is executed, perform the following operations:
  • the foreground sub-image of each frame and the background image of the target video data are image-fused to obtain an exposure image.
  • the processor 602 obtains, in the key image of each frame, a foreground sub-image that includes the foreground object, specifically for:
  • a foreground sub-image including the foreground object is acquired in the key image of each frame.
  • the foreground object is a pedestrian
  • the processor 602 obtains a foreground sub-image including the foreground object in the key image of each frame, specifically:
  • Pedestrian recognition is performed on the key image of each frame according to a pedestrian recognition algorithm to obtain a foreground sub-image including the pedestrian.
  • the processor 602 is further configured to perform image fusion on the foreground sub-image of each frame and the background image of the target video data, and process the target video data before obtaining the exposed image to obtain the The background image.
  • the processor 602 is configured according to an image selection algorithm corresponding to the target video data. Select at least two frames of key images from the target video data, specifically for:
  • the at least two frames of key images are obtained.
  • the processor 602 uses the target video data as an input of the image selection algorithm to obtain the at least two frames of key images, specifically for:
  • the acquired image is taken as the key image.
  • the processor 602 uses the target video data as an input of the image selection algorithm to obtain the at least two frames of key images, specifically for:
  • the image to which the target foreground sub-image belongs is determined as a key image.
  • the processor 602 performs image fusion on the foreground sub-image and the background image of each frame to obtain an exposure image, specifically for:
  • the foreground sub-image and the background image are image-fused to obtain the exposed image.
  • the processor 602 is further configured to: after acquiring a foreground sub-image including the foreground object in each key image of each frame, according to time information of the foreground sub-image, in the target video data Obtaining an image in which time information is greater than time information of the foreground sub-image;
  • the processor 602 is further configured to perform image fusion on the foreground sub-image and each acquired image to update the acquired image;
  • the processor 602 is further configured to update the target video data according to the updated image, where the updated target video data includes the updated image.
  • the processor 602 described in the embodiment of the present invention may be implemented in the embodiment of the present invention.
  • the implementation of the image processing method described in the embodiment of the present invention may also be implemented, and details are not described herein again.
  • the program may be stored in a computer readable storage medium, and the storage medium may include: Flash disk, Read-Only Memory (ROM), Random Access Memory (RAM), disk or optical disk.

Abstract

An image processing method and apparatus, and a terminal. The method comprises: according to a selection algorithm for an image corresponding to target video data, selecting at least two frames of a key image from the target video data, wherein each of the frames of the key image comprises a foreground object; acquiring, from each of the frames of the key image, a foreground sub-image containing the foreground object; and performing image fusion on each frame of the foreground sub-image and a background image of the target video data, so as to obtain an exposure image. The present present is convenient and quick to operate, and can effectively realize multi-exposure and reduce the photography difficulty.

Description

一种图像处理方法、装置以及终端Image processing method, device and terminal 技术领域Technical field
本申请涉及图像处理技术领域，尤其涉及一种图像处理方法、装置以及终端。The present application relates to the field of image processing technologies, and in particular, to an image processing method, apparatus, and terminal.
背景技术Background technique
多重曝光是一种拍摄技法，多重曝光技术的原理是：通过两次或多次曝光，将一个运动物体在一个时间片段内的影像都记录在一张图片上，可以呈现出魔术般无中生有的效果。但是上述多重曝光技术需要多步精细的操作，拍摄难度较大。Multiple exposure is a shooting technique. The principle of multiple exposure technology is to record the image of a moving object in a time segment on a picture by two or more exposures, which can show magical effects. . However, the above multiple exposure technique requires a multi-step fine operation, and the shooting is difficult.
发明内容Summary of the invention
本发明实施例提供一种图像处理方法、装置以及终端，操作便捷，可有效实现多重曝光，降低拍摄难度。The embodiment of the invention provides an image processing method, device and terminal, which are convenient to operate, can effectively realize multiple exposures, and reduce shooting difficulty.
本发明实施例第一方面公开了一种图像处理方法，包括：The first aspect of the embodiment of the present invention discloses an image processing method, including:
根据目标视频数据对应的图像选取算法，在所述目标视频数据中选取至少两帧关键图像，各帧所述关键图像均包括前景物体；Selecting at least two frames of key images in the target video data according to an image selection algorithm corresponding to the target video data, wherein the key images of each frame include a foreground object;
在各帧所述关键图像中获取包含所述前景物体的前景子图像；Obtaining a foreground sub-image including the foreground object in the key image of each frame;
将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像。The foreground sub-image of each frame and the background image of the target video data are image-fused to obtain an exposure image.
本发明实施例第二方面公开了一种图像处理装置，包括：The second aspect of the embodiment of the present invention discloses an image processing apparatus, including:
图像获取模块，用于根据目标视频数据对应的图像选取算法，在所述目标视频数据中选取至少两帧关键图像，各帧所述关键图像均包括前景物体；An image obtaining module, configured to select at least two frame key images in the target video data according to an image selection algorithm corresponding to the target video data, where the key images in each frame include a foreground object;
子图像获取模块，用于在各帧所述关键图像中获取包含所述前景物体的前景子图像；a sub-image obtaining module, configured to acquire a foreground sub-image including the foreground object in the key image of each frame;
图像融合模块，用于将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像。And an image fusion module, configured to image fuse the foreground sub-image of each frame and the background image of the target video data to obtain an exposure image.
本发明实施例第三方面公开了一种终端，包括：存储器和处理器， A third aspect of the embodiments of the present invention discloses a terminal, including: a memory and a processor,
所述存储器，用于存储程序指令；The memory is configured to store program instructions;
所述处理器，用于调用所述程序指令，当所述程序指令被执行时，执行以下操作：The processor is configured to invoke the program instruction, and when the program instruction is executed, perform the following operations:
根据目标视频数据对应的图像选取算法，在所述目标视频数据中选取至少两帧关键图像，各帧所述关键图像均包括前景物体；Selecting at least two frames of key images in the target video data according to an image selection algorithm corresponding to the target video data, wherein the key images of each frame include a foreground object;
在各帧所述关键图像中获取包含所述前景物体的前景子图像；Obtaining a foreground sub-image including the foreground object in the key image of each frame;
将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像。The foreground sub-image of each frame and the background image of the target video data are image-fused to obtain an exposure image.
本发明实施例根据目标视频数据对应的图像选取算法，在目标视频数据中选取至少两帧关键图像，在各帧关键图像中获取包含前景物体的前景子图像，将各帧前景子图像和目标视频数据的背景图像进行图像融合，得到曝光图像，相对传统的多重曝光技术需要多步精细的操作，拍摄难度较大，本发明实施例操作便捷，可有效实现多重曝光，降低拍摄难度。According to the image selection algorithm corresponding to the target video data, the embodiment selects at least two frame key images in the target video data, acquires a foreground sub-image including the foreground object in each frame key image, and displays each frame foreground sub-image and target video. The background image of the data is image-fused to obtain an exposed image. Compared with the conventional multiple exposure technology, the multi-step fine operation is required, and the shooting is difficult. The embodiment of the present invention is convenient to operate, and can effectively realize multiple exposures and reduce the shooting difficulty.
附图说明DRAWINGS
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings to be used in the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without paying for creative labor.
图1是本发明实施例公开的一种图像处理方法的流程示意图；1 is a schematic flow chart of an image processing method according to an embodiment of the present invention;
图2是本发明实施例公开的一种背景图像的界面示意图；2 is a schematic diagram of an interface image of a background image according to an embodiment of the present invention;
图3是本发明实施例公开的一种关键图像的界面示意图；3 is a schematic diagram of an interface of a key image disclosed in an embodiment of the present invention;
图4是本发明实施例公开的一种曝光图像的界面示意图；4 is a schematic diagram of an interface of an exposure image disclosed in an embodiment of the present invention;
图5是本发明实施例公开的一种图像处理装置的结构示意图；FIG. 5 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention; FIG.
图6是本发明实施例公开的一种终端的结构示意图。FIG. 6 is a schematic structural diagram of a terminal according to an embodiment of the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本发明一部分实施例，而不是全 部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not An embodiment of the department. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
本发明实施例中的终端可以包括个人电脑、智能手机(如Android手机、iOS手机等)、平板电脑、掌上电脑、移动互联网设备(MID，Mobile Internet Devices)、穿戴式智能设备、飞行器或者无人机地面控制站等。The terminal in the embodiment of the present invention may include a personal computer, a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, a palmtop computer, a mobile Internet device (MID, Mobile Internet Devices), a wearable smart device, an aircraft, or an unmanned person. Ground control station, etc.
其中，目标视频数据可以是通过摄像装置采集到的，也可以是在终端的存储器中或者互联网中获取到的，具体不受本发明实施例的限制。其中摄像装置可以集成在终端中，也可以外接终端。目标视频数据可以包括至少两帧图像。The target video data may be acquired by the camera device, or may be obtained in the memory of the terminal or in the Internet, and is not limited by the embodiment of the present invention. The camera device can be integrated in the terminal or can be connected to the terminal. The target video data may include at least two frames of images.
其中，背景图像所包含的内容可以为目标视频数据所包含的至少两帧图像中的背景。前景物体可以为目标视频数据所包含的至少两帧图像中相对背景的物体，例如行人、动物或者道具(例如滑板、皮球等)等。终端可以在目标视频数据中选取包含前景物体的图像，将在包含前景物体的图像中选取的至少两帧指定图像作为关键图像。前景子图像可以为关键图像中包含前景物体的区域，例如该区域的边缘与前景物体的边缘相重合，又如该区域的边缘与前景物体的边缘之间的距离小于预设距离阈值。曝光图像可以为通过图像融合技术对前景子图像和背景图像进行处理后得到的图像。The content included in the background image may be a background in at least two frames of images included in the target video data. The foreground object may be an object of a relative background in at least two frames of images included in the target video data, such as a pedestrian, an animal, or an item (eg, a skateboard, a ball, etc.). The terminal may select an image including the foreground object in the target video data, and select at least two frame designated images selected from the image including the foreground object as the key image. The foreground sub-image may be an area of the key image containing the foreground object, for example, the edge of the area coincides with the edge of the foreground object, and the distance between the edge of the area and the edge of the foreground object is less than a preset distance threshold. The exposure image may be an image obtained by processing the foreground sub-image and the background image by image fusion technology.
请参阅图1，图1为本发明实施例提供的一种图像处理方法的流程示意图。具体的，如图1所示，本发明实施例的图像处理方法可以包括以下步骤：Please refer to FIG. 1. FIG. 1 is a schematic flowchart diagram of an image processing method according to an embodiment of the present invention. Specifically, as shown in FIG. 1, the image processing method of the embodiment of the present invention may include the following steps:
101、根据目标视频数据对应的图像选取算法，在目标视频数据中选取至少两帧关键图像。101. Select at least two frame key images in the target video data according to an image selection algorithm corresponding to the target video data.
具体地，终端可以预先建立不同视频数据对应的图像选取算法，在需要对目标视频数据进行处理时，终端可以获取目标视频数据对应的图像选取算法，在目标视频数据中选取至少两帧关键图像。其中，图像选取算法用于选取关键图像，关键图像可以包括前景物体。例如，终端可以在目标视频数据所包含的至少两帧图像中获取各个图像所包含的前景子图像，通过前景子图像的统计信息，获取位于中心点的前景子图像，并根据位于中心点的前景子图像的空间信息和时间信息，将选取得到的前景子图像所属的图像确定为关键图像，关键图像可以如图3所示。Specifically, the terminal may pre-establish an image selection algorithm corresponding to different video data. When the target video data needs to be processed, the terminal may acquire an image selection algorithm corresponding to the target video data, and select at least two frame key images in the target video data. The image selection algorithm is used to select a key image, and the key image may include a foreground object. For example, the terminal may acquire the foreground sub-image included in each image in at least two frames of images included in the target video data, obtain the foreground sub-image located at the center point through the statistical information of the foreground sub-image, and according to the foreground located at the center point The spatial information and the time information of the sub-image determine the image to which the selected foreground sub-image belongs as a key image, and the key image may be as shown in FIG. 3 .
可选的，终端可以获取目标视频数据的应用场景，根据预先设定的应用场 景和图像选取算法的对应关系，获取应用场景对应的图像选取算法，将目标视频数据作为图像选取算法的输入，得到至少两帧关键图像。Optionally, the terminal may obtain an application scenario of the target video data, according to a preset application field. The corresponding relationship between the scene and the image selection algorithm is obtained, and an image selection algorithm corresponding to the application scene is obtained, and the target video data is used as an input of the image selection algorithm to obtain at least two key images.
具体地，终端可以预先建立不同应用场景对应的图像选取算法，在需要对目标视频数据进行处理时，终端可以获取目标视频数据的应用场景，获取该应用场景对应的图像选取算法，终端可以将该应用场景对应的图像选取算法作为该目标视频数据对应的图像选取算法，进而将目标视频数据作为该图像选取算法的输入，将该图像选取算法输出的图像作为至少两帧关键图像。其中，应用场景可以包括前景物体的运动姿态，例如跳跃姿态、呈现千手观音姿态或者武术动作姿态等。Specifically, the terminal may pre-establish an image selection algorithm corresponding to different application scenarios. When the target video data needs to be processed, the terminal may acquire an application scenario of the target video data, and obtain an image selection algorithm corresponding to the application scenario, and the terminal may The image selection algorithm corresponding to the application scene is used as an image selection algorithm corresponding to the target video data, and the target video data is used as an input of the image selection algorithm, and the image output by the image selection algorithm is used as at least two frame key images. The application scenario may include a motion gesture of the foreground object, such as a jumping gesture, a thousand-hand Guanyin gesture, or a martial arts action gesture.
可选的，图像选取算法具体可以为：每间隔预设数量帧在目标视频数据中获取一帧图像，将获取到的图像作为关键图像。其中，预设数量帧可以为预先设定的，例如每间隔三帧或者每间隔五帧等。Optionally, the image selection algorithm may be: acquiring a frame image in the target video data every predetermined number of frames, and using the acquired image as a key image. The preset number of frames may be preset, for example, three frames per interval or five frames per interval.
例如，目标视频数据包括10帧图像，终端可以每间隔二帧在目标视频数据中获取一帧图像，即终端可以将第一帧图像、第四帧图像、第七帧图像以及第十帧图像作为关键图像。For example, the target video data includes 10 frames of images, and the terminal may acquire one frame of image in the target video data every two frames, that is, the terminal may use the first frame image, the fourth frame image, the seventh frame image, and the tenth frame image as Key image.
示例性的，当前景物体的运动姿态呈现千手观音姿态或者武术动作姿态，则终端可以确定目标视频数据的应用场景为第一应用场景，进而获取第一应用场景对应的图像选取算法，将目标视频数据作为该图像选取算法的输入，终端可每间隔预设数量帧在目标视频数据中获取一帧图像，将获取到的图像作为关键图像。For example, if the motion posture of the current scene object presents a thousand-hand Guanyin gesture or a martial arts action gesture, the terminal may determine that the application scene of the target video data is the first application scenario, and then acquire an image selection algorithm corresponding to the first application scenario, and the target is obtained. The video data is used as an input of the image selection algorithm, and the terminal may acquire one frame image in the target video data every predetermined number of frames, and use the acquired image as a key image.
可选的，终端可以根据背景图像，在目标视频数据所包含的每一帧图像中获取前景子图像，根据各个前景子图像的空间信息和时间信息，选取目标前景子图像，将目标前景子图像所属图像确定为关键图像。Optionally, the terminal may acquire the foreground sub-image in each frame image included in the target video data according to the background image, select the target foreground sub-image according to the spatial information and time information of each foreground sub-image, and select the target foreground sub-image. The image to which it belongs is determined as a key image.
例如，目标视频数据所包含至少两帧图像中前景物体的运动姿态为跳跃姿态，则终端根据背景图像，在目标视频数据所包含的每一帧图像中获取前景子图像之后，可以根据各个前景子图像的空间信息和时间信息，选取前景物体的运动姿态为起跳、跳跃至最高点以及落地的前景子图像，并将上述选取的前景子图像作为目标前景子图像，进而将前景子图像所属的图像作为关键图像。For example, if the motion image of the foreground object in the at least two frames of the target video data is a skip gesture, the terminal may obtain the foreground sub-image in each frame image included in the target video data according to the background image, and may The spatial information and time information of the image are selected as the foreground sub-image of the foreground object, the jump to the highest point and the foreground sub-image, and the selected foreground sub-image is used as the target foreground sub-image, and then the image of the foreground sub-image belongs. As a key image.
示例性的，当前景物体的运动姿态为跳跃姿态，则终端可以确定目标视频 数据的应用场景为第二应用场景，进而获取第二应用场景对应的图像选取算法，将目标视频数据作为该图像选取算法的输入，终端可根据背景图像，在目标视频数据所包含的每一帧图像中获取前景子图像，根据各个前景子图像的空间信息和时间信息，选取目标前景子图像，将目标前景子图像所属图像确定为关键图像。Exemplarily, if the motion posture of the current scene object is a jumping posture, the terminal can determine the target video. The application scenario of the data is a second application scenario, and then an image selection algorithm corresponding to the second application scenario is obtained, and the target video data is used as an input of the image selection algorithm, and the terminal may use each frame included in the target video data according to the background image. The foreground sub-image is acquired in the image, and the target foreground sub-image is selected according to the spatial information and the time information of each foreground sub-image, and the image to which the target foreground sub-image belongs is determined as the key image.
102、在各帧关键图像中获取包含前景物体的前景子图像。102. Obtain a foreground sub-image including a foreground object in each frame key image.
具体地，终端根据目标视频数据对应的图像选取算法，在目标视频数据中选取至少两帧关键图像之后，可以对各帧关键图像进行处理，在关键图像中获取包含前景物体的前景子图像。如图3所示，前景子图像可以包括一个运动姿态为起跳的行人。Specifically, the terminal selects at least two frame key images in the target video data according to the image selection algorithm corresponding to the target video data, and then processes each frame key image to obtain a foreground sub-image including the foreground object in the key image. As shown in FIG. 3, the foreground sub-image may include a pedestrian with a moving posture as a take-off.
可选的，终端可以根据背景图像，在各帧关键图像中获取包含前景物体的前景子图像。Optionally, the terminal may acquire a foreground sub-image including the foreground object in each frame key image according to the background image.
具体地，终端可以将背景图像与至少两帧关键图像进行比较，在各帧关键图像中获取包含前景物体的前景子图像。例如，终端可以基于全局变化因素，将背景图像与关键图像进行比较。而且由于场景的多变性，这里还选用了切换分支算法的策略，即根据目标视频数据的应用场景来选取实用的子算法，根据该子算法对背景图像和关键图像进行处理，获取包含前景物体的前景子图像。其中，在一个关键图像中可以获取至少一个前景子图像。Specifically, the terminal may compare the background image with at least two frame key images, and acquire a foreground sub-image including the foreground object in each frame key image. For example, the terminal can compare the background image to the key image based on global variation factors. Moreover, due to the variability of the scene, the strategy of switching the branch algorithm is also selected, that is, the practical sub-algorithm is selected according to the application scene of the target video data, and the background image and the key image are processed according to the sub-algorithm to obtain the foreground object. Prospect sub-image. Wherein, at least one foreground sub-image can be acquired in one key image.
可选的，当前景物体为行人时，终端可以根据行人识别算法对各帧关键图像进行行人识别，得到包含行人的前景子图像。Optionally, when the current scene object is a pedestrian, the terminal may perform pedestrian recognition on each frame key image according to a pedestrian recognition algorithm to obtain a foreground sub-image including a pedestrian.
具体地，终端可以对关键图像进行人脸识别，当在关键图像中识别到人脸时，终端可以确定前景物体为行人，进而终端可以根据行人识别算法对关键图像进行行人识别，得到包含行人的前景子图像。Specifically, the terminal may perform face recognition on the key image. When the face is recognized in the key image, the terminal may determine that the foreground object is a pedestrian, and the terminal may perform pedestrian recognition on the key image according to the pedestrian recognition algorithm to obtain a pedestrian-containing Prospect sub-image.
例如，终端确定前景物体为行人时，可以在关键图像中进行特征提取，以得到方向梯度直方图(Histogram of Oriented Gradient，HOG)特征，进而将HOG特征作为SVM分类器的输入，以得到包含行人的前景子图像。For example, when the terminal determines that the foreground object is a pedestrian, the feature extraction may be performed in the key image to obtain a Histogram of Oriented Gradient (HOG) feature, and then the HOG feature is used as an input of the SVM classifier to obtain a pedestrian. The foreground sub-image.
又如，终端可以通过卷积神经网络特征(Regions with Convolutional Neural Network，R-CNN)、快速卷积神经网络特征Fast-RCNN或者较快卷积神经网络特征Faster-RCNN等行人识别算法对关键图像进行行人识别，得到包含行人 的前景子图像。For example, the terminal can use the pedestrian recognition network (R-CNN), the fast convolutional neural network feature Fast-RCNN or the faster convolutional neural network feature Faster-RCNN to identify the key image. Conduct pedestrian identification and get pedestrians The foreground sub-image.
本发明实施例根据行人识别算法对关键图像进行行人识别，得到包含行人的前景子图像，即使目标视频数据中的前景物体处于非运动状态(即静止状态)，终端也可以通过上述方法得到前景子图像，可提高前景物体的识别效率，有效实现多重曝光。The embodiment of the present invention performs pedestrian recognition on a key image according to a pedestrian recognition algorithm to obtain a foreground sub-image including a pedestrian. Even if the foreground object in the target video data is in a non-moving state (ie, a stationary state), the terminal can obtain the foreground sub-subject by the above method. The image can improve the recognition efficiency of foreground objects and effectively achieve multiple exposures.
可选的，终端在关键图像中获取包含前景物体的前景子图像之后，可以根据前景子图像的时间信息，在目标视频数据中获取时间信息大于前景子图像的时间信息的图像，将前景子图像和各个获取到的图像进行图像融合，以对获取到的图像进行更新，根据更新后的图像，对目标视频数据进行更新，更新后的目标视频数据包括更新后的图像。Optionally, after acquiring the foreground sub-image including the foreground object in the key image, the terminal may acquire, in the target video data, an image in which the time information is greater than the time information of the foreground sub-image according to the time information of the foreground sub-image, and the foreground sub-image Image fusion is performed with each acquired image to update the acquired image, and the target video data is updated according to the updated image, and the updated target video data includes the updated image.
例如，终端在目标视频数据中选取了4帧关键图像，分别为目标视频数据中的第一帧图像、第四帧图像、第七帧图像以及第十帧图像；在第一帧图像中获取到的第一个前景子图像所包含的前景物体的运动姿态为助跑，在第四帧图像中获取到的第二个前景子图像所包含的前景物体的运动姿态为起跳，在第七帧图像中获取到的第三个前景子图像所包含的前景物体的运动姿态为跳跃至最高点，在第十帧图像中获取到的第四个前景子图像所包含的前景物体的运动姿态为落地。终端可以确定第一个前景子图像的时间信息为第一帧，则目标视频数据中时间信息大于第一帧的图像为第2-10帧图像，进而将第一个前景子图像分别和第2-10帧图像进行图像融合，得到更新后的第2-10帧图像。同理，终端可以确定第二个前景子图像的时间信息为第四帧，则目标视频数据中时间信息大于第四帧的图像为第5-10帧图像，进而将第二个前景子图像分别和上述更新后的第5-10帧图像进行图像融合，得到更新后的第5-10帧图像。同理，终端可以确定第三个前景子图像的时间信息为第七帧，则目标视频数据中时间信息大于第七帧的图像为第8-10帧图像，进而将第三个前景子图像分别和上述更新后的第8-10帧图像进行图像融合，得到更新后的第8-10帧图像。终端可以确定第四个前景子图像的时间信息为第十帧，目标视频数据中不存在时间信息大于第十帧的图像，则终端可以对目标视频数据进行更新，更新后的目标视频数据包括更新后的图像，例如目标视频数据包括第一帧图像，以及更新后的第2-10帧图像，其中更新后的第二帧图像为第一个前景子图像和第二帧图 像进行图像融合后得到的，更新后的第五帧图像为第一个前景子图像、第二个前景子图像和第五帧图像进行图像融合后得到的，更新后的第八帧图像为第一个前景子图像、第二个前景子图像、第三个前景子图像和第五帧图像进行图像融合后得到的，更新后的第十帧图像为第一个前景子图像、第二个前景子图像、第三个前景子图像、第四个前景子图像和第五帧图像进行图像融合后得到的。For example, the terminal selects four key images in the target video data, which are the first frame image, the fourth frame image, the seventh frame image, and the tenth frame image in the target video data, and are acquired in the first frame image. The motion image of the foreground object contained in the first foreground sub-image is the run-up, and the motion pose of the foreground object included in the second foreground sub-image acquired in the fourth frame image is the take-off, in the seventh frame image. The motion posture of the foreground object included in the acquired third foreground sub-image is jumped to the highest point, and the motion posture of the foreground object included in the fourth foreground sub-image acquired in the tenth frame image is landing. The terminal may determine that the time information of the first foreground sub-image is the first frame, and the time information of the target video data is larger than the image of the first frame is the image of the second to the tenth frame, and then the first foreground sub-image and the second - 10 frames of images are image-fused to obtain an updated image of the 2nd-10th frame. Similarly, the terminal may determine that the time information of the second foreground sub-image is the fourth frame, and the time information of the target video data is greater than the image of the fourth frame is the 5-10th image, and then the second foreground sub-image is respectively The image is blended with the updated 5-10th image to obtain the updated 5-10th image. Similarly, the terminal may determine that the time information of the third foreground sub-image is the seventh frame, and the image in which the time information in the target video data is larger than the seventh frame is the 8-10th image, and then the third foreground sub-image is respectively Image fusion is performed with the updated image of the 8th-10th frame to obtain an updated image of the 8th-10th frame. The terminal may determine that the time information of the fourth foreground sub-image is the tenth frame, and the target video data does not have the image whose time information is greater than the tenth frame, the terminal may update the target video data, and the updated target video data includes an update. The subsequent image, for example, the target video data includes the first frame image, and the updated 2-10th frame image, wherein the updated second frame image is the first foreground sub image and the second frame image After the image fusion is performed, the updated fifth frame image is obtained by image fusion of the first foreground sub-image, the second foreground sub-image, and the fifth frame image, and the updated eighth frame image is A foreground sub-image, a second foreground sub-image, a third foreground sub-image and a fifth frame image are obtained by image fusion, and the updated tenth frame image is the first foreground sub-image and the second foreground The sub-image, the third foreground sub-image, the fourth foreground sub-image, and the fifth frame image are obtained by image fusion.
103、将各帧前景子图像和目标视频数据的背景图像进行图像融合，得到曝光图像。103. Perform image fusion on each frame foreground sub-image and the background image of the target video data to obtain an exposure image.
具体地，终端可以将所有前景子图像和背景图像进行图像融合，得到曝光图像，曝光图像可以如图4所示。其中，背景图像可以是终端对目标视频数据进行处理得到的，也可以是终端通过摄像装置采集、在本地存储器或者通过互联网获取得到的。Specifically, the terminal may perform image fusion on all foreground sub-images and background images to obtain an exposure image, and the exposure image may be as shown in FIG. 4 . The background image may be obtained by processing the target video data by the terminal, or may be obtained by the terminal by the camera, acquired in a local memory, or obtained through the Internet.
可选的，终端将各帧前景子图像和目标视频数据的背景图像进行图像融合，得到曝光图像之前，可以对目标视频数据进行处理，得到背景图像，背景图像可以如图2所示。Optionally, the terminal performs image fusion on the foreground image of each frame and the background image of the target video data, and before the obtained image is obtained, the target video data may be processed to obtain a background image, and the background image may be as shown in FIG. 2 .
可选的，终端可以获取各帧前景子图像在前景子图像所属关键图像中的位置，根据位置，将前景子图像和背景图像进行图像融合，得到曝光图像。Optionally, the terminal may obtain a position of each foreground sub-image in the key image to which the foreground sub-image belongs, and combine the foreground sub-image and the background image according to the position to obtain an exposure image.
例如，第一个前景子图像所包含前景物体位于第一帧图像的右侧，则终端根据该位置将第一个前景子图像和背景图像进行融合，得到曝光图像，该曝光图像中第一个前景子图像所包含的前景物体位于曝光图像的右侧，且该前景物体与曝光图像各个边缘之间的距离，和该前景物体与第一帧图像对应边缘之间的距离相同。For example, if the foreground object contained in the first foreground sub-image is located on the right side of the image of the first frame, the terminal fuses the first foreground sub-image and the background image according to the position to obtain an exposure image, and the first image in the exposure image The foreground object contained in the foreground sub-image is located on the right side of the exposure image, and the distance between the foreground object and each edge of the exposure image is the same as the distance between the foreground object and the corresponding edge of the first frame image.
本发明实施例根据目标视频数据对应的图像选取算法，在目标视频数据中选取关键图像，在关键图像中获取包含前景物体的前景子图像，将前景子图像和目标视频数据的背景图像进行图像融合，得到曝光图像，操作便捷，可有效实现多重曝光，降低拍摄难度。According to the image selection algorithm corresponding to the target video data, the embodiment selects a key image in the target video data, acquires a foreground sub-image including the foreground object in the key image, and performs image fusion on the foreground sub-image and the background image of the target video data. , the exposure image is obtained, the operation is convenient, the multiple exposure can be effectively realized, and the shooting difficulty is reduced.
本发明实施例还提供了一种计算机存储介质，该计算机存储介质中存储有程序指令，所述程序执行时可包括如图1对应实施例中的图像处理方法的部分或全部步骤。 The embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores program instructions, and the program may include some or all of the steps of the image processing method in the corresponding embodiment of FIG. 1 .
请参阅图5，为本发明实施例提供的一种图像处理装置的结构示意图。本实施例中所描述的图像处理装置，包括：FIG. 5 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus described in this embodiment includes:
图像获取模块501，用于根据目标视频数据对应的图像选取算法，在所述目标视频数据中选取至少两帧关键图像，各帧所述关键图像均包括前景物体；The image obtaining module 501 is configured to select at least two frame key images in the target video data according to an image selection algorithm corresponding to the target video data, where the key images of each frame include a foreground object;
子图像获取模块502，用于在各帧所述关键图像中获取包含所述前景物体的前景子图像；a sub-image obtaining module 502, configured to acquire, in the key image of each frame, a foreground sub-image including the foreground object;
图像融合模块503，用于将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像。The image fusion module 503 is configured to image fuse the foreground sub-image of each frame and the background image of the target video data to obtain an exposure image.
可选的，所述子图像获取模块502，具体用于根据所述背景图像，在各帧所述关键图像中获取包含所述前景物体的前景子图像。Optionally, the sub-image obtaining module 502 is configured to acquire a foreground sub-image including the foreground object in the key image of each frame according to the background image.
可选的，所述前景物体为行人，则所述子图像获取模块502，具体用于根据行人识别算法对各帧所述关键图像进行行人识别，得到包含所述行人的前景子图像。Optionally, the foreground object is a pedestrian, and the sub-image obtaining module 502 is configured to perform pedestrian recognition on the key image of each frame according to a pedestrian recognition algorithm to obtain a foreground sub-image including the pedestrian.
可选的，所述图像获取模块501，还用于所述图像融合模块503将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像之前，对所述目标视频数据进行处理，得到所述背景图像。Optionally, the image obtaining module 501 is further configured to: the image fusion module 503 performs image fusion on the foreground sub-image of each frame and the background image of the target video data, and obtains the target image before the exposure image is obtained. The video data is processed to obtain the background image.
可选的，所述图像获取模块501，具体用于：Optionally, the image obtaining module 501 is specifically configured to:
获取所述目标视频数据的应用场景；Obtaining an application scenario of the target video data;
根据预先设定的应用场景和图像选取算法的对应关系，获取所述应用场景对应的图像选取算法；Acquiring an image selection algorithm corresponding to the application scenario according to a corresponding relationship between the preset application scenario and the image selection algorithm;
将所述目标视频数据作为所述图像选取算法的输入，得到所述至少两帧关键图像。Using the target video data as an input of the image selection algorithm, the at least two frames of key images are obtained.
可选的，所述图像获取模块501将所述目标视频数据作为所述图像选取算法的输入，得到所述关键图像，具体用于：Optionally, the image obtaining module 501 uses the target video data as an input of the image selection algorithm to obtain the key image, specifically for:
每间隔预设数量帧在所述目标视频数据中获取一帧图像；Obtaining one frame of image in the target video data every predetermined number of frames;
将获取到的图像作为所述关键图像。The acquired image is taken as the key image.
可选的，所述图像获取模块501将所述目标视频数据作为所述图像选取算法的输入，得到所述关键图像，具体用于： Optionally, the image obtaining module 501 uses the target video data as an input of the image selection algorithm to obtain the key image, specifically for:
根据所述背景图像，在所述目标视频数据所包含的每一帧图像中获取前景子图像；Obtaining a foreground sub-image in each frame image included in the target video data according to the background image;
根据各个所述前景子图像的空间信息和时间信息，选取目标前景子图像；Selecting a target foreground sub-image according to spatial information and time information of each of the foreground sub-images;
将所述目标前景子图像所属图像确定为关键图像。The image to which the target foreground sub-image belongs is determined as a key image.
可选的，所述图像融合模块503，具体用于：Optionally, the image fusion module 503 is specifically configured to:
获取各帧所述前景子图像在所述前景子图像所属关键图像中的位置；Obtaining, in each frame, a position of the foreground sub-image in a key image to which the foreground sub-image belongs;
根据所述位置，将所述前景子图像和所述背景图像进行图像融合，得到所述曝光图像。According to the position, the foreground sub-image and the background image are image-fused to obtain the exposed image.
可选的，所述图像获取模块501，还用于所述子图像获取模块502在各帧所述关键图像中获取包含所述前景物体的前景子图像之后，根据所述前景子图像的时间信息，在所述目标视频数据中获取时间信息大于所述前景子图像的时间信息的图像；Optionally, the image obtaining module 501 is further configured to: after the sub-image obtaining module 502 acquires a foreground sub-image including the foreground object in each key image of each frame, according to time information of the foreground sub-image Obtaining, in the target video data, an image in which time information is greater than time information of the foreground sub-image;
所述图像融合模块503，还用于将所述前景子图像和各个获取到的图像进行图像融合，以对所述获取到的图像进行更新；The image fusion module 503 is further configured to image fuse the foreground sub-image and each acquired image to update the acquired image;
所述图像处理装置还包括：The image processing apparatus further includes:
更新模块504，用于根据更新后的图像，对所述目标视频数据进行更新，更新后的目标视频数据包括所述更新后的图像。The update module 504 is configured to update the target video data according to the updated image, and the updated target video data includes the updated image.
可以理解的是，本发明实施例的图像处理装置的各功能模块的功能可根据上述方法实施例中的方法具体实现，其具体实现过程可以参照上述方法实施例的相关描述，此处不再赘述。It is to be understood that the functions of the functional modules of the image processing apparatus of the embodiments of the present invention may be specifically implemented according to the method in the foregoing method embodiments, and the specific implementation process may refer to the related description of the foregoing method embodiments, and details are not described herein again. .
本发明实施例中图像获取模块501根据目标视频数据对应的图像选取算法，在目标视频数据中选取关键图像，子图像获取模块502在关键图像中获取包含前景物体的前景子图像，图像融合模块503将前景子图像和目标视频数据的背景图像进行图像融合，得到曝光图像，操作便捷，可有效实现多重曝光，降低拍摄难度。In the embodiment of the present invention, the image obtaining module 501 selects a key image in the target video data according to the image selection algorithm corresponding to the target video data, and the sub-image acquiring module 502 acquires a foreground sub-image including the foreground object in the key image, and the image fusion module 503 The foreground image and the background image of the target video data are image-fused to obtain an exposed image, which is convenient to operate, and can effectively achieve multiple exposures and reduce shooting difficulty.
请参阅图6，为本发明实施例提供的一种终端的结构示意图。本实施例中所描述的终端，包括：存储器601和处理器602。上述处理器602和存储器601通过总线连接。 FIG. 6 is a schematic structural diagram of a terminal according to an embodiment of the present invention. The terminal described in this embodiment includes: a memory 601 and a processor 602. The above processor 602 and memory 601 are connected by a bus.
上述处理器602可以是中央处理单元(Central Processing Unit，CPU)，该处理器还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。The processor 602 may be a central processing unit (CPU), and the processor may be another general-purpose processor, a digital signal processor (DSP), or an application specific integrated circuit (ASIC). ), a Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, and the like. The general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
上述存储器601可以包括只读存储器和随机存取存储器，并向处理器602提供指令和数据。存储器601的一部分还可以包括非易失性随机存取存储器。其中：The above memory 601 can include read only memory and random access memory and provides instructions and data to the processor 602. A portion of the memory 601 may also include a non-volatile random access memory. among them:
所述存储器601，用于存储程序指令；The memory 601 is configured to store program instructions;
所述处理器602，用于调用所述程序指令，当所述程序指令被执行时，执行以下操作：The processor 602 is configured to invoke the program instruction, and when the program instruction is executed, perform the following operations:
根据目标视频数据对应的图像选取算法，在所述目标视频数据中选取至少两帧关键图像，各帧所述关键图像均包括前景物体；Selecting at least two frames of key images in the target video data according to an image selection algorithm corresponding to the target video data, wherein the key images of each frame include a foreground object;
在各帧所述关键图像中获取包含所述前景物体的前景子图像；Obtaining a foreground sub-image including the foreground object in the key image of each frame;
将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像。The foreground sub-image of each frame and the background image of the target video data are image-fused to obtain an exposure image.
可选的，所述处理器602在各帧所述关键图像中获取包含所述前景物体的前景子图像，具体用于：Optionally, the processor 602 obtains, in the key image of each frame, a foreground sub-image that includes the foreground object, specifically for:
根据所述背景图像，在各帧所述关键图像中获取包含所述前景物体的前景子图像。According to the background image, a foreground sub-image including the foreground object is acquired in the key image of each frame.
可选的，所述前景物体为行人，则所述处理器602在各帧所述关键图像中获取包含所述前景物体的前景子图像，具体用于：Optionally, the foreground object is a pedestrian, and the processor 602 obtains a foreground sub-image including the foreground object in the key image of each frame, specifically:
根据行人识别算法对各帧所述关键图像进行行人识别，得到包含所述行人的前景子图像。Pedestrian recognition is performed on the key image of each frame according to a pedestrian recognition algorithm to obtain a foreground sub-image including the pedestrian.
可选的，所述处理器602，还用于将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像之前，对所述目标视频数据进行处理，得到所述背景图像。Optionally, the processor 602 is further configured to perform image fusion on the foreground sub-image of each frame and the background image of the target video data, and process the target video data before obtaining the exposed image to obtain the The background image.
可选的，所述处理器602根据目标视频数据对应的图像选取算法，在所述 目标视频数据中选取至少两帧关键图像，具体用于：Optionally, the processor 602 is configured according to an image selection algorithm corresponding to the target video data. Select at least two frames of key images from the target video data, specifically for:
获取所述目标视频数据的应用场景；Obtaining an application scenario of the target video data;
根据预先设定的应用场景和图像选取算法的对应关系，获取所述应用场景对应的图像选取算法；Acquiring an image selection algorithm corresponding to the application scenario according to a corresponding relationship between the preset application scenario and the image selection algorithm;
将所述目标视频数据作为所述图像选取算法的输入，得到所述至少两帧关键图像。Using the target video data as an input of the image selection algorithm, the at least two frames of key images are obtained.
可选的，所述处理器602将所述目标视频数据作为所述图像选取算法的输入，得到所述至少两帧关键图像，具体用于：Optionally, the processor 602 uses the target video data as an input of the image selection algorithm to obtain the at least two frames of key images, specifically for:
每间隔预设数量帧在所述目标视频数据中获取一帧图像；Obtaining one frame of image in the target video data every predetermined number of frames;
将获取到的图像作为所述关键图像。The acquired image is taken as the key image.
可选的，所述处理器602将所述目标视频数据作为所述图像选取算法的输入，得到所述至少两帧关键图像，具体用于：Optionally, the processor 602 uses the target video data as an input of the image selection algorithm to obtain the at least two frames of key images, specifically for:
根据所述背景图像，在所述目标视频数据所包含的每一帧图像中获取前景子图像；Obtaining a foreground sub-image in each frame image included in the target video data according to the background image;
根据各个所述前景子图像的空间信息和时间信息，选取目标前景子图像；Selecting a target foreground sub-image according to spatial information and time information of each of the foreground sub-images;
将所述目标前景子图像所属图像确定为关键图像。The image to which the target foreground sub-image belongs is determined as a key image.
可选的，所述处理器602将各帧所述前景子图像和所述背景图像进行图像融合，得到曝光图像，具体用于：Optionally, the processor 602 performs image fusion on the foreground sub-image and the background image of each frame to obtain an exposure image, specifically for:
获取各帧所述前景子图像在所述前景子图像所属关键图像中的位置；Obtaining, in each frame, a position of the foreground sub-image in a key image to which the foreground sub-image belongs;
根据所述位置，将所述前景子图像和所述背景图像进行图像融合，得到所述曝光图像。According to the position, the foreground sub-image and the background image are image-fused to obtain the exposed image.
可选的，所述处理器602，还用于在各帧所述关键图像中获取包含所述前景物体的前景子图像之后，根据所述前景子图像的时间信息，在所述目标视频数据中获取时间信息大于所述前景子图像的时间信息的图像；Optionally, the processor 602 is further configured to: after acquiring a foreground sub-image including the foreground object in each key image of each frame, according to time information of the foreground sub-image, in the target video data Obtaining an image in which time information is greater than time information of the foreground sub-image;
所述处理器602，还用于将所述前景子图像和各个获取到的图像进行图像融合，以对所述获取到的图像进行更新；The processor 602 is further configured to perform image fusion on the foreground sub-image and each acquired image to update the acquired image;
所述处理器602，还用于根据更新后的图像，对所述目标视频数据进行更新，更新后的目标视频数据包括所述更新后的图像。The processor 602 is further configured to update the target video data according to the updated image, where the updated target video data includes the updated image.
具体实现中，本发明实施例中所描述处理器602可执行本发明实施例图1 提供的图像处理方法中所描述的实现方式，也可执行本发明实施例图5所描述的图像处理装置的实现方式，在此不再赘述。In a specific implementation, the processor 602 described in the embodiment of the present invention may be implemented in the embodiment of the present invention. The implementation of the image processing method described in the embodiment of the present invention may also be implemented, and details are not described herein again.
需要说明的是，对于前述的各个方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明并不受所描述的动作顺序的限制，因为依据本申请，某一些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本申请所必须的。It should be noted that, for the foregoing various method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence. Because some steps may be performed in other orders or concurrently in accordance with the present application. In the following, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：闪存盘、只读存储器(Read-Only Memory，ROM)、随机存取器(Random Access Memory，RAM)、磁盘或光盘等。A person skilled in the art may understand that all or part of the various steps of the foregoing embodiments may be performed by a program to instruct related hardware. The program may be stored in a computer readable storage medium, and the storage medium may include: Flash disk, Read-Only Memory (ROM), Random Access Memory (RAM), disk or optical disk.
以上对本发明实施例所提供的一种控制终端的控制方法、装置、设备及飞行器进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。 The control method, device, device and aircraft of the control terminal provided by the embodiment of the present invention are described in detail. The principle and implementation manner of the present invention are described in the following. The description of the above embodiment is only The method for understanding the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in specific embodiments and application scopes. The description should not be construed as limiting the invention.

Claims (27)

  1. 一种图像处理方法，其特征在于，所述方法包括：An image processing method, the method comprising:
    根据目标视频数据对应的图像选取算法，在所述目标视频数据中选取至少两帧关键图像，各帧所述关键图像均包括前景物体；Selecting at least two frames of key images in the target video data according to an image selection algorithm corresponding to the target video data, wherein the key images of each frame include a foreground object;
    在各帧所述关键图像中获取包含所述前景物体的前景子图像；Obtaining a foreground sub-image including the foreground object in the key image of each frame;
    将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像。The foreground sub-image of each frame and the background image of the target video data are image-fused to obtain an exposure image.
  2. 如权利要求1所述的方法，其特征在于，所述在各帧所述关键图像中获取包含所述前景物体的前景子图像，包括：The method of claim 1, wherein the acquiring a foreground sub-image comprising the foreground object in the key image of each frame comprises:
    根据所述背景图像，在各帧所述关键图像中获取包含所述前景物体的前景子图像。According to the background image, a foreground sub-image including the foreground object is acquired in the key image of each frame.
  3. 如权利要求1所述的方法，其特征在于，所述前景物体为行人；The method of claim 1 wherein said foreground object is a pedestrian;
    所述在各帧所述关键图像中获取包含所述前景物体的前景子图像，包括：Obtaining, in the key image of each frame, a foreground sub-image including the foreground object, including:
    根据行人识别算法对各帧所述关键图像进行行人识别，得到包含所述行人的前景子图像。Pedestrian recognition is performed on the key image of each frame according to a pedestrian recognition algorithm to obtain a foreground sub-image including the pedestrian.
  4. 如权利要求1所述的方法，其特征在于，所述将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像之前，还包括：The method according to claim 1, wherein the image fusion of the foreground sub-image of each frame and the background image of the target video data to obtain an exposure image further comprises:
    对所述目标视频数据进行处理，得到所述背景图像。The target video data is processed to obtain the background image.
  5. 如权利要求1所述的方法，其特征在于，所述根据目标视频数据对应的图像选取算法，在所述目标视频数据中选取至少两帧关键图像，包括：The method according to claim 1, wherein the image selection algorithm corresponding to the target video data selects at least two frame key images in the target video data, including:
    获取所述目标视频数据的应用场景；Obtaining an application scenario of the target video data;
    根据预先设定的应用场景和图像选取算法的对应关系，获取所述应用场景对应的图像选取算法；Acquiring an image selection algorithm corresponding to the application scenario according to a corresponding relationship between the preset application scenario and the image selection algorithm;
    将所述目标视频数据作为所述图像选取算法的输入，得到所述至少两帧关键图像。 Using the target video data as an input of the image selection algorithm, the at least two frames of key images are obtained.
  6. 如权利要求5所述的方法，其特征在于，所述将所述目标视频数据作为所述图像选取算法的输入，得到所述至少两帧关键图像，包括：The method of claim 5, wherein the using the target video data as an input of the image selection algorithm to obtain the at least two frames of key images comprises:
    每间隔预设数量帧在所述目标视频数据中获取一帧图像；Obtaining one frame of image in the target video data every predetermined number of frames;
    将获取到的图像作为所述关键图像。The acquired image is taken as the key image.
  7. 如权利要求5所述的方法，其特征在于，所述将所述目标视频数据作为所述图像选取算法的输入，得到所述至少两帧关键图像，包括：The method of claim 5, wherein the using the target video data as an input of the image selection algorithm to obtain the at least two frames of key images comprises:
    根据所述背景图像，在所述目标视频数据所包含的每一帧图像中获取前景子图像；Obtaining a foreground sub-image in each frame image included in the target video data according to the background image;
    根据各个所述前景子图像的空间信息和时间信息，选取目标前景子图像；Selecting a target foreground sub-image according to spatial information and time information of each of the foreground sub-images;
    将所述目标前景子图像所属图像确定为所述关键图像。An image to which the target foreground sub-image belongs is determined as the key image.
  8. 如权利要求1所述的方法，其特征在于，所述将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像，包括：The method according to claim 1, wherein the image fusion of the foreground sub-image of each frame and the background image of the target video data to obtain an exposure image comprises:
    获取各帧所述前景子图像在所述前景子图像所属关键图像中的位置；Obtaining, in each frame, a position of the foreground sub-image in a key image to which the foreground sub-image belongs;
    根据所述位置，将所述前景子图像和所述背景图像进行图像融合，得到所述曝光图像。According to the position, the foreground sub-image and the background image are image-fused to obtain the exposed image.
  9. 如权利要求1所述的方法，其特征在于，所述在各帧所述关键图像中获取包含所述前景物体的前景子图像之后，还包括：The method according to claim 1, wherein after acquiring the foreground sub-image including the foreground object in the key image of each frame, the method further comprises:
    根据所述前景子图像的时间信息，在所述目标视频数据中获取时间信息大于所述前景子图像的时间信息的图像；Obtaining, in the target video data, an image in which time information is greater than time information of the foreground sub-image according to time information of the foreground sub-image;
    将所述前景子图像和各个获取到的图像进行图像融合，以对所述获取到的图像进行更新；Performing image fusion on the foreground sub-image and each acquired image to update the acquired image;
    根据更新后的图像，对所述目标视频数据进行更新，更新后的目标视频数据包括所述更新后的图像。The target video data is updated according to the updated image, and the updated target video data includes the updated image.
  10. 一种图像处理装置，其特征在于，所述装置包括： An image processing apparatus, characterized in that the apparatus comprises:
    图像获取模块，用于根据目标视频数据对应的图像选取算法，在所述目标视频数据中选取至少两帧关键图像，各帧所述关键图像均包括前景物体；An image obtaining module, configured to select at least two frame key images in the target video data according to an image selection algorithm corresponding to the target video data, where the key images in each frame include a foreground object;
    子图像获取模块，用于在各帧所述关键图像中获取包含所述前景物体的前景子图像；a sub-image obtaining module, configured to acquire a foreground sub-image including the foreground object in the key image of each frame;
    图像融合模块，用于将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像。And an image fusion module, configured to image fuse the foreground sub-image of each frame and the background image of the target video data to obtain an exposure image.
  11. 如权利要求10所述的装置，其特征在于，The device of claim 10 wherein:
    所述子图像获取模块，具体用于根据所述背景图像，在各帧所述关键图像中获取包含所述前景物体的前景子图像。And the sub-image obtaining module is configured to acquire a foreground sub-image including the foreground object in the key image of each frame according to the background image.
  12. 如权利要求10所述的装置，其特征在于，所述前景物体为行人；The device of claim 10 wherein said foreground object is a pedestrian;
    所述子图像获取模块，具体用于根据行人识别算法对各帧所述关键图像进行行人识别，得到包含所述行人的前景子图像。The sub-image obtaining module is specifically configured to perform pedestrian recognition on the key image of each frame according to a pedestrian recognition algorithm to obtain a foreground sub-image including the pedestrian.
  13. 如权利要求10所述的装置，其特征在于，The device of claim 10 wherein:
    所述图像获取模块，还用于所述图像融合模块将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像之前，对所述目标视频数据进行处理，得到所述背景图像。The image acquisition module is further configured to: the image fusion module performs image fusion on the foreground sub-image of each frame and the background image of the target video data, and processes the target video data before obtaining the exposed image to obtain The background image.
  14. 如权利要求10所述的装置，其特征在于，所述图像获取模块，具体用于：The device according to claim 10, wherein the image acquisition module is specifically configured to:
    获取所述目标视频数据的应用场景；Obtaining an application scenario of the target video data;
    根据预先设定的应用场景和图像选取算法的对应关系，获取所述应用场景对应的图像选取算法；Acquiring an image selection algorithm corresponding to the application scenario according to a corresponding relationship between the preset application scenario and the image selection algorithm;
    将所述目标视频数据作为所述图像选取算法的输入，得到所述至少两帧关键图像。Using the target video data as an input of the image selection algorithm, the at least two frames of key images are obtained.
  15. 如权利要求14所述的装置，其特征在于，所述图像获取模块将所述 目标视频数据作为所述图像选取算法的输入，得到所述关键图像，具体用于：The apparatus of claim 14 wherein said image acquisition module The target video data is used as an input of the image selection algorithm to obtain the key image, specifically for:
    每间隔预设数量帧在所述目标视频数据中获取一帧图像；Obtaining one frame of image in the target video data every predetermined number of frames;
    将获取到的图像作为所述关键图像。The acquired image is taken as the key image.
  16. 如权利要求14所述的装置，其特征在于，所述图像获取模块将所述目标视频数据作为所述图像选取算法的输入，得到所述关键图像，具体用于：The device according to claim 14, wherein the image acquisition module uses the target video data as an input of the image selection algorithm to obtain the key image, specifically for:
    根据所述背景图像，在所述目标视频数据所包含的每一帧图像中获取前景子图像；Obtaining a foreground sub-image in each frame image included in the target video data according to the background image;
    根据各个所述前景子图像的空间信息和时间信息，选取目标前景子图像；Selecting a target foreground sub-image according to spatial information and time information of each of the foreground sub-images;
    将所述目标前景子图像所属图像确定为所述关键图像。An image to which the target foreground sub-image belongs is determined as the key image.
  17. 如权利要求10所述的装置，其特征在于，所述图像融合模块，具体用于：The device according to claim 10, wherein the image fusion module is specifically configured to:
    获取各帧所述前景子图像在所述前景子图像所属关键图像中的位置；Obtaining, in each frame, a position of the foreground sub-image in a key image to which the foreground sub-image belongs;
    根据所述位置，将所述前景子图像和所述背景图像进行图像融合，得到所述曝光图像。According to the position, the foreground sub-image and the background image are image-fused to obtain the exposed image.
  18. 如权利要求10所述的装置，其特征在于，The device of claim 10 wherein:
    所述图像获取模块，还用于所述子图像获取模块在各帧所述关键图像中获取包含所述前景物体的前景子图像之后，根据所述前景子图像的时间信息，在所述目标视频数据中获取时间信息大于所述前景子图像的时间信息的图像；The image acquisition module is further configured to: after the sub-image acquisition module acquires a foreground sub-image including the foreground object in each key image of each frame, according to time information of the foreground sub-image, in the target video Obtaining an image in the data that has time information greater than time information of the foreground sub-image;
    所述图像融合模块，还用于将所述前景子图像和各个获取到的图像进行图像融合，以对所述获取到的图像进行更新；The image fusion module is further configured to image fuse the foreground sub-image and each acquired image to update the acquired image;
    所述图像处理装置还包括：The image processing apparatus further includes:
    更新模块，用于根据更新后的图像，对所述目标视频数据进行更新，更新后的目标视频数据包括所述更新后的图像。And an update module, configured to update the target video data according to the updated image, where the updated target video data includes the updated image.
  19. 一种终端，其特征在于，包括：存储器和处理器；A terminal, comprising: a memory and a processor;
    所述存储器，用于存储程序指令； The memory is configured to store program instructions;
    所述处理器，用于调用所述程序指令，当所述程序指令被执行时，执行以下操作：The processor is configured to invoke the program instruction, and when the program instruction is executed, perform the following operations:
    根据目标视频数据对应的图像选取算法，在所述目标视频数据中选取至少两帧关键图像，各帧所述关键图像均包括前景物体；Selecting at least two frames of key images in the target video data according to an image selection algorithm corresponding to the target video data, wherein the key images of each frame include a foreground object;
    在各帧所述关键图像中获取包含所述前景物体的前景子图像；Obtaining a foreground sub-image including the foreground object in the key image of each frame;
    将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像。The foreground sub-image of each frame and the background image of the target video data are image-fused to obtain an exposure image.
  20. 如权利要求19所述的终端，其特征在于，所述处理器在各帧所述关键图像中获取包含所述前景物体的前景子图像，具体用于：The terminal according to claim 19, wherein the processor acquires a foreground sub-image including the foreground object in the key image of each frame, specifically for:
    根据所述背景图像，在各帧所述关键图像中获取包含所述前景物体的前景子图像。According to the background image, a foreground sub-image including the foreground object is acquired in the key image of each frame.
  21. 如权利要求19所述的终端，其特征在于，所述前景物体为行人；The terminal according to claim 19, wherein said foreground object is a pedestrian;
    所述处理器在各帧所述关键图像中获取包含所述前景物体的前景子图像，具体用于：The processor acquires a foreground sub-image including the foreground object in the key image of each frame, specifically for:
    根据行人识别算法对各帧所述关键图像进行行人识别，得到包含所述行人的前景子图像。Pedestrian recognition is performed on the key image of each frame according to a pedestrian recognition algorithm to obtain a foreground sub-image including the pedestrian.
  22. 如权利要求19所述的终端，其特征在于，The terminal of claim 19, wherein:
    所述处理器，还用于将各帧所述前景子图像和所述目标视频数据的背景图像进行图像融合，得到曝光图像之前，对所述目标视频数据进行处理，得到所述背景图像。The processor is further configured to image fuse the foreground sub-image of each frame and the background image of the target video data, and process the target video data to obtain the background image before obtaining the exposed image.
  23. 如权利要求19所述的终端，其特征在于，所述处理器根据目标视频数据对应的图像选取算法，在所述目标视频数据中选取至少两帧关键图像，具体用于：The terminal according to claim 19, wherein the processor selects at least two frame key images in the target video data according to an image selection algorithm corresponding to the target video data, specifically for:
    获取所述目标视频数据的应用场景；Obtaining an application scenario of the target video data;
    根据预先设定的应用场景和图像选取算法的对应关系，获取所述应用场景 对应的图像选取算法；Obtaining the application scenario according to a corresponding relationship between a preset application scenario and an image selection algorithm Corresponding image selection algorithm;
    将所述目标视频数据作为所述图像选取算法的输入，得到所述至少两帧关键图像。Using the target video data as an input of the image selection algorithm, the at least two frames of key images are obtained.
  24. 如权利要求23所述的终端，其特征在于，所述处理器将所述目标视频数据作为所述图像选取算法的输入，得到所述至少两帧关键图像，具体用于：The terminal according to claim 23, wherein the processor uses the target video data as an input of the image selection algorithm to obtain the at least two frame key images, specifically for:
    每间隔预设数量帧在所述目标视频数据中获取一帧图像；Obtaining one frame of image in the target video data every predetermined number of frames;
    将获取到的图像作为所述关键图像。The acquired image is taken as the key image.
  25. 如权利要求23所述的终端，其特征在于，所述处理器将所述目标视频数据作为所述图像选取算法的输入，得到所述至少两帧关键图像，具体用于：The terminal according to claim 23, wherein the processor uses the target video data as an input of the image selection algorithm to obtain the at least two frame key images, specifically for:
    根据所述背景图像，在所述目标视频数据所包含的每一帧图像中获取前景子图像；Obtaining a foreground sub-image in each frame image included in the target video data according to the background image;
    根据各个所述前景子图像的空间信息和时间信息，选取目标前景子图像；Selecting a target foreground sub-image according to spatial information and time information of each of the foreground sub-images;
    将所述目标前景子图像所属图像确定为关键图像。The image to which the target foreground sub-image belongs is determined as a key image.
  26. 如权利要求19所述的终端，其特征在于，所述处理器将各帧所述前景子图像和所述背景图像进行图像融合，得到曝光图像，具体用于：The terminal according to claim 19, wherein the processor combines the foreground sub-image and the background image of each frame to obtain an exposure image, specifically for:
    获取各帧所述前景子图像在所述前景子图像所属关键图像中的位置；Obtaining, in each frame, a position of the foreground sub-image in a key image to which the foreground sub-image belongs;
    根据所述位置，将所述前景子图像和所述背景图像进行图像融合，得到所述曝光图像。According to the position, the foreground sub-image and the background image are image-fused to obtain the exposed image.
  27. 如权利要求19所述的终端，其特征在于，The terminal of claim 19, wherein:
    所述处理器，还用于在各帧所述关键图像中获取包含所述前景物体的前景子图像之后，根据所述前景子图像的时间信息，在所述目标视频数据中获取时间信息大于所述前景子图像的时间信息的图像；The processor is further configured to: after obtaining a foreground sub-image including the foreground object in each key image of each frame, obtain time information in the target video data that is greater than the time information according to time information of the foreground sub-image An image of time information of the foreground sub-image;
    所述处理器，还用于将所述前景子图像和各个获取到的图像进行图像融合，以对所述获取到的图像进行更新；The processor is further configured to image fuse the foreground sub-image and each acquired image to update the acquired image;
    所述处理器，还用于根据更新后的图像，对所述目标视频数据进行更新，更新后的目标视频数据包括所述更新后的图像。 The processor is further configured to update the target video data according to the updated image, where the updated target video data includes the updated image.
PCT/CN2017/108314 2017-10-30 2017-10-30 Image processing method and apparatus, and terminal WO2019084712A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/CN2017/108314 WO2019084712A1 (en) 2017-10-30 2017-10-30 Image processing method and apparatus, and terminal
CN201780009967.8A CN108702463B (en) 2017-10-30 2017-10-30 Image processing method and device and terminal
CN202011396682.4A CN112541414A (en) 2017-10-30 2017-10-30 Image processing method and device and terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/108314 WO2019084712A1 (en) 2017-10-30 2017-10-30 Image processing method and apparatus, and terminal

Publications (1)

Publication Number Publication Date
WO2019084712A1 true WO2019084712A1 (en) 2019-05-09

Family

ID=63844127

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/108314 WO2019084712A1 (en) 2017-10-30 2017-10-30 Image processing method and apparatus, and terminal

Country Status (2)

Country Link
CN (2) CN112541414A (en)
WO (1) WO2019084712A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113313788A (en) * 2020-02-26 2021-08-27 北京小米移动软件有限公司 Image processing method and apparatus, electronic device, and computer-readable storage medium
CN113808066A (en) * 2020-05-29 2021-12-17 Oppo广东移动通信有限公司 Image selection method and device, storage medium and electronic equipment
EP4068794A4 (en) * 2019-12-30 2022-12-28 Beijing Bytedance Network Technology Co., Ltd. Image processing method and apparatus

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110070551B (en) * 2019-04-29 2020-06-30 北京字节跳动网络技术有限公司 Video image rendering method and device and electronic equipment
CN114795072B (en) * 2022-07-01 2022-10-04 广东欧谱曼迪科技有限公司 Endoscope light source control method and device, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103035013A (en) * 2013-01-08 2013-04-10 东北师范大学 Accurate moving shadow detection method based on multi-feature fusion
CN105847694A (en) * 2016-04-27 2016-08-10 乐视控股（北京）有限公司 Multiple exposure shooting method and system based on picture synthesis
CN105959535A (en) * 2016-04-27 2016-09-21 乐视控股（北京）有限公司 Multiple exposure method and system based on picture synthesis
US20170134754A1 (en) * 2015-11-06 2017-05-11 Raytheon Company Efficient video data representation and content based video retrieval framework

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100189053B1 (en) * 1996-05-06 1999-06-01 이대원 Digital still camera with the multi-exposure function
EP2606637B1 (en) * 2010-08-23 2016-09-21 Red.Com, Inc. High dynamic range video
CN103327253B (en) * 2013-06-26 2015-05-13 努比亚技术有限公司 Multiple exposure method and camera shooting device
CN106851125B (en) * 2017-03-31 2020-10-16 努比亚技术有限公司 Mobile terminal and multiple exposure shooting method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103035013A (en) * 2013-01-08 2013-04-10 东北师范大学 Accurate moving shadow detection method based on multi-feature fusion
US20170134754A1 (en) * 2015-11-06 2017-05-11 Raytheon Company Efficient video data representation and content based video retrieval framework
CN105847694A (en) * 2016-04-27 2016-08-10 乐视控股（北京）有限公司 Multiple exposure shooting method and system based on picture synthesis
CN105959535A (en) * 2016-04-27 2016-09-21 乐视控股（北京）有限公司 Multiple exposure method and system based on picture synthesis

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4068794A4 (en) * 2019-12-30 2022-12-28 Beijing Bytedance Network Technology Co., Ltd. Image processing method and apparatus
US11798596B2 (en) 2019-12-30 2023-10-24 Beijing Bytedance Network Technology Co., Ltd. Image processing method and apparatus
CN113313788A (en) * 2020-02-26 2021-08-27 北京小米移动软件有限公司 Image processing method and apparatus, electronic device, and computer-readable storage medium
CN113808066A (en) * 2020-05-29 2021-12-17 Oppo广东移动通信有限公司 Image selection method and device, storage medium and electronic equipment

Also Published As

Publication number Publication date
CN108702463B (en) 2020-12-29
CN112541414A (en) 2021-03-23
CN108702463A (en) 2018-10-23

Similar Documents

Publication Publication Date Title
WO2019084712A1 (en) Image processing method and apparatus, and terminal
US11003893B2 (en) Face location tracking method, apparatus, and electronic device
US10284789B2 (en) Dynamic generation of image of a scene based on removal of undesired object present in the scene
US10580140B2 (en) Method and system of real-time image segmentation for image processing
US10917571B2 (en) Image capture device control based on determination of blur value of objects in images
US10134165B2 (en) Image distractor detection and processing
US10121256B2 (en) Temporal saliency map
KR20230013243A (en) Maintain a fixed size for the target object in the frame
US10467498B2 (en) Method and device for capturing images using image templates
US10620826B2 (en) Object selection based on region of interest fusion
US20180268207A1 (en) Method for automatic facial impression transformation, recording medium and device for performing the method
US11113507B2 (en) System and method for fast object detection
WO2018102880A1 (en) Systems and methods for replacing faces in videos
US11367196B2 (en) Image processing method, apparatus, and storage medium
CN113496208B (en) Video scene classification method and device, storage medium and terminal
KR102572986B1 (en) Object Tracking Based on Custom Initialization Points
US10432853B2 (en) Image processing for automatic detection of focus area
US10089721B2 (en) Image processing system and method for object boundary smoothening for image segmentation
EP3234865A1 (en) Techniques for providing user image capture feedback for improved machine language translation
US20180053294A1 (en) Video processing system and method for deformation insensitive tracking of objects in a sequence of image frames
CN108229281B (en) Neural network generation method, face detection device and electronic equipment
CN112219218A (en) Method and electronic device for recommending image capture mode
WO2022206679A1 (en) Image processing method and apparatus, computer device and storage medium
KR102372711B1 (en) Image photographing apparatus and control method thereof
CN111476063B (en) Target tracking method, device, storage medium and electronic equipment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17930638

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17930638

Country of ref document: EP

Kind code of ref document: A1