WO2020220292A1 - 电子增稳方法、图像采集设备、可移动平台 - Google Patents

电子增稳方法、图像采集设备、可移动平台 Download PDF

Info

Publication number
WO2020220292A1
WO2020220292A1 PCT/CN2019/085265 CN2019085265W WO2020220292A1 WO 2020220292 A1 WO2020220292 A1 WO 2020220292A1 CN 2019085265 W CN2019085265 W CN 2019085265W WO 2020220292 A1 WO2020220292 A1 WO 2020220292A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
stabilized
posture data
posture
acquisition device
Prior art date
Application number
PCT/CN2019/085265
Other languages
English (en)
French (fr)
Inventor
李兵
邹文
马骏
林光远
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2019/085265 priority Critical patent/WO2020220292A1/zh
Priority to JP2019544054A priority patent/JP2021525009A/ja
Priority to CN201980001010.8A priority patent/CN110235431B/zh
Priority to US16/538,946 priority patent/US10674087B1/en
Priority to US16/862,742 priority patent/US11025823B2/en
Publication of WO2020220292A1 publication Critical patent/WO2020220292A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6811—Motion detection based on the image signal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6812—Motion detection based on additional sensors, e.g. acceleration sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/683—Vibration or motion blur correction performed by a processor, e.g. controlling the readout of an image memory
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/72—Modifying the appearance of television pictures by optical filters or diffusing screens
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the embodiment of the present invention relates to the field of control technology, in particular to an electronic stabilization method, an image acquisition device, a movable platform, and a machine-readable storage medium.
  • EIS electronic image stabilization
  • the EIS algorithm performs posture correction on videos (or images) acquired by the image acquisition devices. For example, after a frame of image is exposed, the EIS algorithm performs low-pass filtering on the spatial attitude data collected by the gyroscope within a period of time before the exposure time to generate a smooth curve (ie, target attitude). Then, the EIS algorithm calculates the posture compensation amount according to the target posture curve, and corrects the video based on the posture compensation amount, so that the user can watch a relatively smooth video picture.
  • the EIS algorithm needs to use a frequency-domain low-pass filter in the process of generating a smooth curve.
  • the frequency-domain low-pass filter generates a second-level delay in the filtering process. If the user rotates during this delay period
  • the real-time posture of the image acquisition equipment and the target posture will have a large deviation, resulting in jitter or twitching of the video picture, which reduces the viewing experience.
  • the embodiment of the present invention provides an electronic stabilization method, an image acquisition device, a movable platform, and a machine-readable storage medium.
  • an embodiment of the present invention provides an electronic stabilization method, including:
  • an embodiment of the present invention provides an image acquisition device, including a processor, an image sensor, and a spatial attitude sensor; the processor is respectively communicatively connected with the image sensor and the spatial attitude sensor; the processor is configured to :
  • an embodiment of the present invention provides a movable platform including a body, a power supply battery provided on the body, a power system, a flight controller, and the image acquisition device according to the second aspect, the power supply battery It can supply power to the power system, and the power system provides flight power for the drone.
  • an embodiment of the present invention provides a machine-readable storage medium with a number of computer instructions stored on the machine-readable storage medium, and the computer instructions implement the steps of the method described in the third aspect when executed.
  • the first posture data and the second posture data can be used to obtain the information of the image acquisition device.
  • the target posture corresponding to the exposure moment is set; then, the image to be stabilized can be stabilized according to the target posture to obtain a stabilized target image.
  • the second posture data can be used to determine the movement of the image acquisition device after the exposure of the image to be stabilized, so as to ensure that a smooth target posture can be obtained after the actual movement of the image acquisition device is filtered, avoiding post-stabilization
  • the phenomenon of volatility or fluctuations in the image is helpful to improve the stability of the display and the viewing experience.
  • FIG. 1 is a schematic flowchart of an electronic stabilization method provided by an embodiment of the present invention
  • FIG. 2 is a schematic diagram of a process for acquiring a target posture according to an embodiment of the present invention
  • Figure 3 is a schematic diagram of the actual posture and the target posture provided by an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a target posture provided by an embodiment of the present invention.
  • FIG. 5 is a schematic flowchart of another electronic stabilization method provided by an embodiment of the present invention.
  • FIG. 6 is a schematic flowchart of another electronic stabilization method provided by an embodiment of the present invention.
  • EIS electronic image stabilization
  • the EIS algorithm performs posture correction on videos (or images) acquired by the image acquisition devices. For example, after a frame of image is exposed, the EIS algorithm performs low-pass filtering on the spatial attitude data collected by the gyroscope within a period of time before the exposure time to generate a smooth curve (ie, target attitude). Then, the EIS algorithm calculates the posture compensation amount according to the target posture curve, and corrects the video based on the posture compensation amount, so that the user can watch a smoother video picture.
  • the EIS algorithm needs to use a frequency-domain low-pass filter in the process of generating a smooth curve.
  • the frequency-domain low-pass filter generates a second-level delay in the filtering process. If the user rotates during this delay period
  • the real-time posture of the image acquisition equipment and the target posture will have a large deviation, resulting in jitter or twitching of the video picture, which reduces the viewing experience.
  • the embodiment of the present invention provides an electronic stabilization method.
  • the inventive concept is to use the posture data before and after the exposure time of the image to be stabilized to perform low-frequency filtering on the actual posture, so as to obtain a smoother Target posture: Afterwards, the stabilization image is stabilized based on the target posture, so as to ensure the stability of the video image, avoid jitter or twitching, and improve the viewing experience.
  • FIG. 1 is a schematic flowchart of an electronic stabilization method provided by an embodiment of the present invention, which can be applied to image acquisition devices, such as (sports) cameras, cameras, handheld pan-tilts, etc.
  • an electronic stabilization method includes steps 101 to 104, wherein:
  • step 101 a frame of an image to be stabilized and its exposure time are acquired.
  • the image sensor in the image acquisition device can collect original images, and these original images are the images to be stabilized.
  • the image sensor can be a CCD, a camera, or the like.
  • the image sensor will also obtain the time stamp of the image to be stabilized.
  • the time stamp can be the exposure time of the image to be stabilized, or it can be based on the exposure time.
  • the preset algorithm generation is not limited in this application. In the subsequent embodiments, the time stamp is described by taking the exposure time as an example.
  • the processor in the image acquisition device can communicate with the image sensor, so that the processor can obtain the image to be stabilized and its exposure time from the image sensor.
  • a first-in first-out memory FIFO
  • the processor stores the acquired image to be stabilized in the FIFO.
  • the image sensor collects the image to be stabilized and stores it in the FIFO, and then the processor directly reads the image to be stabilized from the FIFO.
  • the processor before the processor acquires a frame of the image to be stabilized, it can first acquire a preset number of frames of the image to be stabilized collected by the image sensor, so as to ensure sufficient delay and facilitate the frequency domain low-pass filter. Filtering.
  • the preset number can be set in advance, for example, to acquire the image to be stabilized within 1 second, if the acquisition frequency of the image sensor is 30fps, the preset number is 30 frames; if the acquisition frequency of the image sensor is 60fps, the preset Let the number be 60 frames.
  • the preset number can also be associated with a frequency-domain low-pass filter preset in the image acquisition device. For example, if the frequency-domain low-pass filter takes a short time to filter the actual posture of the image acquisition device, it can be appropriate Increase the preset data; if the frequency domain low-pass filter takes a long time to filter the actual posture of the image acquisition device, you can appropriately decrease the preset data.
  • the preset number in this embodiment can be adjusted according to specific scenarios.
  • the image buffer to be stabilized can be realized, the corresponding solution falls within the protection scope of this application.
  • the frequency domain low-pass filter includes at least one of the following: FIR filter and IIR filter. It should be noted that technical personnel can select a suitable frequency domain low-pass filter according to a specific scenario, and if the frequency domain low-pass filter can be realized, the corresponding filter falls within the protection scope of the present application.
  • step 102 first posture data before the exposure time and second posture data after the exposure time are acquired; the number of the first posture data is one or more, and the number of the second posture data At least one or more.
  • the spatial attitude sensor in the image acquisition device can periodically collect the attitude data of the image acquisition device at different moments.
  • the spatial attitude sensor may include a three-axis gyroscope, a three-axis accelerometer, a three-axis electronic floppy disk, GPS, etc.
  • the technician can make a selection according to a specific scenario, which is not limited here.
  • the period of the space attitude sensor collecting attitude data can be related to the period of the image sensor collecting images to be stabilized, for example, 300 attitude data are collected within 1 second, and 30 frames of images to be stabilized are collected; of course, the space attitude sensor collects attitude data
  • the period of the image sensor can also be independent of the period of the image sensor to be stabilized. Technicians can make adjustments according to specific scenarios, which are not limited here.
  • the spatial attitude sensor when the spatial attitude sensor collects attitude data, it also generates time stamps of the attitude data, such as the acquisition time, the marker generated by the preset algorithm based on the acquisition time, etc., which is not limited in this application.
  • Time stamp matching means that the time stamps are the same or the time stamp difference is less than the set threshold.
  • the setting threshold can be set according to a specific scenario, for example, 0.01 second, which is not limited here.
  • the processor may obtain the exposure time in step 101, and obtain the first posture data before the exposure time and the second posture data after the exposure time based on the exposure time.
  • the quantity of the first posture data can be one or more
  • the quantity of the second posture data can be one or more.
  • adding second posture data on the basis of the first posture data can increase the time span of the posture data.
  • the posture data further includes third posture data corresponding to the exposure time.
  • the processor may merge the third posture data into the first posture data and the second posture data.
  • the third posture data is used as the last posture data
  • the second posture data the first posture data Three posture data are used as the first posture data.
  • the posture data may not be used. Technicians can make adjustments according to specific scenarios, which are not limited here.
  • the first posture data corresponds to a first time period
  • the second posture data corresponds to a second time period
  • the first time period corresponding to the first posture data refers to the first posture data in the first posture data
  • the time difference corresponding to the time stamp of the last posture data it should be noted that if the first posture data or the second posture data includes only one posture data, it corresponds to a time. In this case, the time can be replaced with a smaller preset value, such as 0.01 second.
  • the image stored in the FIFO in step 101 corresponds to the third time period, and the third time period is less than the sum of the first time period and the second time period, so that the first frame or the last frame of the image to be enhanced can be sufficient
  • the posture data ensures the subsequent stabilization effect.
  • the value range of the first time period and the second time period may include 0.5 second to 1.0 second.
  • the first time period corresponding to the first attitude data and the second time period corresponding to the second attitude data are the same, for example, both are 0.5 seconds.
  • a symmetrical frequency domain low-pass filter can be used to improve the filtering speed.
  • the first time period corresponding to the first posture data and the second time period corresponding to the second posture data are different, and a symmetrical frequency domain low-pass filter may be used to improve the filtering accuracy.
  • the storage address of the posture data can be used as the storage address of the posture data while buffering each frame of the image to be stabilized
  • the characteristic data of the image to be stabilized in the frame so that the processor can read the posture data from the corresponding storage address when reading the image to be stabilized in each frame, thereby improving the reading efficiency.
  • the second posture data of the image to be stabilized in the previous frame and the first posture data of the image to be stabilized in the following frame may not overlap, thereby reducing data calculations. the amount.
  • the second posture data of the image to be stabilized in the previous frame and the first posture data of the image to be stabilized in the following frame overlap, so as to ensure that the target posture obtained subsequently is smoother.
  • step 103 the target posture corresponding to the exposure moment where the image acquisition device is located is acquired according to the first posture data and the second posture data.
  • the processor may obtain a preset frequency domain low-pass filter (corresponding to step 201).
  • the frequency domain low-pass filter may include at least one of the following: FIR filter and IIR filter.
  • the processor can use both the first attitude data and the second attitude data as the input data of the frequency domain low-pass filter, and input them to the frequency domain low-pass filter, and the frequency domain low-pass filter filters the first attitude data And a high-frequency signal greater than the cutoff frequency in the second attitude data to obtain a low-frequency signal that does not exceed the cutoff frequency in the first attitude data and the second attitude data (corresponding to step 202).
  • the frequency domain low pass filter has the function of frequency domain transformation
  • the first attitude data and the second attitude data can be directly input to the frequency domain low pass filter; if the frequency domain low pass filter does not With the function of frequency domain transformation, it is necessary to perform frequency domain transformation on the first attitude data and the second attitude data, and then input the frequency domain transformed first attitude data and second attitude data into the frequency domain low-pass filter.
  • the frequency domain transform method can refer to related technologies, which is not limited here.
  • the value range of the cut-off frequency of the frequency domain low-pass filter is 0.5 Hz to 10 Hz.
  • the cutoff frequency of the frequency domain low-pass filter is 0.5 Hz. It is understandable that the lower the cut-off frequency of the frequency-domain low-pass filter, the stronger its ability to filter out high-frequency signals in the attitude data, and the smoother the target attitude obtained, that is, the slower the motion state of the image acquisition device. The lower the impact on subsequent display of video images.
  • the processor may generate the target posture corresponding to the exposure moment of the image acquisition device based on the low-frequency signal that does not exceed the cutoff frequency in the first posture data and the second posture data (corresponding to step 203).
  • the monotonous movement of the image acquisition device refers to the movement of the image acquisition device in one direction, including uniform speed, acceleration, deceleration, and so on.
  • the monotonic movement refers to the movement of the image acquisition device in one direction, including uniform speed, acceleration, deceleration, and so on.
  • Figure 3 The actual posture of the image acquisition device is shown in Figure 3 (a), where the actual exposure point ET in Figure 3 (a), the exposure time T0, the posture data includes the exposure time The first posture data in the first time period T1 before T0, and the second posture data in the second time period T2 after the exposure time T0.
  • the processor collects the first posture data, where the time for collection, storage, and preprocessing is Delta-t1.
  • the first posture data corresponds to a piece of actual posture IT.
  • the median point EE of the first attitude data is located at T1/2. Since the image to be stabilized at the actual exposure point ET is subsequently stabilized by using the data at the median point EE, there is a delay Delta-t2 between the median point EE and the actual exposure point ET, where Delta-t2 is equal to T1/ 2.
  • This delay is due to the processing result obtained by the low-pass filter, that is, the deviation between the median point EE and the actual exposure point ET, or it can be understood as a filtering error. If the image acquisition device shakes during the delay Delta-t2, the medium The data at the value point EE will be biased when stabilizing the image to be stabilized.
  • the processor collects the first posture data and the second posture data, where the first posture data and the second posture data collection, storage, and preprocessing takes the time Delta-t1, the first posture The data and the second posture data correspond to an actual posture IT.
  • the processor can obtain the target posture IE based on the first posture data and the second posture data.
  • the median point EE of the first attitude data and the second attitude data is located at T0, that is, coincides with the actual exposure point ET, so that (b)
  • the delay Delta-t2 of T1/2 in the figure, that is, the delay Delta-t2 is equal to zero.
  • the image acquisition device does not jitter, so the result of stabilizing the image to be stabilized using the data at the median point EE is more accurate .
  • Fig. 4 shows a schematic diagram of an actual posture and a target posture. See Fig. 4, where the curve designated by number 1 is the actual posture of the image acquisition device, and the curve designated by number 2 is the target posture of the image acquisition device. Taking the two rectangular areas 10 and 20 in Fig. 4 as an example, there is a jitter part 11 in the actual posture 1 in the rectangular area 10. After low-pass filtering in the frequency domain, the area 12 corresponding to the jitter part 11 in the target posture 2 has been smoothed. The results corresponding to the numbers 21 and 22 in the rectangular area 20 are similar, and will not be described here.
  • the gap between the two becomes larger and larger at any time, and the delay between the position of the median point EE and the actual exposure point becomes larger and larger. It is understandable that due to the addition of the second attitude data in the second time period, the delay between the median point EE and the actual exposure point will still be less than the time between the median point EE and the actual exposure point ET shown in the figure (b)
  • the delay, that is, IE in (c) is smoother than IE in (b).
  • the delay between the median point EE and the actual exposure point ET may not exceed the preset delay threshold, that is, the zero frequency of the target attitude
  • the zero frequency delay with the actual posture of the image acquisition device does not exceed the preset delay threshold.
  • the value range of the delay threshold may include 0 to 0.5 seconds, and optionally, the delay threshold may be 0.1 seconds or 0.5 seconds.
  • step 104 the stabilized image is stabilized according to the target posture, and the stabilized target image is obtained.
  • the processor divides the image to be stabilized according to the set mode to obtain multiple sub-images (corresponding to step 501).
  • the setting mode may include at least one of the following: grid division and uniform division.
  • the processor stitches at least one of the multiple sub-images based on the target posture corresponding to the exposure time of the image to be stabilized to obtain a frame of stitched image; the stitched image is the stabilized target image (corresponding to step 502).
  • the size of the target image will be smaller than the size of the image to be stabilized.
  • the target image is a part of the image cut from the image to be stabilized.
  • the target pose may be close to the edge of the image to be stabilized.
  • the target pose needs to be translated appropriately to ensure that the target image does not contain the edge area of the image to be stabilized or the blank area outside. .
  • the processor directly stabilizes the image to be stabilized according to the target posture corresponding to the exposure time of each stabilized image, and the stabilization process can refer to the solution shown in FIG. 5.
  • the processor can obtain the previous frame of the target image. If the target image does not exceed the boundary of the image to be stabilized, the processor directly stabilizes each image to be stabilized; if the boundary of the target image and the image to be stabilized If the boundaries of are coincident, the processor maintains that the boundary of the target image does not exceed the boundary of the image to be stabilized, and stabilizes each stabilized image.
  • the input video can be stabilized, so that after the video is output and displayed, a relatively stable image can be displayed.
  • the exposure time at which the image acquisition device is located can be acquired according to the first posture data and the second posture data.
  • Corresponding target posture; afterwards, the image to be stabilized can be stabilized according to the target posture to obtain a stabilized target image.
  • the second posture data can be used to determine the movement of the image acquisition device after the exposure of the image to be stabilized, so as to ensure that a smooth target posture can be obtained after the actual movement of the image acquisition device is filtered, avoiding post-stabilization
  • the phenomenon of volatility or fluctuations in the image is helpful to improve the stability of the display and the viewing experience.
  • An embodiment of the present invention also provides an image acquisition device, including a processor, an image sensor, and a spatial attitude sensor; the processor is respectively communicatively connected with the image sensor and the spatial attitude sensor; the processor is configured to:
  • the processor before the processor is used to obtain a frame of image to be stabilized, it is also used to:
  • a preset number of frames to be stabilized images collected by the image sensor in the image acquisition device are acquired; the to-be-stabilized images include a time stamp.
  • the image to be stabilized is stored in a first-in first-out memory capable of storing a preset number of frames of images.
  • the processor before the processor is used to obtain a frame of image to be stabilized, it is also used to:
  • the posture data collected by the spatial posture sensor in the image acquisition device includes the first posture data and the second posture data, and the third posture data at the time of exposure; the posture data includes Time stamp; there is a posture data matching the time stamp of each image to be stabilized in the posture data.
  • the first time period corresponding to the first posture data is the same as the second time period corresponding to the second posture data; or,
  • the first time period corresponding to the first posture data and the second time period corresponding to the second posture data are different.
  • the value range of the first time period and the second time period includes 0.5 to 1.0 seconds.
  • the image stored in the first-in first-out memory where the image to be stabilized is located corresponds to a third time period, and the third time period is less than the sum of the first time period and the second time period.
  • the processor configured to obtain the target posture corresponding to the exposure moment where the image acquisition device is located according to the first posture data and the second posture data includes:
  • the first posture data and the second posture data are input to a frequency-domain low-pass filter, and the frequency-domain low-pass filter filters out the first posture data and the second posture data.
  • the target posture corresponding to the exposure moment where the image acquisition device is located is generated.
  • the delay between the zero frequency of the target posture and the zero frequency of the actual posture of the image acquisition device does not exceed a preset delay threshold.
  • the value range of the delay threshold includes 0 to 0.5 seconds.
  • the cut-off frequency of the frequency domain low-pass filter has a value range of 0.5 Hz to 10 Hz.
  • the frequency domain low-pass filter includes at least one of the following: FIR filter and IIR filter.
  • the processor is configured to stabilize the image to be stabilized according to the target attitude, and obtaining the stabilized target image includes:
  • the target posture corresponding to the exposure moment of the image to be stabilized as a reference at least one of the plurality of sub-images is spliced to obtain a spliced image; the spliced image is the stabilized target image.
  • the embodiment of the present invention also provides a movable platform, including a body, a power supply battery provided on the body, a power system, a flight controller, and the image acquisition device as described in the above embodiment, the power supply battery can be The power system provides power, and the power system provides flight power for the drone.
  • the embodiment of the present invention also provides a machine-readable storage medium, characterized in that a number of computer instructions are stored on the machine-readable storage medium, and when the computer instructions are executed, the method described in FIGS. 1 to 5 is implemented. step.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Studio Devices (AREA)
  • Adjustment Of Camera Lenses (AREA)
  • Image Processing (AREA)

Abstract

一种电子增稳方法、图像采集设备、可移动平台。一种电子增稳方法，包括：获取一帧待增稳图像及其曝光时刻；获取曝光时刻之前的第一姿态数据和曝光时刻之后的第二姿态数据；第一姿态数据的数量为一个或者多个，第二姿态数据的数量至少为一个或者多个；根据第一姿态数据和第二姿态数据获取图像采集设备所处曝光时刻对应的目标姿态；根据目标姿态增稳待增稳图像，获得增稳后的目标图像。本实施例中利用第二姿态数据可以确定待增稳图像曝光之后的图像采集设备的运动情况，从而保证对图像采集设备的实际运动滤波后可以得到一条平滑的目标姿态，避免增稳后的图像出现波动或的现象，有利于提升显示的稳定度和观看体验。

Description

电子增稳方法、图像采集设备、可移动平台 技术领域
本发明实施例涉及控制技术领域，尤其涉及电子增稳方法、图像采集设备、可移动平台、机器可读存储介质。
背景技术
目前，大部分图像采集设备(例如运动相机)设置有电子增稳算法，由于电子增稳算法(Electric Image Stabilization，EIS)对图像采集设备获取的视频(或者图像)进行姿态矫正。例如，在一帧图像曝光后，EIS算法对曝光时刻之前一段时间内陀螺仪采集的空间姿态数据进行低通滤波，生成一条平滑曲线(即目标姿态)。然后，EIS算法根据目标姿态曲线计算出姿态补偿量，并基于姿态补偿量来对视频进行矫正，从而使用户观看到比较流畅的视频画面。
相关技术中，EIS算法生成平滑曲线的过程中，需要利用频域低通滤波器，然而，频域低通滤波器在滤波过程中会产生秒级别的延时，若用户在这延时期间转动图像采集设备，则图像采集设备的实时姿态和目标姿态会产生较大的偏差，导致视频画面出现抖动或者抽动，降低观看体验。
发明内容
本发明实施例提供一种电子增稳方法、图像采集设备、可移动平台、机器可读存储介质。
第一方面，本发明实施例提供一种电子增稳方法，包括：
获取一帧待增稳图像及其曝光时刻；
获取所述曝光时刻之前的第一姿态数据和所述曝光时刻之后的第二姿态数据；所述第一姿态数据的数量为一个或者多个，所述第二姿态数据的数量至少为一个或者多个；
根据所述第一姿态数据和所述第二姿态数据获取图像采集设备所处所述曝光时刻对应的目标姿态；
根据所述目标姿态增稳所述待增稳图像，获得增稳后的目标图像。
第二方面，本发明实施例提供一种图像采集设备，包括处理器、图像传感器和空间姿态传感器；所述处理器分别与所述图像传感器和所述空间姿态传感器通信连接；所述处理器用于：
获取一帧待增稳图像及其曝光时刻；
获取所述曝光时刻之前的第一姿态数据和所述曝光时刻之后的第二姿态数据；所述第一姿态数据的数量为一个或者多个，所述第二姿态数据的数量至少为一个或者多个；
根据所述第一姿态数据和所述第二姿态数据获取图像采集设备所处所述曝光时刻对应的目标姿态；
根据所述目标姿态增稳所述待增稳图像，获得增稳后的目标图像。
第三方面，本发明实施例提供一种可移动平台，包括机体、设于所述机体上的供电电池、动力系统、飞行控制器和如第二方面所述的图像采集设备，所述供电电池能够为所述动力系统供电，所述动力系统为所述无人机提供飞行动力。
第四方面，本发明实施例提供一种机器可读存储介质，所述机器可读存储介质上存储有若干计算机指令，所述计算机指令被执行时实现第三方面所述方法的步骤。
由上述的技术方案可见，本实施例中通过获取曝光时刻之前的第一姿态数据和所述曝光时刻之后的第二姿态数据，可以利用根据第一姿态数据和第二姿态数据获取图像采集设备所处所述曝光时刻对应的目标姿态；之后，可以根据所述目标姿态增稳所述待增稳图像，获得增稳后的目标图像。这样，本实施例中利用第二姿态数据可以确定待增稳图像曝光之后的图像采集设备的运动情况，从而保证对图像采集设备的实际运动滤波后可以得到一条平滑的目标姿态，避免增稳后的图像出现波动或的现象，有利于提 升显示的稳定度和观看体验。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的一种电子增稳方法的流程示意图；
图2是本发明实施例提供的一种获取目标姿态的流程示意图；
图3是本发明实施例提供的实际姿态和目标姿态的示意图；
图4是本发明实施例提供的一种目标姿态的示意图；
图5是本发明实施例提供的另一种电子增稳方法的流程示意图；
图6是本发明实施例提供的另一种电子增稳方法的流程示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。另外，在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
目前，大部分图像采集设备(例如运动相机)设置有电子增稳算法，由于电子增稳算法(Electric Image Stabilization，EIS)对图像采集设备获取的视频(或者图像)进行姿态矫正。例如，在一帧图像曝光后，EIS算法对曝光时刻之前一段时间内陀螺仪采集的空间姿态数据进行低通滤波，生成一条平滑曲线(即目标姿态)。然后，EIS算法根据目标姿态曲线计算出姿态补偿量，并基于姿态补偿量来对视频进行矫正，从而使用户观看 到比较流畅的视频画面。
相关技术中，EIS算法生成平滑曲线的过程中，需要利用频域低通滤波器，然而，频域低通滤波器在滤波过程中会产生秒级别的延时，若用户在这延时期间转动图像采集设备，则图像采集设备的实时姿态和目标姿态会产生较大的偏差，导致视频画面出现抖动或者抽动，降低观看体验。
为此，本发明实施例提供了一种电子增稳方法，其发明构思在于，利用待增稳图像的曝光时刻之前和之后的姿态数据，对实际姿态进行低频滤波，从而可以得到一条更加平滑的目标姿态；之后基于目标姿态对待增稳图像进行增稳，从而保证视频画面稳定，避免出现抖动或者抽动的现象，提升观看体验。
图1是本发明实施例提供的一种电子增稳方法的流程示意图，可以应用于图像采集设备，例如(运动)相机、摄像头、手持云台等。参见图1，一种电子增稳方法包括步骤101～步骤104，其中：
在步骤101中，获取一帧待增稳图像及其曝光时刻。
本实施例中，图像采集设备中的图像传感器可以采集原始图像，这些原始图像即是待增稳图像。其中，图像传感器可以为CCD、摄像头等。图像传感器在采集各帧待增稳图像的过程中，还会获取该待增稳图像的时间标记，例如，时间标记可以为该帧待增稳图像的曝光时刻，还可以为基于曝光时刻，利用预先设置的算法生成，本申请不作限定。后续实施例中时间标记均以曝光时刻为例进行描述。
本实施例中，图像采集设备中的处理器可以与图像传感器进行通信，这样处理器可以从图像传感器处获取到待增稳图像及其曝光时刻。本实施例中，图像采集设备中可以设置先入先出存储器(FIFO)，处理器将所获取的待增稳图像存储到FIFO。或者说，图像传感器采集待增稳图像后将其存储到FIFO，然后处理器从FIFO直接读取待增稳图像。
本实施例中，在处理器获取一帧待增稳图像之前，可以先获取图像传感器所采集的预设数量帧待增稳图像，从而保证有足够的延时，方便频域 低通滤波器进行滤波。
其中，预设数量可以预先设置，例如获取1秒钟时间内的待增稳图像，若图像传感器的采集频率为30fps，则预设数量为30帧；若图像传感器的采集频率为60fps，则预设数量为60帧。
另外，该预设数量还可以根据图像采集设备中预先设置的频域低通滤波器相关联，例如，若频域低通滤波器对图像采集设备的实际姿态滤波所用时间较短，则可以适当调大预设数据；若频域低通滤波器对图像采集设备的实际姿态滤波所用时间较长，则可以适当调小预设数据。
即，本实施例中预设数量可以根据具体场景进行调整，在能够实现待增稳图像缓存的情况下，相应方案落入本申请的保护范围。
本实施例中，频域低通滤波器包括以下至少一种：FIR滤波器、IIR滤波器。需要说明的是，技术人员可以根据具体场景选择合适的频域低通滤波器，在能够实现频域低通滤波的情况下，相应的滤波器落入本申请的保护范围。
在步骤102中，获取所述曝光时刻之前的第一姿态数据和所述曝光时刻之后的第二姿态数据；所述第一姿态数据的数量为一个或者多个，所述第二姿态数据的数量至少为一个或者多个。
本实施例中，图像采集设备中的空间姿态传感器可以周期性的采集图像采集设备在不同时刻的姿态数据。其中空间姿态传感器可以包括三轴陀螺仪、三轴加速度计、三轴电子软盘、GPS等，技术人员可以根据具体场景进行选择，在此不作限定。
空间姿态传感器采集姿态数据的周期可以和图像传感器采集待增稳图像的周期相关联，例如，1秒钟内采集300个姿态数据，采集30帧待增稳图像；当然，空间姿态传感器采集姿态数据的周期也可以与图像传感器采集待增稳图像的周期无关。技术人员可以根据具体场景进行调整，在此不作限定。
本实施例中，空间姿态传感器在采集姿态数据时，同样生成姿态数据 的时间标记，如采集时间、基于采集时间利用预先设置的算法生成的标记等，本申请不作限定。
可理解的是，姿态数据内存在一个与各帧待增稳图像的时间标记相匹配的姿态数据。时间标记匹配是指，时间标记相同或者时间标记的差值小于设定阈值。其中，设定阈值可以根据具体场景进行设置，例如0.01秒，在此不作限定。
本实施例中，处理器可以获取步骤101中的曝光时刻，基于曝光时刻获取曝光时刻之前的第一姿态数据和曝光时刻之后的第二姿态数据。其中，第一姿态数据的数量可以为一个或者多个，第二姿态数据的数量可以为一个或者多个。与相关技术中仅采用曝光时刻之前的姿态(可理解为本申请第一姿态数据)相比较，本实施例中在第一姿态数据的基础上增加第二姿态数据，可以增加姿态数据的时间跨度，保证图像采集设备在曝光时刻之后的低频运动不会影响到目标姿态。
在一实施例下，姿态数据还包括曝光时刻对应的第三姿态数据。此情况下，处理器可以将第三姿态数据合并到第一姿态数据和第二姿态数据中，在第一姿态数据中，第三姿态数据作为最后一个姿态数据，在第二姿态数据中，第三姿态数据作为第一个姿态数据。当然，在第一姿态数据和第二姿态数据的数量较多时，也可以不采用姿态数据。技术人员可以根据具体场景进行调整，在此不作限定。
在一些实施例中，第一姿态数据对应第一时间段，第二姿态数据对应第二时间段，其中，第一姿态数据对应的第一时间段是指第一姿态数据内第一个姿态数据和最后一个姿态数据对应时间标记的时间差。需要说明的是，若第一姿态数据或者第二姿态数据仅包括一个姿态数据，则对应一个时刻，此情况下可以将时刻采用一个较小的预设值替代，例如0.01秒。
需要说明的是，步骤101中FIFO内所存储图像对应第三时间段，第三时间段小于第一时间段和第二时间段之和，这样可以第一帧或者最后一帧待增强图像对应足够的姿态数据，保证后续的增稳效果。
在一示例中，第一时间段和第二时间段的取值范围可以包括0.5秒～1.0秒。考虑到低通滤波的类型及其运行效率，第一姿态数据对应的第一时间段和第二姿态数据对应的第二时间段相同，例如两者均为0.5秒。此情况下，可以采用对称的频域低通滤波器，从而提升滤波速度。
在另一示例中，第一姿态数据对应的第一时间段和第二姿态数据对应的第二时间段不相同，可以采用对称的频域低通滤波器，从而提升滤波准确率。
在实际应用中，考虑到每帧待增稳图像对应一组姿态数据(即第一姿态数据和第二姿态数据)，因此在缓存每帧待增稳图像的同时可以将姿态数据的存储地址作为该帧待增稳图像的特征数据，这样处理器在读取各帧待增稳图像时可以从相应的存储地址读取姿态数据，提高读取效率。
在又一实施例中，对于相邻两帧待增稳图像，前一帧待增稳图像的第二姿态数据和后一帧待增稳图像的第一姿态数据可以不重叠，从而减少数据计算量。或者，前一帧待增稳图像的第二姿态数据和后一帧待增稳图像的第一姿态数据存在重叠部分，从而保证后续所得的目标姿态更加平滑。
在步骤103中，根据所述第一姿态数据和所述第二姿态数据获取图像采集设备所处所述曝光时刻对应的目标姿态。
本实施例中，参见图2，处理器可以获取预先设置的频域低通滤波器(对应步骤201)。频域低通滤波器可以包括以下至少一种：FIR滤波器、IIR滤波器。
然后，处理器可以将第一姿态数据和第二姿态数据均作为频域低通滤波器的输入数据，并输入到频域低通滤波器，由频域低通滤波器滤除第一姿态数据和第二姿态数据中大于截止频率的高频信号，以获得第一姿态数据和第二姿态数据中不超过截止频率的低频信号(对应步骤202)。可理解的是，若频域低通滤波器具有频域变换的功能，则可以将第一姿态数据和第二姿态数据直接输入到频域低通滤波器；若，频域低通滤波器不具有频域变换的功能，则需要对第一姿态数据和第二姿态数据进行频域变换， 再将频域变换后的第一姿态数据和第二姿态数据输入到频域低通滤波器。其中，频域变换的方法可以参考相关技术，在此不作限定。
其中，频域低通滤波器的截止频率的取值范围为0.5Hz～10Hz。本实施例中，频域低通滤波器的截止频率为0.5Hz。可理解的是，频域低通滤波器的截止频率越低，其滤除姿态数据中高频信号的能力越强，所得到的目标姿态也就越平滑，即图像采集设备的运动状态越缓慢，对后续显示视频图像的影响越低。
之后，处理器可以基于第一姿态数据和第二姿态数据中不超过截止频率的低频信号生成图像采集设备所处曝光时刻对应的目标姿态(对应步骤203)。
以图像采集设备单调运动为例，其中单调运动是指图像采集设备朝向一个方向运动，包括匀速、加速、减速等。以匀速运动为例，参见图3，图像采集设备的实际姿态如图3中(a)图所示，其中图3中(a)图中实际曝光点ET，曝光时刻T0，姿态数据包括曝光时刻T0之前第一时间段T1内的第一姿态数据，以及曝光时刻T0之后第二时间段T2内的第二姿态数据。
参见图3中(b)图，处理器采集第一姿态数据，其中采集、存储和预处理等工作所用时间为Delta-t1，第一姿态数据对应一条实际姿态IT，基于该第一姿态数据可以得到目标姿态IE。在采用频域低通滤波器为中值滤波器的情况下，第一姿态数据的中值点EE位于T1/2处。由于后续对实际曝光点ET处待增稳图像采用中值点EE处的数据增稳，因此在中值点EE和实际曝光点ET之间存在延时Delta-t2，其中Delta-t2等于T1/2。该延时是由于低通滤波器所得到的处理结果即中值点EE与实际曝光点ET的偏差，或者可理解为滤波误差，若图像采集设备在延时Delta-t2期间发生抖动，采用中值点EE处的数据对待增稳图像进行增稳会发生偏差。
参见图3中(c)图，处理器采集第一姿态数据和第二姿态数据，其中第一姿态数据和第二姿态数据采集、存储和预处理等工作所用时间为 Delta-t1，第一姿态数据和第二姿态数据对应一条实际姿态IT，以T1和T2相等为例，处理器基于第一姿态数据和第二姿态数据可以得到目标姿态IE。在采用频域低通滤波器为中值滤波器的情况下，第一姿态数据和第二姿态数据的中值点EE位于T0处，即与实际曝光点ET重合，从而可以避免出现(b)图中所存在的T1/2的延时Delta-t2即延时Delta-t2等于0。这样后续对ET处待增稳图像采用中值点EE处的数据增稳时，图像采集设备未发生抖动，这样采用中值点EE处的数据对待增稳图像进行增稳的结果是比较准确的。
需要说明的是，在第一时间段和第二时间段相同时，中值点EE和实际曝光点的位置重合或者接近，即图像采集设备的实际姿态和目标姿态是重合的。考虑到处理器获取和存储姿态数据需要延时Delta-t1，实际姿态和目标姿态之间的波动可以如图4所示。图4示出了一种实际姿态和目标姿态的示意图，参见图4，其中标号1所指代曲线为图像采集设备的实际姿态，标号2所指代曲线为图像采集设备的目标姿态。以图4中的2个矩形区域10和20为例，矩形区域10内实际姿态1上存在抖动部分11，经过频域低通滤波后，目标姿态2上对应抖动部分11的区域12已经平滑，矩形区域20内标号21和标号22对应的结果类似，在此不再表述。
本实施例中，在第一时间段和第二时间段不同时，随时两者差距越来越大，中值点EE和实际曝光点的位置之间的延时会越来越大。可理解的是，由于增加了第二时间段内的第二姿态数据，中值点EE和实际曝光点的延时仍然会小于(b)图所示中值点EE和实际曝光点ET之间的延时，即(c)图中IE比(b)图中IE要平滑。
可理解的是，为保证电子增稳方法的效率，在一实施例中，中值点EE和实际曝光点ET之间的延时不可以超过预先设置的延时阈值，即目标姿态的零频率与图像采集设备的实际姿态的零频率的延时不超过预先设置的延时阈值。其中，延时阈值的取值范围可以包括0～0.5秒，可选地，延时阈值可以为0.1秒或0.5秒。
在步骤104中，根据所述目标姿态增稳所述待增稳图像，获得增稳后的目标图像。
本实施例中，参见图5，处理器按照设定方式分割所述待增稳图像，得到多个子图像(对应步骤501)。其中设定方式可以包括以下至少一种：栅格分割、均匀分割。然后，处理器以待增稳图像的曝光时刻对应的目标姿态为基准，拼接多个子图像中的至少一个，得到一帧拼接图像；拼接图像即是增稳后的目标图像(对应步骤502)。
可理解的是，目标图像的尺寸会小于待增稳图像的尺寸。换言之，目标图像是从待增稳图像剪切一部分图像。
需要说明的是，在拼接过程中，目标姿态可能靠近待增稳图像的边缘，此情况下需要适当平移目标姿态，从而保证目标图像内不包含待增稳图像的边缘区域或者之外的空白区域。
在一示例中，处理器直接根据各增稳图像的曝光时刻对应的目标姿态对待增稳图像进行增稳，增稳过程可参见图5所示方案。
在一示例中，处理器可以获取前一帧目标图像，若目标图像未超过待增稳图像的边界，则处理器直接对各增稳图像进行增稳；若目标图像的边界与待增稳图像的边界重合，则处理器保持目标图像的边界不超过待增稳图像的边界的情况下，对各增稳图像进行增稳。
可理解的是，本实施例中仅介绍了两种增稳图像的示例，技术人员可以根据具体场景选择合适的图像增稳方法，相应方案落入本申请的保护范围。
参见图6，处理器依次对各帧待增稳图像进行增稳后，可以实现对输入视频的增稳，这样在视频输出并显示后，可以显示比较稳定的图像。
可见，本实施例中通过获取曝光时刻之前的第一姿态数据和所述曝光时刻之后的第二姿态数据，可以利用根据第一姿态数据和第二姿态数据获取图像采集设备所处所述曝光时刻对应的目标姿态；之后，可以根据所述目标姿态增稳所述待增稳图像，获得增稳后的目标图像。这样，本实施例 中利用第二姿态数据可以确定待增稳图像曝光之后的图像采集设备的运动情况，从而保证对图像采集设备的实际运动滤波后可以得到一条平滑的目标姿态，避免增稳后的图像出现波动或的现象，有利于提升显示的稳定度和观看体验。
本发明实施例还提供了一种图像采集设备，包括处理器、图像传感器和空间姿态传感器；所述处理器分别与所述图像传感器和所述空间姿态传感器通信连接；所述处理器用于：
获取一帧待增稳图像及其曝光时刻；
获取所述曝光时刻之前的第一姿态数据和所述曝光时刻之后的第二姿态数据；所述第一姿态数据的数量为一个或者多个，所述第二姿态数据的数量至少为一个或者多个；
根据所述第一姿态数据和所述第二姿态数据获取图像采集设备所处所述曝光时刻对应的目标姿态；
根据所述目标姿态增稳所述待增稳图像，获得增稳后的目标图像。
在一实施例中，所述处理器用于获取一帧待增稳图像之前，还用于：
获取图像采集设备中的图像传感器所采集的预设数量帧待增稳图像；所述待增稳图像包括时间标记。
在一实施例中，所述待增稳图像存储到能够存储预设数量帧图像的先入先出存储器。
在一实施例中，所述处理器用于获取一帧待增稳图像之前，还用于：
获取所述图像采集设备中的空间姿态传感器所采集的姿态数据；所述姿态数据包括所述第一姿态数据和所述第二姿态数据，以及曝光时刻的第三姿态数据；所述姿态数据包括时间标记；所述姿态数据中存在一个与各待增稳图像的时间标记相匹配的一个姿态数据。
在一实施例中，所述第一姿态数据对应的第一时间段和所述第二姿态数据对应的第二时间段相同；或者，
所述第一姿态数据对应的第一时间段和所述第二姿态数据对应的第二 时间段不相同。
在一实施例中，所述第一时间段和所述第二时间段的取值范围包括0.5～1.0秒。
在一实施例中，对于相邻两帧待增稳图像，前一帧待增稳图像的第二姿态数据和后一帧待增稳图像的第一姿态数据存在重叠部分。
在一实施例中，所述待增稳图像所在先入先出存储器所存储图像对应第三时间段，所述第三时间段小于所述第一时间段和所述第二时间段之和。
在一实施例中，所述处理器用于根据所述第一姿态数据和所述第二姿态数据获取图像采集设备所处所述曝光时刻对应的目标姿态，包括：
获取预先设置的频域低通滤波器；
将所述第一姿态数据和所述第二姿态数据输入到频域低通滤波器，由所述频域低通滤波器滤除所述第一姿态数据和所述第二姿态数据中大于截止频率的高频信号，以获得所述第一姿态数据和所述第二姿态数据中不超过所述截止频率的低频信号；
基于所述第一姿态数据和所述第二姿态数据中不超过所述截止频率的低频信号生成图像采集设备所处所述曝光时刻对应的目标姿态。
在一实施例中，所述目标姿态的零频率与所述图像采集设备的实际姿态的零频率的延时不超过预先设置的延时阈值。
在一实施例中，所述延时阈值的取值范围包括0～0.5秒。
在一实施例中，所述频域低通滤波器的截止频率的取值范围为0.5Hz～10Hz。
在一实施例中，所述频域低通滤波器包括以下至少一种：FIR滤波器、IIR滤波器。
在一实施例中，所述处理器用于根据所述目标姿态增稳所述待增稳图像，获得增稳后的目标图像包括：
按照设定方式分割所述待增稳图像，得到多个子图像；
以所述待增稳图像的曝光时刻对应的目标姿态为基准，拼接所述多个 子图像中的至少一个，得到一帧拼接图像；所述拼接图像即是增稳后的目标图像。
本发明实施例还提供了一种可移动平台，包括机体、设于所述机体上的供电电池、动力系统、飞行控制器和如上述实施例所述的图像采集设备，所述供电电池能够为所述动力系统供电，所述动力系统为所述无人机提供飞行动力。
本发明实施例还提供了一种机器可读存储介质，其特征在于，所述机器可读存储介质上存储有若干计算机指令，所述计算机指令被执行时实现图1～图5所述方法的步骤。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上对本发明实施例所提供的检测装置和方法进行了详细介绍，本发明中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。

Claims (30)

  1. 一种电子增稳方法，其特征在于，包括：
    获取一帧待增稳图像及其曝光时刻；
    获取所述曝光时刻之前的第一姿态数据和所述曝光时刻之后的第二姿态数据；所述第一姿态数据的数量为一个或者多个，所述第二姿态数据的数量至少为一个或者多个；
    根据所述第一姿态数据和所述第二姿态数据获取图像采集设备所处所述曝光时刻对应的目标姿态；
    根据所述目标姿态增稳所述待增稳图像，获得增稳后的目标图像。
  2. 根据权利要求1所述的电子增稳方法，其特征在于，获取一帧待增稳图像之前，所述方法还包括：
    获取图像采集设备中的图像传感器所采集的预设数量帧待增稳图像；所述待增稳图像包括时间标记。
  3. 根据权利要求2所述的电子增稳方法，其特征在于，所述待增稳图像存储到能够存储预设数量帧图像的先入先出存储器。
  4. 根据权利要求2所述的电子增稳方法，其特征在于，获取一帧待增稳图像之前，所述方法还包括：
    获取所述图像采集设备中的空间姿态传感器所采集的姿态数据；所述姿态数据包括所述第一姿态数据和所述第二姿态数据，或者曝光时刻的第三姿态数据；所述姿态数据包括时间标记；所述姿态数据中存在一个与各待增稳图像的时间标记相匹配的一个姿态数据。
  5. 根据权利要求1所述的电子增稳方法，其特征在于，所述第一姿态数据对应的第一时间段和所述第二姿态数据对应的第二时间段相同；或者，
    所述第一姿态数据对应的第一时间段和所述第二姿态数据对应的第二时间段不相同。
  6. 根据权利要求5所述的电子增稳方法，其特征在于，所述第一时间 段和所述第二时间段的取值范围包括0.5～1.0秒。
  7. 根据权利要求5所述的电子增稳方法，其特征在于，对于相邻两帧待增稳图像，前一帧待增稳图像的第二姿态数据和后一帧待增稳图像的第一姿态数据存在重叠部分。
  8. 根据权利要求5所述的电子增稳方法，其特征在于，所述待增稳图像所在先入先出存储器所存储图像对应第三时间段，所述第三时间段小于所述第一时间段和所述第二时间段之和。
  9. 根据权利要求1所述的电子增稳方法，其特征在于，根据所述第一姿态数据和所述第二姿态数据获取图像采集设备所处所述曝光时刻对应的目标姿态包括：
    获取预先设置的频域低通滤波器；
    将所述第一姿态数据和所述第二姿态数据输入到频域低通滤波器，由所述频域低通滤波器滤除所述第一姿态数据和所述第二姿态数据中大于截止频率的高频信号，以获得所述第一姿态数据和所述第二姿态数据中不超过所述截止频率的低频信号；
    基于所述第一姿态数据和所述第二姿态数据中不超过所述截止频率的低频信号生成图像采集设备所处所述曝光时刻对应的目标姿态。
  10. 根据权利要求9所述的电子增稳方法，其特征在于，所述目标姿态的零频率与所述图像采集设备的实际姿态的零频率的延时不超过预先设置的延时阈值。
  11. 根据权利要求10所述的电子增稳方法，其特征在于，所述延时阈值的取值范围包括0～0.5秒。
  12. 根据权利要求9所述的电子增稳方法，其特征在于，所述频域低通滤波器的截止频率的取值范围为0.5Hz～10Hz。
  13. 根据权利要求9所述的电子增稳方法，其特征在于，所述频域低通滤波器包括以下至少一种：FIR滤波器、IIR滤波器。
  14. 根据权利要求1所述的电子增稳方法，其特征在于，根据所述目 标姿态增稳所述待增稳图像，获得增稳后的目标图像包括：
    按照设定方式分割所述待增稳图像，得到多个子图像；
    以所述待增稳图像的曝光时刻对应的目标姿态为基准，拼接所述多个子图像中的至少一个，得到一帧拼接图像；所述拼接图像即是增稳后的目标图像。
  15. 一种图像采集设备，其特征在于，包括处理器、图像传感器和空间姿态传感器；所述处理器分别与所述图像传感器和所述空间姿态传感器通信连接；所述处理器用于：
    获取一帧待增稳图像及其曝光时刻；
    获取所述曝光时刻之前的第一姿态数据和所述曝光时刻之后的第二姿态数据；所述第一姿态数据的数量为一个或者多个，所述第二姿态数据的数量至少为一个或者多个；
    根据所述第一姿态数据和所述第二姿态数据获取图像采集设备所处所述曝光时刻对应的目标姿态；
    根据所述目标姿态增稳所述待增稳图像，获得增稳后的目标图像。
  16. 根据权利要求15所述的图像采集设备，其特征在于，所述处理器用于获取一帧待增稳图像之前，还用于：
    获取图像采集设备中的图像传感器所采集的预设数量帧待增稳图像；所述待增稳图像包括时间标记。
  17. 根据权利要求16所述的图像采集设备，其特征在于，所述待增稳图像存储到能够存储预设数量帧图像的先入先出存储器。
  18. 根据权利要求16所述的图像采集设备，其特征在于，所述处理器用于获取一帧待增稳图像之前，还用于：
    获取所述图像采集设备中的空间姿态传感器所采集的姿态数据；所述姿态数据包括所述第一姿态数据和所述第二姿态数据，以及曝光时刻的第三姿态数据；所述姿态数据包括时间标记；所述姿态数据中存在一个与各待增稳图像的时间标记相匹配的一个姿态数据。
  19. 根据权利要求15所述的图像采集设备，其特征在于，所述第一姿态数据对应的第一时间段和所述第二姿态数据对应的第二时间段相同；或者，
    所述第一姿态数据对应的第一时间段和所述第二姿态数据对应的第二时间段不相同。
  20. 根据权利要求19所述的图像采集设备，其特征在于，所述第一时间段和所述第二时间段的取值范围包括0.5～1.0秒。
  21. 根据权利要求19所述的图像采集设备，其特征在于，对于相邻两帧待增稳图像，前一帧待增稳图像的第二姿态数据和后一帧待增稳图像的第一姿态数据存在重叠部分。
  22. 根据权利要求19所述的图像采集设备，其特征在于，所述待增稳图像所在先入先出存储器所存储图像对应第三时间段，所述第三时间段小于所述第一时间段和所述第二时间段之和。
  23. 根据权利要求15所述的图像采集设备，其特征在于，所述处理器用于根据所述第一姿态数据和所述第二姿态数据获取图像采集设备所处所述曝光时刻对应的目标姿态，包括：
    获取预先设置的频域低通滤波器；
    将所述第一姿态数据和所述第二姿态数据输入到频域低通滤波器，由所述频域低通滤波器滤除所述第一姿态数据和所述第二姿态数据中大于截止频率的高频信号，以获得所述第一姿态数据和所述第二姿态数据中不超过所述截止频率的低频信号；
    基于所述第一姿态数据和所述第二姿态数据中不超过所述截止频率的低频信号生成图像采集设备所处所述曝光时刻对应的目标姿态。
  24. 根据权利要求23所述的图像采集设备，其特征在于，所述目标姿态的零频率与所述图像采集设备的实际姿态的零频率的延时不超过预先设置的延时阈值。
  25. 根据权利要求24所述的图像采集设备，其特征在于，所述延时阈 值的取值范围包括0～0.5秒。
  26. 根据权利要求23所述的图像采集设备，其特征在于，所述频域低通滤波器的截止频率的取值范围为0.5Hz～10Hz。
  27. 根据权利要求24所述的电子增稳方法，其特征在于，所述频域低通滤波器包括以下至少一种：FIR滤波器、IIR滤波器。
  28. 根据权利要求15所述的图像采集设备，其特征在于，所述处理器用于根据所述目标姿态增稳所述待增稳图像，获得增稳后的目标图像包括：
    按照设定方式分割所述待增稳图像，得到多个子图像；
    以所述待增稳图像的曝光时刻对应的目标姿态为基准，拼接所述多个子图像中的至少一个，得到一帧拼接图像；所述拼接图像即是增稳后的目标图像。
  29. 一种可移动平台，其特征在于，包括机体、设于所述机体上的供电电池、动力系统、飞行控制器和如权利要求15～28任一项所述的图像采集设备，所述供电电池能够为所述动力系统供电，所述动力系统为所述无人机提供飞行动力。
  30. 一种机器可读存储介质，其特征在于，所述机器可读存储介质上存储有若干计算机指令，所述计算机指令被执行时实现权利要求1～14任一项所述方法的步骤。
PCT/CN2019/085265 2019-04-30 2019-04-30 电子增稳方法、图像采集设备、可移动平台 WO2020220292A1 (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
PCT/CN2019/085265 WO2020220292A1 (zh) 2019-04-30 2019-04-30 电子增稳方法、图像采集设备、可移动平台
JP2019544054A JP2021525009A (ja) 2019-04-30 2019-04-30 電子式手ぶれ補正方法、画像取得装置及び移動可能なプラットフォーム
CN201980001010.8A CN110235431B (zh) 2019-04-30 2019-04-30 电子增稳方法、图像采集设备、可移动平台
US16/538,946 US10674087B1 (en) 2019-04-30 2019-08-13 Electronic stabilization method, image acquisition device, and movable platform
US16/862,742 US11025823B2 (en) 2019-04-30 2020-04-30 Electronic stabilization method, image acquisition device, and movable platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/085265 WO2020220292A1 (zh) 2019-04-30 2019-04-30 电子增稳方法、图像采集设备、可移动平台

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/538,946 Continuation US10674087B1 (en) 2019-04-30 2019-08-13 Electronic stabilization method, image acquisition device, and movable platform

Publications (1)

Publication Number Publication Date
WO2020220292A1 true WO2020220292A1 (zh) 2020-11-05

Family

ID=67855286

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/085265 WO2020220292A1 (zh) 2019-04-30 2019-04-30 电子增稳方法、图像采集设备、可移动平台

Country Status (4)

Country Link
US (2) US10674087B1 (zh)
JP (1) JP2021525009A (zh)
CN (1) CN110235431B (zh)
WO (1) WO2020220292A1 (zh)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110235431B (zh) * 2019-04-30 2021-08-24 深圳市大疆创新科技有限公司 电子增稳方法、图像采集设备、可移动平台
CN112204956B (zh) * 2019-10-10 2023-01-10 深圳市大疆创新科技有限公司 拍摄装置及其控制方法、可移动平台
WO2021114031A1 (zh) * 2019-12-09 2021-06-17 深圳市大疆创新科技有限公司 一种目标检测方法和装置
WO2021138768A1 (zh) * 2020-01-06 2021-07-15 深圳市大疆创新科技有限公司 图像处理方法、设备、可移动平台、成像装置及存储介质
CN113966608A (zh) * 2020-05-19 2022-01-21 深圳市大疆创新科技有限公司 图像增稳方法、变焦相机、云台相机与无人机
CN114390186B (zh) * 2020-10-19 2023-05-19 华为技术有限公司 视频拍摄方法及电子设备
WO2022178703A1 (zh) * 2021-02-24 2022-09-01 深圳市大疆创新科技有限公司 电子增稳方法及装置、可移动平台、成像装置
CN116095487A (zh) * 2021-11-05 2023-05-09 Oppo广东移动通信有限公司 图像防抖方法、装置、电子设备及计算机可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6222882B1 (en) * 1994-01-31 2001-04-24 Samsung Electronics Co., Ltd. Adaptive motion estimation method of a luminance signal
CN104796596A (zh) * 2014-01-20 2015-07-22 联想(北京)有限公司 一种信息处理方法及电子设备
CN106060249A (zh) * 2016-05-19 2016-10-26 维沃移动通信有限公司 一种拍照防抖方法及移动终端
CN108600622A (zh) * 2018-04-12 2018-09-28 联想(北京)有限公司 一种视频防抖的方法及装置

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080100716A1 (en) * 2006-11-01 2008-05-01 Guoyi Fu Estimating A Point Spread Function Of A Blurred Digital Image Using Gyro Data
CN101489035A (zh) * 2008-01-16 2009-07-22 三洋电机株式会社 摄像装置及抖动校正方法
US9071756B2 (en) * 2012-12-11 2015-06-30 Facebook, Inc. Systems and methods for digital video stabilization via constraint-based rotation smoothing
JP6091255B2 (ja) * 2013-02-28 2017-03-08 オリンパス株式会社 ブレ量検出装置及び撮像装置
JP6017381B2 (ja) * 2013-07-22 2016-11-02 オリンパス株式会社 像ブレ補正装置及び撮像装置
JP6448218B2 (ja) * 2014-05-12 2019-01-09 キヤノン株式会社 撮像装置、その制御方法および情報処理システム
FR3027143B1 (fr) * 2014-10-10 2016-11-11 Parrot Appareil mobile, notamment drone a voilure tournante, muni d'une camera video delivrant des sequences d'images corrigees dynamiquement de l'effet "wobble"
EP3274256B1 (en) * 2015-03-27 2024-09-11 AeroVironment, Inc. Unmanned aircraft navigation system and method
CN105611179B (zh) * 2016-03-28 2018-09-04 广东欧珀移动通信有限公司 一种用于手持拍照防抖的多帧优选方法、装置及移动终端
US9639935B1 (en) * 2016-05-25 2017-05-02 Gopro, Inc. Apparatus and methods for camera alignment model calibration
CN106375669B (zh) * 2016-09-30 2019-08-06 天津远度科技有限公司 一种稳像方法、装置和无人机
US20180278823A1 (en) * 2017-03-23 2018-09-27 Intel Corporation Auto-exposure technologies using odometry
CN107770437B (zh) * 2017-09-08 2020-03-17 温州大学 无人机摄影摄像系统及其位移补偿机构
CN109379536B (zh) * 2018-12-29 2020-07-31 深圳看到科技有限公司 画面生成方法、装置、终端及对应的存储介质
CN110235431B (zh) * 2019-04-30 2021-08-24 深圳市大疆创新科技有限公司 电子增稳方法、图像采集设备、可移动平台

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6222882B1 (en) * 1994-01-31 2001-04-24 Samsung Electronics Co., Ltd. Adaptive motion estimation method of a luminance signal
CN104796596A (zh) * 2014-01-20 2015-07-22 联想(北京)有限公司 一种信息处理方法及电子设备
CN106060249A (zh) * 2016-05-19 2016-10-26 维沃移动通信有限公司 一种拍照防抖方法及移动终端
CN108600622A (zh) * 2018-04-12 2018-09-28 联想(北京)有限公司 一种视频防抖的方法及装置

Also Published As

Publication number Publication date
US20200351443A1 (en) 2020-11-05
CN110235431B (zh) 2021-08-24
JP2021525009A (ja) 2021-09-16
CN110235431A (zh) 2019-09-13
US11025823B2 (en) 2021-06-01
US10674087B1 (en) 2020-06-02

Similar Documents

Publication Publication Date Title
WO2020220292A1 (zh) 电子增稳方法、图像采集设备、可移动平台
CN112740652B (zh) 用于稳定视频的系统和方法
JP6219400B2 (ja) キャプチャされたビデオの向きを調節するためのシステムおよび方法
WO2018095262A1 (zh) 视频稳像方法及装置
CN114788260A (zh) 多相机视频稳定
KR20180032529A (ko) 카메라 모듈, 고체 촬상 소자, 전자 기기 및 촬상 방법
Bell et al. A non-linear filter for gyroscope-based video stabilization
WO2018053400A1 (en) Improved video stabilization for mobile devices
WO2018030166A1 (ja) 像ブレ補正装置、光学機器、および像ブレ補正方法
CN110049246A (zh) 电子设备的视频防抖方法、装置和电子设备
WO2019000664A1 (zh) 一种信息处理方法及电子设备
CN112204946A (zh) 数据处理方法、装置、可移动平台及计算机可读存储介质
US9049374B2 (en) Imaging apparatus with camera shake correction function
US20220174217A1 (en) Image processing method and device, electronic device, and computer-readable storage medium
US11688409B2 (en) Multi-band noise gate
TW201536050A (zh) 可修正偏移視角的影像擷取方法、電腦程式產品暨其影像擷取裝置
WO2021138768A1 (zh) 图像处理方法、设备、可移动平台、成像装置及存储介质
CN115604583A (zh) 图像处理方法、图像处理器、电子设备及存储介质
WO2022147703A1 (zh) 跟焦方法、装置、拍摄设备及计算机可读存储介质
CN112204956B (zh) 拍摄装置及其控制方法、可移动平台
CN115209055A (zh) 拍摄方法、装置、电子设备及可读存储介质
JP6230316B2 (ja) 像振れ補正装置及びその制御方法、並びにプログラム
JP2015061200A (ja) 画像表示装置、画像表示方法、および画像表示プログラム
CN118212137A (zh) 一种视频处理方法、装置、设备和存储介质
KR20170009654A (ko) 영상에디팅을 수행하는 영상촬영장치 및 방법

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2019544054

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19927178

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19927178

Country of ref document: EP

Kind code of ref document: A1