WO2022027876A1 - 增稳处理模组、无人机摄像系统以及图像增稳处理方法 - Google Patents

增稳处理模组、无人机摄像系统以及图像增稳处理方法 Download PDF

Info

Publication number
WO2022027876A1
WO2022027876A1 PCT/CN2020/132014 CN2020132014W WO2022027876A1 WO 2022027876 A1 WO2022027876 A1 WO 2022027876A1 CN 2020132014 W CN2020132014 W CN 2020132014W WO 2022027876 A1 WO2022027876 A1 WO 2022027876A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
camera system
motion
lens
unit
Prior art date
Application number
PCT/CN2020/132014
Other languages
English (en)
French (fr)
Inventor
柳云龙
Original Assignee
苏州臻迪智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州臻迪智能科技有限公司 filed Critical 苏州臻迪智能科技有限公司
Publication of WO2022027876A1 publication Critical patent/WO2022027876A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6812—Motion detection based on additional sensors, e.g. acceleration sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/683—Vibration or motion blur correction performed by a processor, e.g. controlling the readout of an image memory
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/685—Vibration or motion blur correction performed by mechanical compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof

Definitions

  • the present invention generally relates to the technical field of image processing, and in particular, to a stabilization processing module for processing low-frequency and high-amplitude jitter, a UAV camera system including the module, and an image stabilization processing method.
  • the stabilization processing of the camera system mounted on the UAV usually relies on the gimbal, but the gimbal equipment not only occupies the space of the UAV, but also increases the manufacturing cost.
  • the inventor hopes to provide an image processing solution that relies on software calculation and processing for stabilization, which can replace the gimbal to achieve the same purpose, thereby saving the space of the drone and saving the manufacturing cost.
  • the software stabilization technology in the existing sports DV is usually only aimed at small high-frequency movements, and has no stabilization effect on large-scale, low-frequency movements, such as skiing, cycling, and such sharp turning movements. Therefore, when a sports DV currently on the market shoots a large-scale and low-frequency moving picture, the image will have serious tilt and shake, resulting in poor user experience.
  • the present invention provides a stabilization processing module that can be used in a UAV camera system, including:
  • a motion sensing unit configured to sense motion of the camera system
  • a motion estimation unit coupled to the motion sensing unit, configured to calculate the rotation angle of the image according to the output of the motion sensing unit;
  • An image processing unit coupled to the motion estimation unit, is configured to perform reverse compensation on the image output by the camera system according to the rotation angle of the image.
  • the motion sensing unit includes an inertial measurement unit
  • the inertial measurement unit is mounted on the camera board of the camera system and is perpendicular or parallel to the optical axis direction of the lens of the camera system
  • the inertial measurement unit is The measuring unit can measure the angular velocity and acceleration of the lens in real time.
  • the motion estimation unit is configured to calculate the roll angle and pitch angle of the lens through a complementary filtering algorithm.
  • the motion estimation unit is configured to calculate the motion frequency of the lens through a complementary filtering algorithm, and when the motion frequency is lower than a preset value, process the output image by the image processing unit ; When the motion frequency is higher than the preset value, perform anti-shake processing on the camera lens.
  • the image processing unit is configured to crop the image to remove black borders after performing inverse compensation on the image.
  • the present invention also provides an unmanned aerial vehicle camera system, comprising:
  • a camera configured to capture images of the surroundings
  • the above-mentioned stabilization processing module is configured to receive and process the image captured by the camera
  • a display module coupled to the stabilization processing module, is configured to display the image processed by the image processing unit.
  • the camera includes a fisheye lens, and the shooting resolution is 3840*2160.
  • the present invention also provides a method for image stabilization processing using the stabilization processing module described above, comprising:
  • the image output by the camera system is reversely compensated according to the rotation angle of the image by the image processing unit.
  • the motion sensing unit includes an inertial measurement unit
  • the inertial measurement unit is mounted on the camera board of the camera system and is perpendicular or parallel to the optical axis direction of the lens of the camera system
  • the method also includes:
  • the angular velocity and acceleration of the lens are measured in real time by the inertial measurement unit.
  • the method further includes:
  • the motion estimation unit calculates the roll angle and the pitch angle of the lens through a complementary filtering algorithm.
  • the method further includes:
  • the motion estimation unit calculates the motion frequency of the lens through a complementary filtering algorithm, and when the motion frequency is lower than a preset value, the image processing unit processes the output image; when the motion frequency is higher than the When the preset value is set, anti-shake processing is performed on the camera lens.
  • Preferred embodiments of the present invention provide a stabilization processing module that can be used in a drone camera system, a drone camera system including the stabilization processing module, and a method for image stabilization.
  • the stabilization processing module senses the motion of the camera system through the motion sensing unit, calculates the rotation angle of the image through the motion estimation unit, and performs reverse compensation on the image output by the camera system through the image processing unit.
  • the stabilization processing module can improve the quality of the moving pictures captured by the UAV camera system. It not only adjusts the small high-frequency jitter, but also avoids the impact of the large low-frequency jitter on the output picture; it is installed on the UAV camera system. , small size and light weight, replacing the gimbal equipment, not only greatly saves the space on the drone, but also reduces the manufacturing cost of the drone.
  • FIG. 1 schematically shows a stabilization processing module according to a preferred embodiment of the present invention
  • FIG. 2 schematically shows that the stabilization processing module performs reverse compensation on the output image according to a preferred embodiment of the present invention
  • FIG. 3 schematically shows that the stabilization processing module performs reverse compensation on the output image according to a preferred embodiment of the present invention
  • FIG. 4 schematically shows a drone camera system according to a preferred embodiment of the present invention
  • Figure 5A schematically shows an output image of a camera system lens according to a preferred embodiment of the present invention
  • FIG. 5B schematically shows that the stabilization processing module performs reverse compensation on the output image according to a preferred embodiment of the present invention
  • FIG. 5C schematically shows the cropping of the output image by the stabilization processing module according to a preferred embodiment of the present invention
  • FIG. 6 schematically shows a drone camera system according to a preferred embodiment of the present invention
  • FIG. 7 shows a method for image stabilization processing according to a preferred embodiment of the present invention.
  • connection should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection Connection, or integral connection: it can be a mechanical connection, an electrical connection or can communicate with each other; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal communication of two elements or the interaction of two elements relation.
  • connection should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection Connection, or integral connection: it can be a mechanical connection, an electrical connection or can communicate with each other; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal communication of two elements or the interaction of two elements relation.
  • a first feature "on” or “under” a second feature may include direct contact between the first and second features, or may include the first and second features Not directly but through additional features between them.
  • the first feature being “above”, “over” and “above” the second feature includes that the first feature is directly above and diagonally above the second feature, or simply means that the first feature is level higher than the second feature.
  • the first feature “below”, “below” and “beneath” the second feature includes the first feature being directly above and obliquely above the second feature, or simply means that the first feature has a lower level than the second feature.
  • the software stabilization technology in the original sports DV is usually only aimed at small high-frequency motions, and has no stabilization effect on large-scale and low-frequency motions, such as skiing, cycling, and sharp turning motions.
  • the present invention provides a Image processing module with horizontal stabilization function.
  • the present invention provides a stabilization processing module 10 that can be used in a UAV camera system, including: a motion sensing unit 11 , a motion estimation unit 12 and an image processing unit unit 13.
  • the motion sensing unit 11 is configured to sense the motion of the camera system. For example, the drone with the camera system rotates, dives, climbs, jumps at high altitude, or encounters bumps when the weather changes, etc., which may cause the camera system It accelerates the movement in different directions or swings around the axis, resulting in the jitter of the shooting picture.
  • the motion estimation unit 12 is coupled to the motion sensing unit 11, and is configured to calculate the rotation angle of the image according to the output of the motion sensing unit 11, and the rotation angle of the image refers to the real-time rotation angle of the image captured by the camera system mounted on the drone , in the case where the camera system has no angular velocity and/or angular acceleration, the real-time captured image is the displayed image. When there is image shake, the captured image needs to be stabilized first, and then the processed image is displayed. .
  • the image processing unit 13 is coupled to the motion estimation unit 12, and is configured to perform reverse compensation on the image output by the camera system according to the rotation angle of the image. According to a preferred embodiment of the present invention, the image output by the camera system is performed in the following manner: Reverse compensation.
  • the output image is reversely adjusted along the vertical direction.
  • the picture captured by the camera system is a spherical picture perpendicular to the optical axis of the lens of the camera system, and reverse adjustment along the vertical direction refers to reverse adjustment along the pitch and elevation directions of the camera system lens on the spherical picture.
  • the UAV camera system adopts a fisheye lens, which can shoot a large wide-angle spherical picture, which is convenient for subsequent picture adjustment according to the angle change of the camera system. In this way, the field of view angle retained after the stabilization processing is turned on will be large enough.
  • the motion sensing unit 11 includes an inertial measurement unit 110, which is a device for measuring the three-axis attitude angle (or angular rate) and acceleration of an object.
  • the usual inertial measurement unit contains three single-axis accelerometers and three single-axis gyroscopes.
  • the three single-axis accelerometers detect the independent three-axis acceleration signals of the object in the carrier coordinate system, while the three single-axis gyroscopes detect The angular velocity signal of the carrier relative to the navigation coordinate system measures the angular velocity and acceleration of the object in the three-dimensional space, and then calculates the attitude of the object. As shown in FIG.
  • the inertial measurement unit 110 is installed on the camera board of the camera system, and is perpendicular or parallel to the optical axis direction of the lens of the camera system (as shown by the x-axis in FIG. 2 ), and the inertial measurement unit 110 can measure in real time The angular velocity and acceleration of the camera system lens.
  • the motion estimation unit 12 is configured to calculate the roll angle and the pitch angle of the lens of the camera system through a complementary filtering algorithm.
  • the complementary filtering algorithm refers to using the angle measured by the gyroscope as the optimum in a short period of time, and periodically averaging the angles sampled by the accelerometer to correct the angle measured by the gyroscope. For a long time, the angle measured by the accelerometer is preferred, and the measurement proportion of the accelerometer is increased. Then, the high-frequency signal of the accelerometer is suppressed by low-pass filtering, and the low-frequency signal of the gyroscope is suppressed by high-pass filtering, and the signals of the entire frequency band are obtained by summing up.
  • the motion estimation unit 12 is configured to calculate the motion frequency of the lens of the camera system through a complementary filtering algorithm, and when the motion frequency is lower than a preset value, the image processing unit 13 processes the output image;
  • the motion frequency is higher than the preset value, anti-shake processing is performed on the camera lens. That is, when the motion frequency of the lens is lower than the preset value, it means that the UAV camera system is in low-frequency motion, and the output image is processed by the stabilization processing module 10 provided by the present invention; when the motion frequency of the lens is high
  • the preset value it means that the UAV camera system is in high-frequency motion, and the output image can be adjusted through the built-in anti-shake processing device in the camera system.
  • the image processing unit 13 is configured to crop the image to remove black borders after performing reverse compensation on the output image.
  • 5A shows an image 51 captured in real time by the lens of the camera system, and the captured image is shaken due to the vibration of the drone, so the image 51 deviates from the normal horizontal orientation.
  • the image processing unit generates an image 52 after performing reverse compensation on the output image 51, and keeps the image 52 in a horizontal orientation, which is convenient for the user to view.
  • the image processing unit cuts and removes black borders to obtain an image 53 , and the resolution of the image 53 is smaller than that of the image 52 .
  • the image processing unit After the image processing unit performs reverse compensation on the output image, the image will have black borders, so it needs to be cropped and then displayed to make the displayed image more beautiful. Of course, doing so will reduce a certain resolution and lose part of the field of view. Set the resolution of the camera system lens to 3840*2160p, and after the stabilization process is turned on, the calibrated image resolution will be reduced to 1080p, which can still achieve good results.
  • FIG. 1 shows that the motion sensing unit 11 , the motion estimation unit 12 and the image processing unit 13 are all arranged on the drone. It is also possible to separate the motion sensing unit 11 , the motion estimation unit 12 and the image processing unit 13 It is installed on the drone body and the controller of the drone.
  • the motion sensing unit 11 and the motion estimation unit 12 may be provided on the drone, and the image processing unit 13 may be provided on the remote controller of the drone, for example, integrated in an APP for controlling the drone.
  • the drone body transmits the real-time collected image and the image rotation angle calculated by the motion estimation unit 12 to the remote controller of the drone, and the image processing unit performs corresponding image processing according to the above-mentioned method according to the image and the image rotation angle. process and display the processed image on the remote control.
  • the motion estimation unit 12 may also be provided on the remote controller of the drone.
  • the present invention further provides a drone camera system 20 , comprising: a camera 21 , the above-mentioned stabilization processing module 10 and a display module 22 .
  • the camera 21 is configured to capture surrounding images.
  • the stabilization processing module 10 described above is configured to receive and process images captured by the camera 21 .
  • the display module 22 is coupled to the stabilization processing module 10 and is configured to display the image processed by the image processing unit 13 .
  • the camera 21 of the drone camera system 20 includes a fisheye lens, and the shooting resolution is 3840*2160.
  • the present invention further provides a method 30 for image stabilization processing using the stabilization processing module 10 as described above, including:
  • step S301 the motion of the camera system is sensed in real time by the motion sensing unit 11 .
  • the motion sensing unit 11 For example, when a drone equipped with a camera system rotates, dives, climbs, jumps at a high altitude, or encounters bumps when the weather changes, it may cause the camera system to accelerate at different angles, resulting in jitters in the captured images.
  • step S302 the rotation angle of the image is calculated by the motion estimation unit 12 according to the output of the motion sensing unit 11 .
  • the rotation angle of the image refers to the real-time rotation angle of the image captured by the camera system mounted on the drone. If the camera system has no angular velocity and/or angular acceleration, the image captured in real time is the displayed image. In the case of , it is necessary to perform stabilization processing on the captured image first, and then display the processed image.
  • step S303 reverse compensation is performed on the image output by the camera system through the image processing unit 13 according to the rotation angle of the image.
  • the motion sensing unit 11 includes an inertial measurement unit 110, and the inertial measurement unit 110 is mounted on the camera board of the camera system and is perpendicular or parallel to the optical axis direction of the lens of the camera system (as shown in FIG. 4), the image stabilization processing method 30 further includes:
  • the angular velocity and acceleration of the lens of the camera system are measured in real time by the inertial measurement unit 110 .
  • the method 30 for image stabilization processing further includes:
  • the motion estimation unit 12 calculates the roll angle and pitch angle of the lens of the camera system through the complementary filtering algorithm.
  • the method 30 for image stabilization processing further includes:
  • the motion estimation unit 12 calculates the motion frequency of the lens of the camera system through the complementary filtering algorithm.
  • the output image is processed by the image processing unit 13; when the motion frequency is higher than the preset value, the output image is processed.
  • the camera lens is subjected to anti-shake processing. That is, when the motion frequency of the lens is lower than the preset value, it means that the UAV camera system is in low-frequency motion, and the output image is processed by the stabilization processing module 10 provided by the present invention; when the motion frequency of the lens is high
  • the preset value it means that the UAV camera system is in high-frequency motion, and the output image can be adjusted through the built-in anti-shake processing device in the camera system.
  • Preferred embodiments of the present invention provide a stabilization processing module that can be used in a drone camera system, a drone camera system including the stabilization processing module, and a method for image stabilization.
  • the stabilization processing module senses the motion of the camera system through the motion sensing unit, calculates the rotation angle of the image through the motion estimation unit, and performs reverse compensation on the image output by the camera system through the image processing unit.
  • the stabilization processing module can improve the quality of the moving pictures captured by the UAV camera system. It not only adjusts the small high-frequency jitter, but also avoids the impact of the large low-frequency jitter on the output picture; it is installed on the UAV camera system. , small size and light weight, replacing the gimbal equipment, not only greatly saves the space on the drone, but also reduces the manufacturing cost of the drone.

Abstract

本发明提供了一种可用于无人机摄像系统的增稳处理模组，包括：运动感测单元，配置成可感测摄像系统的运动；运动估计单元，与所述运动感测单元耦接，配置成可根据所述运动感测单元的输出计算图像的旋转角度；图像处理单元，与所述运动估计单元耦接，配置成可根据所述图像的旋转角度对所述摄像系统输出的图像进行反向补偿。

Description

增稳处理模组、无人机摄像系统以及图像增稳处理方法 技术领域
本发明大致涉及图像处理技术领域，尤其涉及一种处理低频高幅抖动的增稳处理模组、包括该模组的无人机摄像系统以及图像增稳处理的方法。
背景技术
现有技术中，无人机搭载的摄像系统增稳处理通常是依靠云台，但云台设备不仅占用无人机的空间还会增加制造成本。发明人希望提供一种依靠软件的计算和处理进行增稳的图像处理方案，替代云台达到相同的目的，从而节约了无人机的空间，节省了制造成本。
现有的运动DV中的软件增稳技术通常只是针对小幅高频的运动，对大幅、低频的运动，例如：滑雪、骑行这种大幅转弯的运动没有增稳的效果。因此，目前市面上的运动DV在拍摄大幅、低频的运动画面时，图像会产生较为严重的倾斜和抖动，造成了用户的体验不佳。
背景技术部分的内容仅仅是公开人所知晓的技术，并不当然代表本领域的现有技术。
发明内容
有鉴于现有技术的至少一个缺陷，本发明提供一种可用于无人机摄像系统的增稳处理模组，包括：
运动感测单元，配置成可感测摄像系统的运动；
运动估计单元，与所述运动感测单元耦接，配置成可根据所述运动感测单元的输出计算图像的旋转角度；
图像处理单元，与所述运动估计单元耦接，配置成可根据所述图像的旋转角度对所述摄像系统输出的图像进行反向补偿。
根据本发明的一个方面，其中所述运动感测单元包括惯性测量单元，所述惯性测量单元安装在摄像系统的相机板上，并与摄像系统的镜头的光轴方向垂直或平行，所述惯性测量单元可实时测量所述镜头的角速度和加速度。
根据本发明的一个方面，其中所述运动估计单元配置成通过互补滤波算法计算所述镜头的横滚角、俯仰角。
根据本发明的一个方面，其中所述运动估计单元配置成通过互补滤波算法计算所述镜头的运动频率，当所述运动频率低于预设值时，通过所述图像处理单元对输出图像进行处理；当所述运动频率高于所述预设值时，对所述摄像镜头进行防抖处理。
根据本发明的一个方面，其中所述图像处理单元配置成在对所述图像进行反向补偿后，对所述图像进行裁剪以去除黑边。
本发明还提供一种无人机摄像系统，包括：
摄像机，配置成可拍摄周围图像；
如上所述的增稳处理模组，配置成可接收所述摄像机拍摄的图像并进行处理；
显示模块，与所述增稳处理模组耦接，配置成可显示经过所述图像处理单元处理的图像。
根据本发明的一个方面，其中所述摄像机包括鱼眼镜头，拍摄分辨率为3840*2160。
本发明还提供一种使用上所述的增稳处理模组进行图像增稳处理的方法，包括：
通过运动感测单元实时感测摄像系统的运动；
通过运动估计单元根据所述运动感测单元的输出计算图像的旋转角度；
通过图像处理单元根据所述图像的旋转角度对所述摄像系统输出的图像进行反向补偿。
根据本发明的一个方面，其中所述运动感测单元包括惯性测量单元，所述惯性测量单元安装在摄像系统的相机板上，并与摄像系统的镜头的光轴方 向垂直或平行，所述方法还包括：
通过所述惯性测量单元实时测量所述镜头的角速度和加速度。
根据本发明的一个方面，所述方法还包括：
所述运动估计单元通过互补滤波算法计算所述镜头的横滚角、俯仰角。
根据本发明的一个方面，所述方法还包括：
所述运动估计单元通过互补滤波算法计算所述镜头的运动频率，当所述运动频率低于预设值时，通过所述图像处理单元对输出图像进行处理；当所述运动频率高于所述预设值时，对所述摄像镜头进行防抖处理。
本发明的优选实施例提供了一种可用于无人机摄像系统的增稳处理模组、一种包括该增稳处理模组的无人机摄像系统，以及对图像进行防抖处理的方法。该增稳处理模组通过运动感测单元感测摄像系统的运动，通过运动估计单元计算图像的旋转角度，通过图像处理单元对摄像系统输出的图像进行反向补偿。该增稳处理模组可提升无人机摄像系统拍摄的运动画面的质量，既调整了小幅高频的抖动，又避免了大幅低频的抖动对输出画面的影响；安装在无人机摄像系统上，体积小，重量轻，取代了云台设备，不但大大节省了无人机上的空间，而且降低了无人机的制造成本。
附图说明
附图用来提供对本发明的进一步理解，并且构成说明书的一部分，与本发明的实施例一起用于解释本发明，并不构成对本发明的限制。在附图中：
图1示意性地示出了根据本发明的一个优选实施例的增稳处理模组；
图2示意性地示出了根据本发明的一个优选实施例的增稳处理模组对输出图像进行反向补偿；
图3示意性地示出了根据本发明的一个优选实施例的增稳处理模组对输出图像进行反向补偿；
图4示意性地示出了根据本发明的一个优选实施例的无人机摄像系统；
图5A示意性地示出了根据本发明的一个优选实施例的摄像系统镜头的 输出图像；
图5B示意性地示出了根据本发明的一个优选实施例的增稳处理模组对输出图像进行反向补偿；
图5C示意性地示出了根据本发明的一个优选实施例的增稳处理模组对输出图像进行裁剪；
图6示意性地示出了根据本发明的一个优选实施例的无人机摄像系统；
图7示出了根据本发明的一个优选实施例进行图像增稳处理的方法。
具体实施方式
在下文中，仅简单地描述了某些示例性实施例。正如本领域技术人员可认识到的那样，在不脱离本发明的精神或范围的情况下，可通过各种不同方式修改所描述的实施例。因此，附图和描述被认为本质上是示例性的而非限制性的。
在本发明的描述中，需要理解的是，术语"中心"、"纵向"、"横向"、"长度"、"宽度"、"厚度"、"上"、"下"、"前"、"后"、"左"、"右"、"竖直"、"水平"、"顶"、"底"、"内"、"外"、"顺时针"、"逆时针"等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本发明的限制。此外，术语"第一"、"第二"仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有"第一"、"第二"的特征可以明示或者隐含地包括一个或者更多个所述特征。在本发明的描述中，"多个"的含义是两个或两个以上，除非另有明确具体的限定。
在本发明的描述中，需要说明的是，除非另有明确的规定和限定，术语"安装"、"相连"、"连接"应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接:可以是机械连接，也可以是电连接或可以相互通讯；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连 通或两个元件的相互作用关系。对于本领域的普通技术人员而言，可以根据具体情况理解上述术语在本发明中的具体含义。
在本发明中，除非另有明确的规定和限定，第一特征在第二特征之"上"或之"下"可以包括第一和第二特征直接接触，也可以包括第一和第二特征不是直接接触而是通过它们之间的另外的特征接触。而且，第一特征在第二特征"之上"、"上方"和"上面"包括第一特征在第二特征正上方和斜上方，或仅仅表示第一特征水平高度高于第二特征。第一特征在第二特征"之下"、"下方"和"下面"包括第一特征在第二特征正上方和斜上方，或仅仅表示第一特征水平高度小于第二特征。
下文的公开提供了许多不同的实施方式或例子用来实现本发明的不同结构。为了简化本发明的公开，下文中对特定例子的部件和设置进行描述。当然，它们仅仅为示例，并且目的不在于限制本发明。此外，本发明可以在不同例子中重复参考数字和/或参考字母，这种重复是为了简化和清楚的目的，其本身不指示所讨论各种实施方式和/或设置之间的关系。此外，本发明提供了的各种特定的工艺和材料的例子，但是本领域普通技术人员可以意识到其他工艺的应用和/或其他材料的使用。
以下结合附图对本发明的实施例进行说明，应当理解，此处所描述的实施例仅用于说明和解释本发明，并不用于限定本发明。
原有的运动DV中的软件增稳技术通常只是针对小幅高频的运动，对大幅、低频的运动，例如：滑雪、骑行这种大幅转弯的运动没有增稳的效果，本发明提供一种具有水平增稳功能的图像处理模组。
根据本发明的一个优选实施例，如图1所示，本发明提供一种可用于无人机摄像系统的增稳处理模组10，包括：运动感测单元11、运动估计单元12和图像处理单元13。其中运动感测单元11配置成可感测摄像系统的运动，例如，搭载摄像系统的无人机在高空旋转、俯冲、爬升、跳跃，或遇到天气变化时的颠簸等，都可能导致摄像系统的在不同的方向上加速运动或者绕轴 摆动，从而造成拍摄画面的抖动。运动估计单元12与运动感测单元11耦接，配置成可根据运动感测单元11的输出计算图像的旋转角度，该图像的旋转角度是指无人机搭载的摄像系统拍摄画面的实时旋转角度，在该摄像系统无角速度和/或角加速度的情况下，该实时拍摄的画面即为显示图像，当存在画面抖动的情况时，需要先对拍摄画面进行增稳处理，再显示处理后的图像。图像处理单元13与运动估计单元12耦接，配置成可根据图像的旋转角度对摄像系统输出的图像进行反向补偿，根据本发明的一个优选实施例，通过以下方式对摄像系统输出的图像进行反向补偿。
如图2所示，根据运动感测单元11实时输出的摄像系统镜头的俯仰角的变化，将输出图像沿着竖直方向反向调整。优选地，该摄像系统拍摄的画面为垂直于摄像系统镜头的光轴的球面画面，沿着竖直方向反向调整指在该球面画面上沿着摄像系统镜头的俯、仰方向反向调整。
如图3所示，根据运动感测单元11实时输出的摄像系统镜头的横滚角的变化，对输出图像进行反向补偿，使反向补偿后的显示图像与上一帧显示图像在水平方向上所成的角度不发生变化。
优选的，该无人机摄像系统采用鱼眼镜头，可拍摄大广角球面画面，便于后续根据该摄像系统的角度变化进行画面调整。这样开启增稳处理校准后保留的视场角会足够大。
根据本发明的一个优选实施例，运动感测单元11包括惯性测量单元110，惯性测量单元(Inertial Measurement Unit)是测量物体三轴姿态角(或角速率)以及加速度的装置。通常的惯性测量单元包含了三个单轴的加速度计和三个单轴的陀螺，三个单轴的加速度计检测物体在载体坐标系统独立三轴的加速度信号，而三个单轴的陀螺检测载体相对于导航坐标系的角速度信号，测量物体在三维空间中的角速度和加速度，并以此解算出物体的姿态。如图4所示，惯性测量单元110安装在摄像系统的相机板上，并与摄像系统的镜头的光轴方向(如图2中x轴所示)垂直或平行，惯性测量单元110可实时测量摄像系统镜头的角速度和加速度。
根据本发明的一个优选实施例，运动估计单元12配置成通过互补滤波算法计算摄像系统的镜头的横滚角、俯仰角。互补滤波算法指在短时间内采用陀螺仪测得的角度做为最优，定时对加速度计采样来的角度进行取平均值来校正陀螺仪的测得的角度。长时间以加速度计测得的角度为优，加大加速计的测算比重。再通过低通滤波抑制加速度计的高频信号，通过高通滤波抑制陀螺仪的低频信号，相加得到整个频带的信号。
根据本发明的一个优选实施例，运动估计单元12配置成通过互补滤波算法计算摄像系统的镜头的运动频率，当该运动频率低于预设值时，通过图像处理单元13对输出图像进行处理；当该运动频率高于预设值时，对摄像镜头进行防抖处理。也即，当镜头的运动频率低于预设值时，说明该无人机摄像系统处于低频运动中，通过本发明提供的增稳处理模组10对输出图像进行处理；当镜头的运动频率高于预设值时，说明该无人机摄像系统处于高频运动中，可以通过摄像系统中内置的防抖处理装置，对输出画面进行调整。
根据本发明的一个优选实施例，如图5A、5B、5C所示，图像处理单元13配置成在对输出图像进行反向补偿后，对图像进行裁剪以去除黑边。其中，图5A示出了摄像系统镜头实时拍摄的图像51，并且由于无人机的震动造成了拍摄画面的抖动，因此图像51偏离了正常的水平定向。图5B中，图像处理单元对输出图像51进行反向补偿后生成图像52，保持图像52仍然处于水平定向，便于用户观看。如图5C所示，图像处理单元裁剪去除黑边后得到图像53，图像53的分辨率小于图像52。在图像处理单元对输出图像做了反向补偿后，图像会产生黑边，所以需要先裁剪再显示，使得显示的图像更加美观。当然，这样做会降低一定的分辨率，损失一部分视场角。设置摄像系统镜头拍摄画面的分辨率达到3840*2160p，则开启增稳处理后，校准后的图像分辨率会降低到1080p，依然可以达到很好的效果。
图1中示出了所述运动感测单元11、运动估计单元12和图像处理单元13均设置在无人机上，也可以构思将运动感测单元11、运动估计单元12和图像处理单元13分离地设置在无人机本体上以及无人机的控制器上。根据一 个实施例，运动感测单元11、运动估计单元12可设置在无人机上，图像处理单元13设置在无人机的遥控器上，例如集成在用于控制无人机的APP中。无人机本体向无人机的遥控器传输实时采集的图像以及由运动估计单元12所计算的图像旋转角度，所述图像处理单元根据所述图像以及图像旋转角度，根据上述方法对图像进行相应处理，并将处理后的图像显示在遥控器上。或者可替换的，运动估计单元12也可以设置在无人机的遥控器上。
根据本发明的一个优选实施例，如图6所示，本发明还提供一种无人机摄像系统20，包括：摄像机21、如上所述的增稳处理模组10和显示模块22。摄像机21配置成可拍摄周围图像。如上所述的增稳处理模组10配置成可接收摄像机21拍摄的图像并进行处理。显示模块22与增稳处理模组10耦接，配置成可显示经过图像处理单元13处理的图像。
根据本发明的一个优选实施例，该无人机摄像系统20的摄像机21包括鱼眼镜头，拍摄分辨率为3840*2160。
根据本发明的一个优选实施例，如图7所示，本发明还提供一种使用如上所述的增稳处理模组10进行图像增稳处理的方法30，包括：
在步骤S301中，通过运动感测单元11实时感测摄像系统的运动。例如，搭载摄像系统的无人机在高空旋转、俯冲、爬升、跳跃，或遇到天气变化时的颠簸等，都可能导致摄像系统的在不同的角度上加速运动，从而造成拍摄画面的抖动。
在步骤S302中，通过运动估计单元12根据运动感测单元11的输出计算图像的旋转角度。该图像的旋转角度是指无人机搭载的摄像系统拍摄画面的实时旋转角度，在该摄像系统无角速度和/或角加速度的情况下，该实时拍摄的画面即为显示图像，当存在画面抖动的情况时，需要先对拍摄画面进行增稳处理，再显示处理后的图像。
在步骤S303中，通过图像处理单元13根据该图像的旋转角度对该摄像系统输出的图像进行反向补偿。
根据本发明的一个优选实施例，运动感测单元11包括惯性测量单元110， 该惯性测量单元110安装在摄像系统的相机板上，并与摄像系统的镜头的光轴方向垂直或平行(如图4所示)，图像增稳处理的方法30还包括：
通过惯性测量单元110实时测量摄像系统镜头的角速度和加速度。
根据本发明的一个优选实施例，图像增稳处理的方法30还包括：
运动估计单元12通过互补滤波算法计算摄像系统镜头的横滚角、俯仰角。
根据本发明的一个优选实施例，图像增稳处理的方法30还包括：
运动估计单元12通过互补滤波算法计算摄像系统镜头的运动频率，当该运动频率低于预设值时，通过图像处理单元13对输出图像进行处理；当该运动频率高于预设值时，对摄像镜头进行防抖处理。也即，当镜头的运动频率低于预设值时，说明该无人机摄像系统处于低频运动中，通过本发明提供的增稳处理模组10对输出图像进行处理；当镜头的运动频率高于预设值时，说明该无人机摄像系统处于高频运动中，可以通过摄像系统中内置的防抖处理装置，对输出画面进行调整。
本发明的优选实施例提供了一种可用于无人机摄像系统的增稳处理模组、一种包括该增稳处理模组的无人机摄像系统，以及对图像进行防抖处理的方法。该增稳处理模组通过运动感测单元感测摄像系统的运动，通过运动估计单元计算图像的旋转角度，通过图像处理单元对摄像系统输出的图像进行反向补偿。该增稳处理模组可提升无人机摄像系统拍摄的运动画面的质量，既调整了小幅高频的抖动，又避免了大幅低频的抖动对输出画面的影响；安装在无人机摄像系统上，体积小，重量轻，取代了云台设备，不但大大节省了无人机上的空间，而且降低了无人机的制造成本。
最后应说明的是：以上所述仅为本发明的优选实施例而已，并不用于限制本发明，尽管参照前述实施例对本发明进行了详细的说明，对于本领域的技术人员来说，其依然可以对前述各实施例所记载的技术方案进行修改，或 者对其中部分技术特征进行等同替换。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (11)

  1. 一种可用于无人机摄像系统的增稳处理模组，包括：
    运动感测单元，配置成可感测摄像系统的运动；
    运动估计单元，与所述运动感测单元耦接，配置成可根据所述运动感测单元的输出计算图像的旋转角度；
    图像处理单元，与所述运动估计单元耦接，配置成可根据所述图像的旋转角度对所述摄像系统输出的图像进行反向补偿。
  2. 如权利要求1所述的增稳处理模组，其中所述运动感测单元包括惯性测量单元，所述惯性测量单元安装在摄像系统的相机板上，并与摄像系统的镜头的光轴方向垂直或平行，所述惯性测量单元可实时测量所述镜头的角速度和加速度。
  3. 如权利要求1或2所述的增稳处理模组，其中所述运动估计单元配置成通过互补滤波算法计算所述镜头的横滚角、俯仰角。
  4. 如权利要求1或2所述的增稳处理模组，其中所述运动估计单元配置成通过互补滤波算法计算所述镜头的运动频率，当所述运动频率低于预设值时，通过所述图像处理单元对输出图像进行处理；当所述运动频率高于所述预设值时，对所述摄像镜头进行防抖处理。
  5. 如权利要求1或2所述的增稳处理模组，其中所述图像处理单元配置成在对所述图像进行反向补偿后，对所述图像进行裁剪以去除黑边。
  6. 一种无人机摄像系统，包括：
    摄像机，配置成可拍摄周围图像；
    如权利要求1-5中任一项所述的增稳处理模组，配置成可接收所述摄像机拍摄的图像并进行处理；
    显示模块，与所述增稳处理模组耦接，配置成可显示经过所述图像处理单元处理的图像。
  7. 如权利要求6所述的无人机摄像系统，其中所述摄像机包括鱼眼镜头，拍摄分辨率为3840*2160。
  8. 一种使用权利要求1-5中任一项所述的增稳处理模组进行图像增稳处理的方法，包括：
    通过运动感测单元实时感测摄像系统的运动；
    通过运动估计单元根据所述运动感测单元的输出计算图像的旋转角度；
    通过图像处理单元根据所述图像的旋转角度对所述摄像系统输出的图像进行反向补偿。
  9. 如权利要求8所述的方法，其中所述运动感测单元包括惯性测量单元，所述惯性测量单元安装在摄像系统的相机板上，并与摄像系统的镜头的光轴方向垂直或平行，所述方法还包括：
    通过所述惯性测量单元实时测量所述镜头的角速度和加速度。
  10. 如权利要求8或9所述的方法，还包括：
    所述运动估计单元通过互补滤波算法计算所述镜头的横滚角、俯仰角。
  11. 如权利要求8或9所述的方法，还包括：
    所述运动估计单元通过互补滤波算法计算所述镜头的运动频率，当所述运动频率低于预设值时，通过所述图像处理单元对输出图像进行处理；当所述运动频率高于所述预设值时，对所述摄像镜头进行防抖处理。
PCT/CN2020/132014 2020-08-07 2020-11-27 增稳处理模组、无人机摄像系统以及图像增稳处理方法 WO2022027876A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010788869.2A CN111866395B (zh) 2020-08-07 2020-08-07 增稳处理模组、无人机摄像系统以及图像增稳处理方法
CN202010788869.2 2020-08-07

Publications (1)

Publication Number Publication Date
WO2022027876A1 true WO2022027876A1 (zh) 2022-02-10

Family

ID=72972269

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/132014 WO2022027876A1 (zh) 2020-08-07 2020-11-27 增稳处理模组、无人机摄像系统以及图像增稳处理方法

Country Status (2)

Country Link
CN (1) CN111866395B (zh)
WO (1) WO2022027876A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111866395B (zh) * 2020-08-07 2022-05-13 苏州臻迪智能科技有限公司 增稳处理模组、无人机摄像系统以及图像增稳处理方法
WO2022178703A1 (zh) * 2021-02-24 2022-09-01 深圳市大疆创新科技有限公司 电子增稳方法及装置、可移动平台、成像装置
WO2023010318A1 (zh) * 2021-08-04 2023-02-09 深圳市大疆创新科技有限公司 云台控制方法、装置、云台和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090316010A1 (en) * 2008-06-20 2009-12-24 Sony Corporation Image pickup device, image blur correcting method, and program
CN104714353A (zh) * 2013-12-12 2015-06-17 佳能株式会社 图像抖动校正设备及其控制方法、光学装置以及摄像装置
CN106257911A (zh) * 2016-05-20 2016-12-28 上海九鹰电子科技有限公司 用于视频图像的图像稳定方法和装置
CN111866395A (zh) * 2020-08-07 2020-10-30 苏州臻迪智能科技有限公司 增稳处理模组、无人机摄像系统以及图像增稳处理方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103763461B (zh) * 2013-12-31 2015-06-10 合一网络技术(北京)有限公司 一种视频抖动检测方法及装置
CN105578146B (zh) * 2016-01-07 2019-01-11 浙江宇视科技有限公司 一种机芯防抖处理方法及装置
CN206181178U (zh) * 2016-08-31 2017-05-17 深圳零度智能飞行器有限公司 一种航拍相机
CN106506940A (zh) * 2016-10-19 2017-03-15 天津奇幻岛科技有限公司 一种补偿相机在直线滑移云台上晃动的方法
CN110012224B (zh) * 2019-03-26 2021-07-09 Oppo广东移动通信有限公司 摄像头防抖系统、方法、电子设备和计算机可读存储介质
CN110312072A (zh) * 2019-05-14 2019-10-08 深圳市歌美迪实业有限公司 一种流媒体后视镜图像抖动补偿方法及其装置
CN110213490B (zh) * 2019-06-25 2020-09-29 浙江大华技术股份有限公司 一种图像防抖方法、装置、电子设备及存储介质
CN110166697B (zh) * 2019-06-28 2021-08-31 Oppo广东移动通信有限公司 摄像头防抖方法、装置、电子设备和计算机可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090316010A1 (en) * 2008-06-20 2009-12-24 Sony Corporation Image pickup device, image blur correcting method, and program
CN104714353A (zh) * 2013-12-12 2015-06-17 佳能株式会社 图像抖动校正设备及其控制方法、光学装置以及摄像装置
CN106257911A (zh) * 2016-05-20 2016-12-28 上海九鹰电子科技有限公司 用于视频图像的图像稳定方法和装置
CN111866395A (zh) * 2020-08-07 2020-10-30 苏州臻迪智能科技有限公司 增稳处理模组、无人机摄像系统以及图像增稳处理方法

Also Published As

Publication number Publication date
CN111866395B (zh) 2022-05-13
CN111866395A (zh) 2020-10-30

Similar Documents

Publication Publication Date Title
WO2022027876A1 (zh) 增稳处理模组、无人机摄像系统以及图像增稳处理方法
US11480291B2 (en) Camera system using stabilizing gimbal
US11930273B2 (en) Vibration damping gimbal sleeve for an aerial vehicle
US8896697B2 (en) Video motion compensation and stabilization gimbaled imaging system
JP6234679B2 (ja) 外乱を引き起こす動きを最小化しながら搭載型カメラによる撮影を行うための回転翼無人機の操縦方法
JP2015204633A (ja) 安定化した一続きの画像を提供するビデオカメラが設けられた回転翼無人機
CN106210544B (zh) 一种移动终端及其拍摄视频的防抖处理方法、系统
US11076082B2 (en) Systems and methods for digital video stabilization
US10150576B2 (en) Vibration damping gimbal sleeve for an aerial vehicle
WO2020227998A1 (zh) 图像增稳控制方法、拍摄设备和可移动平台
WO2018024239A1 (zh) 混合式图像稳定系统
KR102338479B1 (ko) 3축 짐벌 구조를 이용한 공중 촬영 장치 및 이의 제어 방법
KR101921625B1 (ko) 영상 안정화 기반의 원격 이동체용 영상처리 장치
CN217935703U (zh) 机载大幅面航空摄影系统装置
WO2023010318A1 (zh) 云台控制方法、装置、云台和存储介质
WO2022253018A1 (zh) 无人机视角的视频显示方法及显示系统
WO2022134036A1 (zh) 可移动平台及其控制方法、装置
WO2021232424A1 (zh) 飞行辅助方法及装置、无人飞行器、遥控器、显示器、无人飞行器系统和存储介质
CN114296479B (zh) 一种基于图像的无人机对地面车辆跟踪方法及系统
JP7195880B2 (ja) 空間安定化装置
TWM619972U (zh) 行駛系統的操控裝置
WO2022253017A1 (zh) 无人机的控制方法及控制系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20948414

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20948414

Country of ref document: EP

Kind code of ref document: A1