WO2019183826A1 - 图像处理方法、装置和无人机 - Google Patents

图像处理方法、装置和无人机 Download PDF

Info

Publication number
WO2019183826A1
WO2019183826A1 PCT/CN2018/080807 CN2018080807W WO2019183826A1 WO 2019183826 A1 WO2019183826 A1 WO 2019183826A1 CN 2018080807 W CN2018080807 W CN 2018080807W WO 2019183826 A1 WO2019183826 A1 WO 2019183826A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
yuv
bayer
component
processor
Prior art date
Application number
PCT/CN2018/080807
Other languages
English (en)
French (fr)
Inventor
麻军平
张强
庹伟
余良
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201880031978.0A priority Critical patent/CN110622505A/zh
Priority to PCT/CN2018/080807 priority patent/WO2019183826A1/zh
Publication of WO2019183826A1 publication Critical patent/WO2019183826A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/85—Camera processing pipelines; Components thereof for processing colour signals for matrixing
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots

Definitions

  • Embodiments of the present invention relate to the field of image processing technologies, and in particular, to an image processing method, apparatus, and drone.
  • CMOS image sensors are widely used in digital imaging because of their low cost and improved imaging quality.
  • Mobile phones, digital cameras, digital cameras, and aerial mapping in specialized fields are widely used in CMOS. Take photos with your mobile phone and camera, record videos, and almost everyone uses them every day.
  • CMOS devices output Bayer images because of the sensitization characteristics, that is, each pixel point, only one color component of R (Red, red), G (Green, green), B (Blue, blue), such as Figure 1 shows.
  • the Bayer image needs to be interpolated (for example, a demosaicing algorithm) to obtain an interpolated image, which is called an RGB image, which can be used for display so that the user can see the captured image with the naked eye.
  • RGB image which can be used for display so that the user can see the captured image with the naked eye.
  • Each pixel of the RGB image has three color components of R, G, and B.
  • the image needs to be compressed to reduce the space occupied by the image.
  • the YUV image is suitable for compression processing. Therefore, the RGB image needs to be converted into a YUV image, wherein the Y component represents the brightness information of the image, U The V component is the color difference information of the image.
  • each pixel of the RGB image has three color components of R, G, and B, and the data amount is three times that of the Bayer image. Therefore, the amount of data of the YUV image obtained by the RGB image conversion is large. The compression pressure on the YUV image is large.
  • Embodiments of the present invention provide an image processing method, apparatus, and drone for reducing the amount of data of a YUV image and reducing the pressure of image compression.
  • an embodiment of the present invention provides an image processing method, including: acquiring a Bayer image by an image sensor; directly converting the Bayer image into a YUV image; and compressing the YUV image and storing the compression The latter YUV image.
  • each image unit in the Bayer image includes an R component, a B component, and two G components.
  • the YUV image is a YUV422 image.
  • the converting the Bayer image directly into a YUV image comprises:
  • Each image unit in the Bayer image is directly converted into two YUV 422 pixels by a transformation matrix.
  • the conversion matrix is an invertible matrix.
  • an embodiment of the present invention provides an imaging apparatus, including: an image sensor, a processor, and a memory, wherein the processor is communicatively coupled to the image sensor and the memory; and the image sensor is configured to acquire a Bayer image
  • the processor for converting the Bayer image directly into a YUV image and compressing the YUV image; the memory for storing the compressed YUV image.
  • each image unit in the Bayer image includes an R component, a B component, and two G components.
  • the YUV image is a YUV422 image.
  • the processor is specifically configured to directly convert each image unit in the Bayer image into two YUV422 pixels through a conversion matrix.
  • the conversion matrix is an invertible matrix.
  • an embodiment of the present invention provides a drone, including: a rack, a pan/tilt, and an imaging device; the pan/tilt is mounted on the rack, and the pan/tilt is used to carry the imaging device ;
  • the imaging device includes an image sensor, a processor and a memory, the processor being communicatively coupled to the image sensor and the memory; the image sensor for acquiring a Bayer image; the processor for The Bayer image is directly converted to a YUV image, and the YUV image is compressed; the memory is used to store the compressed YUV image.
  • each image unit in the Bayer image includes an R component, a B component, and two G components.
  • the YUV image is a YUV422 image.
  • the processor is specifically configured to directly convert each image unit in the Bayer image into two YUV422 pixels through a conversion matrix.
  • the conversion matrix is an invertible matrix.
  • an embodiment of the present invention provides a readable storage medium, where the readable storage medium stores a computer program; when the computer program is executed, the following operations are performed: acquiring a Bayer image by using an image sensor; The Bayer image is directly converted into a YUV image; and the YUV image is compressed, and the compressed YUV image is stored.
  • each image unit in the Bayer image includes an R component, a B component, and two G components.
  • the YUV image is a YUV422 image.
  • the converting the Bayer image directly into a YUV image comprises:
  • Each image unit in the Bayer image is directly converted into two YUV 422 pixels by a transformation matrix.
  • the conversion matrix is an invertible matrix.
  • the image processing method, device and drone provided by the embodiments of the present invention convert the acquired Bayer image into a YUV image directly, and then compress and store the YUV image. Since the present embodiment converts the Bayer image directly into a YUV image, unlike the prior art, there is no need to convert the Bayer image into an intermediate image (for example, an RGB image), thereby avoiding the generation of an intermediate image having a large amount of data. Therefore, the amount of YUV image data obtained by directly converting the Bayer image is less than that in the prior art, the pressure of image compression is reduced, and the amount of data of the YUV image after compression processing is less, thereby saving image storage resources.
  • an intermediate image for example, an RGB image
  • Figure 1 is a schematic diagram of components of a pixel in a Bayer image
  • FIG. 2 is a schematic architectural diagram of an unmanned flight system in accordance with an embodiment of the present invention.
  • FIG. 3 is a flowchart of an image processing method according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of each image unit in a Bayer image according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a YUV422 image according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of directly converting each image unit in a Bayer image into two YUV422 pixels according to an embodiment of the present invention
  • FIG. 7 is a schematic structural diagram of an image forming apparatus according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of a drone according to an embodiment of the present invention.
  • Embodiments of the present invention provide an image processing method, apparatus, and drone.
  • the drone involved therein may be a rotorcraft, for example, a multi-rotor aircraft driven by air by a plurality of urging means, and embodiments of the present invention are not limited thereto.
  • FIG. 2 is a schematic architectural diagram of an unmanned flight system in accordance with an embodiment of the present invention. This embodiment is described by taking a rotorcraft unmanned aerial vehicle as an example.
  • the unmanned aerial vehicle system 100 can include an unmanned aerial vehicle 110, a pan/tilt head 120, a display device 130, and a control device 140.
  • the unmanned aerial vehicle 110 may include a power system 150, a flight control system 160, and a rack.
  • the UAV 110 can be in wireless communication with the control device 140 and the display device 130.
  • the rack can include a fuselage and a tripod (also known as a landing gear).
  • the fuselage may include a center frame and one or more arms coupled to the center frame, the one or more arms extending radially from the center frame.
  • the stand is coupled to the fuselage for supporting when the UAV 110 is landing.
  • Power system 150 may include one or more electronic governors (referred to as ESCs) 151, one or more propellers 153, and one or more electric machines 152 corresponding to one or more propellers 153, wherein motor 152 is coupled Between the electronic governor 151 and the propeller 153, the motor 152 and the propeller 153 are disposed on the arm of the unmanned aerial vehicle 110; the electronic governor 151 is configured to receive the driving signal generated by the flight control system 160 and provide driving according to the driving signal. Current is supplied to the motor 152 to control the rotational speed of the motor 152. Motor 152 is used to drive propeller rotation to power the flight of unmanned aerial vehicle 110, which enables unmanned aerial vehicle 110 to achieve one or more degrees of freedom of motion.
  • ESCs electronic governors
  • the UAV 110 can be rotated about one or more axes of rotation.
  • the above-described rotating shaft may include a roll axis, a yaw axis, and a pitch axis.
  • the motor 152 can be a DC motor or an AC motor.
  • the motor 152 may be a brushless motor or a brushed motor.
  • Flight control system 160 may include flight controller 161 and sensing system 162.
  • the sensing system 162 is used to measure the attitude information of the unmanned aerial vehicle, that is, the position information and state information of the UAV 110 in space, for example, three-dimensional position, three-dimensional angle, three-dimensional speed, three-dimensional acceleration, and three-dimensional angular velocity.
  • Sensing system 162 can include, for example, at least one of a gyroscope, an ultrasonic sensor, an electronic compass, an Inertial Measurement Unit (IMU), a vision sensor, a global navigation satellite system, and a barometer.
  • the global navigation satellite system can be a Global Positioning System (GPS).
  • GPS Global Positioning System
  • the flight controller 161 is used to control the flight of the unmanned aerial vehicle 110, for example, the flight of the unmanned aerial vehicle 110 can be controlled based on the attitude information measured by the sensing system 162. It should be understood that the flight controller 161 may control the UAV 110 in accordance with pre-programmed program instructions, or may control the UAV 110 in response to one or more control commands from the control device 140.
  • the pan/tilt 120 can include a motor 122.
  • the pan/tilt is used to carry the imaging device 123.
  • the flight controller 161 can control the motion of the platform 120 via the motor 122.
  • the platform 120 may further include a controller for controlling the motion of the platform 120 by controlling the motor 122.
  • the platform 120 can be independent of the UAV 110 or a portion of the UAV 110.
  • the motor 122 can be a DC motor or an AC motor.
  • the motor 122 may be a brushless motor or a brushed motor.
  • the pan/tilt can be located at the top of the UAV or at the bottom of the UAV.
  • the imaging device 123 may be, for example, a device for capturing an image such as a camera or a video camera, and the imaging device 123 may communicate with the flight controller and perform shooting under the control of the flight controller.
  • the imaging device 123 of the present embodiment includes at least a photosensitive element, such as a Complementary Metal Oxide Semiconductor (CMOS) sensor or a Charge-coupled Device (CCD) sensor.
  • CMOS Complementary Metal Oxide Semiconductor
  • CCD Charge-coupled Device
  • Display device 130 is located at the ground end of unmanned aerial vehicle system 100, can communicate with unmanned aerial vehicle 110 wirelessly, and can be used to display attitude information for unmanned aerial vehicle 110. In addition, an image taken by the imaging device can also be displayed on the display device 130. It should be understood that the display device 130 may be a stand-alone device or may be integrated in the control device 140.
  • the control device 140 is located at the ground end of the unmanned aerial vehicle system 100 and can communicate with the unmanned aerial vehicle 110 in a wireless manner for remote manipulation of the unmanned aerial vehicle 110.
  • FIG. 3 is a flowchart of an image processing method according to an embodiment of the present invention. As shown in FIG. 3, the method in this embodiment can be applied to a device having an imaging device, such as a drone, a mobile phone, a digital camera, and the like.
  • the methods can include:
  • the Bayer image is acquired, for example, the output Bayer image can be acquired by an image sensor (for example, a CMOS sensor).
  • the Bayer image is, for example, as shown in FIG. 1, and any pixel in the Bayer image includes only one color component, such as an R component, a B component, and a G component.
  • the Bayer image is directly converted into a YUV image.
  • the YUV image is, for example, a YUV422 image or a YUV444 image.
  • the embodiment may also save the compressed YUV image.
  • the YUV image may be compressed, but not limited to compression of the YUV image, and other processing may be performed on the YUV image, for example, to other components or devices. .
  • the compressed YUV image may be stored, but not limited to storing the compressed YUV image, and the compressed YUV image may be further processed, for example, transmitted to other components or device.
  • the acquired Bayer image is directly converted into a YUV image, and then the YUV image is compressed and stored. Since the present embodiment converts the Bayer image directly into a YUV image, unlike the prior art, there is no need to convert the Bayer image into an intermediate image (for example, an RGB image), thereby avoiding the generation of an intermediate image having a large amount of data. Therefore, the amount of YUV image data obtained by directly converting the Bayer image is less than that in the prior art, the pressure of image compression is reduced, and the amount of data of the YUV image after compression processing is less, thereby saving image storage resources.
  • an intermediate image for example, an RGB image
  • the Bayer image described above may include a plurality of image elements, each image unit including four color components, an R component, a B component, and two G components, respectively.
  • FIG. 4 is a schematic diagram of each image unit in a Bayer image according to an embodiment of the present invention.
  • FIG. 4 shows an image unit, and other image units in the Bayer image are similar, and each image unit includes Bayer.
  • FIG. 4 is an example of an image unit in a Bayer image, and the image unit in the Bayer image is not limited to that shown in FIG. 4; for example, an image unit in a Bayer image (ie, a 2*2 bayer unit) ), from top left to bottom right, it may be: GRBG or GBRG or BGGR.
  • the YUV image is a YUV 422 image, that is, the present embodiment converts the Bayer image directly into a YUV 422 image when performing the above S302.
  • the YUV image is, for example, as shown in FIG. 5.
  • FIG. 5 is a schematic diagram of a YUV422 image according to an embodiment of the present invention.
  • one possible implementation of S302 is to convert each image element in the Bayer image directly into two YUV 422 pixels, such as shown in FIG.
  • every 2*2 pixels (i.e., 4 pixels) in the Bayer image is directly converted into 1*2 (i.e., two) adjacent pixels in the YUV422 image.
  • two adjacent pixels in the YUV422 image are pixels including a Y component and a U component, and pixels of a Y component and a V component, and two pixels of the YUV422 image obtained after conversion include four components. , respectively, U component, V component, and two Y components.
  • the image unit in the Bayer image also includes four components, namely, an R component, a B component, and two G components. It can be seen that, in this embodiment, the R component, the B component, and the two G components are directly converted into a U component, a V component, and two Y components, that is, four components in the Bayer image are converted into 4 in the YUV422 image.
  • the amount of data before and after conversion is basically the same, which greatly reduces the amount of data of the YUV image and reduces the pressure of image compression and storage.
  • the present embodiment converts each image element in the Bayer image directly into two YUV 422 pixels by a transformation matrix. Since each picture element includes four components, the conversion matrix can be, for example, a 4*4 matrix. This embodiment can obtain the components of two YUV 422 pixels by multiplying the four components of each image unit as a vector by the conversion matrix.
  • the present embodiment multiplies four components of each image unit as a vector by a conversion matrix, and multiplies the result with an offset vector (the offset vector is a column vector, the column vector is The 4*1 matrix is added to obtain a column vector (including four elements), which are respectively used as components of two YUV422 pixels, for example, the first element is used as the Y component in the first YUV422 pixel.
  • the second element is the U component of the first YUV 422 pixel
  • the third element is the Y component of the second YUV 422 pixel
  • the fourth element is the V component of the second YUV 422 pixel. For example, as shown below.
  • I is a 2*2 conversion matrix and A is an offset vector.
  • the conversion matrix of this embodiment is not limited thereto.
  • the conversion matrix may also be adjusted based on this example.
  • the element for obtaining the Y component in the offset vector is 0; the element for obtaining the U component in the offset vector is 128; and the offset vector is used to obtain the V component The element is 128.
  • the conversion matrix is an invertible matrix, such that after converting the Bayer image to the YUV image, the Bayer image can also be recovered from the YUV image by transforming the inverse matrix of the matrix.
  • a computer storage medium is also provided in the embodiment of the present invention.
  • the computer storage medium stores program instructions, and the program may include some or all of the steps of the image processing methods in the foregoing embodiments.
  • FIG. 7 is a schematic structural diagram of an image forming apparatus according to an embodiment of the present invention.
  • the image forming apparatus 700 of the present embodiment may include: an image sensor 701, a processor 702, and a memory 703; The image sensor 701 and the memory 703 are communicatively coupled.
  • the image sensor 701 is configured to acquire a Bayer image.
  • the processor 702 is configured to directly convert the Bayer image into a YUV image; and compress the YUV image.
  • the memory 703 is configured to store the compressed YUV image
  • each image unit in the Bayer image includes an R component, a B component, and two G components.
  • the YUV image is a YUV422 image.
  • the processor 702 is specifically configured to:
  • Each image unit in the Bayer image is directly converted into two YUV 422 pixels by a transformation matrix.
  • the conversion matrix is an invertible matrix.
  • the memory 703 can also be used to store program instructions that, when invoked, are executed by the processor 702.
  • the imaging device of the present embodiment can be used to perform the technical solutions of the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.
  • FIG. 8 is a schematic structural diagram of a drone according to an embodiment of the present invention.
  • the drone 800 of the present embodiment includes a rack 810, a pan/tilt 820, and an imaging device 830.
  • the pan/tilt 820 is mounted on the rack 810, and the pan/tilt 820 is used to carry the imaging device 830.
  • the imaging device 830 may include an image sensor 831, a processor 832, and a memory 833; the processor 832 is communicatively coupled to the image sensor 831 and the memory 833.
  • the image sensor 831 is configured to acquire a Bayer image.
  • the processor 832 is configured to directly convert the Bayer image into a YUV image; and compress the YUV image.
  • the memory 833 is configured to store the compressed YUV image.
  • each image unit in the Bayer image includes an R component, a B component, and two G components.
  • the YUV image is a YUV422 image.
  • the processor 832 is specifically configured to:
  • Each image unit in the Bayer image is directly converted into two YUV 422 pixels by a transformation matrix.
  • the conversion matrix is an invertible matrix.
  • the memory 833 can also be used to store program instructions that, when invoked, the processor 832 executes the program instructions of the above described scheme.
  • the unmanned aerial vehicle of this embodiment can be used to implement the technical solutions of the foregoing method embodiments, and the implementation principle and technical effects thereof are similar, and details are not described herein again.
  • the foregoing program may be stored in a computer readable storage medium, and the program is executed when executed.
  • the foregoing storage medium includes: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and the like, which can store program codes. Medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

本发明实施例提供一种图像处理方法、装置和无人机，所述方法包括：通过图像传感器获取Bayer图像；将所述Bayer图像直接转换为YUV图像；以及对所述YUV图像进行压缩并存储。由于本实施例是将Bayer图像直接转换为YUV图像，与现有技术不同的是，无需将Bayer图像转换为中间图像(例如RGB图像)的过程，从而避免了产生数据量较大的中间图像，因此，直接转换Bayer图像得到的YUV图像数据量与现有技术中相比较少，减少了图像压缩的压力，而且压缩处理后的YUV图像的数据量更少，进而节省了图像存储资源。

Description

图像处理方法、装置和无人机 技术领域
本发明实施例涉及图像处理技术领域，尤其涉及一种图像处理方法、装置和无人机。
背景技术
互补金属氧化物半导体(Complementary Metal Oxide Semiconductor，CMOS)图像传感器因为成本较低，且成像质量逐步提高，在数字成像领域应用十分广泛。手机摄像头，数码相机，数字摄像机，以及专业领域的航空测绘等等都在广泛使用CMOS。用手机、相机进行拍照，记录视频，几乎每个人每天都在使用。
其中，CMOS器件因为感光特性的原因，输出的是Bayer图像，即每个像素点，只有R(Red，红)、G(Green，绿)、B(Blue，蓝)中的一个颜色分量，如图1所示。为了显示图像，需要将Bayer图像进行插值处理(例如解马赛克算法)，得到插值后的图像，称为RGB图像，该RGB图像可以用于显示，以使用户用肉眼能看到拍摄到的图像，其中RGB图像的每一个像素点都具有R、G、B三个颜色分量。为了保存图像，需要对图像进行压缩处理以减少图像占用的空间，其中，YUV图像适用于做压缩处理，因此，还需要将RGB图像转换为YUV图像，其中，Y分量表示图像的亮度信息，U、V分量是图像的色差信息。
但是，现有技术中，RGB图像的每一个像素点都具有R、G、B三个颜色分量，其数据量为Bayer图像的三倍，因此，由RGB图像转换得到YUV图像的数据量大，对YUV图像的压缩压力较大。
发明内容
本发明实施例提供一种图像处理方法、装置和无人机，用于减少YUV图像的数据量，减少图像压缩的压力。
第一方面，本发明实施例提供一种图像处理方法，所述方法包括：通过 图像传感器获取Bayer图像；将所述Bayer图像直接转换为YUV图像；以及对所述YUV图像进行压缩，并存储压缩后的所述YUV图像。
在一种可能的设计中，所述Bayer图像中的每个图像单元包括R分量、B分量以及两个G分量。
在一种可能的设计中，所述YUV图像为YUV422图像。
在一种可能的设计中，所述将所述Bayer图像直接转换为YUV图像包括：
通过转换矩阵将所述Bayer图像中的每个图像单元直接转换为两个YUV422像素。
在一种可能的设计中，所述转换矩阵为可逆矩阵。
第二方面，本发明实施例提供一种成像装置，包括：图像传感器、处理器和存储器，所述处理器与所述图像传感器和所述存储器通信连接；所述图像传感器，用于获取Bayer图像；所述处理器，用于将所述Bayer图像直接转换为YUV图像，以及对所述YUV图像进行压缩；所述存储器，用于存储压缩后的所述YUV图像。
在一种可能的设计中，所述Bayer图像中的每个图像单元包括R分量、B分量以及两个G分量。
在一种可能的设计中，所述YUV图像为YUV422图像。
在一种可能的设计中，所述处理器，具体用于：通过转换矩阵将所述Bayer图像中的每个图像单元直接转换为两个YUV422像素。
在一种可能的设计中，所述转换矩阵为可逆矩阵。
第三方面，本发明实施例提供一种无人机，包括：机架、云台以及成像装置；所述云台搭载在所述机架上，且所述云台用于承载所述成像装置；
所述成像装置包括图像传感器、处理器和存储器，所述处理器与所述图像传感器和所述存储器通信连接；所述图像传感器，用于获取Bayer图像；所述处理器，用于将所述Bayer图像直接转换为YUV图像，以及对所述YUV图像进行压缩；所述存储器，用于存储压缩后的所述YUV图像。
在一种可能的设计中，所述Bayer图像中的每个图像单元包括R分量、B分量以及两个G分量。
在一种可能的设计中，所述YUV图像为YUV422图像。
在一种可能的设计中，所述处理器，具体用于：通过转换矩阵将所述Bayer 图像中的每个图像单元直接转换为两个YUV422像素。
在一种可能的设计中，所述转换矩阵为可逆矩阵。
第四方面，本发明实施例提供一种可读存储介质，所述可读存储介质上存储有计算机程序；所述计算机程序在被执行时，实现如下操作：通过图像传感器获取Bayer图像；将所述Bayer图像直接转换为YUV图像；以及对所述YUV图像进行压缩，并存储压缩后的所述YUV图像。
在一种可能的设计中，所述Bayer图像中的每个图像单元包括R分量、B分量以及两个G分量。
在一种可能的设计中，所述YUV图像为YUV422图像。
在一种可能的设计中，所述将所述Bayer图像直接转换为YUV图像包括：
通过转换矩阵将所述Bayer图像中的每个图像单元直接转换为两个YUV422像素。
在一种可能的设计中，所述转换矩阵为可逆矩阵。
本发明实施例提供的图像处理方法、装置和无人机，通过将获取到的Bayer图像直接转换为YUV图像，然后对该YUV图像进行压缩并存储。由于本实施例是将Bayer图像直接转换为YUV图像，与现有技术不同的是，无需将Bayer图像转换为中间图像(例如RGB图像)的过程，从而避免了产生数据量较大的中间图像，因此，直接转换Bayer图像得到的YUV图像数据量与现有技术中相比较少，减少了图像压缩的压力，而且压缩处理后的YUV图像的数据量更少，进而节省了图像存储资源。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为Bayer图像中像素的分量的一种示意图；
图2是根据本发明的实施例的无人飞行系统的示意性架构图；
图3为本发明一实施例提供的图像处理方法的流程图；
图4为本发明一实施例提供的Bayer图像中每个图像单元的示意图；
图5为本发明一实施例提供的YUV422图像的示意图；
图6为本发明一实施例提供的将Bayer图像中的每个图像单元直接转换为两个YUV422像素的示意图；
图7为本发明一实施例提供的成像装置的结构示意图；
图8为本发明一实施例提供的无人机的结构示意图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本发明的实施例提供了图像处理方法、装置和无人机。其中涉及的无人机可以是旋翼飞行器(rotorcraft)，例如，由多个推动装置通过空气推动的多旋翼飞行器，本发明的实施例并不限于此。
图2是根据本发明的实施例的无人飞行系统的示意性架构图。本实施例以旋翼无人飞行器为例进行说明。
无人飞行系统100可以包括无人飞行器110、云台120、显示设备130和控制装置140。其中，无人飞行器110可以包括动力系统150、飞行控制系统160和机架。无人飞行器110可以与控制装置140和显示设备130进行无线通信。
机架可以包括机身和脚架(也称为起落架)。机身可以包括中心架以及与中心架连接的一个或多个机臂，一个或多个机臂呈辐射状从中心架延伸出。脚架与机身连接，用于在无人飞行器110着陆时起支撑作用。
动力系统150可以包括一个或多个电子调速器(简称为电调)151、一个或多个螺旋桨153以及与一个或多个螺旋桨153相对应的一个或多个电机152，其中电机152连接在电子调速器151与螺旋桨153之间，电机152和螺旋桨153设置在无人飞行器110的机臂上；电子调速器151用于接收飞行控制系统160产生的驱动信号，并根据驱动信号提供驱动电流给电机152，以控制电机152的转速。电机152用于驱动螺旋桨旋转，从而为无人飞行器110的 飞行提供动力，该动力使得无人飞行器110能够实现一个或多个自由度的运动。在某些实施例中，无人飞行器110可以围绕一个或多个旋转轴旋转。例如，上述旋转轴可以包括横滚轴、偏航轴和俯仰轴。应理解，电机152可以是直流电机，也可以交流电机。另外，电机152可以是无刷电机，也可以是有刷电机。
飞行控制系统160可以包括飞行控制器161和传感系统162。传感系统162用于测量无人飞行器的姿态信息，即无人飞行器110在空间的位置信息和状态信息，例如，三维位置、三维角度、三维速度、三维加速度和三维角速度等。传感系统162例如可以包括陀螺仪、超声传感器、电子罗盘、惯性测量单元(Inertial Measurement Unit，IMU)、视觉传感器、全球导航卫星系统和气压计等传感器中的至少一种。例如，全球导航卫星系统可以是全球定位系统(Global Positioning System，GPS)。飞行控制器161用于控制无人飞行器110的飞行，例如，可以根据传感系统162测量的姿态信息控制无人飞行器110的飞行。应理解，飞行控制器161可以按照预先编好的程序指令对无人飞行器110进行控制，也可以通过响应来自控制装置140的一个或多个控制指令对无人飞行器110进行控制。
云台120可以包括电机122。云台用于携带成像装置123。飞行控制器161可以通过电机122控制云台120的运动。可选地，作为另一实施例，云台120还可以包括控制器，用于通过控制电机122来控制云台120的运动。应理解，云台120可以独立于无人飞行器110，也可以为无人飞行器110的一部分。应理解，电机122可以是直流电机，也可以是交流电机。另外，电机122可以是无刷电机，也可以是有刷电机。还应理解，云台可以位于无人飞行器的顶部，也可以位于无人飞行器的底部。
成像装置123例如可以是照相机或摄像机等用于捕获图像的设备，成像装置123可以与飞行控制器通信，并在飞行控制器的控制下进行拍摄。本实施例的成像装置123至少包括感光元件，该感光元件例如为互补金属氧化物半导体(Complementary Metal Oxide Semiconductor，CMOS)传感器或电荷耦合元件(Charge-coupled Device，CCD)传感器。
显示设备130位于无人飞行系统100的地面端，可以通过无线方式与无人飞行器110进行通信，并且可以用于显示无人飞行器110的姿态信息。另 外，还可以在显示设备130上显示成像装置拍摄的图像。应理解，显示设备130可以是独立的设备，也可以集成在控制装置140中。
控制装置140位于无人飞行系统100的地面端，可以通过无线方式与无人飞行器110进行通信，用于对无人飞行器110进行远程操纵。
图3为本发明一实施例提供的图像处理方法的流程图，如图3所示，本实施例的方法可以应用于无人机、手机、数码相机等具有成像装置的设备中，本实施例的方法可以包括：
S301、通过图像传感器获取Bayer图像。
本实施例中，获取Bayer图像，例如可以通过图像传感器(例如CMOS传感器)获取输出的Bayer图像。该Bayer图像例如如图1所示，Bayer图像中的任一个像素点只包括一个颜色分量，例如R分量、B分量、G分量。
S302、将所述Bayer图像直接转换为YUV图像。
本实施例中，在获得Bayer图像之后，再将Bayer图像直接转换为YUV图像。该YUV图像例如是YUV422图像或者YUV444图像。
S303、对所述YUV图像进行压缩，并存储压缩后的所述YUV图像。
本实施例中，在将Bayer图像转换为YUV图像之后，对YUV图像进行压缩。可选地，本实施例还可以对压缩后的YUV图像进行保存。
需要说明的是，在将Bayer图像直接转换为YUV图像之后，可以对YUV图像进行压缩，但不局限于对YUV图像进行压缩，也可以对YUV图像做其它的处理，例如传输给其它组件或设备。
需要说明的是，在YUV图像进行压缩之后，可以存储压缩后的YUV图像，但不局限于存储压缩后的YUV图像，也可以对压缩后的YUV图像做其它的处理，例如传输给其它组件或设备。
本实施例中，通过将获取到的Bayer图像直接转换为YUV图像，然后对该YUV图像进行压缩并存储。由于本实施例是将Bayer图像直接转换为YUV图像，与现有技术不同的是，无需将Bayer图像转换为中间图像(例如RGB图像)的过程，从而避免了产生数据量较大的中间图像，因此，直接转换Bayer图像得到的YUV图像数据量与现有技术中相比较少，减少了图像压缩的压力，而且压缩处理后的YUV图像的数据量更少，进而节省了图像存储资源。
在一些实施例中，上述Bayer图像可以包括多个图像单元，每个图像单 元包括四个颜色分量，分别为R分量、B分量和两个G分量。例如如图4所示，图4为本发明一实施例提供的Bayer图像中每个图像单元的示意图，图4中示出一个图像单元，Bayer图像中其它图像单元类似，每个图像单元包括Bayer图像的四个像素，即2*2像素，其中，每个图像单元中第一行第一列的像素为R分量，第一行第二列的像素为G分量，第二行第一列的像素为G分量，第二行第二列的像素为B分量，即从左上到右下排列方式为RGGB。需要说明的是，图4是以Bayer图像中的图像单元的一个例子示出，Bayer图像中的图像单元并不限于图4所示；例如Bayer图像中的图像单元(即2*2的bayer单元)，从左上到右下排列方式还可能是：GRBG或者GBRG或者BGGR。
在一些实施例中，YUV图像为YUV422图像，即本实施例在执行上述S302时是将Bayer图像直接转换为YUV422图像。其中，YUV图像例如如图5所示，图5为本发明一实施例提供的YUV422图像的示意图。
在一些实施例中，S302的一种可能的实现方式为：将Bayer图像中的每个图像单元直接转换为两个YUV422像素，例如如图6所示。本实施例是将Bayer图像中每2*2个像素(即4个像素)直接转换为YUV422图像中的1*2个(即两个)相邻像素。其中，根据图5所示，YUV422图像中的两个相邻像素为包括Y分量和U分量的像素，以及Y分量和V分量的像素，转换后获得的YUV422图像的两个像素包括四个分量，分别为U分量、V分量和两个Y分量。而且根据图4所示，Bayer图像中图像单元也包括四个分量，即R分量、B分量和两个G分量。由此可知，本实施例是将即R分量、B分量和两个G分量，直接转换为U分量、V分量和两个Y分量，即将Bayer图像中的四个分量转换为YUV422图像中的4个分量，转换前后的数据量基本不变，与现有技术相比，极大减少了YUV图像的数据量，减小了图像压缩和存储的压力。
在一些实施例中，本实施例通过转换矩阵将所述Bayer图像中的每个图像单元直接转换为两个YUV422像素。由于每个图像单元包括四个分量，该转换矩阵例如可以为4*4的矩阵。本实施例可以通过将每个图像单元的四个分量作为一个向量与该转换矩阵相乘，以获得两个YUV422像素的分量。
在一些实施例中，本实施例将每个图像单元的四个分量作为一个向量与 转换矩阵相乘，再将相乘的结果与偏置向量(该偏置向量为列向量，该列向量为4*1的矩阵)相加，获得一个列向量(包括四个元素)，将该四个元素分别作为两个YUV422像素的分量，例如将第一个元素作为第一个YUV422像素中的Y分量，第二元素作为第一个YUV422像素中的U分量，第三个元素作为第二个YUV422像素中的Y分量，第四个元素作为第二YUV422像素中的V分量。例如如下所示。
Figure PCTCN2018080807-appb-000001
其中，I为2*2的转换矩阵，A为偏置向量。
在一些实施例中，
Figure PCTCN2018080807-appb-000002
需要说明的是，本实施例的转换矩阵不限于此，例如转换矩阵也可以基于此例子做些调整后得到的。
在一些实施例中，
Figure PCTCN2018080807-appb-000003
其中，所述偏置向量中用于获得Y分量的元素为0；所述偏置向量中用于获得所述U分量的元素为128；所述偏置向量中用于获得所述V分量的元素为128。
因此，
Figure PCTCN2018080807-appb-000004
在一些实施例中，该转换矩阵为可逆矩阵，这样本实施例在将Bayer图像转换为YUV图像之后，还可以通过转换矩阵的逆矩阵，从YUV图像中恢复出Bayer图像。
在一些实施例中，
Figure PCTCN2018080807-appb-000005
例如：
Figure PCTCN2018080807-appb-000006
本发明实施例中还提供了一种计算机存储介质，该计算机存储介质中存储有程序指令，所述程序执行时可包括上述各实施例中的图像处理方法的部分或全部步骤。
图7为本发明一实施例提供的成像装置的结构示意图，如图7所示，本实施例的成像装置700可以包括：图像传感器701、处理器702和存储器703；所述处理器702与所述图像传感器701和所述存储器703通信连接。
所述图像传感器701，用于获取Bayer图像。
所述处理器702，用于将所述Bayer图像直接转换为YUV图像；以及对所述YUV图像进行压缩。
所述存储器703，用于存储压缩后的所述YUV图像
在一些实施例中，所述Bayer图像中的每个图像单元包括R分量、B分量以及两个G分量。
在一些实施例中，所述YUV图像为YUV422图像。
在一些实施例中，所述处理器702，具体用于：
通过转换矩阵将所述Bayer图像中的每个图像单元直接转换为两个YUV422像素。
在一些实施例中，所述转换矩阵为可逆矩阵。
在一些实施例中，存储器703还可以用于存储程序指令，在所述程序指令被调用时处理器702执行上述方案的程序指令。
本实施例的成像装置，可以用于执行上述各方法实施例的技术方案，其实现原理和技术效果类似，此处不再赘述。
图8为本发明一实施例提供的无人机的结构示意图，如图8所示，本实施例的无人机800包括：机架810、云台820和成像装置830。其中，所述云台820搭载在所述机架810上，且所述云台820用于承载所述成像装置830。其中，成像装置830可以包括：图像传感器831、处理器832和存储器833；所述处理器832与所述图像传感器831和所述存储器833通信连接。
所述图像传感器831，用于获取Bayer图像。
所述处理器832，用于将所述Bayer图像直接转换为YUV图像；以及对所述YUV图像进行压缩。
所述存储器833，用于存储压缩后的所述YUV图像。
在一些实施例中，所述Bayer图像中的每个图像单元包括R分量、B分量以及两个G分量。
在一些实施例中，所述YUV图像为YUV422图像。
在一些实施例中，所述处理器832，具体用于：
通过转换矩阵将所述Bayer图像中的每个图像单元直接转换为两个YUV422像素。
在一些实施例中，所述转换矩阵为可逆矩阵。
在一些实施例中，存储器833还可以用于存储程序指令，在所述程序指令被调用时处理器832执行上述方案的程序指令。
本实施例的无人机，可以用于执行上述各方法实施例的技术方案，其实现原理和技术效果类似，此处不再赘述。
本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于一计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：只读内存(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。

Claims (20)

  1. 一种图像处理方法，其特征在于，所述方法包括：
    通过图像传感器获取Bayer图像；
    将所述Bayer图像直接转换为YUV图像；以及
    对所述YUV图像进行压缩，并存储压缩后的所述YUV图像。
  2. 根据权利要求1所述的方法，其特征在于，所述Bayer图像中的每个图像单元包括R分量、B分量以及两个G分量。
  3. 根据权利要求1或2所述的方法，其特征在于，所述YUV图像为YUV422图像。
  4. 根据权利要求2所述的方法，其特征在于，所述将所述Bayer图像直接转换为YUV图像包括：
    通过转换矩阵将所述Bayer图像中的每个图像单元直接转换为两个YUV422像素。
  5. 根据权利要求4所述的方法，其特征在于，所述转换矩阵为可逆矩阵。
  6. 一种成像装置，其特征在于，包括：图像传感器、处理器和存储器，所述处理器与所述图像传感器和存储器通信连接；
    所述图像传感器，用于获取Bayer图像；
    所述处理器，用于将所述Bayer图像直接转换为YUV图像；以及对所述YUV图像进行压缩；
    所述存储器，用于存储压缩后的所述YUV图像。
  7. 根据权利要求6所述的成像装置，其特征在于，所述Bayer图像中的每个图像单元包括R分量、B分量以及两个G分量。
  8. 根据权利要求6或7所述的成像装置，其特征在于，所述YUV图像 为YUV422图像。
  9. 根据权利要求7所述的成像装置，其特征在于，所述处理器，具体用于：
    通过转换矩阵将所述Bayer图像中的每个图像单元直接转换为两个YUV422像素。
  10. 根据权利要求9所述的成像装置，其特征在于，所述转换矩阵为可逆矩阵。
  11. 一种无人机，其特征在于，包括：机架、云台以及成像装置；所述云台搭载在所述机架上，且所述云台用于承载所述成像装置；
    所述成像装置包括图像传感器、处理器和存储器，所述处理器与所述图像传感器和所述存储器通信连接；
    所述图像传感器，用于获取Bayer图像；
    所述处理器，用于将所述Bayer图像直接转换为YUV图像；以及对所述YUV图像进行压缩；
    所述存储器，用于存储压缩后的所述YUV图像。
  12. 根据权利要求11所述的无人机，其特征在于，所述Bayer图像中的每个图像单元包括R分量、B分量以及两个G分量。
  13. 根据权利要求11或12所述的无人机，其特征在于，所述YUV图像为YUV422图像。
  14. 根据权利要求12所述的无人机，其特征在于，所述处理器，具体用于：
    通过转换矩阵将所述Bayer图像中的每个图像单元直接转换为两个YUV422像素。
  15. 根据权利要求14所述的无人机，其特征在于，所述转换矩阵为可逆矩阵。
  16. 一种可读存储介质，其特征在于，所述可读存储介质上存储有计算机程序；所述计算机程序在被执行时，实现如下操作：
    通过图像传感器获取Bayer图像；
    将所述Bayer图像直接转换为YUV图像；以及
    对所述YUV图像进行压缩，并存储压缩后的所述YUV图像。
  17. 根据权利要求16所述的可读存储介质，其特征在于，所述Bayer图像中的每个图像单元包括R分量、B分量以及两个G分量。
  18. 根据权利要求16或17所述的可读存储介质，其特征在于，所述YUV图像为YUV422图像。
  19. 根据权利要求17所述的可读存储介质，其特征在于，所述将所述Bayer图像直接转换为YUV图像包括：
    通过转换矩阵将所述Bayer图像中的每个图像单元直接转换为两个YUV422像素。
  20. 根据权利要求19所述的可读存储介质，其特征在于，所述转换矩阵为可逆矩阵。
PCT/CN2018/080807 2018-03-28 2018-03-28 图像处理方法、装置和无人机 WO2019183826A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880031978.0A CN110622505A (zh) 2018-03-28 2018-03-28 图像处理方法、装置和无人机
PCT/CN2018/080807 WO2019183826A1 (zh) 2018-03-28 2018-03-28 图像处理方法、装置和无人机

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/080807 WO2019183826A1 (zh) 2018-03-28 2018-03-28 图像处理方法、装置和无人机

Publications (1)

Publication Number Publication Date
WO2019183826A1 true WO2019183826A1 (zh) 2019-10-03

Family

ID=68059114

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/080807 WO2019183826A1 (zh) 2018-03-28 2018-03-28 图像处理方法、装置和无人机

Country Status (2)

Country Link
CN (1) CN110622505A (zh)
WO (1) WO2019183826A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101977330A (zh) * 2010-11-12 2011-02-16 北京空间机电研究所 一种基于YUV变换的Bayer图像压缩方法
CN103220460A (zh) * 2012-01-20 2013-07-24 华晶科技股份有限公司 影像处理方法及其装置
CN103384307A (zh) * 2012-05-03 2013-11-06 三星电子株式会社 图像处理设备和方法
CN106794901A (zh) * 2016-10-21 2017-05-31 深圳市大疆创新科技有限公司 处理故障的方法、飞行器、服务器和控制设备
WO2018012925A1 (ko) * 2016-07-14 2018-01-18 엘지이노텍(주) 이미지 생성 방법 및 생성 장치

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1913626A (zh) * 2006-09-12 2007-02-14 张冕 一种无线汽车后视装置及其实现方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101977330A (zh) * 2010-11-12 2011-02-16 北京空间机电研究所 一种基于YUV变换的Bayer图像压缩方法
CN103220460A (zh) * 2012-01-20 2013-07-24 华晶科技股份有限公司 影像处理方法及其装置
CN103384307A (zh) * 2012-05-03 2013-11-06 三星电子株式会社 图像处理设备和方法
WO2018012925A1 (ko) * 2016-07-14 2018-01-18 엘지이노텍(주) 이미지 생성 방법 및 생성 장치
CN106794901A (zh) * 2016-10-21 2017-05-31 深圳市大疆创新科技有限公司 处理故障的方法、飞行器、服务器和控制设备

Also Published As

Publication number Publication date
CN110622505A (zh) 2019-12-27

Similar Documents

Publication Publication Date Title
US11336837B2 (en) System, method, and mobile platform for supporting bracketing imaging
WO2021078270A1 (zh) 一种可拆换的云台相机、飞行器、系统及其云台拆换方法
US11181809B2 (en) Focusing method, imaging device, and unmanned aerial vehicle
US10855904B2 (en) Focusing method and apparatus, image photographing method and apparatus, and photographing system
US20180113462A1 (en) Position-based soft stop for a 3-axis gimbal
US20160352992A1 (en) Image Stabilization Mechanism
WO2022032538A1 (zh) 全景拍摄方法、装置、系统及计算机可读存储介质
US20200154026A1 (en) Imaging control method, imaging device and unmanned aerial vehicle
WO2019227441A1 (zh) 可移动平台的拍摄控制方法和设备
US11076082B2 (en) Systems and methods for digital video stabilization
WO2020181494A1 (zh) 参数同步方法、拍摄装置和可移动平台
WO2020172800A1 (zh) 可移动平台的巡检控制方法和可移动平台
WO2020227998A1 (zh) 图像增稳控制方法、拍摄设备和可移动平台
US20220103799A1 (en) Image data processing method and apparatus, image processing chip, and aircraft
WO2020014953A1 (zh) 一种图像处理方法及设备
WO2019183789A1 (zh) 无人机的控制方法、装置和无人机
US20210136340A1 (en) Color temperature adjustment method, control terminal, and movable platform
WO2019227438A1 (zh) 一种图像处理方法、设备、飞行器、系统及存储介质
WO2021168821A1 (zh) 可移动平台的控制方法和设备
WO2021253173A1 (zh) 图像处理方法、装置及巡检系统
CN110291772A (zh) 影像拍摄器和手持云台、可移动平台
WO2021031184A1 (zh) 图像处理方法、设备和可移动平台
WO2019183826A1 (zh) 图像处理方法、装置和无人机
WO2020237429A1 (zh) 遥控设备的控制方法和遥控设备
US20200412945A1 (en) Image processing apparatus, image capturing apparatus, mobile body, image processing method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18911916

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18911916

Country of ref document: EP

Kind code of ref document: A1