CN107257427B - Nine-lens unmanned aerial vehicle panoramic camera and image processing method thereof - Google Patents

Abstract

The invention discloses a nine-lens unmanned aerial vehicle panoramic camera and an image processing method thereof. The panoramic camera consists of a mounting bracket and an image processing system. The image processing system comprises 9 identical image sensors and 9 fixed focus lenses with identical specifications. The installing support is formed by connecting 2 longitudinal side panels, 4 inclined panels, 2 transverse side panels, a bottom plate and corresponding fixing plates. The image sensing module is arranged on each inclined panel, the longitudinal side panel, the transverse side panel and the bottom plate, and the fixing plate is used for fixing the mounting bracket. The image processing system comprises an image sensing module, a programmable gate array, a peripheral circuit, a wireless transmission module, a video compression processing chip and the peripheral circuit thereof. The invention can shoot a large-range image signal at the same time, shoot image data from a plurality of visual angles, and improve the efficiency of shooting the image data by the unmanned aerial vehicle and the abundant image data. And the nine image sensors are uniformly controlled, so that image splicing noise at different shutter moments and in different control modes is eliminated.

Description

Nine-lens unmanned aerial vehicle panoramic camera and image processing method thereof

Technical Field

The invention belongs to the field of cameras, and particularly relates to a camera borne by an unmanned aerial vehicle. In particular to a nine-lens unmanned aerial vehicle panoramic camera and an image processing method thereof.

Background

The panoramic camera system carried by unmanned aerial vehicle is widely applied to various fields of national economy and national defense construction, and plays an important role in national construction and safety protection. Along with the continuous progress of science and technology, unmanned aerial vehicle panoramic camera is continuously promoting. However, the prior art has shortcomings and needs to be further improved in quality technology. Chinese patent CN201610912717.2, a panoramic camera system carried by unmanned aerial vehicle and an operation method thereof, uses polyhedrons as carriers, integrates camera equipment, GPS and BDS positioning and OSD image processing into a whole, has simple structure and smaller volume, omits an indirect process link, and has more comprehensive and accurate shooting and more timely information feedback. The method has the defects that dead angles exist for splicing, different lenses cannot be matched with the image sensor, and a splicing algorithm is complex. Chinese patent CN201610552295.2 (method and device for shooting panoramic pictures by unmanned aerial vehicle) can control unmanned aerial vehicle to automatically rotate and shoot images, so that the operation convenience of shooting panoramic pictures is greatly improved on the premise of not adopting panoramic cameras. The method has the defects that only a single lens is adopted, the time difference exists in the spliced images, and the splicing algorithm is complex. The panoramic camera is carried on the unmanned aerial vehicle to collect aerial images according to the Chinese patent CN201320554125.X panoramic image unmanned aerial vehicle collection system, and the panoramic image unmanned aerial vehicle collection system has the advantages of being wide in collection range and high in collection efficiency. The lens of the panoramic camera faces different directions, can collect images of an object from different angles, and has the advantage of high collection precision. The method has the defects that dead angles exist for splicing, different lenses cannot be matched with the image sensor, and a splicing algorithm is complex.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a multi-lens panoramic structure system construction method and an image processing method aiming at the related structure.

The aim of the invention is achieved in that: nine camera lens unmanned aerial vehicle panoramic camera, its characterized in that: the panoramic camera consists of two parts, a mounting bracket and an image processing system; the image processing system is arranged on the mounting bracket; the installing support is fixed in unmanned aerial vehicle bottom.

The image processing system comprises 9 identical image sensors and 9 fixed focus lenses with identical specifications, wherein the pixels sensed by the image sensors are longitudinally x pixels, and transversely y pixels (x > y), and the length and the width of the sensing pixels are w; setting the focal length of the focal lens as z, and setting the length of the sensing surface of the image sensor as xw and the width as yw;

order the

I.e., a is equal to twice the arctangent function of 0.5xw/z, β is equal to twice the arctangent function of 0.5yw/z,

units of α and β are degrees;

the mounting bracket is formed by connecting a plurality of flat plates and comprises a bottom plate ABCDEFGH (30), side panels, inclined panels and a fixed plate; the side panels include longitudinal side panels and transverse side panels.

The fixing plate comprises a longitudinal side fixing plate, an inclined plane fixing plate and a transverse side fixing plate, and the longitudinal side fixing plate, the inclined plane fixing plate and the transverse side fixing plate jointly form the fixing plate.

The side panels and the bevel panels form a support plate.

The image sensing module is arranged on the supporting plate and the bottom plate, and the supporting plate is connected with the bottom plate and the fixing plate mounting bracket is fixed on the bottom mounting plane of the unmanned aerial vehicle; all the fixing plates are on one plane and parallel to the bottom plate ABCDEFGH.

The distance between the plane where the fixing plate is positioned and the bottom plate is set as u, and u is larger than 2yw; when in installation, the bottom plate is parallel to the installation plane of the bottom of the unmanned aerial vehicle.

The side panels include longitudinal side panels and transverse side panels refer to: longitudinal side panel ABJI, longitudinal side panel EFQR, transverse side panel GHVd, transverse side panel CDNM.

The inclined plate means inclined plate HAfe, inclined plate BCLK, inclined plate DEPO, inclined plate FGTS.

The fixed plate comprises a longitudinal side fixed plate, an inclined plane fixed plate and a transverse side fixed plate, and is characterized in that: longitudinal side fixing plate IJji, longitudinal side fixing plate QRrq, inclined fixing plate effhg, inclined fixing plate KLlk, inclined fixing plate OPpo, inclined fixing plate STts, lateral side panel GHVd, lateral side panel CDNM, lateral side fixing plate dVvc, lateral side fixing plate MNnm.

The supporting plate comprises a bevel panel HAfe, a bevel panel BCLK, a bevel panel DEPO, a bevel panel FGTS, a longitudinal side panel ABJI, a longitudinal side panel EFQR, a transverse side panel GHVd and a transverse side panel CDNM, the image sensing module is arranged on the supporting plate and the bottom plate ABCDEFGH, and the supporting plate is connected with the bottom plate and the fixing plate.

All the fixing plates are in one plane and parallel to the bottom plate ABCDEFGH means: longitudinal side fixing plate IJji, longitudinal side fixing plate QRrq, transverse side fixing plate dVvc, transverse side fixing plate MNnm and inclined fixing plate effhg, inclined fixing plate KLlk, inclined fixing plate OPpo, inclined fixing plate STts are on one plane and parallel to bottom plate ABCDEFGH.

The size of the base plate is determined according to the size of the unmanned aerial vehicle; AB and EF are mutually parallel, ABEF is rectangular, the length AB is set to be a times the length xw of the sensing surface of the image sensor, namely ab=ef= axw, and a is required to be larger than 3; cd=hg=ayw; the CD edge is parallel to the HG edge and is perpendicular to the AB and EF edges; becc, AHGF are isosceles trapezoids, ah=gf=bc=ed=aw (x+y)/2;

AF＝BE

the longitudinal side panels ABJI, EFQR are the same size, and are rectangular, with ab=ij= axw

The lateral side panels GHVd, CDNM are the same in size, as exemplified by lateral side panel CDNM35-2,

CD＝MN＝ayw；

the inclined plates HAfe, BCLK, DEPO, FGTS are rectangular in size, and are described as inclined plates BCLK,

BC＝KL＝aw(x+y)/2；

the longitudinal side fixing plates IJji and QRrq have the same size and are rectangular; take the longitudinal side fixing plate IJji as an example for illustration;

ij=ji= axw; ii=ji, and Ii, jj are longer than 3 millimeters;

the sizes of the transverse side fixing plates dVvc and MNN are the same, and the transverse side fixing plates dVvc and MNN are rectangular; fixing MNN with lateral side

For illustration;

mn=nm=ayw; mm=nn, and Mm, nn lengths are greater than 3 millimeters;

the inclined plane fixing plates efhg, KLlk, OPpo, STts are rectangular and have the same size; taking the bevel securing plate KLlk as an example,

KL＝lk＝aw(x+y)/2；

kk=ll, and Kk, ll are greater than 3 millimeters in length.

The image processing system consists of an image sensing module, a programmable gate array, a peripheral circuit, a wireless transmission module, a video compression processing chip and a peripheral circuit thereof;

the nine image sensing modules are respectively mounted on the bottom plate ABCDEFGH, the lateral side panels GHVd and CDNM, the longitudinal side panels ABJI and EFQR and the inclined panel HAfe, BCLK, DEPO, FGTS, and are connected with the programmable gate array and the peripheral circuit through control signal lines and pixel brightness signal connecting lines.

The programmable gate array and the peripheral circuit are connected with nine image sensing modules through control signal lines and pixel brightness signal connecting lines, are connected with the wireless transmission module through communication connecting lines, and are connected with the video compression processing chip and the peripheral circuit thereof through video output signal connecting lines and video compression signal connecting lines.

The wireless transmission module is connected with the programmable gate array and the peripheral circuit through communication connection wires, and transmits the data processed by the programmable gate array and the peripheral circuit to a remote place in a wireless transmission mode.

The video compression processing chip and the peripheral circuit thereof are connected with the programmable gate array and the peripheral circuit through the video output signal connecting wire and the video compression signal connecting wire.

The image sensing module is composed of a lens, a mounting box and a sensing circuit.

The mounting box is a sealed and light-tight box and is a cube or a cylinder, and the mounting box is provided with two parallel planes, one plane is used for mounting a lens, and the other plane is used for being fixed on a bottom plate, a side plate or a bevel panel. The center of the plane for installing the lens is provided with a circular inner wire opening for installing the lens, and the diameter of the wire opening is the same as that of the outer wire opening of the lens.

The lens is a fixed focus lens, and different focal length lenses are selected according to the requirement; the specifications of the nine lenses are identical.

The sensing circuit is used for sensing the image of the lens, the sensing circuit is provided with an image sensor, the sensing plane of the image sensor on the sensing circuit is ensured to be parallel to the lens, and the center of the lens on the image sensor is projected on the center of the image sensor; when the installation box is installed on the bottom plate, the side panel or the inclined panel, the sensing plane of the image sensor is ensured to be parallel to the plane where the bottom plate, the side panel or the inclined panel is positioned;

an image sensing module mounted on the base plate ABCDEFGH, the longitudinal edge of the image sensor being parallel to the AB edge; the exact center of the image sensor is aligned with the exact center of the bottom plate.

The image sensor comprises 2 image sensing modules which are arranged on 2 longitudinal side panels ABJI and EFQR, and longitudinal sides of the image sensor are respectively parallel to AB and EF sides; the center of the image sensor is aligned with the center of the side panel.

The image sensor comprises 2 image sensing modules which are arranged on 2 lateral side panels GHTD and CDNM, and the lateral sides of the image sensor are respectively parallel to the CD and GH sides; the center of the image sensor is aligned with the center of the side panel.

The longitudinal sides of the image sensors and the BC side, the DE side, the FG side and the HA side of the inclined panel are respectively provided with an angle arctan (y/x) degrees; the exact center of the image sensor is aligned with the exact center of the diagonal panels.

The programmable gate array and the peripheral circuit uniformly control the working mode of the image sensing module, receive the signals of the image sensing module, process the signals of the image sensing module, send the processed image sensing data to the video compression processing chip and the peripheral circuit thereof, receive the image data compressed by the video compression processing chip and the peripheral circuit thereof, and send the compressed image data to the wireless transmission module for wireless transmission.

Programmable gate array program flow:

the first step: the same control signals are sent to the nine image sensors, and the sent image sensor control signals are as follows: a field synchronizing signal, a line synchronizing signal, a clock signal, a driving timing signal, an exposure control signal, and a second step;

and a second step of: receiving pixel brightness signals of nine image sensors, and turning to a third step;

and a third step of: performing BAYER image processing and white balance processing on pixel brightness signals of the nine image sensors to obtain RGB signals of each pixel after the white balance processing, and turning to a fourth step;

fourth step: calling an image stitching subroutine, and turning to a fifth step;

fifth step: sending the spliced RGB format image signals to a video compression processing chip, and turning to a sixth step;

sixth step: receiving compressed image data from a video compression processing chip, and turning to a seventh step;

seventh step: and (3) sending the received compressed image data to a wireless transmission module for wireless transmission, and turning to the first step. The image stitching subroutine is carried out according to the following steps: setting PX longitudinal pixels and PY transverse pixels of the spliced image, and sequentially sequencing the longitudinal pixels from left to right according to natural numbers, wherein sequencing numbers are 1, 2, 3, 4, … …, PX-1 and PX; the transverse pixels are sequentially ordered according to natural numbers from top to bottom and are ordered into 1, 2, 3, 4, … …, PY-1 and PY; image stitching subroutine of pixels:

the first step: px=x+3x, py=3y; entering a second step;

and a second step of: the images are spliced as follows:

RGB data obtained by the image sensor of the substrate mounting is spliced in the longitudinal pixel sequence number from x +1 to 2x,

the lateral pixel sequence number is within the interval from y+1 to 2 y;

RGB data obtained by the image sensor installed on the longitudinal side panel ABJI are spliced in a section from x+1 to 2x of longitudinal pixel serial numbers and from 1 to y of transverse pixel serial numbers;

the RGB data obtained by the image sensor installed by the EFQR of the longitudinal side panel are spliced in a section from x+1 to 2x of longitudinal pixel serial numbers and from 2y+1 to 3y of transverse pixel serial numbers;

RGB data obtained by the image sensor mounted on the inclined panel HAfe is spliced with the vertical pixel serial number from 1

X, the lateral pixel sequence number is in the interval from 1 to y;

RGB data obtained by an image sensor mounted on a bevel panel BCLK is spliced in a vertical pixel sequence number from

2x+1 to 3x, the lateral pixel sequence numbers being in the interval from 1 to y;

RGB data obtained by an image sensor mounted on a diagonal panel DEPO are spliced in a longitudinal pixel sequence number from

2x+1 to 3x, the lateral pixel sequence numbers being within the interval 2y+1 to 3y;

RGB data obtained by an image sensor mounted on a diagonal panel FGTS is spliced to a vertical pixel number from 1

X, the lateral pixel sequence number is in the interval from 2y+1 to 3y;

RGB data obtained by an image sensor installed on a GHTD side panel is spliced on a longitudinal pixel serial number slave

1 to x, the lateral pixel sequence number is in the interval from y+1 to 2 y;

RGB data obtained by an image sensor installed on a lateral side panel CDNM is spliced on a longitudinal pixel serial number slave

2x+1 to 3x, the lateral pixel sequence numbers being within the interval from y+1 to 2 y;

and a third step of: and returning to the main program after the images are spliced.

The invention has the positive effects that:

1. and meanwhile, large-range image signals are shot, so that the efficiency of shooting image data by the unmanned aerial vehicle is improved.

2. The nine image sensors are uniformly controlled, so that image splicing noise caused by different shutter moments and different control modes can be eliminated.

3. Image data can be shot from a plurality of view angles in the flight process of the unmanned aerial vehicle, and more shooting view angles and richer image data are provided for image data post-processing.

Drawings

Fig. 1 is a schematic view of the overall structure of the present invention.

Fig. 2 is a schematic view of a mounting bracket structure.

Fig. 3 is a schematic perspective view of a mounting bracket.

Fig. 4 is a schematic view of the floor structure of the mounting bracket.

Fig. 5 is a schematic view of the longitudinal side panel structure of the mounting bracket.

Fig. 6 is a schematic view of the lateral side panel structure of the mounting bracket.

Fig. 7 is a schematic view of a bevel fixing plate structure of the mounting bracket.

Fig. 8 is a schematic view of a longitudinal side mounting plate structure of the mounting bracket.

Fig. 9 is a schematic view of a lateral side mounting plate structure of a mounting bracket.

Fig. 10 is a schematic view of a bevel fixing plate structure of a mounting bracket.

Fig. 11 is a block diagram of an image processing system.

Fig. 12 is a block diagram of an image sensing module.

Fig. 13 is a flow chart of a programmable gate array program.

FIG. 14 is a flow chart of an image stitching subroutine.

Fig. 15 is a first portion of a programmable gate array circuit diagram.

Fig. 16 is a second portion of a programmable gate array circuit diagram.

Fig. 17 is a third portion of a programmable gate array circuit diagram.

Fig. 18 is a fourth portion of a programmable gate array circuit diagram.

Fig. 19 shows a first part of the video compression processing circuit in the image compression and peripheral circuits.

Fig. 20 shows a second part of the video compression processing circuit in the image compression and peripheral circuits.

Fig. 21 is an image sensor control circuit in an image sensor and its peripheral circuits.

Fig. 22 is a first portion of an image sensor circuit.

Fig. 23 is a second portion of the image sensor circuit.

Fig. 24 is a wireless communication module circuit.

In the figure, 1 unmanned plane, 2 mounting bracket, 3 image processing system, 4 unmanned plane bottom mounting plane, 6-1, 6-2 fixed plate, 7-1, 7-2 support plate, 8-1 to 8-9 image sensing module, 9 programmable gate array and peripheral circuit, 10 wireless transmission module, 11 video compression processing chip and its peripheral circuit, 12 lens, 13 mounting box, 14 sensing circuit, 30 backplane ABCDEFGH,31-1 longitudinal side panel ABJI,31-2 longitudinal side panel EFQR,32-1 inclined panel HAfe,32-2 inclined panel BCLK,32-3 inclined panel DEPO,32-4 inclined panel FGTS,33-1 longitudinal side fixed plate IJji,33-2 longitudinal side fixed plate QRrq,34-1 inclined fixed plate efhg,34-2 inclined fixed plate KLlk,34-3 inclined fixed plate OPpo,34-4 inclined fixed plate STts,35-1 lateral side panel GHVd,35-2 lateral CDNM,36-1 lateral side panel MNnm, 36-2 lateral side fixed plate MNnm;

the support plates 7-1, 7-2 include inclined panels HAfe32-1, BCLK32-2, DEPO32-3, FGTS32-4, longitudinal side panels ABJI31-1, EFQR31-2, and lateral side panels GHVd35-1, CDNM 35-2;

the fixing plates 6-1, 6-2 include longitudinal side fixing plates IJji33-1, QRrq33-2, transverse side fixing plates dVvc36-1, MNN 36-2, and inclined fixing plates effg 34-1, KLlk34-2, OPpo34-3, STts 34-4.

The side panels include longitudinal side panels ABJI, EFQR and transverse side panels GHVd, CDNM.

Detailed Description

The panoramic camera consists of two parts, a mounting bracket and an image processing system. The image processing system is mounted on the mounting bracket. The installing support is fixed in unmanned aerial vehicle bottom. As shown in fig. 1.

See fig. 2. The mounting bracket is formed by connecting a plurality of flat plates, the flat plates are made of light materials, and the connection mode of the flat plates is shown in figure 2.

The bracket comprises a bottom plate ABCDEFGH30, a longitudinal side panel ABJI31-1, a longitudinal side panel EFQR31-2, a bevel panel HAfe32-1, a bevel panel BCLK32-2, a bevel panel DEPO32-3, a bevel panel FGTS32-4, a longitudinal side surface fixing plate IJji33-1, a longitudinal side surface fixing plate QRrq33-2, a bevel surface fixing plate efhg34-1, a bevel surface fixing plate KLlk34-2, a bevel surface fixing plate OPpo34-3, a bevel surface fixing plate STts34-4, a transverse side surface side panel GHTD 35-1, a transverse side surface side panel CDNM35-2, a transverse side surface fixing plate dVvc36-1 and a transverse side surface fixing plate MNN 36-2.

Longitudinal side panel ABJI31-1 is connected to chassis ABCDEFGH by an AB edge and longitudinal side panel EFQR31-2 is connected to chassis ABCDEFGH by an EF edge.

The bevel panel BCLK32-2 is connected to the chassis ABCDEFGH by the BC edge, the bevel panel DEPO32-3 is connected to the chassis ABCDEFGH by the DE edge, the bevel panel FGTS32-4 is connected to the chassis ABCDEFGH by the FG edge, and the bevel panel HAfe32-1 is connected to the chassis ABCDEFGH30 by the AH edge.

The longitudinal side fixing plate iji 33-1 is connected to the longitudinal side panel ABJI31-1 by IJ edge, and the longitudinal side fixing plate QRrq33-2 is connected to the longitudinal side panel EFQR31-2 by QR edge.

The bevel fixing plate KLlk34-2 is connected with the bevel panel BCLK32-2 through KL edge, the bevel fixing plate OPpo34-3 is connected with the bevel panel DEPO32-3 through OP edge, the bevel fixing plate STts34-4 is connected with the bevel panel FGTS32-4 through TS edge, and the bevel fixing plate efhg34-1 is connected with the bevel panel HAfe32-1 through ef edge.

The lateral side panel CDNM35-2 is connected to the chassis ABCDEFGH30 by a CD edge and the lateral side panel GHVd35-1 is connected to the chassis ABCDEFGH30 by a GH edge.

Lateral side fixing plate MNnm36-2 is connected to lateral side panel CDNM35-2 by MN side, and lateral side fixing plate dVvc36-1 is connected to lateral side panel GHVd35-1 by dV side.

See fig. 3. Unmanned aerial vehicle bottom mounting plane 4 refers to the mounting plane that unmanned aerial vehicle bottom was used for installing panoramic camera, and when unmanned aerial vehicle level was placed, unmanned aerial vehicle bottom mounting plane was in on the horizontal plane.

The bottom panel ABCDEFGH30 is mounted parallel to the bottom mounting plane of the unmanned aerial vehicle.

The support plates 7-1, 7-2 include inclined plates HAfe32-1, BCLK32-2, DEPO32-3, FGTS32-4, longitudinal side panels ABJI31-1, EFQR31-2, and lateral side panels GHVd35-1, CDNM35-2, the image sensing modules are mounted on the support plates 7-1, 7-2, and the support plates 7-1, 7-2 connect the bottom plate 30 and the fixing plates 6-1, 6-2.

The fixing plates 6-1 and 6-2 comprise longitudinal side fixing plates IJji33-1 and QRrq33-2, transverse side fixing plates dVvc36-1, MNN 36-2 and inclined plane fixing plates effg 34-1, KLlk34-2, OPpo34-3 and STts34-4, and the fixing plates 6-1 and 6-2 fix the mounting bracket on the bottom mounting plane of the unmanned aerial vehicle; the longitudinal side fixing plates IJji33-1, QRrq33-2, the transverse side fixing plates dVvc36-1, MNN 36-2 and the inclined plane fixing plates efag 34-1, KLlk34-2, OPpo34-3, STts34-4 are on one plane and parallel to the bottom plate ABCDEFGH 30.

Let the distance between the plane of the fixing plates 6-1, 6-2 and the bottom plate 30 be u, u being greater than 2yw.

See fig. 4-10.

The size of the bottom plate ABCDEFGH30 is determined according to the size of the unmanned aerial vehicle; AB and EF are parallel to each other, and the length AB is set to be a times the length xw of the sensing surface of the image sensor, i.e., ab=ef= axw, and a is required to be greater than 3; cd=hg=ayw; the CD edge is parallel to the HG edge and is perpendicular to the AB and EF edges; becc, AHGF are isosceles trapezoids, ah=gf=bc=ed=aw (x+y)/2;

AF＝BE

the longitudinal side panels ABJI31-1 and EFQR31-2 are the same in size, and the transverse side panel ABJI31-1 is taken as an example for illustration: the longitudinal side panel ABJI31-1 is rectangular, wherein ab=ij= axw

The lateral side panels GHTD 35-1 and CDNM35-2 are the same in size, and the lateral side panel CDNM35-2 is exemplified

CD＝MN＝ayw；

The same dimensions of the inclined plate HAfe32-1, the inclined plate BCLK32-2, the inclined plate DEPO32-3 and the inclined plate FGTS32-4 are rectangular, and are described by the inclined plate BCLK32-2,

BC＝KL＝aw(x+y)/2；

the longitudinal side fixing plates IJji33-1 and the longitudinal side fixing plates QRrq33-2 are the same in size and are rectangular;

take the longitudinal side fixing plate iji 33-1 as an example;

ij=ji= axw; ii=ji, and Ii, jj are longer than 3 millimeters;

the transverse side fixing plate dVvc36-1 and the transverse side fixing plate MNN 36-2 are rectangular and have the same size; the side fixing plate MNN 36-2 is taken as an example for illustration;

mn=nm=ayw; mm=nn, and Mm, nn lengths are greater than 3 millimeters; the inclined plane fixing plate efhg34-1, the inclined plane fixing plate KLlk34-2, the inclined plane fixing plate OPpo34-3 and the inclined plane fixing plate STts34-4 are rectangular and have the same size; taking the bevel securing plate KLlk34-2 as an example,

KL＝lk＝aw(x+y)/2；

kk=ll, and Kk, ll are greater than 3 millimeters in length.

See fig. 11. The image processing system is composed of image sensing modules 8-1 to 8-9, a programmable gate array, a peripheral circuit 9, a wireless transmission module 10, a video compression processing chip and a peripheral circuit 11 thereof.

Nine image sensing modules 8-1 to 8-9 are respectively arranged on a bottom plate ABCDEFGH30, lateral side panels GHTD 35-1 and CDNM35-2, longitudinal side panels ABJI31-1 and EFQR31-2, inclined panels HAfe32-1, BCLK32-2, DEPO32-3 and FGTS32-4, and are connected with a programmable gate array and a peripheral circuit through control signal lines and pixel brightness signal connecting lines.

The programmable gate array and the peripheral circuit 9 are connected with nine image sensing modules 8-1 to 8-9 through control signal lines and pixel brightness signal connection lines, are connected with a wireless transmission module 10 through communication connection lines, and are connected with a video compression signal connection line, a video compression processing chip and a peripheral circuit 11 thereof through video output signal connection lines.

The wireless transmission module 10 is connected with the programmable gate array and the peripheral circuit 9 through communication connection wires, and transmits the data processed by the programmable gate array and the peripheral circuit 9 to a remote place through a wireless transmission mode.

The video compression processing chip and its peripheral circuit 11 are connected to the programmable gate array and the peripheral circuit 9 via video output signal connection lines and video compression signal connection lines.

See fig. 12. The image sensing module is composed of a lens 12, a mounting case 13, and a sensing circuit 14.

The mounting box is a sealed and light-tight box and is a cube or a cylinder, and the mounting box is provided with two parallel planes, one plane is used for mounting a lens, and the other plane is used for being fixed on a bottom plate, a side plate or a bevel panel. The center of the plane for installing the lens is provided with a circular inner wire opening for installing the lens, and the diameter of the wire opening is the same as that of the outer wire opening of the lens.

The lens is a fixed focus lens, and different focal length lenses are selected according to the requirement; the specifications of the nine lenses are identical.

The sensing circuit is used for sensing the image of the lens, the sensing circuit is provided with an image sensor, the sensing plane of the image sensor on the sensing circuit is ensured to be parallel to the lens, and the center of the lens on the image sensor is projected on the center of the image sensor; when the installation box is installed on the bottom plate, the side panel or the inclined panel, the sensing plane of the image sensor is ensured to be parallel to the plane where the bottom plate, the side panel or the inclined panel is located.

An image sensor module mounted on the chassis ABCDEFGH30, the longitudinal sides of the image sensor being parallel to the AB sides. The exact center of the image sensor is aligned with the exact center of the bottom plate.

The image sensor comprises 2 image sensing modules which are arranged on 2 longitudinal side panels ABJI31-1 and EFQR31-2, and the longitudinal sides of the image sensor are respectively parallel to the AB side and the EF side; the center of the image sensor is aligned with the center of the side panel.

2 image sensing modules mounted on the 2 lateral side panels GHTD 35-1 and CDNM35-2, and the lateral sides of the image sensor are respectively parallel to the CD and GH sides; the center of the image sensor is aligned with the center of the side panel.

The longitudinal sides of the image sensor and the BC side, the DE side, the FG side and the HA side of the inclined panel are arranged at an angle arctan (y/x) degree; the exact center of the image sensor is aligned with the exact center of the diagonal panels.

In this embodiment, the image sensor is ICX205 of SONY corporation; x=1360, y=1024, w=4.65 um, let u=4 yw=19.0 mm.

The lens was an industrial lens of Tenglong, japan Co. Model: TAMRON 17HF; focal length: 16mm; namely: z=16 mm;

the mounting bracket in this embodiment is made of a lightweight aluminum alloy, and the side lengths of the panels are as follows:

the mounting bracket is made of light aluminum alloy, and the side length of each panel is calculated as follows:

a＝10；

AB＝IJ＝EF＝QR＝axw＝10*1024*4.65um＝63.2mm

CD＝MN＝HG＝Vd＝BD＝ayw＝10*1360*4.65um＝47.6mm

/>

AH＝GF＝BC＝ED＝aw(x+y)/2＝55.4mm；

the first to fourth portions of the circuit diagram of the programmable gate array in the peripheral circuit of this embodiment are shown in fig. 15 to 18. U1, X3S1200, is a programmable gate array, xilnx company.

The video compression processing circuit of the image compression and its peripheral circuits is shown in fig. 19 and 20. The video compression chip U12 is manufactured by TOKYO company and is model TE3310.

Fig. 21 is an image sensor control circuit in an image sensor and its peripheral circuits.

U6: TI company, model: SN74LVC16245.

The image sensor circuit is shown in fig. 22 and 23. The UCI is manufactured by SONY company and is model ICX205.

Fig. 24 is a wireless communication module circuit. U7 is CC2520, a wireless transmission chip/manufactured by TEXAS INSTRUMENTS company of the United states. U14, CC2591, wireless transmission front-end chip/manufactured by TEXAS INSTRUMENTS company of the United states.

The image processing method of the nine-lens unmanned aerial vehicle panoramic camera comprises the following steps:

the programmable gate array and the peripheral circuit 9 uniformly control the working modes of the image sensing modules 8-1 to 8-9, receive the signals of the image sensing modules, process the signals of the image sensing modules, send the processed image sensing data to the video compression processing chip and the peripheral circuit 11 thereof, receive the image data compressed by the video compression processing chip and the peripheral circuit thereof, and send the compressed image data to the wireless transmission module 10 for wireless transmission.

See fig. 13.

Program flow of programmable gate array:

the first step: the same control signals are sent to the nine image sensors, and the sent image sensor control signals are as follows: a field synchronizing signal, a line synchronizing signal, a clock signal, a driving timing signal, an exposure control signal, and a second step;

and a second step of: receiving pixel brightness signals of nine image sensors, and turning to a third step;

and a third step of: performing BAYER image processing and white balance processing on pixel brightness signals of the nine image sensors to obtain RGB signals of each pixel after the white balance processing, and turning to a fourth step;

fourth step: calling an image stitching subroutine, and turning to a fifth step;

fifth step: sending the spliced RGB format image signals to a video compression processing chip, and turning to a sixth step;

sixth step: receiving compressed image data from a video compression processing chip, and turning to a seventh step;

seventh step: and (3) sending the received compressed image data to a wireless transmission module for wireless transmission, and turning to the first step.

See fig. 14.

The image stitching subroutine is carried out according to the following steps: setting PX longitudinal pixels and PY transverse pixels of the spliced image, and sequentially sequencing the longitudinal pixels from left to right according to natural numbers, wherein sequencing numbers are 1, 2, 3, 4, … …, PX-1 and PX; the transverse pixels are sequentially ordered according to natural numbers from top to bottom and are ordered into 1, 2, 3, 4, … …, PY-1 and PY; image stitching subroutine of pixels:

the first step: px=x+3x, py=3y; entering a second step;

and a second step of: the images are spliced as follows:

RGB data obtained by the image sensor installed on the bottom plate are spliced in a section from x+1 to 2x of longitudinal pixel serial numbers and from y+1 to 2y of transverse pixel serial numbers;

RGB data obtained by an image sensor installed on the longitudinal side panel ABJI31-1 are spliced in a section from x+1 to 2x of longitudinal pixel serial numbers and from 1 to y of transverse pixel serial numbers;

RGB data obtained by the image sensor installed on the longitudinal side panel EFQR31-2 are spliced in a section from x+1 to 2x of longitudinal pixel sequence numbers and from 2y+1 to 3y of transverse pixel sequence numbers;

RGB data obtained by the image sensor installed on the inclined panel HAfe32-1 are spliced in a section from 1 to x of longitudinal pixel serial numbers and 1 to y of transverse pixel serial numbers;

RGB data obtained by the image sensor installed on the inclined panel BCLK32-2 are spliced in a section from the longitudinal pixel sequence number 2x+1 to 3x and the transverse pixel sequence number from 1 to y;

RGB data obtained by the image sensor installed on the inclined panel DEPO32-3 are spliced in a section from 2x+1 to 3x of longitudinal pixel sequence numbers and from 2y+1 to 3y of transverse pixel sequence numbers;

RGB data obtained by the image sensor installed on the inclined panel FGTS32-4 are spliced in a section from 1 to x of longitudinal pixel serial numbers and from 2y+1 to 3y of transverse pixel serial numbers;

the RGB data obtained by the image sensor installed on the GHTD of the lateral side panel are spliced in a section from 1 to x of longitudinal pixel serial numbers and from y+1 to 2y of lateral pixel serial numbers;

RGB data obtained by an image sensor installed on a lateral side panel CDNM are spliced in a section from 2x+1 to 3x of longitudinal pixel serial numbers and from y+1 to 2y of lateral pixel serial numbers;

and a third step of: and returning to the main program after the images are spliced.

Claims (3)

1. Nine camera lens unmanned aerial vehicle panoramic camera, its characterized in that: the panoramic camera consists of two parts, a mounting bracket and an image processing system; the image processing system is arranged on the mounting bracket; the mounting bracket is fixed at the bottom of the unmanned aerial vehicle;

the image processing system comprises 9 identical image sensors and 9 fixed focus lenses with identical specifications, wherein the pixels sensed by the image sensors are longitudinally x pixels, and transversely y pixels (x > y), and the length and the width of the sensing pixels are w; setting the focal length of the focal lens as z, and setting the length of the sensing surface of the image sensor as xw and the width as yw;

order the

I.e., a is equal to twice the arctangent function of 0.5xw/z, β is equal to twice the arctangent function of 0.5yw/z,

units of α and β are degrees;

the mounting bracket is formed by connecting a plurality of flat plates and comprises a bottom plate ABCDEFGH (30), side panels, inclined panels and a fixed plate; the side panels include longitudinal side panels and transverse side panels;

the fixing plate comprises a longitudinal side fixing plate, an inclined plane fixing plate and a transverse side fixing plate, and the longitudinal side fixing plate, the inclined plane fixing plate and the transverse side fixing plate jointly form fixing plates (6-1, 6-2);

the side panels and the inclined panels form support plates (7-1, 7-2);

the image sensing modules are arranged on the supporting plates (7-1, 7-2) and the bottom plate ABCDEFGH (30), and the supporting plates (7-1, 7-2) are connected with the bottom plate ABCDEFGH (30) and the fixing plates (6-1, 6-2);

the mounting brackets of the fixing plates (6-1, 6-2) are fixed on the bottom mounting plane of the unmanned aerial vehicle; all the fixing plates are on one plane and parallel to the bottom plate ABCDEFGH (30);

the distance between the plane where the fixing plates (6-1, 6-2) are positioned and the bottom plate ABCDEFGH (30) is set as u, and u is larger than 2yw; when in installation, the bottom plate is parallel to the installation plane at the bottom of the unmanned aerial vehicle;

the size of the bottom panel ABCDEFGH (30) is determined according to the size of the unmanned aerial vehicle; AB and EF are mutually parallel, ABEF is rectangular, the length AB is set to be a times the length xw of the sensing surface of the image sensor, namely ab=ef= axw, and a is required to be larger than 3; cd=hg=ayw; the CD edge is parallel to the HG edge and is perpendicular to the AB and EF edges; becc, AHGF are isosceles trapezoids, ah=gf=bc=ed=aw (x+y)/2;

AF＝BE

longitudinal side panels ABJI (31-1) and EFQR (31-2) are the same in size, and longitudinal side panel ABJI (31-1) is illustrated as a rectangular shape, wherein ab=ij= axw

The lateral side panel GHTD (35-1) and the lateral side panel CDNM (35-2) are the same in size, and the lateral side panel CDNM (35-2) is taken as an example for illustration

CD＝MN＝ayw；

The inclined plate HAfe (32-1), the inclined plate BCLK (32-2), the inclined plate DEPO (32-3) and the inclined plate FGTS (32-4) are rectangular in size, and the inclined plate BCLK (32-2) is taken as an example for explanation,

BC＝KL＝aw(x+y)/2；

the longitudinal side fixing plate IJji (33-1) and the longitudinal side fixing plate QRrq (33-2) are rectangular and have the same size; take the longitudinal side fixing plate iji (33-1) as an example;

ij=ji= axw; ii=ji, and Ii, jj are longer than 3 millimeters;

the transverse side fixing plate dVvc (36-1) and the transverse side fixing plate MNN (36-2) are rectangular and have the same size; the side fixing plate MNNM (36-2) is exemplified;

mn=nm=ayw; mm=nn, and Mm, nn lengths are greater than 3 millimeters;

the inclined plane fixing plates efhg (34-1), the inclined plane fixing plate KLlk (34-2), the inclined plane fixing plate OPpo (34-3) and the inclined plane fixing plate STts (34-4) are rectangular and have the same size; taking the bevel securing plate KLlk (34-2) as an example,

KL＝lk＝aw(x+y)/2；

kk=ll, and Kk, ll lengths are greater than 3 millimeters;

the side panels include longitudinal side panels and transverse side panels refer to: longitudinal side panel ABJI (31-1), longitudinal side panel EFQR (31-2), transverse side panel GHTD (35-1), transverse side panel CDNM (35-2); the inclined panel refers to an inclined panel HAfe (32-1), an inclined panel BCLK (32-2), an inclined panel DEPO (32-3) and an inclined panel FGTS (32-4);

the fixed plate comprises a longitudinal side fixed plate, an inclined plane fixed plate and a transverse side fixed plate, and is characterized in that: longitudinal side fixing plate IJji (33-1), longitudinal side fixing plate QRrq (33-2), inclined fixing plate efag (34-1), inclined fixing plate KLlk (34-2), inclined fixing plate OPpo (34-3), inclined fixing plate STts (34-4), lateral side plate GHTD (35-1), lateral side plate CDNM (35-2), lateral side fixing plate dVvc (36-1), lateral side fixing plate MNN (36-2);

the supporting plates (7-1, 7-2) comprise inclined plates HAfe (32-1), inclined plates BCLK (32-2), inclined plates DEPO (32-3), inclined plates FGTS (32-4), longitudinal side plates ABJI (31-1), longitudinal side plates EFQR (31-2) and transverse side plates GHVd (35-1), transverse side plates CDNM (35-2), the image sensing modules are arranged on the supporting plates (7-1, 7-2) and the bottom plate ABCDEFGH (30), and the supporting plates (7-1, 7-2) are connected with the bottom plate ABCDEFGH (30) and the fixing plates (6-1, 6-2);

all the fixing plates are in one plane and parallel to the bottom plate ABCDEFGH (30) means: longitudinal side fixing plate IJji (33-1), longitudinal side fixing plate QRrq (33-2), transverse side fixing plate dVvc (36-1), transverse side fixing plate MNN (36-2) and inclined fixing plate effg (34-1), inclined fixing plate KLlk (34-2), inclined fixing plate OPpo (34-3), inclined fixing plate STts (34-4) are on one plane and parallel to bottom plate ABCDEFGH (30).

2. The nine-lens unmanned aerial vehicle panoramic camera of claim 1, wherein:

the image processing system consists of an image sensing module (8-1-8-9), a programmable gate array, a peripheral circuit (9), a wireless transmission module (10), a video compression processing chip and a peripheral circuit (11) thereof;

nine image sensing modules (8-1-8-9) are respectively arranged on a bottom plate ABCDEFGH (30), a lateral side panel GHTD (35-1), a lateral side panel CDNM (35-2), a longitudinal side panel ABJI (31-1), a longitudinal side panel EFQR (31-2), a diagonal panel HAfe (32-1), a diagonal panel BCLK (32-2), a diagonal panel DEPO (32-3) and a diagonal panel FGTS (32-4), and are connected with a programmable gate array and a peripheral circuit through control signal lines and pixel brightness signal connecting lines;

the programmable gate array and the peripheral circuit (9) are connected with nine image sensing modules (8-1-8-9) through control signal lines and pixel brightness signal connecting lines, are connected with the wireless transmission module (10) through communication connecting lines, and are connected with the video compression processing chip and the peripheral circuit (11) thereof through video output signal connecting lines;

the wireless transmission module (10) is connected with the programmable gate array and the peripheral circuit (9) through communication connection wires, and transmits the data processed by the programmable gate array and the peripheral circuit (9) to a remote place in a wireless transmission mode;

the video compression processing chip and the peripheral circuit (11) thereof are connected with the programmable gate array and the peripheral circuit (9) through video output signal connecting wires and video compression signal connecting wires;

the image sensing module consists of a lens (12), a mounting box (13) and a sensing circuit (14);

the mounting box is a sealed and light-tight box and is a cube or a cylinder, and is provided with two parallel planes, one plane is used for mounting a lens, and the other plane is used for being fixed on the bottom plate, the side plate or the inclined panel; a circular inner wire opening for installing the lens is formed in the center of the plane for installing the lens, and the diameter of the wire opening is the same as that of the outer wire opening of the lens;

the lens is a fixed focus lens, and different focal length lenses are selected according to the requirement; the specifications of the nine lenses are identical;

the sensing circuit is used for sensing the image of the lens, the sensing circuit is provided with an image sensor, the sensing plane of the image sensor on the sensing circuit is ensured to be parallel to the lens, and the center of the lens on the image sensor is projected on the center of the image sensor; when the mounting box is mounted on the bottom plate, the side panel or the inclined panel, the sensing plane of the image sensor is ensured to be parallel to the plane of the bottom plate, the side panel or the inclined panel;

an image sensing module mounted on the chassis ABCDEFGH (30), a longitudinal side of the image sensor being parallel to the AB side; the center of the image sensor is aligned with the center of the bottom plate;

2 image sensing modules mounted on 2 longitudinal side panels ABJI (31-1) and EFQR (31-2), and longitudinal sides of the image sensor are parallel to AB and EF sides respectively; the center of the image sensor is aligned with the center of the side panel;

2 image sensing modules mounted on 2 lateral side panels GHTD (35-1) and CDNM (35-2), and lateral sides of the image sensor are parallel to the CD and GH sides respectively; the center of the image sensor is aligned with the center of the side panel

4 image sensing modules mounted on 4 inclined panels HAfe (32-1), BCLK (32-2), DEPO (32-3) and FGTS (32-4), wherein the included angles between the longitudinal sides of the image sensor and the BC sides, DE sides, FG sides and HA sides of the inclined panels mounted on the image sensor are arctan (y/x) degrees; the exact center of the image sensor is aligned with the exact center of the diagonal panels.

3. The image processing method of the nine-lens unmanned aerial vehicle panoramic camera of claim 1, wherein:

the programmable gate array and the peripheral circuit (9) uniformly control the working modes of the image sensing modules (8-1-8-9), receive the signals of the image sensing modules, process the signals of the image sensing modules, send the processed image sensing data to the video compression processing chip and the peripheral circuit (11) thereof, receive the image data compressed by the video compression processing chip and the peripheral circuit thereof, and send the compressed image data to the wireless transmission module (10) for wireless transmission;

programmable gate array program flow:

the first step: the same control signals are sent to the nine image sensors, and the sent image sensor control signals are as follows: a field synchronizing signal, a line synchronizing signal, a clock signal, a driving timing signal, an exposure control signal, and a second step;

and a second step of: receiving pixel brightness signals of nine image sensors, and turning to a third step;

and a third step of: performing BAYER image processing and white balance processing on pixel brightness signals of the nine image sensors to obtain RGB signals of each pixel after the white balance processing, and turning to a fourth step;

fourth step: calling an image stitching subroutine, and turning to a fifth step;

fifth step: sending the spliced RGB format image signals to a video compression processing chip, and turning to a sixth step;

sixth step: receiving compressed image data from a video compression processing chip, and turning to a seventh step; seventh step: the received compressed image data is sent to a wireless transmission module for wireless transmission, and the first step is switched to;

the image stitching subroutine is carried out according to the following steps: setting PX longitudinal pixels and PY transverse pixels of the spliced image, and sequentially sequencing the longitudinal pixels from left to right according to natural numbers, wherein sequencing numbers are 1, 2, 3, 4, … …, PX-1 and PX; the transverse pixels are sequentially ordered according to natural numbers from top to bottom and are ordered into 1, 2, 3, 4, … …, PY-1 and PY; image stitching subroutine of pixels:

the first step: px=3x, py=3y; entering a second step;

and a second step of: the images are spliced as follows:

RGB data obtained by the image sensor installed on the bottom plate are spliced in a section from x+1 to 2x of longitudinal pixel serial numbers and from y+1 to 2y of transverse pixel serial numbers;

RGB data obtained by an image sensor installed on a longitudinal side panel ABJI is spliced on a longitudinal pixel serial number slave

x+1 to 2x, the lateral pixel sequence number is in the interval from 1 to y;

RGB data obtained by an EFQR-mounted image sensor of a longitudinal side panel are spliced on a longitudinal pixel serial number slave

x+1 to 2x, the lateral pixel sequence number is within the interval from 2y+1 to 3y;

RGB data obtained by the image sensor mounted on the inclined panel HAfe are spliced in the longitudinal pixel sequence number from 1 to

x, the transverse pixel sequence number is in the interval from 1 to y;

RGB data obtained by the image sensor installed on the inclined panel BCLK is spliced with the vertical pixel serial number from 2x+1

To 3x, the lateral pixel sequence number is in the interval from 1 to y;

RGB data obtained by the image sensor installed on the inclined panel DEPO are spliced in the longitudinal pixel sequence number from 2x+1

To 3x, the lateral pixel sequence number is in the interval 2y+1 to 3y;

RGB data obtained by the image sensor mounted on the inclined panel FGTS is spliced with the vertical pixel serial number from 1 to

x, the lateral pixel sequence number is within the interval from 2y+1 to 3y;

the RGB data obtained by the image sensor installed on the GHTD of the lateral side panel are spliced in a section from 1 to x of longitudinal pixel serial numbers and from y+1 to 2y of lateral pixel serial numbers;

RGB data obtained by an image sensor installed on a lateral side panel CDNM are spliced in a section from 2x+1 to 3x of longitudinal pixel serial numbers and from y+1 to 2y of lateral pixel serial numbers;

and a third step of: and returning to the main program after the images are spliced.

Images