CN117699082A

CN117699082A - Unmanned aerial vehicle system for autonomous light and small photoelectric panoramic observation

Info

Publication number: CN117699082A
Application number: CN202311735641.7A
Authority: CN
Inventors: 高山; 汪饶; 胡航; 王立昂; 孙楚雁
Original assignee: 717Th Research Institute of CSSC
Current assignee: 717Th Research Institute of CSSC
Priority date: 2023-12-14
Filing date: 2023-12-14
Publication date: 2024-03-15

Abstract

The invention relates to an unmanned aerial vehicle system for autonomous light and small photoelectric panoramic observation, which comprises an unmanned aerial vehicle, a photoelectric pod and a control module, wherein the unmanned aerial vehicle is respectively in communication connection with the photoelectric pod and the control module and is used for acquiring control instructions and analyzing flight task information and panoramic observation task information, and executing corresponding types of flight tasks according to the flight task information, current posture information and current position information of the unmanned aerial vehicle and sending out unmanned aerial vehicle state information; the photoelectric pod is arranged on the unmanned aerial vehicle and is in communication connection with the unmanned aerial vehicle, and is used for acquiring photoelectric observation information of a corresponding type according to the panoramic observation task information; the control module is in communication connection with the unmanned aerial vehicle, and is used for sending control instructions, receiving unmanned aerial vehicle state information and photoelectric observation information, and adjusting the control instructions according to the unmanned aerial vehicle state information and the photoelectric observation information. According to the invention, panoramic observation is realized without depending on a flying hand, so that the labor is saved, and the working efficiency is improved.

Description

Unmanned aerial vehicle system for autonomous light and small photoelectric panoramic observation

Technical Field

The invention relates to the technical field of unmanned aerial vehicles, in particular to an autonomous light and small photoelectric panoramic observation unmanned aerial vehicle system.

Background

The unmanned aerial vehicle is an unmanned aerial vehicle which is operated by using a radio remote control device and a self-contained program control device. Compared with manned aircraft, it has the advantages of small size, low cost, convenient use, low environmental requirement, high flexibility, etc. With the rapid development of the unmanned aerial vehicle industry, unmanned aerial vehicles have been applied to various industries, such as environmental inspection, travel shooting, agricultural sprinkling, express transportation, and the like. However, most unmanned aerial vehicles used for civil use currently require specialized unmanned aerial vehicle flight personnel (hereinafter simply referred to as a flight crew) to perform operations to complete the flight and additional tasks. For example, the task of an optoelectronic unmanned aerial vehicle to perform observations such as patrol, surveillance, and tracking requires that the unmanned aerial vehicle's flight crew be involved in the flight and optoelectronic operations throughout. Because the degree of dependence on the flying hands is higher, the intelligent degree and the working efficiency of completing the expected task by adopting the unmanned aerial vehicle at the present stage still have larger lifting space.

Disclosure of Invention

Aiming at the technical problems in the prior art, the invention provides an unmanned aerial vehicle system for autonomous light and small photoelectric panoramic observation, so as to improve the intelligent degree and the working efficiency of photoelectric information acquisition by using an unmanned aerial vehicle.

The technical scheme for solving the technical problems is as follows: an unmanned aerial vehicle system of independently light small-size photoelectricity panorama is surveyd, includes unmanned aerial vehicle, photoelectricity nacelle and control module, wherein:

the unmanned aerial vehicle is respectively in communication connection with the photoelectric pod and the control module and is used for acquiring control instructions and analyzing flight task information and panoramic observation task information; the system is also used for executing the corresponding type of flight task according to the flight task information, the current gesture information and the current position information of the unmanned aerial vehicle and sending out the state information of the unmanned aerial vehicle;

the photoelectric pod is arranged on the unmanned aerial vehicle and is in communication connection with the unmanned aerial vehicle, and is used for acquiring photoelectric observation information of a corresponding type according to the panoramic observation task information;

the control module is in communication connection with the unmanned aerial vehicle, and is used for sending out control instructions, receiving unmanned aerial vehicle state information and photoelectric observation information, and adjusting the control instructions according to the unmanned aerial vehicle state information and the photoelectric observation information.

On the basis of the technical scheme, the invention can be improved as follows.

Further, the control module comprises a remote control module and an industrial personal computer, wherein the remote control module is respectively in communication connection with the unmanned aerial vehicle and the industrial personal computer, and is used for transmitting information interacted between the unmanned aerial vehicle and the industrial personal computer, and also is used for providing a manual control mode for the unmanned aerial vehicle; the industrial personal computer is used for sending out control instructions, receiving unmanned aerial vehicle state information and photoelectric observation information, and adjusting the control instructions according to the unmanned aerial vehicle state information and the photoelectric observation information.

Further, the types of flight tasks include a fixed point hover mode and a course flight mode, wherein,

the fixed point hover mode includes: acquiring current attitude information and current position information of the unmanned aerial vehicle, and controlling the unmanned aerial vehicle to keep in the current attitude and the current position according to the current attitude information and the current position information;

the airline flight mode includes: and acquiring a route instruction from the flight task information, acquiring a plurality of waypoint position information from the route instruction, sequencing, reading first flight speed information from the route instruction, sequentially flying through each waypoint position according to the first flight speed information and the sequencing of the waypoints, and switching to the fixed-point hovering mode at the last waypoint position.

Further, the types of panoramic observation tasks include a panoramic stitching mode, a region search mode, and/or a target tracking mode, wherein:

when the panoramic stitching mode is executed, the type of the flight task is a fixed-point hovering mode;

when the area search mode is executed, the type of the flight task is a fixed-point hovering mode;

the type of the flight mission is a fixed point hover mode and/or a course flight mode when the target tracking mode is performed.

Further, the panorama stitching mode includes the following steps:

acquiring a panoramic stitching instruction from the panoramic observation task information, reading the electric quantity of the unmanned aerial vehicle and the type of the current flight task according to the unmanned aerial vehicle state information, if the electric quantity of the unmanned aerial vehicle is higher than an electric quantity threshold value and the unmanned aerial vehicle is in a fixed-point hovering mode, entering the panoramic stitching mode, otherwise, reporting back that the panoramic stitching instruction is invalid;

analyzing second flight speed information, hover point position information of the unmanned aerial vehicle and a photoelectric zeroing instruction from the panoramic stitching instruction;

controlling the unmanned aerial vehicle to fly to reach the hovering point of the unmanned aerial vehicle according to the second flying speed information and the hovering point position information of the unmanned aerial vehicle, and controlling the photoelectric pod to point to a preset azimuth angle and a preset pitch angle according to the photoelectric zeroing instruction;

acquiring real-time position information of the unmanned aerial vehicle from the unmanned aerial vehicle state information, and starting an azimuth scanning instruction and an image splicing instruction if the real-time position information of the unmanned aerial vehicle is consistent with hover point position information of the unmanned aerial vehicle;

and controlling the circumferential rotation of the optoelectronic pod according to the azimuth scanning instruction, acquiring image information of continuous frames, reading corresponding view angles, azimuth angles and pitch angles in the image information according to the image splicing instruction, calculating image projections according to the view angles, azimuth angles and pitch angles of the images, calculating characteristic points in each frame of image according to the image projections, correcting the characteristic points, and splicing according to the characteristic points to obtain a panoramic image.

Further, the area search mode includes the steps of:

acquiring an area search instruction from the panoramic observation task information, reading the electric quantity of the unmanned aerial vehicle and the type of the current flight task according to the unmanned aerial vehicle state information, if the electric quantity of the unmanned aerial vehicle is higher than an electric quantity threshold value and the unmanned aerial vehicle is in a fixed-point hovering mode, entering the area search mode, otherwise, notifying that the area search instruction is invalid;

acquiring position information of each vertex of a search area from the area search instruction, calculating position information of the center of the search area according to the position information of each vertex, and sending a first geographic follow-up instruction to the optoelectronic pod according to the position information of the center of the search area;

the photoelectric pod enters a first geographic follow-up mode according to the first geographic follow-up instruction, and the azimuth angle and the pitch angle of the photoelectric pod are adjusted according to the first geographic follow-up instruction, so that the field of view of the photoelectric pod covers the central position of the searching area;

acquiring real-time position information and real-time attitude information of the unmanned aerial vehicle from the unmanned aerial vehicle state information, and calculating the distance from the photoelectric pod to the central position of the search area according to the real-time position information and the real-time attitude information of the unmanned aerial vehicle;

Calculating an optimal field angle of the photoelectric pod according to the calculated distance and the position information of each vertex, wherein the optimal field angle meets the following conditions: the photoelectric imaging range of the photoelectric pod comprises a search area, and the optimal field angle is as small as possible;

and taking the optimal field angle as an adjustment target, controlling the photoelectric pod to adjust the field angle to the optimal field angle, and acquiring image information in the optimal field angle range.

Further, the target tracking mode includes the steps of:

acquiring a target tracking instruction from the panoramic observation task information, reading the electric quantity of the unmanned aerial vehicle and the type of the current flight task according to the unmanned aerial vehicle state information, if the electric quantity of the unmanned aerial vehicle is higher than an electric quantity threshold value and the unmanned aerial vehicle is in a fixed-point hovering mode, entering the target tracking mode, otherwise, notifying that the target tracking instruction is invalid;

acquiring the position information of the target to be tracked from the target tracking instruction, and sending a second geographic follow-up instruction to the optoelectronic pod according to the position information of the target to be tracked;

the photoelectric pod enters a second geographic follow-up mode according to the second geographic follow-up instruction, and the azimuth angle and the pitch angle of the photoelectric pod are adjusted according to the second geographic follow-up instruction so that the position of the tracking target is located at the center of the field of view of the photoelectric pod;

Sending a target tracking instruction to the photoelectric pod, wherein the photoelectric pod enters an image tracking mode according to the target tracking instruction so as to track a target to be tracked in an image center acquired by the photoelectric pod;

acquiring real-time position information and real-time attitude information of the unmanned aerial vehicle from the unmanned aerial vehicle state information, and calculating the slant distance from the photoelectric pod to the tracking target according to the real-time position information and the real-time attitude information of the unmanned aerial vehicle;

if the slant distance is larger than the distance threshold, calculating a connecting line midpoint between the photoelectric pod and the tracking target, taking longitude of the connecting line midpoint, latitude of the connecting line midpoint and current height information of the unmanned aerial vehicle as position information of the target waypoint, and hovering after the unmanned aerial vehicle flies to reach the position of the target waypoint by adopting a preset default speed.

Further, the photoelectric pod is connected with the unmanned aerial vehicle through a conductive slip ring, a pitching mechanism is arranged on the photoelectric pod, and a photoelectric observation information acquisition end of the photoelectric pod is arranged on the pitching mechanism.

Further, the photoelectric observation information acquisition end of the photoelectric pod comprises a visible light image acquisition module, an infrared image acquisition module and/or a laser detection module which are/is arranged in the photoelectric pod, wherein the visible light image acquisition module is used for acquiring visible light image information in a view field range, the infrared image acquisition module is used for acquiring infrared image information in the view field range, and the laser detection module is used for acquiring laser scanning information in the view field range.

Further, be equipped with first wireless communication module, inertial navigation system and satellite navigation system on the unmanned aerial vehicle, first wireless communication module be used for with control module carries out communication connection, inertial navigation system is used for obtaining unmanned aerial vehicle real-time gesture information, satellite navigation system is used for obtaining unmanned aerial vehicle real-time position information.

The beneficial effects of the invention are as follows:

the unmanned aerial vehicle system for autonomous light and small photoelectric panoramic observation provided by the invention meets the application requirement of controlling the light and small unmanned aerial vehicle to carry out panoramic observation by using the instruction under the condition of not depending on a flying hand. The system can realize that all unmanned aerial vehicle flies are independent, and the photoelectric panoramic observation task is executed by all unmanned aerial vehicles, the panoramic observation function is realized by a preset instruction, the manpower is saved, the unmanned aerial vehicle and the photoelectric pod are not required to be operated by the flies in real time, and the work efficiency of task execution is improved.

Drawings

Fig. 1 is a schematic diagram of a usage scenario of an autonomous light-small-sized photoelectric panoramic observation unmanned aerial vehicle system according to an embodiment of the present invention;

fig. 2 is a schematic cross-linking relationship diagram of an autonomous light-small photoelectric panoramic observation unmanned aerial vehicle system according to an embodiment of the present invention.

In the drawings, the list of components represented by the various numbers is as follows:

1. unmanned aerial vehicle, 2, photoelectricity nacelle, 3, remote control unit, 4, industrial computer.

Detailed Description

The principles and features of the present invention are described below with reference to the drawings, the examples are illustrated for the purpose of illustrating the invention and are not to be construed as limiting the scope of the invention.

Referring to fig. 1, fig. 1 is a schematic diagram of an usage scenario of an autonomous light-small photoelectric panoramic observation unmanned aerial vehicle system according to an embodiment of the present invention. As shown in fig. 1, the unmanned aerial vehicle system for autonomous light and small photoelectric panoramic observation provided by the embodiment of the invention comprises an unmanned aerial vehicle 1, a photoelectric pod 2 and a control module, wherein:

The control module is connected with the unmanned aerial vehicle in a wireless communication manner, and is used for sending out control instructions, receiving unmanned aerial vehicle state information and photoelectric observation information, and adjusting the control instructions according to the unmanned aerial vehicle state information and the photoelectric observation information.

It will be appreciated that the optoelectronic pod is an aircraft mounted optoelectronic sensing device that can be used to acquire image information over a field of view. The photoelectric pod obtains power supply and flight state information of the unmanned aerial vehicle from the unmanned aerial vehicle end, and outputs video, image information and state information of the photoelectric pod to the unmanned aerial vehicle end.

On the basis of the technical scheme, the invention can be improved as follows.

As shown in fig. 1 and 2, the control module includes a remote control module 3 and an industrial personal computer 4, where the remote control module 3 is respectively in communication connection with the unmanned aerial vehicle 1 and the industrial personal computer 4, and is used for transmitting information interacted between the unmanned aerial vehicle 1 and the industrial personal computer 4, and also is used for providing a manual control mode for the unmanned aerial vehicle 1; the industrial personal computer 4 is used for sending out a control instruction, receiving unmanned aerial vehicle state information and photoelectric observation information, and adjusting the control instruction according to the unmanned aerial vehicle state information and the photoelectric observation information.

It can be understood that, in the state of unmanned aerial vehicle and photoelectric pod control independent of the flying hand, the remote control module mainly realizes the function of command transmission, and as a communication connection medium between the industrial personal computer and the unmanned aerial vehicle, the industrial personal computer is mainly used for processing the received data and sending out control commands; for safety reasons, the flight crew can take over the unmanned aerial vehicle through the remote control module in case of emergency, and at this time, the flight crew can directly control the unmanned aerial vehicle and the optoelectronic pod through the remote control module.

In one possible implementation manner, the photoelectric pod is connected with the unmanned aerial vehicle through a conductive slip ring so as to facilitate 360-degree azimuth rotation of the photoelectric pod, and thus, continuous frame image information of all azimuth angles in an observed area is obtained; the photoelectric nacelle is provided with a pitching mechanism, a photoelectric observation information acquisition end of the photoelectric nacelle is arranged on the pitching mechanism, and the pitching angle of the photoelectric nacelle can be adjusted through the pitching mechanism so as to enlarge the field of view.

Still further, the photoelectric observation information acquisition end of the photoelectric pod comprises a visible light image acquisition module, an infrared image acquisition module and/or a laser detection module which are/is arranged in the photoelectric pod, wherein the visible light image acquisition module is used for acquiring visible light image information in a view field range, the infrared image acquisition module is used for acquiring infrared image information in the view field range, and the laser detection module is used for acquiring laser scanning information, such as obstacle detection state information/laser ranging state information, in the view field range.

It can be understood that the observation of the period with better visibility in the observed area can be realized through the visible light image information acquired by the visible light image acquisition module; the infrared image information acquired by the infrared image acquisition module can realize the observation of the period with poor visibility in the observed area, such as cloudy days, rainy days, night and the like; the laser scanning information collected by the laser detection module can be used for measuring the distance of the target in the observed area, and the target can be more accurately identified and tracked by combining the visible light image information and/or the infrared image information.

In one possible implementation manner, the unmanned aerial vehicle is provided with a first wireless communication module, an inertial navigation system and a satellite navigation system, wherein the first wireless communication module is used for being in communication connection with a remote control module in the control module so as to realize information interaction between the unmanned aerial vehicle and a ground station; the inertial navigation system is used for acquiring real-time attitude information of the unmanned aerial vehicle, and the satellite navigation system is used for acquiring real-time position information of the unmanned aerial vehicle.

Fig. 2 is a schematic diagram showing an internal cross-linking relationship of an autonomous light-small photoelectric panoramic observation unmanned aerial vehicle system, and fig. 2 shows that the system forms a local area network through wireless and wired communication modes, and an industrial personal computer of a photoelectric pod, an unmanned aerial vehicle, a remote control module and a ground station respectively distributes different IP addresses with network segments to realize mutual communication.

With reference to fig. 1 and 2, the optoelectronic pod is mounted on an unmanned aerial vehicle, and a continuous zoom visible light television, a fixed focus infrared and a laser range finder are used for reconnaissance monitoring work. The optoelectronic pod transmits optoelectronic video and task attribute information (e.g., imaging related status) to the drone via a network protocol. The task attribute information at least comprises unmanned aerial vehicle state information, panoramic observation task types corresponding to received photoelectric videos and the like.

The unmanned aerial vehicle carries a photoelectric pod, a wireless transmission link, an inertial navigation sensor and a satellite navigation system. The unmanned aerial vehicle sends unmanned aerial vehicle state information (corresponding to telemetry data in fig. 2, including information such as airplane motion state and system state) to the optoelectronic pod through a serial port, the pod control information analyzed from the panoramic observation task information is transmitted to the optoelectronic pod through a network protocol, and the acquired telemetry data (such as unmanned aerial vehicle state information), optoelectronic video and task attribute information are transmitted to a remote control module through a network protocol of a wireless link.

The remote control module comprises a wireless transmission link antenna and a remote control interface. The remote control module transmits the control instruction (namely photoelectricity in fig. 2 and unmanned aerial vehicle remote control data) for controlling the unmanned aerial vehicle and the photoelectric pod to the unmanned aerial vehicle through a network protocol of a wireless link, and transmits photoelectricity video and corresponding task attribute information transmitted back by the unmanned aerial vehicle to the ground station through a wired network protocol. The flight task of the aircraft can be controlled by a control instruction preset on the industrial personal computer, and when an unexpected situation occurs, the aircraft can be switched to the aircraft to control the flight task of the aircraft by a flight hand through the remote control module.

And running photoelectric monitoring software on an industrial personal computer of the ground station to perform data processing, unmanned aerial vehicle flight control, photoelectric pod control and photoelectric data display on the photoelectric pod. The industrial personal computer of the ground station transmits photoelectric remote control data to the remote control module through a wired network protocol.

In one possible embodiment, the types of flight tasks include a fixed point hover mode and a course flight mode, wherein,

the fixed point hover mode includes: acquiring current attitude information and current position information (longitude, latitude and altitude) of the unmanned aerial vehicle through various sensors, navigation systems and the like carried by the unmanned aerial vehicle, and controlling the unmanned aerial vehicle to keep in the current attitude and the current position according to the current attitude information and the current position information;

It can be appreciated that the fixed point hover mode, i.e., the unmanned aerial vehicle obtains current location information (longitude, latitude, altitude) through sensors and navigation systems, and the unmanned aerial vehicle remains hovering at the current location through gesture control. The route flight mode is to fly according to a preset route. The fixed-point hovering mode and the route flying mode can be flexibly switched according to the panoramic observation task type.

In one embodiment, the type of the panoramic observation task includes a panoramic stitching mode, a region searching mode, and/or a target tracking mode, wherein:

It can be understood that the type of the panoramic observation task is matched with the type of the proper flight task, so that more accurate photoelectric observation information can be acquired.

Further, the panorama stitching mode includes the following steps:

acquiring a panoramic stitching instruction from the panoramic observation task information, reading the electric quantity of the unmanned aerial vehicle and the type of the current flight task according to the unmanned aerial vehicle state information, if the electric quantity of the unmanned aerial vehicle is higher than an electric quantity threshold value and the unmanned aerial vehicle is in a fixed-point hovering mode, entering the panoramic stitching mode, otherwise, returning the panoramic stitching instruction to a control module by the unmanned aerial vehicle;

Analyzing second flight speed information, hover point position information (longitude, latitude and altitude) of the unmanned aerial vehicle and a photoelectric zeroing instruction from the panoramic stitching instruction;

controlling the unmanned aerial vehicle to fly to reach the hovering point of the unmanned aerial vehicle according to the second flying speed information and the hovering point position information of the unmanned aerial vehicle, and controlling the direction of the photoelectric pod to be at a preset azimuth angle and a preset pitch angle, for example, at a preset azimuth angle of 0 DEG and a pitch angle of 0 DEG according to the photoelectric zeroing instruction;

acquiring real-time position information (longitude, latitude and altitude) of the unmanned aerial vehicle from the unmanned aerial vehicle state information, and starting an azimuth scanning instruction and an image splicing instruction if the real-time position information of the unmanned aerial vehicle is consistent with hover point position information (longitude, latitude and altitude) of the unmanned aerial vehicle;

and controlling the circumferential rotation of the optoelectronic pod according to the azimuth scanning instruction, acquiring image information of continuous frames, reading a view angle, an azimuth angle and a pitch angle corresponding to the images in the image information according to the image splicing instruction, calculating image projection according to the view angle, the azimuth angle and the pitch angle of the images, calculating characteristic points in each frame of images according to the image projection, correcting the characteristic points, and splicing according to the characteristic points to obtain a panoramic image.

It can be understood that the panoramic stitching mode in this embodiment refers to setting the unmanned aerial vehicle fixed-point hovering position by an instruction at the control module, enabling the photoelectric pod to move circumferentially independently according to the set instruction, stitching multiple frames of images to form images with larger images, for example, panoramic images with 360-degree continuous azimuth, and stitching and displaying the images. And the panoramic image is spliced by adopting a characteristic point splicing and correcting mode, so that the panoramic image with smaller distortion can be obtained.

Further, the area search mode includes the steps of:

acquiring an area search instruction from the panoramic observation task information, reading the electric quantity of the unmanned aerial vehicle and the type of the current flight task according to the unmanned aerial vehicle state information, if the electric quantity of the unmanned aerial vehicle is higher than an electric quantity threshold (for example, 30% of the rated electric quantity of a battery) and the unmanned aerial vehicle is in a fixed-point hovering mode, entering an area search mode, otherwise, returning the area search instruction to a control module by the unmanned aerial vehicle;

acquiring position information (longitude, latitude and altitude) of each vertex of a search area from the area search instruction, calculating search area center position information (longitude, latitude and altitude) according to the position information of each vertex, and sending a first geographic follow-up instruction to the optoelectronic pod according to the search area center position information;

The photoelectric pod enters a first geographic follow-up mode according to the first geographic follow-up instruction, and the azimuth angle and the pitch angle of the photoelectric pod are adjusted according to the first geographic follow-up instruction, so that the field of view of the photoelectric pod covers the central position of the searching area; more specifically, the first geographic follower mode includes: reading a target geographic position from a geographic follow-up instruction, and calculating an azimuth angle and a pitch angle of the unmanned aerial vehicle in real time by the photoelectric pod according to the target geographic position, the position information of the current unmanned aerial vehicle, the flight attitude of the current unmanned aerial vehicle and the self attitude of the current unmanned aerial vehicle, and adjusting the azimuth angle and the pitch angle to enable an imaging visual axis of the photoelectric pod to pass through (point to) the target geographic position;

calculating an optimal field angle of the electro-optical pod according to the calculated distance and position information (longitude, latitude and altitude) of each vertex, wherein the optimal field angle satisfies the following conditions: the photoelectric imaging range of the photoelectric pod comprises a search area, and the optimal field angle is as small as possible;

It can be understood that the area searching mode refers to setting a searching area through instructions, and the unmanned aerial vehicle onboard photoelectric pod automatically observes the searching area.

Further, the target tracking mode includes the steps of:

the photoelectric pod enters a second geographic follow-up mode according to the second geographic follow-up instruction, and the azimuth angle and the pitch angle of the photoelectric pod are adjusted according to the second geographic follow-up instruction so that the position of the tracking target is located at the center of the field of view of the photoelectric pod; the working principle of the second geographic follow-up mode is similar to that of the first geographic follow-up mode, and reference may be made to the working principle of the first geographic follow-up mode, which is not described herein again;

acquiring real-time position information and real-time attitude information of the unmanned aerial vehicle from the unmanned aerial vehicle state information, and calculating the slant distance from the photoelectric pod to the tracking target according to the real-time position information and the real-time attitude information of the unmanned aerial vehicle; the measurement of the inclined distance can utilize laser ranging data collected by the photoelectric pod;

It is understood that the target tracking refers to a function of setting a target tracking position by an instruction, and automatically detecting a target near the target tracking position by an unmanned aerial vehicle-mounted photoelectric pod to continuously track and observe the target. For the target tracking mode, the photoelectric pod receives a target tracking instruction and acquires a target image of an imaging center of the photoelectric pod. The photoelectric pod uses image tracking to acquire the coordinates of the tracked target in the image in real time, and adjusts the visual axis direction in real time according to the coordinates so as to keep the target in the center of the image.

The present invention will now be illustrated with reference to a specific implementation scenario.

In this implementation scene, an unmanned aerial vehicle system of independently light small-size photoelectricity panorama is surveyed constitutes and includes unmanned aerial vehicle, photoelectricity nacelle, remote control module and industrial computer, wherein:

(1) Unmanned aerial vehicle: a multi-rotor flying platform integrating a flying control processor, wireless data communication, inertial navigation sensors and a satellite navigation system. Besides flight control and mission control, the unmanned aerial vehicle can carry the photoelectric pod for flight, and can realize downloading of collected photoelectric observation information and uploading of control instructions. Flight control software running on the unmanned aerial vehicle adjusts flight parameters to adapt to carrying 2kg photoelectric pods, and can realize fixed-point hovering flight and route flight. A schematic of a drone with a mounted optoelectronic pod can be seen with reference to fig. 1.

(2) Photoelectric pod: the system can realize the functions of optical imaging to output video, laser ranging/obstacle detection, target tracking, panoramic scanning and geographic guiding. And outputting newly increased azimuth angle and pitch angle of software running in the photoelectric pod and downloading the newly increased azimuth angle and pitch angle together with an image frame.

(3) And a remote control module: and the communication antenna is carried, the communication antenna is in wireless communication with the unmanned aerial vehicle, the wired communication with the industrial personal computer is realized, and the remote measurement data (such as unmanned aerial vehicle state information), the photoelectric video and task attribute information are forwarded and downloaded, and the control instruction is forwarded and uploaded. The software running in the remote control module is provided with an instruction transparent transmission function, and control instructions can be transmitted between the unmanned aerial vehicle and the industrial personal computer. In an emergency, the drone craft may switch to the manual mode through the remote control device.

(4) The industrial personal computer: the remote control module is connected in a wired way, and the ground photoelectric monitoring software of the unmanned aerial vehicle is operated. The industrial personal computer realizes the functions of video data display, photoelectric pod and unmanned aerial vehicle state display and the like through photoelectric monitoring software, and also realizes the uploading of control instructions (including pod control information and unmanned aerial vehicle remote control information). After the photoelectric monitoring software on the industrial personal computer enters the panoramic observation task mode, the designated panoramic stitching, area searching or target tracking tasks can be completed by the unmanned aerial vehicle automatically and remotely according to the panoramic observation task type until the task mode exits.

The following are the functions and performances realized by the unmanned aerial vehicle, the photoelectric pod, the remote control module, the industrial personal computer and matched software.

1. Photoelectric nacelle

The photoelectric nacelle adopts a two-axis gyro stabilizing platform, and is provided with a conductive slip ring to realize 360-degree continuous rotation and 10-degree pitching rotation of azimuth angles. The photoelectric pod is internally provided with a visible light camera, an infrared camera and a laser range finder, and outputs image frames, a field angle, an azimuth angle and a pitch angle in an H.264 format at a frame rate of 30 fps. The optoelectronic pod outputs the RTSP network video stream and SEI data.

2. Unmanned aerial vehicle and remote control module

The unmanned aerial vehicle is provided with a four-axis propeller, the body structure is composed of carbon fibers and polymer resin materials, the empty weight is about 10kg, and the carrying flying weight is about 13kg. The unmanned aerial vehicle supplies power to the photoelectric pod, sends flight state information to the photoelectric pod, analyzes panoramic observation task information obtained by a control instruction, and receives information and video data returned by the photoelectric pod. And a wireless transmission function is realized between the unmanned aerial vehicle and the remote control module. The remote control module is used for automatically controlling or manually controlling the unmanned aerial vehicle to fly through a preset instruction. The unmanned aerial vehicle supports functions of one-key take-off, one-key return voyage, one-key landing, fixed-point hovering and route flying.

3. Industrial control computer

And the industrial personal computer is connected with the remote control module through a USB, receives photoelectric video, telemetry data (unmanned aerial vehicle state information) and task attribute information in a network UDP protocol format, and sends the control instruction for controlling the unmanned aerial vehicle and the operation of the photoelectric pod. An example of the function of the photoelectric monitoring software of the industrial personal computer is shown in table 1.

Table 1 function of photoelectric monitoring software of industrial personal computer

The system state display mainly displays the state information of the unmanned aerial vehicle, which is sent by each sensor on the unmanned aerial vehicle in real time, and comprises the state information of the system running state, the field angle, the azimuth angle, the pitch angle and the telemetry device loaded on the unmanned aerial vehicle, such as the laser ranging state, the obstacle detection state and the like.

The photoelectric video image display consists of visible light image display and infrared image display. The visible light display is to receive a visible light image downloaded by a photoelectric pod on the unmanned aerial vehicle, calculate pixel distribution characteristics such as brightness contrast and the like of a statistical image, and self-adaptively adjust a display image. The pixel position, the target type and the number of the target are marked in the real-time image and the spliced image. The infrared image display is to receive an infrared image transmitted by an optoelectronic pod on the unmanned aerial vehicle, calculate and count pixel distribution characteristics such as brightness contrast of the infrared image, and adaptively adjust and display the infrared image.

The information in the task attribute information display includes: satellite navigation system time, instruction feedback, azimuth angle and pitch angle of the optoelectronic pod, visible light view angle, infrared view angle, visible light sensor state, infrared sensor state, laser range finder state, laser range finding distance/effectiveness, longitude/latitude/altitude/speed of the flying platform, other sensors and data processing unit state, on-board optoelectronic current function or task mode and the like. The task attribute information also displays the result of laser ranging and the state of a tracking target, including the position and speed of the target, and displays the information of the obstacle on the detection route in the state that the obstacle detection is started.

The observation task remote control mainly sends control instructions about unmanned aerial vehicle and photoelectric pod control to a remote control module to realize: the movement of the photoelectric pod, zooming and focusing of optical equipment, tracking/stopping tracking of the photoelectric equipment, laser ranging, target positioning and resolving, flight path planning and the like. The observation task remote control also comprises a panoramic observation function: panorama concatenation function, area search function and target tracking function.

The panoramic image is spliced and displayed, and image information and system information downloaded by the photoelectric pod on the unmanned aerial vehicle are processed. The method takes out partial image data, and transmits information related to splicing and target detection identification information to panoramic display software for display. And the panoramic image stitching function is used for outputting panoramic stitching images with 360-degree visual angles after stitching, wherein the input visible light images to be stitched are 1920 ANG 1080 pixels, and the overlapping of adjacent images is not less than 20 percent. Specifically, the image information is combined with the airplane azimuth and attitude information and the photoelectric pod view angle and attitude information to correct and splice each frame of image with different visual angles, so that a seamless image is formed.

The target detection and identification mainly displays target identification results sent by photoelectric identification software carried by the unmanned aerial vehicle, wherein the target identification results comprise pixel positions, target types and numbers of targets.

The following is an implementation mode for realizing various panoramic observation tasks by the light and small-sized photoelectric panoramic observation unmanned aerial vehicle system in the implementation scene, and covers the behaviors of photoelectric monitoring software on an industrial personal computer, and photoelectric loads in an unmanned aerial vehicle and a photoelectric pod. The following processes are all started from photoelectric monitoring software, the unmanned aerial vehicle and the photoelectric load are controlled to execute actions through wireless link transmission of a remote control module, and meanwhile the photoelectric monitoring software displays photoelectric video data transmitted under the photoelectric load in real time. Before the following process begins, the unmanned aerial vehicle needs to take off and hover. The following flow does not include the flow of unmanned aerial vehicle return voyage and landing after the task is finished.

1. Panorama stitching mode

(1) The photoelectric monitoring software reads a panoramic splicing instruction which is locally stored and preset (or receives the input of an operator from an interface), and acquires the flight state (including the current flight type) of the unmanned aerial vehicle from telemetry data (unmanned aerial vehicle state information): if the electric quantity of the unmanned aerial vehicle is higher than an electric quantity threshold (for example, 30% of the rated electric quantity of a battery) and the unmanned aerial vehicle hovers, entering the next step; otherwise, the panoramic stitching instruction is returned to be invalid.

(2) The photoelectric monitoring software takes out parameters in the panoramic stitching instruction: hovering point longitude, latitude, altitude and flight speed of the unmanned aerial vehicle (corresponding to the second flight speed information). And the photoelectric monitoring software automatically codes a route instruction (sets longitude, latitude and altitude of a hovering point of the unmanned aerial vehicle and sets flying speed) according to the acquired parameters, and sends the route instruction to the unmanned aerial vehicle through a wireless link. The photoelectric monitoring software automatically codes a photoelectric zeroing instruction (the photoelectric zeroing instruction is used for controlling the photoelectric pod to stop the current task and points to the azimuth angle of 0 degrees and the pitch angle of 0 degrees), and the photoelectric zeroing instruction is sent to the unmanned aerial vehicle through a wireless link and then forwarded to the photoelectric pod.

Second flight speed information, hover point position information of unmanned aerial vehicle and photoelectric zeroing instruction

(3) And the unmanned aerial vehicle receives the route instruction, analyzes the position information (longitude, latitude and altitude) of the hovering point and the second flying speed information of the unmanned aerial vehicle in the instruction, automatically forms a flying task, flies to the waypoint with the appointed longitude, latitude and altitude at the set speed, and hovers after reaching the appointed waypoint.

(4) And after receiving a photoelectric zeroing instruction, the photoelectric pod points to the azimuth angle of 0 degrees and stops after the pitch angle of 0 degrees.

(5) The photoelectric monitoring software circularly reads real-time position information of the unmanned aerial vehicle from telemetry data (unmanned aerial vehicle state information), and sends a photoelectric azimuth scanning instruction to the photoelectric pod after judging that the unmanned aerial vehicle reaches a designated waypoint, and starts an image splicing function.

(6) The photoelectric pod receives an azimuth scanning instruction and starts azimuth scanning: and setting the pitch angle of the photoelectric pod to-20 degrees in the scanning process, and continuously rotating the azimuth angle in the circumferential direction.

(7) The photoelectric monitoring software circularly executes the image splicing process: and continuously receiving the image downloaded by the photoelectric pod, reading the view angle, azimuth angle and pitch angle corresponding to the image from the task attribute data, calculating image projection, calculating characteristic points in the image, splicing and correcting the characteristic points of the projection image, and displaying the panoramic image after projection correction on an interface. The photoelectric monitoring software waits for a task ending instruction which is preset in a local storage mode (or receives an operator input from an interface) while splicing the process. If the task ending instruction is obtained, sending a photoelectric zeroing instruction to the photoelectric pod and stopping image splicing; otherwise, continuing image stitching.

2. Region search mode

(1) The photoelectric monitoring software reads a local storage preset (or receives an operator input from an interface) area search instruction, and acquires the unmanned aerial vehicle flight state from telemetry data: if the electric quantity of the unmanned aerial vehicle is higher than 30% of the rated electric quantity of the battery and the unmanned aerial vehicle hovers, entering the next step; otherwise, the callback area search instruction is invalid.

(2) The photoelectric monitoring software takes out parameters in the area searching instruction: the position information (including longitude, latitude, and altitude) of 4 vertexes of the search area is searched, and the position information (including longitude, latitude, and altitude) of the center of the search area is calculated according to the position information of 4 vertexes. The photoelectric monitoring software uses longitude, latitude and altitude of the central position of the area as control parameters to send a first geographic follow-up instruction to the photoelectric load in the photoelectric pod.

(3) The photoelectric load receives a first geographic follow-up instruction and enters a geographic follow-up mode: and adjusting azimuth angles and pitch angles of the photoelectric pods in real time according to the first geographic follow-up instructions, so that the photoelectric pods point to longitudes, latitudes and altitudes in the geographic follow-up instructions, and even if the fields of view of the photoelectric pods cover the central position of the search area.

(4) The photoelectric monitoring software judges that the photoelectric load enters a first geographic follow-up mode and points to the central position of the search area from the telemetry data, and if the photoelectric load enters the first geographic follow-up mode and points to the central position of the search area, the next step is carried out; otherwise, step (4) of the area search mode is circularly executed.

(5) The photoelectric monitoring software reads real-time position information and real-time posture information of the unmanned aerial vehicle, calculates the distance from the photoelectric pod to the central position of the searching area according to the real-time position information and the real-time posture information of the unmanned aerial vehicle, and then calculates the optimal field angle of the photoelectric pod by using the calculated distance and the position information of 4 vertexes of the searching area, wherein the optimal field angle needs to be satisfied: the range of the photoelectric imaging at the optimal field angle includes the search area and the optimal field angle is as small as possible. The photoelectric monitoring software sends a view angle adjusting instruction to the photoelectric load in the photoelectric pod, and the control parameter is the optimal view angle.

(6) The photoelectric load receives the angle-of-view adjustment instruction, acquires the optimal angle of view in the control parameters, and adjusts the angle of view to the optimal angle of view.

(7) The optoelectronic monitoring software waits for a task end instruction that is locally stored preset (or received operator input from an interface). If the task ending instruction is acquired, a photoelectric zeroing instruction is sent to the photoelectric pod; otherwise, the steps (5), (6) and (7) of the area search mode are circularly executed.

3. Target tracking mode

(1) The photoelectric monitoring software reads a target tracking instruction which is locally stored and preset (or receives an input of an operator from an interface), acquires the flight state of the unmanned aerial vehicle from telemetry data, reads the electric quantity and the current flight type of the unmanned aerial vehicle according to the unmanned aerial vehicle state information, and enters the next step if the battery electric quantity of the unmanned aerial vehicle is higher than 30% of the rated battery electric quantity and the unmanned aerial vehicle hovers; otherwise, the callback target tracking instruction is invalid.

(2) The photoelectric monitoring software takes out the control parameters in the target tracking instruction: location information (longitude, latitude, altitude) of the target to be tracked. The photoelectric monitoring software uses longitude, latitude and altitude of the target position to be tracked as control parameters to send a second geographic follow-up instruction to the photoelectric load in the photoelectric pod.

(3) The photoelectric load receives a second geographic follow-up instruction and enters a second geographic follow-up mode: and the azimuth angle and the pitch angle of the photoelectric pod are adjusted in real time, so that the photoelectric pod points to the longitude, latitude and altitude in the second geographic follow-up instruction.

(4) The photoelectric monitoring software acquires unmanned aerial vehicle state information from the telemetry data, judges whether the photoelectric load enters a second geographic follow-up mode and points to the target position to be tracked according to the unmanned aerial vehicle state information, and enters the next step if the photoelectric load enters the second geographic follow-up mode and points to the target position to be tracked; otherwise, the step (4) is circularly executed.

(5) The photoelectric monitoring software sends a target tracking instruction to the photoelectric load in the photoelectric pod.

(6) The photoelectric load receives the target tracking instruction, and tracks the target to be tracked in the center of the image through the deep learning target detection and recognition result so as to enter an image tracking mode.

(7) The photoelectric monitoring software reads longitude, latitude and altitude of the photoelectric pod pointing to the target position to be tracked from the telemetry data, reads the position information of the unmanned aerial vehicle, and calculates the inclined distance from the unmanned aerial vehicle to the pointing position (the target position to be tracked) according to the target position to be tracked and the position information of the unmanned aerial vehicle. If the skew is larger than a skew threshold, for example, the skew exceeds 200m, calculating the position information of the midpoint of the connecting line of the unmanned aerial vehicle and the target to be tracked. Taking longitude and latitude of the midpoint of the connecting line, the current altitude and the default speed of the unmanned aerial vehicle as the position control parameters of the target waypoint of the route instruction. And the photoelectric monitoring software sends the newly generated route instruction to the unmanned aerial vehicle.

(8) And the unmanned aerial vehicle receives the route instruction, analyzes the longitude, latitude, altitude and speed in the route instruction, automatically forms a flight task, and flies to a target waypoint with the designated longitude, latitude and altitude at a set default speed. The unmanned aerial vehicle hovers after reaching a specified target waypoint.

(9) And the photoelectric monitoring software circularly reads the position information of the unmanned aerial vehicle from the telemetry data, and after judging that the unmanned aerial vehicle reaches a designated target waypoint, the unmanned aerial vehicle enters the next step.

(10) The optoelectronic monitoring software waits for a task end instruction that is locally stored preset (or received operator input from an interface). If a task ending instruction is acquired, a photoelectric zeroing instruction is sent to the photoelectric pod, and a hovering instruction is sent to the unmanned aerial vehicle; otherwise, the steps (7), (8), (9) and (10) of the target tracking mode are circularly executed.

(11) After the unmanned aerial vehicle receives the hovering instruction, hovering at the current position.

The unmanned aerial vehicle system for autonomous light and small photoelectric panoramic observation provided by the embodiment of the invention meets the application requirement of controlling the light unmanned aerial vehicle to carry out panoramic observation only by using the control instruction of the industrial personal computer under the condition of not depending on a flying hand. The panoramic observation function of the system is realized by the preset instruction, and the unmanned aerial vehicle and the photoelectric load do not need to be operated by a flight hand in real time, so that the labor is saved, and the working efficiency is improved. For safety reasons, the flight crew can take over the unmanned aerial vehicle through the remote controller in case of emergency, so as to ensure the operation safety of the unmanned aerial vehicle in case of accidents.

Like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.

It should be noted that, the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," "outer," and the like indicate an azimuth or a positional relationship based on that shown in the drawings, or that the technical product is conventionally put in place when used, merely for convenience in describing the present technology and simplifying the description, and do not indicate or imply that the apparatus or element to be referred must have a specific azimuth, be configured and operated in a specific azimuth, and thus should not be construed as limiting the present technology. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. The terms "first," "second," "third," and the like are therefore used solely to distinguish one from another and are not to be construed as indicating or implying a relative importance. In the description of the present invention, unless otherwise indicated, the meaning of "a plurality" is two or more.

Furthermore, the terms "horizontal," "vertical," "overhang," and the like do not denote a requirement that the component be absolutely horizontal or overhang, but rather may be slightly inclined. As "horizontal" merely means that its direction is more horizontal than "vertical", and does not mean that the structure must be perfectly horizontal, but may be slightly inclined.

In the description of the present technology, it should also be noted that, unless explicitly specified and limited otherwise, the terms "disposed," "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the terms in the art will be understood in a specific manner by those of ordinary skill in the art.

The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims

1. The utility model provides an unmanned aerial vehicle system of independently light small-size photoelectricity panorama observation which characterized in that, including unmanned aerial vehicle, photoelectricity nacelle and control module, wherein:

2. The unmanned aerial vehicle system for autonomous light and small photoelectric panoramic observation according to claim 1, wherein the control module comprises a remote control module and an industrial personal computer, wherein the remote control module is respectively in communication connection with the unmanned aerial vehicle and the industrial personal computer, is used for transmitting information interacted between the unmanned aerial vehicle and the industrial personal computer, and is also used for providing a manual control mode for the unmanned aerial vehicle; the industrial personal computer is used for sending out control instructions, receiving unmanned aerial vehicle state information and photoelectric observation information, and adjusting the control instructions according to the unmanned aerial vehicle state information and the photoelectric observation information.

3. An autonomous lightweight miniaturized optoelectronic panoramic viewing unmanned aerial vehicle system according to claim 1 or 2, wherein the types of flight tasks include a fixed point hover mode and a course flight mode, wherein,

4. An autonomous lightweight miniaturized optoelectronic panoramic viewing drone system as claimed in claim 3, wherein the types of panoramic viewing tasks include a panoramic stitching mode, a region search mode, and/or a target tracking mode, wherein:

5. The autonomous lightweight miniaturized optoelectronic panoramic viewing unmanned aerial vehicle system of claim 4, wherein the panoramic stitching mode comprises the steps of:

6. The autonomous lightweight, miniature optoelectronic panoramic viewing drone system of claim 4, wherein the area search mode comprises the steps of:

7. The autonomous lightweight, miniature optoelectronic panoramic viewing drone system of claim 4, wherein the target tracking mode comprises the steps of:

8. The unmanned aerial vehicle system for autonomous light and small photoelectric panoramic observation according to claim 1, wherein the photoelectric pod is connected with the unmanned aerial vehicle through a conductive slip ring, a pitching mechanism is arranged on the photoelectric pod, and a photoelectric observation information acquisition end of the photoelectric pod is arranged on the pitching mechanism.

9. The unmanned aerial vehicle system for autonomous light and small photoelectric panoramic observation according to claim 8, wherein the photoelectric observation information acquisition end of the photoelectric pod comprises a visible light image acquisition module, an infrared image acquisition module and/or a laser detection module which are arranged in the photoelectric pod, wherein the visible light image acquisition module is used for acquiring visible light image information in a view field range, the infrared image acquisition module is used for acquiring infrared image information in the view field range, and the laser detection module is used for acquiring laser scanning information in the view field range.

10. The unmanned aerial vehicle system for autonomous light and small photoelectric panoramic observation according to claim 1, wherein a first wireless communication module, an inertial navigation system and a satellite navigation system are arranged on the unmanned aerial vehicle, the first wireless communication module is used for being in communication connection with the control module, the inertial navigation system is used for acquiring real-time attitude information of the unmanned aerial vehicle, and the satellite navigation system is used for acquiring real-time position information of the unmanned aerial vehicle.