WO2023102911A1

WO2023102911A1 - Data collection method, data presentation method, data processing method, aircraft landing method, data presentation system and storage medium

Info

Publication number: WO2023102911A1
Application number: PCT/CN2021/137164
Authority: WO
Inventors: 缪宝杰; 沈劭劼; 陈晓智; 林毅
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2021-12-10
Filing date: 2021-12-10
Publication date: 2023-06-15
Also published as: CN117916155A

Abstract

A data collection method, a data presentation method, a data processing method, an aircraft landing method, a data presentation system and a storage medium. The data presentation system comprises a vehicle and an aircraft, wherein the vehicle is in communication connection with the aircraft; the vehicle comprises a plurality of display areas located at different positions; the aircraft is used for collecting, during the process of executing a flight task, task data of a corresponding flight task, and transmitting the task data back to the vehicle; and the vehicle is used for generating display information according to the task data, and controlling the display areas to display the display information; and display information generated by task data of different types of flight tasks is displayed in the display areas located at different positions. It is convenient for a user to directly view the display information from the display areas at different positions, thereby satisfying the viewing requirements of the user for different types of flight tasks.

Description

Data acquisition method, data display method, data processing method, aircraft landing method, data display system and storage medium

technical field

The present application relates to the technical field of aircraft and vehicles, in particular, to a data acquisition method, a data display method, a data processing method, an aircraft landing method, a data display system, and a storage medium.

Background technique

Aircraft such as unmanned aerial vehicles (UAVs) can be piloted using radio remote control equipment and self-contained program controls, or operated completely or intermittently autonomously by an onboard computer. At present, when the aircraft is used independently as an individual, it can be widely used in scenarios such as aerial photography, agricultural plant protection, express delivery, disaster rescue surveying and mapping, news reporting, power inspection, disaster relief, or film and television shooting. In order to further expand the application space of the aircraft, the operation scenarios where the aircraft is used in combination with other objects (such as vehicles) need to be further explored and improved.

Contents of the invention

In view of this, one of the purposes of this application is to provide a data collection method, a data display method, a data processing method, an aircraft landing method, a data display system and a storage medium.

In the first aspect, an embodiment of the present application provides a data display system, including a vehicle and an aircraft, the vehicle is connected to the aircraft in communication; the vehicle includes multiple display areas located at different locations;

The aircraft is used to collect mission data corresponding to the flight mission during the execution of the flight mission, and send the mission data back to the vehicle;

The vehicle is used to generate display information according to the mission data, and control the display area to display the display information, wherein the display information generated by mission data of different types of flight missions is displayed in the display area at different positions.

This embodiment realizes the display information generated by the task data of different types of flight missions, and displays them in the display areas at different positions, which is convenient for users to intuitively view the display information from the display areas at different positions, thereby satisfying the needs of users for different types of flight tasks. The task's viewing needs.

In a second aspect, the embodiment of the present application provides a data collection method, the method is applied to an aircraft, and the method includes:

Responding to the control instructions of the flight mission, collecting mission data during the execution of the flight mission;

sending the task data to a vehicle, the task data is used to generate display information, and the display information is used to display on the vehicle, wherein the vehicle includes a plurality of display areas, and the plurality of display areas are located at For different positions of the vehicle, display information generated from mission data of different types of flight missions is displayed on the display area at different positions.

In the third aspect, the embodiment of the present application provides a data presentation method, the method is applied to a vehicle, and the vehicle includes multiple display areas located in different positions, and the method includes:

Obtain the mission data collected during the flight mission of the aircraft;

generating display information according to the task data;

The display area is controlled to display the display information, wherein the display information generated by mission data of different types of flight missions is displayed on the display area at different positions.

In a fourth aspect, the embodiment of the present application provides a data display system, including a vehicle and an aircraft, and the vehicle is communicatively connected to the aircraft;

The aircraft is used to collect task data corresponding to the flight task during the execution of the flight task, and send the task data back to the vehicle;

The vehicle is used for generating display information according to the mission data, and displaying the display information in a display area, wherein mission data of different types of flight missions generate different types of display information.

In this embodiment, the aircraft can be controlled to perform different flight missions according to actual needs, so that different types of display information can be generated based on mission data of different types of flight missions to meet different viewing needs of users.

In the fifth aspect, the embodiment of the present application provides a data processing method applied to a vehicle, including:

When the aircraft is fixed to the vehicle, the first traffic information collected by the aircraft and the environmental information collected by the vehicle's on-board sensor are obtained, and the first traffic information and the environmental information collected by the on-board sensor are fused as the environment of the vehicle perceptual information reference;

When the aircraft performs a flight mission, acquire the second traffic information collected by the aircraft and the pose information of the aircraft when collecting the second traffic information, and use the second traffic information as the The vehicle's navigation information reference.

In this embodiment, two ways of using the aircraft in the vehicle scene are provided. When the aircraft is fixed on the vehicle, it assists the vehicle to perceive the surrounding environment more comprehensively; Thereby, it is beneficial to improve the driving safety of the vehicle.

In a sixth aspect, the embodiment of the present application provides a data processing method applied to a camera, the method comprising:

In the process of performing the shooting task, detecting whether the camera is connected to the vehicle;

If the camera is connected to the vehicle, the image collected by performing the shooting task is sent to the vehicle; the image is used to trigger the vehicle to use the image to recognize user gestures or detect the driver's state; the user Gestures are used at least to control an aircraft communicatively linked with said vehicle to perform a flight mission;

If the camera is not connected to the vehicle, storing images collected by executing the shooting task.

In this embodiment, a method of using a detachable camera in a vehicle scene is provided. When the camera is connected to the vehicle, it can be used to control the vehicle or aircraft.

In the seventh aspect, the embodiment of the present application provides a method for landing an aircraft, including:

In response to a landing trigger, searching for a vehicle to be landed, and determining a relative pose relationship between the aircraft and the vehicle to be landed;

estimating predicted motion information of the vehicle to be landed according to the motion information of the aircraft and the relative pose relationship, so as to plan the landing path of the aircraft;

The aircraft is controlled to land on the vehicle according to the landing path.

This embodiment provides a method for an aircraft to land on a moving vehicle, by accurately estimating the predicted motion information of the vehicle to be landed based on the motion information of the aircraft and the relative pose relationship between the aircraft and the vehicle to be landed, so that the planning is more accurate The landing path can realize the accurate landing of the aircraft on the moving vehicle.

In an eighth aspect, the embodiment of the present application provides an aircraft, including:

body;

a power system, arranged on the fuselage, used to provide power for the aircraft; and,

One or more processors are arranged in the fuselage and are used to execute the method described in any one of the second aspect or the seventh aspect.

In a ninth aspect, the embodiment of the present application provides a vehicle, including:

body;

The power system is arranged on the body and is used to provide power for the vehicle;

a plurality of display areas located at different locations for displaying display information for flight missions; and,

One or more processors are arranged on the fuselage and are used to execute the method described in any one of the third aspect or the fifth aspect.

In a tenth aspect, the embodiment of the present application provides a camera, including:

a lens assembly for transmitting light;

Photosensitive element, which is used to convert light into electrical signals and generate images;

memory for storing images; and,

A processor, configured to execute the method described in the sixth aspect.

In the eleventh aspect, the embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores executable instructions, and when the executable instructions are executed by a processor, the second aspect and the third aspect are implemented. , the method according to any one of the fifth aspect, the sixth aspect or the seventh aspect.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1 is a schematic diagram of a data display system provided by an embodiment of the present application;

Fig. 2 is a schematic structural diagram of an aircraft provided by an embodiment of the present application;

Fig. 3 is a schematic diagram of a smart cockpit of a vehicle provided by an embodiment of the present application;

Fig. 4 is a schematic diagram of different user gestures and their meanings provided by the embodiment of the present application;

FIG. 5A and FIG. 5B are schematic diagrams of a user manipulating different input devices provided by an embodiment of the present application;

Fig. 5C is a schematic diagram of different display areas in a vehicle provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of a scene of a traffic information collection task provided by an embodiment of the present application;

Fig. 7 is a schematic diagram of the display information of the traffic information collection task provided by the embodiment of the present application displayed on the HUD display;

FIG. 8 is a schematic diagram of another traffic information collection task provided by the embodiment of the present application;

FIG. 9 is a schematic diagram of a scene of an aerial photography task provided by an embodiment of the present application;

Fig. 10A is a schematic diagram of a scene in which a second camera is set in a vehicle according to an embodiment of the present application;

Fig. 10B is a schematic diagram of a scene where the second camera is used alone according to the embodiment of the present application;

Figure 11 is a schematic diagram of several different visual signs provided by the embodiment of the present application;

Fig. 12 is a schematic diagram of processing logic for aircraft landing provided by the embodiment of the present application;

Figure 13A and Figure 13B are different schematic diagrams of the data display system provided by the embodiment of the present application;

FIG. 14 is a schematic flow chart of a data collection method provided by an embodiment of the present application;

Fig. 15 is a schematic flow chart of a data presentation method provided by an embodiment of the present application;

FIG. 16 is a schematic flow chart of a data processing method provided by an embodiment of the present application;

Fig. 17 is a schematic flowchart of another data processing method provided by the embodiment of the present application;

FIG. 18 is a schematic flowchart of a landing method for an aircraft provided in an embodiment of the present application.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

In view of the problems in the related technologies, the embodiment of the present application provides a data display system, please refer to Figure 1, the data display system includes a vehicle 100 and an aircraft 200, and the vehicle and the aircraft are connected in communication; the vehicle includes multiple displays located in different positions area; the aircraft can perform different types of flight missions and collect mission data corresponding to the flight missions; the vehicle can display the display information generated by the mission data of different types of flight missions in display areas at different positions based on different types of flight missions, It is convenient for the user to visually view the displayed information from the display areas at different positions, so as to meet the user's viewing requirements for different types of flight missions.

Wherein, it will be obvious to those skilled in the art that any type of aircraft can be used without limitation, and the embodiments of the present application can be applied to various types of aircraft. For example, the aircraft may be a small or a large aircraft. In some embodiments, the aircraft may be a rotorcraft, for example, a multi-rotor aircraft propelled by multiple propulsion devices through the air. Embodiments of the present application are not limited thereto, and the aircraft may also be other types of aircraft.

In an exemplary embodiment, please refer to FIG. 2 , the aircraft 200 may include a power system 210 , a flight control system 220 , a frame, and a gimbal 230 carried on the frame. The aircraft 200 may be an industrial application aircraft, which requires cyclic operation.

The frame may include the fuselage and undercarriage (also known as landing gear). The fuselage may include a center frame and one or more arms connected to the center frame, and the one or more arms extend radially from the center frame. The tripod is connected with the fuselage and is used for supporting the aircraft 200 when it lands.

The power system 210 may include one or more electronic governors (abbreviated as ESCs) 211, one or more propellers 213 and one or more motors 212 corresponding to the one or more propellers 213, wherein the motors 212 are connected to Between the electronic governor 211 and the propeller 213, the motor 212 and the propeller 213 are arranged on the machine arm of the aircraft 200; the electronic governor 211 is used to receive the driving signal generated by the flight control system 220, and provide the driving current to the The motor 212 is used to control the rotation speed of the motor 212 . The motor 212 is used to drive the propeller to rotate, so as to provide power for the flight of the aircraft 200 , and the power enables the aircraft 200 to realize movement of one or more degrees of freedom. In some embodiments, aircraft 200 may rotate about one or more axes of rotation. For example, the rotation axis may include a roll axis (Roll), a yaw axis (Yaw) and a pitch axis (pitch). It should be understood that the motor 212 may be a DC motor or an AC motor. In addition, the motor 212 may be a brushless motor or a brushed motor.

Flight control system 220 may include flight controller 221 and sensing system 222 . The sensing system 222 is used to measure the attitude information of the aircraft, that is, the position information and status information of the aircraft 200 in space, such as three-dimensional position, three-dimensional angle, three-dimensional velocity, three-dimensional acceleration and three-dimensional angular velocity. The sensing system 222 may include, for example, at least one of sensors such as a gyroscope, an ultrasonic sensor, an electronic compass, an inertial measurement unit (Inertial Measurement Unit, IMU), a visual sensor, a global navigation satellite system, and a barometer. For example, the global navigation satellite system may be the Global Positioning System (GPS). The flight controller 221 is used to control the flight of the aircraft 200 , for example, the flight of the aircraft 200 may be controlled according to the attitude information measured by the sensing system 222 . It should be understood that the flight controller 221 can control the aircraft 200 according to pre-programmed instructions, or can control the aircraft 200 by responding to one or more remote control signals from the vehicle.

The gimbal 230 may include a motor 232 . The pan-tilt is used to carry the photographing device 233 . The flight controller 221 can control the movement of the gimbal 230 through the motor 232 . It should be understood that the gimbal 230 may be independent from the aircraft 200 or be a part of the aircraft 200 . It should be understood that the motor 232 may be a DC motor or an AC motor. In addition, the motor 232 can be a brushless motor or a brushed motor. It should also be understood that the gimbal can be located on the top of the aircraft or on the bottom of the aircraft.

The photographing device 233 can be, for example, a camera or a video camera or other equipment for capturing images. The photographing device 233 can communicate with the flight controller and take photos under the control of the flight controller. The photographing device 233 of this embodiment includes at least a photosensitive element, such as a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) sensor or a charge-coupled device (Charge-coupled Device, CCD) sensor. It can be understood that the camera device 233 can also be directly fixed on the aircraft 200, so that the gimbal 230 can be omitted.

In an exemplary embodiment, the vehicle may be a passenger vehicle (such as a self-driving vehicle), and the vehicle generally includes a chassis, a body, an engine and electrical equipment. The engine is the power plant of the vehicle, which is used to generate power; the chassis is used to support the engine and the body, and the chassis can drive the vehicle according to the power generated by the engine; the body is installed on the frame of the chassis for the driver, passengers to ride or Cargo loading; electrical equipment includes power sources and electrical equipment, for example, power sources include batteries and generators, and electrical equipment includes engine starting systems or other electrical devices. Optionally, the vehicle further includes an on-board sensor for sensing environmental information of the surrounding environment of the vehicle. Optionally, the vehicle also includes an automatic driving system for assisting the driver in driving.

In some embodiments, please refer to FIG. 1 , the embodiment of the present application provides a data display system, the data display system includes a vehicle 100 and an aircraft 200, the vehicle 100 is connected to the aircraft 200 in communication; the vehicle 100 includes multiple display areas at different locations.

The aircraft 200 is used for collecting mission data corresponding to the flight mission during the execution of the flight mission, and sending the mission data back to the vehicle 100 .

The vehicle 100 is used to generate display information according to the mission data, and control the display area to display the display information, wherein, the display information generated by the mission data of different types of flight missions is displayed in the display area at different positions .

In this embodiment, the vehicle 100 displays the display information generated by the mission data of different types of flight missions in the display areas at different positions, so that users can intuitively view the display information from the display areas at different positions, Thereby satisfying users' viewing needs for different types of flight missions.

In some embodiments, the aircraft 200 can be placed in the vehicle 100, for example, the body of the vehicle 100 is provided with an aircraft 200 apron, and the aircraft 200 can be placed in the apron of the aircraft 200, and when it is ready to perform a flight mission, stop from the aircraft 200 The landing pad takes off to perform the flight mission, and lands in the aircraft 200 parking pad after performing the flight mission. The process of taking off or landing of the aircraft 200 will be further described below.

In some embodiments, the vehicle 100 is configured to send a flight mission control instruction to the aircraft 200; the aircraft 200 is configured to respond to the flight mission control instruction, execute the flight mission and collect mission data corresponding to the flight mission. In this embodiment, the vehicle 100 generates a control instruction for controlling the aircraft 200 to perform a flight mission, providing a new control method for the aircraft 200 .

Exemplarily, the vehicle 100 includes at least one input device; the input device includes one or more external input devices, and/or one or more built-in input devices; the control command can be manipulated by the user in the vehicle 100 device generated. In this embodiment, when the vehicle 100 includes multiple input devices, the user can select a convenient input device to control the aircraft 200 according to actual needs, so as to meet the user's personalized use requirements.

It can be understood that the embodiments of the present application do not impose any limitation on the control content of the control instruction, and specific settings may be made according to actual application scenarios. In one example, the control command is used to control the flight direction of the aircraft 200 , such as raising the flight altitude, moving forward/backward/left/right and so on. In another example, the control instruction is used to control the aircraft 200 to collect images, and control the camera attitude angle (Yaw, Roll, Pitch). In yet another example, the control instruction is used to control the aircraft 200 to take off from the vehicle 100 or land on the vehicle 100 .

In an exemplary embodiment, the built-in input device may include any of the following: built-in first camera, voice collection component, central control panel or steering wheel control; the external input device may include any of the following: remote control terminal, somatosensory A remote control, or a second camera communicatively connected to the vehicle 100 through a pluggable interface; the user can select one of the input devices to control the aircraft 200 according to actual needs, so as to facilitate the use of the user.

For example, please refer to FIG. 3 . FIG. 3 shows a schematic diagram of a smart cockpit of a vehicle 100 , where input devices and display areas may be set. For example, the input device includes (3) a steering wheel control, (5) a camera, and (7) a somatosensory remote control, wherein the steering wheel control can be a virtual control or a physical control; the camera can be a built-in first camera or a pluggable interface A second camera communicatively coupled to the vehicle 100 . For example, the display area includes (1) HUD (Head Up Display) display, (2) 3D instrument panel, (4) central control screen area A and (6) central control screen area B; among them, central control screen area A and central control screen Screen B area is two different display areas of the central control screen, which can display the same content or different content. Of course, FIG. 3 is an example and does not constitute a limitation on the input device and the display area.

Optionally, in the case where there are multiple input devices, in order to avoid using at least two input devices at the same time to cause multiple identical or conflicting control instructions to affect the flight safety of the aircraft 200, it can be set for the same The same or conflicting control instructions generated at all times are invalid, so as to ensure the orderly generation of the control instructions and the safe and orderly execution of the control instructions by the aircraft 200 .

Exemplarily, for example, in the case of including two input devices, user A manipulates input device 1 to control aircraft 200 to move forward (at this moment, vehicle 100 generates a first control command), while user B manipulates input device 2 to control aircraft 200 to retreat (At this time, the vehicle 100 generates the second control command), the first control command conflicts with the second control command, and the first control command and the second control command are invalid; or user A manipulates the input device 1 to control the aircraft 200 to move forward, while user B The manipulation input device 2 also controls the aircraft 200 to move forward. If the control commands generated by the two are the same, they will be invalidated and not sent to the aircraft 200 . In other cases, the control commands generated by the user's manipulation of the input device are all valid. For example, user A manipulates the input device 1 to control the aircraft 200 to move forward, while user B manipulates the input device 2 to control the aircraft 200 to take pictures. If they are not the same or conflict, they can be sent to the aircraft 200.

In some embodiments, the external input device can be directly communicated with the aircraft 200. For example, the remote control terminal can be directly communicated with the aircraft 200, and the remote control terminal can generate control instructions for the aircraft 200 according to the user's manipulation behavior on the remote control terminal and send them directly. Give the aircraft 200, no need to transfer through other equipment. Or, the external input device can also be communicated with the aircraft 200 through the vehicle 100; in one example, the vehicle 100 can be used as a transfer station, and the remote control terminal can forward the generated control instructions to the aircraft 200 via the vehicle 100; in another example Among them, the external input device can also send the collected user manipulation data (such as the user image collected by the camera, the operation data received by the remote control terminal, etc.) to the vehicle 100, and the vehicle 100 generates control instructions according to the user manipulation data, and sends Give the aircraft 200.

In a possible implementation, the control instruction can be generated according to user gestures collected by the built-in first camera and/or the second camera connected to the vehicle 100 through a pluggable interface, such as the first camera and/or the second camera Can be set at the position facing the seat in the vehicle 100, so that the shooting range of the first camera and/or the second camera includes the seat range in the vehicle 100, then when the user sits on the seat in the vehicle 100, the first camera And/or the second camera can collect the image of the user, and then recognize the image of the user to obtain the gesture of the user, and generate a corresponding control command according to the meaning indicated by the gesture of the user.

It can be understood that the meanings represented by different user gestures may be specifically set according to actual application scenarios, which is not limited in this embodiment. In an example, please refer to FIG. 4 , which exemplarily shows the meanings of different gestures. For example, in FIG. 4 , the user shows a "thumbs up" gesture. After the vehicle 100 captures the user's gesture through the built-in first camera and/or the second camera communicatively connected to the vehicle 100 through a pluggable interface, The ascent control instruction may be generated and sent to the aircraft 200, and the aircraft 200 performs an ascent flight action based on the ascent control instruction.

Exemplarily, in order to improve the security of the control of the aircraft 200, before the user gesture is recognized from the user image to control the aircraft 200, user identity verification can also be performed according to the user image, for example, face recognition and verification can be performed according to the user image, If the verification is successful, the aircraft 200 is controlled according to the user's gesture recognized from the user image, thereby improving the safety of the aircraft 200 in use.

In a possible implementation, the control instruction can be generated according to the voice signal collected by the voice collection component. For example, after the vehicle 100 obtains the voice signal collected by the voice collection component, it performs voice recognition on the voice signal to obtain the semantic content, and then according to the semantic content Generate control instructions. In this embodiment, voice control is used to allow the user to focus on driving, and to ensure driving safety of the vehicle 100 while controlling the aircraft 200 .

In a possible implementation manner, the control instruction can be generated according to the operation data received by one or more of the central control panel, the steering wheel control and the remote control terminal. In one example, please refer to FIG. 5A and FIG. 5B , which show the central control panel 101 and the steering wheel control 102 in the vehicle 100. In FIG. 5A, the user can operate the central control panel 10 to control the aircraft 200. In FIG. 5B , the user can operate the steering wheel controls 20 to control the aircraft 200 .

In a possible implementation manner, the control instruction may be generated according to the motion sensing data collected by the motion sensing remote controller. Exemplarily, the motion-sensing remote control includes a shift lever, and the user can operate the shift lever to control the aircraft 200 .

In some embodiments, in order to ensure the flight safety of the aircraft 200, the distance between the aircraft 200 and the vehicle 100 is kept within a preset distance range during the flight mission, and the preset distance range can be based on the aircraft 200 and the vehicle. The communication distance between 100 is determined, so as to ensure that the aircraft 200 and the vehicle 100 maintain communication all the time, and prevent the aircraft 200 from being lost. For example, the aircraft 200 can send its own position information to the vehicle 100 in real time (real-time transmission refers to sending position information at a preset frequency, such as sending the latest position currently acquired by the aircraft 200 at a frequency of 50 times per second with the aircraft 200 information), the vehicle 100 can also know its own position based on its own satellite positioning module, and determine the distance between the aircraft 200 and the vehicle 100 according to the position of the aircraft 200 and the position of the vehicle 100, if the distance between the aircraft 200 and the vehicle 100 exceeds In the preset distance range, the vehicle 100 can control the vehicle 100 body to decelerate to ensure that the distance between the aircraft 200 and the vehicle 100 is kept within the preset distance range to prevent the aircraft 200 from being lost.

Optionally, the vehicle 100 can also control the aircraft 200 to accelerate to ensure that the distance between the aircraft 200 and the vehicle 100 is kept within a preset distance range, so as to prevent the aircraft 200 from being lost.

In some embodiments, the aircraft 200 can perform different flight missions in different scenarios. For example, in an assisted driving scenario, the aircraft 200 may perform a traffic information collection task, and the task data collected during the execution of the traffic information collection task may be traffic information. For example, in a traveling scene, the aircraft 200 may perform an aerial photography mission, and the mission data collected during the aerial photography mission may be aerial photography data. For example, in a game experience scene, the aircraft 200 can perform a flight game task, and the task data collected during the aerial photography task can be real-time image transmission, real-time audio data, or real-time audio and video data.

It can be understood that, after the aircraft 200 takes off from the vehicle 100, the aircraft 200 may land on the vehicle 100 after completing one flight mission, or land on the vehicle 100 after completing two or more different flight missions, This embodiment does not limit this.

In some embodiments, the aircraft 200 is provided with one or more sensors, and mission data of different types of flight missions may be collected using different types of sensors. Different types of sensors may include sensors with different types of collected data, for example, a camera is used to collect image data, and an acceleration sensor is used to collect acceleration data, and the types of the two sensors are different. Alternatively, a sensor is an electronic device used to convert one form of energy into another, and different types of sensors can include sensors that process different types of energy, such as cameras for converting light signals into electrical signals, audio capture Components are used to convert sound signals into electrical signals, and the two transducers are of different types.

It can be understood that the sensors used to collect mission data in different types of flight missions may also be partly the same, which is not limited in this embodiment.

Exemplarily, the sensor includes one or more of the following: at least one camera, one or more sensors for sensing the surrounding environment, or at least one audio collection component. Exemplarily, at least one camera can be carried in the aircraft 200 through a gimbal. Sensors used to perceive the surrounding environment include but are not limited to lidar, vision sensors, millimeter-wave radar or ultrasonic sensors, and the like.

The flight tasks include one or more of the following tasks: aerial photography tasks, traffic information collection tasks or flight game tasks. Among them, the aerial photography task can use at least one camera to collect aerial photography data; the traffic information collection task can use at least one camera and one or more sensors for perceiving the surrounding environment to collect traffic information; the flight game task can use at least one camera to collect Real-time images, or use at least one audio collection component to collect real-time audio data, or use one less camera and at least one audio collection component to collect real-time audio and video data. In this embodiment, the user can control the aircraft 200 to perform different types of flight missions according to actual needs, so as to obtain different types of mission data.

Wherein, the flight game task and the aerial photography task have different real-time requirements for the data collected by at least one camera, and the real-time requirement of the flight game task is higher than that of the aerial photography task.

In some embodiments, considering that different types of flight missions are generated according to different needs of users, the flight trajectories of different types of flight missions are also different. For example, the flight trajectory of some flight missions can be preset, and the flight trajectory of some flight missions is generated in real time under the control of the user.

In a possible implementation, the flight trajectory of the aerial photography task can be obtained in any of the following ways: setting the flight trajectory according to the position of the vehicle 100 , or planning the flight trajectory according to the selected shooting area. In this embodiment, the flight trajectory of the aerial photography task can be a flight trajectory set according to the position of the vehicle 100 or the selected shooting area. After the user controls the aircraft 200 to take off through the input device of the vehicle 100, the aircraft 200 can follow the set The flight track automatically performs aerial photography tasks without real-time user operations, which reduces the user's operating steps and improves user experience.

In one example, the vehicle 100 can be determined as the following object, so that the flight trajectory of the following vehicle 100 can be set according to the position of the vehicle 100, and the distance between the aircraft 200 and the vehicle 100 can be maintained during the flight following the vehicle 100. Within the preset distance range, for example, the preset distance range may be determined according to the preset imaging size of the vehicle 100 in the shooting frame, so as to ensure that the required aerial photography data is obtained.

In another example, the vehicle 100 has a variety of trajectory modes pre-stored, such as an inverted flight mode, an orbiting flight mode, a spiral flight mode, a soaring flight mode or a comet flight mode. The shooting area and the target trajectory mode are determined, and then the vehicle 100 or the aircraft 200 can determine the flight trajectory of the aircraft 200 according to the selected shooting area and target trajectory mode.

For example, in the inverted flight mode, the aircraft 200 can fly obliquely upward relative to the target object (such as the vehicle 100 or other objects selected by the user) in the selected shooting area, and the aircraft 200 can shoot the target object at the starting point. Close-up; at the end point, the aircraft 200 shoots the panorama of the environment where the target is located in the way of obliquely looking down.

For example, in the orbiting flight mode, the aircraft 200 may fly around the target within the selected shooting area. The flight trajectory of the aircraft 200 is roughly a circular trajectory. Throughout the circular trajectory, the aircraft 200 takes a 360° close-up of the target.

For example, in the spiral flight mode, the aircraft 200 can fly outwards in a spiral relative to the target within the selected shooting area. The flight path of the aircraft 200 is a spiral curve. In one example, the spiral curve may be a spiral defined by the Fibonacci sequence. At the starting point, the aircraft 200 takes a close-up shot of the target object; at the middle point and the end point, the aircraft 200 takes a panoramic view of the environment where the target object is located from the front and rear respectively in a bird's-eye view.

For example, in the soaring mode, the aircraft 200 can fly upwards relative to the target within the selected shooting area. The flight path of the aircraft 200 is roughly an L-shaped curve. At the starting point, the aircraft 200 takes a close-up shot of the target object; at the end point, the aircraft 200 takes a panoramic view of the environment where the target object is located in a way of shooting directly above.

For example, in the comet flight mode, the aircraft 200 can circle around the target in the selected shooting area and then fly out obliquely upward. The flight trajectory of the aircraft 200 is similar to that of a comet. In the initial stage, the aircraft 200 takes close-up close-ups of the target object at close to 360°; in the final stage, the aircraft 200 takes a panoramic view of the environment where the target object is located in the way of obliquely looking down.

In a possible implementation, the flight trajectory of the traffic information collection task is obtained in any of the following ways: setting the flight trajectory according to the navigation information of the vehicle 100 , or planning the flight trajectory according to the area selected in the navigation interface. In this embodiment, the navigation information of the vehicle 100 or the selected area in the navigation interface is used to set the flight trajectory, so that the aircraft 200 can collect effective traffic information that is conducive to the safe driving of the vehicle 100 during the flight process of the set flight trajectory. .

In one example, the navigation information of the vehicle 100 includes untraveled road sections in the planned route of the vehicle 100, and the vehicle 100 or the aircraft 200 can set the flight trajectory of the aircraft 200 according to the untraveled road sections of the vehicle 100, so that the aircraft 200 can fly along The trajectory collects traffic information on road sections that the vehicle 100 has not traveled, and the vehicle 100 can re-plan the navigation route based on the traffic information collected by the aircraft 200 .

In another example, the user can select an area to be detected in the navigation interface of the vehicle 100, such as a parking lot, and can plan the flight trajectory according to the selected parking lot, so that the aircraft 200 can traffic the parking lot area along the flight trajectory. information collection, the vehicle 100 can know the occupancy of parking spaces in the parking lot area based on the traffic information collected by the aircraft 200 , and can plan a navigation route to an empty parking space.

In a possible implementation, the flight trajectory of the flight game task can be generated according to real-time control instructions, that is, in the flight game task, the user can manipulate the input device (such as a built-in input device or an external input device) in the vehicle 100 in real time, Furthermore, the vehicle 100 generates real-time control instructions according to the user's manipulation behavior and sends them to the aircraft 200. The aircraft 200 can fly based on the real-time control instructions, and use at least one camera and/or at least one audio collection component to collect real-time audio during the flight. /video data.

Optionally, in order to ensure the safety of the aircraft 200 when performing the flying game mission, the flight trajectory of the flying game mission is located within a preset space range. For example, the preset space range is a parking lot or other spaces where flying games can be played.

In some embodiments, considering that different types of flight missions are generated for different needs of users, in different types of flight missions, the types of displayed information that users expect to see are also different. Exemplarily, the types of display information generated by mission data collected by different types of sensors are different. For example, different sensors such as cameras and laser radars are installed in the aircraft 200. The display information generated by the point cloud collected by the radar belongs to different categories. Exemplarily, the same or different task data are processed differently, and the types of display information obtained are also different. For example, image data is processed in two different ways, clipping processing and semantic recognition, and the display information obtained respectively belongs to different types. type. Exemplarily, if the functions of the two kinds of display information are not the same, they belong to different types of display information. For example, the display information of the traffic information collection task is used to assist safe driving, and the display information of the aerial photography task is used to take pictures to commemorate. Different kinds of display information.

In a possible implementation manner, the mission data collected by the aerial photography mission may be aerial photography data collected by at least one camera, and the vehicle 100 may generate an aerial photography image based on the aerial photography data and display it in the display area. The aerial image may be an unedited aerial video or image, or an aerial video or image obtained by editing the aerial data using a preset editing template.

In a possible implementation, the task data of the traffic information collection task may be traffic information collected by at least one camera and one or more sensors used to perceive the surrounding environment, then the vehicle 100 may generate road condition awareness information based on the traffic information and displayed in the display area. Exemplarily, the road condition awareness information includes, but is not limited to, an environment model of a traffic environment, position markers of objects, position markers of obstacles, navigation indication information pointing to objects, or navigation indication information pointing to destinations, and the like. Wherein, the environment model of the traffic environment may be a map or a three-dimensional model of the current traffic environment generated based on traffic information; objects include but are not limited to parking spaces in parking lots, road markings in roads, lane lines or drivable areas, etc. Obstacles include but are not limited to other vehicles 100, dynamic objects such as pedestrians, electric vehicles, or small animals.

Exemplarily, the vehicle 100 can determine the position of the target object in the traffic environment according to the traffic information, and then obtain the position mark of the target object according to the position of the target object, then the display area of the vehicle 100 can display the position mark of the target object, in order to further clarify the target The location of the object can also be superimposed and displayed with the location mark of the target object and the map. For example, the display area of the vehicle 100 can display a map with the location mark of the target object superimposed; The three-dimensional model of the current traffic environment is superimposed and displayed, which is not limited in this embodiment. Wherein, the map may be a map of the current traffic environment generated based on traffic information, or a map acquired by the vehicle 100 from other platforms based on its own satellite positioning module.

Exemplarily, the vehicle 100 may determine the position of the target object in the traffic environment according to the traffic information collected by the traffic information collection task, and then generate navigation indication information pointing to the target object according to the position of the target object, and send the navigation indication information pointing to the target object are displayed in the display area of the vehicle 100 . In an example, for example, in a parking lot scene, the target object is an empty parking space, and the navigation instruction information pointing to the target object may be a navigation path to the empty parking space. In another example, for example, in the parking lot scene, the target object is the exit of the parking lot, and the navigation indication information pointing to the target object may be a navigation path to the exit of the parking lot.

In a possible implementation, the mission data of the flying game mission may be the real-time picture collected by the at least one camera and/or the real-time audio collected by the at least one audio collection component, then the vehicle 100 may combine the real-time picture with the preset The game screen generated by setting rendering rules, or the game screen with sound effects generated based on real-time audio and video data combined with preset rendering rules, or the game sound effects generated based on real-time audio data combined with preset rendering rules.

In some embodiments, considering that users have different viewing requirements for the display information generated by the mission data of different types of flight missions, the display information generated by the mission data of different types of flight missions can be displayed in different positions of the display area for easy viewing by the user. In one example, please refer to FIG. 5C. FIG. 5C shows five display areas 103 located at different positions. The display information of the flight mission is displayed on the display area at different positions.

Exemplarily, the multiple display areas located at different positions may include built-in display devices or external display devices, and the built-in display devices include but are not limited to central control screens, HUD displays, instrument panels, or are arranged in front of passenger seats monitors, etc.; external display devices include but are not limited to remote control terminals or head-mounted devices. Wherein, the external display device can be directly communicated with the aircraft 200 without going through other transfer devices; or, the external display device can also be communicated with the aircraft 200 through the vehicle 100 . In an example, please refer to Figure 3, which shows display areas located in different positions, such as (1) HUD (Head Up Display) display, (2) 3D instrument panel, (4) area A of the central control screen and (6) Area B of the central control panel.

In one example, the display information of the traffic information collection task can be displayed on the central control screen or HUD display, the display information of the aerial photography task can be displayed on the central control screen, HUD display, remote control terminal or head-mounted device, and the flight game task display information Display information can be displayed on a HUD display or a head-mounted device. In another example, when the central control screen includes at least two display areas (as shown in FIG. 3 ), different display areas of the central control screen can also display display information of different types of traffic information collection tasks, such as in In Figure 3, the display information of the traffic information collection task is displayed in area A of the central control screen, and the display information of the aerial photography task is displayed in area B of the central control screen.

In a possible implementation manner, the attribute values of the specific attributes of the multiple display regions are different; the specific attributes include one or more of the following: brightness, resolution, or frame rate. Exemplarily, the multiple display areas include a first display area and a second display area located at different positions; the attribute value of the specific attribute of the first display area is superior to the attribute value of the specific attribute of the second display area. Different types of display information corresponding to different types of flight missions can be displayed in different display areas according to the display requirements of the display information. For example, the display information of flight game missions has higher requirements for specific attributes of the display area. However, the display information corresponding to the aerial photography task has relatively low requirements on the specific attributes of the display area, so the display information of the flying game can be displayed in the first display area, and the display information of the aerial photography task can be displayed in the second display area. Of course, it may also include a third display area, a fourth display area, etc., which are not limited in this embodiment. Display areas with different display performance can be selected according to actual needs to display display information of different types of flight missions.

In a possible implementation manner, the multiple display areas include a first display area and a second display area located at different positions. The first display area may include a light-transmitting area for passing through ambient light and a display area for displaying the display information, the light-transmitting area and the display area overlap at least partially, that is, the first display area may be transparent or translucent For example, the first display area is a HUD display. In addition to viewing the displayed information, the user can also see the surrounding environment of the vehicle 100 through the first display area. Exemplarily, the first display area may be located on the front window of the vehicle 100 or in front of the driver's seat, so that the driver can intuitively see the displayed information while taking care of driving. The second display area includes an opaque display area, which can display display information of flight missions. For example, the second area may be a central control screen, a display screen arranged in front of the passenger seat, a remote control terminal, or a head-mounted device.

For the display information of different types of flight tasks, the display information of different types of flight tasks can be displayed in different display areas according to the reference degree of the display information for safe driving. For example, the display information of traffic information collection tasks is important for safe driving. The reference degree of the traffic information collection task is the highest, and the display information corresponding to the aerial photography task is relatively low for safe driving. It can be considered to display the display information of the traffic information collection task in the first display area, and display the display information of the aerial photography task in the second display area. Two display areas.

In a possible implementation, the plurality of display areas include a first display area and a second display area located at different positions; The angle between them is greater than the angle between the direction of the line connecting the first display area and the headrest of the driver's seat and the forward direction of the vehicle 100 . That is to say, the first display area is set at a position that is more convenient for the driver to view. When the driver is driving the vehicle 100, the degree of line-of-sight adjustment required to view the first display area is smaller than that required to view the second display area. The degree of sight direction adjustment enables the driver to see the displayed information intuitively and conveniently while taking into account the driving, which is conducive to ensuring driving safety. Exemplarily, the first display area may be a HUD display or an instrument panel; the second display area may be a central control screen, or a display arranged in front of a passenger seat, and the like. Exemplarily, in the case where the central control screen includes at least two display areas (as shown in FIG. 3 ), the first display area may be a display area close to the driver in the central control screen (such as the central control screen in FIG. 3 Area A), the second display area may be a display area far away from the driver in the central control screen (such as area B of the central control screen in FIG. 3 ).

In an exemplary embodiment, display information corresponding to different types of flight missions may be displayed in display areas with different levels of user attention based on reference levels of different types of flight missions to the driving safety of the vehicle 100 . As mentioned above, if the first display area is an area that is convenient and intuitive for the user, the user pays more attention to the first display area, and secondly to the second display area. Therefore, the display information corresponding to the missions that are strongly related to the driving safety of the vehicle 100 can be displayed in the first display area, and the display information corresponding to the missions that are weakly related to the driving safety of the vehicle 100 can be displayed in the second display area. In this embodiment, the display information related to driving safety is displayed in the first display area that the driver can easily see, which is beneficial to further improve driving safety.

Taking the aerial photography task and the traffic information collection task as examples, considering that the display information corresponding to the traffic information collection task is closely related to the driving safety of the vehicle 100, the display information corresponding to the traffic information collection task can be displayed in a position that the driver can intuitively see. For example, the display information corresponding to the traffic information collection task is displayed in the first display area that the driver mainly pays attention to, for example, the first display area is the HUD display or the central control screen A area in Figure 3; If the display information of the vehicle 100 is not closely related to the driving safety of the vehicle 100, the display information corresponding to the aerial photography task can be displayed in the second display area that the driver pays second attention to. For example, the second display area is the central control screen B in FIG. 3 area, a monitor or remote control terminal installed in front of the passenger seat, etc.

In an exemplary embodiment, after the vehicle 100 generates display information (hereinafter referred to as the first display information for convenience) according to the mission data of the flight mission, if the vehicle 100 does not currently display the display information of other flight missions (hereinafter referred to as the first display information), called the second display information), the first display information can be directly displayed in the first display area where the user pays the most attention and can be seen intuitively, and subsequently if the vehicle 100 needs to display the second display information of other flight missions, And the second display information is strongly related to the driving safety of the vehicle 100 and the first display information is weakly related to the driving safety of the vehicle 100, then the second display information can be displayed in the first display area that the driver mainly pays attention to, and the first display information can be switched. Display in the second display area that the driver pays second attention to. In this embodiment, the flexible switching of display information is realized, which is beneficial to meet the user's viewing requirements.

Taking aerial photography tasks and traffic information collection tasks as examples, the aircraft 200 collects task data (such as aerial photography data) during the execution of aerial photography tasks and sends them back to the vehicle 100. The vehicle 100 generates display information (such as aerial photography images) according to the task data. If the vehicle 100 did not execute the traffic information collection task before performing the aerial photography task, that is, did not generate the display information corresponding to the traffic information collection task, then the vehicle 100 can display the display information corresponding to the aerial photography task in the first display area; if the aircraft 200 Continue to execute the traffic information collection task after performing the aerial photography task, or the aircraft 200 takes into account the execution of the traffic information collection task during the execution of the photography task, and the aircraft 200 collects the task data (such as traffic information) of the traffic information collection task and sends it back to Vehicle 100, after the vehicle 100 generates display information according to the task data of the traffic information collection task, considering that the display information of the traffic information collection task is strongly related to the driving safety of the vehicle 100, its display priority is higher than that of the aerial photography task. is high, the display information corresponding to the traffic information collection task can be displayed in the first display area that the user pays the most attention to and can be seen intuitively, and the display information corresponding to the aerial photography task can be switched and displayed in the second focus of the user. Second display area.

In an exemplary embodiment, the display information corresponding to the same flight mission may also be displayed in different display areas at the same time or at different times. Taking the traffic information collection task as an example, the task data of the traffic information collection task includes traffic information in the traffic information. For example, the vehicle 100 can generate display information according to the location information and type information of the target object extracted from the traffic information. In one example, for example, if the target object is a parking space, the type information indicates the occupancy state of the parking space, including the "occupied" type and the "unoccupied" type (that is, an empty parking space). In another example, for example, if the target object is a traffic sign (such as a traffic light), the type information represents different colors of the red street light, such as a red light type, a green light type, and a yellow light type.

The vehicle 100 may display the display information corresponding to the traffic information collection task in the first display area, for example, the first display area is a HUD display, wherein the display position of the display information in the first display area is based on The relative positional relationship between the target object and the vehicle 100 is determined, so as to ensure that the display information can be displayed in accordance with the surrounding environment of the vehicle 100 seen by the user through the light-transmitting area of the first display area, so as to provide accurate guidance information for the user.

The vehicle 100 may also display the display information corresponding to the traffic information collection task in a second display area, for example, the second display area is a central control screen, wherein the display position of the display information in the second display area is based on the The position information of the target object and the display scale of the object indicated by the second display area are determined. The display ratio of the object may refer to the display ratio of the map or three-dimensional model of the current traffic environment displayed in the second display area, so as to ensure that the displayed information can be displayed in accordance with the map or three-dimensional model of the current traffic environment, so as to provide users with accurate information. boot information.

In an exemplary embodiment, here is an exemplary description of the traffic information collection task:

In a possible implementation, in the parking lot scene, please refer to FIG. 6 , in figure (a), the vehicle 100 equipped with the aircraft 200 is driving on the road, if it is detected that the vehicle 100 has driven to the vicinity of the destination , the vehicle 100 can output prompt information whether to allow the aircraft 200 to perform the traffic information collection task, the prompt information can be visual information or auditory information, and the traffic information collection task is used to allow the aircraft 200 to detect the parking lot near the destination. The detection data of 200 can clarify the occupancy of parking spaces in the parking lot so as to find empty parking spaces. The vehicle 100 includes at least one input device. For example, in FIG. 6 (b), the user can operate the central control screen to confirm that the aircraft 200 is to perform the traffic information collection task, and the vehicle 100 can generate a control instruction for the traffic information collection task and sent to the aircraft 200. Optionally, the parking lot may also be selected by the user in the navigation interface.

In Figure 6(c), the aircraft 200 can take off from a designated position (such as the roof) of the vehicle 100 in response to a control command to perform the traffic information collection task, wherein the flight trajectory of the traffic information collection task is in the area where the parking lot is located. Inner planning, for example, the flight trajectory can be a bow-shaped trajectory covering the parking lot. The aircraft 200 collects task data and sends it back to the vehicle 100 during the flight according to the planned flight trajectory. For example, the task data may be the environmental information of the parking lot.

The vehicle 100 can create a parking lot map according to the task data, and can know the occupancy of the parking spaces in the parking lot and determine the positions of the empty parking spaces through the task data, and then can plan the navigation path of the vehicle 100 according to the positions of the empty parking spaces, so that the vehicle 100 Ability to go to the location of an empty parking space based on the navigation path. For example, in (d) of FIG. 6 , a parking lot map superimposed with position signs of vacant parking spaces and a navigation route can be displayed in the display area of the vehicle 100 (such as the central control screen of the vehicle 100 ). Optionally, the vehicle 100 may also obtain the parking lot map from other platforms based on its own satellite positioning module, which is not limited in this embodiment. When the user confirms to park according to the navigation route, as shown in (e) of FIG. 6 , the vehicle 100 may drive to an empty parking space based on the navigation route to complete the parking behavior. In this embodiment, the detection of the parking lot by the aircraft 200 is beneficial to quickly find an empty parking space and improve parking efficiency.

Optionally, referring to FIG. 7, the display area includes a HUD display. After the display information is generated according to the task data, the vehicle 100 may also display the display information on the HUD display. For example, the display information may include label information of the location of an empty parking space. (As shown in A in Figure 7), the parking space marking information of the empty parking space (as shown in B in Figure 7), the global navigation path pointing to the empty parking space (as shown in C in Figure 7) and the current navigation path pointing to the empty parking space (as shown in Figure 7 D in 7). Wherein, the display position of the displayed information on the HUD display can be determined according to the relative positional relationship between objects (empty parking spaces, other vehicles 100, buildings or trees, etc.) The surrounding environment of the vehicle 100 seen by the display matches the display, so that accurate guidance information can be provided for the user.

Optionally, when the user wants to go from the parking lot to the next destination, the user can also control the aircraft 200 to execute the traffic information collection task through the vehicle 100. The task data collected by the traffic information collection task includes the distance between the vehicle 100 and the exit of the parking lot. environment information of the area in between, the vehicle 100 can generate a map of the area between the vehicle 100 and the exit of the parking lot according to the task data, determine the position of the exit of the parking lot, and obtain the navigation pointing to the exit of the parking lot according to the position of the exit of the parking lot Instruction information (such as a navigation path pointing to the exit of the parking lot), and then can display a map superimposed with the parking lot exit mark and/or navigation path in the display area (such as the central control screen or HUD display); optionally, The vehicle 100 can also obtain the parking lot map from other platforms based on its own satellite positioning module, which is not limited in this embodiment.

In another possible implementation manner, the traffic information collection task is used to detect untraveled road sections in the planned route of the vehicle 100 to determine traffic conditions of the untraveled road sections by the vehicle 100 . Please refer to FIG. 8, as shown in (a) of FIG. 8, if the vehicle 100 encounters a traffic jam during driving, due to the limited detection range of the vehicle 100, it is impossible to detect the cause of the congestion ahead, as shown in ( b) As shown in the figure, the user can control the aircraft 200 to perform traffic information collection tasks through the central control panel of the vehicle 100 , for example, the vehicle 100 can send control instructions for traffic information collection tasks to the aircraft 200 . Certainly, other input devices (such as a steering wheel control, a somatosensory remote control or a camera, etc.) may also be manipulated to generate control instructions. As shown in Figure 8 (c), the aircraft 200 takes off from a designated position (such as the roof) of the vehicle 100 to perform a traffic information collection task, and the task data of the traffic information collection task includes the traffic information of the untraveled road section of the vehicle 100, The aircraft 200 then transmits the collected mission data back to the vehicle 100 .

The vehicle 100 can generate display information according to the mission data, as shown in FIG. The display area of the vehicle 100 (such as the central control screen) displays a map superimposed with position marks of obstacles (such as other vehicles 100), which includes a map of the untraveled road section of the vehicle 100, and the content displayed in the display area can let the user The reason for the congestion in front of the road section that the vehicle 100 has not traveled is clarified, so that the vehicle 100 can re-plan the navigation route to the destination and avoid the congested road section. As shown in (e) of Figure 8, after the aircraft 200 completes the task of collecting traffic information, the user can control the aircraft 200 to land through the input device (such as the steering wheel control) of the vehicle 100, and accordingly, the vehicle 100 can generate a 200 of the landing control command, as shown in (f) of FIG. 8 , the aircraft 200 may land on the roof of the vehicle 100 in response to the landing control command. In this embodiment, the aircraft 200 is used to perform the traffic information collection task, and the risk factors beyond the visual range of the vehicle 100 and human beings can be observed in advance, so that the vehicle 100 can carry out automatic or autonomous decision-making and planning for the next driving task. The objective shooting of the aircraft 200 in the air can reflect more realistic and uninterrupted human driving behavior, and can more efficiently collect data on specific road sections and special driving scenarios, such as on-ramps, frequent traffic jams, etc.

In an exemplary embodiment, here is an exemplary description of the aerial photography task: please refer to FIG. 9, as shown in (a) and (b) of FIG. If the driver or passenger wants to record the scenery along the way, he can control the aircraft 200 from the vehicle 100 by manipulating the input devices in the vehicle 100 (such as the central control panel, steering wheel controls, audio collection components, remote control terminal, somatosensory remote control or camera, etc.) To take off to perform the aerial photography mission, correspondingly, the vehicle 100 generates a control command of the aerial photography mission based on the user manipulation input device and sends it to the aircraft 200 .

As shown in (c) of Figure 9, assuming that the flight trajectory of the aerial photography task is set based on the position of the vehicle 100, such as following the flight trajectory of the vehicle 100, the aircraft 200 follows the vehicle 100 during the aerial photography task. Shooting, obtain mission data (such as aerial photography data) of the aerial photography task and send it back to the vehicle 100 . Optionally, during the shooting process, the aircraft 200 can adjust the composition of the captured image or video through the preset camera AI algorithm, so as to improve the display effect. As shown in (d) of Figure 9, the vehicle 100 can generate an aerial image based on the aerial data. For example, the vehicle 100 can use a preset editing template to edit the aerial data to obtain an aerial image, and display the aerial image in the display area of the vehicle 100 (such as displayed on the control screen). Optionally, users can also share the effects of aerial photography to third-party social platforms. As shown in (e) of Figure 9, after the aircraft 200 performs the aerial photography task, the user can control the aircraft 200 to land through the input device of the vehicle 100 (such as steering wheel controls, central control screen, voice control, or gesture control, etc.) , correspondingly, the vehicle 100 may generate a landing control instruction sent to the aircraft 200, as shown in (f) of FIG. 9 , the aircraft 200 may land on the roof of the vehicle 100 in response to the landing control instruction. This embodiment realizes that in the automatic driving state, the user can control the aircraft 200 to perform aerial photography tasks through the vehicle 100 anytime and anywhere (except in the restricted flight area), so as to efficiently complete the photography work and improve the user experience.

In an exemplary embodiment, the flying game task is exemplified here: in order to improve the safety of the flying game task, the flying game task is carried out within a preset space range, such as a parking lot or other possible A space for flying games; or the flying game task is executed when the driving speed of the vehicle 100 is lower than a preset value (such as 20km/h) or in a non-driving state. When the user is in the space of the vehicle 100, the user can control the aircraft 200 (such as the FVP aircraft 200) to take off from the vehicle 100 through the input device in the vehicle 100 to perform flight game tasks. The aircraft 200 can be controlled by input devices such as a camera (gesture control) or an audio collection component (voice control), and passengers can control the aircraft through input devices such as a remote control terminal, a somatosensory remote control, a camera (gesture control) or an audio collection component (voice control). 200. Correspondingly, the vehicle 100 can send a real-time control instruction of the flight game mission to the aircraft 200 , and the aircraft 200 executes the flight game mission based on the real-time control instruction, collects real-time images and sends them back to the vehicle 100 . The vehicle 100 can generate a game screen according to the real-time screen combined with preset rendering rules. For example, the game screen can be a real-time screen superimposed with speed, flight trajectory or virtual information (such as virtual gold coins, virtual characters, etc.), and then the vehicle 100 can display the game screen Displayed in the display area, for example, the displayed information may be displayed on a HUD display or displayed on a head-mounted device, and the user may have an immersive flight experience of controlling/viewing the aircraft 200 .

In some embodiments, when the display area where the display information corresponding to the flight mission is located is not fixed, the user can also switch and display the display information corresponding to the flight mission to other display areas based on actual needs. The vehicle 100 includes at least one input device, and the input device includes one or more external input devices, and/or one or more built-in input devices; the user in the vehicle 100 can manipulate the built-in input device or the external input device to generate a switching control command , the vehicle 100 switches and displays the display information in other display areas in response to the display switching instruction. In this embodiment, the user can switch the display information to a desired display area for display according to the actual needs of the user, so as to meet the personalized needs of the user.

Among them, the built-in input device includes any of the following: a built-in first camera, a voice collection component, a central control panel or a steering wheel control; an external input device includes any of the following: a remote control terminal, a somatosensory remote control, or through a pluggable interface A second camera communicatively coupled to the vehicle 100 . The switching control instruction can be generated according to any of the following data collected by the input device: user gestures collected by the first camera and/or the second camera; voice signals collected by the voice collection component; one of the central control screen, the steering wheel control and the remote control terminal One or more types of received operation data; or motion sensory data collected by the sensory remote controller.

Taking gesture control as an example, the vehicle 100 is provided with a camera (such as a built-in first camera and/or a second camera communicatively connected to the vehicle 100 through a pluggable interface), and the vehicle 100 can acquire user images collected by the camera, according to Determine whether to generate the display switching instruction from the user gesture recognized in the user image, for example, after the user gesture is recognized, the instruction corresponding to the recognized user gesture can be determined according to the pre-stored correspondence between the gesture and the instruction. Of course, in addition to gesture control, facial expression control can also be used, and its control logic is similar to that of gesture control, which will not be repeated here.

In some embodiments, the vehicle 100 may be provided with a built-in first camera and/or a second camera in which the camera communicates with the vehicle 100 through a pluggable interface, and the first camera and/or the second camera may also be To perform the shooting task, the vehicle 100 can display the image or video obtained by the first camera and/or the second camera in any of the display areas, or use the first camera and/or the second camera to perform the shooting task. The image or video and the display information corresponding to the flight mission are superimposed and displayed in the display area. In this embodiment, various display effects are realized to meet the personalized viewing needs of users.

In some application scenarios, such as live broadcast scenarios, video conference scenarios, demonstration scenarios or survey scenarios, etc., the vehicle 100 can also send the images obtained by the first camera and/or the second camera to perform the shooting task and the mission data of the flight mission. To the remote device, so that the display information generated by the mission data of the flight mission and the image obtained by the first camera and/or the second camera performing the shooting mission are displayed on the remote device, so as to realize effective interaction of information. Exemplarily, the image obtained by the first camera and/or the second camera performing the shooting task and the display information may also be superimposed and displayed in the remote device. In one example, for example, if the image or video obtained by the first camera and/or the second camera performing the shooting task is a scene where the user manipulates the aircraft 200 to perform the flight task, then the display information corresponding to the flight task can be combined with the first camera and/or The superimposed display of the image or video obtained by the second camera performing the shooting task can let the viewer understand the flight effect produced by the user's manipulation behavior.

In some embodiments, the vehicle 100 includes a second camera communicatively connected to the vehicle 100 through a pluggable interface, please refer to FIG. 10A and FIG. 10B , FIG. 10A shows a scene where the second camera 104 is set in the vehicle 100 , FIG. 10B shows a scene where the second camera 104 is used independently. When the second camera 104 is communicatively connected to the vehicle 100 through a pluggable interface, in addition to collecting user gestures, the second camera 104 can also be used for other functions. After the vehicle 100 detects that the body of the vehicle 100 is in communication with the second camera 104, it can acquire the user image captured by the second camera 104, and use the user image to perform different functions in different working modes. Exemplarily, the working mode includes a first working mode and a second working mode; the first working mode indicates to recognize the user image to obtain user gestures; the second working mode indicates to use the user image to monitor the driver's driving Status (i.e. DMS detection).

In a possible implementation manner, the vehicle 100 is provided with a switching control, and after detecting that the vehicle 100 body is in communication with the second camera 104, the switching control can change the current working mode from the first working mode to the first working mode in response to the switching trigger. Switching to the second working mode, the vehicle 100 uses the user image to monitor the driving state of the driver after switching; or the switching control can switch the current working mode from the second working mode to the first working mode in response to the switching trigger, After switching, the vehicle 100 recognizes the user image to obtain the user gesture, so as to control the vehicle 100 or the aircraft 200 according to the user gesture.

Optionally, when the second camera 104 is communicatively connected to the vehicle 100 through the pluggable interface, the vehicle 100 can also charge the second camera 104 through the pluggable interface. Optionally, the second camera 104 can be magnetically fixed with the vehicle 100 through a pluggable interface.

Optionally, the vehicle 100 can be provided with a pluggable interface at one or more different positions, so as to establish a connection with the second camera 104 through the pluggable interface, and the user can place the second camera 104 in the vehicle 100 according to actual needs In different positions of the vehicle 100, multiple second cameras 104 can also be placed in the vehicle 100. For example, as shown in FIG. 104 fixed connections are conducive to meeting the individual needs of users.

Optionally, the second camera 104 is a camera with processing capabilities and operational controls. When the second camera 104 is placed independently of the vehicle 100, the second camera 104 establishes a near field communication connection with the vehicle 100, and the near field communication connection can be used to The vehicle 100 is controlled to open and close the lock. Optionally, when the second camera 104 is a camera with processing capabilities and operational controls, the second camera 104 can also use the collected images to recognize user gestures or monitor the driver's state, and its processing logic is the same as that of the vehicle using the image processing The logic is similar and will not be repeated here.

Optionally, when the second camera 104 is placed independently of the vehicle 100, the second camera 104 will perform the shooting task to obtain images or videos stored, and when it is detected that the second camera 104 is connected to the vehicle 100 through a pluggable interface , and send the stored image or video to the vehicle 100 .

In some embodiments, the aircraft 200 can be placed at a designated location of the vehicle 100 . Exemplarily, the body of the vehicle 100 is provided with an aircraft 200 apron, and the aircraft 200 can be placed in the apron of the aircraft 200 . Exemplarily, the specified location may be the roof of the vehicle 100 .

In some embodiments, the aircraft 200 may take off from a designated position of the vehicle 100 to perform flight missions, for example, the aircraft 200 may take off from a designated position of the vehicle 100 in response to a take-off trigger. Wherein, when the aircraft 200 takes off, the vehicle 100 may be in a driving state or a non-driving state, such as parked in a parking lot.

Exemplarily, in order to ensure the flight safety of the aircraft 200, when the vehicle 100 is moving relative to the ground, that is, when the vehicle 100 is in a driving state, the aircraft 200 can take off from a designated position of the vehicle 100 to within a preset height range , and maintain a relative displacement within the preset horizontal distance range with the vehicle 100 in the horizontal direction; wherein, the preset height range and the preset horizontal distance range can be determined according to the communication distance between the vehicle 100 and the aircraft 200, so as to ensure that the vehicle 100 During the driving process, the aircraft 200 can always maintain a communication state with the vehicle 100 to prevent the aircraft 200 from being lost, thereby helping to ensure the flight safety of the aircraft 200 .

Of course, when the vehicle 100 is in a non-driving state, after the aircraft 200 takes off from the designated position of the vehicle 100, it can also fly within the preset altitude range and maintain a preset horizontal distance from the vehicle 100 in the horizontal direction. Relative displacement, so as to ensure the flight safety of the aircraft 200.

In some embodiments, the aircraft 200 may land at a designated location of the vehicle 100 after completing the flight mission. Exemplarily, the body of the vehicle 100 is provided with an aircraft 200 apron, and the aircraft 200 can land on the apron of the aircraft 200 after completing the flight mission. Wherein, when the aircraft 200 lands, the vehicle 100 may be in a driving state or in a non-driving state, such as parked in a parking lot.

When the vehicle 100 is in a driving state, in order to realize the safe and reliable landing of the aircraft 200 , the aircraft 200 may land to a designated position when the vehicle 100 is moving at a constant speed after completing the flight mission. Exemplarily, after the aircraft 200 completes the flight mission, it can send the instruction information of completing the flight mission to the vehicle 100, and the vehicle 100 can control the vehicle 100 to move at a constant speed or report to the driver after receiving the instruction information of completing the flight mission sent by the aircraft 200. The operator sends a reminder message to keep the vehicle 100 at a constant speed.

Of course, in order to ensure the safety of the aircraft 200, some special circumstances can also automatically trigger the aircraft 200 to land at the designated position of the vehicle 100, for example, the power of the aircraft 200 is lower than a preset value, or some parts in the aircraft 200 are damaged, or The aircraft 200 may be automatically triggered to land on the designated position of the vehicle 100 when the aircraft 200 hits an obstacle or the like.

In some possible implementations, when the aircraft 200 needs to land at a designated position of the vehicle 100, the aircraft 200 may respond to the landing trigger (such as after completing the flight mission or the above-mentioned situation occurs), search for the vehicle 100 to be landed, and Determine the relative pose relationship between the aircraft 200 and the vehicle 100 to be landed; estimate the predicted motion information of the vehicle 100 to be landed according to the motion information and the relative pose relationship of the aircraft 200, so as to plan the landing path of the aircraft 200; The aircraft 200 is controlled to land on the vehicle 100 , for example, to a designated position of the vehicle 100 .

The search for the vehicle 100 to be landed may include the following two optional implementations:

In an optional implementation, considering that the aircraft 200 is communicatively connected with the vehicle 100, the vehicle 100 can send its own motion information to the aircraft 200, and the aircraft 200 can receive the motion information of the vehicle 100 sent by the vehicle 100, and according to The motion information of the vehicle 100 acquires the target motion trajectory; and the aircraft 200 can observe one or more objects on the ground according to the sensors used to perceive the surrounding environment (such as visual sensors, laser radars, millimeter wave radars, ultrasonic sensors or infrared sensors, etc.) Moving objects, determine the trajectory of one or more moving objects on the ground; then the aircraft 200 can search for one or more moving objects according to the difference between the target trajectory and the trajectory of one or more moving objects. Landed vehicles 100. Exemplarily, the aircraft 200 may determine the similarity between the target trajectory and the trajectory of one or more moving objects, and determine the moving object corresponding to the trajectory with the largest similarity as the vehicle 100 to be landed. Wherein, the motion information of the vehicle 100 can be the odometer information of the vehicle 100, the odometer information includes the pose information and the speed information of the vehicle 100, the motion information of the vehicle 100 can be the sensor (such as pose sensor, inertial Vehicle 100 unit or satellite positioning module, etc.) to obtain.

In another possible implementation, the designated position of the vehicle 100 is provided with landing visual signs, such as several visual signs as shown in Figure 11, of course, other styles of visual signs are also possible. There are no restrictions on this; the aircraft 200 is equipped with a visual sensor, and the visual sensor can collect images on the ground side towards the ground, and the aircraft 200 can identify whether there is a landing visual sign from the image, and determine the landing site according to the recognized landing visual sign. Vehicle 100. Exemplarily, the aircraft 200 has pre-stored landing visual signs, and the aircraft 200 can match the sign data recognized from the image with the pre-stored landing visual signs, and determine the vehicle 100 to be landed from the matched images if the matching is consistent.

After determining the vehicle 100 to be landed, the aircraft 200 needs to determine the relative pose relationship between the aircraft 200 and the vehicle 100 to be landed, which may include the following several optional implementations:

In a possible implementation, after the vehicle 100 to be landed is determined, the aircraft 200 can track the vehicle 100 to be landed to obtain the trajectory of the vehicle 100 to be landed according to the data collected by the sensor used to perceive the surrounding environment. The trajectory of the vehicle 100 to be landed determines the relative pose relationship between the aircraft 200 and the vehicle 100 to be landed.

In a possible implementation, considering the communication connection between the aircraft 200 and the vehicle 100, the vehicle 100 may send its own movement information to the aircraft 200; the communication connection between the aircraft 200 and the vehicle 100 includes a near field communication connection. The aircraft 200 can determine the relative distance between the aircraft 200 and the vehicle 100 based on the near field communication connection with the vehicle 100, and then determine the distance between the aircraft 200 and the vehicle 100 according to the movement information of the vehicle 100 sent by the vehicle 100, the movement information of the aircraft 200 and the relative distance. The relative pose relationship between 100. Exemplarily, the near-field communication connection includes a WIFI connection or a UWB connection; the motion information of the vehicle 100 includes pose information and speed information of the vehicle 100, and the motion information of the vehicle 100 can be based on sensors of the vehicle 100 (such as a pose sensor, an inertial vehicle 100 unit or satellite positioning module, etc.);

Exemplarily, if the near field communication connection is a WIFI connection, the aircraft 200 can determine the relative distance between the aircraft 200 and the vehicle 100 based on the signal strength of the WIFI connection; or, the WIFI fingerprint can be obtained according to the signal characteristics of the vehicle 100 to be landed, Determine the relative distance between the aircraft 200 and the vehicle 100 according to the WIFI fingerprint; or, set a wifi router in the vehicle 100, and the aircraft 200 can determine the relative distance between the aircraft 200 and the vehicle 100 (wifi router) based on Wi-Fi RTT positioning technology .

Exemplarily, UWB is a wireless carrier communication technology, that is, it does not use a sinusoidal carrier, but uses nanosecond-level non-sinusoidal narrow pulses to transmit data, so it occupies a wide spectrum range. UWB is a technology that uses nanosecond-level narrow pulses to transmit wireless signals, and is suitable for high-speed, short-distance wireless personal communications. The UWB positioning technology adopts TOF (time-of-flight) distance measurement, which mainly uses the flight time of the signal between the aircraft 200 and the vehicle 100 to be landed to measure the relative distance between the aircraft 200 and the vehicle 100 .

In one example, after obtaining the relative distance between the vehicle 100 and the aircraft 200, the motion information of the vehicle 100 and the motion information of the aircraft 200 sent by the vehicle 100, considering that both the motion information of the vehicle 100 and the motion information of the aircraft 200 are based on their own The sensor is collected, and the two are not in the same coordinate system, so the relative pose relationship between the motion information of the vehicle 100 and the motion information of the aircraft 200 can be found through the relative distance between the vehicle 100 and the aircraft 200 . For example, the aircraft 200 can obtain the vehicle trajectory of the vehicle 100 in the vehicle 100 coordinate system within a preset time period according to the motion information of the vehicle 100, and obtain the coordinates of the aircraft 200 in the aircraft 200 within the same preset time period according to the motion information of the aircraft 200. The trajectory of the aircraft 200 under the system, and obtain the relative distance between the vehicle 100 and the aircraft 200 within the same preset time period, and determine based on the vehicle trajectory, the aircraft 200 trajectory, and the relative distance between the vehicle 100 and the aircraft 200 within the same preset time period The relative pose relationship between the aircraft 200 and the vehicle 100 is obtained.

In a possible implementation, the roof of the vehicle 100 is equipped with a TOF sensor, and the aircraft 200 is provided with a material with a specific reflectivity for the waves emitted by the TOF sensor; For example, the material may be placed on the abdomen of the aircraft 200 (ie, the position where the aircraft 200 faces the ground during flight). In response to the landing trigger of the aircraft, the TOF sensor can emit beams towards the sky, such as the laser pulse of the lidar, the ultrasonic wave of the ultrasonic sensor, and obtain the reflected echo data; since there are generally not many objects in the sky and roadside leaves , buildings have low reflectivity, and the aircraft 200 has and is equipped with materials with specific reflectivity to the waves emitted by the TOF sensor, so the waves emitted by the TOF sensor have a high reflectivity, then the vehicle 100 can use the TOF Among the echo data collected by the sensor, the data whose reflectivity is greater than or equal to a specific reflectivity is determined as the target data corresponding to the aircraft 200 , and the relative pose relationship between the aircraft 200 and the vehicle 100 is determined according to the target data and fed back to the aircraft 200 . Exemplarily, the TOF sensor includes lidar, millimeter wave radar, infrared sensor or ultrasonic sensor.

Among them, the material with a specific reflectivity of the wave can be a material that conforms to the Beer-Lambert law, and the Beer-Lambert law (also known as Beer law, Lambert-Beer law. Beer-Lambert-Bonguer law) is the The law of absorption. It applies to all electromagnetic radiation and to all light-absorbing substances. Including gases, solids, liquids, molecules, atoms and ions. The physical meaning of the Beer-Lambert law is: when a beam of parallel monochromatic light passes through a uniform non-scattering light-absorbing substance vertically. Its absorbance A is proportional to the concentration c of the light-absorbing substance and the thickness l of the absorbing layer. When the medium contains many light-absorbing components, as long as there is no interaction between the components, at a certain wavelength, the total absorbance of the medium is the sum of the absorbance of each component at that wavelength. This rule is called the addition of absorbance. sex. Based on the Beer-Lambert law, materials can be designed that have different reflectivities for certain wavelengths.

In one example, the lidar is mounted on the roof of the vehicle 100 and the aircraft 200 is provided with a material having a specific reflectivity for laser pulses. The basic principle of lidar is to transmit a signal (laser pulse) to the target, and then compare the received signal reflected from the target (target echo) with the transmitted signal, and then obtain relevant information about the target, such as target distance, azimuth, Parameters such as height, speed, attitude, and even shape. In response to the landing trigger of the aircraft 200, the vehicle 100 can use the lidar to emit laser pulses towards the sky. Since there are generally not many objects in the sky and the reflectivity of roadside leaves and buildings is low, the aircraft 200 has and set There is a material with a specific reflectivity to the laser pulse, so it has a relatively high reflectivity to the laser pulse, then the vehicle 100 can determine the three-dimensional point in the point cloud collected by the lidar with a reflectivity greater than or equal to the specific reflectivity as the corresponding point of the aircraft 200. The target three-dimensional point, according to the depth information and angle information of the target three-dimensional point, determines the relative pose relationship between the aircraft 200 and the vehicle 100 and feeds it back to the aircraft 200 .

In an exemplary embodiment, please refer to FIG. 12. FIG. 12 shows a schematic diagram of the landing processing of the aircraft 200, including five parts, sensors, image detection, body coordinate system trajectory planning, trajectory tracking and controller, and flight control (i.e. flight controller). Optionally, image detection, body coordinate system trajectory planning, trajectory tracking and controller can also be integrated in the flight controller. The processing logic of the landing of the aircraft 200 involves three coordinate systems, namely the world coordinate system, the body coordinate system and the tag coordinate system. The world coordinate system uses the starting point of the aircraft 200 as the origin of the coordinate system, the body coordinate system uses the center of gravity of the aircraft 200 as the origin of the coordinate system, and the tag coordinate system uses the visual marker as the center as the origin of the coordinate system.

During the landing mission of the aircraft 200 in response to a landing trigger (such as a landing trigger command), the aircraft first observes a visual identifier (such as an AprilTag array) through a monocular camera. Since the positioning result of the aircraft in the world coordinate system is known, it can be Perform coordinate transformation on the pose of the visual marker, transform it into the world coordinate system, and then use the Kalman linear filter for smoothing to generate the pose and speed of the vehicle 100 to be landed with the visual marker in the world coordinate system . The aircraft itself uses a binocular camera and an inertial measurement unit for positioning. Through the extended Kalman filter, the attitude and velocity feedback of the aircraft are obtained, and then the acquired attitude, velocity and other data are sent to the trajectory tracking and controller for attitude control. .

During the landing mission of the aircraft 200, the aircraft uses the relative pose between the aircraft and the vehicle 100 to perform trajectory planning in the body coordinate system, and then passes the trajectory of the high-order continuous body coordinate system through the vehicle 100 to be landed. The rotation and translation matrix in the world coordinate system is projected onto the world coordinate system to achieve the smoothness of the landing process of the aircraft. The aircraft 200 can transform the pose of the vehicle 100 to be landed in the world coordinate system and the pose of the aircraft obtained in the above manner into the body coordinate system through coordinate rotation and translation, and determine the body coordinates between the aircraft and the vehicle 100 to be landed. The relative pose under the system, set the position of the aircraft in the tag coordinate system as the starting point of the landing trajectory, calculate the expected landing target position, and then set the 0.5 meters in front of the visual marker as the end point of the landing trajectory under the tag coordinate system, and then set Constraints such as maximum speed and acceleration on the landing trajectory, use the MINCO trajectory planning period to plan a landing trajectory suitable for the flight of the aircraft. The landing trajectory is sent to the trajectory tracking and controller, and the rotation and translation matrix of the vehicle 100 to be landed in the world coordinate system is projected to the world coordinate system, so as to perform feedback control to control the landing of the aircraft to the expected landing point.

In some embodiments, when the aircraft 200 is placed on the vehicle 100, the aircraft 200 can be used as one of the devices of the vehicle 100 for sensing the surrounding environment, and the aircraft 200 can collect traffic information of the environment around the vehicle 100 and send it to the vehicle 100, so as to assist the vehicle 100 in perceiving the surrounding environment. Exemplarily, the aircraft 200 can collect traffic information of the vehicle 100 to detect the blind area, so as to assist the vehicle 100 to perceive the surrounding environment accurately and comprehensively.

In an exemplary embodiment, when the aircraft 200 is fixed on the vehicle 100, the vehicle 100 can fuse the first traffic information in the traffic environment collected by the aircraft 200 with the environmental information collected by the vehicle sensor of the vehicle 100, as the environment information of the vehicle 100 Perceptual information reference; when the aircraft 200 executes the traffic information collection task, the vehicle 100 can collect the second traffic information in the traffic environment collected by the aircraft 200 (in order to facilitate the distinction from the first traffic information, here the task data corresponding to the traffic information collection task referred to as the second traffic information) and the pose information when the aircraft 200 collects the second traffic environment information are used as a reference for the navigation information of the vehicle 100 . In this embodiment, when the aircraft 200 is fixed on the vehicle 100, the sensing range of the aircraft 200 is limited, so the aircraft 200 can be used as one of the devices of the vehicle 100 for sensing the surrounding environment, and then combined with the on-board sensors in the vehicle 100, to achieve accurate And perceive the surrounding environment of the vehicle 100 in all directions; when the aircraft 200 performs a flight mission, the aircraft 200 can perceive traffic information at a longer distance, and observe risk factors beyond the visual range of the vehicle 100 and humans in advance, which can provide navigation for the vehicle 100 for reference.

Wherein, when the aircraft 200 performs a flight mission, the distance between the detected vehicle 100 and objects in the traffic environment is longer. The first traffic information includes first distances between the vehicle 100 and multiple objects in the traffic environment; the second traffic information includes second distances between the vehicle 100 and multiple objects in the traffic environment; wherein, among the multiple first distances The largest of is smaller than the largest of the plurality of second distances.

In a possible implementation, considering that the altitude of the aircraft 200 is different in the scenario of performing a flight mission and in the scenario of being fixed on the vehicle 100, the orientation of the sensor used to collect the first traffic information in the aircraft 200 is the same as that used for The sensors for collecting the second traffic information have different orientations. Exemplarily, when the aircraft 200 is fixed on the vehicle 100, the height of the aircraft 200 is not much different from the height of the vehicle 100, and the sensor used to collect the first traffic information faces the front or side of the aircraft 200 to realize the detection blind spot of the vehicle 100 To perceive. When the aircraft 200 is performing a flight mission, the aircraft 200 is in a state of overlooking the ground, and in order to observe the road environment, the sensor for collecting the second traffic information faces downward of the aircraft 200 .

In a possible implementation, when the aircraft 200 is fixed on the vehicle 100, in one case, the aircraft 200 is provided with a rotatable sensor, for example, the aircraft 200 includes a camera mounted on the gimbal, and the aircraft 200 can The orientation of the sensor needs to be adjusted so that the sensor can face the detection blind spot of the vehicle 100, and assist the vehicle 100 to perceive the surrounding environment in an all-round way; Orientation, the aircraft 200 can sense the perception needs of the vehicle 100 (such as detecting the orientation of the blind spot), and use the sensor facing a specific orientation to collect the first traffic information, assisting the vehicle 100 to perceive the surrounding environment in an all-round way.

Exemplarily, the body of the vehicle 100 is provided with a landing pad for the aircraft 200 , and the aircraft 200 is fixedly connected to the body of the vehicle 100 when it lands on the landing pad. In one example, the aircraft 200 is magnetically fixed to the vehicle 100 through the pluggable interface when it lands on the apron, and the aircraft 200 can be communicatively connected to the vehicle 100 through the pluggable interface, and the collected first traffic information is passed through the pluggable interface. The interface is transmitted to vehicle 100 . Optionally, the vehicle 100 can also charge the aircraft 200 through the pluggable interface.

In a possible implementation, when the aircraft 200 is fixed on the vehicle 100, the vehicle 100 can obtain the pose information of the aircraft 200 relative to the vehicle 100 when it is fixed on the vehicle 100; according to the pose information, the first traffic information collected by the aircraft 200 and The environment information collected by the on-board sensors of the vehicle 100 is fused, and the fused information is used as a reference for the environment perception information of the vehicle 100 .

In a possible implementation, when the aircraft 200 performs a flight mission, the vehicle 100 can obtain the pose information when the aircraft 200 collects the second traffic information, and determine the position of the target in the traffic environment based on the pose information and the second traffic information Information; use the position information of the target as a reference for navigation information. Exemplarily, the target objects include but are not limited to parking spaces, road signs, lane lines, or dynamic objects. Exemplarily, the vehicle 100 can plan the navigation route of the vehicle 100 according to the location information of the target. In a parking lot scenario, for example, if the target object is an empty parking space, the vehicle 100 may plan a navigation route to the empty parking space according to the location information of the empty parking space.

Further, in order to improve the planning accuracy of the navigation path, the vehicle 100 may obtain the pose information of the vehicle 100, and then plan the navigation path of the vehicle 100 according to the pose information of the vehicle 100 and the location information of the target. In this embodiment, the navigation route planning is performed based on the relative relationship between the pose information of the vehicle 100 and the position information of the target object, which is beneficial to improve the planning accuracy of the navigation route.

In a possible implementation, when the aircraft 200 performs a flight mission, after the vehicle 100 determines the position information of the target object in the traffic environment based on the pose information and the second traffic information, it can display The area displays a map overlaid with location markers of objects. Wherein, the map may be a map generated based on the second traffic information, or a map obtained by the vehicle 100 located in its own satellite positioning system, which is not limited in this embodiment. Alternatively, the vehicle 100 may also display a three-dimensional model superimposed with the position mark of the target in the display area according to the position information of the target, and the three-dimensional model may be generated by the vehicle 100 based on the pose information and the second traffic information The 3D model of the current traffic environment may also be the 3D model of the current traffic environment acquired through other platforms.

In a possible implementation, when the aircraft 200 performs a flight mission, after the vehicle 100 obtains the navigation route, it may also display a map with the navigation route superimposed in the display area, or may display the position of the target object superimposed in the display area A map that marks and navigates the route, making it easy for the user to know what to do next.

In an exemplary embodiment, please refer to FIG. 13A and FIG. 13B , which illustrate a system diagram of an aircraft 200 and a vehicle 100 . In FIG. 13A, the aircraft 200 includes a flight controller, a sensing system and a load system; wherein, the sensing system may include at least one visual sensor, an inertial measurement module, and a GNSS module. It can be understood that the sensing system in FIG. 13A All are examples, and may also include laser radar, millimeter wave radar or ultrasonic sensors, etc.; the load system may include a pan-tilt, at least one main camera and a wireless data transmission module arranged on the pan-tilt; the flight controller may include sensory module, and decision planning and control module.

Among them, the perception module is used to construct a local map based on the data collected by the sensor system. For example, the local map can be obtained in the following way: combine the motion data collected by the inertial measurement module to process the image (such as a grayscale image) collected by the visual sensor to obtain Depth map and semantic recognition results, and then obtain a local map based on the depth map and semantic recognition results. The decision planning and control module is used for planning the flight trajectory of the aircraft 200 and controlling the flight of the aircraft 200 .

The aircraft 200 can receive control instructions from the vehicle 100 through the data transmission module, and then make mission decisions according to the control instructions to generate specific instructions for different types of flight missions or specific instructions for acquisition tasks fixed on the vehicle 100, and then The gimbal and/or aircraft 200 are controlled based on characteristic commands. For example, for the control of the gimbal, the flight controller can control the motion of the gimbal according to specific instructions and the pose estimation data of the gimbal. For example, for the flight control of the aircraft 200, the flight controller can plan the flight trajectory based on specific instructions combined with the local map generated by the aircraft 200; Perform flight control based on the aircraft 200 attitude determined based on the data collected by each sensor in the sensing system. For example, in FIG. The positioning data collected by the positioning module is obtained after performing multi-sensor fusion positioning.

And the aircraft 200 can transmit the mission data collected during the flight mission to the vehicle 100 through the data transmission module, as shown in Figure 13A, the mission data includes but not limited to the aircraft 200 pose data, gimbal pose data or main camera acquisition image data, etc. It can be understood that the mission data fed back to the vehicle 100 in FIG. 13A is only a description of the distance, and the mission data corresponding to different types of flight missions are different; The pose data, the gimbal pose data, and the image data collected by the main camera can also include traffic environment information collected by other sensors, local maps obtained in the perception module, etc.; The mission data may include image data captured by the main camera.

In FIG. 13B , the vehicle 100 includes on-board sensors, an aircraft 200 data analysis module, an air perception module, a vehicle-side perception module, an air-ground fusion module, a planning control module and an HMI module (human-machine interface). The following uses the traffic information collection task as an example to illustrate:

The aircraft 200 data analysis module includes a wireless data transmission unit, which is used to receive the mission data sent back by the aircraft 200, which may include data such as the camera video data of the aircraft 200, the position and attitude of the aircraft 200; Analyze it, convert it into a data format suitable for algorithm processing, and output it to the air perception module for processing.

Vehicle-side sensors refer to the sensor hardware on the vehicle side, which may include binocular cameras, monocular cameras, IMU, GPS, lidar and other sensors.

The aerial perception module is used to realize beyond-the-horizon traffic scene perception based on the mission data sent back by the aircraft 200, and may include the following sensing functions: parking space detection, road sign detection, drivable area detection, traffic flow analysis, traffic scene recognition, etc. For different scenarios, different perception functions will be applied. For example, for parking scenarios, functions such as parking space detection and road sign detection will be activated; for congestion scenarios, functions such as traffic flow analysis and traffic scene recognition will be activated.

The vehicle-side perception module is used to realize ground perception functions based on the data collected by the vehicle-side sensors, which may include functions such as dynamic object detection, drivable area detection, lane line detection, road sign detection, or parking position estimation.

The air-ground fusion module is used to fuse the results output by the aerial perception module and the vehicle-side perception module. Through technologies such as relative pose estimation, coordinate system alignment, and static map fusion, the vehicle-side perception range is extended to hundreds of meters. Even to a distance of several kilometers.

The planning control module is used to plan the vehicle's driving trajectory according to the space-ground fusion perception result and the user's intention, and then realize the control of the vehicle 100 . The HMI module is used to realize over-the-horizon multi-scenario automatic driving visualization and human-computer interaction based on the results of air-ground fusion perception, which may include real-time image transmission display of aircraft 200, control of aircraft 200, automatic driving or automatic parking in car congestion, etc.

The various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features, but due to space limitations, they are not described one by one, so the various technical features in the above embodiments can be combined arbitrarily It also belongs to the scope disclosed in this specification.

Correspondingly, referring to FIG. 14 , an embodiment of the present application provides a data collection method, the method is applied to an aircraft, and the method includes:

In step S101, in response to a control command of the flight mission, mission data is collected during the execution of the flight mission.

In step S102, the task data is sent to the vehicle, the task data is used to generate display information, and the display information is used to display on the vehicle, wherein the vehicle includes a plurality of display areas, a plurality of The display areas are located at different positions of the vehicle, and the display information generated from mission data of different types of flight tasks is displayed on the display areas at different positions.

In this embodiment, the display information generated by the mission data of different types of flight missions is displayed in the display areas of different positions, which is convenient for users to intuitively view the display information from the display areas of different positions, thereby satisfying the needs of users for different types of missions. Viewing requirements for flight missions.

Optionally, the control instruction is sent by the vehicle to the aircraft.

Optionally, it also includes: taking off from a designated location of the vehicle to perform the flight mission, and landing to the designated location after completing the flight mission.

Optionally, the taking off from the designated position of the vehicle to perform the flight mission includes: taking off from the designated position of the vehicle to within a preset height range, and maintaining a preset altitude with the vehicle in a horizontal direction. Relative displacement within a horizontal distance range. Improve the flight safety of the aircraft and prevent the aircraft from being lost.

Optionally, said landing to the designated location after completing the flight mission includes: landing to the designated location while the vehicle is moving at a constant speed after completing the flight mission. Improve the landing safety of the aircraft.

Optionally, during the execution of the flight mission, the distance between the aircraft and the vehicle is kept within a preset distance range. Improve the flight safety of the aircraft.

Optionally, the aircraft is provided with one or more sensors; mission data of different types of flight missions are collected using different types of sensors.

Optionally, the sensor includes one or more of the following: at least one camera, one or more sensors for sensing the surrounding environment, or at least one audio collection component; the flight mission includes one or more of the following tasks: Aerial photography tasks, traffic information collection tasks or flight game tasks.

Wherein, the aerial photography task uses the at least one camera to collect aerial photography data; the traffic information collection task uses the at least one camera and the one or more sensors for sensing the surrounding environment to collect traffic information in the traffic environment; The flying game mission uses the real-time images collected by the at least one camera and/or the real-time audio collected by the at least one audio collection component.

Optionally, the flight trajectories of different types of flight missions are different.

Optionally, the flight tasks include one or more of the following tasks: aerial photography tasks, traffic information collection tasks, or flight game tasks. The flight trajectory of the aerial photography task is obtained according to any of the following methods: setting the flight trajectory according to the position of the vehicle, or planning the flight trajectory according to the selected shooting area. The flight trajectory of the traffic information collection task is obtained in any of the following ways: setting the flight trajectory according to the navigation information of the vehicle, or planning the flight trajectory according to the area selected in the navigation interface. The flight trajectory of the flying game mission is generated according to real-time control instructions, and the flight trajectory of the flying game mission is located within a preset space range.

Optionally, the flight mission includes a flight game mission. The method further includes: executing the flying game mission when the speed of the vehicle is lower than a preset value or the vehicle is in a non-driving state.

Optionally, it also includes: in response to the landing trigger, searching for the vehicle to be landed, and determining the relative pose relationship between the aircraft and the vehicle to be landed; according to the motion information of the aircraft and the relative position Estimate the predicted motion information of the vehicle to be landed based on attitude relationship, so as to plan the landing path of the aircraft; and control the aircraft to land on the vehicle according to the landing path. Realize the accurate landing of the aircraft to the moving vehicle.

Optionally, the searching for the vehicle to be landed includes: determining the movement trajectory of one or more moving objects on the ground; receiving the movement information of the vehicle sent by the vehicle, and obtaining A target movement trajectory: according to the difference between the target movement trajectory and the movement trajectory of the one or more moving objects, find the vehicle to be landed from the one or more moving objects.

Optionally, the vehicle is provided with landing visual signs. The searching for the vehicle to be landed includes: collecting an image on the ground side, and recognizing the landing visual sign from the image to determine the vehicle to be landed.

Optionally, the determining the relative pose relationship between the aircraft and the vehicle to be landed includes: tracking the vehicle to be landed to obtain the trajectory of the vehicle to be landed; The motion trajectory of the landing vehicle determines the relative pose relationship between the aircraft and the vehicle to be landing.

Optionally, the determining the relative pose relationship between the aircraft and the vehicle to be landed includes: determining the relative distance between the aircraft and the vehicle based on a near field communication connection with the vehicle ; Determine the relative pose relationship between the aircraft and the vehicle according to the motion information of the vehicle sent by the vehicle, the motion information of the aircraft, and the relative distance.

Optionally, the near-field communication connection includes a WIFI connection or a UWB connection; the motion information of the vehicle includes pose information and speed information of the vehicle; the motion information of the aircraft includes the pose information and speed information.

Optionally, a TOF sensor is installed on the roof of the vehicle, and the aircraft is provided with a material having a specific reflectivity for the waves emitted by the TOF sensor; the relative pose relationship between the aircraft and the vehicle is determined by The vehicle sends the data to the aircraft, wherein the vehicle determines the data whose reflectance is greater than or equal to the specific reflectance in the echo data collected by the TOF sensor as the target data corresponding to the aircraft, according to the The target data determines the relative pose relationship between the aircraft and the vehicle.

Optionally, the TOF sensor includes a laser radar, a millimeter wave radar, an infrared sensor or an ultrasonic sensor.

Optionally, a laser radar is installed on the roof of the vehicle, and the aircraft is provided with materials having a specific reflectivity for laser pulses. The relative pose relationship between the aircraft and the vehicle is sent by the vehicle to the aircraft, wherein, the vehicle collects the points in the point cloud collected by the lidar with a reflectivity greater than or equal to the specific reflectivity The three-dimensional point is determined as a target three-dimensional point corresponding to the aircraft, and the relative pose relationship between the aircraft and the vehicle is determined according to the depth information and angle information of the target three-dimensional point.

Optionally, the method further includes: when the aircraft is fixed on the vehicle, collecting first traffic information in a traffic environment in response to a control command of the vehicle.

Optionally, the mission data collected when the aircraft performs the flight mission includes second traffic information in the traffic environment. The orientation of the sensor used to collect the first traffic information in the aircraft is different from the orientation of the sensor used to collect the second traffic information.

Optionally, the sensor used for collecting the first traffic information is directed toward the front or side of the aircraft; the sensor used for collecting the second traffic information is directed downward of the aircraft.

Optionally, the body of the vehicle is provided with an apron, and the aircraft is fixedly connected to the body of the vehicle when it lands on the apron.

Please refer to FIG. 15 , an embodiment of the present application provides a data presentation method, the method is applied to a vehicle, and the vehicle includes multiple display areas located in different positions, and the method includes:

In step S201, the mission data collected during the execution of the flight mission by the aircraft is acquired.

In step S202, display information is generated according to the task data.

In step S203, the display area is controlled to display the display information, wherein the display information generated by mission data of different types of flight missions is displayed on the display area at different positions.

Optionally, further comprising: sending a control instruction of the flight mission to the aircraft, so that the aircraft responds to the control instruction of the flight mission, executes the flight mission and collects mission data corresponding to the flight mission .

Optionally, the vehicle includes at least one input device; the input device includes one or more external input devices, and/or one or more built-in input devices. The control instruction is generated by a user in the vehicle manipulating the built-in input device or the external input device. Users can choose convenient input devices to control the aircraft according to actual needs to meet the user's individual needs.

Optionally, the built-in input device includes any of the following: a built-in first camera, a voice collection component, a central control screen or a steering wheel control. The external input device includes any one of the following: a remote control terminal, a motion-sensing remote control, or a second camera communicatively connected to the vehicle through a pluggable interface.

Wherein, the control instruction is generated according to any of the following data collected by the input device: user gestures collected by the first camera and/or the second camera; voice signals collected by the voice collection component; The operation data received by one or more of the central control panel, the steering wheel control and the remote control terminal; or the somatosensory data collected by the somatosensory remote controller.

Optionally, the external input device can be directly communicatively connected with the aircraft; or, the external input device is communicatively connected with the aircraft through the vehicle.

Optionally, the method further includes: after receiving the instruction information sent by the aircraft to complete the flight mission, controlling the vehicle to move at a constant speed or sending a prompt message to the driver to keep the vehicle at a constant speed. Improve the landing safety of the aircraft.

Optionally, during the execution of the flight mission, the distance between the aircraft and the vehicle is kept within a preset distance range. The method further includes: if the distance between the aircraft and the vehicle exceeds the preset distance range, controlling the vehicle to decelerate. Improve the flight safety of the aircraft and prevent the aircraft from being lost.

Optionally, different types of mission data generate different types of display information.

Optionally, the flight tasks include one or more of the following tasks: aerial photography tasks, traffic information collection tasks, or flight game tasks. Wherein, the display information corresponding to the aerial photography task includes: aerial photography images generated according to the collected aerial photography data. The display information corresponding to the traffic information collection task includes one or more of the following: road condition perception information generated according to the collected traffic information in the traffic environment. The display information corresponding to the flying game task includes: a game screen generated according to the collected real-time screen combined with preset rendering rules, or a game screen with sound effects generated according to the collected real-time audio and video data combined with preset rendering rules.

Optionally, the road condition awareness information includes at least one or more of the following: position markers of objects, position markers of obstacles, navigation indication information pointing to the target object, or navigation indication information pointing to a destination.

Optionally, attribute values of specific attributes of the plurality of display regions are different; the specific attribute includes one or more of the following: brightness, resolution or frame rate.

Optionally, the plurality of display areas include a first display area and a second display area located at different positions; wherein, the attribute value of the specific attribute of the first display area is better than the attribute of the specific attribute of the second display area value. Display areas with different display performance can be selected according to actual needs to display display information of different types of flight missions.

Optionally, the plurality of display areas include a first display area and a second display area located at different positions. Wherein, the first display area includes a light-transmitting area for passing ambient light and a display area for displaying the display information; the second display area includes an opaque display area, and the opaque display area area is used to display the display information. There is at least a partial overlap between the light-transmitting area and the display area. Optionally, the first display area is located on the front window of the vehicle or in front of the driving seat.

Optionally, the multiple display areas include a first display area and a second display area located at different positions; The included angle is larger than the included angle between the direction of the line connecting the first display area and the headrest of the driver's seat and the forward direction of the vehicle. Display information of different types of flight missions can be displayed by selecting display areas with different degrees of convenience for viewing according to actual needs.

Optionally, the plurality of display areas include a first display area and a second display area located at different positions; the flight task includes an aerial photography task and/or a traffic information collection task; wherein, the traffic information collection task corresponds to The display information is displayed in the first display area; the display information corresponding to the aerial photography task is displayed in the second display area.

Optionally, the plurality of display areas include a first display area and a second display area located at different positions; the flight task includes an aerial photography task and/or a traffic information collection task. The method further includes: when the aircraft performs an aerial photography task, displaying display information corresponding to the aerial photography task in the first display area; and after the aircraft performs the traffic information collection task, displaying the traffic information The display information corresponding to the acquisition task is displayed in the first display area, and the display information corresponding to the aerial photography task is switched and displayed in the second display area.

Optionally, the flight task includes a traffic information collection task; the vehicle is used to generate display information according to the position information and type information of the target object extracted from the task data.

Optionally, the multiple display areas include a first display area and a second display area located at different positions; the vehicle is used to display the display information in the first display area and/or the second display area; wherein , the display position of the display information in the first display area is determined according to the relative positional relationship between the target and the vehicle in the traffic environment; the display position of the display information in the second display area is determined according to The position information of the target object and the display scale of the object indicated by the second display area are determined.

Optionally, the method further includes: switching and displaying the display information in another display area in response to a display switching instruction.

Optionally, the vehicle is provided with a camera. The method further includes: acquiring a user image captured by the camera, and determining whether to generate the display switching instruction according to a user gesture recognized from the user image.

Optionally, the camera includes a built-in first camera and/or a second camera communicatively connected to the vehicle through a pluggable interface.

Optionally, the vehicle includes a second camera communicatively connected to the vehicle through a pluggable interface. The method further includes: displaying the image or video obtained by the second camera performing a shooting task in any one of the display areas.

Optionally, the method further includes: sending the image or video obtained by the second camera and the mission data of the flight mission to a remote device.

Optionally, the image or video obtained by the second camera performing the shooting mission and the display information generated by the mission data of the flight mission are superimposed and displayed in the remote device.

Optionally, the vehicle includes a second camera communicatively connected to the vehicle through a pluggable interface. The method further includes: after detecting that the vehicle body is in communication with the second camera, acquiring a user image captured by the second camera, and using the user image to perform different functions in different working modes.

Optionally, the working mode includes a first working mode and a second working mode; the first working mode indicates identifying the user image to obtain user gestures; the second working mode indicates using the user image to monitor the driver driving status.

Optionally, the vehicle includes a switch control. The method further includes: switching the current working mode from the first working mode to the second working mode, or switching the current working mode from the second working mode to the first working mode in response to the switching trigger of the switching control .

Optionally, the flight task includes a traffic information collection task. The method further includes: when the aircraft is fixed on the vehicle, fusing the first traffic information in the traffic environment collected by the aircraft with the environmental information collected by the on-board sensor of the vehicle, as the vehicle's environmental awareness Information reference; when the aircraft performs a traffic information collection task, the second traffic information in the traffic environment collected by the aircraft and the pose information when the aircraft collects the second traffic environment information are used as the vehicle navigation information reference.

Optionally, the fusing the first traffic information in the traffic environment collected by the aircraft with the environment information collected by the on-board sensor of the vehicle includes: The pose information of the vehicle; according to the pose information, the first traffic information collected by the aircraft and the environment information collected by the on-board sensor of the vehicle are fused.

Optionally, using the second traffic information in the traffic environment collected by the aircraft and the pose information when the aircraft collects the second traffic environment information as the navigation information reference of the vehicle includes: obtaining The pose information when the aircraft collects the second traffic information, and determine the position information of the target object in the traffic environment based on the pose information and the second traffic information; use the position information of the target object as the Refer to the above navigation information.

Optionally, referring to the location information of the target as the navigation information includes: planning a navigation route of the vehicle according to the location information of the target.

Optionally, planning the navigation route of the vehicle according to the position information of the target includes: acquiring pose information of the vehicle, and planning according to the pose information of the vehicle and the position information of the target The vehicle's navigation path.

Optionally, the method further includes: displaying a map superimposed with a position mark of the target in the display area according to the position information of the target.

Optionally, the method further includes: displaying a map superimposed with the navigation route in the display area.

In some embodiments, the embodiment of the present application also provides a data display system, including a vehicle and an aircraft, and the vehicle is communicatively connected to the aircraft;

For the relevant parts of the data display system, please refer to the above description.

In some embodiments, referring to FIG. 16 , there is also provided a data processing method applied to a vehicle, including:

In step S301, when the aircraft is fixed to the vehicle, the first traffic information collected by the aircraft and the environmental information collected by the on-board sensor of the vehicle are obtained, and the first traffic information and the environmental information collected by the on-board sensor are fused together as The environment perception information of the vehicle is referenced.

In step S302, when the aircraft performs a flight mission, the second traffic information collected by the aircraft and the pose information of the aircraft when collecting the second traffic information are obtained, and the The second traffic information is referred to as the navigation information of the vehicle.

For the relevant parts of the data processing method, please refer to the description in the above data display system.

In some embodiments, referring to FIG. 17 , there is also provided a data processing method applied to a camera, the method comprising:

In step S401, it is detected whether the camera is connected to the vehicle during the execution of the shooting task.

In step S402, if the camera is connected to the vehicle, the image collected by performing the shooting task is sent to the vehicle; the image is used to trigger the vehicle to use the image to recognize user gestures or detect the driver State; the user gesture is at least used to control an aircraft communicatively connected with the vehicle to perform a flight mission.

In step S403, if the camera is not connected to the vehicle, the image collected by executing the shooting task is stored.

In some embodiments, please refer to FIG. 18 , a method for landing an aircraft is also provided, including:

In step S501, in response to a landing trigger, a vehicle to be landed is searched for, and a relative pose relationship between the aircraft and the vehicle to be landed is determined.

In step S502, predictive motion information of the vehicle to be landed is estimated according to the motion information of the aircraft and the relative pose relationship, so as to plan a landing path of the aircraft.

In step S503, the aircraft is controlled to land on the vehicle according to the landing path.

In some embodiments, an aircraft is also provided, comprising:

body;

One or more processors are arranged in the fuselage and are used to execute the above method.

In some embodiments, a vehicle is also provided, comprising:

body;

In some embodiments, a camera is also provided, comprising:

a lens assembly for transmitting light;

memory for storing images; and,

A processor is configured to execute the above method.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, which are executable by a processor of an apparatus to perform the above method. For example, the non-transitory computer readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

A non-transitory computer-readable storage medium, enabling the terminal to execute the above method when instructions in the storage medium are executed by a processor of the terminal.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. There is no such actual relationship or order between them. The term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements but also other elements not expressly listed elements, or also elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The methods and devices provided by the embodiments of the present application have been described in detail above. The principles and implementation methods of the present application have been explained by using specific examples in this paper. The descriptions of the above embodiments are only used to help understand the methods and methods of the present application. core idea; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation and application scope. In summary, the content of this specification should not be construed as limiting the application .

Claims

A data display system, characterized in that it includes a vehicle and an aircraft, the vehicle is connected in communication with the aircraft; the vehicle includes a plurality of display areas located in different positions;

The aircraft is used to collect mission data corresponding to the flight mission during the execution of the flight mission, and send the mission data back to the vehicle;

The vehicle is used to generate display information according to the mission data, and control the display area to display the display information, wherein the display information generated by mission data of different types of flight missions is displayed in the display area at different positions.
The system according to claim 1, wherein the vehicle is used to send control instructions of the flight mission to the aircraft;

The aircraft is used for executing the flight mission and collecting mission data corresponding to the flight mission in response to the control instruction of the flight mission.
The system according to claim 2, wherein the vehicle includes at least one input device; the input device includes one or more external input devices, and/or one or more built-in input devices;

The control instruction is generated by a user in the vehicle manipulating the built-in input device or the external input device.
The system according to claim 3, wherein the built-in input device comprises any of the following: a built-in first camera, a voice collection component, a central control panel or a steering wheel control;

The external input device includes any of the following: a remote control terminal, a motion-sensing remote control, or a second camera communicatively connected to the vehicle through a pluggable interface;

Wherein, the control instruction is generated according to any of the following data collected by the input device: user gestures collected by the first camera and/or the second camera; voice signals collected by the voice collection component; The operation data received by one or more of the central control panel, the steering wheel control and the remote control terminal; or the somatosensory data collected by the somatosensory remote controller.
The system according to claim 3, wherein the external input device is directly communicatively connected with the aircraft; or, the external input device is communicatively connected with the aircraft through the vehicle.
The system according to claim 1, wherein the aircraft is used to take off from a designated location of the vehicle to perform the flight mission, and land to the designated location after completing the flight mission.
The system according to claim 6, wherein the aircraft is used to take off from a designated position of the vehicle to within a preset altitude range, and maintain a horizontal distance from the vehicle within a preset horizontal distance range Relative displacement.
The system according to claim 6, wherein the aircraft is used to land to the designated position when the vehicle is moving at a constant speed after completing the flight mission.
The system according to claim 6, wherein the vehicle is configured to control the vehicle to move at a constant speed or send prompt information to the driver to keep the vehicle at a constant speed after receiving the instruction information sent by the aircraft to complete the flight mission.
The system according to claim 1, characterized in that, during the execution of the flight mission, the distance between the aircraft and the vehicle is kept within a preset distance range;

The vehicle is used to control the vehicle to decelerate if the distance between the aircraft and the vehicle exceeds the preset distance range.
The system according to claim 1, wherein the aircraft is provided with one or more sensors;

Mission data for different types of missions are collected using different types of sensors.
The system according to claim 11, wherein the sensor comprises one or more of the following: at least one camera, one or more sensors for sensing the surrounding environment, or at least one audio collection component;

Described flying task comprises following one or more tasks: aerial photography task, traffic information collection task or flight game task;

Wherein, the aerial photography task uses the at least one camera to collect aerial photography data;

The traffic information collection task uses the at least one camera and the one or more sensors for sensing the surrounding environment to collect traffic information in the traffic environment;

The flying game mission uses the real-time images collected by the at least one camera and/or the real-time audio collected by the at least one audio collection component.
The system according to claim 1, wherein the flight trajectories of different types of missions are different.
The system according to claim 13, wherein the flight tasks include one or more of the following tasks: aerial photography tasks, traffic information collection tasks or flight game tasks;

The flight trajectory of the aerial photography task is obtained according to any of the following methods: setting the flight trajectory according to the position of the vehicle, or planning the flight trajectory according to the selected shooting area;

The flight trajectory of the traffic information collection task is obtained in any of the following ways: setting the flight trajectory according to the navigation information of the vehicle, or planning the flight trajectory according to the selected area in the navigation interface;

The flight trajectory of the flying game mission is generated according to real-time control instructions, and the flight trajectory of the flying game mission is located within a preset space range.
The system of claim 1, wherein different types of mission data generate different types of display information for different types of missions.
The system according to claim 15, wherein the flight tasks include one or more of the following tasks: aerial photography tasks, traffic information collection tasks or flight game tasks;

Wherein, the display information corresponding to the aerial photography task includes: aerial photography images generated according to the collected aerial photography data;

The display information corresponding to the traffic information collection task includes one or more of the following: road condition perception information generated according to the collected traffic information in the traffic environment;

The display information corresponding to the flying game task includes: a game screen generated according to the collected real-time screen combined with preset rendering rules, or a game screen with sound effects generated according to the collected real-time audio and video data combined with preset rendering rules.
The system according to claim 16, wherein the road condition awareness information includes at least one or more of the following: position markers of targets, position markers of obstacles, navigation indication information pointing to the targets, or Navigation instructions to a destination.
The system according to claim 1, wherein the flight mission comprises a flight game mission;

The aircraft is used to execute the flying game task when the speed of the vehicle is lower than a preset value or the vehicle is in a non-driving state.
The system according to claim 1, wherein the attribute values of the specific attributes of the plurality of display areas are different; the specific attributes include one or more of the following: brightness, resolution or frame rate.
The system according to claim 19, wherein the plurality of display areas comprises a first display area and a second display area located at different positions;

Wherein, the attribute value of the specific attribute of the first display area is better than the attribute value of the specific attribute of the second display area.
The system according to claim 1, wherein the plurality of display areas comprise a first display area and a second display area located at different positions;

Wherein, the first display area includes a light-transmitting area for passing ambient light and a display area for displaying the display information;

The second display area includes an opaque display area for displaying the display information.
The system according to claim 21, wherein the light transmission area and the display area overlap at least partially.
The system according to claim 21, wherein the first display area is located on a front window of the vehicle or in front of a driver's seat.
The system according to claim 1, wherein the plurality of display areas comprise a first display area and a second display area located at different positions;

The included angle between the direction of the line connecting the second display area and the headrest of the driver's seat and the forward direction of the vehicle is larger than the direction of the line connecting the first display area and the headrest of the driver's seat and the direction of the vehicle. The angle between the heading directions.
The system according to any one of claims 19 to 24, wherein the plurality of display areas include a first display area and a second display area located at different positions;

The flight tasks include aerial photography tasks and/or traffic information collection tasks;

Wherein, the display information corresponding to the traffic information collection task is displayed in the first display area;

The display information corresponding to the aerial photography task is displayed in the second display area.
The system according to claim 1, wherein the plurality of display areas comprise a first display area and a second display area located at different positions; the flight tasks include aerial photography tasks and/or traffic information collection tasks;

The vehicle is used to display the display information corresponding to the aerial photography task in the first display area when the aircraft performs the aerial photography task; and collect the traffic information after the aircraft performs the traffic information collection task The display information corresponding to the task is displayed in the first display area, and the display information corresponding to the aerial photography task is switched and displayed in the second display area.
The system according to claim 1, wherein the flight task comprises a traffic information collection task;

The vehicle is used for generating display information according to the position information and type information of the target object extracted from the mission data.
The system according to claim 27, wherein the plurality of display areas comprises a first display area and a second display area located at different positions;

The vehicle is used to display the display information in the first display area and/or the second display area;

Wherein, the display position of the display information in the first display area is determined according to the relative positional relationship between the target object and the vehicle in the traffic environment;

The display position of the display information in the second display area is determined according to the position information of the target object and the display ratio of the object indicated by the second display area.
The system according to claim 1, wherein the vehicle is configured to switch and display the display information in other display areas in response to a display switching instruction.
The system of claim 29, wherein the vehicle is provided with a camera;

The vehicle is configured to acquire the user image captured by the camera, and determine whether to generate the display switching instruction according to the user gesture recognized from the user image.
The system according to claim 30, wherein the camera comprises a built-in first camera and/or a second camera communicatively connected to the vehicle through a pluggable interface.
The system of claim 1, wherein the vehicle includes a second camera communicatively connected to the vehicle via a pluggable interface;

The vehicle is further configured to display the image or video obtained by the second camera in any of the display areas.
The system according to claim 32, wherein the vehicle is further configured to send the image or video obtained by the second camera during the shooting mission and mission data of the flight mission to a remote device.
The system according to claim 33, characterized in that the image or video obtained by the second camera performing the shooting mission and the display information generated by the mission data of the flight mission are superimposed and displayed in the remote device.
The system of claim 1, wherein the vehicle includes a second camera communicatively connected to the vehicle via a pluggable interface;

The vehicle is configured to acquire a user image captured by the second camera after detecting that the vehicle body is in communication with the second camera, and use the user image to perform different functions in different working modes.
The system according to claim 35, wherein the working modes include a first working mode and a second working mode;

The first working mode indicates identifying the user image to obtain user gestures;

The second working mode indicates using the user image to monitor the driving state of the driver.
The system of claim 36, wherein the vehicle includes a toggle control;

The switching control is used for switching the current working mode from the first working mode to the second working mode, or switching the current working mode from the second working mode to the first working mode in response to the switching trigger.
The system according to claim 1, wherein the aircraft is used for responding to a landing trigger, searching for a vehicle to be landed, and determining the relative pose relationship between the aircraft and the vehicle to be landed; according to Estimating predicted motion information of the vehicle to be landed from the motion information of the aircraft and the relative pose relationship to plan a landing path of the aircraft; controlling the aircraft to land on the vehicle according to the landing path.
The system according to claim 38, wherein the aircraft is used to determine the trajectory of one or more moving objects on the ground; receiving the vehicle's movement information sent by the vehicle, and according to the vehicle Obtaining a target trajectory from the motion information; and searching for a vehicle to be landed from the one or more moving objects according to the difference between the target trajectory and the trajectory of the one or more moving objects.
The system of claim 38, wherein the vehicle is provided with a landing visual marker;

The aircraft is used to collect images on the ground side, and identify the landing visual signs from the images to determine the vehicle to be landed.
The system according to claim 39 or 40, wherein the aircraft is used to track the vehicle to be landed to obtain the trajectory of the vehicle to be landed, and determine the The relative pose relationship between the aircraft and the vehicle to be landed.
The system according to claim 38, wherein the aircraft is configured to determine the relative distance between the aircraft and the vehicle based on a near field communication connection with the vehicle; The motion information of the vehicle, the motion information of the aircraft and the relative distance determine the relative pose relationship between the aircraft and the vehicle.
The system according to claim 42, wherein the near field communication connection comprises a WIFI connection or a UWB connection;

The motion information of the vehicle includes pose information and speed information of the vehicle;

The motion information of the aircraft includes pose information and speed information of the aircraft.
The system according to claim 38, wherein a TOF sensor is installed on the roof of the vehicle, and the aircraft is provided with a material having a specific reflectivity for waves emitted by the TOF sensor;

The vehicle is used to determine the data whose reflectivity is greater than or equal to the specific reflectivity in the echo data collected by the TOF sensor as the target data corresponding to the aircraft, and determine the relationship between the aircraft and the aircraft according to the target data. The relative pose relationship of the vehicle is fed back to the aircraft.
The system according to claim 44, wherein the TOF sensor comprises a laser radar, a millimeter wave radar, an infrared sensor or an ultrasonic sensor.
The system according to claim 38, wherein a lidar is installed on the roof of the vehicle, and the aircraft is provided with a material having a specific reflectivity to laser pulses;

The vehicle is also used to determine a three-dimensional point whose reflectivity is greater than or equal to the specific reflectivity in the point cloud collected by the lidar as the target three-dimensional point corresponding to the aircraft, according to the depth information of the target three-dimensional point and The angle information determines the relative pose relationship between the aircraft and the vehicle and is fed back to the aircraft.
The system according to claim 1, wherein the flight task comprises a traffic information collection task;

When the aircraft is fixed on the vehicle, the vehicle is used to fuse the first traffic information in the traffic environment collected by the aircraft with the environmental information collected by the vehicle's on-board sensor, as the environment perception information of the vehicle refer to;

When the aircraft performs a traffic information collection task, the vehicle is used to use the second traffic information in the traffic environment collected by the aircraft and the pose information when the aircraft collects the second traffic environment information as the Refer to the navigation information for the vehicle described above.
The system according to claim 47, wherein the orientation of the sensor used to collect the first traffic information in the aircraft is different from the orientation of the sensor used to collect the second traffic information.
The system according to claim 48, wherein the sensor for collecting the first traffic information faces the front or side of the aircraft;

The sensor for collecting the second traffic information faces downward of the aircraft.
The system according to claim 47, wherein the first traffic information includes first distances between the vehicles and multiple objects in the traffic environment; the second traffic information includes distances between the vehicles and traffic objects respectively. a second distance of the plurality of objects in the environment;

Wherein, the maximum among the multiple first distances is smaller than the maximum among the multiple second distances.
The system according to claim 47, wherein the body of the vehicle is provided with an apron, and the aircraft is fixedly connected to the body of the vehicle when it lands on the apron.
The system according to claim 47, wherein when the aircraft is fixed on the vehicle, the vehicle is used to obtain the pose information of the aircraft relative to the vehicle when it is fixed on the vehicle; according to the The pose information is fused with the first traffic information collected by the aircraft and the environment information collected by the on-board sensor of the vehicle.
The system according to claim 47, wherein when the aircraft performs a traffic information collection task, the vehicle is used to obtain the pose information of the aircraft when collecting the second traffic information, based on the pose The information and the second traffic information determine the position information of the target object in the traffic environment; the position information of the target object is used as the reference of the navigation information.
The system according to claim 53, wherein the vehicle is used to plan a navigation route of the vehicle according to the location information of the target.
The system according to claim 54, wherein the vehicle is used to obtain the pose information of the vehicle, and plan the navigation path of the vehicle according to the pose information of the vehicle and the position information of the target .
The system according to claim 53, wherein the vehicle is further configured to display a map superimposed with a location mark of the object in the display area according to the location information of the object.
The system according to claim 54 or 55, wherein the vehicle is further configured to display a map superimposed with the navigation route in the display area.
A data acquisition method, characterized in that the method is applied to an aircraft, and the method comprises:

Responding to the control instructions of the flight mission, collecting mission data during the execution of the flight mission;

sending the task data to a vehicle, the task data is used to generate display information, and the display information is used to display on the vehicle, wherein the vehicle includes a plurality of display areas, and the plurality of display areas are located at For different positions of the vehicle, display information generated from mission data of different types of flight missions is displayed on the display area at different positions.
The method of claim 58, wherein the control commands are sent from the vehicle to the aircraft.
The method of claim 58, further comprising:

The vehicle takes off from a designated location to perform the flight mission, and lands at the designated location after completing the flight mission.
The method according to claim 60, wherein said taking off from a designated location of said vehicle to perform said flight mission comprises:

Take off from a designated position of the vehicle to within a preset height range, and maintain a relative displacement with the vehicle within a preset horizontal distance range in a horizontal direction.
The method according to claim 60, wherein said landing to said designated location after completing said flight task comprises:

After completing the flight mission, the vehicle lands at the specified position while the vehicle is moving at a constant speed.
The method according to claim 58, characterized in that, during the execution of the flight mission, the distance between the aircraft and the vehicle is kept within a preset distance range.
The method of claim 58, wherein the aircraft is provided with one or more sensors;

Mission data for different types of missions are collected using different types of sensors.
The method according to claim 64, wherein the sensor comprises one or more of the following: at least one camera, one or more sensors for sensing the surrounding environment, or at least one audio collection component;

The flight tasks include one or more of the following tasks: aerial photography tasks, traffic information collection tasks or flight game tasks;

Wherein, the aerial photography task uses the at least one camera to collect aerial photography data;

The traffic information collection task uses the at least one camera and the one or more sensors for sensing the surrounding environment to collect traffic information in the traffic environment;

The flying game mission uses the real-time images collected by the at least one camera and/or the real-time audio collected by the at least one audio collection component.
The method according to claim 58, wherein the flight trajectories of different types of missions are different.
The method according to claim 66, wherein the flight missions include one or more of the following missions: aerial photography missions, traffic information collection missions or flight game missions;

The flight trajectory of the aerial photography task is obtained according to any of the following methods: setting the flight trajectory according to the position of the vehicle, or planning the flight trajectory according to the selected shooting area;

The flight trajectory of the traffic information collection task is obtained in any of the following ways: setting the flight trajectory according to the navigation information of the vehicle, or planning the flight trajectory according to the selected area in the navigation interface;

The flight trajectory of the flying game mission is generated according to real-time control instructions, and the flight trajectory of the flying game mission is located within a preset space range.
The method according to claim 58, wherein the flight mission comprises a flight game mission;

The method also includes:

When the speed of the vehicle is lower than a preset value or the vehicle is in a non-driving state, the flying game task is executed.
The method of claim 58, further comprising:

In response to a landing trigger, searching for a vehicle to be landed, and determining a relative pose relationship between the aircraft and the vehicle to be landed;

estimating predicted motion information of the vehicle to be landed according to the motion information of the aircraft and the relative pose relationship, so as to plan the landing path of the aircraft;

The aircraft is controlled to land to the vehicle according to the landing path.
The method according to claim 69, wherein said searching for the vehicle to be landed comprises:

Determine the trajectory of one or more moving objects on the ground;

receiving motion information of the vehicle sent by the vehicle, and acquiring a target motion trajectory according to the motion information of the vehicle;

According to the difference between the target movement trajectory and the movement trajectory of the one or more moving objects, the vehicle to be landed is searched from the one or more moving objects.
The method of claim 69, wherein the vehicle is provided with landing visual signs;

Said finding the vehicle to be landed includes:

An image of the ground side is collected, and the landing visual sign is recognized from the image to determine the vehicle to be landed.
The method according to claim 70 or 71, wherein the determining the relative pose relationship between the aircraft and the vehicle to be landed comprises:

tracking the vehicle to be landed to obtain a trajectory of the vehicle to be landed;

The relative pose relationship between the aircraft and the vehicle to be landed is determined according to the trajectory of the vehicle to be landed.
The method according to claim 69, wherein the determining the relative pose relationship between the aircraft and the vehicle to be landed comprises:

determining a relative distance between the aircraft and the vehicle based on a near field communication connection with the vehicle;

A relative pose relationship between the aircraft and the vehicle is determined according to the motion information of the vehicle, the motion information of the aircraft, and the relative distance sent by the vehicle.
The method according to claim 73, wherein the near field communication connection comprises a WIFI connection or a UWB connection;

The motion information of the vehicle includes pose information and speed information of the vehicle;

The motion information of the aircraft includes pose information and speed information of the aircraft.
The method according to claim 69, wherein a TOF sensor is installed on the roof of the vehicle, and the aircraft is provided with a material having a specific reflectivity for waves emitted by the TOF sensor;

The relative pose relationship between the aircraft and the vehicle is sent by the vehicle to the aircraft, wherein the vehicle uses the reflectivity in the echo data collected by the TOF sensor to be greater than or equal to the specific reflectivity The data of the aircraft is determined as the target data corresponding to the aircraft, and the relative pose relationship between the aircraft and the vehicle is determined according to the target data.
The method according to claim 75, wherein the TOF sensor comprises a laser radar, a millimeter wave radar, an infrared sensor or an ultrasonic sensor.
The method according to claim 69, wherein a lidar is installed on the roof of the vehicle, and the aircraft is provided with a material having a specific reflectivity to laser pulses;

The relative pose relationship between the aircraft and the vehicle is sent by the vehicle to the aircraft, wherein, the vehicle collects the points in the point cloud collected by the lidar with a reflectivity greater than or equal to the specific reflectivity The three-dimensional point is determined as a target three-dimensional point corresponding to the aircraft, and the relative pose relationship between the aircraft and the vehicle is determined according to the depth information and angle information of the target three-dimensional point.
The method of claim 58, further comprising:

When the aircraft is fixed on the vehicle, first traffic information in the traffic environment is collected in response to a control command of the vehicle.
The method according to claim 78, characterized in that the mission data collected when the aircraft executes the flight mission includes the second traffic information in the traffic environment;

The orientation of the sensor used to collect the first traffic information in the aircraft is different from the orientation of the sensor used to collect the second traffic information.
The method according to claim 79, wherein the sensor for collecting the first traffic information faces the front or side of the aircraft;

The sensor for collecting the second traffic information faces downward of the aircraft.
The method according to claim 78, characterized in that, the body of the vehicle is provided with an apron, and the aircraft is fixedly connected to the body of the vehicle when it lands on the apron.
A data display method, characterized in that the method is applied to a vehicle, and the vehicle includes a plurality of display areas located at different positions, and the method includes:

Obtain the mission data collected during the flight mission of the aircraft;

generating display information according to the task data;

The display area is controlled to display the display information, wherein the display information generated by mission data of different types of flight missions is displayed on the display area at different positions.
The method of claim 82, further comprising:

Sending the control instruction of the flight mission to the aircraft, so that the aircraft responds to the control instruction of the flight mission, executes the flight mission and collects mission data corresponding to the flight mission.
The method according to claim 83, wherein the vehicle includes at least one input device; the input device includes one or more external input devices, and/or one or more built-in input devices;

The control instruction is generated by a user in the vehicle manipulating the built-in input device or the external input device.
The method according to claim 84, wherein the built-in input device includes any of the following: a built-in first camera, a voice collection component, a central control panel or a steering wheel control;

The external input device includes any of the following: a remote control terminal, a motion-sensing remote control, or a second camera communicatively connected to the vehicle through a pluggable interface;

Wherein, the control instruction is generated according to any of the following data collected by the input device: user gestures collected by the first camera and/or the second camera; voice signals collected by the voice collection component; The operation data received by one or more of the central control panel, the steering wheel control and the remote control terminal; or the somatosensory data collected by the somatosensory remote controller.
The method according to claim 84, characterized in that, the external input device is directly communicatively connected with the aircraft; or, the external input device is communicatively connected with the aircraft through the vehicle.
The method of claim 82, further comprising:

After receiving the instruction information sent by the aircraft to complete the flight task, the vehicle is controlled to move at a constant speed or the driver is sent prompt information to keep the vehicle at a constant speed.
The method according to claim 82, characterized in that, during the execution of the flight mission, the distance between the aircraft and the vehicle is kept within a preset distance range;

The method also includes:

If the distance between the aircraft and the vehicle exceeds the preset distance range, the vehicle is controlled to decelerate.
The method of claim 82, wherein different types of mission data generate different types of display information.
The method according to claim 89, wherein the flight tasks include one or more of the following tasks: aerial photography tasks, traffic information collection tasks or flight game tasks;

Wherein, the display information corresponding to the aerial photography task includes: aerial photography images generated according to the collected aerial photography data;

The display information corresponding to the traffic information collection task includes one or more of the following: road condition perception information generated according to the collected traffic information in the traffic environment;

The display information corresponding to the flying game task includes: a game screen generated according to the collected real-time screen combined with preset rendering rules, or a game screen with sound effects generated according to the collected real-time audio and video data combined with preset rendering rules.
The method according to claim 90, wherein the road condition awareness information includes at least one or more of the following: position markers of targets, position markers of obstacles, navigation indication information pointing to the targets, or Navigation instructions to a destination.
The method according to claim 82, wherein the attribute values of the specific attributes of the plurality of display areas are different; the specific attribute includes one or more of the following: brightness, resolution or frame rate.
The method according to claim 92, wherein the plurality of display areas comprise a first display area and a second display area located at different positions;

Wherein, the attribute value of the specific attribute of the first display area is better than the attribute value of the specific attribute of the second display area.
The method according to claim 82, wherein the plurality of display areas comprise a first display area and a second display area located at different positions;

Wherein, the first display area includes a light-transmitting area for passing through ambient light and a display area for displaying the display information;

The second display area includes an opaque display area for displaying the display information.
The method according to claim 94, wherein there is at least a partial overlap between the light-transmitting area and the display area.
The method according to claim 94, wherein the first display area is located on a front window of the vehicle or in front of a driver's seat.
The method according to claim 82, wherein the plurality of display areas comprise a first display area and a second display area located at different positions;

The included angle between the direction of the line connecting the second display area and the headrest of the driver's seat and the forward direction of the vehicle is larger than the direction of the line connecting the first display area and the headrest of the driver's seat and the direction of the vehicle. The angle between the heading directions.
The method according to any one of claims 92 to 97, wherein the plurality of display areas include a first display area and a second display area located at different positions;

The flight tasks include aerial photography tasks and/or traffic information collection tasks;

Wherein, the display information corresponding to the traffic information collection task is displayed in the first display area;

The display information corresponding to the aerial photography task is displayed in the second display area.
The method according to claim 82, wherein the plurality of display areas include a first display area and a second display area located at different positions; the flight task includes an aerial photography task and/or a traffic information collection task;

The method also includes:

When the aircraft performs an aerial photography task, display the display information corresponding to the aerial photography task in the first display area; and after the aircraft performs the traffic information collection task, display the display information corresponding to the traffic information collection task display in the first display area, and switch and display the display information corresponding to the aerial photography task in the second display area.
The method according to claim 82, wherein the flight task comprises a traffic information collection task;

The vehicle is used for generating display information according to the position information and type information of the target object extracted from the task data.
The method according to claim 100, wherein the plurality of display areas comprise a first display area and a second display area located at different positions;

The vehicle is used to display the display information in the first display area and/or the second display area;

Wherein, the display position of the display information in the first display area is determined according to the relative positional relationship between the target object and the vehicle in the traffic environment;

The display position of the display information in the second display area is determined according to the position information of the target object and the display ratio of the object indicated by the second display area.
The method of claim 82, further comprising:

In response to the display switching instruction, the display information is switched and displayed in other display areas.
The method of claim 102, wherein the vehicle is provided with a camera;

The method also includes:

Acquiring a user image captured by the camera, and determining whether to generate the display switching instruction according to a user gesture recognized from the user image.
The method according to claim 103, wherein the camera comprises a built-in first camera and/or a second camera communicatively connected to the vehicle through a pluggable interface.
The method of claim 82, wherein the vehicle includes a second camera communicatively coupled to the vehicle via a pluggable interface;

The method also includes:

The image or video obtained by the second camera performing the shooting task is displayed in any one of the display areas.
The method according to claim 105, further comprising:

Sending the image or video obtained by the second camera during the shooting mission and the mission data of the flight mission to the remote device.
The method according to claim 106, characterized in that the image or video obtained by the second camera performing the shooting mission and the display information generated by the mission data of the flight mission are superimposed and displayed in the remote device.
The method of claim 82, wherein the vehicle includes a second camera communicatively coupled to the vehicle via a pluggable interface;

The method also includes:

After detecting that the vehicle body is connected in communication with the second camera, the user image captured by the second camera is acquired, and the user image is used to perform different functions in different working modes.
The method according to claim 108, wherein the working mode comprises a first working mode and a second working mode;

The first working mode indicates identifying the user image to obtain user gestures;

The second working mode indicates using the user image to monitor the driving state of the driver.
The method of claim 109, wherein the vehicle includes a toggle control;

The method also includes:

In response to the switching trigger of the switching control, the current working mode is switched from the first working mode to the second working mode, or the current working mode is switched from the second working mode to the first working mode.
The method according to claim 82, wherein the flight task comprises a traffic information collection task;

The method also includes:

When the aircraft is fixed on the vehicle, the first traffic information in the traffic environment collected by the aircraft is fused with the environment information collected by the on-board sensor of the vehicle, and used as a reference for the vehicle's environment perception information;

When the aircraft executes the traffic information collection task, the second traffic information in the traffic environment collected by the aircraft and the pose information when the aircraft collects the second traffic environment information are used as the navigation information of the vehicle refer to.
The method according to claim 111, wherein the fusing the first traffic information in the traffic environment collected by the aircraft with the environmental information collected by the on-board sensor of the vehicle comprises:

Obtaining the pose information of the aircraft relative to the vehicle when it is fixed on the vehicle;

The first traffic information collected by the aircraft is fused with the environment information collected by the on-board sensor of the vehicle according to the pose information.
The method according to claim 111, wherein the second traffic information in the traffic environment collected by the aircraft and the pose information when the aircraft collects the second traffic environment information are used as the Vehicle navigation information reference, including:

Obtaining pose information when the aircraft collects the second traffic information, and determining position information of objects in the traffic environment based on the pose information and the second traffic information;

The position information of the target is used as the reference of the navigation information.
The method according to claim 113, wherein said referring to the position information of the target as the navigation information comprises:

A navigation route of the vehicle is planned according to the position information of the target object.
The method according to claim 114, wherein planning the navigation route of the vehicle according to the position information of the target comprises:

The pose information of the vehicle is acquired, and the navigation path of the vehicle is planned according to the pose information of the vehicle and the position information of the target.
The method according to claim 113, further comprising:

A map superimposed with a position mark of the target is displayed in the display area according to the position information of the target.
The method according to claim 114 or 115, further comprising:

A map with the navigation route superimposed is displayed in the display area.
A data display system, characterized in that it includes a vehicle and an aircraft, and the vehicle is connected to the aircraft in communication;

The aircraft is used to collect mission data corresponding to the flight mission during the execution of the flight mission, and send the mission data back to the vehicle;

The vehicle is used for generating display information according to the mission data, and displaying the display information in a display area, wherein mission data of different types of flight missions generate different types of display information.
A data processing method, characterized in that it is applied to a vehicle, comprising:

When the aircraft is fixed to the vehicle, the first traffic information collected by the aircraft and the environmental information collected by the vehicle's on-board sensor are obtained, and the first traffic information and the environmental information collected by the on-board sensor are fused as the environment of the vehicle perceptual information reference;

When the aircraft performs a flight mission, acquire the second traffic information collected by the aircraft and the pose information of the aircraft when collecting the second traffic information, and use the second traffic information as the The vehicle's navigation information reference.
A data processing method, characterized in that it is applied to a camera, the method comprising:

In the process of performing the shooting task, detecting whether the camera is connected to the vehicle;

If the camera is connected to the vehicle, the image collected by performing the shooting task is sent to the vehicle; the image is used to trigger the vehicle to use the image to recognize user gestures or detect the driver's state; the user Gestures are used at least to control an aircraft communicatively linked with said vehicle to perform a flight mission;

If the camera is not connected to the vehicle, storing images collected by executing the shooting task.
A landing method for an aircraft, characterized in that it comprises:

In response to a landing trigger, searching for a vehicle to be landed, and determining a relative pose relationship between the aircraft and the vehicle to be landed;

estimating predicted motion information of the vehicle to be landed according to the motion information of the aircraft and the relative pose relationship, so as to plan the landing path of the aircraft;

The aircraft is controlled to land on the vehicle according to the landing path.
An aircraft, characterized in that it comprises:

body;

a power system, arranged on the fuselage, used to provide power for the aircraft; and,

One or more processors, arranged in the fuselage, are used to execute the method described in any one of claims 58 to 81,121.
A vehicle, characterized in that it comprises:

body;

The power system is arranged on the body and is used to provide power for the vehicle;

a plurality of display areas located at different locations for displaying display information for flight missions; and,

One or more processors, arranged in the fuselage, are used to execute the method described in any one of claims 82-117, 119.
A camera, characterized in that it comprises:

a lens assembly for transmitting light;

Photosensitive element, which is used to convert light into electrical signals and generate images;

memory for storing images; and,

A processor configured to perform the method of claim 120.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores executable instructions, and when the executable instructions are executed by a processor, the implementation of any one of claims 58-117, 119-121 described method.