WO2022077238A1

WO2022077238A1 - Imaging display method, remote control terminal, device, system, and storage medium

Info

Publication number: WO2022077238A1
Application number: PCT/CN2020/120705
Authority: WO
Inventors: 汪成; 周振
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-10-13
Filing date: 2020-10-13
Publication date: 2022-04-21
Also published as: CN113906481A

Abstract

An imaging display method, a remote control terminal (30), a device, a system, and a storage medium. The present invention is applied to the remote control terminal (30). The remote control terminal (30) is communicationally connected to a mobile platform, and an imaging device (21) and a point cloud acquisition device (22) are mounted on the mobile platform; the acquisition ranges of the imaging device (21) and the point cloud acquisition device (22) at least partially overlap. The method comprises: acquiring, in real time, a two-dimensional image acquired by the imaging device (21) and a three-dimensional point cloud acquired by the point cloud acquisition device (22) (S101); and simultaneously displaying, on an interactive interface of the remote control terminal (30), the two-dimensional image and a three-dimensional image indicated by the three-dimensional point cloud, so as to indicate a scanning result of scanning a target object by the point cloud acquisition device (22) (S102). By means of the imaging display method, the integrity of three-dimensional point cloud acquisition can be learned visually with reference to a two-dimensional image.

Description

Imaging display method, remote control terminal, device, system and storage medium

technical field

The present application relates to the technical field of computer software, and in particular, to an imaging display method, a remote control terminal, an apparatus, a system and a storage medium.

Background technique

3D reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of 3D objects. It is the basis for processing, operating and analyzing its properties in a computer environment.

In an implementation manner of the related art, a measuring instrument can scan the appearance surface of a three-dimensional object to collect point cloud data, perform three-dimensional reconstruction according to the point cloud data, and obtain a three-dimensional model of the three-dimensional object. However, the scanning of three-dimensional objects in the related art is usually based on the judgment of the target object with the naked eye, and there may be missed sampling (for example, a certain area of the three-dimensional object may not be scanned) or false sampling (for example, the collected point cloud The data does not belong to a three-dimensional object), resulting in incomplete or incorrect point cloud data collected, which in turn leads to wrong modeling. Further, from collecting point cloud data to finding errors in the generated three-dimensional model, re-collect the point cloud The entire closed loop of data will be very time-consuming and labor-intensive.

SUMMARY OF THE INVENTION

In view of this, one of the objectives of the present application is to provide an imaging display method, a remote control terminal, an apparatus, a system and a storage medium.

In the first aspect, an embodiment of the present application provides an imaging display method, which is applied to a remote control terminal, where the remote control terminal is used for communication and connection with a movable platform, and the movable platform is installed with an imaging device and a point cloud collection device; The acquisition ranges of the imaging device and the point cloud acquisition device at least partially overlap; the method includes:

acquiring the two-dimensional image collected by the imaging device and the three-dimensional point cloud collected by the point cloud collecting device in real time;

The two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the remote control terminal to instruct the point cloud collecting device to scan the scanning result of the target object.

In a second aspect, an embodiment of the present application provides a remote control terminal, including a processor, a memory for storing instructions executable by the processor, and a wireless communicator;

The wireless communicator is used to communicate and connect with the movable platform, and acquires in real time the two-dimensional image collected by the imaging device mounted on the movable platform and the three-dimensional point cloud collected by the point cloud collecting device mounted on the movable platform ; the acquisition ranges of the imaging device and the point cloud acquisition device at least partially overlap;

When the processor executes the executable instructions, it is used to: simultaneously display the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal to instruct the point cloud collection device Scan result of scanning target object.

In a third aspect, an embodiment of the present application provides an imaging display device, which is applied to a remote control terminal, where the remote control terminal is used for communication and connection with a movable platform, and the movable platform is installed with an imaging device and a point cloud collection device ; the acquisition ranges of the imaging device and the point cloud acquisition device at least partially overlap;

The apparatus includes one or more processors, operating individually or collectively, to perform the following operations:

In a fourth aspect, an embodiment of the present application provides an imaging display system, including a remote control terminal and a movable platform, the remote control terminal is used for communicating with a movable platform, and the movable platform is installed with an imaging device and a point cloud a collection device; the collection ranges of the imaging device and the point cloud collection device at least partially overlap;

The movable platform is used to acquire the two-dimensional image collected by the imaging device and the three-dimensional point cloud collected by the point cloud collecting device in real time, and transmit them to the remote control terminal;

The remote control terminal is used to acquire the two-dimensional image and the three-dimensional point cloud transmitted by the movable platform in real time, and simultaneously display the two-dimensional image and the three-dimensional point cloud indication on the interactive interface of the remote control terminal The three-dimensional image to instruct the point cloud acquisition device to scan the scan result of the target object.

In a fifth aspect, embodiments of the present application provide a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the method described in the first aspect.

An imaging display method, a remote control terminal, a device, a system and a storage medium provided by the embodiments of the present application are applied to a remote control terminal, and the remote control terminal is used for communication and connection with a movable platform, and the movable platform is installed with an imaging device and a point cloud acquisition device; the acquisition ranges of the imaging device and the point cloud acquisition device overlap at least partially, so as to ensure that the two-dimensional image acquired by the imaging device can serve as a reference for the scanning results of the point cloud acquisition device , which also allows users to deepen their understanding of the acquisition environment through 2D images and further ensure the accurate acquisition of 3D point clouds.

During the process of collecting the three-dimensional point cloud by the point cloud collecting device, the remote control terminal can obtain the two-dimensional image collected by the imaging device and the three-dimensional point cloud collected by the point cloud collecting device in real time, and the remote control terminal can obtain the three-dimensional point cloud collected by the point cloud collecting device in real time. The two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the device to instruct the point cloud collection device to scan the scanning result of the target object. The user can see the two-dimensional image and the three-dimensional image at the same time. With the aid of the two-dimensional image, the user can clearly understand the real-time scanning result of the point cloud acquisition device through the comparison of the two-dimensional image and the three-dimensional image displayed at the same time. Intuitively understand the completeness of the 3D point cloud collection, and can make judgments more directly and in a timely manner, thereby ensuring accurate and complete scanning of the target object, and at the same time avoiding the subsequent supplementary collection process, which is conducive to saving supplementary collection costs.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

1 and 4 to 16 are different schematic diagrams of an interactive interface provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a scenario provided by an embodiment of the present application;

3 is a schematic flowchart of an imaging display method provided by an embodiment of the present application;

17 is a schematic structural diagram of a remote control terminal provided by an embodiment of the present application;

FIG. 18 is a schematic diagram of an imaging display system provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In view of the problems existing in the related art, an embodiment of the present application provides an imaging display method, which is applied to a remote control terminal, where the remote control terminal is used for communication and connection with a movable platform, and the movable platform is installed with an imaging device and a point Cloud acquisition device; the acquisition range of the imaging device and the point cloud acquisition device at least partially overlap, so as to ensure that the two-dimensional image acquired by the imaging device can serve as a reference for the scanning result of the point cloud acquisition device, and also This enables users to deepen their understanding of the acquisition environment through 2D images and further ensure the accurate acquisition of 3D point clouds.

During the process of collecting the three-dimensional point cloud by the point cloud collecting device, the remote control terminal can obtain the two-dimensional image collected by the imaging device and the three-dimensional point cloud collected by the point cloud collecting device in real time, and the remote control terminal can obtain the three-dimensional point cloud collected by the point cloud collecting device in real time. The two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the device to instruct the point cloud collection device to scan the scanning result of the target object. For example, please refer to FIG. 1, the user can simultaneously see the two-dimensional image Image and 3D image, under the auxiliary reference of 2D image, through the comparison of the 2D image and 3D image displayed at the same time, the user can clearly understand the real-time scanning result of the point cloud acquisition device, in other words, the user can The completeness of the 3D point cloud collection can be intuitively understood under the reference of the 2D image, and the judgment can be made more directly and in a timely manner, thereby ensuring accurate and complete scanning of the target object, and at the same time avoiding the subsequent supplementary collection process, which is beneficial to Save on supplementary collection costs.

The remote control terminal includes, but is not limited to, a remote controller, a smart phone, a computer, or a tablet device with an interactive interface, and the remote control terminal can communicate with the mobile platform based on a short range wireless communication protocol or a mobile communication protocol; Wireless communication protocols include but are not limited to infrared protocol, WiFi protocol, Bluetooth protocol, UWB protocol or ZigBee protocol; the mobile communication protocol includes but is not limited to 3G communication protocol, 4G communication protocol, GSM communication protocol or GPRS communication protocol.

The movable platform includes, but is not limited to, an unmanned aerial vehicle, an unmanned vehicle, an unmanned vessel, or a mobile robot. An imaging device and a point cloud collecting device are installed on the movable platform, the imaging device is used for collecting two-dimensional images, and the point cloud collecting device is used for scanning a target object to collect a three-dimensional point cloud. The acquisition ranges of the imaging device and the point cloud acquisition device at least partially overlap, that is to say, the imaging device and the point cloud acquisition device are installed in close positions, so as to ensure that the two-dimensional images acquired by the imaging device can be It serves as a reference for the scanning situation of the point cloud acquisition device, and also enables the user to deepen the understanding of the acquisition environment through the two-dimensional image, and further ensures the accurate acquisition of the three-dimensional point cloud.

The point cloud collection device includes but is not limited to: lidar, binocular vision sensor and structured light depth camera.

The lidar is used to transmit a laser pulse sequence to a target object, then receive the laser pulse sequence reflected from the target object, and generate a three-dimensional point cloud according to the reflected laser pulse sequence. In one example, the lidar can determine the reception time of the reflected laser pulse sequence, eg, by detecting the rising edge time and/or the falling edge time of the electrical signal pulse to determine the reception time of the laser pulse sequence. In this way, the lidar can calculate TOF (Time of flight, time of flight) by using the receiving time information and the transmitting time of the laser pulse sequence, so as to determine the distance from the target object to the lidar. The lidar is a self-illuminating sensor, does not depend on light source illumination, is less disturbed by ambient light, can work normally even in a closed environment without light, and has wide applicability.

The binocular vision sensor obtains two images of the target object from different positions based on the principle of parallax, and obtains three-dimensional geometric information by calculating the positional deviation between the corresponding points of the two images, thereby generating a three-dimensional point cloud. The binocular vision sensor has low hardware requirements, and correspondingly, it can also reduce the cost. It only needs to be an ordinary CMOS (Complementary Metal Oxide Semiconductor) camera. As long as the light is suitable, both indoor and outdoor environments can be used. use, so it also has certain applicability.

The structured light depth camera is to project light with certain structural characteristics to the target object and then collect it. This kind of light with certain structure will collect different image phase information due to different depth areas of the subject, and then Convert it into depth information to obtain a 3D point cloud. Structured light depth cameras are also self-illuminating sensors, which do not depend on light source illumination and are less disturbed by ambient light.

The two-dimensional images that can be collected by the imaging device include color images, grayscale images, infrared images, and the like. The imaging device includes at least one or more of the following: a visible light camera, a grayscale camera, and an infrared camera.

The imaging device may capture a sequence of images at a specified frame rate. In some embodiments, the sequence of images may be captured at a standard frame rate such as about 24p, 25p, 30p, 48p, 50p, 60p, 72p, 90p, 100p, 120p, 300p, 50i or 60i. In some embodiments, less than or equal to about every 0.0001 second, 0.0002 second, 0.0005 second, 0.001 second, 0.002 second, 0.005 second, 0.01 second, 0.02 second, 0.05 second, 0.1 second, 0.2 second, 0.5 second, 1 Capture image sequences at frame rates of one image per second, 2 seconds, 5 seconds, or 10 seconds.

The imaging device may have adjustable capture parameters. Under different capture parameters, the imaging device may capture different images despite being subjected to exactly the same external conditions (eg location, lighting). Capture parameters may include exposure (eg, exposure time, shutter speed, aperture, film speed), gain, gamma, region of interest, binning/subsampling, pixel clock, offset, trigger, ISO, and the like. Exposure-related parameters can control the amount of light reaching the image sensor in the imaging device. For example, shutter speed can control the amount of time light reaches the image sensor and aperture can control the amount of light that reaches the image sensor in a given time. A gain-related parameter controls the amplification of the signal from the optical sensor. ISO controls the level of sensitivity of the camera to the available light.

In an exemplary embodiment, a lidar and a visible light camera are mounted on the movable platform. In one example, the lidar and the visible light camera may operate at the same frame rate. In another example, the lidar and the visible light camera may also operate at different frame rates.

In an embodiment, the imaging device and the point cloud collecting device are directly installed on the movable platform, which may be fixed installation or detachable installation, and this embodiment does not do anything about it. limit. Then, during the movement of the movable platform (such as moving, rotating, or flipping, etc.), the imaging device and the point cloud collecting device can be driven to collect in different directions, thereby realizing a complete scan of the target object.

In another embodiment, the imaging device and the point cloud collecting device may be installed on the movable platform through a pan/tilt head, which may be fixed installation or detachable installation, which is not the case in this embodiment. make any restrictions. Then, during the movement of the movable platform (such as moving, rotating, or flipping, etc.), and/or during the rotation of the PTZ, the imaging device and the point cloud collecting device can be driven to perform in different directions. acquisition, so as to achieve a complete scan of the target object. Wherein, and/or represent both or one of the two.

In an exemplary application scenario, please refer to FIG. 2 , taking the movable platform as an example of an unmanned aerial vehicle for illustration, the imaging device 21 and the point cloud collecting device 22 are installed on the On the unmanned aerial vehicle 10, the unmanned aerial vehicle 10 is connected in communication with the remote control terminal 30, and the user can control the unmanned aerial vehicle 10 through the remote control terminal 30. For example, the user can control the imaging device 21 through the remote control terminal 30 to collect two images. 3D image and the control of the point cloud collection device 22 to collect the 3D point cloud, and in the process, the remote control terminal 30 acquires the 2D image collected by the imaging device 21 and the point cloud collection device 22 in real time. 3D point cloud, and simultaneously display the 2D image and the 3D image indicated by the 3D point cloud on the interactive interface of the remote control terminal 30 to instruct the point cloud collection device 22 to scan the scan result of the target object, For example, please refer to Fig. 1, the user can see the 2D image and the 3D image at the same time. With the auxiliary reference of the 2D image, the user can clearly understand the point cloud collection by comparing the 2D image and the 3D image displayed at the same time. The real-time scanning results of the device, in other words, the user can intuitively understand the completeness of the 3D point cloud collection under the reference of the 2D image, and can make judgments more directly and in a timely manner, thereby ensuring accurate and complete scanning of the target object. At the same time, the subsequent supplementary collection process can also be avoided, which is beneficial to saving supplementary collection costs.

Please refer to FIG. 3 , which is a schematic flowchart of an imaging display method provided by an embodiment of the present application, which is applied to a remote control terminal, and the remote control terminal is used for communication and connection with a movable platform, and the movable platform is installed with an imaging device and a remote control terminal. A point cloud collection device; the collection ranges of the imaging device and the point cloud collection device at least partially overlap; the method includes:

In step S101, the two-dimensional image collected by the imaging device and the three-dimensional point cloud collected by the point cloud collecting device are acquired in real time.

In step S102, the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the remote control terminal to instruct the point cloud collecting device to scan the scanning result of the target object.

In one embodiment, the remote control terminal may control the imaging device to collect two-dimensional images and control the point cloud collecting device to collect three-dimensional point clouds. Specifically, at least one point cloud recording control is also displayed on the interactive interface of the remote control terminal, and the point cloud recording control is used to indicate the acquisition state of the imaging device and/or the point cloud acquisition device. Before step S101, the remote control terminal controls the imaging device to start collecting two-dimensional images and controls the point cloud collecting device to start scanning the target object in response to the user's operation on the point cloud recording control on the interactive interface. Collect 3D point clouds. In an example, please refer to FIG. 4 , where a point cloud recording control is displayed on the interactive interface as an example for description. The user can operate the point cloud recording control by clicking, double-clicking, long-pressing, etc., and the remote control terminal responds to The user's operation on the point cloud recording control on the interactive interface generates a control instruction sent to the movable platform, and then the movable platform controls the imaging device to start collecting two-dimensional images and controls according to the control instruction The point cloud acquisition device starts to scan the target object to acquire a three-dimensional point cloud.

Before the user operates the point cloud recording control to control the imaging device through the remote control terminal to start collecting two-dimensional images and control the point cloud collecting device to start scanning the target object to collect the three-dimensional point cloud, the user can also The acquisition parameters of the point cloud acquisition device are set on the remote control terminal based on their own requirements, so as to ensure that the 3D point cloud acquired by the point cloud acquisition device based on the set acquisition parameters meets the user's use requirements. Specifically, a setting control is also displayed on the interactive interface; the setting control is used to instruct the setting of the acquisition parameters of the point cloud acquisition device; the remote control terminal can respond to the user's operation on the setting control , displaying a setting page on the interactive interface, where the setting page displays setting information of the acquisition parameters of the point cloud acquisition device; and, in response to a user's setting operation on the setting page for the setting information, Acquire the acquisition parameters set by the point cloud acquisition device; after the user setting is completed, the user can operate the point cloud recording space, and then the remote control terminal responds to the user's operation on the point cloud recording control, according to the The collection parameters set by the point cloud collection device control the point cloud collection device to start scanning the target object to collect the three-dimensional point cloud; in this embodiment, before the point cloud collection device is controlled to collect the point cloud, the user can The collection parameters of the cloud collection device are set to meet the individual needs of users.

In an example, referring to FIG. 5 , the interactive interface also displays a setting control, and the user can operate the setting control by clicking, double-clicking, long-pressing, etc., and then displaying a setting page on the interactive interface. The setting page displays the setting information of the acquisition parameters of the point cloud acquisition device, the acquisition parameters include but are not limited to sampling frequency, scan mode, echo quantity, echo mode, whether with visible light pixels, rain and fog enhancement and strong denoising mode ; For example, the acquisition frequency can be selected from 80, 160 or 240; the scan mode can be selected from coverage mode or sweep mode; the number of echoes can be selected from 1, 2 or 3; when echo data is selected as 1, the echoes are displayed In the column of mode, the echo mode can select the nearest echo or the strongest echo; and whether with visible light pixels, rain and fog enhancement and strong denoising mode can be selected through the switch control; for example, please refer to Figure 5, when the user clicks , double-click, long-press, etc. to operate the scan mode column, the scan mode interface is displayed so that the user can choose the overlay mode or the scan mode, and other columns such as the sampling frequency, echo quantity, echo mode and scan mode This column operates in a similar way. After setting the acquisition parameters of the point cloud acquisition device, the user can operate the point cloud recording control to control the imaging device to start acquiring two-dimensional images through the remote control terminal and control the point cloud acquisition device according to the settings. After the acquisition parameters start to scan the target object to acquire a three-dimensional point cloud, so as to obtain a three-dimensional point cloud that meets the user's needs.

In order to further ensure the accuracy of 3D point cloud collection and subsequent 3D reconstruction, after the user operates the point cloud recording control, the interactive interface can display a recording preparation page, and the remote control terminal controls the movable platform Check whether its various components are running normally, and then receive the recording inspection result returned by the movable platform; if the recording inspection result indicates normal, the remote control terminal controls the imaging device to start collecting two-dimensional images and controls the The point cloud collection device starts to scan the target object to collect a three-dimensional point cloud; if the recording inspection result indicates an abnormality, abnormal information is displayed on the recording preparation page to remind the user.

In one example, referring to FIG. 6 , after the user operates the point cloud recording control, the interactive interface may display a recording preparation page, and the interactive interface displays each device to be inspected installed on the movable platform As well as the set acquisition parameters of the point cloud acquisition device, the device to be inspected may include, for example, an inertial measurement unit (IMU), a GPS locator, an RTK locator and other devices used to assist subsequent three-dimensional reconstruction. At the same time, the remote control terminal sends a recording inspection instruction to the movable platform, and the recording inspection instruction may include the acquisition parameters of the point cloud collection device set by the user; the movable platform may check each of its own based on the recording inspection instruction Whether the device is running normally and the point cloud acquisition device is set based on the acquisition parameters included in the recording detection instruction, and then the recording inspection result is returned to the remote control terminal; if the recording inspection result indicates normal, then in the The recording preparation page displays normal information to notify the user, and the remote control terminal controls the imaging device to start collecting two-dimensional images and the point cloud collecting device to start scanning the target object to collect three-dimensional point clouds; if the recording checks If the result indicates an abnormality, abnormal information is displayed on the recording preparation page to remind the user, and the imaging device and the point cloud acquisition device are not controlled to perform acquisition.

Next, during the process of collecting a two-dimensional image by the imaging device and collecting a three-dimensional point cloud by the point cloud collecting device, the user can control the imaging device and the point cloud collecting device to pause according to their actual needs through a remote control terminal Acquisition, continue acquisition or end acquisition, that is, the remote control terminal can control the imaging device and the point cloud acquisition device to suspend acquisition, continue acquisition or end acquisition in response to the user's operation on the point cloud recording control, so as to satisfy the User's personalized needs.

It can be understood that the control of the imaging device and the point cloud acquisition device to start acquisition, pause acquisition, continue acquisition or end acquisition can be controlled by one or more point cloud recording controls. The number is not limited in this embodiment. In an example, two point cloud recording controls are used for illustration here: please refer to Fig. 7, before starting the acquisition, a point cloud recording control is displayed on the interactive interface (for the convenience of distinction, it is referred to as the first point here Cloud recording control), after the user operates the first point cloud recording control, the remote control terminal controls the imaging device to start collecting two-dimensional images and controls the point cloud collecting device to start scanning the target object to collect three-dimensional points cloud; and another point cloud recording control is added on the interactive interface (for the convenience of distinction, referred to as the second point cloud recording control here), the added second point cloud recording control is displayed as a "pause" button, and the user can operate The "Pause" button is used to control the imaging device and the point cloud acquisition device to pause acquisition; after operating the "Pause" button, the second point cloud recording control is changed to a "Continue" button, and the user can operate the "Pause" button. "Continue" button to control the imaging device and the point cloud acquisition device to continue acquisition; during the process of acquiring a two-dimensional image by the imaging device and acquiring a three-dimensional point cloud by the point cloud acquisition device, the user can operate the The point cloud recording control is used to control the imaging device and the point cloud acquisition device through the remote control terminal to end acquisition, so as to meet the individual needs of the user.

During the process of collecting a two-dimensional image by the imaging device and collecting a three-dimensional point cloud by the point cloud collecting device, the remote control terminal can obtain the two-dimensional image collected by the imaging device and the three-dimensional image collected by the point cloud collecting device in real time. point cloud, and simultaneously display the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal to instruct the point cloud acquisition device to scan the scanning result of the target object, for example, please refer to Fig. 1. The user can see the two-dimensional image and the three-dimensional image at the same time. With the auxiliary reference of the two-dimensional image, through the comparison of the two-dimensional image and the three-dimensional image displayed at the same time, the user can clearly understand the real-time scanning of the point cloud acquisition device. As a result, in other words, the user can intuitively understand the completeness of the 3D point cloud collection under the reference of the 2D image, and can make judgments more directly and in a timely manner, so as to ensure the accurate and complete scanning of the target object. Avoiding the subsequent supplementary collection process is beneficial to saving supplementary collection costs. It can be understood that the embodiments of the present application do not impose any restrictions on the specific manner of displaying the two-dimensional image and the three-dimensional image simultaneously, and specific settings may be made according to actual application scenarios.

In one embodiment, referring to FIG. 1 , when the two-dimensional image and the three-dimensional image are displayed simultaneously, the two-dimensional image and the three-dimensional image may be displayed side by side on the interactive interface, so that the user can With the reference of the two-dimensional image, the scope and completeness of the three-dimensional point cloud collection can be intuitively understood, and it is also convenient to view the target object scanned this time from the two-dimensional image.

In another embodiment, when the two-dimensional image and the three-dimensional image are displayed at the same time, the two-dimensional image and the three-dimensional image may be superimposed and displayed on the interactive interface; as one implementation manner, the The two-dimensional image and the three-dimensional image may be displayed on different layers, and the layers are superimposed and displayed on the interactive interface, for example, the layer where the three-dimensional image is located is overlaid on the layer where the two-dimensional image is located. The superimposed display of the two-dimensional image and the three-dimensional image in this embodiment can clearly show the corresponding relationship between the two, and has a more intuitive reference for the scanning result of the target object scanned by the point cloud acquisition device, so that the user can understand more clearly The range and completeness of 3D point cloud collection.

In an implementation manner, with the reference of the two-dimensional image, the user can be informed of the scanning result of the point cloud collection device. When the scanning result indicates that the target object has not been scanned or that the scanned target object is not When complete, the user can make judgments more directly and in a timely manner, and can control the point cloud collection device to scan the target object according to the user's operation on the remote control terminal, thereby ensuring accurate and complete scanning of the target object. , thereby ensuring the integrity and accuracy of the collected 3D point cloud.

Exemplarily, a drone is equipped with a camera and a point cloud collection device. The camera is used to acquire two-dimensional images, and the point cloud acquisition device is used to scan objects and generate point clouds, and the point clouds are used to construct three-dimensional images. In the two-dimensional image, objects of preset categories in the two-dimensional image can be recognized, such as tall buildings, roads, bridges, etc. After comparing the 2D image with the 3D image, identify the unscanned objects. When the unscanned object belongs to the preset category, a bounding box is displayed on the periphery of the unscanned object. The user can click on the bounding box, and in response to the click operation, the drone automatically adjusts the pose of the point cloud acquisition device to scan the unscanned object. In another embodiment, in response to the clicking operation, the UAV plans a route according to the positions of a plurality of unscanned objects, so that the UAV performs flight and scanning operations according to the route.

The operation of the user on the remote control terminal includes, but is not limited to, the user's operation on the two-dimensional image and/or the three-dimensional image displayed on the interactive interface, or the user's operation on a specified control. For example, the user can conveniently control the point cloud acquisition device to scan the target object by double-clicking, long-pressing or sliding the two-dimensional image and/or the three-dimensional image, thereby simplifying the user's operation steps and improving the User experience.

If the imaging device and the point cloud collecting device are directly installed on the movable platform, when the remote control terminal controls the point cloud collecting device to scan the target object, according to the user's control of the remote control The operation of the terminal can adjust the pose of the movable platform, so that the point cloud collection device scans toward the target object, thereby ensuring accurate and complete scanning of the target object. In one example, the user can conveniently adjust the pose of the movable platform by manipulating the two-dimensional image and/or the three-dimensional image, for example, the two-dimensional image and/or the three-dimensional image can be slid, and the remote control terminal can slide the 2D image and/or the 3D image according to the sliding direction of the user. to adjust the pose of the movable platform, so as to adjust the scanning direction of the target object by the point cloud collection device.

In one example, when the point cloud collecting device is not facing the target object, the movable platform can be controlled to move, rotate or flip, etc. according to the user's operation on the remote control terminal, so that the point cloud The acquisition device can scan towards the target object, thereby ensuring accurate scanning of the target object. In another example, after the point cloud collecting device completes scanning one area of the target object, the movable platform can be controlled to move, rotate or flip, etc. according to the user's operation on the remote control terminal, so as to This enables the point cloud collection device to scan toward another area of the target object, thereby ensuring complete scanning of the target object.

If the imaging device and the point cloud collection device are installed on the movable platform through a pan-tilt; then the remote control terminal controls the point cloud collection device to scan the target object, according to the user's control of the remote control The operation of the terminal can adjust the pose of the movable platform and/or the pan/tilt, so that the point cloud acquisition device scans toward the target object, thereby ensuring accurate and complete scanning of the target object. In one example, the user can conveniently adjust the pose of the movable platform and/or the pan/tilt by manipulating the two-dimensional image and/or the three-dimensional image, for example, the two-dimensional image and/or the three-dimensional image can be slid, and the remote control terminal The pose of the movable platform and/or the gimbal is adjusted according to the sliding direction of the user, so as to adjust the scanning direction of the target object by the point cloud collection device. The point cloud collection device may be based on the adjusted Scan the target object in the scanning direction.

In one example, when the point cloud collecting device is not facing the target object, the movable platform can be controlled to move, rotate or flip, etc., according to the user's operation on the remote control terminal, and/or control all The pan/tilt is rotated so that the point cloud collecting device can scan towards the target object, thereby ensuring accurate scanning of the target object.

The above implementation method is that under the reference of the two-dimensional image, the user intuitively obtains the scanning result of the target object by the point cloud acquisition device and controls the operation through the remote control terminal. In order to further simplify the user's operation steps, another implementation method is used. , the remote control terminal can compare the two-dimensional image with the three-dimensional image, so as to obtain the scanning result of the point cloud collecting device scanning the target object, when the scanning result indicates that the target object is not scanned or the scanning When the arrived target object is incomplete, the remote control terminal controls the point cloud collection device to scan the target object according to the scanning result. In this embodiment, no user participation is required, and the remote control terminal directly acquires the scanning result and automatically controls the point cloud acquisition device based on the scanning result, which simplifies the user's operation steps and also helps to improve the scanning efficiency.

If the imaging device and the point cloud collecting device are directly installed on the movable platform, the remote control terminal can compare the two-dimensional image and the three-dimensional image to obtain the target object scanned by the point cloud collecting device and then adjust the pose of the movable platform according to the scanning result, so that the point cloud collection device scans toward the target object, thereby ensuring accurate and complete scanning of the target object.

If the imaging device and the point cloud collecting device are installed on the movable platform through a pan/tilt, the remote control terminal can compare the two-dimensional image and the three-dimensional image to obtain the target object scanned by the point cloud collecting device and then adjust the pose of the movable platform and/or the pan/tilt according to the scan results, so that the point cloud acquisition device scans towards the target object, so as to ensure the accuracy of the target object. Scan and full scan.

In one embodiment, in order to further meet the different needs of users, the interactive interface further displays a single-dual-screen switching control, and the single-dual-screen switching control is used to instruct the interactive interface to display only the three-dimensional image, or, Simultaneously display the two-dimensional image and the three-dimensional image; please refer to FIG. 8, when the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the remote control terminal, the remote control terminal Only the three-dimensional image can be displayed on the interactive interface in response to the user's operation on the single-dual-screen switching control, so as to meet the user's personalized needs.

In one embodiment, a two-dimensional image display control is further displayed on the interactive interface, and the two-dimensional image display control is used to instruct to display the two-dimensional image on the interactive interface, and display the two-dimensional image on the interactive interface. The collection range of the three-dimensional point cloud is displayed on the top; please refer to FIG. 9, when the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the remote control terminal, the remote control terminal can respond According to the user's operation of the two-dimensional image display control, the two-dimensional image is displayed on the interactive interface, and the collection range of the three-dimensional point cloud is displayed on the two-dimensional image. In this embodiment, the collection range of the 3D point cloud can be visually displayed in the 2D image, and the collection range of the 3D point cloud can be accurately located under the reference of the 2D image, so that the user can have a better understanding of the collection environment and the collection range of the 3D point cloud. Clear cognition, thus ensuring complete and accurate collection of 3D point clouds.

Wherein, considering that the acquisition range of the imaging device and the point cloud acquisition device are not completely overlapping, the position of the acquisition range of the three-dimensional point cloud in the two-dimensional image is based on the imaging device and the point cloud acquisition device. It is determined by the external parameter conversion relationship of the 3D point cloud, so as to ensure the accuracy of the position where the collection range of the 3D point cloud is located in the 2D image. It can be understood that the acquisition of the external parameter conversion relationship can be specifically set according to the actual application scenario, for example, the external parameter conversion relationship can be obtained by calibrating the imaging device and the point cloud collecting device.

Wherein, the size of the collection range of the three-dimensional point cloud in the two-dimensional image is determined according to the focal length of the imaging device. When the imaging device zooms and shoots, the size of the collection range of the 3D point cloud in the 2D image will be enlarged or reduced according to the zoom factor of the imaging device, so that the size displayed in the 2D image will be enlarged or reduced. The collection range of the three-dimensional point cloud can be adapted to the two-dimensional image, so as to ensure the accuracy of the displayed collection range of the three-dimensional point cloud.

In one embodiment, the movable platform may be installed with at least two PTZs, wherein one PTZ is installed with at least an imaging device and a point cloud acquisition device, and the other PTZs are installed with at least an imaging device; in an implementation manner , please refer to FIG. 10 , the two-dimensional image collected by the imaging device of one of the pan-tilts and the two-dimensional image collected by the point cloud collecting device are simultaneously displayed on the interactive interface, that is, the two-dimensional image displayed on the interactive interface The two-dimensional image collected by the imaging device of one of the pan-tilts, and the interactive interface may also display the first thumbnails of the two-dimensional images collected by the imaging devices of the other pan-tilts (for example, as shown in FIG. 10 ). shown in the lower right). After the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the remote control terminal, the remote control terminal may, in response to the user's operation on the first thumbnail, display the The two-dimensional image acquired by the imaging device of one of the pan/tilts displayed on the interactive interface is switched to the two-dimensional image indicated by the first thumbnail. In this embodiment, the user can watch the two-dimensional images collected by the imaging devices of different pan-tilts, realize multi-dimensional observation under the reference of the two-dimensional images of different angles, and can compare the differences between the two-dimensional images of different imaging devices, so that the Users can more intuitively understand the completeness and accuracy of 3D point cloud acquisition, and can make judgments more directly and in a timely manner, thereby ensuring accurate and complete scanning of the target object, and avoiding subsequent supplementary acquisition processes. Conducive to saving the cost of supplementary collection.

In an example, the user can quickly switch images by dragging and other operations. For the convenience of distinction, for example, the two-dimensional image currently displayed on the interactive interface is named as the first image. At this time, the first thumbnail indicates the The two-dimensional image is named as the second image, the user drags the first thumbnail to the location of the first image, and the remote control terminal switches the first image to the second image indicated by the first thumbnail according to the drag operation. image, correspondingly, the first thumbnail image at this time indicates the first image.

In one embodiment, the remote control terminal may adjust the poses of the at least two PTZs in response to the user's operation on the first thumbnail, the two-dimensional image and/or the three-dimensional image, so as to So that the imaging devices installed on the at least two PTZs can collect two-dimensional images from different directions and the point cloud collecting device can scan the target object; The convenient operation of the two-dimensional image and/or the three-dimensional image can quickly and conveniently control the orientation of each pan/tilt, so as to obtain two-dimensional images collected from different angles, and realize the reference of the two-dimensional images from different angles. Multi-dimensional observation ensures the integrity and correctness of the target object scanning. Wherein, the and/or represent three or any combination of the three. The operation of the user on the first thumbnail, the two-dimensional image and/or the three-dimensional image includes but is not limited to a sliding operation, a double-click operation, a long-press operation, and the like.

In one example, for example, the two-dimensional image and/or the first thumbnail can be slid, and the remote control terminal adjusts the pose of the corresponding pan/tilt according to the sliding direction of the user, thereby adjusting the position of the pan/tilt on the pan/tilt. According to the shooting angle of the imaging device, two-dimensional images collected from different angles are acquired, and multi-dimensional observation is realized under the reference of two-dimensional images from different angles.

In one example, when the scanning result indicates that the target object is not scanned or the scanned target object is incomplete, the remote control terminal can adjust the pose of the corresponding pan/tilt according to the user's operation of sliding the three-dimensional image, Thus, the scanning direction of the target object by the point cloud collecting device on the PTZ is adjusted.

In one embodiment, referring to FIG. 11 , the interactive interface may also display a second thumbnail image of the map where the pan/tilt is located; the two-dimensional image and After the three-dimensional image indicated by the three-dimensional point cloud, the remote control terminal may simultaneously display the map and the three-dimensional image on the interactive interface in response to the user's operation on the second thumbnail. In this embodiment, the map and the 3D map can be displayed at the same time. With the reference of the map, the user can intuitively understand the location of the target object, which further ensures the accuracy and integrity of the 3D point cloud collection.

It can be understood that the embodiments of the present application do not impose any restrictions on the specific manner of displaying the map and the three-dimensional image simultaneously, and specific settings may be made according to actual application scenarios.

In one embodiment, when the map and the three-dimensional image are displayed at the same time, the map and the three-dimensional image can be displayed side by side on the interactive interface, so that the user can intuitively understand the three-dimensional point with reference to the map. The geographic location of the cloud collection.

In another embodiment, when the map and the three-dimensional image are displayed at the same time, the map and the three-dimensional image may be superimposed and displayed on the interactive interface; The three-dimensional image may be displayed on different layers, and the layers are displayed in a superimposed manner on the interactive interface, for example, the layer where the three-dimensional image is located is overlaid on the layer where the map is located. In this embodiment, the superimposed display of the map and the three-dimensional image can clearly show the corresponding relationship between the two, so that the user can more clearly understand the geographic location of the three-dimensional point cloud collection.

In an embodiment, a rendering control is further displayed on the interactive interface; the rendering control is used to instruct to perform color rendering on the three-dimensional image; before displaying the three-dimensional image or during the process of displaying the three-dimensional image, The remote control terminal may display a rendering page on the interactive interface in response to the user's operation on the rendering control, and the rendering page displays at least one rendering mode, and the rendering mode is used to indicate the color rendering mode of the three-dimensional image. ; and then perform color rendering on the three-dimensional image based on the rendering mode selected by the user on the rendering page. This embodiment implements color rendering of a three-dimensional image, which is beneficial to improve the user's visual experience.

Wherein, the rendering mode includes at least one of the following: a reflectance rendering mode, a height rendering mode, and a two-dimensional image rendering mode; the reflectance rendering mode is used to indicate the reflectivity according to each three-dimensional point in the three-dimensional point cloud. coloring; the height rendering mode is used to indicate that coloring is performed according to the height information of each 3D point in the 3D point cloud; the 2D image rendering mode is used to indicate the pixels according to the pixels in the 2D image value to generate coloring information for the 3D point cloud. In each rendering mode, there is also a color matching function, and users can set different color combinations based on their own needs to meet the user's experience.

Wherein, a rendering close control is displayed on the rendering page, and the rendering close control instructs to restore the three-dimensional image after color rendering, for example, restore the three-dimensional image to a gray style. In one embodiment, a measurement control is also displayed on the interactive interface; the measurement control is used to instruct to perform spatial measurement on the three-dimensional image; the user can perform a measurement function according to his own needs, and the remote control terminal can respond to the user. For the operation of the measurement control, a measurement page is displayed on the interactive interface, and the three-dimensional image is displayed on the measurement page; then in response to the user's operation on the three-dimensional image, at least one measurement point is acquired, and according to the At least one measurement point acquires distance information and/or angle information.

In one example, when acquiring the measurement point, the remote control terminal may display the measurement point on the measurement page of the interactive interface, and considering that the user's finger is too large, if the finger is directly placed on the To operate at the location of the measurement point, the finger may cover the measurement point, which may affect the accuracy of the measurement result because the specific position of the measurement point cannot be seen. Therefore, the remote control terminal can respond to the user's operation, displaying a moving marker near the measurement point; and, in response to the user's operation on the moving marker, determining the distance where the moving marker is located relative to the measuring point. In this embodiment, the operation for the measurement point is equivalent to the operation for the moving marker, so that the user can clearly see the specific position of the measurement point and also ensures the accuracy of the measurement result.

In an example, please refer to FIG. 12, on the measurement page, when it is detected that the user clicks on the three-dimensional image, the remote control terminal acquires a measurement point according to the click position of the user and displays it; considering the user's finger If it is too large, if you put your finger directly on the position of the measurement point to operate, the finger may cover the measurement point, which may cause operation errors because the specific position of the measurement point cannot be seen, thus affecting the accuracy of the measurement result. Therefore, the remote control terminal can display a moving mark near the measuring point in response to the user touching the position of the measuring point, such as in the form of a "moving bubble", and the user can move the moving mark, such as placing the mobile phone on the " The "moving bubble" can be dragged on the "moving bubble", and then the position where the moving marker stays is the moving position, and the remote control terminal can determine the distance of the current moving marker relative to the measuring point.

In one example, please refer to FIG. 13 , the remote control terminal acquires two measurement points and displays them on the measurement page in response to the user clicking on different positions of the three-dimensional image twice, and then acquires the two measurement points distance information and displayed on the measurement page.

In one example, please refer to FIG. 14 , in response to the user clicking different positions of the three-dimensional image three times, the remote control terminal acquires three measurement points and displays them on the measurement page, and then acquires one of the three measurement points. The distance information, area information and/or angle information between the two devices are displayed on the measurement page.

In one embodiment, a playback control is also displayed on the interactive interface; the playback control is used to at least indicate a pre-collected 3D point cloud and/or a 2D image; please refer to FIG. 15 , the remote control terminal can respond to The user's operation on the playback control displays a playback list that displays at least the pre-collected 3D point cloud and/or 2D image, and then responds to the user's operation of the 3D point cloud and/or 2D image. The operation is to display the 3D image and/or the 2D image indicated by the 3D point cloud on the interactive interface. In this embodiment, the user may choose to play back the pre-collected three-dimensional point cloud and/or two-dimensional image according to actual needs.

16, when the 3D image indicated by the pre-collected 3D point cloud is displayed, a supplementary recording control is also displayed on the interactive interface, and the supplementary recording control is used to instruct the supplementary acquisition of the 3D point cloud. The three-dimensional point cloud can be checked and filled according to its own needs; the remote control terminal can control the point cloud acquisition device to supplement the acquisition of the three-dimensional point cloud in response to the user's operation of the supplementary recording control.

In an implementation manner, the remote control terminal can control the point cloud acquisition device to perform supplementary scanning on the target object according to the density of the three-dimensional point cloud, so as to supplement the acquisition of the three-dimensional point cloud; for example, by setting a density threshold, when a certain area is When the density of the three-dimensional point cloud is less than the density threshold, the point cloud collection device is controlled to perform supplementary scanning on this area of the target object.

In another implementation manner, it is considered that when the 3D point cloud is collected by the point cloud collection device, the imaging device will also be used to collect the 2D image, and the collection range of the imaging device and the point cloud collection device is at least partially overlap, so the two-bit image can serve as a reference, the remote control terminal can acquire the pre-collected two-dimensional image corresponding to the pre-collected three-dimensional point cloud, and according to the pre-collected three-dimensional point cloud According to the comparison result between the indicated three-dimensional image and the pre-collected two-dimensional image, the point cloud collection device is controlled to perform supplementary scanning on the target object to supplement the collection of the three-dimensional point cloud. For example, the leaked collection area can be determined by comparing with the two-dimensional image, and the point cloud collection device can be controlled to perform supplementary scanning on the leaked collection area of the target object according to the leaked collection area.

Correspondingly, referring to FIG. 17 , an embodiment of the present application further provides a remote control terminal 30, including a processor 31, a memory 32 for storing executable instructions of the processor 31, and a wireless communicator 33;

The wireless communicator 33 is used to communicate and connect with the movable platform, and acquires in real time the two-dimensional images collected by the imaging device mounted on the movable platform and the three-dimensional images collected by the point cloud collecting device mounted on the movable platform. a point cloud; the acquisition ranges of the imaging device and the point cloud acquisition device at least partially overlap;

When the processor 31 executes the executable instructions, it is configured to: simultaneously display the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal, so as to instruct the point cloud to collect The device scans the scan result of the target object.

The processor 31 executes the executable instructions included in the memory 32, and the processor 31 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processors) Processor, DSP), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 32 stores executable instructions of the imaging display method, and the memory 32 may include at least one type of storage medium, including flash memory, hard disk, multimedia card, card-type memory (eg, SD or DX memory, etc.) , Random Access Memory (RAM), Static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Disk, CD and so on. Also, the remote control terminal may cooperate with a network storage device that performs the storage function of the memory through a network connection. The memory 32 may be an internal storage unit of the remote control terminal 30 , such as a hard disk or a memory of the remote control terminal 30 . The memory 32 can also be an external storage remote control terminal of the remote control terminal 30, such as a plug-in hard disk equipped on the remote control terminal 30, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card ( Flash Card), etc. Further, the memory 32 may also include both an internal storage unit of the remote control terminal 30 and an external storage remote control terminal. The memory 32 is used to store the computer program 33 and other programs and data required by the remote control terminal. The memory 32 may also be used to temporarily store data that has been or will be output.

In one embodiment, the processor 31 is further configured to: when the scan result indicates that the target object is not scanned or the scanned target object is incomplete, control the point according to the user's operation on the remote control terminal. The cloud collection device scans the target object.

In one embodiment, when controlling the point cloud collecting device to scan the target object, the processor 31 is specifically configured to: adjust the pose of the movable platform, so that the point cloud collecting device Scan towards the target object.

In one embodiment, the imaging device and the point cloud acquisition device are mounted on the movable platform through a pan-tilt;

When controlling the point cloud collection device to scan the target object, the processor 31 is specifically configured to: adjust the pose of the movable platform and/or the PTZ, so that the point cloud collection is performed The device scans towards the target object.

In an embodiment, the user's operation on the remote control terminal at least includes: the user's operation on the two-dimensional image and/or the three-dimensional image.

In an embodiment, the two-dimensional image and the three-dimensional image are displayed side by side or superimposed on the interactive interface.

In one embodiment, the two-dimensional image and the three-dimensional image are displayed on different layers, and the layers are displayed superimposed on the interactive interface.

In an embodiment, at least one point cloud recording control is further displayed on the interactive interface, and the point cloud recording control is used to indicate the acquisition state of the imaging device and the point cloud acquisition device.

Before acquiring the two-dimensional image collected by the imaging device and the three-dimensional point cloud collected by the point cloud collecting device in real time, the processor 31 is further configured to: in response to the user recording the point cloud on the interactive interface The operation of the control controls the imaging device to start collecting two-dimensional images and the point cloud collecting device to start scanning the target object to collect three-dimensional point clouds.

In one embodiment, after the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the remote control terminal, the processor 31 is further configured to: The operation of the point cloud recording control controls the imaging device and the point cloud acquisition device to suspend acquisition, continue acquisition or end acquisition.

In one embodiment, a setting control is further displayed on the interactive interface; the setting control is used to instruct to set the acquisition parameters of the point cloud acquisition device.

Before the control of the imaging device to start collecting a two-dimensional image and the control of the point cloud collecting device to start scanning the target object to collect a three-dimensional point cloud, the processor 31 is further configured to:

In response to the user's operation of the setting control, a setting page is displayed on the interactive interface, and the setting page displays the setting information of the acquisition parameters of the point cloud acquisition device; and,

In response to the user's setting operation for the setting information on the setting page, acquiring the collection parameters set by the point cloud collection device;

According to the acquisition parameters set by the point cloud acquisition device, the point cloud acquisition device is controlled to start scanning the target object to acquire a three-dimensional point cloud.

In an embodiment, the acquisition parameters include at least one of the following: sampling frequency, scan mode, echo quantity, echo mode, whether or not to have visible light pixels, rain and fog enhancement, and strong denoising mode.

In one embodiment, the interactive interface further displays a single-dual-screen switching control, and the single-dual-screen switching control is used to instruct the interactive interface to display only the three-dimensional image, or simultaneously display the two-dimensional image and the two-dimensional image. the three-dimensional image.

When simultaneously displaying the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal, the processor 31 is further configured to: respond to the user's operation on the single-dual-screen switching control , and only the three-dimensional image is displayed on the interactive interface.

In one embodiment, a two-dimensional image display control is further displayed on the interactive interface, and the two-dimensional image display control is used to instruct to display the two-dimensional image on the interactive interface, and display the two-dimensional image on the interactive interface. The acquisition range of the 3D point cloud is displayed on the top.

When displaying the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal, the processor 31 is further configured to: in response to a user's operation on the two-dimensional image display control, The two-dimensional image is displayed on the interactive interface, and the acquisition range of the three-dimensional point cloud is displayed on the two-dimensional image.

In an embodiment, the position of the collection range of the three-dimensional point cloud in the two-dimensional image is determined according to the external parameter conversion relationship between the imaging device and the point cloud collecting device.

In one embodiment, the size of the collection range of the three-dimensional point cloud in the two-dimensional image is determined according to the focal length of the imaging device.

In one embodiment, the imaging device includes at least one of the following: a visible light camera, a grayscale camera, or an infrared camera; the point cloud collection device includes at least one of the following: a lidar, a binocular vision sensor, and a structured light depth camera .

In one embodiment, the movable platform is installed with at least two cradles, one of which is installed with at least an imaging device and a point cloud acquisition device, and the other cradles are installed with at least an imaging device.

The two-dimensional image displayed on the interactive interface is the two-dimensional image collected by the imaging device of one of the pan/tilts, and the interactive interface also displays the second image of the two-dimensional image collected by the imaging device of the other pan/tilt. a thumbnail.

After the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the remote control terminal, the processor 31 is further configured to: respond to the user's response to the first thumbnail image to switch the two-dimensional image captured by the imaging device of one of the pan/tilts displayed on the interactive interface to the two-dimensional image indicated by the first thumbnail.

In an embodiment, the processor 31 is further configured to: in response to a user's operation on the first thumbnail, the two-dimensional image and/or the three-dimensional image, adjust the pose.

In one embodiment, the interactive interface further displays a second thumbnail image of the map where the pan/tilt is located.

After the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are displayed on the interactive interface of the remote control terminal, the processor 31 is further configured to: respond to the user's selection of the second thumbnail image operation, displaying the map and the three-dimensional image simultaneously on the interactive interface.

In one embodiment, the map and the three-dimensional image are superimposed and displayed on the interactive interface.

In one embodiment, a rendering control is further displayed on the interactive interface; the rendering control is used to instruct to perform color rendering on the three-dimensional image.

The processor 31 is further configured to: in response to a user's operation on the rendering control, display a rendering page on the interactive interface, where the rendering page displays at least one rendering mode, and the rendering mode is used to indicate the three-dimensional image color rendering mode; color rendering is performed on the three-dimensional image based on the rendering mode selected by the user on the rendering page.

In one embodiment, the rendering mode includes at least one of the following: a reflectance rendering mode, a height rendering mode, and a two-dimensional image rendering mode; wherein, the reflectance rendering mode is used to indicate the The reflectivity of each three-dimensional point is used for coloring; the height rendering mode is used to indicate that coloring is performed according to the height information of each three-dimensional point in the three-dimensional point cloud; the two-dimensional image rendering mode is used to indicate that according to the two-dimensional image rendering mode The pixel values of the pixels in the 3D image generate the coloring information of the 3D point cloud.

In one embodiment, a measurement control is further displayed on the interactive interface; the measurement control is used to instruct to perform spatial measurement on the three-dimensional image.

The processor 31 is further configured to: in response to the user's operation on the measurement control, display a measurement page on the interactive interface, where the three-dimensional image is displayed on the measurement page; in response to the user's operation on the three-dimensional image , acquire at least one measurement point, and acquire distance information and/or angle information according to the at least one measurement point.

In one embodiment, when the at least one measurement point is acquired, the processor 31 is further configured to: display the at least one measurement point on the interactive interface; and in response to the user's operation on the measurement point, Displaying a moving marker near the measurement point; and, in response to a user's operation on the moving marker, determining a distance of the location of the moving marker relative to the measuring point.

In one embodiment, a playback control is further displayed on the interactive interface; the playback control is used to at least indicate a pre-collected 3D point cloud and/or a 2D image.

The processor 31 is further configured to: in response to the user's operation on the playback control, display a playback list, where the playback list displays at least the three-dimensional point cloud and/or the two-dimensional image collected in advance; For the operation of the 3D point cloud and/or the 2D image, the 3D image and/or the 2D image indicated by the 3D point cloud is displayed on the interactive interface.

In one embodiment, when the 3D image indicated by the pre-collected 3D point cloud is displayed, a supplementary recording control is further displayed on the interactive interface, and the supplementary recording control is used to instruct supplementary acquisition of the 3D point cloud.

The processor 31 is further configured to: in response to the user's operation on the supplementary recording control, control the point cloud acquisition device to supplement the acquisition of the three-dimensional point cloud.

In one embodiment, when controlling the point cloud collection device to supplement the collection of the 3D point cloud, the processor 31 is specifically configured to: control the point cloud collection device to supplement the target object according to the density of the 3D point cloud. Scanning to supplement the collection of 3D point clouds; or, obtaining a pre-collected 2D image corresponding to the pre-collected 3D point cloud, and according to the pre-collected 3D image indicated by the According to the comparison result of the two-dimensional images, the point cloud collecting device is controlled to perform supplementary scanning on the target object, so as to supplement and collect the three-dimensional point cloud.

The various embodiments described herein can be implemented using computer readable media such as computer software, hardware, or any combination thereof. For hardware implementation, the embodiments described herein can be implemented using application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays ( FPGA), processors, controllers, microcontrollers, microprocessors, electronic units designed to perform the functions described herein are implemented. For software implementation, embodiments such as procedures or functions may be implemented with separate software modules that allow the performance of at least one function or operation. The software codes may be implemented by a software application (or program) written in any suitable programming language, which may be stored in memory and executed by a controller.

For details of the implementation process of the functions and functions of the various units in the above-mentioned terminal, please refer to the implementation process of the corresponding steps in the above method, which will not be repeated here.

Embodiments of the present application also provide an imaging display device, which is applied to a remote control terminal, where the remote control terminal is used for communication and connection with a movable platform, and the movable platform is installed with an imaging device and a point cloud collection device; the imaging device At least partially overlaps with the collection range of the point cloud collection device;

Referring to FIG. 18 , an embodiment of the present application further provides an imaging display system, including a remote control terminal and a movable platform, where the remote control terminal is configured to communicate with a movable platform, and the movable platform is installed with an imaging device and a movable platform. a point cloud collection device; the collection ranges of the imaging device and the point cloud collection device at least partially overlap;

In one embodiment, the movable platform includes at least an unmanned aerial vehicle, an unmanned vehicle, an unmanned watercraft, and a mobile robot.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium, such as a memory including instructions, executable by a processor of an apparatus to perform the above-described method. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the terminal, enable the terminal to execute the above method.

It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. The terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion such that a process, method, article or device comprising a list of elements includes not only those elements, but also other not expressly listed elements, or also include elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

The methods and devices provided by the embodiments of the present application have been introduced in detail above, and specific examples are used to illustrate the principles and implementations of the present application. At the same time, for those of ordinary skill in the art, according to the idea of the application, there will be changes in the specific implementation and application scope. In summary, the content of this specification should not be construed as a limitation to the application. .

Claims

An imaging display method, characterized in that it is applied to a remote control terminal, the remote control terminal is used for communication and connection with a movable platform, and the movable platform is installed with an imaging device and a point cloud collection device; the imaging device and the The collection ranges of the point cloud collection devices at least partially overlap; the method includes:

acquiring the two-dimensional image collected by the imaging device and the three-dimensional point cloud collected by the point cloud collecting device in real time;

The two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the remote control terminal to instruct the point cloud collecting device to scan the scanning result of the target object.
The method of claim 1, further comprising:

When the scanning result indicates that the target object is not scanned or the scanned target object is incomplete, the point cloud collection device is controlled to scan the target object according to the user's operation on the remote control terminal.
The method according to claim 2, wherein the controlling the point cloud collection device to scan the target object comprises:

The pose of the movable platform is adjusted so that the point cloud acquisition device scans toward the target object.
The method according to claim 2, wherein the imaging device and the point cloud collecting device are installed on the movable platform through a pan/tilt;

The controlling the point cloud collection device to scan the target object includes:

Adjusting the pose of the movable platform and/or the pan/tilt head so that the point cloud collection device scans toward the target object.
The method according to claim 2, wherein the user's operation on the remote control terminal at least comprises: the user's operation on the two-dimensional image and/or the three-dimensional image.
The method according to claim 1, wherein the two-dimensional image and the three-dimensional image are displayed side by side or superimposed on the interactive interface.
The method according to claim 1, wherein the two-dimensional image and the three-dimensional image are displayed on different layers, and the layers are displayed superimposed on the interactive interface.
The method according to claim 1, wherein at least one point cloud recording control is further displayed on the interactive interface, and the point cloud recording control is used to instruct the imaging device and the point cloud collecting device to collect state;

Before acquiring the two-dimensional image collected by the imaging device and the three-dimensional point cloud collected by the point cloud collecting device in real time, the method further includes:

In response to the user's operation on the point cloud recording control on the interactive interface, the imaging device is controlled to start collecting two-dimensional images and the point cloud collecting device is controlled to start scanning the target object to collect three-dimensional point clouds.
The method according to claim 8, wherein after simultaneously displaying the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal, the method further comprises:

In response to the user's operation on the point cloud recording control, the imaging device and the point cloud acquisition device are controlled to suspend acquisition, continue acquisition or end acquisition.
The method according to claim 8, wherein a setting control is further displayed on the interactive interface; the setting control is used to instruct the setting of the acquisition parameters of the point cloud acquisition device;

Before the controlling the imaging device to start collecting a two-dimensional image and controlling the point cloud collecting device to start scanning the target object to collect a three-dimensional point cloud, the method further includes:

In response to the user's operation of the setting control, a setting page is displayed on the interactive interface, and the setting page displays the setting information of the acquisition parameters of the point cloud acquisition device; and,

In response to the user's setting operation for the setting information on the setting page, acquiring the collection parameters set by the point cloud collection device;

The controlling the point cloud acquisition device to start scanning the target object to acquire a three-dimensional point cloud includes:

According to the acquisition parameters set by the point cloud acquisition device, the point cloud acquisition device is controlled to start scanning the target object to acquire a three-dimensional point cloud.
The method according to claim 10, wherein the acquisition parameters include at least one of the following: sampling frequency, scanning mode, number of echoes, echo mode, whether with visible light pixels, rain and fog enhancement, and strong denoising mode.
The method according to claim 1, wherein the interactive interface further displays a single-dual-screen switching control, and the single-dual-screen switching control is used to instruct the interactive interface to display only the three-dimensional image, or, at the same time, displaying the two-dimensional image and the three-dimensional image;

When simultaneously displaying the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal, the method further includes:

In response to the user's operation on the single-dual-screen switching control, only the three-dimensional image is displayed on the interactive interface.
The method according to claim 1, wherein a two-dimensional image display control is further displayed on the interactive interface, and the two-dimensional image display control is used to instruct the two-dimensional image to be displayed on the interactive interface, and displaying the collection range of the 3D point cloud on the 2D image;

When displaying the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal, the method further includes:

In response to the user's operation on the two-dimensional image display control, the two-dimensional image is displayed on the interactive interface, and the collection range of the three-dimensional point cloud is displayed on the two-dimensional image.
The method according to claim 13, wherein the position of the collection range of the three-dimensional point cloud in the two-dimensional image is determined according to the external parameter conversion relationship between the imaging device and the point cloud collecting device.
The method according to claim 13, wherein the size of the collection range of the three-dimensional point cloud in the two-dimensional image is determined according to the focal length of the imaging device.
The method according to claim 1, wherein the imaging device comprises at least one of the following: a visible light camera, a grayscale camera, or an infrared camera;

The point cloud collection device includes at least one of the following: a lidar, a binocular vision sensor, and a structured light depth camera.
The method according to claim 1, wherein the movable platform is installed with at least two PTZs, wherein one PTZ is installed with at least an imaging device and a point cloud acquisition device, and the other PTZs are installed with at least an imaging device;

The two-dimensional image displayed on the interactive interface is the two-dimensional image collected by the imaging device of one of the pan/tilts, and the interactive interface also displays the second image of the two-dimensional image collected by the imaging device of the other pan/tilt. a thumbnail;

After simultaneously displaying the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal, the method further includes:

In response to the user's operation on the first thumbnail, the two-dimensional image captured by the imaging device of one of the pan/tilts displayed on the interactive interface is switched to the two-dimensional image indicated by the first thumbnail.
The method of claim 17, further comprising:

In response to a user's manipulation of the first thumbnail image, the two-dimensional image and/or the three-dimensional image, the poses of the at least two PTZs are adjusted.
The method according to claim 1, wherein the interactive interface further displays a second thumbnail of the map where the PTZ is located;

After the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are displayed on the interactive interface of the remote control terminal, the method further includes:

In response to the user's operation on the second thumbnail, the map and the three-dimensional image are simultaneously displayed on the interactive interface.
The method according to claim 19, wherein the map and the three-dimensional image are superimposed and displayed on the interactive interface.
The method according to claim 1, wherein a rendering control is also displayed on the interactive interface; the rendering control is used for instructing to perform color rendering on the three-dimensional image;

The method also includes:

In response to the user's operation on the rendering control, a rendering page is displayed on the interactive interface, and the rendering page displays at least one rendering mode, and the rendering mode is used to indicate the color rendering mode of the three-dimensional image;

The three-dimensional image is color-rendered based on the rendering mode selected by the user on the rendering page.
The method according to claim 21, wherein the rendering mode comprises at least one of the following: a reflectance rendering mode, a height rendering mode, and a two-dimensional image rendering mode;

Wherein, the reflectance rendering mode is used to indicate that the coloring is performed according to the reflectivity of each three-dimensional point in the three-dimensional point cloud;

The height rendering mode is used to indicate that coloring is performed according to the height information of each 3D point in the 3D point cloud;

The two-dimensional image rendering mode is used to instruct to generate the coloring information of the three-dimensional point cloud according to the pixel values of the pixels in the two-dimensional image.
The method according to claim 1, wherein a measurement control is further displayed on the interactive interface; the measurement control is used for instructing to perform spatial measurement on the three-dimensional image;

The method also includes:

In response to the user's operation of the measurement control, displaying a measurement page on the interactive interface, where the three-dimensional image is displayed on the measurement page;

In response to a user's operation on the three-dimensional image, at least one measurement point is acquired, and distance information and/or angle information is acquired according to the at least one measurement point.
The method according to claim 23, wherein when acquiring at least one measurement point, the method further comprises:

displaying the at least one measurement point on the interactive interface;

in response to user manipulation of the measurement point, displaying a moving marker near the measurement point; and,

In response to the user's operation on the moving marker, the distance of the location where the moving marker is located relative to the measurement point is determined.
The method according to claim 1, wherein a playback control is further displayed on the interactive interface; the playback control is used to at least indicate a pre-collected 3D point cloud and/or a 2D image;

The method also includes:

In response to the user's operation of the playback control, displaying a playback list, where the playback list displays at least a pre-collected 3D point cloud and/or a 2D image;

In response to the user's operation on the three-dimensional point cloud and/or the two-dimensional image, the three-dimensional image and/or the two-dimensional image indicated by the three-dimensional point cloud is displayed on the interactive interface.
The method according to claim 25, wherein when the 3D image indicated by the pre-collected 3D point cloud is displayed, a supplementary recording control is further displayed on the interactive interface, and the supplementary recording control is used to indicate supplementary recording Collect 3D point cloud;

The method also includes:

In response to the user's operation on the supplementary recording control, the point cloud acquisition device is controlled to supplementally acquire the three-dimensional point cloud.
The method according to claim 26, wherein the controlling the point cloud collection device to supplement the collection of the 3D point cloud comprises:

According to the density of the three-dimensional point cloud, the point cloud acquisition device is controlled to perform supplementary scanning on the target object to supplement the acquisition of the three-dimensional point cloud; or,

Acquire a pre-collected two-dimensional image corresponding to the pre-collected three-dimensional point cloud, and control the The point cloud acquisition device performs supplementary scanning on the target object to supplement the acquisition of the three-dimensional point cloud.
The method according to claim 1, wherein the movable platform comprises at least an unmanned aerial vehicle, an unmanned vehicle, an unmanned watercraft and a mobile robot.
A remote control terminal, comprising a processor, a memory for storing executable instructions of the processor, and a wireless communicator;

The wireless communicator is used to communicate and connect with the movable platform, and acquires in real time the two-dimensional image collected by the imaging device mounted on the movable platform and the three-dimensional point cloud collected by the point cloud collecting device mounted on the movable platform ; the acquisition ranges of the imaging device and the point cloud acquisition device at least partially overlap;

When the processor executes the executable instructions, it is used to: simultaneously display the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal to instruct the point cloud collection device Scanning result of the target object.
The remote control terminal according to claim 29, wherein the processor is further configured to: when the scanning result indicates that the target object is not scanned or the scanned target object is incomplete, according to the user's feedback on the remote control terminal operation to control the point cloud collection device to scan the target object.
The remote control terminal according to claim 30, wherein when controlling the point cloud collection device to scan the target object, the processor is specifically configured to: adjust the pose of the movable platform to The point cloud collection device is made to scan towards the target object.
The remote control terminal according to claim 30, wherein the imaging device and the point cloud collecting device are installed on the movable platform through a pan-tilt;

When controlling the point cloud collecting device to scan the target object, the processor is specifically configured to: adjust the pose of the movable platform and/or the pan/tilt, so that the point cloud collecting device Scan towards the target object.
The remote control terminal according to claim 30, wherein the user's operation on the remote control terminal at least comprises: the user's operation on the two-dimensional image and/or the three-dimensional image.
The remote control terminal according to claim 29, wherein the two-dimensional image and the three-dimensional image are displayed side by side or superimposed on the interactive interface.
The remote control terminal according to claim 29, wherein the two-dimensional image and the three-dimensional image are displayed on different layers, and the layers are displayed superimposed on the interactive interface.
The remote control terminal according to claim 29, wherein at least one point cloud recording control is further displayed on the interactive interface, and the point cloud recording control is used to instruct the imaging device and the point cloud collecting device collection status;

Before the real-time acquisition of the two-dimensional image collected by the imaging device and the three-dimensional point cloud collected by the point cloud collecting device, the processor is further configured to: record a control on the point cloud on the interactive interface in response to the user operation, controlling the imaging device to start collecting two-dimensional images and controlling the point cloud collecting device to start scanning the target object to collect three-dimensional point clouds.
The remote control terminal according to claim 36, wherein after the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the remote control terminal, the processor further It is used for: controlling the imaging device and the point cloud acquisition device to suspend acquisition, continue acquisition or end acquisition in response to the user's operation on the point cloud recording control.
The remote control terminal according to claim 36, wherein a setting control is further displayed on the interactive interface; the setting control is used to instruct the setting of the acquisition parameters of the point cloud acquisition device;

Before the controlling the imaging device to start collecting the two-dimensional image and the point cloud collecting device starting to scan the target object to collect the three-dimensional point cloud, the processor is further configured to:

In response to the user's operation of the setting control, a setting page is displayed on the interactive interface, and the setting page displays the setting information of the acquisition parameters of the point cloud acquisition device; and,

In response to the user's setting operation for the setting information on the setting page, acquiring the collection parameters set by the point cloud collection device;

According to the acquisition parameters set by the point cloud acquisition device, the point cloud acquisition device is controlled to start scanning the target object to acquire a three-dimensional point cloud.
The remote control terminal according to claim 38, wherein the acquisition parameters include at least one of the following: sampling frequency, scan mode, number of echoes, echo mode, whether with visible light pixels, rain and fog enhancement, and strong denoising mode .
The remote control terminal according to claim 29, wherein the interactive interface further displays a single-dual-screen switching control, and the single-dual-screen switching control is used to instruct the interactive interface to display only the three-dimensional image, or, Simultaneously displaying the two-dimensional image and the three-dimensional image;

When simultaneously displaying the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal, the processor is further configured to: in response to a user's operation on the single-dual-screen switching control, Only the three-dimensional image is displayed on the interactive interface.
The remote control terminal according to claim 29, wherein a two-dimensional image display control is further displayed on the interactive interface, and the two-dimensional image display control is used to instruct to display the two-dimensional image on the interactive interface , and display the collection range of the 3D point cloud on the 2D image;

When displaying the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud on the interactive interface of the remote control terminal, the processor is further configured to: in response to the user's operation on the two-dimensional image display control, in The two-dimensional image is displayed on the interactive interface, and the collection range of the three-dimensional point cloud is displayed on the two-dimensional image.
The remote control terminal according to claim 41, wherein the position of the collection range of the 3D point cloud in the 2D image is determined according to the external parameter conversion relationship between the imaging device and the point cloud collection device .
The remote control terminal according to claim 41, wherein the size of the collection range of the three-dimensional point cloud in the two-dimensional image is determined according to the focal length of the imaging device.
The remote control terminal according to claim 29, wherein the imaging device comprises at least one of the following: a visible light camera, a grayscale camera or an infrared camera;

The point cloud collection device includes at least one of the following: a lidar, a binocular vision sensor, and a structured light depth camera.
The remote control terminal according to claim 29, wherein the movable platform is installed with at least two PTZs, wherein one PTZ is installed with at least an imaging device and a point cloud acquisition device, and the other PTZ is installed with at least an imaging device ;

The two-dimensional image displayed on the interactive interface is the two-dimensional image collected by the imaging device of one of the pan/tilts, and the interactive interface also displays the second image of the two-dimensional image collected by the imaging device of the other pan/tilt. a thumbnail;

After the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the remote control terminal, the processor is further configured to:

In response to the user's operation on the first thumbnail, the two-dimensional image captured by the imaging device of one of the pan/tilts displayed on the interactive interface is switched to the two-dimensional image indicated by the first thumbnail.
The remote control terminal according to claim 45, wherein the processor is further configured to: in response to a user's operation on the first thumbnail, the two-dimensional image and/or the three-dimensional image, adjust the Describe the poses of at least two heads.
The remote control terminal according to claim 29, wherein the interactive interface further displays a second thumbnail image of the map where the PTZ is located;

After the two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are displayed on the interactive interface of the remote control terminal, the processor is further configured to:

In response to the user's operation on the second thumbnail, the map and the three-dimensional image are simultaneously displayed on the interactive interface.
The remote control terminal according to claim 47, wherein the map and the three-dimensional image are superimposed and displayed on the interactive interface.
The remote control terminal according to claim 29, wherein a rendering control is further displayed on the interactive interface; the rendering control is used to instruct to perform color rendering on the three-dimensional image;

The processor is also used to:

In response to the user's operation on the rendering control, a rendering page is displayed on the interactive interface, and the rendering page displays at least one rendering mode, and the rendering mode is used to indicate the color rendering mode of the three-dimensional image;

The three-dimensional image is color-rendered based on the rendering mode selected by the user on the rendering page.
The remote control terminal according to claim 49, wherein the rendering mode comprises at least one of the following: a reflectance rendering mode, a height rendering mode, and a two-dimensional image rendering mode;

Wherein, the reflectance rendering mode is used to indicate that the coloring is performed according to the reflectivity of each three-dimensional point in the three-dimensional point cloud;

The height rendering mode is used to indicate that coloring is performed according to the height information of each 3D point in the 3D point cloud;

The two-dimensional image rendering mode is used to instruct to generate the coloring information of the three-dimensional point cloud according to the pixel values of the pixels in the two-dimensional image.
The remote control terminal according to claim 29, wherein a measurement control is further displayed on the interactive interface; the measurement control is used for instructing to perform spatial measurement on the three-dimensional image;

The processor is also used to:

In response to the user's operation of the measurement control, displaying a measurement page on the interactive interface, where the three-dimensional image is displayed on the measurement page;

In response to a user's operation on the three-dimensional image, at least one measurement point is acquired, and distance information and/or angle information is acquired according to the at least one measurement point.
The remote control terminal according to claim 51, wherein when acquiring at least one measurement point, the processor is further configured to:

displaying the at least one measurement point on the interactive interface;

in response to user manipulation of the measurement point, displaying a moving marker near the measurement point; and,

In response to the user's operation on the moving marker, the distance of the location where the moving marker is located relative to the measurement point is determined.
The remote control terminal according to claim 29, wherein a playback control is further displayed on the interactive interface; the playback control is used to at least indicate a pre-collected 3D point cloud and/or a 2D image;

The processor is also used to:

In response to the user's operation of the playback control, displaying a playback list, where the playback list displays at least a pre-collected 3D point cloud and/or a 2D image;

In response to the user's operation on the three-dimensional point cloud and/or the two-dimensional image, the three-dimensional image and/or the two-dimensional image indicated by the three-dimensional point cloud is displayed on the interactive interface.
The remote control terminal according to claim 53, wherein when displaying the three-dimensional image indicated by the pre-collected three-dimensional point cloud, a supplementary recording control is further displayed on the interactive interface, and the supplementary recording control is used to indicate Supplementary collection of 3D point cloud;

The processor is further configured to: in response to the user's operation on the supplementary recording control, control the point cloud acquisition device to supplement the acquisition of the three-dimensional point cloud.
The remote control terminal according to claim 54, wherein when controlling the point cloud collection device to supplement the collection of three-dimensional point clouds, the processor is specifically configured to:

According to the density of the three-dimensional point cloud, the point cloud acquisition device is controlled to perform supplementary scanning on the target object to supplement the acquisition of the three-dimensional point cloud; or,

Acquire a pre-collected two-dimensional image corresponding to the pre-collected three-dimensional point cloud, and control the The point cloud acquisition device performs supplementary scanning on the target object to supplement the acquisition of the three-dimensional point cloud.
An imaging display device is characterized in that, it is applied to a remote control terminal, and the remote control terminal is used for communication connection with a movable platform, and the movable platform is installed with an imaging device and a point cloud collection device; the imaging device and The collection ranges of the point cloud collection devices at least partially overlap;

The apparatus includes one or more processors, operating individually or collectively, to perform the following operations:

acquiring the two-dimensional image collected by the imaging device and the three-dimensional point cloud collected by the point cloud collecting device in real time;

The two-dimensional image and the three-dimensional image indicated by the three-dimensional point cloud are simultaneously displayed on the interactive interface of the remote control terminal to instruct the point cloud collecting device to scan the scanning result of the target object.
An imaging display system is characterized in that it includes a remote control terminal and a movable platform, the remote control terminal is used for communication connection with a movable platform, and the movable platform is installed with an imaging device and a point cloud collection device; the imaging device The acquisition range of the device and the point cloud acquisition device at least partially overlap;

The movable platform is used to acquire the two-dimensional image collected by the imaging device and the three-dimensional point cloud collected by the point cloud collecting device in real time, and transmit them to the remote control terminal;

The remote control terminal is used to acquire the two-dimensional image and the three-dimensional point cloud transmitted by the movable platform in real time, and simultaneously display the two-dimensional image and the three-dimensional point cloud indication on the interactive interface of the remote control terminal The three-dimensional image to instruct the point cloud acquisition device to scan the scan result of the target object.
The system of claim 57, wherein the movable platform includes at least an unmanned aerial vehicle, an unmanned vehicle, an unmanned watercraft, and a mobile robot.
A computer-readable storage medium, characterized in that a computer program is stored thereon, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 28.