WO2020150975A1

WO2020150975A1 - Image display method, image control method, device, and movable platform

Info

Publication number: WO2020150975A1
Application number: PCT/CN2019/073028
Authority: WO
Inventors: 翁超; 周琦; 丘力
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-01-24
Filing date: 2019-01-24
Publication date: 2020-07-30
Also published as: CN111373743A

Abstract

Embodiments of the present invention provide an image display method, an image control method, a device, and a movable platform. The image display method comprises: sending a code stream transmission instruction to a photography device, wherein the photography device comprises at least two image sensors; images respectively captured by the at least two image sensors have one-to-one correspondence to at least two code streams; the code stream transmission instruction is used for instructing the photography device to send the transmission modes of the at least two code streams; receiving, according to the code stream transmission instruction, a display code stream sent by the photography device; and superimposing and displaying, according to the display code stream, on an image display device, the images respectively captured by the at least two image sensors. The code stream transmission instruction clearly instructs the transmission modes of a plurality of code streams, and reduces the complexity of transmission of a plurality of code streams between the photography device and the image display device, and the display mode in picture-in-picture, so that the transmission and display of the code streams are more flexible.

Description

Image display method, image control method, equipment and movable platform

Technical field

The present invention relates to the technical field of image processing, in particular to an image display method, an image control method, equipment and a movable platform.

Background technique

With the rapid development of UAVs, the application of UAVs has become more and more extensive. The drone can be equipped with a camera. When the drone is flying, the camera can collect images at various angles and attitudes. The image collected by the camera is transmitted to the ground control terminal through the drone, and the image can be displayed on the ground control terminal.

Generally, cameras are classified into visible light cameras and infrared cameras. The visible light camera can collect images in places with better lighting conditions. Infrared cameras can collect images at night or in places with weak lighting conditions. With the development of cameras, two lenses can be set on the camera for visible light shooting and infrared shooting.

Since the camera can obtain different code streams through different lenses, the control and display of the image between the camera and the ground control terminal are more complicated, and it is urgent to solve.

Summary of the invention

The invention provides an image display method, an image control method, equipment and a movable platform, which simplifies the complexity of the transmission of multiple code streams and the picture-in-picture display mode between the shooting equipment and the image display equipment.

In a first aspect, the present invention provides an image display method, including:

Send a code stream transmission instruction to the shooting device; wherein, the shooting device includes at least two image sensors, and the pictures taken by the at least two image sensors correspond to at least two code streams one-to-one; the code stream transmission instruction is used for Instruct the shooting device to send the at least two code streams in a sending manner;

Receiving the display code stream sent by the shooting device according to the code stream transmission instruction;

The images taken by the at least two image sensors are superimposed and displayed on the image display device according to the display code stream.

In a second aspect, the present invention provides an image control method, which is applied to a photographing device, the photographing device includes at least two image sensors, and the pictures respectively captured by the at least two image sensors correspond one-to-one with at least two code streams; The method includes:

Receiving a code stream transmission instruction sent by an image display device; the code stream transmission instruction is used to instruct the shooting device to send the transmission mode of the at least two code streams;

Sending a display code stream to the image display device according to the code stream transmission instruction.

In a third aspect, the present invention provides an image display device, including: a memory, a processor, a communication interface, and a display;

The display is used to display images taken by a photographing device;

The memory is used to store program code;

The processor calls the program code, and when the program code is executed, is used to perform the following operations:

Controlling the communication interface to send a code stream transmission instruction to a photographing device; wherein the photographing device includes at least two image sensors, and the pictures respectively taken by the at least two image sensors correspond to at least two code streams in a one-to-one correspondence; The code stream transmission instruction is used to instruct the shooting device to send the transmission mode of the at least two code streams;

Controlling the communication interface to receive the display code stream sent by the shooting device according to the code stream transmission instruction;

The pictures taken by the at least two image sensors are superimposed and displayed on the display according to the display code stream.

In a fourth aspect, the present invention provides a photographing device, including: a memory, a processor, a communication interface, and at least two image sensors, where the pictures respectively taken by the at least two image sensors correspond to at least two code streams in a one-to-one correspondence;

The memory is used to store program code;

Controlling the communication interface to receive the code stream transmission instruction sent by the image display device; the code stream transmission instruction is used to instruct the photographing device to send the transmission mode of the at least two code streams;

Controlling the communication interface to send a display code stream to the image display device according to the code stream transmission instruction.

In a fifth aspect, the present invention provides a movable platform including: a device body and the photographing device as provided in any embodiment of the fourth aspect loaded on the device body.

In a sixth aspect, the present invention provides a storage medium, including: a readable storage medium and a computer program, the computer program is used to implement the image display method provided in any embodiment of the first aspect or any embodiment provided in the second aspect Image control method.

In a seventh aspect, the present invention provides a program product. The program product includes a computer program (that is, an execution instruction), and the computer program is stored in a readable storage medium. The processor may read the computer program from a readable storage medium, and the processor executes the computer program to implement the image display method provided by any embodiment of the first aspect or the image control method provided by any embodiment of the second aspect.

The invention provides an image display method, an image control method, equipment and a movable platform. Send a code stream transmission instruction to the shooting device through the image display device, receive the display code stream sent by the shooting device according to the code stream transmission instruction, and superimpose and display the pictures taken by at least two image sensors on the image display device according to the display code stream, simplifying The complexity of the transmission of multiple streams and the picture-in-picture display between the shooting device and the image display device makes the transmission and display of the stream more flexible.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

Figure 1 is a system architecture diagram of the mobile platform system involved in the present invention;

FIG. 2A is a schematic diagram of communication within the mobile platform system involved in the present invention;

FIG. 2B is another schematic diagram of communication inside the mobile platform system involved in the present invention;

3 is a flowchart of an image display method provided by Embodiment 1 of the present invention;

4A to 4C are schematic diagrams of superimposed and displayed interfaces in the image display method provided by Embodiment 1 of the present invention;

FIG. 5 is a flowchart of an image control method provided by Embodiment 1 of the present invention;

6 is a schematic structural diagram of an image display device provided by Embodiment 1 of the present invention;

FIG. 7 is a schematic structural diagram of a photographing device provided in Embodiment 1 of the present invention.

detailed description

The invention provides an image display method, an image control method, equipment and a movable platform, which can be applied to a movable platform system. Fig. 1 is a system architecture diagram of the mobile platform system involved in the present invention. As shown in FIG. 1, the movable platform system may include: a movable platform 11, a photographing device 12 and an image display device 13. Among them, the photographing equipment 12 is mounted on the movable platform 11. An application program can be installed on the image display device 13, through which the movable platform 11 can be controlled, and related information of the movable platform 11 and/or the photographing device 13 can be displayed. For example, the image display device 13 may display an image photographed by the photographing device 13. Optionally, the movable platform system may further include a control device 14. The control device 14 can be used to control the movable platform 11. It should be noted that the present invention does not limit the implementation of the movable platform 11, the photographing device 12, the image display device 13, and the control device 14. Optionally, the movable platform 11 may be an unmanned aerial vehicle, an unmanned car, or the like. The drone may be a rotorcraft, for example, a multi-rotor aircraft propelled by multiple propulsion devices through the air. The photographing device 12 may be a photographing device, a photographing device, or the like. The image display device 13 may be a smart phone, a tablet computer, or the like. The control device 14 may be a rocker remote controller or the like.

Optionally, FIG. 2A is a schematic diagram of communication within the mobile platform system involved in the present invention. As shown in FIG. 1 and FIG. 2A, the mobile platform 11 and the photographing device 12 may communicate based on a communication protocol in a wireless or wired manner. The communication protocol may be an internal communication protocol of the movable platform 11, or other existing communication protocols, or a customized communication protocol between the movable platform 11 and the photographing device 12. The mobile platform 11 and the image display device 13 can communicate wirelessly through a wireless network. The wireless network may be a wireless ad hoc network, such as a wireless local area network (Wireless Fidelity, WIFI). The wireless network may also be a public wireless communication network.

Optionally, FIG. 2B is another schematic diagram of communication within the mobile platform system involved in the present invention. As shown in FIG. 1 and FIG. 2B, the mobile platform 11 and the photographing device 12 may communicate based on a communication protocol in a wireless or wired manner. For the communication mode, refer to the description of FIG. 2A, which will not be repeated here. The movable platform 11 and the image display device 13 can also communicate with each other through the control device 14. Optionally, the control device 14 and the image display device 13 may be wirelessly connected, that is, wireless communication is performed through a wireless network. Optionally, the control device 14 and the image display device 13 may be connected in a wired manner, for example, through a universal serial bus (Universal Serial Bus, USB) interface. The mobile platform 11 and the control device 14 can communicate wirelessly through a wireless network.

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

Exemplarily, in each embodiment of the present invention, the movable platform 11 may be a drone, and the photographing device 12 may be a camera.

FIG. 3 is a flowchart of an image display method provided by Embodiment 1 of the present invention. In the image display method provided in this embodiment, the execution subject may be an image display device. The image display device is used to display the picture obtained by the photographing device. As shown in FIG. 3, the image display method provided in this embodiment may include:

S301: Send a code stream transmission instruction to the shooting device.

Wherein, the photographing device includes at least two image sensors, and the pictures respectively taken by the at least two image sensors correspond to at least two code streams in a one-to-one correspondence. The code stream transmission instruction is used to instruct the shooting device to send at least two code streams.

Specifically, the photographing device includes at least two image sensors. This embodiment does not limit the number of image sensors, implementation manners, and whether they are arranged in one lens. For example, the number of image sensors is two. The two image sensors include visible light image sensor and infrared image sensor. Or, both image sensors are visible light image sensors. The two image sensors may be set in the same lens of the shooting device, or the two image sensors may be set in different lenses of the shooting device. For each image sensor, the picture taken by the image sensor corresponds to a code stream. Through the code stream transmission instruction, you can clearly instruct the shooting device to send at least two code streams.

The following is an example to describe the sending mode of at least two code streams.

Assume that the camera includes 3 image sensors, 2 visible light image sensors and 1 infrared image sensor respectively.

Optionally, in one transmission mode, three code streams corresponding to three image sensors can be respectively transmitted.

Optionally, in another transmission mode, the three code streams corresponding to the three image sensors can be fused in a picture-in-picture mode first to form a fused code stream. At this time, the fused code stream needs to be transmitted. The so-called "picture in picture (PIP)" is a way of presenting video content, which means that while a video is played in full screen, another video is simultaneously played on a small area of the playing screen.

Optionally, in another transmission mode, the two code streams corresponding to the two visible light image sensors can be fused in a picture-in-picture mode first to form a fused code stream. At this time, it is necessary to transmit the fused code stream and the code stream corresponding to the infrared image sensor.

Optionally, in another transmission mode, the code stream corresponding to one visible light image sensor and the code stream corresponding to the infrared image sensor may be fused in a picture-in-picture mode first to form a fused code stream. At this time, it is necessary to transmit the fused code stream and the code stream corresponding to the remaining visible light image sensor.

Since there are multiple ways to achieve the transmission of at least two code streams, by sending a code stream transmission instruction to the shooting device, the shooting device can be clearly instructed to send at least two code streams, which simplifies the shooting device and the image display device. The complexity of the transmission and display of multiple streams.

S302: Receive the display code stream sent by the shooting device according to the code stream transmission instruction.

S303. Superimpose and display the pictures taken by at least two image sensors on the image display device according to the display code stream.

Specifically, the shooting device sends the display code stream to the image display device according to the code stream transmission instruction and the code streams respectively corresponding to the multiple image sensors. Correspondingly, the image display device receives the display code stream, and realizes superimposed display of the pictures taken by multiple image sensors.

The following describes the superimposed display of the pictures taken by at least two image sensors through an example.

Assume that the camera includes 3 image sensors, 2 visible light image sensors and 1 infrared image sensor respectively. The pictures taken by the two visible light image sensors are called picture 1 and picture 2 respectively. The picture taken by the infrared image sensor is called picture 3.

Optionally, in one example, see Figure 4A. 4A is a schematic diagram of a superimposed display interface in the image display method provided by Embodiment 1 of the present invention. The screen 1, the screen 2 and the screen 3 are respectively displayed through a display window, and the three display windows are correspondingly called the display window 1, the display window 2 and the display window 3. Among them, the display size of the display window 1 is the largest. Display window 2 is displayed superimposed on the lower left corner of display window 1. The display window 3 is displayed superimposed on the lower right corner of the display window 1. The display window 2 and the display window 3 have no overlapping area.

Optionally, in another example, see Figure 4B. 4B is a schematic diagram of a superimposed display interface in the image display method provided by Embodiment 1 of the present invention. Screen 1, Screen 2, and Screen 3 are displayed through the same display window, which is called display window 1. Among them, display window 1 displays picture 1 in full screen. Picture 2 is superimposed and displayed in the lower left corner of Picture 1. Picture 3 is superimposed and displayed in the lower right corner of Picture 1. Screen 2 and Screen 3 have no overlapping area.

Optionally, in yet another example, see Figure 4C. 4C is a schematic diagram of a superimposed display interface in the image display method provided by Embodiment 1 of the present invention. Screen 1, Screen 2, and Screen 3 are displayed through two display windows, which are called display window 1 and display window 2, respectively. Among them, the display size of the display window 1 is the largest. Display window 2 is superimposed and displayed below window 1. Display window 1 displays screen 1. The screen 2 is displayed on the left side of the display window 2, and the screen 3 is displayed on the right side of the display window 2. Screen 2 and Screen 3 have no overlapping area.

It should be noted that this embodiment does not limit the number of display windows, the display size and display position of each display window. For a scene where a display window displays multiple pictures, this embodiment does not limit the overlay relationship between the pictures. For a scene in which there are multiple display windows, this embodiment does not limit the coverage relationship between the display windows.

It should be noted that the display size and display position of each display window, the coverage relationship between multiple screens in a display window, and the coverage relationship between multiple display windows can be used by the user through the image display device or the control device Set up.

It can be seen that the image display method provided in this embodiment is applied to a scene in which a photographing device includes multiple image sensors, and the image display device superimposes and displays pictures taken by the multiple image sensors. The shooting device includes a plurality of image sensors, and the code stream corresponding to each image sensor needs to be transmitted to the image display device. The image display device sends a code stream transmission instruction to the shooting device, which clearly indicates the sending mode of multiple code streams. Therefore, the image display device can receive the display code stream sent by the photographing device according to the code stream transmission instruction, so as to achieve superimposed display of the pictures taken by the multiple image sensors. The image display method provided in this embodiment simplifies the complexity of the transmission of multiple streams and the picture-in-picture display mode between the shooting device and the image display device, and makes the transmission and display of the stream more flexible.

Optionally, in the application scenario shown in FIG. 2A, instructions and code streams may be transmitted between the image display device and the shooting device through a movable platform. Optionally, in the application scenario shown in FIG. 2B, the image display device is connected to the control device. Commands and code streams can be transmitted between the image display device and the shooting device through the movable platform and the control device.

Wherein, the instruction includes any instruction transmitted between the image display device and the shooting device, for example, a code stream transmission instruction. The code stream includes any code stream transmitted between the image display device and the shooting device, for example, the display code stream.

Optionally, in this embodiment, the code stream transmission instruction may be input by the user. Optionally, in the application scenario shown in FIG. 2A, the code stream transmission instruction may be input by the user through the image display device. For example, the user inputs a code stream transmission instruction by operating a control button set on the image display device or performing a touch operation on the touch screen. This embodiment does not limit the touch operation, for example, a click operation, a sliding operation, etc. are performed on the touch display screen. Optionally, in the application scenario shown in FIG. 2B, the code stream transmission instruction may be input by the user through the image display device or the control device.

Optionally, in S303, before superimposing and displaying pictures taken by at least two image sensors on the image display device according to the display code stream, it may further include:

Decode the display code stream.

Specifically, the shooting device captures the code stream. The code stream refers to stream media data after encoding. The shooting device sends a display code stream to the image display device. Correspondingly, after receiving the display code stream, the image display device needs to decode the display code stream.

Optionally, in the application scenario shown in FIG. 2A, the display code stream is decoded by the image display device. Optionally, in the application scenario shown in FIG. 2B, the display code stream is decoded through an image display device or a control device connected to the image display device.

This embodiment provides an image display method, including: sending a code stream transmission instruction to a shooting device, receiving a display code stream sent by the shooting device according to the code stream transmission instruction, and superimposing and displaying at least two images on the image display device according to the display code stream Images taken by the sensors separately. The image display method provided in this embodiment simplifies the complexity of the transmission of multiple streams and the picture-in-picture display mode between the shooting device and the image display device, and makes the transmission and display of the stream more flexible.

In the following, on the basis of the first embodiment shown in FIG. 3, different implementations of the code stream transmission instruction will be described in detail.

The second embodiment of the present invention provides an image display method. In this embodiment, the code stream transmission instruction is used to instruct the shooting device to send each of the at least two code streams separately. The display code stream includes at least two code streams.

The following is an example to illustrate. Assume that the camera includes two image sensors, namely a visible light image sensor and an infrared image sensor. The visible light image sensor corresponds to code stream 1, and the infrared image sensor corresponds to code stream 2. The shooting device sends code stream 1 and code stream 2 to the image display device respectively. At this time, the display code stream includes code stream 1 and code stream 2. Correspondingly, the image display device receives code stream 1 and code stream 2 sent by the shooting device.

By sending at least two code streams corresponding to at least two image sensors respectively by the shooting device, the processing complexity of the code stream of the shooting device is reduced, and the hardware cost of the shooting device is reduced.

Correspondingly, in S303, superimposing and displaying pictures taken by at least two image sensors on the image display device according to the display code stream may include:

The pictures taken by the at least two image sensors are superimposed and displayed according to the at least two code streams.

Optionally, superimposing and displaying pictures taken by at least two image sensors according to at least two code streams may include:

Get the window mode. The window mode is used to indicate to superimpose and display the pictures taken by at least two image sensors in one display window, or to display pictures taken by at least two image sensors in at least two superimposed display windows.

According to the window mode, superimpose and display according to at least two streams.

Specifically, the window mode is used to indicate that the images taken by at least two image sensors are superimposed and displayed in one display window, as shown in FIG. 4B. The window mode is used to indicate that pictures taken by at least two image sensors are respectively displayed in at least two superimposed display windows, as shown in FIG. 4A. The principle is similar and will not be repeated here.

It should be noted that this embodiment does not limit the implementation manner of obtaining the window mode. Optionally, the window mode preset in the image display device may be obtained, or the window mode input by the user through the image display device or the control device may be obtained.

Hereinafter, according to the number of display windows, the image display device will superimpose and display the pictures respectively taken by at least two image sensors.

Optionally, in one implementation manner, there is one display window.

Correspondingly, superimposing and displaying images captured by at least two image sensors according to at least two code streams may include:

At least two streams are merged to generate a picture-in-picture stream.

According to the picture-in-picture code stream, the pictures taken by at least two image sensors are superimposed and displayed in the display window.

In this implementation manner, the image display device performs fusion processing on at least two code streams to generate a picture-in-picture code stream, also known as a PIP code stream. Therefore, according to the PIP code stream, the pictures taken by the multiple image sensors included in the photographing device can be superimposed and displayed in one display window. Among them, before fusing at least two code streams, each code stream needs to be decoded. Optionally, each bit stream can be decoded through an image display device or a control device.

By merging multiple streams, the picture-in-picture mode display is realized in one display window, which reduces the number of display windows and is convenient for users to view.

Optionally, fusing at least two code streams to generate a picture-in-picture code stream may include:

Get the first display mode. Wherein, the first display mode is used to indicate the display size, display position, and coverage relationship of the pictures respectively taken by the at least two image sensors in the display window.

At least two code streams are merged according to the first display mode to generate a picture-in-picture code stream.

Optionally, the first display mode is input by the user. Optionally, the first display mode may be input by the user through the image display device or the control device.

Optionally, the first display mode may be preset in the image display device.

Optionally, in another implementation manner, there are at least two display windows.

Acquire the second display mode. Wherein, the second display mode is used to indicate the corresponding relationship between the images captured by the at least two image sensors and the at least two display windows, and the display size, display position, and coverage relationship of the at least two display windows.

According to the second display mode and the at least two code streams, the pictures taken by the at least two image sensors are respectively displayed in the at least two display windows.

In this implementation manner, the number of display windows is at least two. According to the second display mode, the image display device realizes the display in the picture-in-picture mode through multiple display windows. Wherein, the corresponding relationship between the pictures taken by the at least two image sensors and the at least two display windows indicates that each display window displays the picture taken by which image sensor in the photographing device. Among them, each display window needs to decode the code stream corresponding to the image sensor before displaying the picture taken by the image sensor. Optionally, the code stream can be decoded by an image display device or a control device.

By separately displaying the code streams corresponding to multiple image sensors on the shooting device in multiple display windows, the flexibility of the picture-in-picture mode display is improved.

Optionally, the second display mode is input by the user. Optionally, the second display mode may be input by the user through the image display device or the control device.

Optionally, the second display mode may be preset in the image display device.

Optionally, the image display method provided in this embodiment may further include:

Get the movement instruction entered by the user. Wherein, the movement instruction is used to instruct to move the display position of the first display window. The first display window is a non-maximized window.

Move the display position of the first display window according to the move instruction.

The following is an example to illustrate. As shown in Fig. 4A, the display window 1 is a maximized window. Display window 2 and display window 3 are non-maximized windows. The movement instruction input by the user can be used to instruct to move the display window 2 from the lower left corner to the upper right corner.

It should be noted that this embodiment does not limit the implementation manner for the user to input the movement instruction. For example, the user can move on the display screen by dragging the display window to input a movement instruction.

Get the exchange instruction entered by the user. Wherein, the exchange instruction is used to instruct to exchange the screens respectively displayed in the second display window and the third display window. The second display window is a non-maximized window, and the third display window is a maximized window.

The screens respectively displayed on the second display window and the third display window are exchanged and displayed according to the exchange instruction.

The following is an example to illustrate. As shown in Fig. 4A, the display window 1 is a maximized window. Display window 2 and display window 3 are non-maximized windows. The exchange instruction entered by the user can be used to instruct to exchange the screens respectively displayed in the display window 3 and the display window 1. After the interchange display, display window 1 displays screen 3, and display window 3 displays screen 1.

It should be noted that this embodiment does not limit the implementation manner for the user to input the exchange instruction.

Optionally, obtaining the exchange instruction input by the user may include:

The touch operation performed by the user on the second display window is detected.

If the touch operation is a touch operation corresponding to the exchange instruction, the exchange instruction is generated.

This embodiment provides an image display method. In this embodiment, the code stream transmission instruction is used to instruct the shooting device to send each code stream of at least two code streams, and the display code stream includes at least two code streams. The image display device may superimpose and display the pictures taken by the at least two image sensors according to the at least two code streams. The image display method provided in this embodiment simplifies the complexity of the transmission of multiple streams and the picture-in-picture display mode between the shooting device and the image display device, and makes the transmission and display of the stream more flexible.

The third embodiment of the present invention provides an image display method. In this embodiment, the code stream transmission instruction is used to instruct the shooting device to merge at least two code streams to generate a picture-in-picture code stream, and send the picture-in-picture code stream. Display code stream includes picture-in-picture code stream.

The following is an example to illustrate. Assume that the camera includes two image sensors, namely a visible light image sensor and an infrared image sensor. The visible light image sensor corresponds to code stream 1, and the infrared image sensor corresponds to code stream 2. The shooting device merges code stream 1 and code stream 2 to generate a picture-in-picture code stream. At this time, the display code stream includes the picture-in-picture code stream. Correspondingly, the image display device receives the picture-in-picture code stream sent by the photographing device.

The picture-in-picture code stream generated by the fusion of at least two code streams is sent by the shooting device, which reduces the processing complexity of the code stream by the image display device and the complexity of implementing the picture-in-picture display, and reduces the hardware cost of the image display device.

According to the picture-in-picture code stream, the pictures taken by at least two image sensors are displayed in the display window.

Specifically, the number of display windows is one, as shown in Figure 4B. The principle is similar and will not be repeated here. Among them, each bit stream needs to be decoded before the pictures taken by at least two image sensors are displayed in the display window. Optionally, each bit stream can be decoded through an image display device or a control device.

Send the third display mode to the shooting device. Wherein, the third display mode is used to indicate the display size, display position, and coverage relationship in the display window of the pictures respectively captured by the at least two image sensors, so that the shooting device merges at least two streams to generate a picture-in-picture stream.

It should be noted that the third display mode may be preset in the image display device, or input by the user through the image display device or the control device.

This embodiment provides an image display method. In this embodiment, the code stream transmission instruction is used to instruct the shooting device to merge at least two code streams to generate a picture-in-picture code stream, and the display code stream includes the picture-in-picture code stream. The image display device can superimpose and display the pictures taken by at least two image sensors according to the picture-in-picture code stream. The image display method provided in this embodiment simplifies the complexity of the transmission and picture-in-picture display of multiple streams between the shooting device and the image display device, and makes the transmission and display of the stream more flexible.

FIG. 5 is a flowchart of an image control method provided by Embodiment 1 of the present invention. In the image control method provided in this embodiment, the execution subject may be a photographing device. Wherein, the photographing device includes at least two image sensors, and the pictures respectively taken by the at least two image sensors correspond to at least two code streams in a one-to-one correspondence. As shown in FIG. 5, the image control method provided in this embodiment may include:

S501: Receive a code stream transmission instruction sent by the image display device.

Wherein, the code stream transmission instruction is used to instruct the shooting device to send at least two code streams.

S502: Send a display code stream to the image display device according to the code stream transmission instruction.

Among them, the code stream transmission instruction can refer to the image display method provided in the embodiments shown in FIG. 3 and FIG. 4A to FIG.

The image control method provided in this embodiment is applied to a scene in which a photographing device includes multiple image sensors, and the image display device superimposes and displays pictures taken by the multiple image sensors. The shooting device includes a plurality of image sensors, and the code stream corresponding to each image sensor needs to be transmitted to the image display device. The shooting device receives the code stream transmission instruction sent by the image display device, which clearly indicates the sending mode of multiple code streams. Therefore, the photographing device can send the display code stream to the image display device according to the code stream transmission instruction, so that the image display device can superimpose and display the pictures respectively taken by the multiple image sensors. The image control method provided in this embodiment simplifies the complexity of the transmission and picture-in-picture display modes of multiple streams between the shooting device and the image display device, and makes the transmission and display of the stream more flexible.

Optionally, the code stream transmission instruction is used to instruct the shooting device to send each of the at least two code streams separately.

Correspondingly, in S502, sending the display code stream to the image display device according to the code stream transmission instruction may include:

Send at least two code streams to the image display device.

Optionally, the shooting device is mounted on a movable platform, and there are at least two communication channels between the shooting device and the movable platform.

Correspondingly, sending at least two code streams to the image display device may include:

At least two code streams are respectively sent to the movable platform through at least two communication channels, so that the movable platform sends at least two code streams to the image display device.

The following is an example to illustrate.

Assume that the camera includes 3 image sensors, 2 visible light image sensors and 1 infrared image sensor respectively. The code streams corresponding to the two visible light image sensors are code stream 1 and code stream 2. The code stream corresponding to the infrared image sensor is code stream 3. Three communication channels have been established between the camera and the mobile platform, which are called communication channel 1 to communication channel 3. Then, through communication channel 1, the shooting device sends code stream 1 to the movable platform. Through communication channel 2, the shooting device sends code stream 2 to the movable platform. Through communication channel 3, the shooting device sends code stream 3 to the movable platform.

It should be noted that this embodiment does not limit how to establish a communication channel between the shooting device and the movable platform, and the existing communication protocol can be followed.

Optionally, the code stream transmission instruction is used to instruct the shooting device to merge at least two code streams to generate a picture-in-picture code stream, and send the picture-in-picture code stream.

At least two streams are merged to generate a picture-in-picture stream.

Send the picture-in-picture code stream to the image display device.

Optionally, the image control method provided in this embodiment may further include:

Receive the display mode sent by the image display device. Wherein, the display mode is used to indicate the display size, display position, and coverage relationship of the pictures respectively taken by the at least two image sensors in the display window of the image display device.

Correspondingly, fusing at least two code streams to generate a picture-in-picture code stream may include:

According to the display mode, at least two code streams are merged to generate a picture-in-picture code stream.

For the display mode in this embodiment, please refer to the description of the third display mode in the image display method provided in the third embodiment above, and the principle is similar, and will not be repeated here.

This embodiment provides an image control method, including: receiving a code stream transmission instruction sent by an image display device, and sending a display code stream to the image display device according to the code stream transmission instruction. The image control method provided in this embodiment simplifies the complexity of the transmission and picture-in-picture display modes of multiple streams between the shooting device and the image display device, and makes the transmission and display of the stream more flexible.

FIG. 6 is a schematic structural diagram of an image display device provided by Embodiment 1 of the present invention. The image display device provided in this embodiment is used to execute the image display method provided in the foregoing embodiment of the present invention. As shown in FIG. 6, the image display device provided in this embodiment may include: a memory 62, a processor 61, a communication interface 63, and a display 64.

The display 64 is used to display images taken by the photographing device.

The memory 62 is used to store program codes.

The processor 61 calls the program code, and when the program code is executed, it is used to perform the following operations:

The control communication interface 63 sends a code stream transmission instruction to the shooting device. Wherein, the photographing device includes at least two image sensors, and the pictures respectively taken by the at least two image sensors correspond to at least two code streams in a one-to-one correspondence. The code stream transmission instruction is used to instruct the shooting device to send at least two code streams.

The control communication interface 63 receives the display code stream sent by the shooting device according to the code stream transmission instruction.

The pictures taken by at least two image sensors are superimposed and displayed on the display 64 according to the display code stream.

Optionally, the code stream transmission instruction is used to instruct the shooting device to send each of the at least two code streams separately. The display code stream includes at least two code streams.

The processor 61 is specifically used for:

Optionally, the processor 61 is specifically configured to:

Get the window mode. The window mode is used to indicate to superimpose and display the pictures respectively taken by at least two image sensors in one display window, or to display pictures respectively taken by at least two image sensors in at least two superimposed display windows.

According to the window mode, the pictures taken by at least two image sensors are superimposed and displayed according to at least two code streams.

Optionally, there is one display window, and the processor 61 is specifically configured to:

At least two streams are merged to generate a picture-in-picture stream.

Optionally, the processor 61 is specifically configured to:

Get the first display mode. The first display mode is used to indicate the display size, display position, and coverage relationship of the pictures respectively taken by the at least two image sensors in the display window.

Optionally, the first display mode is input by the user.

Optionally, there are at least two display windows, and the processor 61 is specifically configured to:

Acquire the second display mode. The second display mode is used to indicate the correspondence between the pictures respectively captured by the at least two image sensors and the at least two display windows, and the display size, display position, and coverage relationship of the at least two display windows.

Optionally, the second display mode is input by the user.

Optionally, the processor 61 is also used to:

Get the movement instruction entered by the user. The movement instruction is used to instruct to move the display position of the first display window. The first display window is a non-maximized window.

Optionally, the processor 61 is also used to:

Get the exchange instruction entered by the user. The exchange instruction is used to instruct to exchange the screens respectively displayed in the second display window and the third display window. The second display window is a non-maximized window, and the third display window is a maximized window.

Optionally, the processor 61 is specifically configured to:

Optionally, the code stream transmission instruction is used to instruct the shooting device to merge at least two code streams to generate a picture-in-picture code stream, and send the picture-in-picture code stream. Display code stream includes picture-in-picture code stream.

The processor 61 is specifically used for:

Optionally, the processor 61 is also used to:

The control communication interface 63 transmits the third display mode to the photographing device. Wherein, the third display mode is used to indicate the display size, display position, and coverage relationship in the display window of the pictures respectively captured by the at least two image sensors, so that the shooting device merges at least two streams to generate a picture-in-picture stream.

Optionally, the processor 61 is also used to:

Decode the display code stream.

Optionally, the processor 61 is specifically configured to:

The display code stream is decoded through the image display device or the control device connected to the image display device.

Optionally, the code stream transmission instruction is input by the user.

Optionally, the image display device is connected to the control device, and instructions and code streams are transmitted between the image display device and the shooting device through the control device.

The image display device provided in this embodiment is used to implement the image display method provided in the foregoing embodiment of the present invention. The technical principles and technical effects are similar, and will not be repeated here.

FIG. 7 is a schematic structural diagram of a photographing device provided in Embodiment 1 of the present invention. The photographing device provided in this embodiment is used to execute the image control method provided in the foregoing embodiment of the present invention. As shown in FIG. 7, the photographing device provided in this embodiment may include: a memory 72, a processor 71, a communication interface 73, and at least two image sensors 74. The images captured by the at least two image sensors 74 and at least two codes The streams correspond one to one.

The memory 72 is used to store program codes.

The processor 71 calls the program code, and when the program code is executed, it is used to perform the following operations:

The control communication interface 73 receives the code stream transmission instruction sent by the image display device. The code stream transmission instruction is used to instruct the shooting device to send at least two code streams.

The control communication interface 73 sends the display code stream to the image display device according to the code stream transmission instruction.

The processor 71 is specifically used for:

The control communication interface 73 sends at least two code streams to the image display device.

Optionally, the shooting device is mounted on a movable platform, and there are at least two communication channels between the shooting device and the movable platform. The processor 71 is specifically used for:

The control communication interface 73 sends at least two code streams to the movable platform through at least two communication channels, so that the movable platform sends at least two code streams to the image display device.

The processor 71 is specifically used for:

At least two streams are merged to generate a picture-in-picture stream.

The control communication interface 73 sends a picture-in-picture code stream to the image display device.

Optionally, the processor 71 is also used to:

The control communication interface 73 receives the display mode sent by the image display device. The display mode is used to indicate the display size, display position, and coverage relationship of the pictures respectively taken by the at least two image sensors 74 in the display window of the image display device.

The processor 71 is specifically used for:

The photographing device provided in this embodiment is used to execute the image control method provided in the foregoing embodiment of the present invention. The technical principles and technical effects are similar, and will not be repeated here.

The embodiment of the present invention also provides a movable platform, including: a device body and the photographing device provided by the embodiment of the present invention loaded on the device body.

It should be noted that this embodiment does not limit the implementation of the device body. For example, the device body may be a drone or an unmanned vehicle.

It should be noted that this embodiment does not limit the loading method of the photographing device on the device body. For example, a pan-tilt is provided on the device body. The shooting device is detachably connected to the pan-tilt.

A person of ordinary skill in the art can understand that all or part of the steps in the foregoing method embodiments can be implemented by a program instructing relevant hardware. The aforementioned program can be stored in a computer readable storage medium. When the program is executed, it executes the steps including the foregoing method embodiments; and the foregoing storage medium includes: ROM, RAM, magnetic disk, or optical disk and other media that can store program codes.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, but not to limit them; although the embodiments of the present invention are described in detail with reference to the foregoing embodiments, those of ordinary skill in the art It should be understood that: it is still possible to modify the technical solutions described in the foregoing embodiments, or equivalently replace some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the embodiments of the present invention The scope of the technical solution.

Claims

An image display method, characterized by comprising:

Send a code stream transmission instruction to the shooting device; wherein, the shooting device includes at least two image sensors, and the pictures taken by the at least two image sensors correspond to at least two code streams one-to-one; the code stream transmission instruction is used for Instruct the shooting device to send the at least two code streams in a sending manner;

Receiving the display code stream sent by the shooting device according to the code stream transmission instruction;

The images taken by the at least two image sensors are superimposed and displayed on the image display device according to the display code stream.
The method according to claim 1, wherein the code stream transmission instruction is used to instruct the shooting device to send each code stream of the at least two code streams; the display code stream includes the At least two streams;

The superimposing and displaying the pictures taken by the at least two image sensors on an image display device according to the display code stream includes:

The pictures taken by the at least two image sensors are superimposed and displayed according to the at least two code streams.
The method according to claim 2, wherein the superimposing and displaying the pictures taken by the at least two image sensors according to the at least two code streams comprises:

Acquiring a window mode; the window mode is used to instruct to superimpose and display the pictures taken by the at least two image sensors in one display window, or to display the at least two image sensors in at least two superimposed display windows, respectively Images taken separately;

According to the window mode, the pictures taken by the at least two image sensors are superimposed and displayed according to the at least two code streams.
The method according to claim 2 or 3, wherein there is one display window, and the superimposing and displaying the pictures taken by the at least two image sensors according to the at least two code streams comprises:

Fusing the at least two code streams to generate a picture-in-picture code stream;

According to the picture-in-picture code stream, the pictures taken by the at least two image sensors are superimposed and displayed in the display window.
The method according to claim 4, wherein the fusing the at least two code streams to generate a picture-in-picture code stream comprises:

Acquiring a first display mode; the first display mode is used to indicate the display size, display position, and coverage relationship of the pictures respectively taken by the at least two image sensors in the display window;

The at least two code streams are merged according to the first display mode to generate the picture-in-picture code stream.
The method of claim 5, wherein the first display mode is input by a user.
The method according to claim 2 or 3, wherein there are at least two display windows, and the superimposing and displaying the pictures taken by the at least two image sensors according to the at least two code streams comprises:

Acquire a second display mode; the second display mode is used to indicate the correspondence between the pictures taken by the at least two image sensors and the at least two display windows, and the display of the at least two display windows Size, display position and coverage relationship;

According to the second display mode and the at least two code streams, the pictures taken by the at least two image sensors are respectively displayed in the at least two display windows.
The method according to claim 7, wherein the second display mode is input by a user.
The method according to claim 7, wherein the method further comprises:

Acquiring a movement instruction input by a user; the movement instruction is used to instruct to move the display position of the first display window; the first display window is a non-maximized window;

Moving the display position of the first display window according to the moving instruction.
The method according to claim 7, wherein the method further comprises:

Acquire an exchange instruction input by the user; the exchange instruction is used to instruct to exchange the screens respectively displayed in the second display window and the third display window; the second display window is a non-maximized window, and the third display window is Maximize the window;

The screens respectively displayed on the second display window and the third display window are exchanged and displayed according to the exchange instruction.
The method according to claim 10, wherein said acquiring an exchange instruction input by a user comprises:

Detecting a touch operation performed by a user on the second display window;

If the touch operation is a touch operation corresponding to the exchange instruction, the exchange instruction is generated.
The method according to claim 1, wherein the code stream transmission instruction is used to instruct the shooting device to merge the at least two code streams to generate a picture-in-picture code stream, and send the picture-in-picture code stream Code stream; the display code stream includes the picture-in-picture code stream;

The superimposing and displaying the pictures taken by the at least two image sensors on an image display device according to the display code stream includes:

According to the picture-in-picture code stream, the pictures taken by the at least two image sensors are displayed in the display window.
The method according to claim 12, further comprising:

A third display mode is sent to the photographing device; wherein the third display mode is used to indicate the display size, display position, and coverage relationship of the pictures taken by the at least two image sensors in the display window, so as to Enabling the shooting device to fuse the at least two code streams to generate the picture-in-picture code stream.
The method according to any one of claims 1-13, wherein before the superimposing and displaying the pictures taken by the at least two image sensors on an image display device according to the display code stream, the method further comprises:

Decoding the display code stream.
The method according to claim 14, wherein said decoding said display code stream comprises:

The display code stream is decoded through the image display device or a control device connected to the image display device.
The method according to any one of claims 1-13, wherein the code stream transmission instruction is input by a user.
The method according to any one of claims 1-13, wherein the image display device is connected to a control device, and instructions and code streams are transmitted between the image display device and the shooting device through the control device .
An image control method, characterized in that it is applied to a photographing device, the photographing device includes at least two image sensors, and the pictures respectively taken by the at least two image sensors correspond one-to-one with at least two code streams; the method include:

Receiving a code stream transmission instruction sent by an image display device; the code stream transmission instruction is used to instruct the shooting device to send the transmission mode of the at least two code streams;

Sending a display code stream to the image display device according to the code stream transmission instruction.
The method according to claim 18, wherein the code stream transmission instruction is used to instruct the shooting device to send each of the at least two code streams;

The sending a display code stream to the image display device according to the code stream transmission instruction includes:

Sending the at least two code streams to the image display device.
The method according to claim 19, wherein the photographing device is mounted on a movable platform, and there are at least two communication channels between the photographing device and the movable platform; and the displaying to the image The device sending the at least two code streams includes:

The at least two code streams are respectively sent to the movable platform through the at least two communication channels, so that the movable platform sends the at least two code streams to the image display device.
The method according to claim 18, wherein the code stream transmission instruction is used to instruct the shooting device to merge the at least two code streams to generate a picture-in-picture code stream, and send the picture-in-picture code stream Code stream

The sending a display code stream to the image display device according to the code stream transmission instruction includes:

Fusing the at least two code streams to generate the picture-in-picture code stream;

Sending the picture-in-picture code stream to the image display device.
The method according to claim 21, further comprising:

Receiving a display mode sent by the image display device; the display mode is used to indicate the display size, display position, and coverage relationship of the pictures taken by the at least two image sensors in the display window of the image display device;

The fusing the at least two code streams to generate the picture-in-picture code stream includes:

The at least two code streams are merged according to the display mode to generate the picture-in-picture code stream.
An image display device, characterized by comprising: a memory, a processor, a communication interface and a display;

The display is used to display images taken by a photographing device;

The memory is used to store program code;

The processor calls the program code, and when the program code is executed, is used to perform the following operations:

Controlling the communication interface to send a code stream transmission instruction to a photographing device; wherein the photographing device includes at least two image sensors, and the pictures respectively taken by the at least two image sensors correspond to at least two code streams in a one-to-one correspondence; The code stream transmission instruction is used to instruct the shooting device to send the transmission mode of the at least two code streams;

Controlling the communication interface to receive the display code stream sent by the shooting device according to the code stream transmission instruction;

The pictures taken by the at least two image sensors are superimposed and displayed on the display according to the display code stream.
The device according to claim 23, wherein the code stream transmission instruction is used to instruct the photographing device to send each of the at least two code streams separately; the display code stream includes the At least two streams;

The processor is specifically used for:

The pictures taken by the at least two image sensors are superimposed and displayed according to the at least two code streams.
The device according to claim 24, wherein the processor is specifically configured to:

Acquiring a window mode; the window mode is used to instruct to superimpose and display the pictures taken by the at least two image sensors in one display window, or to display the at least two image sensors in at least two superimposed display windows, respectively Images taken separately;

According to the window mode, the pictures taken by the at least two image sensors are superimposed and displayed according to the at least two code streams.
The device according to claim 24 or 25, wherein there is one display window, and the processor is specifically configured to:

Fusing the at least two code streams to generate a picture-in-picture code stream;

According to the picture-in-picture code stream, the pictures taken by the at least two image sensors are superimposed and displayed in the display window.
The device according to claim 26, wherein the processor is specifically configured to:

Acquiring a first display mode; the first display mode is used to indicate the display size, display position, and coverage relationship of the pictures respectively taken by the at least two image sensors in the display window;

The at least two code streams are merged according to the first display mode to generate the picture-in-picture code stream.
The device of claim 27, wherein the first display mode is input by a user.
The device according to claim 24 or 25, wherein there are at least two display windows, and the processor is specifically configured to:

Acquire a second display mode; the second display mode is used to indicate the correspondence between the pictures taken by the at least two image sensors and the at least two display windows, and the display of the at least two display windows Size, display position and coverage relationship;

According to the second display mode and the at least two code streams, the pictures taken by the at least two image sensors are respectively displayed in the at least two display windows.
The device of claim 29, wherein the second display mode is input by a user.
The device according to claim 29, wherein the processor is further configured to:

Acquiring a movement instruction input by a user; the movement instruction is used to instruct to move the display position of the first display window; the first display window is a non-maximized window;

Moving the display position of the first display window according to the moving instruction.
The device according to claim 29, wherein the processor is further configured to:

Acquire an exchange instruction input by the user; the exchange instruction is used to instruct to exchange the screens respectively displayed in the second display window and the third display window; the second display window is a non-maximized window, and the third display window is Maximize the window;

The screens respectively displayed on the second display window and the third display window are exchanged and displayed according to the exchange instruction.
The device according to claim 32, wherein the processor is specifically configured to:

Detecting a touch operation performed by a user on the second display window;

If the touch operation is a touch operation corresponding to the exchange instruction, the exchange instruction is generated.
The device according to claim 23, wherein the code stream transmission instruction is used to instruct the shooting device to merge the at least two code streams to generate a picture-in-picture code stream, and send the picture-in-picture code stream Code stream; the display code stream includes the picture-in-picture code stream;

The processor is specifically used for:

According to the picture-in-picture code stream, the pictures taken by the at least two image sensors are displayed in the display window.
The device according to claim 34, wherein the processor is further configured to:

Control the communication interface to send a third display mode to the photographing device; wherein the third display mode is used to indicate the display size and display position of the pictures respectively taken by the at least two image sensors in the display window And coverage relationship, so that the shooting device merges the at least two code streams to generate the picture-in-picture code stream.
The device according to any one of claims 23-35, wherein the processor is further configured to:

Decoding the display code stream.
The device according to claim 36, wherein the processor is specifically configured to:

The display code stream is decoded through the image display device or a control device connected to the image display device.
The device according to any one of claims 23-35, wherein the code stream transmission instruction is input by a user.
The device according to any one of claims 23-35, wherein the image display device is connected to a control device, and instructions and code streams are transmitted between the image display device and the shooting device through the control device .
A photographing device, which is characterized by comprising: a memory, a processor, a communication interface, and at least two image sensors, and the pictures respectively taken by the at least two image sensors correspond to at least two code streams in a one-to-one correspondence;

The memory is used to store program code;

The processor calls the program code, and when the program code is executed, is used to perform the following operations:

Controlling the communication interface to receive the code stream transmission instruction sent by the image display device; the code stream transmission instruction is used to instruct the photographing device to send the transmission mode of the at least two code streams;

Controlling the communication interface to send a display code stream to the image display device according to the code stream transmission instruction.
The shooting device according to claim 40, wherein the code stream transmission instruction is used to instruct the shooting device to respectively send each of the at least two code streams;

The processor is specifically used for:

Controlling the communication interface to send the at least two code streams to the image display device.
The shooting device according to claim 41, wherein the shooting device is mounted on a movable platform, and there are at least two communication channels between the shooting device and the movable platform; the processor is specifically used in:

Control the communication interface to send the at least two code streams to the movable platform through the at least two communication channels, so that the movable platform sends the at least two code streams to the image display device .
The shooting device according to claim 40, wherein the code stream transmission instruction is used to instruct the shooting device to merge the at least two code streams to generate a picture-in-picture code stream, and send the picture-in-picture code stream Picture stream

The processor is specifically used for:

Fusing the at least two code streams to generate the picture-in-picture code stream;

Controlling the communication interface to send the picture-in-picture code stream to the image display device.
The photographing device according to claim 43, wherein the processor is further configured to:

Control the communication interface to receive the display mode sent by the image display device; the display mode is used to indicate the display size and display position of the pictures respectively taken by the at least two image sensors in the display window of the image display device And coverage relationship;

The processor is specifically used for:

The at least two code streams are merged according to the display mode to generate the picture-in-picture code stream.
A movable platform, characterized by comprising: a device body and the photographing device according to any one of claims 40-44 loaded on the device body.
A storage medium, characterized by comprising: a readable storage medium and a computer program, the computer program being used to implement the image display method according to any one of claims 1-17 or as claimed in claims 18-22 Any one of the image control methods.