WO2020258031A1 - Procédé de commande, système de transmission d'images, dispositif d'affichage et système de véhicule aérien sans pilote - Google Patents

Procédé de commande, système de transmission d'images, dispositif d'affichage et système de véhicule aérien sans pilote Download PDF

Info

Publication number
WO2020258031A1
WO2020258031A1 PCT/CN2019/092799 CN2019092799W WO2020258031A1 WO 2020258031 A1 WO2020258031 A1 WO 2020258031A1 CN 2019092799 W CN2019092799 W CN 2019092799W WO 2020258031 A1 WO2020258031 A1 WO 2020258031A1
Authority
WO
WIPO (PCT)
Prior art keywords
image data
memory
stored
stored image
display controller
Prior art date
Application number
PCT/CN2019/092799
Other languages
English (en)
Chinese (zh)
Inventor
赵财华
吴一凡
刘怀宇
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2019/092799 priority Critical patent/WO2020258031A1/fr
Priority to CN201980008943.XA priority patent/CN111656797A/zh
Publication of WO2020258031A1 publication Critical patent/WO2020258031A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/764—Media network packet handling at the destination 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
    • H04N21/4435—Memory management

Definitions

  • This application relates to the field of image transmission technology, in particular to a control method, an image transmission system, a display device and an unmanned aerial vehicle system.
  • the analog video transmission scheme is usually based on the entire frame transmission, that is, the display device needs to receive a complete frame of image from the video input source before it starts to display.
  • this will introduce a frame of image transmission delay.
  • the video input source comes from high-speed moving equipment, such as a high-speed drone (its maximum speed can reach 280km/h)
  • the delay of the drone's analog video transmission is very important for such a fast speed.
  • Factor because such a frame of image transmission delay is more likely to cause bombing. Therefore, it is necessary to reduce the image transmission delay.
  • the embodiments of the present application provide a control method, an image transmission system, a display device, and an unmanned aerial vehicle system.
  • Image data from a video input source including a plurality of pixels, and writing the image data into the memory according to the arrangement of the plurality of pixels;
  • the display controller reads the stored image data from the memory;
  • an interrupt signal is issued so that the display controller outputs the read stored image data to the display screen; wherein the transmission time is less than one frame of transmission The time required for the image data, and the transmission time length is the time from when the image data is written into the memory to when the display controller outputs the read stored image data to the display screen .
  • the image transmission system of the embodiment of the present application includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program, and when the computer program is executed, the following steps are implemented: For image data of a video input source, the image data includes a plurality of pixels, and the image data is written into a memory according to an arrangement of the plurality of pixels; when the memory stores the image data, the display controller receives The memory reads the stored image data; when the stored image data meets a preset condition, an interrupt signal is issued, so that the display controller outputs the read stored image data to Display screen; wherein the transmission time is less than the time required to transmit one frame of the image data, and the transmission time is from the image data being written to the memory to the display controller will read the stored The duration of the image data output to the display screen.
  • the display device of the embodiment of the present application includes a display screen and the image transmission system of the above embodiment, wherein the image transmission system outputs image data to the display screen.
  • the unmanned aerial vehicle system of the embodiment of the present application includes an unmanned aerial vehicle, a display device connected to the unmanned aerial vehicle, and the image transmission system of the above-mentioned embodiment.
  • the control method, image transmission system, display device, and UAV system of the embodiments of the present application read the image data from the memory when the image data is stored in the memory, and it is not necessary to wait until a complete frame of image data is stored in the memory before reading take.
  • an interrupt signal is issued.
  • the time difference from writing the image data into the memory to outputting the read stored image data to the display screen can be basically fixed.
  • the transmission time is less than the time required to transmit one frame of image data, and a low-delay optimization scheme can be realized.
  • FIG. 1 is a schematic flowchart of an image transmission control method according to an embodiment of the present application
  • FIG. 2 is a schematic diagram of modules of an image transmission system according to an embodiment of the present application.
  • FIG. 3 is a schematic diagram of another module of the image transmission system according to the embodiment of the present application.
  • FIG. 4 is a schematic flowchart of an image transmission control method according to another embodiment of the present application.
  • FIG. 5 is a schematic flowchart of an image transmission control method according to another embodiment of the present application.
  • FIG. 6 is a schematic flowchart of an image transmission control method according to still another embodiment of the present application.
  • FIG. 7 is a schematic flowchart of an image transmission control method according to another embodiment of the present application.
  • FIG. 8 is a schematic diagram of modules of an image transmission system according to another embodiment of the present application.
  • FIG. 9 is a schematic flowchart of an image transmission control method according to another embodiment of the present application.
  • FIG. 10 is a schematic diagram of modules of an image transmission system according to another embodiment of the present application.
  • FIG. 11 is a schematic flowchart of an image transmission control method according to still another embodiment of the present application.
  • FIG. 12 is a schematic flowchart of an image transmission control method according to another embodiment of the present application.
  • FIG. 13 is a schematic flowchart of an image transmission control method according to another embodiment of the present application.
  • 15 is a schematic diagram of modules of an image transmission system according to still another embodiment of the present application.
  • FIG. 16 is a schematic diagram of modules of a display device according to an embodiment of the present application.
  • Fig. 17 is a schematic structural diagram of an unmanned aerial vehicle system according to an embodiment of the present application.
  • UAV system 1000 display device 100, processor 101, memory 102, display controller 103, image transmission system 10, image writing module 12, image conversion unit 124, first writing unit 126, image reading unit 127 , Image scaling unit 128, second writing unit 129, image reading module 14, interrupt module 16, dynamic adjustment module 18, video input source 20, drone 22, pan/tilt 222, camera 224, set-top box 24, display screen 30.
  • first and second are only used for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with “first” and “second” may explicitly or implicitly include one or more of the features. In the description of this application, “multiple” means two or more than two, unless otherwise specifically defined.
  • connection should be interpreted broadly unless otherwise clearly specified and limited.
  • it can be a fixed connection or a detachable connection.
  • Connected or integrally connected it can be mechanically connected, or electrically connected or can communicate with each other; it can be directly connected, or indirectly connected through an intermediate medium, it can be the internal communication of two components or the interaction of two components relationship.
  • connection should be interpreted broadly unless otherwise clearly specified and limited.
  • it can be a fixed connection or a detachable connection.
  • Connected or integrally connected it can be mechanically connected, or electrically connected or can communicate with each other; it can be directly connected, or indirectly connected through an intermediate medium, it can be the internal communication of two components or the interaction of two components relationship.
  • an embodiment of the present application provides an image transmission control method and an image transmission system 10.
  • Step S12 receiving image data from the video input source 20, the image data including a plurality of pixels, and writing the image data into the memory 102 according to the arrangement of the plurality of pixels;
  • Step S14 when the memory 102 stores image data, the display controller 103 reads the stored image data from the memory 102;
  • Step S16 When the stored image data meets the preset conditions, an interrupt signal is issued so that the display controller 103 outputs the read stored image data to the display screen 30; wherein, the transmission time is shorter than the transmission of one frame of image
  • the time required for the data to be transmitted is the time from when the image data is written into the memory 102 to when the display controller 103 outputs the read stored image data to the display screen 30.
  • the image transmission system 10 of the embodiment of the present application includes a memory 102 and a processor 101; the memory 102 is used to store a computer program; the processor 101 is used to execute the computer program and when the computer program is executed, the following steps are implemented: receiving video input from The image data of the source 20, the image data includes multiple pixels, the image data is written into the memory 102 according to the arrangement of the multiple pixels; when the memory 102 stores image data, the display controller 103 reads the stored image data from the memory 102 ; When the stored image data meets the preset conditions, an interrupt signal is issued so that the display controller 103 outputs the read stored image data to the display screen 30; wherein the transmission time is less than the transmission time of one frame of image data The required time, the transmission time is the time from when the image data is written into the memory 102 to when the display controller 103 outputs the read stored image data to the display screen 30.
  • the image transmission control method and image transmission system 10 of the embodiment of the present application reads the image data from the memory 102 when the image data is stored in the memory 102, and does not need to wait until the memory 102 stores a complete frame of image data before reading .
  • an interrupt signal is issued.
  • the transmission time is less than the time required to transmit one frame of image data, and a low-delay optimization scheme can be realized.
  • the processor 101 may be integrated with an image writing module 12, an image reading module 14, and an interrupt module 16.
  • the image writing module 12 is used to receive image data from the video input source 20, the image data includes multiple pixels, and the image data is written into the memory 102 according to the arrangement of the multiple pixels; the image reading module 14 is used to store the image data in the memory 102 When there is image data, the stored image data is read from the memory 102 through the display controller 103; the interrupt module 16 is used to send an interrupt signal when the stored image data meets a preset condition, so that the display controller 103 will read The obtained stored image data is output to the display screen 30; wherein, the transmission time is less than the time required to transmit one frame of image data, and the transmission time is from the time the image data is written into the memory 102 to the display controller 103 that will be read. The length of time the stored image data is output to the display screen 30.
  • the image writing module 12, the image reading module 14, and the interrupt module 16 may be integrated in the processor 101, or each module may exist alone physically, or two or more modules may be integrated together.
  • the above-mentioned functional modules can be executed in the form of hardware or software functional modules. If the above functional modules are executed in the form of software functional modules and sold or used as independent products, they can also be stored in a computer readable storage medium.
  • the image writing module 12 is a chip. In another example, the image writing module 12 is formed by interconnecting multiple chips.
  • the interrupt module 16 may be a frequency divider, the frequency divider is integrated in the display controller 103, and the display controller 103 is integrated in the processor 101. In another example, the interrupt module 16 may be a frequency divider, which is independent of the display controller 103, and both the frequency divider and the display controller 103 are integrated in the processor 101. The specific relationship between each module and each device is not limited here.
  • the image data from the video input source 20 can be received frame by frame in time sequence.
  • the processor 101 can be used to receive image data from the video input source 20 in a time sequence.
  • the image data can be written into the memory 102 according to the arrangement of pixels in the single frame of image data. For example, the image data may be written into the memory 102 row by row, the image data may be written into the memory 102 column by column, or the image data may be written into the memory 102 in other suitable ways, which is not limited herein.
  • the image writing module 12 may write one frame of image data into the memory 102 after receiving one frame of image data.
  • the image writing module 12 can also write image data of less than one frame into the memory 102.
  • the image writing module 12 can also write image data larger than one frame into the memory 102.
  • the amount of image data written into the memory 102 by the image writing module 12 is not limited here.
  • the video input source 20 includes at least one of a drone and a set top box (Set Top Box, STB).
  • the video input source 20 generates image data at a fixed frame rate. For example, a picture with a resolution of 720x480 transmits 60 frames per second, which is 60fps.
  • the video input source 20 includes a drone; in another example, the video input source 20 includes a set-top box; in yet another example, the video input source 20 includes a drone and a set-top box.
  • the delay of image transmission is likely to cause undesirable consequences such as bombing.
  • the maximum speed of the UAV is 280km/h
  • the delay of image transmission can be reduced, so that the image observed by the user is synchronized with the image captured by the drone Higher, so that users can understand the flight environment and status of the drone in time, so that they can control the drone more flexibly and avoid bombing.
  • a set-top box also known as a set-top box or a digital video conversion box, is a device that connects the TV to an external signal source. It can convert the compressed digital signal into TV content and display it on the TV.
  • the delay of image transmission during live broadcast may easily cause the user to be unable to participate in the discussion of the live broadcast content in time.
  • an emergency message such as an earthquake prediction
  • the delay of image transmission may easily cause the user to suffer casualties due to the inability to obtain the earthquake prediction in time.
  • the delay of image transmission can be reduced, so that users can obtain live content or emergency messages in time, thereby improving user experience.
  • the image data may include the Parr system (Phase Alteration Line, PAL), and may also include the National Television Systems Committee (National Television Systems Committee, NTSC).
  • PAL Phase Alteration Line
  • NTSC National Television Systems Committee
  • the specific format of the image data is not limited here.
  • the memory 102 is a dynamic random access memory (Dynamic Random Access Memory, DRAM).
  • DRAM Dynamic Random Access Memory
  • the memory 102 may also be other types of memory, such as random access memory (RAM), complementary metal oxide semiconductor memory (Complementary Metal Oxide Semiconductor Memory), and so on.
  • RAM random access memory
  • complementary metal oxide semiconductor memory Complementary Metal Oxide Semiconductor Memory
  • the specific type of the memory 102 is not limited here.
  • the display controller 103 is a liquid crystal display controller (LCD Controller, LCDC).
  • LCD Controller LCDC
  • the display controller 103 can obtain image data from the memory 102, superimpose and mix them in a certain manner, and combine the processed image data Send to the display screen 30 to display.
  • Interrupt refers to the processor receiving a signal from hardware or software, prompting that an event has occurred and should be paid attention to. This situation is called an interrupt.
  • the display controller 103 outputs the read image data to the display screen 30 after receiving the interrupt signal.
  • the display controller 103 can be adjusted to the same frame rate as the video input source 20, the interrupt signal can be a video synchronization signal (Video synchronization, Vsync), and Vsync is aligned to the output rhythm of the video input source 20, so that The time difference from writing the image data to the memory 102 to outputting the read image data to the display screen 30 is basically constant.
  • Vsync Video synchronization, Vsync
  • step S16 includes:
  • Step S161 When the stored image data exceeds the preset threshold, an interrupt signal is issued, so that the display controller 103 outputs the read stored image data to the display screen 30.
  • the processor 101 is configured to send an interrupt signal when the stored image data exceeds a preset threshold, so that the display controller 103 outputs the read stored image data to the display screen 30.
  • the interrupt module 16 is configured to send an interrupt signal when the stored image data exceeds a preset threshold, so that the display controller 103 outputs the read stored image data to the display screen 30.
  • an interrupt signal is issued when the stored image data meets the preset condition.
  • the stored image data exceeds a preset threshold, including: the row height and/or column height of the stored image data exceeds the preset threshold.
  • the processor 101 is configured to send an interrupt signal when the row height and/or column height of the stored image data exceeds a preset threshold, so that the display controller 103 outputs the read stored image data to Display 30.
  • the row height and/or column height of the stored image data exceeds the preset threshold, including three situations: the row height of the stored image data exceeds the preset threshold, and the column height does not exceed the preset threshold; The column height of the image data exceeds the preset threshold, and the row height does not exceed the preset threshold; the row height and column height of the stored image data both exceed the preset threshold.
  • the preset threshold may be stored in the memory 102 in advance, and the preset threshold may also be set according to user input.
  • the source of the preset threshold is not limited here.
  • the row height and/or column height of the stored image data meets the preset threshold, including: the row height of the stored image data exceeds half of the row height of a complete image, and/or the stored image data
  • the column height of is more than half of the column height of a complete image.
  • the processor 101 is configured to send an interrupt signal when the line height of the stored image data exceeds half of the line height of a complete image, so that the display controller 103 outputs the read stored image data To display 30;
  • the processor 101 is configured to send an interrupt signal when the column height of the stored image data exceeds half of the column height of a complete image, so that the display controller 103 outputs the read stored image data to Display 30;
  • the processor 101 is configured to send an interrupt signal when the row height of the stored image data exceeds half of the row height of a complete frame of image and the column height of the stored image data exceeds half of the column height of a complete frame of image , So that the display controller 103 outputs the read stored image data to the display screen 30.
  • the preset threshold may be one-third of the row height, one-third of the column height, two-thirds of the row height, two-thirds of the column height, and the row height.
  • the specific value of the preset threshold is not limited here.
  • step S16 includes:
  • Step S162 within the preset time period of writing the stored image data into the memory 102, an interrupt signal is issued, so that the display controller 103 outputs the read stored image data to the display screen 30.
  • the processor 101 is configured to issue an interrupt signal within the preset time period of writing the stored image data into the memory 102, so that the display controller 103 outputs the read stored image data to the display screen 30 .
  • the transmission time is less than the time required to transmit one frame of image data.
  • the transmission time is the time from when the image data is written into the memory 102 to when the display controller 103 outputs the read stored image data to the display screen 30 . Therefore, the preset duration is less than the time required to transmit one frame of image data. In this way, the time difference from writing image data to the memory 102 to outputting the read stored image data to the display screen 30 can be basically fixed, and a low-latency optimization solution can also be realized.
  • the value range of the preset duration is 0.5ms-5ms.
  • the preset duration is 0.5ms, 1ms, 1.2ms, 1.8ms, 2ms, 2.6ms, 3.1ms, 4.5ms, 5ms, or other values in the range of 0.5ms-5ms.
  • the value range of the preset duration is 1ms-2ms.
  • the preset duration is 1ms, 1.2ms, 1.8ms, 2ms, or other values within the range of 1ms-2ms.
  • the specific value of the preset duration is not limited here.
  • the display controller 103 receives the interrupt signal and outputs the read image data to the display screen 30. In another example, 0.5 ms after the image data is written into the memory 102, the display controller 103 receives the interrupt signal and outputs the read image data to the display screen 30. In another example, 2 ms after the image data is written into the memory 102, the display controller 103 receives the interrupt signal and outputs the read image data to the display screen 30.
  • step S12 includes:
  • Step S122 Write the image data into the memory 102 line by line.
  • the processor 101 is configured to write image data into the memory 102 line by line.
  • the image writing module 12 is used to write image data into the memory 102 line by line.
  • the image data is written into the memory 102 according to the arrangement of a plurality of pixels. It can be understood that in other embodiments, one frame of image may be written into the memory 102 by interlaced writing or other methods. The specific manner of writing image data into the memory 102 is not limited here.
  • step S12 includes:
  • Step S124 Convert the type of the received image data from the first type to the second type
  • Step S126 Write the second type of image data into the memory 102.
  • the processor 101 is used for converting the type of the received image data from the first type to the second type; and for writing the image data of the second type into the memory 102.
  • the type of the image data generated by the video input source 20 is the first type
  • the image writing module 12 includes an image conversion unit 124 and a first writing unit 126
  • the image conversion unit 124 is used for receiving
  • the type of image data is converted from the first type to the second type; the first writing unit 126 is used to write the image data of the second type into the memory 102.
  • the image data from the video input source 20 is received frame by frame in time sequence and the image data is written into the memory 102.
  • the first type of image data is analog data
  • the second type of image data is digital data.
  • the first type of image data is an analog video shot by a drone
  • the image conversion unit 124 converts the analog video shot by the drone into a digital video, and uses the Mobile Industry Processor Interface (Mobile Industry Processor Interface,
  • the transmitter (Transport, TX) of the camera serial interface (Camera Serial Interface, CSI) of MIPI is output to the camera serial interface (Mobile Industry Processor Interface, MIPI) of the first writing unit 126.
  • the physical layer (Physical, PHY) of the receiving end (Receive, RX) writes it into the memory 102.
  • the display controller 103 After the display controller 103 receives the interrupt signal, it reads the image data from the memory 102 and outputs it to the display screen 30 through the display interface (DSI) of the display controller 103. In this way, the user can see the image taken by the drone on the display screen 30.
  • DSI display interface
  • the first type of image data is analog video output by the set-top box
  • the image conversion unit 124 converts the analog video output by the set-top box into digital video, and outputs it to the first writing unit 126 through the TX of MIPI's CSI.
  • RX of MIPI's CSI After receiving the digital video, the PHY of the RX writes it into the memory 102.
  • the display controller 103 receives the interrupt signal, it reads out the image data from the memory 102 and outputs it to the display screen 30 through the DSI of the display controller 103. In this way, the user can see the image output by the set-top box on the display screen 30.
  • the image conversion unit 124 may be integrated in a conversion device independent of the video input source.
  • the image conversion unit 124 may also be integrated in the video input source 20.
  • the video input source 20 is a video output device with MIPI CSI.
  • step S12 includes:
  • Step S127 Read image data from the memory 102
  • Step S128 performing scaling processing on the image data
  • Step S129 Write the zoomed image data into the memory 102.
  • the processor 101 is used to read image data from the memory 102; used to perform scaling processing on the image data; and used to write the image data after scaling processing into the memory 102.
  • the image writing module 12 includes an image reading unit 127, an image scaling unit 128 and a second writing unit 129, the image reading unit 127 is used to read image data from the memory 102; image scaling unit 128 is used to perform scaling processing on the image data; the second writing unit 129 is used to write the image data after scaling processing into the memory 102.
  • the image reading unit 127, the image scaling unit 128, and the second writing unit 129 may all be integrated in a scaler.
  • the scaler may be integrated in the processor 101.
  • only the image scaling unit 128 may be integrated in the scaler, and the image reading unit 127 and the second writing unit 129 may be integrated in the image reading and writing device.
  • the image reading and writing device After the image reading and writing device reads the image data from the memory 102, it sends the image data to the scaler.
  • the scaler is scaling the image data and can send the processed image data to the image reading and writing device so that the image can be read and written.
  • the device writes the image data after scaling processing into the memory 102.
  • the image reading unit 127, the image scaling unit 128, and the second writing unit 129 may be functional modules of different devices.
  • the image reading unit 127 is a functional module of the image reading device
  • the image scaling unit 128 is a functional module of a scaler
  • the second writing unit 129 is a functional module of the image writing device.
  • the specific forms of the image reading unit 127, the image scaling unit 128, and the second writing unit 129 are not limited here.
  • step S129 the image data after scaling is written into the memory 102 line by line. It can be understood that, in other embodiments, the image data may be written into the memory 102 by interlaced writing or other methods. The specific manner of writing image data into the memory 102 is not limited here.
  • step S128 includes:
  • Step S1282 Perform scaling processing on the resolution of the image data.
  • the processor 101 is configured to perform scaling processing on the resolution of the image data.
  • the image scaling unit 128 is used for scaling the resolution of the image data.
  • the image data is scaled and scaled. It can be understood that by processing the resolution of the image data, the size of the image data can be adjusted, so that the memory 102 can be used more effectively.
  • the resolution of the image data can be reduced, thereby reducing the size of the image data and avoiding occupying too much memory 102 space.
  • the resolution of the image data can be enlarged, so as to enlarge the size of the image data, so that the content of the image is richer, and when displayed on the display screen 30, it is more delicate. Better results.
  • the resolution of the image data can be scaled by methods such as neighborhood interpolation, bilinear interpolation, or bicubic interpolation.
  • the specific method of scaling processing is not limited here.
  • the first type of image data is an analog video taken by a drone
  • the image conversion unit 124 converts the analog video output by the set-top box into a digital video, and outputs it to the first writing unit 126 through the TX of MIPI CSI MIPI CSI RX.
  • the PHY of the RX After receiving the digital video, the PHY of the RX writes it into the memory 102.
  • the scaler reads out the image data from the memory 102 and performs scaling processing on the resolution of the image data, and then writes the processed image data into the memory 102 line by line.
  • the display controller 103 receives the interrupt signal, it reads out the image data from the memory 102 and outputs it to the display screen 30 through the DSI of the display controller 103. In this way, the user can see the image taken by the drone on the display screen 30.
  • step S14 includes:
  • Step S142 The display controller 103 reads the stored image data line by line from the memory 102.
  • the processor 101 is configured to read the stored image data line by line from the memory 102 through the display controller 103.
  • the image reading module 14 is used to read the stored image data line by line from the memory 102 through the display controller 103.
  • the display controller 103 reads the image data from the memory 102. It can be understood that, in other embodiments, the image data may be read from the memory 102 by interlaced reading or other methods. The specific manner of reading the image data from the memory 102 is not limited here.
  • step S16 includes:
  • Step S162 When the stored image data meets the preset condition, an interrupt signal is issued, so that the display controller 103 outputs the read stored image data to the display screen 30 line by line.
  • the processor 101 is configured to send an interrupt signal when the stored image data meets a preset condition, so that the display controller 103 outputs the read stored image data to the display screen 30 line by line.
  • the interrupt module 16 is configured to issue an interrupt signal within a preset time period of writing image data into the memory 102, so that the display controller 103 outputs the read image data to the display screen 30 line by line.
  • an interrupt signal is issued, so that the display controller 103 outputs the read image data to the display screen 30.
  • the display controller 103 may also be allowed to output the read image data to the display screen 30 interlacedly.
  • the specific manner in which the display controller 103 outputs the image data to the display screen 30 is not limited here.
  • the image transmission control method includes:
  • Step S18 If the stored image data meets the preset condition and the interrupt signal is not issued, the interrupt signal is issued again.
  • the processor 101 is configured to issue an interrupt signal again when the stored image data meets the preset condition and the interrupt signal is not issued.
  • the image transmission system 10 includes a dynamic adjustment module 18, and the dynamic adjustment module 18 is used to issue an interrupt signal again when the stored image data meets a preset condition and no interrupt signal is issued.
  • the interrupt signal can be triggered by the divider. It can be understood that after working for a period of time, the frequency divider will accumulate errors, resulting in the time difference from writing the image data to the memory 102 to sending the interrupt signal no longer meets the requirement of the preset duration.
  • the interrupt signal is sent again when the interrupt signal is not sent within the preset time, which can prevent the frequency divider from not sending the interrupt signal within the preset time, thereby realizing the dynamic adjustment of the image transmission , So that the time difference from when the image data is written into the memory 102 to when the interrupt signal is issued always meets the requirement of the preset time length.
  • the preset time length is 0.5ms
  • the image writing module 12 writes image data into the memory 102 at the zero point. After 0.5ms after the zero point, it is detected that the frequency divider does not send an interrupt signal, and the dynamic adjustment module 18 displays The controller 103 sends an interrupt signal to ensure that the display controller 103 outputs the read image data to the display screen 30 within 0.5 ms.
  • the preset time length is 0.5ms
  • the image writing module 12 writes image data into the memory 102 at the zero point. After 0.5ms after the zero point, it is detected that the divider does not send an interrupt signal, and the dynamic adjustment module 18 An instruction is sent to the frequency divider so that the frequency divider sends an interrupt signal again, so as to ensure that the display controller 103 outputs the read image data to the display screen 30 within 0.5 ms.
  • the dynamic adjustment module 18 can directly send the interrupt signal to the display controller 103 again, or the dynamic adjustment module 18 can send the interrupt signal to the frequency divider. An instruction is sent to cause the frequency divider to send an interrupt signal to the display controller 103 again.
  • the specific manner in which the dynamic adjustment module 18 sends an interrupt signal is not limited here.
  • the image transmission control method of this embodiment when the stored image data exceeds the preset threshold and does not issue In the case of an interrupt signal, the interrupt signal is issued again, which can realize the dynamic adjustment of the image transmission, so that the issued interrupt signal always meets the preset condition.
  • the preset threshold value is one half of the line height.
  • the frequency divider does not send an interrupt signal, and it is dynamically adjusted
  • the module 18 sends an interrupt signal to the display controller 103 to ensure that when the line height of the stored image data exceeds half of the line height of a complete image, the display controller 103 outputs the read image data to the display screen 30 .
  • the preset threshold is one half of the line height.
  • the dynamic The adjustment module 18 sends an instruction to the frequency divider so that the frequency divider sends an interrupt signal again, so as to ensure that when the line height of the stored image data exceeds half of the line height of a complete image, the display controller 103 will read The captured image data is output to the display screen 30.
  • the display device 100 of the embodiment of the present application includes a display screen 30 and the image transmission system 10 of any of the above embodiments, wherein the image transmission system 10 outputs image data to the display screen 30.
  • the display device 100 of the embodiment of the present application reads the image data from the memory 102 when the image data is stored in the memory 102, and does not need to wait until the memory 102 stores a complete frame of image data before reading.
  • an interrupt signal is issued.
  • the time difference from writing the image data into the memory 102 to outputting the read stored image data to the display screen 30 can be substantially fixed.
  • the transmission time is less than the time required to transmit one frame of image data, and a low-delay optimization scheme can be realized.
  • the display device 100 is a head-mounted display device, and the head-mounted display device is connected to the drone 22.
  • head-mounted display devices include, but are not limited to, electronic glasses, electronic helmets, or head-mounted parts of electronic devices.
  • the head-mounted display device may be based on augmented reality technology (Augmented Reality, AR) or virtual reality technology (Virtual Reality, VR).
  • AR Augmented Reality
  • VR Virtual Reality
  • the specific form of the head-mounted display device is not limited here.
  • the drone 22 may include a pan/tilt 222 and a camera 224 set on the pan/tilt 222.
  • the gimbal 222 can fix the camera 224 to the drone 22, and can also adjust the posture of the camera 224, thereby expanding the shooting range of the camera 224.
  • the pan/tilt 222 can stabilize and stabilize the camera 224 so as to prevent the camera 224 from being affected by flying, thereby improving the quality of captured images.
  • the image transmission system 10 transmits the image data taken by the camera 224 of the drone 22 to the head-mounted display device, so that the user can observe the picture taken by the drone 22 through the display screen 30 of the head-mounted display device.
  • the head-mounted display device can also process image data, for example, using AR or VR technology to process, and then display the processed image on the display screen 30, thereby enriching the user experience.
  • the unmanned aerial vehicle system 1000 of the embodiment of the present application includes an unmanned aerial vehicle 22, a display device 100 connected to the unmanned aerial vehicle 22, and the image transmission system 10 of any one of the above embodiments.
  • the unmanned aerial vehicle system 1000 of the embodiment of the present application reads the image data from the memory 102 when the image data is stored in the memory 102, and does not have to wait for a complete frame of image data to be stored in the memory 102 before reading.
  • an interrupt signal is issued.
  • the time difference from writing the image data into the memory 102 to outputting the read stored image data to the display screen 30 can be substantially fixed.
  • the transmission time is less than the time required to transmit one frame of image data, and a low-delay optimization scheme can be realized.
  • a "computer-readable medium” can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, device, or device or in combination with these instruction execution systems, devices, or devices.
  • computer readable media include the following: electrical connections (electronic devices) with one or more wiring, portable computer disk cases (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable and editable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM).
  • the computer-readable medium may even be paper or other suitable media on which the program can be printed, because it can be used, for example, by optically scanning the paper or other media, and then editing, interpreting, or other suitable media if necessary. The program is processed in a manner to obtain the program electronically and then stored in the computer memory.
  • each part of this application can be executed by hardware, software, firmware or a combination thereof.
  • multiple steps or methods can be executed by software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a logic gate circuit for performing logic functions on data signals
  • Discrete logic circuits Discrete logic circuits
  • application-specific integrated circuits with suitable combinational logic gates
  • FPGA field programmable gate array
  • a person of ordinary skill in the art can understand that all or part of the steps carried in the above-mentioned implementation method can be performed by a program instructing relevant hardware to complete.
  • the program can be stored in a computer-readable storage medium, and the program can be When it includes one of the steps of the method embodiment or a combination thereof.
  • the functional units in the various embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units may be integrated into one module.
  • the above-mentioned integrated modules can be executed in the form of hardware or software function modules. If the integrated module is executed in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.
  • the aforementioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

Un procédé de commande permettant une transmission d'images comprend les étapes consistant à : (S12) recevoir des données d'images provenant d'une source d'entrée vidéo (20), les données d'images contenant de multiples pixels, et écrire les données d'images dans une mémoire (102) en fonction de l'agencement des multiples pixels ; (S14) lorsque la mémoire (102) stocke les données d'images, lire au moyen d'un contrôleur d'affichage (103) les données d'images stockées provenant de la mémoire (102) ; et (S16) lorsque les données d'images stockées satisfont une condition prédéfinie, émettre un signal d'interruption de telle sorte que le contrôleur d'affichage (103) sort les données d'images stockées et lues sur un écran d'affichage (30), une durée prédéfinie étant inférieure à une période requise pour transmettre une trame des données d'images. La présente invention concerne également un système de transmission d'images (10), un dispositif d'affichage (100) et un système de véhicule aérien sans pilote (1000).
PCT/CN2019/092799 2019-06-25 2019-06-25 Procédé de commande, système de transmission d'images, dispositif d'affichage et système de véhicule aérien sans pilote WO2020258031A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2019/092799 WO2020258031A1 (fr) 2019-06-25 2019-06-25 Procédé de commande, système de transmission d'images, dispositif d'affichage et système de véhicule aérien sans pilote
CN201980008943.XA CN111656797A (zh) 2019-06-25 2019-06-25 控制方法、图像传输系统、显示装置及无人机系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/092799 WO2020258031A1 (fr) 2019-06-25 2019-06-25 Procédé de commande, système de transmission d'images, dispositif d'affichage et système de véhicule aérien sans pilote

Publications (1)

Publication Number Publication Date
WO2020258031A1 true WO2020258031A1 (fr) 2020-12-30

Family

ID=72350341

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/092799 WO2020258031A1 (fr) 2019-06-25 2019-06-25 Procédé de commande, système de transmission d'images, dispositif d'affichage et système de véhicule aérien sans pilote

Country Status (2)

Country Link
CN (1) CN111656797A (fr)
WO (1) WO2020258031A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114679525A (zh) * 2022-03-24 2022-06-28 中电海康集团有限公司 一种用于自动驾驶系统的图像处理装置及图像采集系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140192231A1 (en) * 2013-01-04 2014-07-10 Canon Kabushiki Kaisha Image signal processing apparatus and a control method thereof, and an image pickup apparatus and a control method thereof
CN104253964A (zh) * 2013-06-27 2014-12-31 精工爱普生株式会社 图像处理装置、图像显示装置以及图像处理装置的控制方法
CN105144690A (zh) * 2013-04-26 2015-12-09 富士胶片株式会社 摄像装置
CN106095366A (zh) * 2016-06-07 2016-11-09 北京小鸟看看科技有限公司 一种缩短图像延迟的方法、装置和虚拟现实设备
CN106134173A (zh) * 2014-03-25 2016-11-16 精工爱普生株式会社 显示控制装置和显示设备
WO2019031735A1 (fr) * 2017-08-08 2019-02-14 Samsung Electronics Co., Ltd. Appareil de traitement d'image, procédé de traitement d'image et système d'affichage d'image

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100274397B1 (ko) * 1998-06-18 2000-12-15 구자홍 타임랩스 영상녹화재생기의 메모리 운용방법
JP6243129B2 (ja) * 2013-03-19 2017-12-06 クラリオン株式会社 カメラ画像表示システム
CN107231561A (zh) * 2017-07-11 2017-10-03 Tcl移动通信科技（宁波）有限公司 一种图像数据传输方法、移动终端及存储装置
CN109474768A (zh) * 2017-09-08 2019-03-15 中兴通讯股份有限公司 一种提高图像流畅度的方法及装置
CN109874052B (zh) * 2019-02-21 2022-01-07 海信视像科技股份有限公司 图像处理方法、装置、设备及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140192231A1 (en) * 2013-01-04 2014-07-10 Canon Kabushiki Kaisha Image signal processing apparatus and a control method thereof, and an image pickup apparatus and a control method thereof
CN105144690A (zh) * 2013-04-26 2015-12-09 富士胶片株式会社 摄像装置
CN104253964A (zh) * 2013-06-27 2014-12-31 精工爱普生株式会社 图像处理装置、图像显示装置以及图像处理装置的控制方法
CN106134173A (zh) * 2014-03-25 2016-11-16 精工爱普生株式会社 显示控制装置和显示设备
CN106095366A (zh) * 2016-06-07 2016-11-09 北京小鸟看看科技有限公司 一种缩短图像延迟的方法、装置和虚拟现实设备
WO2019031735A1 (fr) * 2017-08-08 2019-02-14 Samsung Electronics Co., Ltd. Appareil de traitement d'image, procédé de traitement d'image et système d'affichage d'image

Also Published As

Publication number Publication date
CN111656797A (zh) 2020-09-11

Similar Documents

Publication Publication Date Title
US11741916B2 (en) Video frame rate compensation through adjustment of timing of scanout
US8711207B2 (en) Method and system for presenting live video from video capture devices on a computer monitor
US20150242994A1 (en) Method and system for accelerating video preview digital camera
CN104918137B (zh) 一种拼接屏系统播放视频的方法
US8063910B2 (en) Double-buffering of video data
US20120314777A1 (en) Method and apparatus for generating a display data stream for transmission to a remote display
WO2012079453A1 (fr) Système et procédé de traitement de données vidéo, unités cpu et gpu et système de surveillance vidéo
US20150163450A1 (en) Video display system, source device, sink device, and video display method
WO2021092827A1 (fr) Procédé et dispositif de traitement de commutation sans écran noir de signal vidéo
WO2020258031A1 (fr) Procédé de commande, système de transmission d'images, dispositif d'affichage et système de véhicule aérien sans pilote
CN102625086B (zh) 一种用于高清数字矩阵的ddr2存储方法和系统
TW201421250A (zh) 顯示裝置及其影像擷取方法
US9609215B2 (en) Moving-image recording/reproduction apparatus
US10362216B2 (en) Image pickup apparatus of which display start timing and display quality are selectable, method of controlling the same
JP5688255B2 (ja) 映像処理装置、表示装置及び映像処理方法
US9420218B2 (en) Television system
US20140056524A1 (en) Image processing device, image processing method, and program
JP2008131264A (ja) 監視カメラ、画像記録表示装置及び監視カメラシステム
US20230344963A1 (en) Docking device and method with video capturing function
JP6790276B2 (ja) 映像出力制御装置及び映像出力制御方法
US20240163509A1 (en) Apparatus, method and computer program product for distributing video data over a network
US20210400230A1 (en) Video display system, video conversion device, and video display method
KR101212947B1 (ko) 데이터 전송 장치
JP2006270346A (ja) 映像配信システムおよびネットワークカメラ
US20110292292A1 (en) Method and apparatus for displaying video data

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19934509

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19934509

Country of ref document: EP

Kind code of ref document: A1