WO2019104558A1 - Procédé de traitement d'image, équipement de photographie, véhicule aérien sans pilote et dispositif d'extrémité au sol - Google Patents

Procédé de traitement d'image, équipement de photographie, véhicule aérien sans pilote et dispositif d'extrémité au sol Download PDF

Info

Publication number
WO2019104558A1
WO2019104558A1 PCT/CN2017/113657 CN2017113657W WO2019104558A1 WO 2019104558 A1 WO2019104558 A1 WO 2019104558A1 CN 2017113657 W CN2017113657 W CN 2017113657W WO 2019104558 A1 WO2019104558 A1 WO 2019104558A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
encoded data
data
image encoded
processor
Prior art date
Application number
PCT/CN2017/113657
Other languages
English (en)
Chinese (zh)
Inventor
翁超
王铭熙
樊洁
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201780029120.6A priority Critical patent/CN109417594A/zh
Priority to PCT/CN2017/113657 priority patent/WO2019104558A1/fr
Publication of WO2019104558A1 publication Critical patent/WO2019104558A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display

Definitions

  • the embodiments of the present invention relate to the field of drones, and in particular, to an image processing method, a photographing device, a drone, and a ground end device.
  • the drone is equipped with a photographing device through the pan/tilt and controls the photographing device to perform aerial photography.
  • the photographing device includes an image sensor, and the image sensor is used to sense image data, the processor or the photographing device of the drone.
  • the processor needs to perform distortion calibration on the image data collected by the image sensor.
  • Embodiments of the present invention provide an image processing method, a photographing device, a drone, and a ground end device to solve the problem of insufficient performance of the processor, and the bandwidth and performance of the processor when the code stream of the image data collected by the image sensor is large. The problem.
  • a first aspect of the embodiments of the present invention provides an image processing method, including:
  • the main processor acquires image data
  • the main processor encodes the image data to obtain image encoded data
  • the main processor transmits the image encoded data to a slave processor to cause the slave processor to perform distortion calibration on the image data according to the image encoded data.
  • a second aspect of the embodiments of the present invention provides an image processing method, including:
  • the slave processor performs distortion calibration on the image data according to the image encoded data.
  • a third aspect of the embodiments of the present invention provides an image processing method, which is applied to a drone, and includes:
  • a fourth aspect of the present invention provides an image processing method, which is applied to a ground end device, and includes:
  • a fifth aspect of the embodiments of the present invention provides an image processing apparatus, including: a main processor and a slave processor;
  • the main processor is used to:
  • a sixth aspect of the embodiments of the present invention provides an image processing apparatus, including: a main processor and a slave processor;
  • the slave processor is used to:
  • a seventh aspect of the embodiments of the present invention provides a photographing apparatus, including:
  • An eighth aspect of the embodiments of the present invention provides a drone, including:
  • a power system mounted to the fuselage for providing power
  • a photographing device mounted on the body for collecting image data
  • a ninth aspect of the embodiments of the present invention provides a drone, including:
  • a power system mounted to the fuselage for providing power
  • a photographing device mounted on the body for collecting image data
  • One or more first processors and a first communication interface
  • the first processor is configured to:
  • the first communication interface is used to:
  • a tenth aspect of the present invention provides a ground end device, including: a second communication interface and a second processor;
  • the second communication interface is configured to receive image encoded data sent by the drone, and the image encoded data is obtained by encoding, by the drone, image data obtained by the drone;
  • the second processor is configured to perform distortion calibration on the image data according to the image encoded data.
  • the image processing method, the photographing device, the drone and the ground end device provided by the embodiment obtain image data by the main processor, encode the image data, obtain image coded data, and send the image coded data to the slave processor.
  • the slave processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, thereby saving the performance overhead of the main processor and the performance of the main processor.
  • the bandwidth and performance of the main processor can be effectively avoided.
  • FIG. 1 is a flowchart of an image processing method according to an embodiment of the present invention
  • FIG. 2 is a schematic structural diagram of a photographing apparatus according to an embodiment of the present invention.
  • FIG. 3 is a schematic structural diagram of a drone according to an embodiment of the present invention.
  • FIG. 4 is a flowchart of an image processing method according to another embodiment of the present invention.
  • FIG. 5 is a flowchart of an image processing method according to another embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a photographing apparatus according to another embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a drone according to another embodiment of the present invention.
  • FIG. 8 is a flowchart of an image processing method according to another embodiment of the present invention.
  • FIG. 9 is a schematic diagram of a communication system according to another embodiment of the present invention.
  • FIG. 10 is a schematic diagram of a communication system according to another embodiment of the present invention.
  • FIG. 11 is a flowchart of an image processing method according to another embodiment of the present invention.
  • FIG. 12 is a flowchart of an image processing method according to another embodiment of the present invention.
  • FIG. 13 is a structural diagram of an image processing apparatus according to an embodiment of the present invention.
  • FIG. 14 is a structural diagram of an image processing apparatus according to another embodiment of the present invention.
  • 15 is a structural diagram of a drone according to an embodiment of the present invention.
  • FIG. 16 is a structural diagram of a ground end device according to an embodiment of the present invention.
  • a component when referred to as being "fixed” to another component, it can be directly on the other component or the component can be present. When a component is considered to "connect” another component, it can be directly connected to another component or possibly a central component.
  • a photographing device such as a photographing device mounted on a handheld pan/tilt, a photographing device held by a user, a movable platform such as a photographing device mounted on the drone, and the like are provided with an image sensor that can capture image data in real time and The captured image data is sent to the processor, and the image data is distorted and calibrated by the processor to obtain the calibrated image data, and then the calibrated image data is encoded to obtain the encoded image data, and the encoded image data is obtained.
  • the role of the processor is not only to process the image data, but also to process other data. When the performance of the processor is insufficient, if the code stream of the image data is high, the processor will have bandwidth and performance problems.
  • the embodiment provides an image processing method, which is described below in conjunction with a specific embodiment.
  • FIG. 1 is a flowchart of an image processing method according to an embodiment of the present invention. As shown in FIG. 1, the method in this embodiment may include:
  • Step S101 The main processor acquires image data.
  • the image processing method described in this embodiment is applied to the photographing apparatus 20 shown in FIG. 2, and the photographing apparatus 20 includes an image sensor 21, a main processor 22, and a slave processor 23.
  • the image processing method can also be applied to a mobile platform such as a drone equipped with a photographing device.
  • the method is also applicable to other forms of devices having image processing capabilities.
  • the drone 30 includes a main processor 31 and a slave processor 32, and the drone 30 is equipped with a photographing device 35 via a support device 34 such as a pan/tilt.
  • the main processor 31 and the slave processor 32 are electrically connected, and the main processor 31 and the photographing device 35 are electrically connected.
  • the main processor acquires image data, and includes the following feasible implementation manners:
  • a possible implementation manner is that the main processor acquires image data collected by an image sensor.
  • the image sensor 21 can acquire image data in real time and transmit the image data to the main processor 22.
  • the photographing device 35 may be provided with an image sensor that collects image data in real time and transmits the image data to the main processor 31.
  • Another possible implementation manner is that the main processor acquires a captured image corresponding to the shooting control instruction.
  • the photographing apparatus 20 may generate a photographing control command according to a photographing control operation of the user, and optionally, the photographing control command is generated by the main processor 22 according to a photographing control operation of the user.
  • the photographing control operation may be an operation of a user's photographing button or button of the photographing device 20.
  • the photographing device 20 can also be in communication connection with a control device through which the user can control the photographing device 20, and the photographing control operation can also be an operation of a photographing button or button of the control device by the user.
  • the main processor 22 can acquire the image data acquired by the image sensor 21 at the time of the photographing control command generation, and it can be understood that the image sensor 21 is acquired at the photographing control command generating timing.
  • the image data is the captured image.
  • the main processor 31 can control the photographing device 35 to take image data according to the photographing control instruction, and further, the photographing device 35 can transmit the image data photographed by the photographing device 35 to the main processor 31.
  • the shooting control command may be sent to the drone 30 by a control terminal of the drone 30, such as a remote controller.
  • Step S102 the main processor encodes the image data to obtain an image coding number. according to.
  • the main processor 22 encodes the image data collected by the image sensor 21 to obtain image encoded data.
  • the main processor 31 encodes image data captured by the photographing device 35 to obtain image encoded data.
  • Step S103 The main processor sends the image encoded data to a slave processor, so that the slave processor performs distortion calibration on the image data according to the image encoded data.
  • the main processor 22 encodes the image data to obtain image encoded data, and then transmits the image encoded data to the slave processor 23, and the slave processor 23 can decode the image encoded data to obtain image data. Further, the image data obtained after decoding is subjected to distortion calibration.
  • the main processor 22 may send the image encoded data to the slave processor 23 in real time, and the slave processor 23 may decode the image encoded data in real time to obtain image data, and perform image data obtained after decoding. Distortion calibration.
  • the main processor 31 encodes the image data, and after obtaining the image encoded data, the image encoded data is sent to the slave processor 32, and the slave processor 32 can perform image encoded data. Decoding to obtain image data, and further performing distortion calibration on the decoded image data.
  • the main processor 31 may send the image encoded data to the slave processor 32 in real time, and the slave processor 32 may decode the image encoded data in real time to obtain image data, and perform image data obtained after decoding. Distortion calibration.
  • the main processor sends the image encoded data to a slave processor, so that the slave processor performs distortion calibration on the image data according to the image encoded data, including: the main processor And transmitting the image encoded data to the slave processor, so that the slave processor stores the image encoded data, and performs distortion calibration on the image data according to the stored image encoded data.
  • the main processor 22 encodes the image data, and after obtaining the image encoded data, transmits the image encoded data to the slave processor 23, and the slave processor 23 stores the image encoded data, for example, when When the processor 23 receives the image encoded data sent by the main processor 22, the image encoded data is first stored in the memory, and when the slave processor 23 is idle, the image encoded data is acquired from the memory, and the encoded data is decoded. To obtain image data, the image data obtained after decoding is further subjected to distortion calibration.
  • the main processor 31 encodes the image data, and after obtaining the image encoded data, transmits the image encoded data to the slave processor 32, and the slave processor 32 performs the encoded data on the image.
  • the image encoded data transmitted by the main processor 31 is received from the processor 32
  • the image encoded data is first stored in the memory, and when the slave processor 32 is idle, the image encoded data is acquired from the memory,
  • the image encoded data is decoded to obtain image data, and the decoded image data is further subjected to distortion calibration.
  • the main processor sends the image encoded data to the slave processor, and the host processor sends the image encoded data to the slave processor through a data bus.
  • the main processor 22 and the slave processor 23 are connected via a data bus, and the main processor 22 transmits the encoded image encoded data obtained by the main processor 22 to the slave processor 23 via the data bus.
  • the main processor 31 and the slave processor 32 are connected by a data bus, and the main processor 31 transmits the encoded image encoded data obtained by the main processor 31 to the slave processor 32 via the data bus.
  • the image data is acquired by the main processor, the image data is encoded, the image encoded data is obtained, and the image encoded data is sent to the slave processor, so that the slave processor performs distortion calibration on the image data according to the image encoded data.
  • the processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the main processor.
  • the performance of the main processor is insufficient, and the code stream of the image data is large, the main can be effectively avoided. There is a problem with the bandwidth and performance of the processor.
  • FIG. 4 is a flowchart of an image processing method according to another embodiment of the present invention. As shown in FIG. 4, the method in this embodiment may include:
  • Step S401 The slave processor receives image coded data sent by the main processor, where the image coded data is obtained by encoding, by the main processor, the image data acquired by the main processor.
  • the slave processor receives the image encoded data sent by the main processor, and the slave processor receives the image encoded data sent by the main processor through the data bus.
  • the image encoded data transmitted from the main processor 22 is received from the processor 23, and the main processor 22 and the slave processor 23 are connected via a data bus, and the slave processor 23 receives the image transmitted by the main processor 22 via the data bus. Encoded data.
  • the slave processor 32 receives the transmission from the main processor 31.
  • the image encoded data, the main processor 31 and the slave processor 32 are connected by a data bus, and the slave processor 32 receives the image encoded data transmitted from the main processor 31 via the data bus.
  • Step S402 The slave processor performs distortion calibration on the image data according to the image encoded data.
  • the slave processor 23 can decode the image encoded data to obtain image data, and further perform distortion calibration on the decoded image data. .
  • the slave processor 32 may decode the image encoded data to obtain image data, and further obtain the decoded data.
  • the image data is subjected to distortion calibration.
  • the method further includes: the slave processor storing the image encoded data into a memory.
  • the slave processor performs distortion calibration on the image data according to the image encoded data
  • the method includes: the slave processor performing the image data according to the image encoded data stored in the memory. Distortion calibration.
  • the main processor 22 encodes the image data, and after obtaining the image encoded data, transmits the image encoded data to the slave processor 23, and the slave processor 23 stores the image encoded data, for example, when When receiving the image encoded data sent by the main processor 22, the processor 23 first stores the image encoded data into the memory, and when the slave processor 23 is idle, acquires the image encoded data from the memory, and performs the encoded data on the image. Decoding to obtain image data, and further performing distortion calibration on the decoded image data.
  • the main processor 31 encodes the image data, and after obtaining the image encoded data, transmits the image encoded data to the slave processor 32, and the slave processor 32 performs the encoded data on the image.
  • the image encoded data transmitted by the main processor 31 is received from the processor 32
  • the image encoded data is first stored in the memory, and when the slave processor 32 is idle, the image encoded data is acquired from the memory,
  • the image encoded data is decoded to obtain image data, and the decoded image data is further subjected to distortion calibration.
  • the image encoded data sent by the main processor is obtained by the processor, and the image encoded data is obtained by encoding, by the main processor, the image data obtained by the main processor, and the image is distorted by the processor according to the image encoded data. Calibration so that the main processor does not need to be acquired The image data is distorted immediately in the image data, which saves the performance overhead of the main processor. When the performance of the main processor is insufficient, the bandwidth and performance of the main processor can be effectively avoided.
  • FIG. 5 is a flowchart of an image processing method according to another embodiment of the present invention. As shown in FIG. 5, the method in this embodiment may include:
  • Step S501 Receive, by the processor, image encoded data sent by the main processor, where the image encoded data is obtained by encoding, by the main processor, the image data acquired by the main processor.
  • step S501 and step S401 are the same, and are not described here.
  • Step S502 The slave processor stores the image encoded data into a memory.
  • the photographing apparatus 20 further includes a memory 24.
  • the main processor 22 encodes the image data to obtain image encoded data, and then transmits the image encoded data to the slave processor 23, and the slave processor 23 stores the image encoded data in the memory 24.
  • the memory 24 is specifically a Secure Digital Memory Card (SD card), which is only a schematic description, and does not limit the specific form of the memory 24. In other embodiments, the memory 24 may be other Form of memory.
  • SD card Secure Digital Memory Card
  • the drone 30 further includes a memory 33.
  • the main processor 31 encodes the image data to obtain image encoded data, and then transmits the image encoded data to the slave processor 32, and the slave processor 32 stores the image encoded data in the memory 33.
  • the memory 33 may be a secure digital memory card (SD card), which is only a schematic description, and does not limit the specific form of the memory 24. In other embodiments, the memory 33 may be other. Form of memory.
  • Step S503 The slave processor acquires the image encoded data from a memory.
  • the image encoded data is acquired from the memory 24.
  • image encoded data is retrieved from memory 33 when slave processor 32 is relatively idle.
  • Step S504 The slave processor decodes the image encoded data to obtain the image data.
  • the image data is decoded from the processor 23 to obtain image data.
  • the image encoded data is decoded from the processor 32 to obtain image data.
  • Step S505 The slave processor performs distortion calibration on the image data to obtain calibrated image data.
  • the processor 23 performs distortion calibration on the decoded image data to obtain calibrated image data.
  • the decoded image data is subjected to distortion calibration from the processor 32 to obtain calibrated image data.
  • Step S506 The slave processor encodes the calibrated image data to obtain calibrated image encoded data.
  • the slave processor 23 further encodes the calibrated image data to obtain calibrated image encoded data.
  • the slave processor 32 further encodes the calibrated image data to obtain calibrated image encoded data.
  • Step S507 The slave processor stores the calibrated image encoded data into the memory.
  • the slave processor stores the calibrated image encoded data into the memory, including: the slave processor storing the calibrated image encoded data into the memory, and The image encoded data stored in the memory is replaced with the calibrated image encoded data.
  • the slave processor 23 further stores the calibrated image encoded data in the memory 24. Since the pre-calibrated image encoded data is stored in the memory 24, optionally, the slave processor 23 can replace the pre-calibrated image encoded data stored in the memory 24 with the calibrated image encoded data.
  • the slave processor 32 further stores the calibrated image encoded data to the memory 33. Since the pre-calibrated image encoded data is stored in the memory 33, alternatively, the slave processor 32 can replace the pre-calibrated image encoded data stored in the memory 33 with the calibrated image encoded data.
  • the embodiment performs an image on the image encoded data stored in the memory from the processor.
  • the image encoded data stored in the memory may be obtained after the main processor encodes the captured image captured by the photographing device, so that the slave processor only needs to perform image calibration on the captured image captured by the photographing device, without
  • the image data of each frame sensed by the image sensor is image-calibrated, which not only improves the calculation efficiency of the slave processor, but also the main processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, thereby saving the main
  • the performance overhead of the processor can effectively avoid the problem of bandwidth and performance of the main processor when the performance of the main processor is insufficient and the code stream of the image data is large.
  • FIG. 8 is a flowchart of an image processing method according to another embodiment of the present invention.
  • the image processing method provided by this embodiment is applied to a drone. As shown in FIG. 8, the method in this embodiment may include:
  • Step S801 acquiring image data.
  • the communication system includes a drone 90 and a ground end device 91.
  • the drone 90 includes a processor 901, a support device 902 such as a pan/tilt head, a photographing device 903, and a communication system 904, and the photographing device 903 is mounted on the drone 90 via the support device 902.
  • Processor 901 may specifically be a dedicated or other general purpose processor.
  • Communication system 904 can communicate wirelessly with ground end device 91.
  • the ground end device 91 wirelessly communicates with the drone 90 via the antenna 910.
  • the acquiring image data includes: acquiring image data collected by an image sensor of the photographing device mounted on the drone.
  • the photographing apparatus 903 may be provided with an image sensor that collects image data in real time and transmits the image data to the processor 901.
  • the acquiring image data includes: acquiring image data captured by the photographing device mounted on the drone according to a shooting control instruction.
  • the communication system 904 can receive the photographing control command transmitted by the ground end device 91, the processor 901 controls the photographing device 903 to capture image data according to the photographing control command, and the photographing device 903 can transmit the image data captured by the photographing device 903 to the processor 901.
  • Step S802 encoding the image data to obtain image encoded data.
  • the processor 901 After the processor 901 acquires the image data, the image data is encoded to obtain image encoded data.
  • Step S803 Send the image encoded data to a ground end device corresponding to the drone, so that the ground end device performs distortion calibration on the image data according to the image encoded data.
  • the processor 901 transmits the image encoded data to the ground end device 91 corresponding to the drone 90 through the communication system 904, and the ground end device 91 performs distortion calibration on the image data according to the image encoded data. Specifically, the ground end device 91 pairs The image encoded data is decoded to obtain image data, and the decoded image data is further subjected to distortion calibration.
  • the processor 901 can send the image encoded data to the ground end device 91 in real time through the communication system 904, and the ground end device 91 can decode the image encoded data in real time to obtain image data, and obtain the decoded image. The data is calibrated for distortion.
  • the image encoding data is sent to the ground end device corresponding to the drone, so that the ground end device performs distortion calibration on the image data according to the image encoded data, including: Sending the image encoded data to the ground end device corresponding to the drone, so that the ground end device stores the image encoded data, and performs the image data according to the stored image encoded data. Distortion calibration.
  • the ground terminal device 91 includes a processor 911 and a memory 912.
  • the memory 912 may be a secure digital memory card (SD card).
  • SD card secure digital memory card
  • the specific form of the memory 912, in other embodiments, the memory 912 may also be other forms of memory.
  • the processor 901 transmits the image encoded data to the ground end device 91 corresponding to the drone 90 via the communication system 904, and the processor 911 stores the image encoded data to the memory 912.
  • the slave memory 912 The image encoded data is acquired, the image encoded data is decoded to obtain image data, and the decoded image data is further subjected to distortion calibration.
  • the image data is acquired by the drone, the image data is encoded, the image encoded data is obtained, and the image encoded data is sent to the ground end device, so that the ground end device performs distortion calibration on the image data according to the image encoded data,
  • the human-machine processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the processor of the drone, when the performance of the processor is insufficient, and the code stream of the image data is large. Can effectively avoid problems with the bandwidth and performance of the processor.
  • FIG. 11 is a flowchart of an image processing method according to another embodiment of the present invention.
  • the image processing method provided in this embodiment is applied to a ground end device. As shown in FIG. 11, the method in this embodiment may include:
  • Step S1101 Receive image coded data sent by the drone, and the image coded data is obtained by encoding the image data acquired by the drone.
  • the processor 901 of the drone 90 acquires the image data captured by the photographing device 903
  • the image data is encoded to obtain image encoded data
  • the image encoded data is transmitted to the ground end device 91, the ground end.
  • the device 91 receives the image encoded data transmitted by the drone 90.
  • Step S1102 Perform distortion calibration on the image data according to the image encoded data.
  • the processor 911 of the ground end device 91 decodes the image encoded data to obtain image data, and further performs distortion calibration on the decoded image data.
  • step S1101 after receiving the image encoded data sent by the drone, step S1101 further includes: storing the image encoded data into a memory.
  • step S1102 performs distortion calibration on the image data according to the image encoded data, including: performing distortion calibration on the image data according to the image encoded data stored in the memory.
  • the processor 911 After the ground end device 91 receives the image encoded data sent by the drone 90, the processor 911 first stores the image encoded data into the memory, and when the processor 911 is idle, the image encoded data is acquired from the memory. The image encoded data is decoded to obtain image data, and the decoded image data is further subjected to distortion calibration.
  • the image encoding data sent by the UAV is received by the ground end device, and the image encoded data is obtained by encoding the image data acquired by the UAV, and the ground end device distorts the image data according to the image encoded data.
  • the calibration makes the UAV not need to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the UAV processor.
  • the performance of the UAV processor is insufficient, the UAV can effectively avoid the unmanned There is a problem with the bandwidth and performance of the machine.
  • FIG. 12 is a flowchart of an image processing method according to another embodiment of the present invention. As shown in FIG. 12, the method in this embodiment may include include:
  • Step S1201 Receive image encoded data sent by the drone, and the image encoded data is obtained by encoding, by the drone, image data acquired by the drone.
  • Step S1201 is consistent with the specific principles and implementation manners of step S1101, and details are not described herein again.
  • Step S1202 Store the image encoded data in a memory.
  • the processor 911 first stores the image encoded data in the memory 912.
  • Step S1203 Acquiring the image encoded data from a memory.
  • Image encoded data is retrieved from memory 912 when processor 911 is relatively idle.
  • Step S1204 Decoding the image encoded data to obtain the image data.
  • the processor 911 decodes the image encoded data to obtain image data.
  • Step S1205 Perform distortion correction on the image data to obtain calibrated image data.
  • the processor 911 performs distortion calibration on the decoded image data to obtain calibrated image data.
  • Step S1206 Encoding the calibrated image data to obtain calibrated image encoded data.
  • the processor 911 further encodes the calibrated image data to obtain calibrated image encoded data.
  • Step S1207 Store the calibrated image encoded data into the memory.
  • the storing the calibrated image encoded data to the memory comprises: storing the calibrated image encoded data into the memory, and encoding the image stored in the memory The data is replaced with the calibrated image encoded data.
  • Processor 911 further stores the calibrated image encoded data to memory 912. Since the pre-calibrated image encoded data is stored in the memory 912, optionally, the processor 911 may replace the pre-calibrated image encoded data stored in the memory 912 with the calibrated image encoded data.
  • the image encoding data stored in the memory is image-calibrated by the ground end device, and the image encoded data stored in the memory may be obtained by the UAV encoding the captured image captured by the photographing device, so that the ground end is obtained.
  • the device only needs to be photographed on the shooting device
  • the image is captured for image calibration without image calibration for each frame of image data sensed by the image sensor, which not only improves the computational efficiency of the ground end device, but also does not require the drone to instantly acquire the image data.
  • the image data is distorted and calibrated, which saves the performance overhead of the UAV processor. When the performance of the UAV processor is insufficient, and the code stream of the image data is large, the processor of the UAV can be effectively avoided. There is a problem with bandwidth and performance.
  • FIG. 13 is a structural diagram of an image processing apparatus according to an embodiment of the present invention.
  • the image processing apparatus 130 includes a main processor 131 and a slave processor 132.
  • the main processor 131 is configured to: acquire image data; encode the image data to obtain image encoded data; and send the image encoded data to a slave processor, so that the slave processor 132 encodes the data according to the image.
  • the image data is subjected to distortion calibration.
  • the main processor 131 when acquiring the image data, is specifically configured to: acquire image data collected by the image sensor.
  • the main processor 131 when acquiring the image data, is specifically configured to: acquire a captured image corresponding to the shooting control instruction.
  • the main processor 131 sends the image encoded data to the slave processor, so that when the slave processor 132 performs distortion calibration on the image data according to the image encoded data, specifically, the image is used to:
  • the encoded data is transmitted to the slave processor to cause the slave processor 132 to store the image encoded data, and to perform distortion calibration on the image data based on the stored image encoded data.
  • the method is specifically configured to: send the image encoded data to the slave processor through a data bus.
  • the image data is acquired by the main processor, the image data is encoded, the image encoded data is obtained, and the image encoded data is sent to the slave processor, so that the slave processor performs distortion calibration on the image data according to the image encoded data.
  • the processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the main processor.
  • the performance of the main processor is insufficient, and the code stream of the image data is large, the main can be effectively avoided. There is a problem with the bandwidth and performance of the processor.
  • FIG. 14 is a structural diagram of an image processing apparatus according to another embodiment of the present invention; as shown in FIG. 14, the image processing apparatus 140 includes a main processor 141 and a slave processor 142.
  • the slave processor 142 is configured to: receive image coded data sent by the main processor, where the image coded data is obtained by the main processor 141 to encode the image data obtained by the main processor 141; and the image is encoded according to the image coded data. The data is calibrated for distortion.
  • the processor 142 is further configured to: store the image encoded data into a memory; and correspondingly, the slave processor 142 pairs the image according to the image encoded data.
  • the data is subjected to distortion calibration, specifically, the image data is subjected to distortion calibration according to the image encoded data stored in the memory.
  • the method is specifically configured to: acquire the image encoded data from a memory; and encode the data into the image. Decoding to obtain the image data; performing distortion correction on the image data to obtain calibrated image data; encoding the calibrated image data to obtain calibrated image encoded data; encoding the calibrated image Data is stored to the memory.
  • the method is specifically configured to: store the calibrated image encoded data into the memory, and store the memory in the memory The image encoded data is replaced with the calibrated image encoded data.
  • the method when receiving the image encoded data sent by the main processor from the processor 142, the method is specifically configured to: receive the image encoded data sent by the main processor by using a data bus.
  • the image encoded data stored in the memory may be obtained by the main processor encoding the captured image captured by the photographing device, so that the processing is performed. It only needs to perform image calibration on the captured image captured by the shooting device without image calibration for each frame of image data sensed by the image sensor, which not only improves the calculation efficiency of the slave processor, but also does not support the main processor. It is necessary to perform distortion calibration on the image data immediately when the image data is acquired, thereby saving the performance of the main processor. Overhead, when the performance of the main processor is insufficient, and the code stream of the image data is large, the bandwidth and performance of the main processor can be effectively avoided.
  • An embodiment of the present invention provides a photographing apparatus including: an image sensor and the image processing apparatus 130 or the image processing apparatus 140 described in the above embodiments.
  • An embodiment of the present invention provides a drone that includes: a body, a power system, a photographing device, and an image processing device 130 or an image processing device 140 according to the above embodiments, wherein the power system is installed in the The body is for providing power; the photographing device is mounted on the body for collecting image data.
  • Embodiments of the present invention provide a drone.
  • 15 is a structural diagram of a drone according to an embodiment of the present invention.
  • the drone 150 includes: a fuselage, a power system, a photographing device 1501, and one or more first processors 1502. a communication interface 1503, wherein a power system is mounted on the airframe for providing power, the power system comprising at least one of: a motor 1507, a propeller 1506, and an electronic governor 1517, the power system being mounted on the fuselage Used to provide flight power.
  • the photographing device 1501 is mounted to the body through a support device 1504 such as a pan/tilt.
  • the first processor 1502 is configured to: acquire image data collected by the photographing device; and encode the image data to obtain image encoded data.
  • the first communication interface 1503 is configured to: send the image encoded data to a ground end device corresponding to the drone, so that the ground end device performs distortion calibration on the image data according to the image encoded data.
  • the first processor 1502 when acquiring the image data collected by the photographing device, is specifically configured to: acquire image data collected by the image sensor of the photographing device 1501.
  • the first processor 1502 when acquiring the image data collected by the photographing device, is specifically configured to: acquire image data captured by the photographing device 1501 according to the photographing control instruction.
  • the first communication interface 1503 is configured to send the image encoded data to a ground end device corresponding to the drone, so that the ground end device distorts the image data according to the image encoded data.
  • the method is specifically configured to: send the image coded data to a ground end device corresponding to the drone, so that the ground end device performs the image coded data And storing, and performing distortion calibration on the image data according to the stored image encoded data.
  • the image data is acquired by the drone, the image data is encoded, the image encoded data is obtained, and the image encoded data is sent to the ground end device, so that the ground end device performs distortion calibration on the image data according to the image encoded data,
  • the human-machine processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the processor of the drone, when the performance of the processor is insufficient, and the code stream of the image data is large. Can effectively avoid problems with the bandwidth and performance of the processor.
  • FIG. 16 is a structural diagram of a ground end device according to an embodiment of the present invention.
  • the ground end device 160 includes: a second communication interface 161 and a second processor 162.
  • the second communication interface 161 is configured to receive an unmanned person.
  • Image encoded data sent by the machine, the image encoded data is obtained by encoding the image data acquired by the drone; the second processor 162 is configured to perform the image data according to the image encoded data. Distortion calibration.
  • the second processor 162 is further configured to: store the image coded data into the memory; correspondingly, the second processor 162 is configured according to the When the image encoded data is subjected to distortion calibration of the image data, specifically, the image data is subjected to distortion calibration according to the image encoded data stored in the memory.
  • the method is: acquiring the image encoded data from a memory; and encoding the image. Decoding the data to obtain the image data; performing distortion correction on the image data to obtain calibrated image data; encoding the calibrated image data to obtain calibrated image encoded data; and calibrating the image
  • the encoded data is stored to the memory.
  • the method is: storing the calibrated image encoded data into the memory, and storing the The stored image encoded data is replaced with the calibrated image encoding data.
  • the image encoding data stored in the memory is image-calibrated by the ground end device, and the image encoded data stored in the memory may be obtained by the UAV encoding the captured image captured by the photographing device, so that the ground end is obtained.
  • the device only needs to perform image calibration on the captured image captured by the shooting device without performing image calibration on each frame of image data sensed by the image sensor, which not only improves the calculation efficiency of the ground end device, but also does not It is necessary to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the processor of the drone, and when the performance of the processor of the drone is insufficient, and the code stream of the image data is large, Effectively avoid problems with the bandwidth and performance of the drone's processor.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
  • the above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium.
  • the above software functional unit is stored in a storage medium and includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, or A network device or the like) or a processor performs part of the steps of the method described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

Un mode de réalisation de la présente invention concerne un procédé de traitement d'image, un véhicule aérien sans pilote (UAV) et un dispositif d'extrémité au sol. Le procédé comprend : l'acquisition par un processeur principal de données d'image ; le codage par le processeur principal de données d'image afin d'obtenir des données codées d'image ; et la transmission par le processeur principal des données codées d'image à un processeur esclave afin de permettre au processeur esclave d'effectuer un étalonnage de distorsion sur les données d'image en fonction des données codées d'image. Dans le mode de réalisation de la présente invention, les données d'image sont acquises par le processeur principal, et les données d'image sont codées afin d'obtenir les données codées d'image, puis le processeur principal transmet les données codées d'image à un processeur esclave afin de permettre au processeur esclave d'effectuer un étalonnage de distorsion sur les données d'image en fonction des données codées d'image. Le processeur principal n'a pas besoin d'effectuer un étalonnage de distorsion sur les données d'image immédiatement après l'acquisition des données d'image. Ainsi, le surdébit de performance du processeur principal est évité. Lorsque la performance du processeur principal est insuffisante et lorsque le pixel de flux codé des données d'image est grand, des problèmes sur la largeur de bande et les performances du processeur principal peuvent être efficacement évités.
PCT/CN2017/113657 2017-11-29 2017-11-29 Procédé de traitement d'image, équipement de photographie, véhicule aérien sans pilote et dispositif d'extrémité au sol WO2019104558A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201780029120.6A CN109417594A (zh) 2017-11-29 2017-11-29 图像处理方法、拍摄设备、无人机及地面端设备
PCT/CN2017/113657 WO2019104558A1 (fr) 2017-11-29 2017-11-29 Procédé de traitement d'image, équipement de photographie, véhicule aérien sans pilote et dispositif d'extrémité au sol

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/113657 WO2019104558A1 (fr) 2017-11-29 2017-11-29 Procédé de traitement d'image, équipement de photographie, véhicule aérien sans pilote et dispositif d'extrémité au sol

Publications (1)

Publication Number Publication Date
WO2019104558A1 true WO2019104558A1 (fr) 2019-06-06

Family

ID=65463389

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/113657 WO2019104558A1 (fr) 2017-11-29 2017-11-29 Procédé de traitement d'image, équipement de photographie, véhicule aérien sans pilote et dispositif d'extrémité au sol

Country Status (2)

Country Link
CN (1) CN109417594A (fr)
WO (1) WO2019104558A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1780359A (zh) * 2004-11-18 2006-05-31 北京大学 无人机遥感平台中的cmos成像与信息处理系统
CN102419171A (zh) * 2011-08-10 2012-04-18 王桥生 一种基于无人机航拍的灾害监测电子勾绘系统
EP2728308A2 (fr) * 2012-10-31 2014-05-07 Kabushiki Kaisha Topcon Photogrammétrie aérienne et système photogrammétrique aérien
CN105049812A (zh) * 2015-08-07 2015-11-11 清华大学深圳研究生院 一种无人机便携式地面站处理方法及系统
CN106454209A (zh) * 2015-08-06 2017-02-22 航天图景（北京）科技有限公司 基于时空信息融合技术的无人机应急快反数据链系统及方法

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101207825A (zh) * 2007-09-28 2008-06-25 深圳市迈科龙电子有限公司 图像处理平台
CN102724395B (zh) * 2012-06-26 2015-02-04 公安部第三研究所 一种采用双核控制的网络摄像机
JP5982588B2 (ja) * 2013-02-05 2016-08-31 クアルコム，インコーポレイテッド 保証された過渡期限とともに中央処理装置の電力を制御するためのシステムおよび方法
US9049442B2 (en) * 2013-03-15 2015-06-02 Canon Kabushiki Kaisha Moving image encoding apparatus and method for controlling the same
US20150341541A1 (en) * 2014-05-23 2015-11-26 Tikeswar Naik Methods and systems of remote acquisition of digital images or models
CN204291227U (zh) * 2014-11-28 2015-04-22 四川中软科技有限公司 视频信息处理装置
KR102280543B1 (ko) * 2014-12-01 2021-07-26 삼성전자주식회사 전자 디바이스의 작업 수행 장치 및 방법
CN105516604A (zh) * 2016-01-20 2016-04-20 陈昊 一种航拍视频分享方法和系统
CN106341571A (zh) * 2016-08-18 2017-01-18 深圳市金立通信设备有限公司 一种图像处理方法以及终端
CN106341667A (zh) * 2016-11-10 2017-01-18 广西师范大学 基于无人机的三维全景视频远程监控系统及图像采集控制方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1780359A (zh) * 2004-11-18 2006-05-31 北京大学 无人机遥感平台中的cmos成像与信息处理系统
CN102419171A (zh) * 2011-08-10 2012-04-18 王桥生 一种基于无人机航拍的灾害监测电子勾绘系统
EP2728308A2 (fr) * 2012-10-31 2014-05-07 Kabushiki Kaisha Topcon Photogrammétrie aérienne et système photogrammétrique aérien
CN106454209A (zh) * 2015-08-06 2017-02-22 航天图景（北京）科技有限公司 基于时空信息融合技术的无人机应急快反数据链系统及方法
CN105049812A (zh) * 2015-08-07 2015-11-11 清华大学深圳研究生院 一种无人机便携式地面站处理方法及系统

Also Published As

Publication number Publication date
CN109417594A (zh) 2019-03-01

Similar Documents

Publication Publication Date Title
US20180063512A1 (en) Image streaming method and electronic device for supporting the same
CN109076194B (zh) 一种视频处理方法、设备、飞行器、系统及存储介质
US20190110077A1 (en) Method and electronic device for processing raw image acquired through camera by using external electronic device
CN103945001A (zh) 一种图片分享方法及装置
CN109756763B (zh) 用于基于优先级处理图像的电子装置及其操作方法
EP3664442A1 (fr) Procédé et dispositif de transmission d'image, plate-forme mobile, dispositif de surveillance, et système
US11109012B2 (en) Carriage of PCC in ISOBMFF for flexible combination
KR102397924B1 (ko) 이미지 보정 방식 및 이미지의 특징 정보에 기반하여 이미지를 보정하는 전자 장치 및 방법
WO2019019128A1 (fr) Procédé, dispositif, et système de transmission vidéo
CN111788603A (zh) 用于以第二图像处理方案在外部电子设备中校正以第一图像处理方案校正的图像的电子设备和方法
US20190251670A1 (en) Electronic device and method for correcting images using external electronic device
WO2019091191A1 (fr) Procédé et appareil de traitement de données
US20210166354A1 (en) Chrominance Denoising
WO2018058575A1 (fr) Procédé et système de traitement de données d'images de vol et dispositif au sol
WO2018176303A1 (fr) Procédé, système et dispositif d'émission et de réception de vidéo, et véhicule aérien sans pilote
US11393078B2 (en) Electronic device and method for correcting image on basis of image transmission state
WO2022000350A1 (fr) Procédé de transmission vidéo, plateforme mobile, équipement terminal, système de transmission vidéo et support d'enregistrement
WO2019104558A1 (fr) Procédé de traitement d'image, équipement de photographie, véhicule aérien sans pilote et dispositif d'extrémité au sol
US11330227B2 (en) Electronic device for compressing image acquired by using camera, and operation method therefor
CN110771174A (zh) 视频处理的方法、地面控制端及存储介质
US11425300B2 (en) Electronic device and method for processing image by electronic device
US11115605B2 (en) Electronic device for selectively compressing image data according to read out speed of image sensor, and method for operating same
KR102460123B1 (ko) 이미지 센서를 이용한 이미지 획득 과정에서 생성된 압축 속성을 이용하여 이미지를 압축하는 전자 장치 및 그 동작 방법
EP4297394A1 (fr) Dispositif électronique comprenant une caméra
WO2019090473A1 (fr) Procédé et système d'affichage pour image panoramique, et procédé de traitement et station au sol pour image panoramique

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17933798

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17933798

Country of ref document: EP

Kind code of ref document: A1