EP4612902A1 - Système de visualisation à distance en temps réel d'un site - Google Patents

Système de visualisation à distance en temps réel d'un site

Info

Publication number
EP4612902A1
EP4612902A1 EP23786198.4A EP23786198A EP4612902A1 EP 4612902 A1 EP4612902 A1 EP 4612902A1 EP 23786198 A EP23786198 A EP 23786198A EP 4612902 A1 EP4612902 A1 EP 4612902A1
Authority
EP
European Patent Office
Prior art keywords
drone
micro
observer
computing device
personal computing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP23786198.4A
Other languages
German (de)
English (en)
Inventor
Jiang Zhou
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority claimed from PCT/EP2023/077358 external-priority patent/WO2024094378A1/fr
Publication of EP4612902A1 publication Critical patent/EP4612902A1/fr
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
    • H04N7/185—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source from a mobile camera, e.g. for remote control
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U10/00—Type of UAV
    • B64U10/80—UAVs characterised by their small size, e.g. micro air vehicles [MAV]
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2201/00—UAVs characterised by their flight controls
    • B64U2201/20—Remote controls

Definitions

  • This invention relates to a system for real-time remote viewing of a site and, in particular, to a system employing drones at the site.
  • Real-time remote viewing can be accomplished using teleconferencing applications such as WhatsApp ® and Skype ® , but these solutions are unwieldy, where a remote viewer has a requirement for particular viewing angles and locations at the site, as he is required to verbally instruct an observer at the site to manually move around his camera to each location.
  • UAVs unmanned aerial vehicles
  • Such systems rely on proprietary, secure, high-speed networks for remote manipulation of a drone.
  • These drones are launched and controlled by a user, such as an infantry soldier, who tracks the drone on a console, but is not necessarily in visual contact with the drone.
  • US Patent No. 10,382,539 B1 relates to methods and apparatus for data control and transfer with an unmanned aerial vehicle, such as a drone.
  • a system which has a drone session server to collect drone session information.
  • a drone user machine is in a client relationship with the drone session server.
  • a drone control machine is in a client relationship with the drone session server and is connected through a network with the drone user machine, and in a peer-to peer relationship with the drone user machine.
  • the drone control machine is configured to relay video data from a drone to the drone user machine through a drone console and the drone control machine and the network via a peer-to-peer connection, wherein the drone console is within visual presence of the drone.
  • the drone control machine sends autopilot commands to the drone console to lift off the drone and direct the drone into a three-dimensional geographical fence.
  • the drone control machine evaluates user commands collected by the drone user machine, which drone user machine is operated by a user that is not within visual presence of the drone. These user commands are relayed to the drone control machine via the peer-to-peer connection to produce enforced limits commands through the drone console to maintain the drone within the three-dimensional geographical fence.
  • the drone control machine sends autopilot commands to the drone to transport the drone from the three-dimensional geographical fence to a land site to complete a drone session.
  • the drone control console is used by a drone operator to fly a drone that is in the visual presence of the operator.
  • the operator is at the drone operator machine, which is not in the visual presence of the drone.
  • the drone user machine may be used to pilot a drone thousands of miles away.
  • the user must be a trained pilot in order to satisfy the flight regulations for the country in which the drone is being flown.
  • the remote drone piloting system includes a pilot endpoint system comprising a pilot endpoint and a controller connected to the pilot endpoint.
  • the system includes a control endpoint system including a control endpoint, a signal adaptor connected to the control endpoint, and a transmitter connected to the signal adaptor.
  • a drone is arranged to communicate drone video data to the control endpoint system.
  • a remote bridge including a server is arranged to connect the pilot endpoint and the control endpoint using an internet protocol when the pilot endpoint and the control endpoint are located at different geographic locations such that the drone operating data and drone video data are communicated amongst the pilot endpoint, control endpoint, and drone in real-time.
  • the pilot controller is a handheld controller that includes controls for piloting the drone.
  • the pilot controller allows the remote pilot utilizing the pilot system to execute control of the drone in real-time by providing commands through the connection to the control endpoint system. Such commands operate to allow control of the flight of the drone, use of a camera on the drone and use of any other onboard drone systems or functionalities.
  • a peer-to-peer audio connection allows the drone pilot and the remote pilot to communicate directly.
  • the drone pilot who controls the drone through the pilot endpoint and the remote pilot, who controls the drone through the control endpoint must be trained to a level that will satisfy the flight regulations for the country in which the drone is being flown.
  • the invention provides a system for real-time remote viewing of a site, the system comprising a micro-drone, a camera mounted on the micro-drone, an observer’s personal computing device, with a system client software package installed thereon, with both the observer’s personal computing device and the micro-drone being locatable at the site, in use, and in communication with each other by a wireless connection, a remote viewer’s personal computing device, a system server, which system server manages the operation of the system, processes data collected by the micro-drone during a flight, and communicates the processed data to the observer’s personal computing device and the remote viewer’s personal computing device, a Web Real-Time Communication (WebRTC) connection between the observer’s personal computing device and the system server for audio, video and data transmission, a WebRTC connection between the remote viewer’s personal computing device and the system server, for audio, video and data transmission,
  • WebRTC Web Real-Time Communication
  • the micro-drone is a low-speed, palm-size micro-drone, with a gross weight of ⁇ 250g and flown by either the remote viewer or the observer within Visual Line of Sight (VLOS) of the observer, with control of the micro-drone by the remote viewer being limited to three commands being go-forward, yaw-left, and yaw-right, with the observer being able to take-over control of the micro-drone at any time during the flight, and
  • VLOS Visual Line of Sight
  • the word “device” in the phrases “observer’s personal computing device” and remote viewer’s computing device”, in this context, means a laptop, tablet, smartphone or the like.
  • server herein encompasses one or more physical or virtual, cloud-based servers, or a combination of both physical and virtual servers.
  • An advantage of the system for real-time remote viewing of a site in accordance with the invention is that, as the micro-drone is being flown by either the remote viewer or the observer within Visual Line of Sight (VLOS) of the observer, neither the observer nor the remote viewer is required to be a trained drone pilot.
  • the remote viewer can be located anywhere in the world as long as he can connect to the Internet.
  • the micro-drone is flown, in accordance with the system in a low or no risk category to third parties.
  • operation of the micro-drone falls within EU regulation Open category A1 C0 (EU Commission Implementing Regulation (EU)2019/947) and can be used by anyone from the public without professional training.
  • a further advantage of the system is that the remote viewer and the observer can communicate with each other directly in real-time by the WebRTC connection. Such communication can include the transmitting of micro-drone control commands from the remote viewer’s personal computing device to the observer’s personal computing device.
  • the server can intervene in a remote viewing session by sending top priority commands such as force-landing, close-camera, etc. to the observer’s personal computing device, which will suppress the commands from the remote viewer’s personal computing device and stop illicit use of remote viewing.
  • top priority commands such as force-landing, close-camera, etc.
  • the maximum speed of the micro-drone is limited to within a range of 80 to 120cm/s, and the micro-drone is required to be flown within a radial distance of between 2 to 10m from the location of the observer’s personal computer device.
  • micro-drone specification is that it falls well within EU regulation Open category A1 C0.
  • a sensor is mounted on the micro-drone and the data collected by the micro-drone during a flight includes data from the camera and data from the sensor.
  • An advantage of this aspect of the invention is that the inclusion of a sensor means that a combination of video data from the micro-drone camera with data collected by a sensor enhances the data being processed by the system server, resulting in additional information for use by the remote observer.
  • An advantage of this aspect of the invention is that, where the micro-drone has Bluetooth ® , pairing the micro-drone with the observer’s personal computing device is straightforward.
  • the observer’s personal computing device first sets up a Bluetooth ® connection with the micro-drone and sends the micro-drone the Wi-Fi credentials, which are used by the micro-drone, to join a Wi-Fi network in which the observer’s personal computing device and the micro-drone exchange IP addresses and set up Wi-Fi connections for video and data transmission.
  • the system has a relay box, locatable in proximity to the observer, which relay box is a compact computing device with at least two Wi-Fi interfaces and a Bluetooth ® interface, and which relay box enables a wireless connection between the micro-drone and the relay box and a Wi-Fi connection and a Bluetooth ® connection between the relay box and the observer’s personal computing device.
  • a relay box locatable in proximity to the observer, which relay box is a compact computing device with at least two Wi-Fi interfaces and a Bluetooth ® interface, and which relay box enables a wireless connection between the micro-drone and the relay box and a Wi-Fi connection and a Bluetooth ® connection between the relay box and the observer’s personal computing device.
  • An advantage of this aspect of the invention is that communication between the micro-drone and the observer’s personal computing device can be achieved by pairing the observer’s personal computing device with the relay box by way of a Bluetooth ® connection.
  • the observer’s personal computer device then sends to the relay box Wi-Fi credentials which are used by a Wi-Fi interface of the relay box to join a Wi-Fi network where the observer’s personal computing device and the relay box exchange IP addresses and set up Wi-Fi connections.
  • the relay box includes a USB interface through which the relay box is connectable to the observer’s personal computing device by a USB cable.
  • An advantage of this aspect of the invention is that no Wi-Fi is required for Internet connection.
  • the relay box connects to the observer’s personal computing device through the USB cable and sets up a Wi-Fi connection with the micro-drone and the Internet is accessible through a 4G/5G network connection on the observer’s personal computing device.
  • the system server processes the data collected by the micro-drone by employing machine learning models, pretrained with large amounts of high-quality measurements, to rectify and enhance the data captured before it is transmitted to the observer’s personal computing device and the remote viewer’s personal computing device.
  • An advantage of this aspect of the invention is that the raw data from the micro-drone, which is of low quality can be enhanced using the machine learning modules.
  • data collected by a basic camera and sensor mounted on the micro-drone can still provide accurate high-grade data to both the remote viewer’s personal computer and the observer’s personal computer device to provide real-time remote viewing of a quality that one would not expect from basic low-cost hardware.
  • the system server spatially and temporally co-locates the enhanced data for each flight, for further analysis by the system server.
  • An advantage of this aspect of the invention is that, for example, when creating a map of a large area, smaller maps generated from separate micro-drone flights can be stitched together using computer vision techniques.
  • This analysis may be accomplished by real-time, on-line processing by the system server or by off-line processing.
  • the analysis can also be sent to a third party’s server for further processing.
  • the position of the micro-drone may be calculated from the GPS coordinates of the observer’s personal computing device.
  • GPS Global Positioning System
  • the advantage of using the GPS coordinates of the observer’s personal computing device is that a basic micro-drone without GPS can be used by the system.
  • the system server for geofencing and a warning is communicated to the observer’s personal computing device and the remote user’s computing device if the observer enters a restricted area or the micro-drone enters into restricted airspace.
  • An advantage of this aspect of the invention is that the location of the micro-drone can be inferred from the GPS coordinates of the observer’s personal computing device.
  • the micro-drones are programmed to fly autonomously and the system client software installed on the observer’s personal computing device monitors the flight of each micro-drone and implements drone swarming and collision avoidance measures, including sending a message to the observer’s personal computing device alerting the observer to the flight of a particular micro-drone.
  • An advantage of this aspect of the invention is that, in such a situation, as with more micro-drones in the air at the same time, it would be next to impossible for the observer to keep track of each micro-drone visually. So, monitoring of the micro-drones by the system client software, including alerting the observer to a potential hazard, makes such multi drone operations possible.
  • the system provides Representational state transfer (REST) Application Programming Interfaces (APIs) and Webhooks to third parties for system integrations.
  • REST Representational state transfer
  • APIs Application Programming Interfaces
  • Webhooks to third parties for system integrations.
  • An advantage of this aspect of the invention is that it facilitates the integration of a customer’s in-house system with the system for real-time remote viewing in accordance with the invention. This integration is feasible as no trained drone pilots are required for operation of the system.
  • the APIs are authorized with JavaScript Object Notation (JSON) Web Tokens RFC 7519.
  • JSON JavaScript Object Notation
  • JSON Web Tokens An advantage of the use of JSON Web Tokens is that their use is a simple but secure method for authorising use of the system by a remote viewer through his company’s in-house system.
  • Each flight of the micro-drone is also authorized by a token containing information about the Time to Live of the remote viewing session, the session id, and the micro-drone registration number, such that unauthorized flights can be prevented, and inappropriate use of remote viewing can be traced.
  • FIG. 1 is schematic representation of a system for real-time remote viewing of a site in accordance with the invention.
  • the system 10 comprises a micro-drone 11, a camera 12 mounted on the micro-drone 11, an observer’s personal computing device 13, operable by an observer, with both the observer’s personal computing device 13 and the micro-drone 11 being locatable at the site.
  • the system 10 also includes a remote viewer’s personal computing device 14, operable by a remote viewer, and a system server 15, which system server 15 manages the operation of the system 10, processes data collected by the micro-drone 11 during a flight, and communicates the processed data to the observer’s personal computing device 13 and the remote viewer’s personal computing device14.
  • the system 10 has a Web Real-Time Communication (WebRTC) connection, indicated by dashed line 16, between the observer’s personal computing device 13 and the system server 15 for audio, video and data transmission, and a WebRTC connection, indicated by dashed line 17, between the remote viewer’s personal computing device 14 and the system server 15, for audio, video and data transmission.
  • WebRTC Web Real-Time Communication
  • the system 10 also has a WebRTC connection, indicated by dashed line 18, between the remote viewer’s personal computing device 14 and observer’s personal computing device 13 for audio, video and data transmission directly.
  • the micro-drone 11 is a low-speed, palm-size micro-drone, with a gross weight of ⁇ 250g and flown by either the remote viewer or the observer within Visual Line of Sight (VLOS) of the observer, with control of the micro-drone 11 by the remote viewer being limited to three commands, being go-forward, yaw-left, and yaw-right, with the observer being able to take-over control of the micro-drone 11 at any time during the flight.
  • the micro-drone used is a Tello manufactured by DJI, DJI Headquarters Building, No.55 Xianyuan Road, Nanshan District, Shenzhen, China.
  • the Tello micro-drone has a gross weight of 85g and is configured so that the maximum speed of the micro-drone 11 is limited to within a range of 80 to 120cm/s, and the micro-drone 11 is required to be flown within a radial distance of between 2 to 10m from the location of the observer’s personal computer device 13.
  • a sensor 19 is mounted on the micro-drone 11 and the data collected by the micro-drone 11 during a flight includes data from the camera 12 and data from the sensor 19, if required.
  • the micro-drone 11 does not have Bluetooth ® capability so the system 10 has a relay box 20, locatable in proximity to the observer, which relay box 20 is a compact computing device with at least two Wi-Fi interfaces 21, 22 and a Bluetooth ® interface 23, and which relay box 20 enables a wireless connection, indicated by dashed line 24, between the micro-drone 11 and the relay box 20 and wireless connections, indicated by dashed line 25, between the relay box 20 and the observer’s personal computing device 13.
  • the relay box 20 is not required as communication between the observer’s personal computing device 13 and the micro-drone 11 can be by a direct Wi-Fi connection, indicated by dashed line 26.
  • the system server 15 processes the data collected by the micro-drone 11 by employing machine learning models, pretrained with large amounts of high-quality measurements, to rectify and enhance the data captured before it is transmitted to the observer’s personal computing device 13 and the remote viewer’s personal computing device 14.
  • system server 15 spatially and temporally co-locates the enhanced data for each flight, for further analysis by the system server 15.
  • the micro-drone 11 does not have Global Positioning System (GPS) capability. However, the position of the micro-drone 11 may be calculated from the GPS coordinates of the observer’s personal computing device 13. In use, as the observer moves around the site the location of the observer’s personal computing device 13 is communicated to the system server 15 for geofencing and a warning is communicated to the observer’s personal computing device 13 and the remote user’s computing device 14 if the observer enters a restricted area or the micro-drone 11 enters into restricted airspace.
  • GPS Global Positioning System
  • the micro-drone 11 is positioned alongside the observer.
  • the observer connects to the micro-drone 11 through Bluetooth ® and Wi-Fi using a client software installed on the observer’s personal computing device 13.
  • the software sets up the wireless connection to the micro-drone in any Wi-Fi network with the following steps:
  • the micro-drone 11 may not have Bluetooth ® connection but only Wi-Fi on board and is configured to a fixed IP address (as is the case with the embodiment of ).
  • a relay box 20 is introduced.
  • the relay box 20 is a small battery-powered single-board computer with two Wi-Fi interfaces 21, 22 and one Bluetooth ® interface 23.
  • One Wi-Fi interface 21 connects to the micro-drone Wi-Fi with a given SSID and password.
  • An IP address ip-drone is assigned to the relay box 20 while the micro-drone 11 has a fixed IP.
  • Another Wi-Fi interface 22 together with the Bluetooth ® interface 23 is used to join the same Wi-Fi network that the observer’s personal computing device 13 is using, and obtains an IP address ip-observer.
  • the second Wi-Fi connection 22 is set up with the same steps as setting up a micro-drone 11 connection to the observer’s personal computing device 13 directly, except by replacing the micro-drone 11 with the relay box 20 in the steps.
  • the control commands, drone videos, and sensor data can be relayed between the micro-drone 11 and the observer’s personal computing device 13.
  • the relay box 20 includes a USB interface through which the relay box is connectable to the observer’s personal computing device 13 by a USB cable.
  • the relay box 20 connects to the observer’s personal computing device 13 through the USB cable and sets up a Wi-Fi connection with the micro-drone 11 and the Internet is accessible through a 4G/5G network connection on the observer’s personal computing device 13.
  • the remote viewer connects to the system server 15 through the WebRTC connection 17 from the remote viewer’s personal computing device 14, using the Internet.
  • the observer connects to the system server 15 through the WebRTC connection 16 from through a platform client software installed on the observer’s personal computing device 13, using the Internet.
  • the remote viewer or viewers, as the case may be, and the observer are located at different geographic locations. With WebRTC connections 16 and 17, the remote viewer(s) and the observer can have a two-way videoconference in sound and vision.
  • the remote viewer(s) can also communicate with each other in the videoconferencing.
  • the system server authenticates, relays, and processes data between the remote viewer(s) and the observer.
  • the remote viewer(s) and the observer can communicate directly with WebRTC connection 18. Communication among the remote viewer(s) can also be set up directly using WebRTC without being bridged by the system server.
  • the video captured by the micro-drone camera 12 is streamed to the observer’s personal computing device 13.
  • the video is streamed to the relay box 20 via the Wi-Fi connection 24 and then forwarded to the observer’s personal computing device 13 by the relay box 20 via the Wi-Fi connection 25.
  • the observer’s personal computing device 13 receives the micro-drone video and forwards it to the remote viewers’ personal computing device 14 via WebRTC connections 16 and 17, or via WebRTC connection 18, directly.
  • the micro-drone 11 video received by the remote viewers’ personal computing device 14 is displayed on a system web interface. Micro-drone control buttons are overlayed on the displayed video.
  • the remote viewers can then control the micro-drone 11 flight using these web buttons, and each remote viewer, one at a time, is given sole ownership over these buttons. No physical control console such as a joystick is needed.
  • the control commands executed from those web buttons are transmitted to the observer’s personal computing device 13 via the WebRTC connections 16 and 17, or 18 directly, then forwarded to the micro-drone via the Wi-Fi connections 24 and 25 or 26 directly.
  • the system’s client software installed on the observer’s personal computing device 13 also displays the micro-drone video on the software interface to ease the flight monitoring for the observer while the observer is video conferencing with the remote viewer(s) at the same time.
  • the client software can take flight control inputs from a keyboard, a computer mouse, a joystick, a touch screen, voice recognition, gesture recognition, etc., and these control commands executed by the observer are forwarded to the micro-drone 11 via the Wi-Fi connections 24 and 25 or 26 directly.
  • the observer and the micro-drone 11 are at the same site, and the observer visually monitors the micro-drone 11 flight.
  • the observer can interfere with the remote control from the remote viewer(s), by executing control commands from the client software, when a potential hazard is recognized by the observer.
  • These control commands from the observer have higher priorities than those control commands from the remote viewer(s). In other words, when both commands from the observer and the remote viewers present, the commands from the observer will be forwarded to the drone.
  • the micro-drone 11 flight status and captured sensor data are also sent to the observer’s personal computing device 13 via the Wi-Fi connections 24 and 25 or 26, directly. Some of the flight status is displayed by the client software by overlaying the parameters on the drone video.
  • the micro-drone 11 flight status and the sensor data are transmitted to the system server 15 together with the drone video via the WebRTC connection 16 for processing using computer vision and machine learning algorithms running on the system server 15.
  • the processed results can be sent to both the observer and the remote viewer(s) via the WebRTC connections 16 and 17 and be displayed on the web interface and the client software interface.
  • the platform deploys micro-drones in the service, which are palm-size with a weight of less than 250g.
  • This weight includes the camera and all the other sensors mounted on the micro-drone 11.
  • the speed of the micro-drone is fixed under 100 cm/s which is a very low speed.
  • a “Stop, Look, Fly” scheme is employed.
  • the implementation of the scheme limits the flight control for the remote viewer(s) to only three commands, go-forward, yaw-left, and yaw-right, which are executed mutually exclusively. No other commands, such as go-backward, speed-change, roll, follow-me, etc ., are available.
  • a remote viewer can watch the area in front of the micro-drone 11 and execute the go-forward command when there is no potential hazard. To change the direction of the micro-drone 11, the remote viewer waits until the micro-drone 11 stops and hovers in the air, then executes a yaw-left or yaw-right command, and checks around. When there is no risk in one direction the remote viewer can give the go-forward command and continue flying in the new direction. Such a procedure repeats during the entire remote piloting of the micro-drone 11.
  • the micro-drone 11 must fly within Visual Line of Sight (VLOS) of an operator.
  • VLOS Visual Line of Sight
  • an observer situates alongside the micro-drone to maintain visual contact with the micro-drone 11 during a flight.
  • the observer can be anyone above 16 years old and it is not necessary for him to be a trained pilot.
  • the distance between the observer and the micro-drone 11 is measured based on the strength of the Bluetooth ® and Wi-Fi signals, in the absence of (GPS) capability on the micro-drone 11.
  • the micro-drone 11 is flown around the observer within a pre-set radius. The radius should be less than 10 meters due to the Bluetooth ® effective range for most Bluetooth ® devices in practice.
  • micro-drone 11 control commands provided in the client software for the observer. Observing the surroundings allows the observer to decide when and where to execute take-off or land commands. Besides the go-forward and yaw commands, the observer can also control the micro-drone using go-backward and roll commands. These commands can overwrite the remote viewer’s commands to avoid collisions. The observer can adjust the hovering height of the micro-drone with go-up and go-down commands and can also execute some AI functions such as follow-me from the client software to ease remote viewing.
  • the geofencing can be achieved by checking the observer’s location to avoid restricted areas and comply with altitude limitations. This is because the micro-drone 11 is within a very short distance of the observer.
  • the observer’s location can be the GPS location from the observer’s device or be inferred based on the IP address of the device or the cell tower that the observer’s device connects to, etc.
  • the observer’s location is not fixed because the observer can keep moving during remote viewing.
  • the observer’s location is sent to a look-up table on the system server 15 through the connection 16 for geofencing. If a location is forbidden, a warning will be displayed on the client software installed on the observer’s personal computing device 13.
  • the system provides some mechanism to minimize the risks to privacy and to the protection of personal data.
  • All data and videos transmitted on the system through the Internet are encrypted. No streamed video and data from the videoconferencing are saved. The non-permitted faces accidentally captured in the micro-drone 11 videos are blurred locally by the client software on the observer’s personal computing device 13 before being sent to the system server 15 for processing.
  • each remote viewing session has a booking of the viewing time on the platform.
  • the viewing time together with the observer’s account information is used for generating a time-based token.
  • the platform authorizes each micro-drone 11 flight with this time-based token to prevent illicit use of remote viewing. Because the observer is within a very short distance of the micro-drone 11 for flight supervision, with the viewing time and the observer’s location having been recorded, an event of privacy and data protection violation can be easily traced back.
  • the remote viewing scenarios of the system are agnostic. To make it versatile, the system provides REST APIs and Webhooks to third parties for system integrations. The APIs are authorized with JSON Web Tokens RFC 7519. With these APIs a third party can make remote viewing through the platform but have their own viewing session management.
  • the remote viewing can be complemented with inspection or aerial survey. Due to the size, weight and battery limitations of a micro-drone, the inspection or the survey may require several rounds of remote viewing by a micro-drone 11 with different sensors in each flight. All these sensor data are transmitted to the system server 15 via connections 24, 25 and 26 or 26 and 16, bridged by the observer’s personal computing device 13.
  • the micro-drone 11 onboard sensor(s) 19 are generally small low-cost sensors with low-quality measurements.
  • machine learning models pretrained with large amounts of high-quality measurements are used to rectify and enhance those low-quality sensor data captured and output measurements of much better quality.
  • these enhanced data are spatially collocated, and temporally collocated on the system server 15 for data analysis.
  • mapping of large areas can be achieved by stitching small maps generated from several flights of the micro-drone 11 using computer vision techniques.
  • the analysis can be either real-time on-line processing or off-line processing.
  • the results are transmitted to the remote viewers’ web interface via connection 17 and the observer’s client software via connection 16 for display.
  • the results can also be sent to third parties’ servers for further processing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Selective Calling Equipment (AREA)

Abstract

Un système (10) de visualisation à distance en temps réel d'un site comprend un micro-drone (11) monté avec une caméra (12), un observateur avec un dispositif informatique personnel de l'observateur (13), un serveur de système (15) et un spectateur à distance avec un dispositif informatique personnel (14) du spectateur à distance. Le système peut nécessiter un boîtier relais (20), si le micro-drone (11) n'a pas de capacité Bluetooth® mais possède le Wi-Fi et est configuré avec une adresse IP fixe. Le micro-drone (11) est un micro-drone de faible vitesse, de taille d'une paume de main, ayant un poids brut inférieur à 250g et est piloté soit par le spectateur à distance, soit par l'observateur à l'intérieur d'une ligne de visée visuelle (VLOS) de l'observateur, la commande du micro-drone (11) par le spectateur à distance étant limitée à trois commandes, qui sont d'avancer, de virer à gauche, et de virer à droite, l'observateur pouvant prendre la commande du micro-drone (11) à tout moment pendant le vol.
EP23786198.4A 2022-11-02 2023-10-03 Système de visualisation à distance en temps réel d'un site Pending EP4612902A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IES20220175 2022-11-02
PCT/EP2023/077358 WO2024094378A1 (fr) 2022-11-02 2023-10-03 Système de visualisation à distance en temps réel d'un site

Publications (1)

Publication Number Publication Date
EP4612902A1 true EP4612902A1 (fr) 2025-09-10

Family

ID=96730989

Family Applications (1)

Application Number Title Priority Date Filing Date
EP23786198.4A Pending EP4612902A1 (fr) 2022-11-02 2023-10-03 Système de visualisation à distance en temps réel d'un site

Country Status (1)

Country Link
EP (1) EP4612902A1 (fr)

Similar Documents

Publication Publication Date Title
US11199859B2 (en) Apparatus and method for network based operation of an unmanned aerial vehicle
Mademlis et al. Challenges in autonomous UAV cinematography: An overview
KR101941643B1 (ko) 멀티 드론 제어 시스템 및 방법
CN109479119B (zh) Uav交互视频广播的系统与方法
US10244211B2 (en) Immersive interactive telepresence
CN104808675B (zh) 基于智能终端的体感飞行操控系统及终端设备
JP6437675B2 (ja) 「無人航空機（ｕａ）を制御するためのパイロットを選択するための方法、無人航空機（ｕａ）を制御するためのパイロットを選択するための装置、無人航空機（ｕａ）を制御するためのパイロットの選択を容易にするための方法、及び無人航空機（ｕａ）
US20160116912A1 (en) System and method for controlling unmanned vehicles
US20200106818A1 (en) Drone real-time interactive communications system
CN107357314A (zh) 一种无人飞行器远程控制方法及系统
CN106982345A (zh) 通过uav网络来促进广角视野视频会议
US9961658B2 (en) Local network for the simultaneous exchange of data between a drone and a plurality of user terminals
US20170269590A1 (en) Controlled device and communication system and method utilizing the same
JP2021036414A (ja) 遠隔操縦システムおよびその操縦装置
CN110772728A (zh) 消防专用无人机、系统及其消防方法
JP2021179976A (ja) 遠隔操縦システムおよびその操縦装置
CN113093811B (zh) 基于智能座舱系统的无人机控制方法以及系统
JP6986303B2 (ja) 遠隔操縦システムおよびその操縦装置
CN219105386U (zh) 图传灭火控制系统及其消防无人机
WO2024094378A1 (fr) Système de visualisation à distance en temps réel d'un site
EP4612902A1 (fr) Système de visualisation à distance en temps réel d'un site
KR20170034461A (ko) 자동항법 및 정보전달 가능한 무인비행시스템
KR102118345B1 (ko) 드론을 활용한 현장 실시간 입체경호서비스 제공시스템
CN211863625U (zh) 消防专用无人机、系统
CN211442806U (zh) 应急救援专用侦察无人机、系统

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20250331

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)