FR3100075A1 - PERSON DETECTION DEVICE BY DRONE - Google Patents

Abstract

The present invention relates to a device (2) for detecting persons for assisting in searching for persons and for monitoring defined perimeters, comprising program modules which can be executed on one or more hardware processing components such as a microprocessor, the device. device (2) being characterized in that it comprises: At least one drone (3) controlled remotely through the device (2), and configured to perform aerial movements, preferably automatically over a zone (Z), preferably according to programmed, predefined and memorized flights, the drone (3) comprising an image capture device (32) for recording images of the overflown zone (Z) and a communication module configured to transmitting and receiving data from a communication module; a computing computer server (4) being configured to allow the construction of flight plans to cover the areas to be searched or to schedule regular rounds monitoring for the drone (3), and for coordinating the actions to be carried out; the server (4) comprising an Artificial Intelligence (AI) module implementing an artificial neural network (5) using a machine learning model specialized in the analysis of aerial images, configured to detect human silhouettes, and a module communication device configured to send and receive data. Figure to be published with the abstract: Fig. 1

Description

Drone person detection device

Technical field of the invention

The present invention generally relates to the field of person detection devices, in particular devices using drones. It also concerns the surveillance of defined areas and perimeters.

prior art

It is known from the prior art the use of drones, for example by the company Dronevolt™ for the surveillance and safety of people in various situations such as the search for a person, the surveillance of an area, the assistance to firefighters. The images are captured in real time and transmitted to a computer screen. However, these drones only allow the transmission of a video stream on a screen, the surveillance of perimeters or the detection of people being carried out, among other things thanks to the analysis of this video stream, by an operator.

Document EP3207435 describes a security system in which a drone is used stationary. When an alarm is received, the computer system autonomously programs the flight of a drone from its fixed initial location to the specific location where the alarm condition is declared, in order to send data, for example videos, of said location. Again, the drone is only used as a support device to send a stream of video data from a particular vantage point.

Similarly, document US20190164019 also describes sending a drone following an alarm to send data from its sensors. The drone intervenes once the alarm is detected, to provide additional data on a location of interest.

The result is a lack of solutions to help with automated perimeter surveillance and detection of people assisted by a computer solution, to help or even replace the operator on the ground, allowing him to keep these resources for use them when detecting a person.

The object of the present invention is therefore to provide a device for detecting people and monitoring perimeters using a drone, assisted by artificial intelligence, making it possible to overcome at least some of the drawbacks of the prior art, and in particular put the charge of people detection on the drone, which then can notify a user of the device.

This object is achieved by a person detection device for assisting in searching for people and for monitoring defined perimeters, comprising program modules executable on one or more processing hardware components such as a microprocessor, the device comprising :
At least one drone remotely controlled through the device, and configured to perform aerial movements, preferably automatically over an area, preferably according to programmed, predefined and memorized flights, the drone comprising a image capture to record images of the overflown area and a communication module configured to send to another communication module and receive data from this other communication module;
a calculation computer server being configured to allow the construction of flight plans covering the areas to be searched or to schedule regular rounds of surveillance for the drone, and to coordinate the actions to be carried out; the server comprising an Artificial Intelligence (AI) module implementing an artificial neural network using a machine learning model specialized in the analysis of aerial images, configured to detect human silhouettes, and a communication module configured to transmit to other communications modules including at least that of the drone and receiving data from other communications modules including at least that of the drone
the system further comprising a database in which the images received from the drone are stored for analysis and use in machine learning for continuous improvement.

According to one feature, the communication module of the device and that of the drone each comprise an encryption and decryption sub-module configured to encrypt the messages to be transmitted and to decrypt the encrypted messages received.

According to another feature, the server further comprises a cartographic application including relief and buildings, the cartographic application being used for the creation of three-dimensional flight plans.

According to another feature, the drone further comprises a thermal camera for detecting human silhouettes, for example at night or in foggy weather.

According to another feature, the artificial neural network comprising an automatic learning model is trained from real image data and/or virtual image data.

According to another feature, the artificial neural network (5) comprises an automatic learning model trained from virtual image data sets, automatically annotated, from digital doubles created by a learning model production module , in particular sets of virtual images of people seen from above, for example seen from a drone.

Another aspect of the invention relates to a person detection system for assisting in searching for people and for monitoring defined perimeters, in which the system comprises at least one detection device as described above, the detection system comprising furthermore an application executed on a communication terminal, preferably general public, for example a smartphone, a tablet or a computer, the communication terminal comprising a communication module configured to transmit and receive data from the device and a graphical interface configured to display information on a screen of the terminal, the communication module of the terminal preferably comprising an encryption and decryption sub-module configured to encrypt the messages to be transmitted to the device or the drone and to decrypt the encrypted messages received from the device or of the drone.

According to another feature, the application of the communication terminal comprises a cartographic application including the relief and the built environment, the cartographic application being used for the creation of three-dimensional flight plans.

According to another feature, the server is configured to send data relating to a geolocated event at the location of the detection of a person to the communication module of the communication terminal, the interface of the communication terminal being configured to display said event geolocated on a virtual map of the search perimeter on the screen of said terminal with the geographical coordinates of the event.

Another aspect of the invention relates to the method for monitoring a perimeter with a previously described system, the method comprising the following steps:
- Programming a drone to perform a predetermined route;
- Realization by the drone of the planned trajectory, capture and recording of aerial images during the journey;
- Sending images to the AI module of the server through the communication modules;
- Real-time or delayed analysis of the images by the AI module of the calculation server (4);
- When a silhouette, preferably human, is identified, an event geolocated at the location of the detection is created by the AI module and is sent to a user's communication device;
- Placement of a marker on a virtual map on the interface of a user's communication terminal.

According to a feature of the process, the images captured by the drone are also sent in real time to a user's communication terminal.

Brief description of figures

• [Fig. 1] représente une vue schématique du système de détection selon certains modes de réalisation,
• |Fig. 2] représente le procédé de l’invention en vue schématique selon certains modes de réalisation.

Other characteristics, details and advantages of the invention will become apparent on reading the following description with reference to the appended figures, which illustrates:

• [Fig. 1] represents a schematic view of the detection system according to certain embodiments,
• |Fig. 2] represents the method of the invention in schematic view according to certain embodiments.

detailed description of the invention

Many combinations can be envisaged without departing from the scope of the invention; the person skilled in the art will choose one or the other depending on the economic, ergonomic, dimensional or other constraints that he will have to respect.

In general, the present invention comprises a person detection device (2) for assisting in searching for people and for monitoring defined perimeters, comprising program modules executable on one or more processing hardware components such as a microprocessor, the device (2) being characterized in that it comprises:
At least one drone (3) remotely controlled through the device (2), and configured to perform aerial movements, preferably automatically over an area (Z), preferably according to programmed, predefined flights and stored, the drone (3) comprising an image capture device (32) for recording images of the area (Z) overflown and a communication module configured to transmit to another communication module and receive data from this other communication module;
a calculation computer server (4) being configured to allow the construction of flight plans covering the areas (Z) to be searched or to schedule regular surveillance rounds for the drone (3), and to coordinate the actions to be carried out; the server (4) comprising an Artificial Intelligence (AI) module implementing an artificial neural network (5) using a machine learning model specialized in the analysis of aerial images, configured to detect human silhouettes, and a module communication module configured to transmit to other communication modules including at least that of the drone and to receive data from other communication modules including at least that of the drone
the server (4) further comprising a database (43) in which the images received from the drone (3) are stored for analysis and use in machine learning for continuous improvement.

This device (2) advantageously allows assistance with searches, for example of missing persons, as well as the monitoring of perimeters. When a person disappears, the emergency services carry out a drone-assisted search (3)s to find them. The addition of an aerial image analysis AI allows an application hosted in the calculation server (4) to build flight plans covering the areas to be searched or to schedule regular surveillance rounds for drones (3), to coordinate the actions to be carried out and to detect human forms. The information is received on the screen interface of the communication terminal (5) fitted to the teams in the field. The drones (3) automatically carry out the programmed flights and record images of the overflown zone (Z). In the event that a human silhouette is identified, the AI of the communication server (4) places a geolocated event at the location of the detection, the detection being able to be made using a thermal camera at night or by foggy weather. In the case of assistance with perimeter surveillance, a drone (3) is released regularly to carry out a round, it automatically carries out the planned trajectory and records images of the monitored area (Z). The images are sent to the server AI (4) which analyzes them in real time, and, if a human silhouette is identified, a geolocated event at the location of the detection is placed and the image is displayed on the terminal communications (5).

By “drone” is meant any unmanned aerial vehicle comprising a computer carried by the unmanned aerial vehicle to control the flight of the unmanned aerial vehicle, the drone being able to communicate with a remote communication server.

In certain embodiments, the communication module of the server (4) and that of the drone (3) each comprise an encryption and decryption sub-module configured to encrypt the messages to be transmitted and to decrypt the encrypted messages received.

This advantageously makes it possible to obtain a device (2), and a system (1) including the device (2), whose data exchanges are safe and secure.

In certain embodiments, the server (4) further comprises a cartographic application including the relief and the building, the cartographic application being used for the creation of three-dimensional flight plans.

This allows the server (4) to integrate data from the mapping application for route programming.

In embodiments, the application (52) of the communication terminal (5) comprises a cartographic application including the relief and the building, the cartographic application being used for the creation of three-dimensional flight plans. In certain embodiments, the route programming is carried out manually from the cartographic application of a communication terminal (5), the user indicating points on the graphic interface (53). The drone (3) will carry out a route passing through all the points, in the agreed order. It is possible to manage certain parameters of the drone (3), such as the altitude and inclination of the camera, before or during the flight of said drone (3). In other embodiments, the route programming is carried out automatically from the cartographic application of a communication terminal (5). From the definition of a perimeter or an area to be monitored (Z), the server (4) of the device (2) calculates a route passing through all the points by carrying out round trips (decided according to criteria or randomly) so that the drone (3) squares the entire area of the zone to be monitored (Z), preferably in a minimum of time. In these embodiments, the angle of the camera is preferably not manually controllable, the server (4) carrying out a calculation by an algorithm which takes into account said inclination. Route programming can be automatic or manual, and the choice is left to the user, from the application (52) of his communication terminal (5).

In some embodiments, the drone (3) further comprises a thermal camera to detect human silhouettes, for example at night or in foggy weather.

This advantageously makes it possible to make the device (2) functional whatever the time and whatever the weather conditions.

In some embodiments, the artificial neural network (5) including a machine learning model is trained from real image data and/or virtual image data.

Advantageously, training the neural network on virtual image data makes it possible to reduce the resources necessary in terms of data and the preparation thereof so that they can be exploited by said neural network. It is necessary that aerial image data be used to correctly train the drone (3) to detect silhouettes seen from above.

In some embodiments, the artificial neural network (5) including a machine learning model is trained from automatically annotated virtual image datasets from digital twins created by a model generation module. learning, in particular sets of virtual images of people seen from above, for example seen from a drone.

Advantageously, this makes it possible to train the artificial neural network (5) without needing to provide it with a large amount of real image data, which would have had to be manually trimmed by an operator to teach said network to discriminate human silhouettes, for example in top view. In other words, it makes it possible to consider doing without the tangible world to produce automatically annotated data files from digital twins. A digital model of the environment, with digital models of the entities to be identified and a rendering engine is used to generate annotated data files.

In some embodiments, digital duplicates are modeled from existing photos of silhouettes, people, or objects. These 3D models can then be integrated into landscapes taken from an image database and/or into virtual images designed by a graphics engine, for example a video game engine.

In some embodiments, the digital doubles are made by photogrammetry and/or by at least one photorealistic scene-creating application configured to create images, preferably by varying angles, light conditions, or targets inserted into the synthetic landscapes , including inside buildings, without moving people.

Auto-annotated datasets are created using applications, such as Blender, to create photorealistic scenes, varying angles, light conditions, or targets inserted into synthetic landscapes, including inside buildings, without moving people.

Parameters for creating virtual datasets can be controlled and randomized. Since the software knows the image to be integrated, the clipping can be done automatically, which saves considerable time.

In embodiments, the datasets may include semi-virtual datasets, in other words datasets comprising virtual elements and real elements. The image data collected, said to be real, can come from “rig” cameras configured to scan people and/or objects. In certain embodiments, which can be combined with the previously described embodiments, the image data collected are images captured by drone. This makes it possible to mechanize, to improve the repeatability of said image capture.

The image data of virtual or real human silhouettes present different environments, outdoors, in all weathers, in all seasons, in various environments. In the same way, the digital doubles include silhouettes of different sizes, colors and allow views from very different angles.

In some embodiments, the artificial neural network (5) retrains itself on the database (43) of the server (4) to improve its detection of people. This has the advantage of continuously improving the efficiency of people detection thanks to the machine learning model, which updates and improves with each detection.

A person detection system (1) for assisting in the search for people and for monitoring defined perimeters is illustrated in a non-limiting manner and by way of example in FIG. 1, the system (1) comprising at least a detection device (2) as described above, the detection system (1) further comprising an application (52) executed on a communication terminal (5), preferably for the general public, for example a smartphone, a tablet or a computer, the communication terminal (5) comprising a communication module configured to transmit and receive data from the device (2) and a graphic interface (53) configured to display information on a screen of the terminal, the communication module of the terminal preferably comprising an encryption and decryption sub-module configured to encrypt the messages to be transmitted to the device (2) or the drone (3) and to decrypt the encrypted messages received from the device (2) or from the d round (3).

This advantageously allows users of the system (1) to receive, in particular in real time, data captured by the drone (3) and information resulting from the analysis of said data by the server (4) of the device (2) integrated into the system (1). For example, once a person has been detected by the AI module on an image received from the drone (3), the server (4) can send to the communication terminal (5) which equips a user the image and the associated data, for example the geolocation of the location where the image was captured. The data can be displayed on the application (52) of the terminal, in particular on a cartographic application making it possible to display the geolocation of the location where a person or a silhouette has been detected. In certain embodiments, a user can, from the application (52), validate the detection, and can decide to intervene.

• Programmation d’un drone (3) pour réaliser un trajet prédéterminé ;
• Réalisation par le drone (3) de la trajectoire planifiée, capture et enregistrement d’images aériennes pendant le trajet ;
• Envoi des images au module d’IA du serveur (4) par l’intermédiaire des modules de communication ;
• Analyse en temps réel ou en décalé des images par le module d’IA du serveur (4) de calcul ;
• Lorsqu’une silhouette, de préférence humaine, est identifiée, un évènement géolocalisé à l’endroit de la détection est créé par le module d’IA et est envoyé au terminal de communication (5) d’un utilisateur ;
• Placement d’un repère sur une carte virtuelle sur l’interface du dispositif (2) de communication d’un utilisateur.

More particularly, Figure 2 illustrates an exemplary non-limiting embodiment of the present invention, in which is described the method of monitoring a perimeter, or detecting a person, using the system (1) described above, the method comprising the following steps:

• Programming a drone (3) to perform a predetermined route;
• Realization by the drone (3) of the planned trajectory, capture and recording of aerial images during the journey;
• Sending images to the AI module of the server (4) through the communication modules;
• Real-time or delayed analysis of the images by the AI module of the calculation server (4);
• When a silhouette, preferably human, is identified, an event geolocated at the location of the detection is created by the AI module and is sent to the communication terminal (5) of a user;
• Placement of a marker on a virtual map on the interface of a user's communication device (2).

In some embodiments of the method, as illustrated by the dotted arrow in Figure 2, the images captured by the drone (3) can also be sent in real time to the communication terminal (5) of a user. This allows users to access data and monitor the area (Z) in real time, which further enables dual image analysis: visual analysis by the user, and automated pattern recognition analysis by the neural network of the device AI module (2).

It will readily be understood from reading the present application that the features of the present invention, as generally described and illustrated in the figures, can be arranged and designed in a wide variety of different configurations. Thus, the description of the present invention and the accompanying figures are not intended to limit the scope of the invention but merely represent selected embodiments.

Those skilled in the art will understand that the technical characteristics of a given embodiment can in fact be combined with characteristics of another embodiment unless the reverse is explicitly mentioned or it is obvious that these characteristics are incompatible. Moreover, the technical characteristics described in a given embodiment can be isolated from the other characteristics of this mode unless the reverse is explicitly mentioned.

It should be apparent to those skilled in the art that the present invention permits embodiments in many other specific forms without departing from the scope defined by the scope of the appended claims, they should be considered by way of illustration and the invention should not be limited to the details given above.

1. Person detection system (1)
2. Person detection device (2)
3.Drone (3)
31. Communications Module
32. Image capture device
4. Server (4) IT
41. Communications Module
42. Artificial Neural Network (5)
43. Database (43)
5. Communication terminal (5)
51. Communications Module
52. Application
53. GUI
Z. Area to be monitored

Claims (12)

Person detection device (2) for assisting in the search for persons and for monitoring defined perimeters, comprising program modules executable on one or more processing hardware components such as a microprocessor, the device (2) being characterized in that it comprises:

• At least one drone (3) remotely controlled through the device (2), and configured to perform aerial movements, preferably automatically over an area (Z), preferably according to programmed, predefined flights and stored, the drone (3) comprising an image capture device (32) for recording images of the area overflown (Z) and a communication module configured to transmit to another communication module and receive data from this other communication module;

a computing server (4) being configured to allow the construction of flight plans covering the areas to be searched or to schedule regular rounds of surveillance for the drone (3), and to coordinate the actions to be carried out;
the server (4) comprising an Artificial Intelligence (AI) module implementing an artificial neural network (5) using a machine learning model specialized in the analysis of aerial images, configured to detect human silhouettes, and a module communication module configured to transmit to other communication modules including at least that of the drone and to receive data from other communication modules including at least that of the drone
the server (4) further comprising a database (43) in which the images received from the drone (3) are stored for analysis and use in machine learning for continuous improvement. Device (2) according to the preceding claim, in which the communication module of the device (2) and that of the drone (3) each comprise an encryption and decryption sub-module configured to encrypt the messages to be transmitted and to decrypt the encrypted messages received. Device (2) according to any one of the preceding claims, in which the device (2) further comprises a cartographic application including the relief and the building, the cartographic application being used for the creation of three-dimensional flight plans. Device (2) according to any one of the preceding claims, in which the drone (3) further comprises a thermal camera for detecting human silhouettes, for example at night or in foggy weather. Device (2) according to any of the preceding claims, wherein the artificial neural network (5) comprising a machine learning model is trained from real image data and/or virtual image data. A device (2) according to any preceding claim, wherein the artificial neural network (5) comprises a machine learning model trained from automatically annotated virtual image datasets from created digital twins by a module for producing the learning model, in particular sets of virtual images of people seen from above, for example seen from a drone. Device (2) according to any one of the preceding claims, in which the artificial neural network (5) re-trains itself on the database (43) of the server (4) to improve its detection of people. Person detection system (1) for assisting in the search for persons and for monitoring defined perimeters, characterized in that the system (1) comprises at least one detection device (2) according to any one of the claims above, the detection system (1) further comprising an application (52) executed on a communication terminal (5), preferably the general public, for example a smartphone, a tablet or a computer, the communication terminal (5) comprising a communication module configured to transmit and receive data from the device (2) and a graphical interface (53) configured to display information on a screen of the terminal, the communication module of the terminal preferably comprising an encryption sub-module and decryption configured to encrypt the messages to be sent to the device (2) or the drone (3) and to decrypt the encrypted messages received from the device (2) or the drone (3). System (1) according to the preceding claim, in which the application (52) of the communication terminal (5) comprises a cartographic application including the relief and the building, the cartographic application being used for the creation of three-dimensional flight plans. System (1) according to any one of claims 8 to 9, in which the server (4) is configured to send data relating to a geolocated event at the location of the detection of a person to the communication module of the terminal of communication (5), the interface of the communication terminal (5) being configured to display said geolocated event on a virtual map of the search perimeter on the screen of said terminal with the geographical coordinates of the event Method for monitoring a perimeter, or detecting a person, with a system (1) according to any one of Claims 8 to 10, characterized in that the method comprises the following steps:

• Programming a drone (3) to perform a predetermined route;
• Realization by the drone (3) of the planned trajectory, capture and recording of aerial images during the journey;
• Sending images to the AI module of the server (4) through the communication modules;
• Real-time or delayed analysis of the images by the AI module of the calculation server (4);
• When a silhouette, preferably human, is identified, an event geolocated at the location of the detection is created by the AI module and is sent to a user's communication device (2);
• Placement of a marker on a virtual map on the interface of the communication terminal (5) of a user.

Method according to the preceding claim, in which the images captured by the drone (3) are also sent in real time to the communication terminal (5) of a user.