WO2023247982A1

WO2023247982A1 - Smart drone system for simulating a moving object

Info

Publication number: WO2023247982A1
Application number: PCT/HU2023/050041
Authority: WO
Inventors: Tamás Bence GERA; Bence KONDÁSZ; Kolumbán Tamás SMAUSZ; Béla HOPP
Original assignee: Szegedi Tudományegyetem
Priority date: 2022-06-22
Filing date: 2023-06-22
Publication date: 2023-12-28
Also published as: FR3137064A3

Abstract

The smart drone system (100) for simulating a moving object comprises at least one drone (200) comprising (a) a stabilizing unit (210), a projection unit (220), an analytical unit (230) for processing and evaluating motion information of at least one user, a wired and/or wireless communication interface (240) for transmitting sensor data originating from its analytical unit (230) and receiving control information; (b) at least one diagnostic unit (300) that can be mounted on said at least one user comprising at least one kinematic sensor (310), and/or at least one biological sensor (320), and a wireless communication interface (340) for transmitting sensor data originating from the kinematic sensors (310) and the biological sensors (320); (c) at least one central control unit (400) comprising a data storage device (410) for storing a computer program, a processor (420) for running said computer program, a wireless communication interface (440); (d) a communication system (700) providing wireless connection between the at least one drone (200), the at least one diagnostic unit (300) belonging to the at least one user and the central computer (400).

Description

Smart drone system for simulating a moving object

The present invention relates to a smart drone system for simulating a moving object. In particular, the invention relates to a drone system with which a motion program for one or more persons can be projected onto a surface, which can be dynamically changed depending on the current motion and physical capabilities of one or more persons.

Many methods and/or systems are currently known by which different motion activities can be displayed for a person, such as an athlete. Due to the widespread usability of flying drones, these drones are playing an increasingly important role in supporting motion activities, especially in increasing their efficiency.

The patent document W02017055080 describes a system and procedure that can be used in a closed space, which is used to support the training of a person performing physical activity. The system selects a feasible exercise depending on the user's environment, and then projects training information related to the exercise onto the surrounding surface using a projector mounted on a flying drone and/or a drone moving on the ground. The disadvantage of the above solution is that the definition of the exercises basically depends on the environment and not on the current physical abilities and/or biological characteristics of the user (e.g. endurance, fitness, speed, heart rate). Another disadvantage of the system is that the user receives information about the correct performance of the exercises only passively, i.e. in the form of image sequences and/or text.

The patent document US2008269016 describes a training system that monitors the position and motion of the athlete performing the training, and then improves the athlete's motion skills by simulating various movements based on this information. The system has a moving unit for simulating movements, which is arranged in a suspended state above an exercise track. The moving unit travels a training route at varying speeds, which the athlete can follow. In this way, the athlete no longer receives information about the performance of the exercises in a passive way, but can see the series of movements in reality during the exercise simulated by the moving unit and then can also perform it. The disadvantage of the above solution is that, beyond the precise control of the various movements, it is not suitable for increasing the efficiency of the training during the training, or for simulating the actual competition load, since the current physical abilities and /or biological characteristics of the person performing the training are not known. It is an object of the invention to eliminate the aforementioned problems by creating a smart drone system that can be used to carry out extensive motion analysis tests by measuring the actual biological characteristics of the person performing the motion, and based on the data thus obtained, the system can visually display a motion program using at least one light signal, where the motion program can be dynamically changed depending on the person's current motion and physical condition.

The above objects are achieved by providing a smart drone system according to claim 1 and a method for operating the drone system according to claim 8. Preferred embodiments of the smart drone system and the method according to the invention are defined by the dependent claims.

The invention will be described in detail below with reference to the drawings. In the drawings:

Figure 1 schematically illustrates the system design of the smart drone system according to the invention;

Figure 2 schematically illustrates the structural arrangement of the flying drone used in the smart drone system according to the invention;

Figure 3 is a functional block diagram of the units of the smart drone system according to the invention,

Figure 4 is a flowchart illustrating the main steps of the method according to the invention,

Figure 5 schematically illustrates the system design of a preferred embodiment of the intelligent drone system according to the invention, and

Figure 6 shows the internal arrangement of an exemplary embodiment of the integrated projection and stabilization unit used in the smart drone system according to the invention in a schematic perspective view.

In the present description, the term "user" generally refers to the person using the drone system according to the invention, typically an athlete who can practice either a team sport or an individual sport. However, the person of the "user" is not limited to athletes, but can be anyone else for whom the drone system according to the invention supports the performance of dynamic motion exercises, so the drone system can be used, for example, in medicine for motion therapy purposes, for the motion development of babies or for the physical conditioning of elderly people, etc. In the present description, the term "light signal" means a stationary or moving image object of different geometry and/or colour projected onto the surface of a motion area, which provides a visual display of the dynamically changeable motion program. The light signal can correspond, for example, to a beam of light simulating an opponent, a path dynamically changing relative to a moving person whose motion the user can follow, or, for example, a stationary landmark to be approached by the user. Depending on the application area of the drone system, the "light signal" can carry a variety of visual information.

In the present description, the term "track" means the area where the motion program is displayed for the user by means of the drone system, and in which area the user executes the displayed motion program. The surface of the track is suitable for the moving user to be able to notice the light signal under all circumstances. The track can be, for example, an outdoor sport track, water sport track or indoor sport track, a medical rehabilitation room, etc. In a closed space, however, sufficient air space must be provided for the free motion of the one or more drones.

As shown in Figures 1-3, the smart drone system 100 comprises at least one unmanned aerial vehicle, hereinafter referred to as drones 200, that can travel at a constant or variable speed along a route that is predetermined or can be changed adaptively during its operation. The drone 200 can be made from components already known for conventional flying drone units, accordingly the drone 200 contains at least a frame, motors, propellers, a control unit for driving the motors, a battery, a navigation unit, a radio transceiver, an on-board computer and a geographic positioning unit, such as a GPS or GLONASS or GALILEO receiver, optionally equipped with an RTK (Real Time Kinematics) unit. The drone 200 may also include one or more of an air flow meter, an optical flow meter, a barometer, a gyroscope, a temperature sensor, a magnetometer and/or a position sensor. The drone 200 may also include a speaker and/or microphone to issue various commands and/or transmit information bi-directionally. If a speaker is used, for example, the drone 200 can give voice instructions to the user regarding the exercise to be performed, and if a microphone is used, the user can give a command to the drone 200 to start, change or stop the motion program.

The high-precision positioning required for the motion simulation of the drone 200 can be performed in a covered motion area using, for example, a radio transceiver, an infrared sensor, a stereo camera, a distance sensor, LIDAR and ultrasonic sensors, whereas in an open motion area, in addition to the afore-mentioned devices, a geographic positioning unit (GPS, GLONASS, GALILEO), optionally equipped with a RTK receiver, may also be used. In order to increase the accuracy of positioning, at least one ground positioning station 500 can be placed near the track, which can have any kind of transceiver using electromagnetic waves, such as radio transceiver, an infrared transceiver or even a transceiver operating in the visible light range.

In order to avoid software malfunctions and/or accidents, the drone 200 may comprise various safety solutions, including a software safety module, a parachute or an airbag for the case of a contact failure (e.g. circuit break, power failure, other fatal error), and a spare battery. The smart drone system 100 may also include a hand-held radio controller 600, with which direct manual control of the drone 200 can be carried out when a security risk is detected.

The smart drone system 100 comprises a stabilization unit 210 mounted on the drone 200, by means of which the light signal 225 for the visual display of the motion program can be moved in any direction independently of the spatial position of the drone 200, or, where appropriate, the light signal 225 can be directed to a selected fixed point. The stabilization unit 210 can be realized in the form of a one-, two- or three-axis stabilizer as a group of electric motors.

The drone 200 comprises at least one projection unit 220 that is configured to project the light signal 225 required for the visual display of the dynamically variable motion program onto the surface of the track. The projection unit 220 has at least one light source 221, which may be carried out, for example, as a small-sized, high-power, single-colour or multi-colour light source, in particular as a laser, LED, or halogen light source 221.

In order to project the light signal 225 more precisely, one or more optical elements may be connected to the light source 221, such as a beam forming unit 222 having at least one lens, an optical fibre and a mirror, and/or a spatial light modulator 223 having at least an LCD panel, an aperture and a micro-mirror panel (DMD).

The projection unit 220 may be, for example, an optical device also capable of projecting several high-brightness light spots or images at the same time.

In a preferred embodiment of the smart drone system 100 according to the invention, the stabilizing unit 210 is integrated with the projection unit 220, and in this way they form an integrated unit 260, as schematically illustrated in Figures 5 and 6. In this embodiment, the light source 221 of the projection unit 220 is a laser source, the operation of which is controlled by a laser control unit 226. The light emitted by the light source 221, in this case the laser beams, is controlled by two servo motors 212, 214 having mutually perpendicular axes. The laser beams emitted by the light source 221 are directed to a mirror 228 adjusted by the servo motors 212, 214, which projects the laser beams onto a specific point on the track. In this case, the light signal 225 projected on the ground is typically a bright spot. The integrated unit 260 is placed in a housing, on the lower side of which a light-transmitting window is provided for the emitted light.

The servo motors 212, 214, which may be brushed or brushless motors, mechanically stabilize the mirror 228 so that the light signal 225 is always projected under the drone 200, at a position on the ground determined by the simulation program, regardless of the angle of inclination of the drone 200, as a point to be followed by the athlete. In this case, the servo motors 212, 214 and the mirror 228 together form the stabilizing unit.

The servo motors 212, 214 can rotate the mirror 228 along two mutually perpendicular axes. Due to its location, the servo motor 214 also functions as a mechanical blocker for the light emitted by the light source 221 (in this case, the laser beams), so that in the event of a malfunction of the drone 200, it prevents potentially dangerous laser beams from exiting from the integrated unit 260. It is noted that the laser control unit 226 is not necessarily a part of the integrated unit 260, as shown in Figure 6, but can even be formed as part of the control electronics of the drone 200 itself.

In another preferred embodiment of the intelligent drone system 100 according to the invention, the integrated unit 260 can also stabilize the projection of the light signal 225 by directing the light emitted by the light source 221 to a digital micromirror device (DMD), the properly controlled micromirrors of which adjust the projected position of the light or the shape and the pattern of the light signal 225 in relation to the position, orientation or tilt angle of the drone 200.

The drone 200 has a wired and/or wireless communication interface 240 for communicating with a central control unit 400.

The functional and physical units of the drone 200 described above are well known to those skilled in the art of drone technology, so the detailed description thereof will be omitted herein.

The at least one drone 200 used in the smart drone system 100 according to the invention comprises an analytical unit 230 mounted on the drone 200, which is used to measure, process and, where appropriate, evaluate motion information originating from at least one user. The analytical unit 230 has one or more sensors 231, which include one or more devices selected from the following group:

- at least one camera, preferably stereo camera, distance sensor, LIDAR and/or ultrasonic sensor for high-precision, real-time determination, for example, of the user's location, position, speed, acceleration and motion coordination, and/or - at least one infrared camera, which can be used to obtain information on the temperature distribution of the user's body, and/or

- at least one photodiode or light detector, by means of which it can be determined whether the user has entered the range of the light signal displayed by the projection unit.

The smart drone system 100 according to the invention further comprises at least one diagnostic unit 300, which acquires and evaluates information about the location and physi- cal/biological characteristics of the moving user during the dynamically variable motion program, and which includes: at least one kinematic sensor 310 for determining the location and/or motion of the user, wherein the kinematic sensor 310 may be, for example, a gyroscope, a pedometer, an accelerometer, a magnetometer, a geographic positioning unit, optionally equipped with an RTK module; and/or at least one biological sensor 320, by means of which information can be obtained about the current physical conditional abilities of the user, and which may be, for example, a heart rate monitor, a blood oxygen level meter, EKG; and a wired and/or wireless communication interface 340 for transmitting sensor data provided by the kinematic sensors 310 and the biological sensors 320.

The diagnostic unit 300 is attached to the user's body and is provided with an appropriately sized battery to power the sensors 310, 320 and the communication interface 340. The diagnostic unit 300 may be carried out, for example, in the form of a smart watch or smartphone.

The smart drone system 100 has a central control unit 400 that includes a wired and/or wireless communication interface 440 for receiving sensor data from the kinematic sensors 310 and the biological sensors 320 and signals from the sensors mounted on the drone 200, a data storage device 410 on the drone 200 for storing the computer program controlling its operation based on the afore-mentioned sensor data, and a processor 420 for running said computer program. The central control unit 400 may be carried out in the form of any computing device performing complex calculations, such as a personal computer, a notebook, a tablet, a smart phone, a smart watch, by means of which the smart drone system 100 can be programmed and controlled in a user-friendly and simple manner.

The smart drone system 100 further comprises a communication system 700 that provides a wireless connection between the drone 200, the user-mounted diagnostic unit 300 and the central control unit 400. The computer program running in the central control unit 400 is configured to send control information about the motion of the drone 200 to the drone 200 based on the sensor data from the diagnostic unit 300 and to control the directionality of the projection unit 220 as well as the position and the motion of the drone 200 over the track depending on the current motion and conditional characteristics of the user. Similar to the motion program, the computer program may also be dynamically modified with taking the user's current motion and conditional characteristics into account, i.e. the motion program can be modified adaptively in response to the user’s actions during performing the exercise.

The drone 200 may further comprise a speaker and/or a microphone, which can be used to issue and receive various voice messages and thereby allowing one- or two-way information transmission.

Hereinafter, the method for operating the drone system according to the invention is described with reference to Figure 4.

In the first step 800 of the method, the drone system is initialized, during which one or more drones 200 are moved along the track to their starting position and the computer program executing the motion program is loaded into the central control unit 400. The diagnostic units 300 fixed to the user are also activated.

In the next step 810, in order to display the motion program, at least one light signal is continuously projected onto the track using the stabilization unit(s) 210 and the projection unit(s) 220 mounted on one or more drones 200.

In step 820, the motion parameters (e.g. direction, speed, acceleration, the instantaneous or average values thereof, etc.) of the user are continuously determined with at least one analytical unit 230.

In step 830, the current position and the current physical/biological characteristics of the user are continuously determined using the diagnostic unit 300, and then in step 840, the central control unit 400 is used to evaluate the health status and the performance of the user based on the sensor data originating from the diagnostic unit 300, as well as the motion of the user on the basis of the motion parameters originating from the analytical unit 230.

In step 850, depending on the result of the evaluation, based on the motion program, the central control unit 400 controls the motion of the one or more drones 200 on a predetermined route or on a route that can be adaptively changed depending on the result of the evaluation, as well as the position and the information content of the light signal to be displayed by the one or more drones 200. The motion program may be modified, for example, so that, based on the information produced by the analytical unit 230, the user's heart rate be kept within a given range, e.g. within the range of 140 to 160 beats.

In a preferred embodiment of the method according to the invention, during step 850, voice messages may be used to help the user to complete the motion program.

Unlike the prior art solutions, the smart drone system 100 according to the inventions has the further advantage that the motion program may be changed depending on the current physical state of the user in addition to the precise control of the various movements of the user. In this way, for example, situations where the user's weaknesses are emphasized can be simulated, thus a large increase in the performance may be achieved.

An additional advantage of the smart drone system 100 according to the invention is its portability, which makes it possible to use it not only with fixed tracks, but practically anywhere where the sensors and the communication system of the smart drone system 100 can be operated.

Taking all of this into account, the 100 smart drone system can be used to create completely user-tailored and innovative educational, training, practicing or leisure motion programs.

Example 1

The smart drone system 100 according to the invention offers the possibility of simulating contacts and positional changes during various sports matches, where, for example, it is necessary to switch from a defensive position to an attacking position and vice versa. By means of the photodiode and/or the light detector of the analytical unit 230 it can be determined whether the user has entered the moving zone projected by the drone 200. When the user reaches the projected moving zone, then:

- dynamically changing forbidden or compulsory zones may be created, and the user's performance and reaction time may be evaluated depending on the extent and/or duration of entering the zones, based on the comparison of the sensor signals of the 300 diagnostic units mounted on the user,

- the drone 200 changes its role and can become the chaser instead of the chased object, so the user will have to run away from the light signal 225 projected by the drone 200 in order not to be „caughf ’ by the drone 200. Example 2

The smart drone system 100 also makes it possible to carry out an artificial tour or a guided tour, where the drone 200 provides information to at least one user, depending on their location, using the speaker, while various informative educational images are projected to the user.

Example 3

The smart drone system 100 may also be used in military training, where the drone 200 can accompany the soldiers through the field exercise with visual light signals or voice commands or can even simulate the soldiers or combat equipment of the enemy.

List of reference numbers

100 - smart drone system

200 - drone

210 - stabilizing unit

212, 214 - servo motor

220 - projection unit

221- light source

222 - beam forming unit

223 - spatial light modulator

225 - light signal

226 - laser control unit

228 - mirror

230 - analytical unit

231 - sensor

240 - communication interface

260 - integrated unit

300 - diagnostic unit

310 - kinematic sensor 320 - biological sensor

340 - communication interface

400 - central control unit

410 - data storage device 420 - processor

440 - communication interface - ground positioning station - hand-held radio controller - communication system

Claims

CLAIMS . A smart drone system (100) for simulating a moving object, characterized in that it comprises: a) at least one drone (200) comprising

• a projection unit (220) for producing a light signal (225),

• a stabilizing unit (210) for stabilizing the ground position of the light signal (225),

• an analytical unit (230) for processing and evaluating motion information of at least one user,

• a wired and/or wireless communication interface (240) for transmitting sensor data originating from its analytical unit (230) and receiving control information, b) at least one diagnostic unit (300) that can be mounted on said at least one user, comprising

• at least one kinematic sensor (310), and/or

• at least one biological sensor (320), and

• a wireless communication interface (340) for transmitting sensor data originating from the kinematic sensors (310) and the biological sensors (320), c) at least one central control unit (400) comprising

• a data storage device (410) for storing a computer program,

• a processor (420) for running said computer program,

• a wireless communication interface (440); d) a communication system (700) providing wireless connection between the at least one drone (200), the at least one diagnostic unit (300) belonging to the at least one user and the central computer (400), wherein the data storage device (410) of the central control unit (400) stores a computer program for generating control information about the motion of the drone (200) based on sensor data originating from the at least one diagnostic unit (300), and for transmitting the generated control information to the at least one drone (200) through the wireless communication network (700), and wherein the stabilizing unit (210) and the projection unit (220) form an integrated unit (260).

2. The smart drone system (100) according to claim 1, characterized in that the projection unit (220) comprises

- a high-power light source (221), in particular a laser, LED or halogen light source,

- a beam forming unit (222), and

- a spatial light modulator (223).

3. The smart drone system (100) according to claim 2, characterized in that said integrated unit (260) comprises two servo motors (202, 204) having mutually perpendicular axes, and a mirror (228) that can be adjusted by the servo motors (212, 214) for directing the light emitted by the light source (221).

4. The smart drone system (100) according to claim 1, characterized in that said integrated unit (260) comprises a digital micromirror device (DMD) as the stabilizing unit (210).

5. The smart drone system (100) according to claim 1, characterized in that the analytical unit (230) mounted on the drone (200) comprises at least one sensor selected from the following group:

• a camera, in particular a stereo camera;

• an infrared camera;

• a light detector, in particular a photodiode;

• a distance sensor, in particular a LIDAR or an ultrasonic sensor.

6. The smart drone system (100) according to claim 1 or 2, characterized in that the drone (200) further comprises at least one microphone and/or speaker.

7. The smart drone system (100) according to any one of the preceding claims, characterized in that a drone (200) further comprises a parachute and/or an airbag and/or a spare battery.

8. The smart drone system (100) according to any one of the preceding claims, characterized in that the at least one diagnostic unit (300) comprises at least one sensor selected from the following group: a gyroscope, a GPS receiver, a pedometer, a radio transceiver, an acceleration sensor, a magnetometer, a heart rate monitor, a blood oxygen level meter, EKG.

9. The smart drone system (100) according to any one of the preceding claims, characterized in that it further comprises a hand-held radio controller (600) for direct manual control of the drone (200).

10. A method for operating a drone system (100) according to any one of the claims 1 to 9, characterized in that the method comprises the steps of initializing (800) the drone system (100), during which one or more drones (200) are moved along the track to their starting position, loading the computer program executing the motion program into the central control unit (400), and activating the diagnostic unit (300) mounted on the user; for displaying the motion program, continuously projecting (810) at least one light signal (225) onto the track by means of the stabilization unit(s) (210) and the projection unit(s) (220) mounted on one or more drones (200); continuously determining (830), by means of the diagnostic unit (300), the user's current position and current physical/biological characteristics;

- by means of the central control unit (400), continuously evaluating (840) the user's state of health and performance based on sensor data originating from the diagnostic unit (300), as well as the user's motion based on the motion parameters originating from the analytical unit (230); and depending on the result of the evaluation, based on the motion program, controlling (850) the motion of one or more drones (200) along a predetermined route or on a route that can be adaptively changed depending on the result of the evaluation, as well as the position and information content of the light signal (225) to be displayed by one or more drones (200).