KR102493856B1 - Method of providing composite video based on athletics video - Google Patents

Abstract

The present invention relates to a method for providing a composite video based on an athletic video. The method for providing a composite video based on an athletic video, according to the present invention, comprises the steps of: receiving, as inputs, a first value specifying a flight height of a drone, a second value specifying a distance between a first sensor and a first point on the surface of a first athlete, and a third value specifying an angular displacement in a direction from the first point toward the first sensor of the drone; transmitting the first value, the second value, and the third value; checking information that the drone is at a height equal to the first value; obtaining, as at least one piece of collection data, measurement values of at least one sensor and video data; deriving at least one command data; generating an initial position setting completion signal; deriving at least one piece of command data; and extracting parts corresponding to the time the drone flew at locations corresponding to the first value, the second value, and the third value and storing the parts as an athletic video. Therefore, the present invention helps to precisely and accurately identify incorrect posture.

Description

Method for providing synthetic video based on exercise video {METHOD OF PROVIDING COMPOSITE VIDEO BASED ON ATHLETICS VIDEO}

The present invention relates to a motion image acquisition system using a drone and a method for acquiring motion images in the system. The present invention relates to a method for acquiring motion data based on a motion image acquired by a motion image acquisition system. The present invention relates to a method for providing a synthetic image based on motion images. An exercise image acquisition system using a drone, a method for acquiring exercise data, and a method for providing a synthetic image according to the present invention can be used for the purpose of improving posture and performance, particularly in exercise training.

With the increase in national income, interest in exercise and sports continues to increase. There are various sports such as track and field events such as running, high jump, and long jump, weight training to increase muscle mass, and ball games such as soccer and badminton. It is common in that applying postures or movements to practice helps to achieve high performance.

Various methods of acquiring exercise images are disclosed for training of professional athletes or ordinary people who want to improve their exercise abilities. The most basic method is to install the camera in a fixed location and record the movement within the camera angle. In a large-scale international sports event or professional league, a professional photographer holding a camera along with a moving athlete takes a picture while moving, or a track for moving a camera along with a moving athlete is sometimes used.

In addition, various methods of acquiring exercise data in a training process are disclosed for training of professional athletes or ordinary people who want to improve their athletic abilities. The most traditional method is for an experienced coach to observe the athlete's movements while training is taking place and, based on his or her experience, advise which posture is better. When an exercise image is obtained, a method in which an experienced coach monitors the obtained image to analyze and evaluate the movement of the player shown in the image is also used.

Among the conventional methods for obtaining exercise images, in the method of installing a camera at a fixed location and taking pictures of the movement within the camera angle, the camera position is fixed, but the player's position can be moved, so the player's position It is difficult to record the player's movements while the camera is out of the camera angle, and since the angle between the camera's shooting direction and the player's body direction can change, it is difficult to obtain images consistently taken from the same angle. Even when a track is used to move the camera, it is difficult to consistently acquire images taken at the same angle unless the player moves in a direction parallel to the length of the track. Among the conventional methods for acquiring exercise images, when a professional photographer is used, high costs are consumed due to the labor cost of the photographer, and it may be difficult to obtain images consistently taken from the same angle depending on the capabilities of the photographer. .

Since the conventional method of obtaining exercise data during training relies on observations that occur with the naked eye of experts such as coaches, there is a limit to the precision of observation itself, so it is difficult to expect efficient posture improvement in the reality where minute differences in posture affect performance. difficult.

An object of the present invention is to provide an exercise image acquisition system using a drone and a method for obtaining an exercise image using a drone, to provide a method for providing exercise data based on a sports training image obtained using a drone, and to provide a method for acquiring exercise data using a drone. It is to provide a method for providing a synthesized image based on a sports training image.

In a motion image acquisition system including a user device, a server, and a drone according to an embodiment of the present invention, the user device: a first value for specifying a flight height of the drone from a user; A second value specifying a distance between a first sensor and a first point on the surface of the first player, and a direction from the first point toward the first sensor of the drone relative to the frontal direction of the first player. Receiving a third value specifying angular displacement - the first point is a point at which the distance to the drone is the shortest on the surface of the first player, and the third value has a value of 0 degrees or more and less than 360 degrees , wherein the first sensor includes a first distance sensor and a camera; configured to transmit the first value, the second value, and the third value to the server and the drone; The drone: transmits information about being at the same height as the first value to the user device while flying at a height equal to the first value; configured to transmit a measurement value of at least one sensor of the drone and image data obtained by a camera included in the first sensor to the user device as at least one collection data, wherein the user device transmits the at least one collection data transmit data to the server, wherein the at least one sensor includes a first sensor, a plurality of radially disposed distance sensors, and an IMU sensor; The server: analyzes the at least one collected data to derive at least one command data for locating the drone at a position corresponding to the second value and the third value, and transmits it to the user device - the user the device transmits the at least one command data to the drone; In response to confirming that the drone is located at a position corresponding to the second value and the third value, configured to transmit an initial location setting completion signal to the user device, wherein the user device sends the initial location setting completion signal transmits to the drone -; The drone outputs a preset sound through a speaker in response to receiving the initial positioning completion signal, and collects the at least one piece of collected data while flying in the front direction of the first player while the first player is exercising. configured to transmit to the user device, wherein the user device transmits the at least one collected data to the server, wherein the server: analyzes the at least one collected data to perform At least one command data for positioning the drone at a position corresponding to the first value, the second value, and the third value based on the first player is derived and transmitted to the user device - the user device transmits the at least one command data to the drone; Based on the at least one collected data, it corresponds to the flight time of the drone at a position corresponding to the first value, the second value, and the third value among the image data included in the at least one collected data. It is configured to extract a part to be stored as an exercise image.

A method for providing exercise data according to an embodiment of the present invention transmits, by a server, at least one command data for causing a drone to rotate around a first stopped player to a user device, and the user device transmits the at least one command data to the user device. transmitting command data of to the drone; While the drone rotates around the stopped first player, at least one first collection data including a measurement value of at least one sensor included in the drone and image data obtained by a camera included in the drone is generated, transmitting to the user device, wherein the user device transmits at least one piece of first collection data to the server; generating, by the server, a three-dimensional human body model of the first player based on the at least one piece of first collected data; In the user device, a first value specifying a flying height of the drone, a second value specifying a distance between a first sensor of the drone and a first point on the surface of a first player, and a front of the first player. A third value specifying an angular displacement in a direction from the first point toward the first sensor of the drone is received based on the direction, and the server and the drone receive the first value, the second value, and the Transmitting the third value - the first point is a point at which the distance to the drone is the shortest on the surface of the first player, the third value has a value of 0 degrees or more and less than 360 degrees, 1 sensor includes a first distance sensor and the camera; Second collected data including image data acquired by the camera and measurement values of at least one sensor included in the drone while the drone flies in the front direction of the first player while the first player is exercising Generating and transmitting at least one to the user device, and transmitting at least one second collection data to the server by the user device; By the server: analyzing the at least one second collected data, and corresponding to the first value, the second value, and the third value based on the first player while the first player is exercising deriving at least one command data for locating the drone at a location and transmitting it to the user device, the user device sending the at least one command data to the drone; Based on the at least one second collected data, among the image data included in the at least one second collected data, the drone flies at a position corresponding to the first value, the second value, and the third value. extracting a portion corresponding to one hour and storing it as an exercise image; extracting joint points from the motion image; And based on the joint point extracted from the motion image and the 3-dimensional human body model of the first player, the joint movable range and joint movable range and joint movable range of the first player, joint movable speed, and periodically repeated motions Checking exercise data indicating a change in operating speed for each period, and displaying the exercise data on the user device.

According to an embodiment of the present invention, a method for providing a synthetic image based on a sports training image includes, by a server: obtaining an exercise image of a first player; checking exercise data of the first player from the exercise image of the first player; obtaining a plurality of partial exercise images corresponding to the first player by parsing the exercise image of the first player for each period of the exercise motion; Of the plurality of partial exercise images corresponding to the first player, the transparency of the first partial exercise image and the second partial exercise image are increased, and the transparency of the first partial exercise image with increased transparency is increased, and the second part is increased. generating a first composite image by overlaying a motion image, motion data corresponding to the first partial motion image, and motion data corresponding to the second partial motion image - the motion data is the movement of the joints of the first player contains range -; and transmitting the first synthesized image to a user device, and displaying the first synthesized image on the user device.

According to an embodiment of the present invention, a method for providing a synthetic video based on a sports training video is provided by a server: a first player is photographed by a drone flying in a frontal direction of the first player while the first player is performing a first exercise. obtaining a first exercise image based on the first image data; obtaining a first background image and a first player image by separating a background region and a first player region from the first exercise image; generating a background image by synthesizing a plurality of frames of the first background image based on the flight speed of the drone and time information at which the first image data was captured; generating a first synthesized image by overlaying a plurality of frames of the first player image on the background image based on the flight speed of the drone and the time information at which the first image data was captured; and transmitting the first synthesized image to a user device, and displaying the first synthesized image on the user device.

According to an embodiment of the present invention, a motion image acquisition system using a drone stores an image obtained by a drone as a motion image while the drone is at a preset position on a cylindrical coordinate based on an athlete as a subject. , it is possible to provide an image acquired at a certain relative position and angle with respect to the subject. In the image acquired in a state where the relative position and angle of the camera to the subject is not maintained constant, the change in the position and view angle of the subject in the image, such as the change in the angle of a specific joint of the subject, constitutes an exercise posture. Numerical exercise data cannot be accurately specified. On the other hand, the motion image acquisition system using a drone according to an embodiment of the present invention makes it possible to secure an image by a camera whose position on a cylindrical coordinate with respect to a subject is constant. Since quantitative data such as joint angles can be extracted with high accuracy through analysis of images acquired in a state where the position on the cylindrical coordinates based on the subject of the camera is constant, motion images using a drone according to an embodiment of the present invention The acquisition system helps pinpoint incorrect postures precisely and accurately.

According to an embodiment of the present invention, a method for providing exercise data based on a sports training image obtained using a drone includes joint angles, etc. Since quantitative data can be extracted with high accuracy, it helps to precisely and accurately identify incorrect exercise postures.

According to an embodiment of the present invention, a method for providing a composite image based on sports training images provides an image by synthesizing images obtained in a state in which a position on a cylindrical coordinate of a camera subject is constant. In the case where the relative position and angle of the camera to the subject are not kept constant, when synthesizing the subject's appearance in the acquired image, the position of the subject in the image and the change in the view angle change, so even through the synthesized image, the specific joint of each subject Since it is impossible to accurately specify exercise data, which are numerical values constituting an exercise posture, such as a change in angle, it is impossible to accurately determine a change in posture. On the other hand, in the method for providing a synthetic image based on sports training images according to an embodiment of the present invention, images obtained by synthesizing images obtained in a state in which the position of the camera on the cylindrical coordinates is constant with respect to the subject are provided, so that posture changes are not performed. Efficient improvement in posture can be made possible by visually checking and comparing directly.

1A illustrates a motion image acquisition system according to various embodiments of the present invention.
1B illustrates a configuration of a server included in a motion image acquisition system according to various embodiments of the present disclosure.
2A and 2B illustrate a motion image acquisition method performed in a motion image acquisition system according to various embodiments of the present disclosure.
2C and 2D illustrate definitions of a first value, a second value, and a third value in a motion image acquisition method according to various embodiments of the present disclosure.
3A and 3B illustrate a method performed in a motion image acquisition system according to various embodiments of the present invention.
4A illustrates an example of a predetermined plurality of preset angles, in accordance with various embodiments of the present invention.
4B illustrates inputs and outputs of a first artificial intelligence model, according to various embodiments of the present invention.
5A illustrates a drone, in accordance with various embodiments of the present invention.
5B and 5C illustrate an operation in which a drone rotates in place according to various embodiments of the present disclosure.
6A to 6D illustrate operations of a drone according to an angle increase command according to various embodiments of the present disclosure.
7 illustrates an avoidance maneuver method performed in a motion image acquisition system according to various embodiments of the present disclosure.
8 illustrates a method of generating a 3D human body model performed in a motion image acquisition system according to various embodiments of the present disclosure.
9 illustrates a motion data acquisition method performed in a motion image acquisition system according to various embodiments of the present disclosure.
10A and 10B illustrate examples of displaying exercise data according to various embodiments of the present disclosure.
11 illustrates a synthetic image providing method according to various embodiments of the present invention.
12a and 12b illustrate examples of synthesized images provided according to various embodiments of the present invention.
13 illustrates a synthetic image providing method according to various embodiments of the present invention.
14A illustrates an example of a background image and a player image obtained according to various embodiments of the present invention.
14B illustrates an example of a background image obtained according to various embodiments of the present invention.
14C illustrates an example of a synthesized image obtained according to various embodiments of the present invention.
15 illustrates a synthetic image providing method according to various embodiments of the present invention.
16A and 16B illustrate examples of synthesized images provided according to various embodiments of the present invention.

As used herein, "comprises" and/or "comprising" does not exclude the presence or addition of one or more other elements other than the recited elements. Various embodiments and terms used herein are not intended to limit the technical features described herein to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, like reference numerals may be used for like or related elements. The singular form of a noun corresponding to an item may include one item or a plurality of items, unless the relevant context clearly dictates otherwise. In the present specification, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A Each of the phrases such as "at least one of , B, or C" may include any one of the items listed together in that phrase, or all possible combinations thereof. Terms such as "first", "second", or "first" or "secondary" may simply be used to distinguish that component from other corresponding components, and may refer to that component in other respects (eg, importance or order) is not limited. When a (eg, first) element is referred to as being “connected” to another (eg, second) element, with or without the terms “functionally” or “communicatively,” it is This means that the certain component can be connected to the other component directly (eg, wired), wirelessly, or through a third component.

Various embodiments of the present specification may be implemented as software including one or more instructions stored in a storage medium (eg, internal memory or external memory) readable by a machine. For example, the processor of the device may call at least one command among one or more commands stored from a storage medium and execute it. This enables the device to be operated to perform at least one function according to the at least one command invoked. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term is used when data is stored semi-permanently in the storage medium. It does not discriminate when it is temporarily stored.

According to one embodiment, the method according to various embodiments disclosed herein may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. A computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store ^TM ) or between two user terminals (e.g. It can be distributed (eg downloaded or uploaded) online, directly between smart phones. In the case of online distribution, at least part of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium such as a manufacturer's server, an application store server, or a relay server's memory.

According to various embodiments, each component (eg, module or program) of the above-described components may include a single object or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. there is. According to various embodiments, one or more components or steps among the aforementioned components may be omitted, or one or more other components or steps may be added. Alternatively or additionally, a plurality of components (eg modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by a corresponding component among the plurality of components prior to the integration. . According to various embodiments, the steps performed by a module, program or other component are executed sequentially, in parallel, iteratively, or heuristically, or one or more of the steps are executed in a different order, or omitted. or one or more other steps may be added.

1A illustrates a motion image acquisition system according to various embodiments of the present invention. The motion image acquisition system 100 may include a drone 110 , a user device 120 , and a server 130 . The drone 110 may photograph an athlete performing an exercise while flying. Here, the athlete may be not only a professional athlete but also anyone who wants to acquire an exercise image. The drone 110 may include a plurality of ranging sensors, an IMU sensor, and a camera. The distance measurement sensor of the drone 110 may include a distance measurement sensor disposed radially toward the outer surface of the drone, a distance measurement sensor disposed toward the ground, and a distance measurement sensor disposed toward the top of the drone. An example of distance measurement sensors disposed radially toward the outer surface of the drone will be described later with reference to FIG. 5A. The distance measurement sensor of the drone 110 may be at least one of an ultrasonic sensor, an infrared sensor, a radar, a PSD sensor, a LiDAR, a ToF sensor, and a stereo camera. The IMU sensor of the drone 110 may include an acceleration sensor, a gyro sensor, and a geomagnetic sensor. The acceleration sensor included in the IMU sensor may be at least one of a piezoelectric type, a piezoresistive type, a capacitive type, and a thermal type. The gyro sensor included in the IMU sensor may be, for example, an MEMS type. The geomagnetic sensor included in the IMU sensor may be a geomagnetic sensor using at least one of a Hall effect, a magnetoresistance effect, and magnetic impedance. The drone 110 may include a plurality of motors and communication modules for flight.

The drone 110 may transmit the measurement value of at least one sensor and the image data obtained by the camera as collection data to the user device 120 as described above. In addition, the drone 110 may receive command data generated by the server 130 from the user device 120 and control a plurality of motors to fly according to the command data.

The user device 120 is a device used by an athlete or a coach, and may include at least one of a smart phone, a tablet PC, or a wearable device. The user device 120 may transmit collected data generated by the drone 110 to the server 130 and transmit command data generated by the server 130 to the drone 110 .

The server 130 may generate command data for controlling flight of the drone 110 based on collected data generated by the drone 110 . In addition, the server 130 may generate a motion image based on the collected data generated by the drone 110, check the motion data based on the motion image, and generate various synthesized images based on the motion image. .

1B illustrates a configuration of a server included in a motion image acquisition system according to various embodiments of the present disclosure. Referring to FIG. 1B , a server 130 may include a communication circuit 131 , a processor 132 , and a memory 135 . The communication circuit 131 may transmit information to or receive information from other electronic devices, and the type of communication supported by the communication circuit 131 is not limited.

The processor 132 performs an operation based on data received through the communication circuit 131 and/or data stored in the memory 135, and transmits at least a part of the result of the operation to another electronic device through the communication circuit 131. or stored in the memory 135.

The processor 132 may include a data learning unit 133 and a data recognizing unit 134 . The data learning unit 133 may generate an artificial intelligence model that receives joint point information extracted from one frame image of the exercise video and outputs an angle at which the corresponding image was photographed. The data recognizing unit 134 may preprocess data and provide the preprocessed data to the data learning unit 133 for learning.

At least one of the data learning unit 133 and the data recognizing unit 134 is implemented in the form of a dedicated hardware chip for artificial intelligence, or implemented as part of an existing general-purpose processor (eg, AP or CPU) or graphics-only processor. It could be.

According to various embodiments, unlike the data learning unit 133 and the data recognition unit 134 represented as being included in the server 130 in FIG. 2, the data learning unit 133 and the data recognition unit 134 are Each may be mounted on a separate electronic device.

In this case, the data learning unit 133 and the data recognizing unit 134 are wired or wirelessly connected to each other, so that the model information generated by the data learning unit 133 is provided to the data recognizing unit 134 or the data recognizing unit. Data input to 134 may be provided to the data learning unit 133 as additional learning data.

At least one of the data learning unit 133 and the data recognizing unit 134 may be implemented as a software module. In this case, the software module may be stored in a computer-readable non-transitory recording medium. At least some of the software modules may be provided by an operating system (OS) or may be provided by a predetermined application.

2A and 2B illustrate a motion image acquisition method performed in a motion image acquisition system according to various embodiments of the present disclosure.

In step 210, the user device 120 may receive a first value, a second value, and a third value specifying the relative position of the drone to the player.

Referring to FIG. 2D , the first value may be a value specifying the flight height h of the drone 200d. The flight height h of the drone 200d is the height of the drone 200d based on the ground on which the athlete 210d is exercising. A distance measurement sensor disposed toward the ground of the drone may measure the flight height (h) of the drone 200d and include it as part of collected data.

Referring to FIG. 2C , the second value may be a value specifying a distance r between the first sensor of the drone 200c and the first point on the surface of the player 210c. The first sensor of the drone 200c may include a distance sensor and a camera disposed toward an outer surface of the drone. The first point on the surface of the player 210c means a point having the shortest distance from the first sensor of the drone 200c among the points on the surface of the player 210c. A distance measurement sensor included in the first sensor of the drone 200c may measure a distance (r, 220c) between the first sensor and a first point on the surface of the player 210c and include it as part of the collected data.

Referring to FIG. 2C again, the third value specifies the angular displacement θ of the direction 220c from the first point to the first sensor of the drone 200c based on the frontal direction 230c of the player 210c. It can be a value that As will be described later with reference to FIG. 3B, the server 130 determines the front direction 230c of the player 210c according to the result of analysis by the artificial intelligence model on the image data acquired by the camera and the measurement value of the IMU sensor. As a reference, the angular displacement (θ) of the direction 220c of the drone 200c toward the first sensor may be determined from the first point, and the drone 200c may be operated according to a comparison result between the angular displacement (θ) and the third value. You can control it. According to various embodiments, the angular displacement θ may be defined as having a positive value in a counterclockwise direction and a negative value in a clockwise direction when looking down vertically at the bow 210c and the drone 200c. . According to various embodiments, the angular displacement θ may be specified as a value greater than or equal to 0 degrees and less than 360 degrees. For example, an angular displacement of -10 degrees can be defined as equal to 350 degrees.

After step 210 is performed, the user device 120 may transmit the first value, the second value, and the third value to the drone 110 and the server 130, respectively.

In step 220, the drone 110 flies at the same height as the first value by adjusting the thrust of at least one motor, and measures the flight height (h) by a distance measuring sensor disposed toward the ground to determine the first value and the first value. If it is confirmed that the drone 110 is at the same height, information that the drone 110 is at the same height as the first value may be transmitted to the user device 120 . The user device 120 may transmit information to the server 130 that the drone 110 is at the same height as the first value, and thus the server 130 may transmit information that the drone 110 is at the same height as the first value. can confirm that

In step 231, the server 130 may obtain collection data generated by the drone 110. The collected data may include a measurement value of at least one sensor included in the drone and image data obtained by a camera included in the first sensor. The drone 110 may generate and transmit collected data to the user device 120, and the user device 120 may transmit the collected data to the server 130.

In step 232, the server 130 may analyze the collected data to determine whether the drone 110 is located at a position corresponding to the second value and the third value. When it is determined in step 232 that the drone 110 is not at a position corresponding to the second value and the third value, the server 130 analyzes the collected data in step 233 and the position corresponding to the second value and the third value At least one command data for locating the drone may be derived and transmitted to the user device 120 . The user device 120 may transmit the command data of step 233 to the drone 110. The server 130 checks whether the drone 110 is at a position corresponding to the second value and the third value in step 232 and derives command data in step 233, with reference to FIGS. 3A and 3B. Explain. Steps 231 to 233 may be repeated until it is confirmed that the drone 110 is located at a position corresponding to the second value and the third value. Also, steps 231 to 233 may be performed when the player is standing in the starting position before starting the exercise.

According to various embodiments, at least one operation included in steps 231 to 233 may be performed by the drone 110. For example, after acquiring collected data, the drone 110 may perform step 232 by a processor included in the drone 110 . Thereafter, the processor of the drone 110 derives command data for moving to a position corresponding to the second value and the third value in step 233, and uses the derived command data to at least one command included in the drone 110. motor can be controlled.

3A and 3B illustrate a method performed in a motion image acquisition system according to various embodiments of the present invention. In step 310, the server 130 may generate a first artificial intelligence model by learning angles at which a plurality of human body images are photographed and joint point information extracted from the plurality of human body images. Specifically, the server 130 may store a human body image database including a plurality of human body images photographed at a plurality of predetermined angles having a value of 0 degrees or more and less than 360 degrees. Step 310 is a step performed before the athlete performs an exercise. The human body image database may associate and store a plurality of human body images and angles at which each image was photographed. The plurality of preset angles may be, for example, 0 degrees, 90 degrees, 180 degrees, and 270 degrees, as shown in FIG. 4A . Then, the server 130 may extract joint point information from each of a plurality of human body images included in the human body image database. Thereafter, the server 130 may generate a first artificial intelligence model by learning angles of a plurality of human body images and joint point information extracted from the plurality of human body images. As shown in FIG. 4B , the first artificial intelligence model may be an artificial intelligence model that receives joint point information of an image and outputs an angle at which the corresponding image is photographed. Since the first artificial intelligence model has learned images taken at a plurality of preset angles, the first artificial intelligence model is an image taken at one angle among 0 degrees, 90 degrees, 180 degrees, and 270 degrees, The angle at which the corresponding image was taken can be output with high accuracy. The learning method for generating the first artificial intelligence model is not limited. For example, the first artificial intelligence model may be generated by various machine learning techniques. For example, at least one of a recurrent neural network (RNN), a convolution neural network (CNN), an artificial neural network (ANN), and a transformer model may be used for learning to generate the first artificial intelligence model.

In step 321, the server 130 may obtain collected data. In step 322, the server 130 may analyze the image data included in the collected data and determine whether the first player is located in the center of the image corresponding to the image data. Referring to FIG. 5A , a drone 500 may include a plurality of distance sensors 511, 512, 513, and 514 arranged radially and measuring a distance toward the side of the drone, and a plurality of distance sensors 511, 512 , 513 , and 514 may measure distances from the drone body to external objects existing in outward directions 521 , 522 , 523 , and 524 . Among the plurality of distance sensors 511 , 512 , 513 , and 514 , the first sensor 511 faces the same direction 521 as the photographing direction of the camera. Returning to step 322, determining that the first player is located in the center of the image corresponding to the image data may mean that the first sensor 511 is facing the first player.

When it is determined in step 322 that the first player is not located in the center of the image corresponding to the image data, the server 130 operates in step 323 until the first player is located in the center of the image corresponding to the image data, Command data for causing the drone to rotate in place around a rotational axis perpendicular to the ground may be transmitted to the user device 120 . Command data transmitted to the user device 120 may be transmitted to the drone 110 . 5B and 5C illustrate an operation in which a drone rotates in place according to various embodiments of the present disclosure. As shown in FIG. 5B , in the drone 500b, the direction 521b toward which the first sensor is directed may be located differently from the direction in which the player 530 is located. Upon receiving the command data to cause the drone to rotate in place around a rotational axis perpendicular to the ground, the drone remains in place and continues to transmit collected data while rotating around a rotational axis perpendicular to the ground (step 321). . That is, steps 321 to 323 are repeated until the first sensor faces the player 530, and in step 322, when it is confirmed that the first player is located in the center of the image corresponding to the image data, the drone and the first player The positional relationship of is shown in FIG. 5C. Referring to FIG. 5C, the drone 500c, compared to the drone 500b, rotates only about a rotational axis perpendicular to the ground while maintaining its position, and the direction 521c to which the first sensor is directed is the player 530. You can tell which direction it is located.

According to various embodiments, at least one operation included in steps 321 to 323 may be performed by the drone 110. For example, after acquiring the collected data, the drone 110 may perform step 322 by a processor included in the drone 110 . Then, in step 323, the processor of the drone 110 derives command data for rotating around a rotational axis perpendicular to the ground in place, and at least one motor included in the drone 110 uses the derived command data. can control.

When it is confirmed in step 322 that the first player is located in the center of the image corresponding to the image data, the server 130 acquires the collected data again in step 331 . The server 130 checks whether the measured value of the first distance sensor corresponds to the second value in step 332 . When it is determined that the measurement value of the first distance sensor does not correspond to the second value, the server 130 provides command data based on the comparison between the measurement value of the distance sensor included in the first sensor and the second value in step 333. may be transmitted to the user device 120 . The user device 120 may transmit the command data received in step 333 to the drone 110. The command data transmitted in step 333 causes the drone to move in a direction opposite to the first direction 521c directed by the first distance sensor when the measurement value of the distance sensor included in the first sensor is smaller than the second value. That is, it may be command data that causes the drone to move farther away from the player. Conversely, when the measured value of the distance sensor included in the first sensor is greater than the second value, the command data causes the drone to move in the first direction 521c, that is, command data to bring the drone closer to the player. can be Steps 331 to 333 may also be repeated until it is determined that the measured value of the first distance sensor corresponds to the second value, similarly to steps 321 to 323 .

According to various embodiments, at least one operation included in steps 331 to 333 may be performed by the drone 110. For example, after obtaining collected data in step 331, the drone 110 may perform step 332 by a processor included in the drone 110. After that, the processor of the drone 110 may derive command data in step 333 and control at least one motor included in the drone 110 using the derived command data.

When it is determined in step 332 that the measurement value of the first distance sensor corresponds to the second value, the server 130 performs operation 340 . In operation 340, the server 130 may check a first angle that is smaller than the third value and has the largest value among the plurality of preset angles. For example, when the plurality of preset angles are 0 degrees, 90 degrees, 180 degrees, and 270 degrees as shown in FIG. 4A and the third value is 100 degrees, the first angle may be 90 degrees. As another example, when the third value is 190 degrees, the first angle may be 180 degrees. According to various embodiments, operation 340 may be performed by a processor of the drone 110.

In step 351, the server 130 may obtain collected data again. In step 352, the server 130 may check whether the first artificial intelligence model confirms that the image corresponding to the image data included in the collected data acquired in step 351 is taken from the first angle. When an image corresponding to the image data included in the collected data obtained in step 351 is input to the first artificial intelligence model, if it is not confirmed that the image was taken at the first angle, the server 130 commands an angle increase in step 353. may be transmitted to the user device 120 as command data. The user device 120 may transmit an angle increase command to the drone 110 .

The angle increase command is a command to fly the drone so that the angular displacement θ increases as shown in FIG. 2C, and will be described with reference to FIGS. 6A to 6D. 6A shows a drone before receiving an angle increase command, and in the drone 600a, the first direction 621a is illustratively directed toward the first player 630, and the angular displacement from the front direction of the player is 270 It is stopped at the leading position. The angle increase command is (a) a command to move the drone by a preset first distance in a direction perpendicular to the first direction 621a, (b) when the first player is located in the center of the image corresponding to the image data and (c) a command to move the drone in the first direction until the measured value of the first distance sensor corresponds to the second value. there is.

First, (a) a drone 600b moved according to a command to move the drone by a predetermined first distance in a direction perpendicular to the first direction 621a is shown in FIG. 6B. Looking at the position of the drone, compared to the previous position 600a, it moved by a preset first distance in the direction 601b perpendicular to the first direction 621a, and there was no rotation of the drone about a rotation axis perpendicular to the ground. . As a result, the first direction 621b is parallel to the first direction 621a before movement, but does not face the first player 630.

Next, (b) the drone 600c moved according to the command to rotate the drone in place on a plane parallel to the ground until the first player is located in the center of the image corresponding to the image data It is shown in Figure 6c. Looking at the position of the drone, the position is the same as the position of the drone 600b before movement, and as a result of rotating about the axis of rotation perpendicular to the ground, the first direction 621c is directed toward the first player 630, and as a result, the first 1 The player is located in the center of the video corresponding to the video data. At this time, the angle θ1 rotated by the drone until the first player is located in the center of the image corresponding to the image data is the angular displacement change (θ1) in the direction toward the drone based on the first player 630 and are the same, and these angular changes are accumulated and recorded by the IMU sensor. Therefore, the server 130 can check the angular displacement change in the direction toward the drone 600c with respect to the first player 630, that is, how many degrees the drone has rotated with respect to the first player 630. .

Next, FIG. 6D shows the drone 600d moving according to the command to move the drone in the first direction until the measurement value of the first distance sensor corresponds to the second value. It can be seen that only the position of the drone 600d is changed while the first direction 621d is maintained the same as the previous first direction 621c.

After performing step 353, steps 351, 352, and 353 may be repeated until the image data corresponding to the image confirmed to have been taken at the first angle is included in the collected data and received by the server 130. . When it is confirmed that the image data was taken at the first angle, the server 130 may set the angle value of the IMU sensor to 0 in step 360 and receive the collected data again in step 361 .

According to various embodiments, at least one operation included in steps 351 to 353 and 360 may be performed by the drone 110 . For example, the drone 110 may perform step 352 by a processor included in the drone 110 after acquiring the collected data. After that, the processor of the drone 110 may generate an angle increase command in step 353 and control at least one motor included in the drone 110 by using the generated angle increase command.

In step 362, the server 130 may check whether the angle value indicated by the IMU sensor corresponds to a value obtained by subtracting the first angle from the third value. When the angle value indicated by the IMU sensor does not correspond to the value obtained by subtracting the first angle from the third value, an angle increase command may be transmitted as command data in step 363 . Steps 361, 362, and 363 may be repeated until the angle value indicated by the IMU sensor corresponds to a value obtained by subtracting the first angle from the third value. When the angle value indicated by the IMU sensor corresponds to the value obtained by subtracting the first angle from the third value, the server 130 confirms that the drone 110 is located at a position corresponding to the second and third values in step 370. can

According to various embodiments, at least one operation included in steps 361 to 363 and 370 may be performed by the drone 110 . For example, after obtaining collected data, the drone 110 may perform step 362 by a processor included in the drone 110 . After that, the processor of the drone 110 may generate an angle increase command in step 363 and control at least one motor included in the drone 110 by using the generated angle increase command.

When it is confirmed through the process shown in FIGS. 3A and 3B that the drone 110 is located at the position corresponding to the second value and the third value in step 232, the server 130 signals the completion of initial location setting in step 240. may be transmitted to the user device 120. The user device 120 may transmit an initial location setting completion signal to the drone. In addition, the user device 120 may output a preset sound through an output device (eg, a speaker) in response to receiving an initial position setting completion signal or display that the initial position setting has been completed through a display. can

According to various embodiments, when step 232 is performed by the processor of the drone 110, step 240 may be performed when the drone 110 generates an initial location setting completion signal and transmits it to the user device 120.

The drone 110 that has generated or received the initial positioning completion signal may also output a preset sound through a speaker in response to the initial positioning completion signal being generated or in response to the initial positioning completion signal being received. . A preset sound output through a speaker of the drone 110 or the user device 120 may be, for example, the same as a sound announcing the start of an actual game.

After operation 240 is performed, the player can start exercising by listening to the sound output through the speaker or seeing the message displayed on the display of the user device 120 . The drone 110 may obtain collected data and transmit it to the user device 120 while flying in the front direction of the player while the player is exercising, and the user device 120 may transmit the collected data to the server 130 ( Step 251).

The server 130 may check whether the exercise is completed in step 252 . According to various embodiments, the server 130 may analyze the image data in the collected data and determine that the exercise is completed when the time during which the player moves at a predetermined speed or less is equal to or longer than a predetermined first time. According to various embodiments, the server 130 may determine that the exercise is complete when receiving a signal indicating that the exercise is over from the user device 120 receiving the player's or coach's input.

When it is determined that the exercise is not completed, that is, until the exercise is over, the server 130 provides at least one command for locating the drone at a position corresponding to the first value, the second value, and the third value in step 253. Data may be derived and transmitted to the user device 120 . The user device 120 may transmit the command data of step 253 to the drone 110.

The process of deriving the command data from the server 130 when the exercise is not completed is similar to that described above. First, since the h value, which is the flying height of the drone 110, is included in the collected data, the server 130 compares the h value in the collected data with the first value, and if the h value is smaller than the first value, the drone is located on the ground. Command data for increasing the propulsion force for the drone may be transmitted, and command data for decreasing the propulsion force for the ground of the drone may be transmitted when the value h is greater than the first value.

The r value, which is the distance between the drone 110 and the player, is a measurement value of a distance sensor included in the first sensor when the player is located in the center of an image corresponding to the image data. Accordingly, the server 130 may control the drone 110 to be at a position corresponding to the second value by performing steps 321 to 323 and steps 331 to 333.

The θ value, which is the angular displacement in the direction from the player to the drone based on the player's frontal direction, is the rotation angle when the drone rotates so that the player is located in the center of the image corresponding to the image data, as described above in FIG. 6C. change as much Since the value of θ is set to be equal to the third value immediately before the start of the exercise, the server 130 rotates the drone so that the player is positioned at the center of the image corresponding to the image data so that the angular displacement can be maintained after the start of the exercise. While doing so, if it is determined that the angular displacement has decreased, an angle increase command may be transmitted as command data, and if it is determined that the angular displacement has increased, an angle decrease command may be transmitted as command data. The contents of the angle reduction command are the same as those described above with reference to FIGS. 6A to 6D , but only the movement direction in FIG. 6B is the direction in which the angular displacement decreases, that is, the direction opposite to the movement direction 601b shown in FIG. 6B.

According to various embodiments, at least one operation included in steps 252 to 253 may be performed by the drone 110 . For example, after obtaining collected data in step 251, the drone 110 may perform step 252 by a processor included in the drone 110. After that, the processor of the drone 110 derives command data for moving to a position corresponding to the second value and the third value in step 253, and uses the derived command data to at least one command included in the drone 110. motor can be controlled.

When it is confirmed that the exercise is complete, the server 130 may store the exercise image based on the collected data during the exercise, that is, the collected data received in step 251 in step 260 . The motion image is obtained by extracting a part corresponding to the time the drone flew at a position corresponding to the first value, the second value, and the third value among the image data included in the at least one collection data collected in step 251. can

According to various embodiments, when at least one operation included in steps 252 to 253 is performed by the drone 110, in step 260, the drone 110 transmits collected data while the exercise is being performed to the server 130. And, among the image data included in the collected data, the server 130 may extract and store a portion corresponding to the time the drone flew at a location corresponding to the first value, the second value, and the third value. .

7 illustrates an avoidance maneuver method performed in a motion image acquisition system according to various embodiments of the present disclosure. The avoidance maneuver is an operation performed to protect the drone 110 or an athlete when an external object approaches the drone 110 while the drone 110 is in flight.

In step 710, the server 130 may confirm that an unidentified object is approaching the drone 110 based on at least one piece of collected data while the first player is exercising. For example, in a situation where it is confirmed through the image data that the first sensor is facing the player, other distance sensors (eg, sensor 512 of FIG. , 513, 514)), when an unidentified object is detected and the distance to the unidentified object is getting closer, it may be determined that the unidentified object is approaching the drone 110 .

In step 720, the server 130 may check the direction in which the unidentified object approaches the drone and whether the object exists on top of the drone. The server 130 may check a direction detected by a sensor that detects an unidentified object as a direction in which the unidentified object approaches the drone. For example, when an unidentified object is detected through the sensor 512 of FIG. 5A and the distance to the unidentified object is getting closer, the direction in which the unidentified object approaches the drone is the detection direction 522 of the sensor 512. You can check that it is in the opposite direction. The server 130 may determine that an object exists at the top of the drone when a measurement value from a distance measuring sensor disposed toward the top of the drone is equal to or less than a predetermined value.

In step 730, the server 130 may check whether a difference between the direction from the first sensor to the first point and the second direction is greater than a preset second angle. The second direction refers to a direction in which an unidentified object approaches the drone.

When it is determined in step 730 that the difference between the direction from the first sensor toward the first point and the second direction is greater than the preset second angle, the server 130 transmits command data for stopping and flying the drone to the user device 120 in step 740. ) can be sent. The user device 120 may transmit command data to the drone 110 . Stationary flight means that the drone 110 maintains its position without rotating or changing its position based on a rotational axis perpendicular to the ground.

When it is determined in step 730 that the difference between the direction from the first sensor to the first point and the second direction is equal to or less than the preset second angle, the server 130 may check whether an object exists on top of the drone in step 750. there is.

When it is determined in step 750 that an object is present on top of the drone, the server 130 may transmit command data for blocking motor operation of the drone in step 760 . The user device 120 may transmit command data to the drone 110 . Upon receiving the command data of step 760, the drone 110 blocks the operation of the motor, and as a result, immediately falls to the ground.

If it is determined in step 750 that there is no object above the drone, the server 130 may transmit command data for rapidly ascending the drone to an altitude equal to or higher than a preset first altitude in step 770 . The user device 120 may transmit command data to the drone 110 . Upon receiving the command data in step 770, the drone 110 rapidly rises to an altitude equal to or higher than the preset first altitude while maintaining coordinates on a plane parallel to the ground.

According to various embodiments, at least one operation included in each step of FIG. 7 may be performed by the drone 110. The checking operation of steps 710 to 730 and 750 may be performed by a processor included in the drone 110 . Steps 740, 760, and 770 are performed by generating corresponding command data in the processor included in the drone 110 and controlling at least one motor included in the drone 110 using the generated command data. can

According to various embodiments, an evasion maneuver method other than that shown in FIG. 7 may be performed. The server 130 or the processor of the drone 110 may confirm that the unidentified object approaches the drone 110 within a predetermined second distance based on at least one piece of collected data while the first player is exercising. When an unidentified object approaching within a predetermined second distance from the drone 110 is identified, the server 130 or the processor of the drone 110 determines the distance between the unidentified object and the drone 110 based on at least one piece of collected data. can be continuously monitored. When the distance between the unidentified object and the drone 110 is confirmed to be less than a predetermined third distance shorter than the second distance, the server 130 or the processor of the drone 110 determines that the object exists on top of the drone based on the collected data. If there is an object on top of the drone, command data to block the motor operation of the drone is generated, and if there is no object on top of the drone, command data to rapidly raise the drone to a preset altitude of the first altitude or higher is generated. can create

The server 130 or the processor of the drone 110 may confirm that the first player approaches the drone 110 within a predetermined second distance based on at least one piece of collected data while the first player is exercising. . Here, the second distance may be a smaller value than the second value described above with reference to FIG. 2C. When it is confirmed that the first player has approached within the predetermined second distance from the drone 110, the server 130 or the processor of the drone 110 sends the drone 110 and the first player based on at least one piece of collected data. It is possible to generate command data such that the distance between them becomes the second value. If the distance between the first player and the drone 110 is determined to be less than a predetermined third distance shorter than the second distance, the server 130 or the processor of the drone 110 places an object on top of the drone based on the collected data. Checks whether there is an object on top of the drone, generates command data to block motor operation of the drone if there is an object on top of the drone, and commands to rapidly raise the drone to an altitude higher than the first preset altitude if there is no object on top of the drone. data can be generated.

Although not shown in the drawings, according to various embodiments, the server 130 may determine and respond to a case in which normal image data cannot be collected because the camera of the drone 110 is obscured by an external object. The server 130 may determine that the camera is covered by an external object when the ratio of the area whose brightness is equal to or less than the preset first brightness in the image corresponding to the video data is equal to or greater than the preset ratio. Command data for maintaining the motor rotation speed, tilt angle, and moving direction of the drone for a set first time period may be transmitted to the user device. The user device 120 may transmit corresponding command data to the drone 110 .

In addition, the server 130 stops transmitting the command data when a state in which the ratio of an area whose brightness is equal to or less than the first preset brightness in the image corresponding to the video data is equal to or greater than the preset ratio continues for a second preset time or longer, and , a message indicating that the drone is switched to manual mode may be transmitted to the user device. In this case, the operation of the drone 110 may be controlled not by the server 130 but by the athlete or coach. For example, a signal for manually controlling the drone 110 may be input to the user device 120 .

According to various embodiments, the server 130 may control the drone 110 to fly along the changed runner as the runner changes in a relay race situation. The server 130 may store the average color value of the top and bottom color of the stopped first player before transmitting the initial position setting completion signal. Thereafter, it may be confirmed that there is a second player different from the first player in the image corresponding to the image data collected while the first player is exercising. Various artificial intelligence models capable of identifying the shape of a person in the image data may be used to identify the second player. If it is confirmed that the second player is in the image, the server 130 may check the average value of the color of the second player's upper garment and the average value of the color of his lower garment.

The server 130 determines whether the difference between the average color value of the second player's shirt and the average color value of the first player's shirt is equal to or less than a preset fourth value, or the difference between the average color value of the second player's shirt and the average color value of the first player's shirt is a preset value. If the value is 5 or less, the moving direction of the second player and the moving direction of the first player can be checked. Here, the fourth value and the fifth value may be sufficiently small values to be determined to be the same color when viewed with the naked eye. According to various embodiments, the server 130 receives an exception processing color from the user device 120, and even if the difference between the average color value of the second player's shirt and the first player's shirt color is equal to or less than a preset fourth value, the first player When the difference between the average color of the 2 player's shirt and the exception processing color is less than or equal to the fourth preset value, and the difference between the average color of the first player's uniform and the exception processing color is less than or equal to the preset fourth value, the moving direction of the second player is not checked. The drone may be controlled to continuously track only the first player without interruption. The exception handling color is a color that is commonly used for sportswear and everyday wear and is rarely used for team division, and may be determined according to a user's input, for example, among black and/or gray. According to various embodiments, the server 130 searches for various keywords representing exercise among posts on social media, recognizes a person's shape in posts filtered through the search results, and sets the top color and bottom color when the person's shape is recognized. After extracting and creating a database of the frequency of appearance of the top and bottom colors extracted from the filtered posts, the top or bottom color with a high frequency of appearance may be designated as an exceptional color. Various keywords indicating exercise may be pre-specified by the manager of the server. According to various embodiments, the server 130 searches for various keywords representing sportswear among posts on the Internet open market, obtains a database of review photos included in posts filtered through the search results, and each review photo included in the database. Recognize the human form in the photo, extract the upper and lower colors when the human form is recognized, and database the frequency of appearance of the upper and lower colors extracted from the filtered posts, You can designate the bottom color as the exception processing color. Various keywords representing sportswear may be designated in advance by a manager of the server.

According to various embodiments, the server 130 transmits designated exception handling color information to the user device, and the user device may display exception handling colors.

Similarly for the bottoms, the server 130 determines that even if the difference between the average color value of the second player's bottoms and the average color value of the first player's bottoms is equal to or less than the preset fifth value, the difference between the average color value of the second player's bottoms and the exception handling color is set in advance. If it is less than the fifth value and the difference between the average color value of the first player's bottom and the exception processing color is less than or equal to the preset fifth value, the drone may be controlled to continuously track only the first player without checking the moving direction of the second player. there is.

After confirming the moving direction of the second player and the moving direction of the first player, the server 130 determines that the two players move in the same direction as the average moving direction of the drone for the second preset time, and the first player moves in the same direction as the drone's average moving direction. When moving in a direction different from the moving direction for 2 hours, the target to be tracked by the drone may be changed from the first player to the second player. That is, the server 130 derives at least one command data for locating the drone at a position corresponding to the first value, the second value, and the third value based on the second player and transmits it to the user device 120. can do. The user device 120 may transmit at least one command data to the drone.

8 illustrates a method of generating a 3D human body model performed in a motion image acquisition system according to various embodiments of the present disclosure. The generation of the 3D human body model of FIG. 8 may be performed while the athlete is stationary before the athlete starts exercising.

In step 810, the server 130 may obtain first collection data collected by the drone 110.

In step 820, the server 130 may check whether the rotational motion is completed. Completion of the rotational motion means that the drone 110 hovered around the player a predetermined number of times. The server 130 may confirm through image analysis that the drone is located at the smallest angle (eg, 0 degree) among a plurality of preset angles, and repeats an angle increase command starting from 0 degree to adjust the angle of the drone. displacement can be increased. Similar to steps 351 to 363 of FIG. 3B, the server 130 operates when the angular displacement θ of the drone 110 is a plurality of preset angles based on the first artificial intelligence model described above with reference to FIG. 4B. This can be checked every time, and the value obtained by adding the measured value of the IMU sensor can be checked as the current angular displacement of the drone. Similar to steps 351 to 363 of FIG. 3B , the server 130 may reset the measured value of the IMU sensor to 0 whenever the angular displacement θ of the drone 110 becomes a plurality of preset angles.

If the rotational movement is not completed, the server 130 may transmit at least one command data for rotating around the stopped first player to the user device 120 in step 830 . The user device 120 may transmit the command data of step 830 to the drone 110. The command data in step 830 may include command data for setting the distance between the first sensor of the drone and the player to a preset first distance and the above-described angle increase command.

When the rotation motion is completed, the server 130 may generate a 3D human body model of the first player in step 840 based on the first collected data collected in step 810 . The server 130 may extract joint points from an image corresponding to the image data included in the first collection data, and generate a three-dimensional human body model of the first player based on joint point information obtained from various angles. there is.

According to various embodiments, at least one operation included in each step of FIG. 8 may be performed by the drone 110 instead of the server 130 . The checking operation of step 820 may be performed by a processor included in the drone 110. Step 830 may be performed by generating command data in a processor included in the drone 110 and controlling at least one motor included in the drone 110 using the generated command data. Step 840 may also be performed by a processor included in the drone 110.

9 illustrates a motion data acquisition method performed in a motion image acquisition system according to various embodiments of the present disclosure. In step 910, the server 130 may obtain an exercise image. Here, the motion image means the motion image stored in operation 260 of FIG. 2B, and the motion image is the relative position of the drone to the player by the first value, the second value, and the third value input to the user device. This is an image acquired when the location is specified.

In operation 920, the server 130 may extract joint points from each image constituting each frame of the motion image.

In operation 930, the server 130 may check the exercise data based on the joint points extracted from the exercise image and the 3D human body model of the first player. The exercise data may include the first player's joint movable range, joint movable speed, and periodic changes in the joint movable range and joint movable speed in a periodically repeated motion. The 3D human body model of the first player means a 3D human body model generated through the process of FIG. 8 . The server 130 may transmit the verified exercise data to the user device, or generate a synthesized image by overlaying the verified exercise data on the exercise image, and transmit the generated synthesized image to the user device.

In operation 940, the user device 120 may display exercise data. Examples of displaying exercise data are shown in FIGS. 10A and 10B. In the example of FIG. 10A , the motion range of the joint and the body contour of the player are displayed as exercise data. In the example of FIG. 10B , changes in joint range of motion for each repetition cycle of a running motion that the athlete periodically repeats and body contours of the athlete are displayed as exercise data.

11 illustrates a synthetic image providing method according to various embodiments of the present invention. In step 1110, the server 130 may obtain an exercise image. Here, the exercise image means a motion image stored in operation 260 of FIG. This is an image acquired when the location is specified.

In step 1120, the server 130 may check the exercise data of the first player from the exercise image of the first player. A method for acquiring exercise data has been described above with reference to FIGS. 8 and 9 .

In step 1130, the server 130 may obtain a plurality of partial exercise images by parsing the exercise image of the first player. When an exercise motion is repeated, the server 130 may parse the exercise image for each repetition period. Alternatively, the server 130 may parse the exercise image according to a predetermined time interval. Alternatively, the server 130 may parse the exercise image according to a predetermined recording point. For example, in the case of a 100m run, exercise images may be parsed at 30m points, 50m points, 80m points, and 100m points. The server 130 corresponds to a certain frame of the motion image at a time point passing through a specific point (30m point, 50m point, 80m point, and 100m point) based on the linear movement speed for each hour of the drone 110 included in the collected data. can decide whether

According to various embodiments, various algorithms for parsing exercise images for each repetition period of repeated exercise motions may be used. For example, the server 130 extracts the angle A formed by the three joint points of the left knee-pelvis-right knee from the motion image based on the joint point analysis, and when the value of the angle A is the largest within a certain time range An exercise image can be parsed based on .

In step 1140, the server 130 sends a first partial exercise image, a second partial exercise image, exercise data corresponding to the first partial exercise image, and exercise data corresponding to the second partial exercise image among a plurality of partial exercise images. A synthesized image can be generated based on this. For example, as shown in FIG. 12A , a synthetic image may be generated by increasing the transparency of the two motion images and overlaying the motion data corresponding to the two partial motion images with increased transparency and the motion data corresponding to the two motion images. .

According to various embodiments, the server 130 may generate a synthesized image by synthesizing three or more partial exercise images and corresponding exercise data. For example, as shown in FIG. 12B , four partial motion images and corresponding motion data may be overlaid and displayed on a synthesized image.

In step 1150, the server 130 may transmit the synthesized video generated in step 1140 to the user device. In step 1160, the user device may display a synthesized image. As shown in FIGS. 12A and 12B , motion data corresponding to different partial images may be displayed in different colors.

Unlike the method described in FIG. 11 in which partial images corresponding to different timings are synthesized and provided within one exercise when the same athlete performs one exercise, different players perform the same exercise according to various embodiments. It is also possible to synthesize and provide partial images corresponding to the same timing in two exercise images based on motion images captured during movement. That is, the server 130 obtains an exercise image of the second player, identifies exercise data of the second player from the exercise image of the second player, parses the exercise image of the second player for each cycle of the exercise motion, A plurality of partial exercise images corresponding to the player are obtained, transparency of a third partial exercise image corresponding to a time point at which a first partial exercise image among the plurality of partial exercise images corresponding to the second player is captured is increased, and transparency is increased. A third partial motion image with increased transparency, motion data corresponding to the first partial motion image, and motion data corresponding to the third partial motion image are overlaid on the increased first partial motion image to generate a second synthetic image; , the second synthesized video may be transmitted to the user device. The user device may display the second synthesized image in the form illustrated in FIGS. 12A and 12B.

13 illustrates a synthetic image providing method according to various embodiments of the present invention. 13 is a method of providing an entire exercise process as a single composite image so that changes in motion during exercise can be seen at once.

In step 1310, the server 130 may obtain a first exercise image of the first player. Here, the motion image means the motion image stored in operation 260 of FIG. 2B, and the motion image is the relative position of the drone to the player by the first value, the second value, and the third value input to the user device. This is an image acquired when the location is specified.

In step 1320, the server 130 may obtain a first background image and a first player image by separating the background region and the first athlete region from the first exercise image. Referring to FIG. 14A , the server 130 may obtain a first background image 1410a and a first player image 1420a from a first exercise image 1400a.

In step 1330, the server 130 may generate a background image by synthesizing a plurality of frames of the first background image. The server 130 synthesizes a plurality of frames of the first background image based on the flight speed of the drone and information on the time when the first image data was captured, which are included in the collected data that is the basis of the first exercise image of the first player. to create a background image. According to various embodiments, the server 130 learns the image of the start line and the end line of the land track, recognizes the start line and the end line shown in the image data based on the similarity determination, and as shown in FIG. 14B, the start line You can create a background image so that lines and arrival lines appear on the background image.

In step 1340, the server 130 may generate a first synthesized image by overlaying a plurality of frames of the first player image on the background image. The server 130 determines which position on the background image the plurality of frames of the first player image corresponds to based on the flight speed of the drone and the time information at which the first image data was captured, and as shown in FIG. 14C Similarly, a first composite image 1420c in which a plurality of frames 1411c, 1412c, 1413c, 1414c, and 1415c of the first player image are overlaid on the background image may be generated.

In step 1350, the server 130 may transmit the first synthesized video to the user device. In step 1360, the user device may display the first synthesized image. When the first synthesized video 1420c is reproduced, only the initial frames 1411c of the first player video are initially displayed, and the frames 1412c corresponding to the timing when the actual player reaches the position 1412c are reproduced. The plurality of frames of the first player image may be sequentially reproduced in such a way that the last frame of the initial frames 1411c remains in a still state.

14A illustrates an example of a background image and a player image obtained according to various embodiments of the present invention.

14B illustrates an example of a background image obtained according to various embodiments of the present invention.

14C illustrates an example of a synthesized image obtained according to various embodiments of the present invention.

15 illustrates a synthetic image providing method according to various embodiments of the present invention. In step 1510, the server 130 may obtain a second exercise image of the second player. The second exercise image may be an image obtained by capturing a scene in which the second athlete performs the same exercise as the first athlete, for example, a 100m run. Similarly here, the motion image means the motion image stored in operation 260 of FIG. It is an image obtained when the position is specified by

In step 1520, the server 130 may obtain a second player image by separating the background area and the second player area from the second exercise image.

In step 1530, the server 130 may generate a second synthesized image by overlaying at least one frame of the second player image on the first synthesized image described above with reference to FIG. 13 . The server 130 determines the position of the plurality of frames of the second player image on the background image based on the flight speed of the drone and the time information at which the image data was captured, which are included in the collected data that is the basis for acquiring the second player image. It is determined whether it corresponds to , and a second synthesized image may be generated by overlaying the corresponding frame on the first synthesized image.

In step 1540, the server 130 may transmit the second synthesized image to the user device, and the user device may display the second synthesized image in step 1550.

As shown in FIG. 16A, a part of the image of the first player and a part of the image of the second player may be overlaid, and the overlaid position eventually corresponds to the reproduction of the actual record of the first player and the second player. As shown in FIG. 16A, the first player 1610a lags behind the second player 1620a at the beginning of the race, while the first player 1610b lags behind the second player 1620b in the middle of the race, as shown in FIG. 16B. ), it can be seen that Athletes can use this as a reference for establishing their own pace strategy by comparing their pace with other runners.

According to various embodiments of the present invention, the drone flies along the athlete after the athlete starts exercising based on recognizing the starting line of the running track from the image data, based on the flight speed and flight direction of the drone. , it is possible to calculate how many meters the drone has advanced from the starting line in the direction of the athlete on a straight track. In addition, the drone is a state in which the player advances several meters from the starting line based on the distance the drone has advanced from the starting line, the distance between the user and the drone, and the angle formed by the direction from the player to the drone based on the frontal direction of the player. Perception calculations can be made and stored as player records. The drone further includes a light source and a light source control device capable of radiating light in a specific direction, and while the second player is exercising, while flying along the second player, while checking the player record of the second player, the first player Light is radiated to a position corresponding to the player's record, and the light source may be controlled so that the position corresponding to the first player's record is displayed to the second player on the main image.

According to various embodiments of the present disclosure, a motion image acquisition method performed in a motion image acquisition system including a user device, a server, and a drone includes a first method for specifying a flight height of the drone from a user in the user device. a value, a second value specifying a distance between a first sensor of the drone and a first point on the surface of a first player, and a first point of the drone from the first point based on a frontal direction of the first player. Receiving a third value specifying the angular displacement in the direction toward the sensor - The first point is a point on the surface of the first player with the shortest distance from the drone, and the third value is 0 degrees or more 360 has a value less than degrees, the first sensor includes a first distance sensor and a camera -, the user device transmits the first value, the second value, and the third value to the server and the drone doing; checking information that the drone is at the same height as the first value; Acquiring a measurement value of at least one sensor of the drone and image data acquired by a camera included in the first sensor as at least one collected data - The at least one sensor is disposed radially from the first sensor including a plurality of distance sensors, and an IMU sensor; deriving at least one command data for locating the drone at a position corresponding to the second value and the third value by analyzing the at least one collected data; generating an initial location setting completion signal in response to determining that the drone is located at a location corresponding to the second value and the third value; In response to the initial positioning completion signal, the drone outputs a preset sound through a speaker and obtains the at least one collection data while flying in the front direction of the first player while the first player is exercising. ; Analyzing the at least one collected data and positioning the drone at a position corresponding to the first value, the second value, and the third value based on the first player while the first player is exercising deriving at least one command data for; And based on the at least one collected data, the time at which the drone flew at a position corresponding to the first value, the second value, and the third value among the image data included in the at least one collected data. A step of extracting the corresponding part and storing it as an exercise image may be included.

According to various embodiments of the present invention, the method includes: storing a human body image database including a plurality of human body images photographed at a plurality of preset angles having a value of 0 degrees or more and less than 360 degrees; extracting joint point information from each of a plurality of human body images included in the human body image database; generating a first artificial intelligence model by learning angles at which the plurality of human body images are photographed and joint point information extracted from the plurality of human body images; In order to position the drone at a position corresponding to the second value and the third value: until the first player is located in the center of the image corresponding to the image data, the drone is perpendicular to the ground in place deriving command data for rotating about a rotation axis that is; When it is determined that the first player is located in the center of the image corresponding to the image data, the distance sensor included in the first sensor continues until the measured value of the first distance sensor corresponds to the second value. If the measured value is smaller than the second value, command data for causing the drone to move in a direction opposite to a first direction toward which the first distance sensor is directed is transmitted to the user device, and a distance sensor included in the first sensor deriving command data for causing the drone to move in the first direction when the measured value of is greater than the second value; if it is confirmed that the measured value of the first distance sensor corresponds to the second value, checking a first angle having a largest value and smaller than the third value among the plurality of preset angles; Until the video data corresponding to the image confirmed to have been taken at the first angle based on the first artificial intelligence model is received, the drone is directed to the first point based on the front direction of the first player. deriving, as command data, an angle increase command for increasing an angular displacement in a direction toward the drone from the angle increase command is a command to move the drone by a preset first distance in a direction perpendicular to the first direction. , A command to rotate the drone in place on a plane parallel to the ground until the first player is located in the center of the image corresponding to the image data, and the measured value of the first distance sensor is the second including a command to move the drone in the first direction until a value is reached; and deriving the angle increase command as command data until the angle value indicated by the IMU sensor is a value obtained by subtracting the first angle from the third value.

According to various embodiments of the present disclosure, based on the at least one collection data while the first player is exercising, confirming that an unidentified object is approaching the drone; determining whether an object exists in a second direction, which is a direction in which the unidentified object approaches the drone, and on top of the drone, based on the confirmation that the unidentified object is approaching the drone; If the difference between the direction from the first sensor to the first point and the second direction is equal to or less than a preset second angle and it is determined that there is no object on top of the drone, move the drone to an altitude equal to or higher than a preset first altitude. deriving command data to jump; When the difference between the direction from the first sensor to the first point and the second direction is equal to or less than a preset second angle and it is determined that an object exists on top of the drone, command data for blocking motor operation of the drone is transmitted. deriving; and deriving command data for stopping and flying the drone when a difference between the second direction and the first direction is greater than a preset second angle.

According to various embodiments of the present invention, in the method, while the first player is exercising, when the ratio of a region having a brightness equal to or less than a preset first brightness in the image corresponding to the video data is greater than or equal to a preset ratio, deriving command data for maintaining a rotational speed of a motor, a tilt angle, and a moving direction of the drone for a first time; When a state in which the ratio of a region whose brightness is equal to or less than the first preset brightness in the image corresponding to the video data is equal to or greater than the preset ratio lasts for a preset second time or longer, derivation of command data is stopped, and the drone is sent to the user device. The method may further include transmitting a message indicating that the device is switched to a manual mode.

According to various embodiments of the present invention, the method may include: before generating an initial position setting completion signal, storing an average color value of an upper garment and an average color value of a lower garment of the stopped first player; If it is confirmed that there is a second player different from the first player in the image corresponding to the video data, checking the average color value of the upper and lower garments of the second player; The difference between the average color value of the second player's shirt and the average color value of the first player's shirt is equal to or less than a preset fourth value, or the difference between the average color value of the second player's shirt and the average color value of the first player's shirt reaches a preset value of 5 If the value is less than or equal to, checking the moving direction of the second player and the moving direction of the first player; and when the second player moves in the same direction as the average moving direction of the drone for the second time period and the first player moves in a direction different from the moving direction of the drone during the second time period, the second player deriving at least one command data for positioning the drone at a position corresponding to the first value, the second value, and the third value based on the player; may further include.

According to various embodiments of the present disclosure, a method for providing exercise data may include deriving at least one command data for causing a drone to rotate around a first player who is stationary; Obtaining at least one first collection data including a measurement value of at least one sensor included in the drone and image data obtained by a camera included in the drone while the drone rotates around the first player when the drone is stopped step; generating a three-dimensional human body model of the first player based on the at least one first collected data; In the user device, a first value specifying a flying height of the drone, a second value specifying a distance between a first sensor of the drone and a first point on the surface of a first player, and a front of the first player. A third value specifying an angular displacement in a direction from the first point toward the first sensor of the drone is received based on the direction, and the server and the drone receive the first value, the second value, and the Transmitting the third value - the first point is a point at which the distance to the drone is the shortest on the surface of the first player, the third value has a value of 0 degrees or more and less than 360 degrees, 1 sensor includes a first distance sensor and the camera; Second collected data including image data acquired by the camera and measurement values of at least one sensor included in the drone while the drone flies in the front direction of the first player while the first player is exercising Obtaining at least one; By analyzing the at least one second collected data, the drone is located at a position corresponding to the first value, the second value, and the third value based on the first player while the first player is exercising. deriving at least one command data for locating; Based on the at least one second collected data, among the image data included in the at least one second collected data, the drone flies at a position corresponding to the first value, the second value, and the third value. extracting a portion corresponding to one hour and storing it as an exercise image; extracting joint points from the motion image; Based on the joint point extracted from the motion image and the 3D human body model of the first player, the joint movable range and the joint movable in the joint movable range, joint movable speed, and periodically repeated motion of the first player. Checking motion data representing a change in speed for each period; and displaying the exercise data on the user device.

According to various embodiments of the present invention, the method further includes generating a synthesized image by overlaying the exercise data on the exercise image, and transmitting the generated synthesized image to a user device, wherein the user device transmits the exercise data. The displaying may include displaying the synthesized image.

According to various embodiments of the present invention, the method includes: storing a human body image database including a plurality of human body images photographed at a plurality of predetermined angles having a value of 0 degrees or more and less than 360 degrees; extracting joint point information from each of a plurality of human body images included in the human body image database; and generating a first artificial intelligence model by learning angles at which the plurality of human body images were photographed and joint point information extracted from the plurality of human body images; may further include.

According to various embodiments of the present invention, deriving at least one command data for causing the drone to rotate around a first player who has stopped; deriving command data for causing the drone to rotate around a rotational axis perpendicular to the ground in place until the first player is located in the center of the image corresponding to the image data; When it is confirmed that the first player is located in the center of the image corresponding to the image data, the distance sensor included in the first sensor until the measured value of the first distance sensor corresponds to the preset first distance. If the measured value of is smaller than the first distance, command data for causing the drone to move in a first direction opposite to the first direction toward which the first distance sensor is directed is derived, and the measured value of the distance sensor included in the first sensor deriving command data for causing the drone to move in the first direction when the distance is greater than the first distance; When it is confirmed that the measured value of the first distance sensor corresponds to the first distance, the image confirmed to have been captured at the first angle, which is the smallest angle among the plurality of predetermined angles, based on the first artificial intelligence model. Until the image data corresponding to is received, an angle increase command for causing the drone to increase an angular displacement in a direction from the first point toward the drone with respect to the front direction of the first player is command data. Deriving as - The angle increase command is a command to move the drone by a preset first distance in a direction perpendicular to the first direction, until the first player is located in the center of the image corresponding to the image data. Includes a command to rotate the drone in place on a plane parallel to the ground, and a command to move the drone in the first direction until a measurement value of the first distance sensor corresponds to the second value -; Based on the first artificial intelligence model, until the image data corresponding to the image confirmed to have been taken at the second angle, which is the second smallest angle among the plurality of preset angles, is received, the angle increase command is executed. deriving as command data; and an IMU sensor included in the drone at the first angle after image data corresponding to an image captured at the first angle is received and before image data corresponding to an image captured at the second angle is received. The method may include determining a value obtained by adding the indicated angle values as an angle between a front direction of the first player and a direction from the first point toward the first sensor of the drone.

According to various embodiments of the present disclosure, the plurality of preset angles may include 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

According to various embodiments of the present disclosure, a method for providing a synthetic image based on a sports training image includes: obtaining an exercise image of a first player; checking exercise data of the first player from the exercise image of the first player; obtaining a plurality of partial exercise images corresponding to the first player by parsing the exercise image of the first player for each period of the exercise motion; Of the plurality of partial exercise images corresponding to the first player, the transparency of the first partial exercise image and the second partial exercise image are increased, and the transparency of the first partial exercise image having increased transparency is increased, and the second part is increased. Generating a first synthetic image by overlaying a motion image, motion data corresponding to the first partial motion image, and motion data corresponding to the second partial motion image - the motion data is the movement of the joints of the first player contains range -; transmitting the first synthesized video to a user device; and displaying the first synthesized image in the user device.

According to various embodiments of the present invention, the exercise data further includes a contour of the body of the first player extracted from the exercise image of the first player, and in the synthesized image, corresponding to the first partial exercise image. The exercise data and the exercise data corresponding to the second partial exercise image may be displayed in different colors.

Acquiring an exercise image of a second player according to various embodiments of the present disclosure; checking exercise data of the second player from the exercise image of the second player; obtaining a plurality of partial exercise images corresponding to the second player by parsing the exercise image of the second player for each period of the exercise motion; Among the plurality of partial exercise images corresponding to the second player, transparency of a third partial exercise image corresponding to a time point at which the first partial exercise image was captured is increased, and the transparency of the first partial exercise image having increased transparency generating a second synthesized image by overlaying the third partial motion image obtained by increasing ?, motion data corresponding to the first partial motion image, and motion data corresponding to the third partial motion image; and transmitting the second synthesized image to a user device, and displaying the second synthesized image on the user device. According to various embodiments, the method includes: in the user device, a first value specifying a flight height of a drone, a second value specifying a distance between a first sensor of the drone and a first point on the surface of a first player. value, and a third value specifying an angular displacement in a direction from the first point to the first sensor of the drone based on the front direction of the first player, and the server and the drone receive the first Transmitting a value, the second value, and the third value - the first point is a point at which the distance to the drone is the shortest on the surface of the first player, and the third value is 0 degrees or more and 360 degrees with a value less than, the first sensor comprises a first distance sensor and a camera; First collection data including image data acquired by the camera and measurement values of at least one sensor included in the drone while the drone flies in the front direction of the first player while the first player is exercising Obtaining at least one; And by analyzing the at least one first collection data, the drone is located at a position corresponding to the first value, the second value, and the third value based on the first player while the first player is exercising. deriving at least one command data for locating ; Further comprising, the obtaining of the exercise image of the first player includes, based on the at least one first collected data, the drone among the image data included in the at least one first collected data has the first value. , extracting a portion corresponding to a flight time at a location corresponding to the second value and the third value and storing the extracted portion as an exercise image of the first player.

According to various embodiments of the present invention, the method may include, before the first player starts exercising: deriving at least one command data for causing the drone to rotate around the stationary first player; Obtaining second collection data including measurement values of at least one sensor included in the drone and image data obtained by the camera while the drone rotates around a first player who is stationary; and generating a 3D human body model of the first player based on the at least one second collected data, wherein the first player is configured to obtain a joint point extracted from an exercise image of the first player. The checking of the range of motion of the joints may include checking the range of motion of the joints of the first player based on the joint points extracted from the motion image and the 3D human body model of the first player.

According to various embodiments of the present disclosure, a method for providing synthetic images based on sports training images includes: first image data captured by a drone flying in a frontal direction of a first athlete while the first athlete performs a first exercise; Obtaining a first exercise image based on; obtaining a first background image and a first player image by separating a background region and a first athlete region from the first exercise image; generating a background image by synthesizing a plurality of frames of the first background image based on the flight speed of the drone and time information at which the first image data was captured; generating a first synthesized image by overlaying at least one frame of the first player image on the background image based on the flight speed of the drone and time information at which the first image data was captured; and transmitting the first synthesized image to a user device, and displaying the first synthesized image on the user device.

According to various embodiments of the present invention, the method: based on second image data captured by a drone flying in the front direction of the second player while the second player performs the first exercise, acquiring motion images; obtaining a second player image by separating a background area and a second player area from the second exercise image; generating a second synthesized image by overlaying at least one frame of the second player image on the first synthesized image, based on the flight speed of the drone and time information at which the second image data was captured; transmitting the second synthesized video to a user device; and displaying the second synthesized image in the user device.

According to various embodiments of the present disclosure, the generating of the second synthesized image may include: changing transparency or color of the plurality of frames of the second player image; and generating a second synthesized image by overlaying the plurality of frames of which the transparency or color is changed on the first synthesized image. can include

According to various embodiments of the present disclosure, the method may include: specifying, in the user device, a first value specifying a flying height of a drone and a distance between a first sensor of the drone and a first point on the surface of a first player. A second value and a third value specifying an angular displacement in a direction from the first point to the first sensor of the drone based on the front direction of the first player are received, and the server and the drone send the information to the server and the drone. Transmitting the first value, the second value, and the third value - the first point is a point on the surface of the first player with the shortest distance from the drone, and the third value is equal to or greater than 0 degrees having a value of less than 360 degrees, the first sensor comprises a first distance sensor and a camera; The drone, before the first player starts the first exercise, at least collects second data including a measurement value of at least one sensor included in the drone and second image data obtained by the camera. acquiring one; At least one command data for locating the drone at a position corresponding to the first value, the second value, and the third value based on the first player by analyzing the at least one second collected data Deriving; - While the first player performs the first exercise, while the drone flies in the front direction of the first player, the measurement value of at least one sensor included in the drone and the camera obtaining at least one piece of first collection data including the first image data obtained by; And by analyzing the at least one first collection data, while the first player performs the first exercise, the first value, the second value, and the third value based on the first player deriving at least one command data for locating the drone at a corresponding location; Further comprising a, wherein the obtaining of the first exercise image comprises: based on the at least one first collected data, the drone among the first image data included in the at least one first collected data The method may include extracting a portion corresponding to a flight time at a location corresponding to the value, the second value, and the third value, and storing the extracted portion as an exercise image of the first player.

According to various embodiments of the present invention, the method, before the first player starts exercising: a plurality of human body images taken at a plurality of predetermined angles having a value of 0 degrees or more and less than 360 degrees. Storing a human body image database comprising; extracting joint point information from each of a plurality of human body images included in the human body image database; and generating a first artificial intelligence model by learning angles at which the plurality of human body images were photographed and joint point information extracted from the plurality of human body images; And, by analyzing the at least one second collected data, the first value, the second value, and the third value based on the first player while the first player performs the first exercise. The step of deriving at least one command data for locating the drone at a position corresponding to the value includes: until the first player is located in the center of the image corresponding to the second image data, the drone is placed in place. deriving command data for rotating about a rotational axis perpendicular to the ground; When it is confirmed that the first player is located in the center of the image corresponding to the second image data, the distance included in the first sensor until the measured value of the first distance sensor corresponds to the second value. When the measurement value of the sensor is smaller than the second value, command data for causing the drone to move in a direction opposite to the first direction toward which the first distance sensor is directed is derived, and the distance sensor included in the first sensor is measured. deriving command data for causing the drone to move in the first direction when the value is greater than the second value; if it is confirmed that the measured value of the first distance sensor corresponds to the second value, checking a first angle having a largest value and smaller than the third value among the plurality of preset angles; Until the second image data corresponding to the image confirmed to have been taken at the first angle based on the first artificial intelligence model is received, the drone operates the first player based on the front direction. Deriving, as command data, an angle increase command for increasing an angular displacement of a vector toward the drone from 1 point - the angle increase command moves the drone by a preset first distance in a direction perpendicular to the first direction A command to rotate the drone in place on a plane parallel to the ground until the first player is located in the center of the image corresponding to the second image data, and a measurement value of the first distance sensor including a command to move the drone in the first direction until the value of t corresponds to the second value; and deriving the angle increase command as command data until an angle value indicated by an IMU sensor included in the drone is a value obtained by subtracting the first angle from the third value.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features of the present invention. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims (5)

A method for providing synthetic images based on sports training images:
storing a human body image database including a plurality of human body images photographed at a plurality of predetermined angles having a value of 0 degrees or more and less than 360 degrees;
extracting joint point information from each of a plurality of human body images included in the human body image database;
generating a first artificial intelligence model by learning angles at which the plurality of human body images are photographed and joint point information extracted from the plurality of human body images;
Second collection data including second image data obtained by a camera included in the drone and a measured value of at least one sensor included in the drone before the first player starts the first exercise by the drone Obtaining at least one;
In the user device, a first value specifying a flying height of the drone, a second value specifying a distance between a first sensor of the drone and a first point on the surface of a first player, and a frontal direction of the first player A third value specifying an angular displacement in a direction of the drone toward the first sensor from the first point as a reference is input, and the first value, the second value, and the third value are transmitted to the server and the drone. Transmitting - The first point is a point on the surface of the first player with the shortest distance from the drone, the third value has a value of 0 degrees or more and less than 360 degrees, and the first sensor 1 with distance sensor and camera -;
At least for locating the drone at a position corresponding to the first value, the second value, and the third value based on the first player by analyzing the at least one second collected data in the server. deriving one command data;
storing an average color value of an upper garment and an average color value of a lower garment of a first athlete who is stationary based on the at least one second collected data;
While the first player performs the first exercise, the drone flies in the front direction of the first player, and the measurement value of at least one sensor included in the drone and the first image obtained by the camera obtaining at least one first collection data comprising data; and
The server analyzes the at least one first collected data, and while the first player performs the first exercise, the first value, the second value, and the third value based on the first player. deriving at least one command data for locating the drone at a position corresponding to the value;
Of the first image data, a portion corresponding to the flight time of the drone at a position corresponding to the first value, the second value, and the third value is extracted and stored as the first exercise image of the first player. doing;
obtaining a first background image and a first player image by separating a background region and a first athlete region from the first exercise image;
generating a background image by synthesizing a plurality of frames of the first background image based on the flight speed of the drone and time information at which the first image data was captured;
generating a first synthesized image by overlaying at least one frame of the first player image on the background image based on the flight speed of the drone and the time information at which the first image data was captured;
transmitting the first synthesized video to a user device; and
Displaying the first synthesized image in the user device
including,
The server analyzes the at least one second collected data, and the first value, the second value, and the third value based on the first player while the first player performs the first exercise. Deriving at least one command data for positioning the drone at a position corresponding to a value includes:
deriving command data for causing the drone to rotate in place about a rotational axis perpendicular to the ground until the first player is located in the center of the image corresponding to the second image data;
When it is confirmed that the first player is located in the center of the image corresponding to the second image data, the distance included in the first sensor until the measured value of the first distance sensor corresponds to the second value. When the measurement value of the sensor is smaller than the second value, command data for causing the drone to move in a direction opposite to the first direction toward which the first distance sensor is directed is derived, and the distance sensor included in the first sensor is measured. deriving command data for causing the drone to move in the first direction when the value is greater than the second value;
if it is confirmed that the measured value of the first distance sensor corresponds to the second value, checking a first angle having a largest value and smaller than the third value among the plurality of preset angles;
Until the second image data corresponding to the image confirmed to have been taken at the first angle based on the first artificial intelligence model is received, the drone operates the first player based on the front direction. Deriving, as command data, an angle increase command for increasing an angular displacement of a vector toward the drone from 1 point - the angle increase command moves the drone by a preset first distance in a direction perpendicular to the first direction A command to rotate the drone in place on a plane parallel to the ground until the first player is located in the center of the image corresponding to the second image data, and a measurement value of the first distance sensor includes a command to move the drone in the first direction until the value of t corresponds to the second value; and
Deriving the angle increase command as command data until an angle value indicated by an IMU sensor included in the drone is a value obtained by subtracting the first angle from the third value
including,
Obtaining the at least one first collected data comprises:
Based on the first image data, confirming that there is a second player different from the first player;
checking the average color value of the second player's upper garment and lower garment;
The difference between the average color value of the second player's shirt and the average color value of the first player's shirt is equal to or less than a preset fourth value, or the difference between the average color value of the second player's shirt and the average color value of the first player's shirt reaches a preset value of 5 confirming that the value is less than or equal to;
Checking the moving direction of the second player and the moving direction of the first player; and
When the first player moves in a direction different from the average moving direction of the drone for a second preset time, and the second player moves in the same direction as the average moving direction of the drone for the second time, changing a subject to be tracked by the drone from the first player to the second player;
including,
Composite image delivery method. The method of claim 1 , wherein the method:
obtaining a second exercise image based on second image data taken by a drone flying in a frontal direction of the second player while the second player performs the first exercise;
obtaining a second player image by separating a background area and a second player area from the second exercise image;
generating a second synthesized image by overlaying at least one frame of the second player image on the first synthesized image, based on the flight speed of the drone and time information at which the second image data was captured;
transmitting the second synthesized video to a user device; and
Displaying the second synthesized image in the user device
Further comprising a, composite image providing method. The method of claim 2, wherein generating the second synthesized image comprises:
changing transparency or color of the plurality of frames of the second player image; and
generating a second composite image by overlaying the plurality of frames of which the transparency or color is changed on the first composite image;
Including, synthetic image providing method. The method of claim 1 , wherein the method:
storing exception handling colors;
Even if the difference between the average color value of the second player's bottoms and the average color value of the first player's bottoms is equal to or less than the preset fifth value, the difference between the average color value of the second player's bottoms and the exception processing color is equal to or less than the preset fifth value, When the difference between the average color of the first player's bottom and the exceptionally processed color is equal to or less than the preset fifth value, the moving direction of the second player and the moving direction of the first player are not checked, and the drone selects an object to track. maintaining as the first player; and
Even if the difference between the average color of the second player's shirt and the average color of the first player's shirt is equal to or less than the preset fifth value, the difference between the average value of the second player's bottom shirt and the exception handling color is equal to or less than the preset fifth value, When the difference between the average color of the top of the first player and the exception processing color is equal to or less than a preset fifth value, the moving direction of the second player and the moving direction of the first player are not checked, and the drone determines the object to be tracked. maintaining as the first player;
Further comprising a, composite image providing method. 5. The method of claim 4, wherein storing the exception processing color comprises:
Obtaining a search result corresponding to a predetermined keyword representing exercise among postings on social media;
Recognizing a human form within the search result;
extracting upper and lower colors based on the recognition of a human form in the search result and storing them as a database; and
storing a color whose frequency of appearance is greater than or equal to a predetermined frequency in the database as an exception processing color;
Including, synthetic image providing method.

Images