KR102457450B1 - Device and method for measuring and sharing excercise record of exercise video - Google Patents

Abstract

A method according to various embodiments of the present disclosure is a method performed in a server device including a communication circuit, a first memory, a second memory, and a processor. The method may comprise the following steps of: receiving, through the communication circuit, an exercise video of a first user's exercise activity from a first user terminal; storing, by the processor, the exercise video received through the communication circuit in the first memory; calculating, by the processor, an exercise record for the exercise activity by accessing the first memory and analyzing the exercise video; and in response to completion of receiving the exercise video, storing, by the processor, the exercise video and the exercise record in the second memory. In addition, the step of calculating the exercise record may include a step of analyzing frames of the exercise video based on a preset algorithm according to a type of the exercise activity and calculating at least one of a performance time, number of performances, and performance distance of the exercise activity. Additionally, the step of storing the exercise video in the first memory may be executed simultaneously with the step of calculating the exercise record. According to the present invention, users can check and improve the exercise records by oneself, thereby enabling self-directed learning.

Description

Device and method for measuring and sharing exercise record of exercise video {DEVICE AND METHOD FOR MEASURING AND SHARING EXCERCISE RECORD OF EXERCISE VIDEO}

Embodiments of the present disclosure relate to a method for measuring and sharing an exercise record of an exercise image using a server device and a server device therefor, and more particularly, to a method of measuring and sharing an exercise record in an exercise image received from a user, and measuring the exercise record in an exercise image received from a user and recording the exercise image and/or exercise record It relates to a method and apparatus for sharing.

Recently, as non-face-to-face activities are active, individuals are increasingly exercising in their own independent space without going to a separate public space such as a gym. Online classes and assessments are increasingly unavoidable in schools, and physical education subjects that require physical movement are no exception. However, when each individual performs an exercise activity non-face-to-face, there is a problem in that it is difficult to check and evaluate an accurate exercise record of the performed activity.

After recording a video of an exercise activity performed by an individual, it is possible to request a record check and evaluation by transmitting the exercise video to a personal coach or teacher. However, it takes a lot of time for a personal coach or teacher to measure an exercise record by individually reproducing the exercise video, and it may be difficult to accurately measure the record. In particular, since objective grade evaluation is required for a large number of students in the school field, there is a difficulty in non-face-to-face evaluation through exercise video.

A method according to various embodiments of the present disclosure is a method performed in a server device including a communication circuit, a first memory, a second memory, and a processor, wherein the movement of the first user from the first user terminal through the communication circuit Receiving, by the processor, the exercise image received through the communication circuit, in the first memory, accessing, by the processor, the first memory and performing the exercise Analyzing the image may include calculating an exercise record for the exercise activity. The step of calculating the exercise record comprises the steps of analyzing the frames of the exercise image based on a preset algorithm according to the type of the exercise activity and calculating at least one of the execution time, the number of times, and the execution distance of the exercise activity. may include In addition, the method includes, by the processor, in response to the completion of the reception of the exercise image, storing the exercise image and the exercise record in the second memory, and storing the exercise image in the first memory The step of storing the exercise record may be executed simultaneously with the step of calculating the exercise record.

A server device according to various embodiments of the present disclosure includes a communication circuit configured to transmit and receive data, a first memory and a second memory, and a processor, wherein the processor is configured to transmit and receive data from a first user terminal through the communication circuit Receive an exercise image captured by a first user's exercise activity, store the exercise image received through the communication circuit by the processor in the first memory, and access the first memory by the processor and analyze the exercise image to calculate an exercise record for the exercise activity. Calculating the exercise record may include analyzing the frames of the exercise image based on a preset algorithm according to the type of exercise activity to calculate at least one of the exercise time, the number of times, and the distance to be performed. can In addition, the processor is configured to store the exercise image and the exercise record in the second memory in response to completion of the reception of the exercise image, and the storing of the exercise image in the first memory includes the exercise It can be executed at the same time as producing the record.

According to embodiments of the present disclosure, since the server device measures and provides an exercise record based on an exercise image received from the user terminal, self-directed learning in which users can check and improve their own exercise record is possible. In addition, teachers or coaches do not need to replay and check the exercise videos recorded by the students in order to check the exercise records of the students, so it is possible to shorten the evaluation time of the students.

According to the embodiments of the present disclosure, a server device may share an exercise record or an exercise image with various user terminals having authority. Therefore, by sharing a student's exercise video or exercise record with parents, parents can understand the level of achievement of their children and provide motivation to their children.

1 is a diagram illustrating a relationship between a server device and user terminals communicating with the server device through a network according to embodiments of the present disclosure.
2A is a block diagram of a first user terminal according to various embodiments of the present disclosure;
2B is a diagram illustrating examples of screens displayed on a display when a first user terminal executes a dedicated application to receive an exercise record measurement service of a server device.
3 is a block diagram of a server device according to various embodiments of the present disclosure.
4A is a block diagram of a second user terminal according to various embodiments of the present disclosure;
4B to 4D are exemplary screens in which a second user terminal displays an exercise image received from a server device and a recorded measurement result on a display.
5A is a block diagram of a third user terminal according to various embodiments of the present disclosure;
5B are exemplary screens displayed on a display of a third user terminal to provide a service of a server device.
6 is a flowchart of a method performed by a processor to upload an exercise video to a server device by a first user terminal.
7 is a flowchart of a method performed by a processor of a server device to store an exercise image for an exercise activity received from a first user terminal and to calculate an exercise record.
8A illustrates a method of calculating an exercise record according to various embodiments performed by a processor of a server device through an analysis module.
8B exemplarily shows positions of objects detected by the processor.
FIG. 9A illustrates a method of calculating an exercise record according to another various exemplary embodiments performed by a processor of a server device through an analysis module.
FIG. 9B illustrates an example of an exercise activity for which an exercise record may be calculated using the method illustrated in FIG. 9A , exemplarily illustrating a sit-up exercise.
10A illustrates a method of calculating an exercise record according to another various exemplary embodiments performed by a processor of a server device through an analysis module.
FIG. 10B exemplarily illustrates an in-place long jump exercise as an example of an athletic activity for which an exercise record may be calculated using the method illustrated in FIG. 10A .
11 is a diagram illustrating a method of calculating an exercise record according to another various exemplary embodiments performed by a processor of a server device through an analysis module.
12A illustrates a method of calculating an exercise record according to another various exemplary embodiments performed by a processor of a server device through an analysis module.
12B exemplarily illustrates a running exercise as an example of an exercise activity for which an exercise record may be calculated using the method illustrated in FIG. 12A .

Embodiments of the present disclosure are exemplified for the purpose of explaining the technical spirit of the present disclosure. The scope of the rights according to the present disclosure is not limited to the embodiments presented below or specific descriptions of these embodiments.

All technical and scientific terms used in this disclosure, unless otherwise defined, have the meanings commonly understood by one of ordinary skill in the art to which this disclosure belongs. All terms used in the present disclosure are selected for the purpose of more clearly describing the present disclosure and not to limit the scope of the present disclosure.

As used in this disclosure, expressions such as "comprising", "including", "having", etc. are open-ended terms connoting the possibility of including other embodiments, unless otherwise stated in the phrase or sentence in which the expression is included. (open-ended terms).

Expressions in the singular described in this disclosure may include the meaning of the plural unless otherwise stated, and the same applies to expressions in the singular in the claims.

Expressions such as “first” and “second” used in the present disclosure are used to distinguish a plurality of components from each other, and do not limit the order or importance of the corresponding components.

As used in this disclosure, the expression "based on" is used to describe one or more factors affecting the act or action of a decision, judgment, or action described in a phrase or sentence in which the expression is included, the expression being It does not exclude additional factors influencing the actions or actions of decisions, judgments or actions.

In the present disclosure, when a component is referred to as being “connected” or “connected” to another component, the component can be directly connected or connectable to the other component, or a new component It should be understood that other components may be connected or may be connected via other components.

The technical spirit of the present disclosure includes various modifications, equivalents, and alternatives of each embodiment described in this document, and embodiments selectively combined from all or part of each embodiment. In addition, the scope of the rights of the technical idea of the present disclosure is not limited to the various embodiments presented below or specific descriptions thereof.

In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "A; Each of the phrases "at least one of B, or C" may include any one of, or all possible combinations of, items listed together in the corresponding one of the phrases. Terms such as "first", "second", or "first" or "second" may simply be used to distinguish an element from other elements in question, and may refer elements to other aspects (e.g., importance or order) is not limited.

As used in this document, the expression "configured to" means "configured to", "having the ability to", "modified to", "made to", "to do", depending on the context. It can have the meaning of "can be". The expression is not limited to the meaning of "specially designed in terms of hardware", and for example, the processor 111 configured to perform a specific operation is a general-purpose processor 111 capable of performing the specific operation by executing software. (generic-purpose processor).

Hereinafter, various embodiments of the present disclosure will be described with reference to the accompanying drawings. In the accompanying drawings and the description of the drawings, the same reference numerals may be assigned to the same or substantially equivalent elements. In addition, in the following description of various embodiments, description of the same or corresponding components may be omitted, but this does not mean that the corresponding components are not included in the embodiments.

1 is a diagram illustrating a relationship between a server device 110 and user terminals 120 , 130 , and 140 communicating with the server device 110 through a network according to embodiments of the present disclosure.

Referring to FIG. 1 , a server device 110 according to various embodiments may be connected to a plurality of user terminals 120 , 130 , and 140 through a network to transmit/receive various data. The server device 110 according to various embodiments may provide a service for measuring and sharing an exercise image and an exercise record with a plurality of user terminals 120 , 130 , and 140 . For example, the server device 110 may provide a service for calculating an exercise record by receiving an exercise image from a student. In addition, the server device 110 may provide a service for sharing the physical activity performed by the student with teachers and parents. In this embodiment, the first user terminal 120 may be a terminal associated with a student, and the second user terminal 130 may be a terminal associated with a teacher. Also, the third user terminal 140 may be a terminal associated with a parent.

The first user terminal 120 according to various embodiments may provide user information to the server device 110 and then perform a registration procedure. According to an embodiment, the student may proceed with the registration procedure by providing his/her user information to the server device 110 using the first user terminal 120 . The user information may include, for example, user identification information such as a user ID and password. Also, for example, in the case of a student, the user information may additionally include affiliation information such as a school name, a class name, and an attendance number.

According to various embodiments, the first user terminal 120 may perform a service login procedure by providing registered user identification information to the server device 110 . According to various embodiments, the first user terminal 120 may capture an exercise image of the user and transmit the exercise image to the server device 110 in order to request an exercise record measurement.

In various embodiments, operations such as the registration procedure, login procedure, and image transmission of the first user terminal 120 described above may be performed through a dedicated application installed in the first user terminal 120 .

The server device 110 according to various embodiments may receive an exercise image from the first user terminal 120 . In various embodiments, the server device 110 may receive an exercise image captured by the first user terminal 120 in real time and simultaneously store the received exercise image in a memory. In various embodiments, the server device 110 may calculate an exercise record by analyzing the frames of the simultaneously received exercise image in real time while storing the exercise image received from the first user terminal 120 .

In an embodiment, when the server device 110 completes calculating the exercise record for the exercise image received from the first user terminal 120 , the server device 110 may store the calculated exercise record together with the exercise image. In various embodiments, the server device 110 may transmit an upload completion notification to the first user terminal 120 when the storage of the exercise image received from the first user terminal 120 is completed. In this case, the server device 110 transmits an upload completion notification including the calculated exercise record to the first user terminal 120 so that the user of the first user terminal 120 can check his/her exercise record.

The second user terminal 130 according to various embodiments of the present disclosure accesses the server device 110 to receive an exercise video and/or an exercise record of another user (eg, a user related to the first user terminal 120 ). can do. For example, the second user terminal 130 may check an exercise video and/or an exercise record on the web through a SaaS (Software as a Service)-based service provided by the server device 110 .

The second user terminal 130 according to various embodiments of the present disclosure may perform a registration procedure by providing user information to the server device 110 . According to an embodiment, a user (eg, a teacher) associated with the second user terminal 130 may perform a registration procedure by providing their user information to the server device 110 using the second user terminal 130 . have. The user information may include, for example, user identification information such as a user ID and password. Also, for example, when the user associated with the second user terminal 130 is a teacher, the user information may additionally include affiliation information such as a school name and/or a class name in charge.

According to various embodiments, the second user terminal 130 may request access to the server device 110 while providing user identification information registered to the server device 110 . In an embodiment, the second user terminal 130 may request the server device 110 to transmit other users' exercise images and/or exercise records. In this case, the second user terminal 130 may provide, for example, at least a portion of user information of another user who will request transmission of an exercise video and/or an exercise record. For example, the second user terminal 130 may request an exercise video and an exercise record for the specific students by providing affiliation information such as a school name, class name, and attendance number for the specific students to the server device 110 . .

In an embodiment, the second user terminal 130 may receive another user's exercise image and/or exercise record from the server device 110 . In an embodiment, the second user terminal 130 may receive a plurality of exercise images and/or exercise records for a specific user. Accordingly, the user of the second user terminal 130 may check the history of the exercise record of a specific other user. Also, in an embodiment, the second user terminal 130 may receive an exercise video and/or an exercise record for a plurality of users. Accordingly, the user of the second user terminal 130 may check the exercise record statistics for a plurality of users.

In an embodiment, the third user terminal 140 may also receive an exercise video and/or an exercise record of a specific user from the server device 110 . The difference from the second user terminal 130 is that the third user terminal 140 receives an exercise video and/or an exercise record through a third-party application. In an embodiment, the third-party application may be an application that receives an API from the server device 110 and is allowed to share an exercise video and/or an exercise record in the server device 110 . For example, various applications managed by a third entity, such as an application of a financial institution or a shopping application, may be used as the third-party application.

2A is a block diagram of a first user terminal 200 according to various embodiments of the present disclosure. The user terminal 200 is a device corresponding to the first user terminal 120 described above. Referring to FIG. 2A , the first user terminal 200 according to various embodiments includes a processor 210 , a memory 220 , a communication circuit 230 , a display 240 , a camera 250 , and a sensor 260 . may include. However, if various embodiments described in the present disclosure can be implemented, it is also possible to omit or substitute some of the components illustrated in FIG. 2A .

According to various embodiments, the processor 210 of the first user terminal 200 may be a configuration capable of performing an operation or data processing related to control and/or communication of each component of the first user terminal 200 . have. The processor 210 may, for example, be operatively connected to components of the first user terminal 200 . The processor 210 may load a command or data received from other components of the first user terminal 200 into the memory 220 , process the command or data stored in the memory 220 , and store the result data. .

At least one communication circuit 230 of the first user terminal 200 according to various embodiments may establish a communication channel with an external device and transmit/receive various data to and from the external device. According to various embodiments, the at least one communication circuit 230 may include a cellular communication module to be configured to be connected to a cellular network (eg, 3G, LTE, 5G, Wibro, or Wimax). The processor 210 according to various embodiments may transmit/receive various data to and from the server device 110 and/or other user terminals (eg, user terminals 130 and 140 ) through at least one communication circuit 230 . .

The memory 220 of the first user terminal 200 according to various embodiments may store instructions for the operation of the processor 210 . The memory 220 according to various embodiments may store various data. In an embodiment, the memory 220 of the first user terminal 200 may store a dedicated application for providing a service of the server device 110 .

The first user terminal 200 according to various embodiments may include a display 240 for displaying data according to a command of the processor 210 . For example, the first user terminal 200 may display various user interfaces provided by the application on the display 240 when executed by the processor 210 .

The first user terminal 200 according to various embodiments may include a camera 250 capable of capturing a video. In an embodiment, the camera 250 may be called by an application installed in the first user terminal 200, and accordingly, video recording may be possible within the application.

The first user terminal 200 according to various embodiments may include one or more sensors 260 . In an embodiment, the sensor 260 may include a distance measuring sensor configured to measure distance information between the first user terminal 200 and the target object. In the first embodiment, the distance measuring sensor may be a Time of Flight (ToF) sensor configured to measure a three-dimensional position of the target object through a time when light emitted to the target object through an infrared wavelength is reflected and returned. However, the sensor used in the first user terminal 200 is not limited thereto, and various types of sensors capable of measuring data that can be used to calculate the distance of the target object may be mounted on the first user terminal 200 . can

In various embodiments, the processor 210 captures a video of a specific object using the camera 250 and at the same time uses the sensor 260 to calculate a distance to the object using the camera 250 and the sensor 260 . can control In an embodiment, distance information measured using the sensor 260 may be stored together with each frame of the captured image, and may be transmitted to an external device through the communication circuit 230 . In an embodiment, the processor 210 may control the distance measurement information to be measured and stored only for frames at a predetermined time interval among the frames of the motion image captured by the camera 250 .

In various embodiments, the processor 210 may control the communication circuit 230 so that an image captured by the camera 250 is transmitted to the server device 110 in a real-time streaming manner through an application. In another embodiment, the processor 210 may read the image selected by the user through the dedicated application from the memory 230 and transmit it to the server device 110 through the communication circuit 230 .

FIG. 2B shows examples of screens displayed on the display 240 when the first user terminal 120 executes a dedicated application to receive the exercise record measurement service of the server device 110 . Referring to FIG. 2B , the application executed in the first user terminal 120 may display a screen for selecting a type of exercise 270a , a recording measurement start screen 270b , and an upload completion notification screen 270c .

In an embodiment, the exercise activity type selection screen 270a is a screen provided to provide a list of measurable types of exercise activity to the user and to allow the user to select a desired exercise activity type. In an embodiment, the exercise activity type selection screen 270a may include a list of selectable types of exercise activity. In an embodiment, the exercise activity type selection screen 270a may be configured to receive a selection of a desired exercise activity type from the user. For example, in the exercise activity type selection screen 270a shown in FIG. 2B , the user may input selection of the exercise activity type by selecting a “record measurement” icon displayed on the type of exercise activity desired to be measured. . The list of selectable types of exercise activities may include, but is not limited to, types of exercise activities such as jumping rope, sit-up, standing long jump, and running, and may include various types of exercise activities.

In one embodiment, the record measurement start screen 270b is a screen for receiving a request for measuring an exercise record from a user. In an embodiment, the first user terminal 200 may proceed to the next step for acquiring an exercise image when the user selects the "measure by image" icon displayed on the recording measurement start screen 270b. . For example, the first user terminal 200 may call the camera 250 to start image capturing when receiving a selection input of “measuring by image” displayed on the recording measurement start screen 270b. . In addition, the first user terminal 200 is a user for receiving the user's selection from the images stored in the memory 230 when receiving a selection input of "measure by image" displayed on the recording measurement start screen 270b interface can be displayed. In one embodiment, the first user terminal 200, when receiving a selection input of the "measure by image" icon displayed on the recording measurement start screen 270b, an option to start image capturing and an already captured image By displaying a screen presenting all of the options to be selected, the user may select an image acquisition method.

In an embodiment, the recording measurement start screen 270b may include an image and/or text containing instructions that a user should refer to when shooting an exercise video. Accordingly, the user can start shooting an exercise video after being familiar with the precautions when shooting the corresponding exercise activity through the instructions displayed on the recording measurement start screen 270b. As an example, the precautions when shooting may include user requirements (eg, exercise posture, shooting environment, etc.) required for the server device 110 to measure an exercise record through an exercise image in the corresponding exercise activity. As an example, in the case of a long jump exercise in place, the user may instruct the user to place a reference object (eg, A4 paper) on a starting line and start shooting.

In an embodiment, the upload completion notification screen 270c may display an uploaded image. Also, according to an embodiment, the upload completion notification screen 270c may display the exercise record calculated by the server device 110 together. Accordingly, the user can not only confirm that the upload is complete through the upload completion notification screen 270c, but also check the uploaded exercise video and his or her exercise record.

3 is a block diagram of a server device 300 according to various embodiments of the present disclosure. The server device 300 is a device corresponding to the above-described server device 110 . If various embodiments described in the present disclosure can be implemented, it is also possible to omit or substitute some of the components illustrated in FIG. 3 . Referring to FIG. 3 , the server device 300 according to various embodiments may include a processor 310 , a first memory 320 , a communication circuit 330 , an analysis module 340 , and a second memory 350 . can However, if various embodiments described in the present disclosure can be implemented, it is also possible to omit or substitute some of the components shown in FIG. 3 . In addition, in the present disclosure, for convenience of description, the analysis module 340 and the processor 310 have been described based on an embodiment configured separately, but the present disclosure is not limited thereto, and a part or the entire configuration of the analysis module 340 is not limited thereto. It may be implemented integrally with the processor 310 .

According to various embodiments, the processor 310 of the server device 300 may be a configuration capable of performing an operation or data processing related to control and/or communication of each component of the server device 300 . The processor 310 may, for example, be operatively connected to components of the server device 300 . The processor 310 loads a command or data received from other components of the server device 300 into the first memory 320 , processes the command or data stored in the first memory 320 , and stores the result data. can

At least one communication circuit 330 of the server device 300 according to various embodiments may establish a communication channel with an external device, and may transmit/receive various data to and from the external device. According to various embodiments, the at least one communication circuit 330 may include a cellular communication module to be connected to a cellular network (eg, 3G, LTE, 5G, Wibro, or Wimax). The processor 310 according to various embodiments may transmit/receive various data to and from other devices (eg, the user terminals 120 , 130 , and 140 ) through at least one communication circuit 330 .

The first memory 320 of the server device 300 according to various embodiments may store instructions for the operation of the processor 310 . The first memory 320 according to various embodiments may store various data. In an embodiment, the first memory 320 of the server device 300 may include a memory area having a low latency for storing an image input received from the first user terminal 200 .

In various embodiments, the server device 300 may include an analysis module 340 for calculating an exercise record by analyzing a received image input. In one embodiment, the analysis module 340, when executed by the processor 310, may be configured to analyze each frame of the exercise image to calculate an exercise record. In an embodiment, the exercise record may include at least one of a time of performing the exercise, the number of times of performing the exercise, and a distance of performing the exercise. For example, when the type of exercise activity is skipping rope or roll up the sit-up, the exercise record may include the number of times the jump rope or roll-up is performed. As another example, when the type of exercise activity is a long jump in place, the exercise record may include a distance performed by the long jump. As another example, when the type of exercise activity is running, the exercise record may include execution time. A specific analysis process performed by the analysis module 340 will be described in detail below.

In various embodiments, the server device 300 may be connected to the second memory 350 for recording an exercise image and exercise. When the exercise record calculation for the received exercise image is completed, the processor 310 of the server device 300 may store the received exercise image and the calculated exercise record in the second memory 350 . In an embodiment, the second memory 350 may be a mass storage capable of long-term storage. In an embodiment, the processor 310 may delete the same image stored in the first memory 320 with respect to the image that has been stored in the second memory 350 .

In an embodiment, when the storage of the exercise image received from the first user terminal 120 in the second memory 350 is completed, the server device 300 may transmit an upload completion notification to the first user terminal 120 . In various embodiments, the upload completion notification may include an exercise record calculated by the analysis module 340 of the server device 300 . In an embodiment, the server device 300 may transmit an image upload completion notification to the web application server to the first user terminal 120 .

4A is a block diagram of a second user terminal 400 according to various embodiments of the present disclosure. The second user terminal 400 is a device corresponding to the above-described second user terminal 130 . Referring to FIG. 4A , the second user terminal 400 according to various embodiments may include a processor 410 , a memory 420 , a communication circuit 430 , and a display 440 . However, if various embodiments described in the present disclosure can be implemented, it is also possible to omit or substitute some of the components illustrated in FIG. 4A . In various embodiments, the processor 410, the memory 420, the communication circuit 430, and the display 440 of the second user terminal 400 of the second user terminal 400 are the processor 210 of the above-described second user terminal 200, respectively; The memory 220 , the communication circuit 230 , and the display 240 may be configured to perform the same and/or similar functions.

In various embodiments, the communication circuit 430 of the second user terminal 400 may be configured to communicate with the server device 110 to receive an exercise image and/or an exercise record. In an embodiment, the second user terminal 400 may display the received exercise image and/or exercise record on the display 440 . In various embodiments, the exercise video and/or the exercise record may be provided on the web through a SaaS-based service provided by the server device 110 . For example, the user of the second user terminal 400 provides an exercise video and/or exercise record from the server device 110 by accessing the web of the server device 110 through a network using the communication circuit 430 . can receive

In various embodiments, the second user terminal 400 may receive the exercise details for each user (eg, a student) from the server device 110 . For example, the second user terminal 130 may receive from the server device 110 an exercise record for a plurality of types of exercise activities (eg, jump rope, sit-up, etc.) performed by the same student. Accordingly, the teacher associated with the second user terminal 400 may make a comprehensive evaluation of various types of exercise activities for the same student. As another example, the second user terminal 400 may receive an exercise record for the same exercise activity performed by the same student multiple times from the server device 110 . Accordingly, the second user terminal 400 allows the user to check the exercise record history for the corresponding exercise activity at a glance.

In various embodiments, the second user terminal 400 may additionally receive an exercise image from the server device 110 . Accordingly, a user (eg, a teacher) associated with the second user terminal 400 may reconfirm the exercise record measurement result provided from the server device 110 through the confirmation of the exercise image.

4B to 4D are exemplary screens in which the second user terminal 400 displays an exercise image received from the server device 110 and a recorded measurement result on the display 440 .

Referring to FIG. 4B , the user terminal 400 may display a screen 450a providing statistical data on exercise activities performed by a plurality of users (eg, all students in the homeroom class). In addition, the second user terminal 400, as shown in Fig. 4c, a plurality of types of exercise activities (eg, jump rope, sit-up, long jump in place, running, etc.) performed by a specific user (eg, a specific student) A screen 450b providing an exercise record for . may be displayed, and as shown in FIG. 4D , a screen 450c providing detailed exercise records and exercise images for each exercise activity may be displayed.

5A is a block diagram of a third user terminal 500 according to various embodiments of the present disclosure. Here, the third user terminal 500 is a device corresponding to the above-described third user terminal 140 . Referring to FIG. 5A , the third user terminal 500 according to various embodiments may include a processor 510 , a memory 520 , a communication circuit 530 , and a display 540 . However, if various embodiments described in the present disclosure can be implemented, it is also possible to omit or substitute some of the components illustrated in FIG. 5A . In various embodiments, the processor 510, the memory 520, the communication circuit 530, and the display 540 of the second user terminal 500 are the processor 410 of the second user terminal 200 described above, respectively. The memory 420 , the communication circuit 430 , and the display 440 may be configured to perform the same and/or similar functions.

The third user terminal 500 according to various embodiments may display an exercise image and/or an exercise record provided by the service of the server device 110 on the display 540 using an installed application. In an embodiment, the application may be a third-party application configured to receive an exercise video and/or an exercise record from the server device 110 by receiving an API from the server device 110 .

The third user terminal 500 according to various embodiments may transmit a connection request with another user to the server device 110 in order to receive the exercise information of the other user through the installed application. As an example, the server device 110 may identify a connection-requested user among users registered in the server device 110 based on user information provided by the third user terminal 500 . In an embodiment, the third user terminal 500 may receive, from the server device 110 , an exercise video and/or an exercise record for users who have requested a connection.

In various embodiments, the third user terminal 500 may receive an exercise image and/or feedback on the exercise record received from the server device 110 from the user. For example, a parent associated with the third user terminal 500 may write feedback praising their children's exercise video and/or exercise record and input it through the application, and the third user terminal 500 may transmit the received feedback to the first user terminal 120 . In an embodiment, the third user terminal 500 may transmit feedback to the first user terminal 120 through a third-party application.

5B are exemplary screens displayed on the display 540 of the third user terminal 500 to provide the service of the server device 110 . For example, a parent, who is a user associated with the third user terminal 500 , may select an “apply to connect my child” icon on the connection application screen 550a in order to check an exercise video of their child. As an example, the server device 110 may identify a child for which connection is requested among users registered in the server device 110 based on the user information of the child received from the third user terminal 500 . In addition, since at least some of the connected children's information is displayed on the display 540 of the third user terminal 500 , the user can check the information of the currently connected children. Also, on the display 540 of the third user terminal 500 , a child image providing screen 550c displaying the connected child's exercise image and exercise record may be displayed.

6 is a flowchart of a method 600 performed by the processor 210 for the first user terminal 200 to upload an exercise video to the server device 110 . In various embodiments, the method 600 may be started by executing a dedicated application on the first user terminal 200 , but is not limited thereto.

The method 600 according to various embodiments may include the step 610 of the processor 210 of the first user terminal 200 requesting a service login to the server device 110 through the communication circuit 230 . have. In an embodiment, the processor 210 may receive user information from a user through an application and transmit the corresponding user information to the server device 110 . For example, the user information may include user identification information such as a user ID and password.

In the method 600 according to various embodiments, the processor 210 transmits the type of the exercise activity selected through the communication circuit 230 to the server device 110 in response to input of a selection regarding the type of the exercise activity from the user. ) may include a step 620 of transmitting. In an embodiment, the first user terminal 200 may display a list of selectable types of exercise activity on the display 240 and allow the user to select a type of exercise activity desired from the list. For example, when the user provides a selection input of a “record measurement” icon provided for each exercise activity on the type selection screen 270a of the exercise activity shown in FIG. 2B , the first user terminal 200 selects such a selection Receiving the input may allow the user to receive a selection regarding the type of athletic activity for which the user wishes to record. The list of selectable exercise activities may include, but is not limited to, exercise activities such as jumping rope, sit-up, standing long jump, and running, and may include various exercise activities.

The method 600 according to various embodiments may include an operation 630 in which the processor 210 of the first user terminal 200 acquires an exercise image. In an embodiment, the processor 210 may acquire a user's exercise scene photographed using the camera 250 as an exercise image. For example, referring to FIG. 2B , the display 240 displays an exercise recording measurement start screen 270b, and on receiving the selection of the "Measure by image" icon displayed on the recording measurement start screen 270 In response, the processor 210 may start capturing an image using the camera 250 . In another embodiment, the processor 210 may acquire the exercise image by reading the previously captured exercise image from the memory 220 . In an embodiment, a separate screen may be provided on the display 240 for receiving a user selection of an image desired to be recorded and measured among the exercise images stored in the memory 220 .

In one embodiment, the processor 210 of the first user terminal 200 operates with the sensor 260 of the first user terminal 200 while the camera 250 captures the motion image to obtain the target object and the second image. 1 The sensor 260 may be controlled to measure distance information between the user terminals 200 . In one embodiment, when the sensor 260 is a ToF sensor, the sensor 260 measures the three-dimensional position of the object to be photographed by measuring the time taken until the infrared beam is emitted to the object and then reflected from the object to be photographed. can

In an embodiment, the processor 210 of the first user terminal 200 may store distance information measured by the sensor 260 with respect to frames of the captured image together with the corresponding frames. In an embodiment, distance information measured by the sensor 260 may be stored only for frames having a predetermined time interval among frames of the captured image.

The method 600 according to various embodiments may include the step 640 of the processor 210 of the first user terminal 200 transmitting the exercise image to the server device 110 through the communication circuit 230 . can In an embodiment, the processor 210 transmits, in real-time, a streaming method to the server device 110 through the communication circuit 230 in real-time while shooting an exercise image by using the camera 150 . can In an embodiment, the processor 210 may be configured to read the exercise image selected by the user from the memory 220 and transmit it to the server device 110 through the communication circuit 230 .

In the method 600 according to various embodiments, the processor 210 of the first user terminal 200 receives a notification of completion of uploading of the exercise video from the server device 110 through the communication circuit 230 ( 650 ). may include. In an embodiment, the upload completion notification from the server device 110 may include an exercise record calculated by the server device 110 with respect to the exercise image requested to be uploaded. For example, referring to FIG. 2B , on the display 240 of the first user terminal 200 , an upload completion notification screen 270c in which a calculated exercise record is displayed together with an uploaded image may be displayed.

7 is a flowchart of a method 700 performed by the processor 310 of the server device 300 to store an exercise image for an exercise activity received from the first user terminal 120 and calculate an exercise record.

In the method 700 according to various embodiments, the processor 310 of the server device 300 receives an exercise image obtained by photographing the exercise activity of the first user from the first user terminal 120 through the communication circuit 330 . It may include a step 710 of In an embodiment, at least some of the frames of the exercise image may include distance information measured by the sensor 260 of the first user terminal 120 . In one embodiment, the method 700 includes, before the server device 300 receives the exercise image from the first user terminal 120 , via the communication circuit 330 , the type of exercise activity from the first user terminal The method may further include receiving an input about the .

In the method 700 according to various embodiments, the server device 300 stores the exercise image received from the first user terminal 120 through the communication circuit 330 in the first memory 320 ( 720 ) may include. In various embodiments, the server device 300 may store the received exercise image frames in the first memory 320 while receiving the exercise image photographed by the first user terminal 120 in real time. In an embodiment, the received motion image may be stored in a memory area having a low latency in the first memory 320 .

In the method 700 according to various embodiments, the processor 310 of the server device 300 accesses the first memory 320 and the exercise image received from the first user terminal 120 through the analysis module 340 . and calculating ( 730 ) an exercise record for the exercise activity by analyzing it. In various embodiments, the processor 310 accesses the first memory 320 at the same time as the step 720 of storing the exercise image received from the first user terminal 120 in the first memory 320 to display the exercise image. A step 730 of calculating an exercise record for the user may be performed. In one embodiment, the server device 300 stores each frame in the first memory 320 while receiving each of the frames of the exercise image in real time, and simultaneously performs image analysis for calculating the exercise record in real time. can Accordingly, exercise record measurement can be performed quickly even for an exercise image captured and sent by the user of the first user terminal 120 in real time.

According to various embodiments, in the step 730 of calculating the exercise record, the processor 310 analyzes the frames of the exercise image based on an algorithm according to the type of exercise activity preset in the analysis module 340 to determine the exercise activity. It may include calculating at least one of the execution time, the number of executions, and the execution distance.

In various embodiments, the step 730 of calculating the exercise record may include detecting the position of the object in at least some of the frames of the exercise image from the first user terminal 120 , and the detected position change of the object. It may include calculating an exercise record based on the . In an embodiment, when the type of exercise activity is jumping rope, the position of the person's ankle is recognized in frames of the exercise image, and the number of executions may be increased based on the recognized change in the position of the ankle. As another example, when the type of exercise activity is sit-up, the number of times of performing may be counted by recognizing the positions of a person's hands and knees in the frames of the image and recognizing the relative positions of the recognized hands and knees. As another example, when the type of exercise activity is the long jump in place, the distance to be performed may be calculated by recognizing the positions of the feet before and after the jump of the person in the frames of the image and calculating the distance difference.

According to various embodiments, the step 730 of calculating the exercise record may include identifying distance information about the first user in the exercise image from at least two frames among the frames of the exercise image from the first user terminal 120, respectively. and calculating an exercise record for the first user based on the identified distance information. In an embodiment, when the type of exercise activity is the long jump in place, the distance record of the long jump in place may be calculated based on distance information stored in frames before and after a person jumps from frames of an image.

In the step 730 of calculating the exercise record according to various embodiments, the exercise record may be calculated by additionally utilizing time information of a plurality of frames of the exercise image received from the first user terminal 120 . As an example, when the type of exercise activity is running, based on distance information obtained from frames of the image, the frame at which the exercise started and the frame after moving by the target distance are specified, and the time information of the frames is additionally considered. The time taken to reach the target distance can be calculated. As another example, when the type of exercise activity is jumping rope, it is also possible to calculate the execution time of jumping rope based on time information of an image frame at the time of starting and completion of jumping rope. Similarly, even when the type of exercise activity is sit-up, the execution time may be calculated based on time information of a frame at the start and completion of the sit-up.

In the method 700 according to various embodiments, when the processor 310 of the server device 300 completes the step 730 of calculating the exercise record, the calculated exercise record is transmitted to the first through the communication circuit 330 . It may include transmitting 740 to the user terminal 120 . In an embodiment, the processor 310 may be implemented to transmit the calculated exercise record to the first user terminal 120 through the web application server.

In the method 700 according to various embodiments, after the processor 310 of the server device 300 completes the step 730 of calculating the exercise record, the calculated exercise record is stored in the first memory 320 . It may include storing the image in the second memory 350 together with the image (750). In an embodiment, if the exercise image is stored in the second memory 350 , the processor 310 may further perform the step of deleting the corresponding exercise image from the first memory 320 . In various embodiments, in the method 700 , upon completion of storage of the exercise image and the exercise record in the second memory 350 , the processor 310 of the server device 300 communicates with the first user through the communication circuit 330 . The method may further include transmitting an upload completion notification to the terminal 120 .

In the method 700 according to various embodiments, the processor 310 of the server device 300 receives a request for information regarding the exercise activity of the first user from the second user terminal 130 through the communication circuit 330 . step 760 . In an embodiment, upon receiving a request for information on exercise activity, the step of confirming the authority of the second user terminal 130 may be additionally performed. As an example, the processor 310 may check whether the teacher associated with the second user terminal 130 has access to the student information associated with the first user terminal 120 .

In the method 700 according to various embodiments of the present disclosure, in response to a request for information regarding the exercise activity of the first user from the second user terminal 130 , the processor 310 is configured to perform a second memory ( ) through the communication circuit 330 . The method may include transmitting ( 770 ) at least one of an exercise image and an exercise record for the first user stored in 350 ) to the second user terminal ( 130 ).

In the method 700 according to various embodiments, the processor 310 of the server device 300 receives a request for information of an athletic activity associated with the first user from the third user terminal 140 through the communication circuit 320 . step 780 . In one embodiment, the method 700 includes, in response to the processor 310 receiving a request for information of an athletic activity associated with the first user from the third user terminal 140 , A step of checking the authority may be additionally performed. In one embodiment, when the parent associated with the third user terminal 140 sends a connection request for the first user (ie, their child) to the server device 300 through the third user terminal 140, the processor ( 310 may check whether the user of the third user terminal 140 has authority to the first user in response to the reception of the connection request. In an embodiment, the server device 300 may identify the connection-requested first user among users registered in the server device 110 based on user information included in the connection request received from the third user terminal 140 . have.

In the method 700 according to various embodiments, the processor 310 of the server device 300 transmits at least one of an exercise image and an exercise record for the first user stored in the second memory 350 to the communication circuit 330 . It may include a step 790 of transmitting to the third user terminal 140 through the.

8A to 12B relate to various exercise record calculation methods available for the processor 310 of the server device 300 to perform the step 730 of calculating the exercise record. As described above, the analysis module 340 of the server device 300 may include a preset algorithm for calculating exercise records for various types of exercise activities. Accordingly, the processor 310 may calculate an exercise record for the received exercise image through a preset algorithm of the analysis module 340 . Hereinafter, specific embodiments of a method for calculating an exercise record will be described with reference to each drawing.

8A illustrates a method 800 of calculating an exercise record according to various embodiments performed by the processor 310 of the server device 300 through the analysis module 340 . The method 800 may be used, for example, to calculate the number of times a jumping rope exercise is performed, but is not limited thereto, and may be used to calculate the number of times to perform various exercise activities for performing a repetitive exercise.

In the method 800 according to various embodiments, the processor 310 of the server device 300 detects the position of the object in at least some of the frames of the motion image received from the first user terminal 120 . 810 may be included. In an embodiment, the object to be detected may include a body or a specific part of an object. In an embodiment, the detection object may include at least one of various parts of the body, such as a hand, a wrist, a foot, an ankle, a knee, and a face. In one embodiment, it may be detected using various image processing methods to detect objects in each frame. For example, a position of a specific body part may be detected in frames of an exercise image through a body part detection function of Open Computer Vision (Open CV).

The method 800 according to various embodiments may include the step 820 of the processor 310 of the server device 300 detecting an external value among positions of objects detected in frames of the motion image. can In an embodiment, the processor 310 may detect a maximum point, which is a transition point when the height of the object detected in the plurality of frames rises and then falls, as the pole point. 8B exemplarily shows positions of objects detected by the processor. As exemplarily shown in FIG. 8B , the processor 310 may detect poles P1, P2, P3, P4, and P5 in which the position (eg, ankle height) of the detected object rises and then falls. .

In the method 800 according to various embodiments, the processor 310 of the server device 300 detects repetitive motion in the motion image based on poles detected in the first plurality of frames during the first period of the motion image. It may include a step 830 of determining whether the In an embodiment, the processor 310 may determine that the exercise is repeated when a predetermined number or more of poles occur in the first plurality of frames during the first period. In one embodiment, the first period may be selected as an estimated time required for a person to perform a predetermined number of jumps or more. Referring to FIG. 8B , for example, when three or more poles (eg, P3, P4, P5) are detected for 4 seconds, it may be determined that a repetitive motion is detected. Specific reference values for the first period and the number of poles used in detecting repetitive motion may be variously changed according to specific embodiments. In an embodiment, the server device 300 may determine whether to perform repetitive motion in consideration of a time difference between poles detected in the first plurality of frames. For example, referring to FIG. 8B , when the time difference between the temporally adjacent first pole P1 and the second pole P2 among the detected poles is more than a predetermined time, the server device 300 performs a simple movement. By judging it, it can be determined that it is not a repetitive exercise.

In the method 800 according to various embodiments of the present disclosure, when the processor 310 of the server device 300 determines that the repetitive exercise is not performed in the first plurality of frames during the first period among the exercise images, Determining the repetition motion based on the poles detected in the first plurality of frames during the subsequent first period ( 830 ) may be performed again. In one embodiment, if performance of the repetitive motion is not detected as a result of performing the step 830 of determining the repetitive motion for the entire frame or a predetermined number of frames of the motion image, the method 800 may end. have.

In the method 800 according to various embodiments, in response to the processor 310 of the server device 300 determining that the repetitive exercise is performed, the object in the second plurality of frames received during the second period of the exercise image. and determining (840) whether the poles of the positions of are detected. In one embodiment, the second period of time may be selected as the estimated time required for a person to perform one jump. In one embodiment, the processor 310 excludes the execution of step 840 for frames before the start time of the first period for which it is determined that the repetitive motion is performed in step 830 of determining whether the repetitive motion is to be performed. can do. Accordingly, it is possible to prevent an error in which a simple movement before the repetition of the exercise is started is included in the exercise record.

In the method 800 according to various embodiments, the processor 310 of the server device 300 increases the number of executions in response to the determination that the poles are detected in the second plurality of frames of the exercise image ( 850 ) may include In an embodiment, the processor 310 performs a plurality of successive steps of determining whether poles are detected in a second plurality of frames received during the second period ( 840 ) and increasing the number of times of performing ( 850 ). By repeatedly executing the frames during the second period, the number of executions may be counted for the entire exercise image. In various embodiments, when the time difference between adjacent poles among poles detected in frames for a plurality of consecutive second periods is a predetermined time or more, the processor 310 excludes the poles from counting the number of executions can do.

In the method 800 according to various embodiments, in response to the processor 310 of the server device 300 determining that no poles are detected in the second plurality of frames received during the second period, the current number of executions is calculated. Method 800 may end after performing step 860 of returning to the last exercise record.

FIG. 9A illustrates a method 900 of calculating an exercise record according to another various exemplary embodiments performed by the processor 310 of the server device 300 through the analysis module 340 . 9B exemplifies a sit-up exercise as an example of an athletic activity that may generate an exercise record using the method 900 . The method 900 may be used to calculate the number of times the sit-up exercise shown in FIG. 9B is performed, but is not limited thereto, and may be used for recording measurement of various types of activities.

In the method 900 according to various embodiments, the processor 310 of the server device 300 detects the positions of at least two objects in frames of an exercise image received from the first user terminal 120 ( 910 ). ) may be included. In an embodiment, the object to be detected may include a body or a specific part of an object. In an embodiment, the detection object may include at least one of various parts of the body, such as a hand, a wrist, a foot, an ankle, a knee, and a face. In one embodiment, it may be detected using various image processing methods to detect objects in each frame. For example, a position of a specific body part may be detected in frames of an exercise image through a body part detection function of Open Computer Vision (Open CV).

In an embodiment, detecting the position of the object in each frame of the motion image 910 may include the processor 310 detecting the position of the object in consideration of color information for at least a portion of each frame. can In an embodiment, when the object to be detected is a body hand, regions having a skin color (eg, “flesh color”) are detected in a frame of a motion image, and the detected wrist position and positions of the regions having the skin color are compared. Accordingly, when the skin color area is located near the wrist position, the detected skin color area may be recognized as the area occupied by the hand.

The method 900 according to various embodiments may include an operation 920 in which the processor 310 of the server device 300 calculates a relative positional relationship between the detected objects. In an embodiment, when the objects to be detected are the hands and knees of the body, the processor 310 detects a knee position, a wrist position, and a hand position in at least some frames of the frames of the motion image, and determines the position of the hand. Select the midpoint of the x-coordinate value (for example, the maximum x-coordinate of the area occupied by the hand) and the x-coordinate of the wrist position as the hand position value, select the x-coordinate of the knee position as the knee position value, and then select the hand position A relative positional relationship between the hand and the knee may be calculated based on the fluctuation relationship between the value and the knee position value. In an embodiment, the processor 310 may calculate a relative positional relationship between the hand and the knee based on the area occupied by the hand and the position of the knee.

The method 900 according to various embodiments may include, in response to the processor 310 of the server device 300 determining that the calculated relative positional relationship satisfies a predetermined criterion, increasing the number of executions ( 930 ). can In an embodiment, the predetermined criterion regarding the relative positional relationship between the hand and the knee may include a case in which the above-described hand position value exceeds the knee position value by more than a predetermined range. For example, referring to FIG. 9B , the position of the hand (eg, the x-coordinate value of the hand position) may include crossing the knee position (eg, the x-coordinate value of the knee position) by a predetermined distance. In an embodiment, the predetermined criterion regarding the relative positional relationship between the hand and the knee is a relative variation relationship between the hand position value and the knee position value calculated for a plurality of consecutive frames of the motion image, the hand position value is the knee In a state smaller than the position value, it increases by exceeding the knee position value, and then it may include indicating a position variation relationship that decreases below the knee position value again. In another embodiment, the predetermined criterion regarding the relative positional relationship between the hand and the knee may include that, among regions occupied by the hand, a ratio of the region of the hand beyond the position of the knee is equal to or greater than a predetermined value. In addition, in one embodiment, the predetermined criterion regarding the relative positional relationship between the hand and the knee is that the relative variation relationship between the area occupied by the hand and the knee position is such that the area in which the hand area crosses the knee position is greater than or equal to a certain ratio. It may include indicating a position variation relationship that decreases below a certain ratio after increasing. According to an embodiment, the processor 310 of the server device 300 determines that the relative positional relationship of the detected objects with respect to the frames does not satisfy a predetermined criterion, the number of executions may be maintained without increasing.

In the method 900 according to various embodiments, if the processor 310 of the server device 300 additionally determines that the increase in the number of executions has not occurred in frames of a predetermined period among the frames of the exercise image, the number of executions is finalized. After performing step 940 of returning to the athletic record, the method 900 may end.

10A illustrates a method 1000 of calculating an exercise record according to various embodiments performed by the processor 310 of the server device 300 through the analysis module 340 . 10B exemplifies an in-place long jump exercise as an example of an athletic activity that may generate an athletic record using the method 1000 . In one embodiment, the method 1000 may be used to calculate the distance to perform of the long jump in place illustrated in FIG. 10B , but is not limited thereto, and may be used to calculate the distance to perform various athletic activities.

The method 1000 according to various embodiments may include an operation 1010 of the processor 310 of the server device 300 setting a virtual coordinate system with respect to a frame of a received motion image. In an embodiment, the processor 310 may recognize a reference object in a frame of a received motion image and set a virtual coordinate system based on the recognized reference object. For example, the reference object may be an object with a fixed standard and size, such as an A4 paper.

The method 1000 according to various embodiments includes a step 1020 of the processor 310 of the server device 300 detecting a jump start position in a first frame among a plurality of frames of a motion image based on a virtual coordinate system. can do. In an embodiment, the processor 310 may detect the position of the toe on the ground in the frame of the motion image as the jumping start position. In an embodiment, the position of the ground in the motion image may be detected based on color information indicating the ground. In an embodiment, the toe position in the motion image may be detected using various image processing methods for detecting the object.

According to another various embodiments, the step 1020 of detecting the jump start position is a starting point display object (eg, the processor 310 of the server device 300) that displays the jump start point promised with the user in the exercise image. , reference object) and determining a jump start position in consideration of a positional relationship with a starting point display object. For example, if the application executed on the first user device 120 instructs the user to place a specific starting point display object at the jump start position, the position of the starting point display object detected in the motion image is specified as the jump start position. can do. For example, a reference object used when setting the virtual coordinate system may also be used as a starting point display object.

The method 1000 according to various embodiments includes a step 1030 of the processor 310 of the server device 300 detecting a jump completion position in a second frame among a plurality of frames of a motion image based on a virtual coordinate system. can do. In one embodiment, the processor 310 may detect the toe position as a jump completion position when the toe position on the ground is recognized in the second frame temporally subsequent to the first frame in the frame of the motion image. Here, the ground position and the toe position in the frames of the motion image may be detected through the same method as the step of detecting the jump start position described above.

The method 1000 according to various embodiments may include a step 1040 of the processor 310 of the server device 300 calculating a jump performance distance based on a jump start position and a jump finish position in a virtual coordinate system. have. In an embodiment, the processor 310 may end the method 1000 after performing the step 1040 of returning the calculated running distance as the final exercise record.

FIG. 11 illustrates a method 1100 for calculating an exercise record according to another various exemplary embodiments performed by the processor 310 of the server device 300 through the analysis module 340 . In one embodiment, the method 1100 may be used to calculate a record of an in-place long jump exercise, but is not limited thereto, and may be used to calculate a distance to perform various athletic activities.

The method 1100 according to various embodiments includes the step 1110 of the processor 310 of the server device 300 specifying at least two frames among the frames of the exercise image received from the first user terminal 120 . may include In an embodiment, the step 1110 of identifying the distance information may include a step of the processor 310 detecting a jump start frame and a jump completion frame among frames of the motion image. In an embodiment, the processor 310 may detect a jump start frame and a jump completion frame in consideration of the positional relationship between the ground and the toe in the frame of the motion image. In an embodiment, the processor 310 may detect the first frame as a jump start frame when a toe position placed on the ground is detected in the first frame of the exercise image. In one embodiment, the processor 310 detects the second frame as a jump completion frame when detecting the toe position placed on the ground in the second frame among the frames after a predetermined time from the first frame of the motion image. can

In an embodiment, the processor 310 of the server device 300 may further include determining whether at least some of the frames of the exercise image satisfy a predetermined criterion for determining the jumping exercise. In one embodiment, the predetermined criterion for judging the jumping motion may include that the amount of change in height and/or the amount of linear movement of the toe detected in a predetermined number of frames subsequent to the jumping start frame exceeds a predetermined size. can Accordingly, the server device 300 can prevent a case in which an exercise record is calculated even when a jump does not actually occur.

The method 1100 according to various embodiments may include an operation 1120 of the processor 310 of the server device 300 identifying distance information for the first user from at least two specified frames. In an embodiment, the distance information includes a distance value measured by a distance measuring sensor of the first user terminal 120 , and is stored together with each frame of an exercise image received from the first user terminal 120 . there may be

The method 1100 according to various embodiments may include an operation 1130 in which the processor 310 of the server device 300 calculates an exercise record for the first user based on the identified distance information. In an embodiment, the processor 310 may calculate a distance to perform a jump based on distance information identified from each of the jump start frame and the jump complete frame. In one embodiment, the processor 310 detects the position of the toe in each of the jump start frame and the jump complete frame when calculating the distance to perform the jump, and then the corresponding toe from the distance information of each of the jump start frame and the jump complete frame. It is possible to identify distance information for a location.

The method 1100 according to various embodiments may end the method 1100 after the processor 310 of the server device 300 performs an operation 1140 of returning the calculated exercise record as a final exercise record. .

FIG. 12A illustrates a method 1200 of calculating an exercise record according to another various exemplary embodiments performed by the processor 310 of the server device 300 through the analysis module 340 . 12B exemplifies a running exercise as an example of an athletic activity for which the method 1200 may be used to generate an exercise record. The method 1200 may be used to calculate the execution time of the running exercise illustrated in FIG. 12B , but is not limited thereto, and may be used to calculate the execution time of various exercise activities.

The method 1200 according to various embodiments includes the step 1210 of specifying, by the processor 310 of the server device 300 , an exercise start frame among the frames of the exercise image received from the first user terminal 120 . can do. In an embodiment, the processor 310 may detect a motion of a person in the motion image, and specify a frame at a moment when a sudden linear movement occurs without a motion of the person as the motion start frame. In an embodiment, recognizing a person in a motion image may be implemented through an AI algorithm trained to recognize a person in the image.

The method 1200 according to various embodiments includes a step 1220 of specifying, by the processor 310 of the server device 300 , an exercise completion frame among the frames of the exercise image received from the first user terminal 120 . can do. In an embodiment, the processor 310 may specify the exercise completion frame based on preset distance information for the frames of the exercise image. In an embodiment, the processor 310 determines whether the first user moves by a predetermined distance from the movement start frame in each frame of the exercise image based on distance information stored together with the frames of the exercise image to be received. can For example, the processor 310 may determine the movement distance of the first user by comparing the distance information of the exercise start frame with the distance information of the frames of the exercise image. In an embodiment, the processor 310 may specify a frame at the moment in which the first user moves by a predetermined distance from the position in the exercise start frame in the frames of the exercise image as the exercise completion frame.

Exemplarily, FIG. 12B shows a movement start frame 1210F and a movement completion frame 1210F when the camera 250 of the first user terminal 120 is installed close to the running completion point side when the type of exercise activity is running. is shown as an example. According to the example of FIG. 12B , the movement completion frame 1220F may be selected as a frame having a distance measurement value in which the distance measurement information is reduced by a predetermined distance than the movement start frame 1210F among a plurality of frames of the motion image. have. For example, when it is necessary to measure the exercise record of a 50m run, a frame in which a difference between the movement start frame 1210F and the distance measurement value of 50m occurs may be determined as the movement completion frame 1220F.

Returning again to FIG. 12A , in the method 1200 according to various embodiments, the processor 310 of the server device 300 calculates the execution time of the exercise activity based on time information between the exercise start frame and the exercise completion frame. Step 1230 may be included.

The method 1200 according to various embodiments ends the method 1200 after performing the step 1240 of returning the execution time calculated by the processor 310 of the server device 300 as a final exercise record measurement result value. can do.

Although the technical idea of the present disclosure has been described by various embodiments above, the technical idea of the present disclosure includes various substitutions, modifications, and changes that can be made within a range that can be understood by those of ordinary skill in the art to which the present disclosure belongs. include It is also to be understood that such substitutions, modifications and alterations may fall within the scope of the appended claims.

Claims (21)

A method performed in a server device comprising communication circuitry, a first memory, a second memory and a processor, the method comprising:
receiving, through the communication circuit, an exercise image captured by a first user's exercise activity from a first user terminal;
storing, by the processor, the exercise image received through the communication circuit in the first memory;
accessing, by the processor, the first memory and analyzing the exercise image to calculate an exercise record for the exercise activity - The step of calculating the exercise record is based on a preset algorithm according to the type of the exercise activity and analyzing the frames of the exercise image to calculate at least one of a performance time, a number of times, and a distance to perform the exercise activity; and
Storing, by the processor, the exercise image and the exercise record in the second memory in response to completion of the reception of the exercise image
including,
Each frame of the exercise image is received from the first user terminal and stored in the first memory, and is used for image analysis for calculating the exercise record in real time at the same time. According to claim 1,
receiving, through the communication circuit, a request for information regarding the exercise activity of the first user from a second user terminal; and
The method further comprising the step of transmitting, by the processor, at least one of the exercise image and the exercise record for the first user to the second user terminal through the communication circuit in response to the information request. According to claim 1,
and, in response to storing the exercise image and the exercise record in the second memory, via the communication circuit, transmitting an upload completion notification including the exercise record to the first user terminal. According to claim 1,
The method further comprising the step of receiving, through the communication circuit, an input regarding the type of the exercise activity from the first user terminal before receiving the exercise image from the first user terminal. According to claim 1,
The step of calculating the exercise record,
detecting the position of the object in at least some of the frames of the motion image; and
Calculating the exercise record based on a change in the position of the object
How to include. 6. The method of claim 5,
Calculating the exercise record based on the change in the position of the object comprises:
detecting a pole at the position of the object detected from the frames of the motion image; and
Counting the number of executions based on a change in the position of the detected pole
How to include. 7. The method of claim 6,
Calculating the exercise record based on the change in the position of the object comprises:
The method further comprising the step of determining that repetitive motion is detected in the motion image when the number of poles detected in the first plurality of frames during the first period of the motion image occurs more than a predetermined number. 8. The method of claim 7,
Calculating the exercise record based on the change in the position of the object comprises:
In response to determining that the repetitive motion is detected in the motion image, the method further comprising the step of increasing the number of executions based on poles detected in a second plurality of frames during a second period of the exercise image. According to claim 1,
The step of calculating the exercise record,
detecting positions of at least two objects in the frames of the motion image;
calculating a relative positional relationship between the detected at least two objects; and
and in response to determining that the relative positional relationship satisfies a predetermined criterion, increasing the number of times of performing. According to claim 1,
The step of calculating the exercise record,
setting a virtual coordinate system for the frame of the motion image;
detecting a jump start position in a first frame among a plurality of frames of the motion image based on the virtual coordinate system;
detecting a jump completion position in a second frame among a plurality of frames of the motion image based on the virtual coordinate system, wherein the second frame is temporally subsequent to the first frame; and
The method comprising the step of calculating the distance based on the jump start position and the jump complete position. 11. The method of claim 10,
The step of detecting the jump completion position includes:
Recognizing the position of the ground and the position of the toe of the first user in the second frame. According to claim 1,
The step of calculating the exercise record,
specifying at least two frames among the frames of the motion image;
identifying distance information for the first user from the specified at least two frames; and
Based on the identified distance information, comprising the step of calculating the exercise record for the first user,
The distance information for the first user is measured by a sensor included in the first user terminal and stored in the at least two frames. 13. The method of claim 12,
The step of specifying at least two frames among the frames of the motion image includes detecting a jump start frame and a jump completion frame among the frames of the motion image,
The step of identifying the distance information for the first user from the specified at least two frames includes obtaining distance information from the jump start frame and the jump complete frame,
Based on the identified distance information, calculating the exercise record for the first user includes calculating the distance to be performed based on distance information obtained from the jump start frame and the jump complete frame How to. According to claim 1,
The step of calculating the exercise record,
specifying an exercise start frame among the frames of the exercise image;
specifying a motion completion frame among the frames of the motion image, wherein the motion completion frame is determined based on predetermined distance information with respect to the motion start frame and the motion completion frame; and
Comprising the step of calculating the execution time based on the time information between the exercise start frame and the exercise completion frame,
The distance information is measured by a sensor included in the first user terminal and stored in the exercise start frame and the exercise complete frame, the method. As a server device,
a communication circuit configured to transmit and receive data;
a first memory and a second memory; and
A processor comprising:
Through the communication circuit, receiving an exercise image captured by the exercise activity of the first user from the first user terminal,
Storing, by the processor, the motion image received through the communication circuit in the first memory,
by the processor, access the first memory and analyze the exercise image to calculate an exercise record for the exercise activity, wherein calculating the exercise record is based on a preset algorithm according to the type of the exercise activity analyzing the frames of the exercise image to calculate at least one of the execution time, the number of execution times, and the execution distance of the exercise activity; and
and in response to completion of reception of the exercise image by the processor, store the exercise image and the exercise record in the second memory;
Each frame of the exercise image is received from the first user terminal and stored in the first memory, and is used for image analysis for calculating the exercise record in real time at the same time. 16. The method of claim 15, wherein the processor,
receiving, through the communication circuit, a request for information regarding the exercise activity of the first user from a second user terminal;
and transmit at least one of the exercise image and the exercise record for the first user to the second user terminal through the communication circuit in response to the information request. 16. The method of claim 15, wherein the processor is configured to:
detecting the position of the object in at least some of the frames of the motion image,
and calculate the exercise record based on a change in the position of the object. The method of claim 15 , wherein the processor is configured to:
detecting positions of at least two objects in the frames of the motion image;
calculating a relative positional relationship between the detected at least two objects,
and increase the number of performing according to determining that the relative positional relationship satisfies a predetermined criterion. The method of claim 15 , wherein the processor is configured to:
Set a virtual coordinate system for the frame of the motion image,
Detecting a jump start position in a first frame among a plurality of frames of the motion image based on the virtual coordinate system,
detecting a jump completion position in a second frame among a plurality of frames of the motion image based on the virtual coordinate system, wherein the second frame is temporally subsequent to the first frame;
The server device, configured to calculate the running distance based on the jump start position and the jump finish position. The method of claim 15 , wherein the processor is configured to:
Each of the distance information for the first user is identified from at least two frames among the frames of the exercise image,
configured to calculate the exercise record for the first user based on the identified distance information,
The distance information for the first user is measured by a sensor included in the first user terminal and stored in the at least two frames, the server device. The method of claim 15 , wherein the processor is configured to:
specifying a motion start frame among the frames of the motion image,
and specifying a motion completion frame among the frames of the motion image - the motion completion frame is moved by a preset distance from the motion starting frame based on distance information stored in each of the motion starting frame and the motion completion frame Determined by the frame of the moment -,
configured to calculate the execution time based on time information between the exercise start frame and the exercise completion frame,
The distance information is measured by a sensor included in the first user terminal and stored in at least some of the frames of the exercise image, the server device.

Images