KR20210043174A - Method for providing exercise coaching function and electronic device performing thereof - Google Patents

Abstract

According to various embodiments, an electronic device comprises: a plurality of cameras which include a first camera for photographing an external object in a first direction, and a second camera for photographing the external object in a second direction; a display; a memory; and a processor. The processor obtains image information for the external object by using the plurality of cameras; generates a first virtual character image corresponding to exercise coaching information stored in the memory based on the image information; generates a second virtual character image corresponding to movement of the external object based on the image information; mutually overlaps the first virtual character image and the second virtual character image, and displays the same through the display. Various embodiments are possible.

Description

A method of providing an exercise coaching function, and an electronic device that performs the same TECHNICAL FIELD [METHOD FOR PROVIDING EXERCISE COACHING FUNCTION AND ELECTRONIC DEVICE PERFORMING THEREOF}

Various embodiments of the present disclosure relate to a method of providing an exercise coaching function and an electronic device performing the same.

According to the development of technology, various types of electronic devices are being developed. In particular, as interest in health increases in recent years, electronic devices that collect user health care information and provide related services are being developed.

Accordingly, various types of electronic devices (eg, wrist-type wearable devices) are being released that collect body information such as a user's weight, heart rate, and blood pressure and provide a service to check the user's health status through the collection.

Some of these electronic devices may provide a notification message instructing the user to exercise every specific time period to promote the user's exercise, or provide a function of outputting an image (or audio) related to a specific exercise method. have. The exercise method-related image at this time may be, for example, an image of an exercise expert's motion.

When the user checks the exercise motion of the exercise expert as an image using the electronic device as described above, and exercises while following the motion, even if the user performs the exercise motion in an incorrect posture, it is not recognized, resulting in a decrease in exercise efficiency. Or you may have the wrong exercise habits.

Various embodiments of the present specification provide an electronic device for coaching an exercise method by feeding back an appropriate exercise posture according to a user's body shape.

An electronic device according to various embodiments of the present disclosure includes: a plurality of cameras including a first camera for photographing an external object in a first direction and a second camera for photographing the external object in a second direction; display; Memory; And a processor, wherein the processor obtains image information on the external object using the plurality of cameras, and based on the image information, a first virtual character image corresponding to exercise coaching information stored in the memory And, based on the image information, a second virtual character image corresponding to the movement of the external object is generated, and the first virtual character image and the second virtual character image are mutually overlapped and displayed through the display. Can be set to

The processor of the electronic device extracts feature points of the external object from the image information, calculates location information and/or space information between the feature points, and generates the first virtual character image based on the calculated information. Can be set to

The processor may be configured to generate the first virtual character image by scaling the virtual character image generated through the exercise coaching information to correspond to the calculated information.

The electronic device further includes an input device, wherein the processor receives a selection input for a specific exercise among the exercise coaching information through the input device, and corresponds to the received selection input based on the image information. The first virtual character image may be generated using the exercise coaching information.

The first camera may be arranged to photograph one side of the external object in the first direction, and the second camera may be arranged to photograph the other side of the external object in the second direction.

The processor determines whether the electronic device has entered the operator mode, and when determining that the electronic device has entered the operator mode, obtains image information on a second external object using the plurality of cameras. And, based on the image information on the second external object, exercise coaching information corresponding to the movement of the second external object may be generated, and the generated exercise coaching information may be stored in the memory.

The electronic device further includes an audio output unit, wherein the processor calculates a degree of correspondence between the first virtual character image and the second virtual character image, and transmits information related to the calculation result to the audio output unit or the It can be output through at least one of the displays.

The processor identifies the movement of the external object corresponding to the first virtual character image, counts the number of repetitive movements of the identified movement of the external object, and calculates the count result to the sound output unit or the display It may be set to output through at least one of them.

The first virtual character image and the second virtual character image may include a video.

The exercise coaching method of an electronic device according to various embodiments of the present disclosure includes a plurality of cameras including a first camera photographing an external object in a first direction and a second camera photographing the external object in a second direction. Obtaining image information on an external object; Generating a first virtual character image corresponding to pre-stored exercise coaching information based on the image information; Generating a second virtual character image corresponding to the movement of the external object based on the image information; And displaying the first virtual character image and the second virtual character image to overlap each other through a display.

The electronic device according to various embodiments may provide an exercise coaching function through which a user can accurately receive feedback on his/her exercise posture through an exercise image of a virtual character corresponding to the user's body shape.

1 is a block diagram of an electronic device according to an embodiment of the present invention.
2 is a flowchart illustrating an exercise coaching method for an electronic device according to various embodiments of the present disclosure.
3 is a flowchart illustrating an exercise coaching method in an exercise mode of an electronic device according to various embodiments of the present disclosure.
4A to 4E are schematic diagrams for explaining an operation of generating exercise coaching information of an electronic device according to various embodiments of the present disclosure.
5A to 5F are schematic diagrams for describing exercise coaching operations of an electronic device according to various embodiments of the present disclosure.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but identical or similar elements are denoted by the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. The suffixes "module" and "unit" for constituent elements used in the following description are given or used interchangeably in consideration of only the ease of preparation of the specification, and do not have meanings or roles that are distinguished from each other by themselves. In addition, in describing the embodiments disclosed in the present specification, when it is determined that a detailed description of related known technologies may obscure the subject matter of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed in the present specification is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention It should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first and second may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, terms such as "comprises" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

Mobile terminals described herein include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation systems, and slate PCs. , Tablet PC (tablet PC), ultrabook (ultrabook), wearable device (wearable device, for example, smartwatch, glass-type terminal (smart glass), HMD (head mounted display)), etc. may be included. have.

However, it is understood that the configuration according to the embodiment described in the present specification may also be applied to fixed terminals such as digital TV, desktop computer, digital signage, etc., except when applicable only to mobile terminals. Technicians will find it easy.

1 is a block diagram illustrating an electronic device according to an embodiment of the present invention.

Referring to FIG. 1, an electronic device 100 according to various embodiments includes a processor 110, a plurality of cameras 120, a memory 130, a display 140, an input device 150, or an audio output. At least two or more of the units 160 may be included. At least one of the above components may be omitted, and one or more other components may be further added.

The processor 110 may generally control one or more other components of the electronic device 100 and may perform various data processing or operations.

For example, the processor 110 loads commands or data received from other components (eg, cameras 120, display 140, or input device 150) into the memory 130 (eg, volatile memory), and , An operation of storing result data generated by processing the command or data in the memory 130 (for example, a nonvolatile memory) may be performed.

The plurality of cameras 120 may capture a still image and a video. For example, the plurality of cameras 120 may include one or more lenses, image sensors, image signal processors, and/or flashes. The cameras 120 may include, for example, a dual camera, a 360 degree camera, or a spherical camera.

The plurality of cameras 120 may include a first camera and a second camera. The first camera and the second camera may acquire (photograph) image information on an external object (eg, a user's body) under the control of the processor 110.

According to various embodiments, the plurality of cameras 120 may be arranged to simultaneously photograph an external object in various directions. For example, the first camera may be arranged to photograph one side of the external object in a first direction, and the second camera may be arranged to photograph the other side of the external object in a second direction different from the first direction. . For example, the electronic device 100 may simultaneously photograph at least two or more of the front, side, and rear surfaces of an external object using the plurality of cameras 120.

The memory 130 may store various data (or information) that can be used in at least one component of the electronic device 100. The memory 130 may include a volatile memory and/or a nonvolatile memory.

For example, data (or information) that may be stored in the memory 130 may include input/output data for software and commands related thereto. For example, the memory 130 may store a specific program that can be executed by the processor 110 as software.

For example, the memory 130 may include image information acquired using a plurality of cameras 120, or predetermined data (or information) generated or processed by the processor 110 based on the image information ( Example: exercise coaching information, virtual character video, etc.) can be stored.

The display 140 may visually display (or output) information processed by the electronic device 100 to the outside. For example, the display 140 may display (or output) execution screen information, graphic user interface (GUI) information, and/or predetermined image information of a program executed in the electronic device 100.

The display 140 may be disposed at a position where the user can check the output screen of the display 140 while the user is photographed as an external object by the plurality of cameras 120. For example, the display 140 may be disposed adjacent to either the first camera or the second camera. For another example, the display 140 may be provided in the form of a head mounted display (HMD) that can be mounted on the user's head, or may be provided in the form of a glasses-type display.

The input device 150 is configured to receive a predetermined command or data from a user, and may include, for example, at least one of a microphone, a mouse, a digital pen, a keyboard, and a push input key. As another example, the input device 150 may be implemented as at least one of a touch panel or a plurality of cameras 120 that may be provided as a part of the display 140. The electronic device 100 may perform a specific function based on a command or data received through the input device 150.

The sound output unit 160 may output audio data processed by the electronic device 100 to the outside, and may include, for example, a speaker or a buzzer. The sound output unit 160 may provide specific information to a user together with or separately from the display 140.

Hereinafter, a method of coaching an exercise of the electronic device 100 will be described with reference to FIGS. 2 and 3.

2 is a flowchart illustrating an exercise coaching method of an electronic device according to various embodiments of the present disclosure.

According to various embodiments, the electronic device 100 may have an operator mode and an exercise mode. Here, the operator mode is a mode for obtaining exercise coaching information by photographing and analyzing the motion of an exercise expert, and storing or updating it in advance, and the exercise mode is a mode for giving a user exercise coaching through the obtained exercise coaching information. It may refer to a mode that provides a function.

Referring to FIG. 2, the processor 110 may identify whether the electronic device 100 enters an operator mode or an exercise mode, and may perform an operation corresponding to the identification result.

For example, if it is determined in operation 210 that the electronic device 100 has entered the operator mode, the processor 110 may perform a series of operations for storing exercise coaching information as an operator mode in operations 220 to 240.

On the other hand, in operation 210, when it is determined that the electronic device 100 has not entered the operator mode, the processor 110 branches to operation 250 as an exercise mode, and performs operations required to provide exercise coaching information to the user. You can do it.

In operation 220, the electronic device 100 may obtain image information on the first external object using the plurality of cameras 120. For example, in the operator mode, the first external object may correspond to the body of an exercise expert.

The plurality of cameras 120 may photograph the first external object from various directions. For example, among the plurality of cameras 120, a first camera may photograph one side of the first external object in a first direction, and a second camera may photograph the other side of the first external object in a second direction. The cameras 120 of the electronic device 100 may further include an additional camera for photographing the first external object in addition to the first camera and the second camera.

The processor 110 may acquire at least two photographed images of the front, side, or rear of the first external object through the plurality of cameras 120, and through this, the shape and motion of the first external object are three-dimensionally Can be identified as.

4A to 4E are schematic diagrams illustrating an operation of generating exercise coaching information by photographing a first external object through at least one of a plurality of cameras in operations 220 and 230 of FIG. 2. Hereinafter, it will be described with reference to FIGS. 2 and 4A to 4E together.

4A is a diagram illustrating a situation in which image information on the front surface of the first external object 410 is obtained through at least one of the plurality of cameras 120.

The processor 110 may extract feature points 420 of the first external object 410 from image information obtained by photographing the first external object 410. For example, the extractable feature points 420 may be pre-configured to correspond to body components related to a person's movement, such as a face, a body skeleton, and a joint point in the first external object 410. For example, the feature points 420 are, as shown in FIG. 4A, feature points 426 corresponding to the eyes of the face, nose, mouth, ears, etc., feature points 424 corresponding to the elbow, or wrist It may include at least one of the feature points 422 corresponding to.

In operation 230, the processor 110 may generate exercise coaching information corresponding to the movement of the first external object 410 based on the image information.

The processor 110 controls the cameras 120 to continuously or periodically acquire image information corresponding to the first external object 410 for a specified reference time, for example, and from image information acquired during the specified reference time. Feature points 420 may be extracted and tracked.

For example, the processor 110 may recognize the movement of the first external object 410 according to a change in position or length of the feature points 420 extracted from the image information.

For example, as shown in FIGS. 4B and 4C, each feature point 420 (eg, 422, 424) in image information photographing the first external object 410 according to the movement of the first external object 410 May be changed, and the processor 110 acquires information related to a change in the position of each of these feature points 420 or a change in length between adjacent feature points 420, and the movement of the first external object 410 Can be recognized.

The processor 110 may generate exercise coaching information based on feature points 420 obtained from image information for the designated reference time.

The processor 110 may generate an image of a virtual character 430 as shown in 4D and 4E through the exercise coaching information. For example, the image of the virtual character 430 may include an image in which the virtual character 430 moves in response to the movement of the first external object 410 in FIGS. 4B and 4C.

In operation 240, the processor 110 may store the generated exercise coaching information (or the generated virtual character 430 image) in the memory 130.

3 is a flowchart illustrating an exercise coaching method in an exercise mode of an electronic device according to various embodiments of the present disclosure, and FIGS. 5A to 5F are schematic diagrams for explaining exercise coaching operations of FIG. 3. Hereinafter, it will be described with reference to FIGS. 3 and 5A to 5F together.

3 and 5A, in operation 310, the processor 110 (FIG. 1) of the electronic device 100 (FIG. 1) uses a plurality of cameras 120 (FIG. 1) to display a second external object 510. ) Image information can be obtained.

The second external object 510 in operation 310 is a target photographed in the exercise mode, and unlike the first external object 410 (FIG. 4A) photographed in the operator mode of operation 220 of FIG. It may be understood as a general user who wants to use the exercise coaching function of the device 100.

As shown in FIG. 5A, the processor 110 may acquire image information on the second external object 510 using at least one of the plurality of cameras 120.

In operation 320, the processor 110 may generate an image of the first virtual character 530 based on the acquired image information.

For example, as shown in FIGS. 5A and 5B, the processor 110 uses the feature points 520 (e.g., 522) of the second external object 510 from image information photographing the second external object 510. , 524, 526) may be extracted, and a first virtual character 530 may be generated based on the feature points 520.

Like the feature points 420 described with reference to FIG. 4A, the feature points 520 in FIGS. 5A and 5B are specific components of the body related to the face, a body skeleton, and joint points of the second external object 510. It can be configured in advance to correspond to. For example, the first virtual character 530 of FIG. 5B may be generated based on the location of each feature point 520 of the second external object 510 of FIG. 5A and/or the spacing information between the feature points 520. have.

5C and 5D, the processor 110 may generate an image of a first virtual character 530 corresponding to the exercise coaching information by using exercise coaching information previously stored in the memory 130.

For example, as illustrated in FIGS. 4D and 4E, the processor 110 may generate an image of the virtual character 430 through exercise coaching information. In FIGS. 5C and 5D, the processor 110 scales the image related to the exercise of the virtual character 430 generated through the exercise coaching information to correspond to the size of the first virtual character 530, An image of the virtual character 530 may be generated.

In other words, the processor 110 may generate an image of the virtual character 430 by using the exercise coaching information obtained by photographing the exercise motion of the exercise expert, and as another user, the second external object 510 of FIG. 5A. ), a scaled first virtual character 530 image corresponding to the body shape of the second external object 510 may be generated.

In this case, in the image of the first virtual character 530, the first virtual character 530 corresponding to the body shape of the second external object 510 of FIG. 5A is shown in FIGS. 5C and 5D. It may include an image that operates in response to a motion motion of the first external object 410 of FIGS. 4B and 4C.

Through the image of the first virtual character 530 shown in FIGS. 5C and 5D, the electronic device 100 is an image in which the first virtual character 530 similar to the user's body shape is operated as it is the exercise movement of an exercise expert. Can be provided to the user.

In FIGS. 5C and 5D, an exercise operation performed by the first virtual character 530 may be determined based on a user's selection. For example, a plurality of exercise movements may be stored in advance in the memory 130, and the processor 110 provides exercise coaching corresponding to a plurality of exercise movements previously stored in the memory 130 through the input device 150. Among the information, a selection input for a specific exercise may be received from the user. The processor 110 may generate an image of the first virtual character 530 performing the selected exercise by using specific exercise coaching information corresponding to the received selection input.

3, 5E, and 5F, in operation 330, the processor 110 generates a second virtual character 540 image corresponding to the movement of the second external object 510 based on the image information. It may be generated, and in operation 340, the image of the second virtual character 540 and the image of the first virtual character 530 may be superimposed and displayed through the display 140.

In operation 330, the image of the second virtual character 540 is a second virtual character 540 corresponding to the movement of the second external object 510 based on image information on the second external object 510 It can correspond to the motion image of.

For example, in the image of the second virtual character 540, the second external object 510 sees an image of the first virtual character 530 displayed through the display 140, and moves along the image in real time. It may include an image of the second virtual character 540 that is photographed and operates in response to the movement of the second external object 510.

That is, the first virtual character 530 image corresponds to an image in which the first virtual character 530 corresponding to the body shape of the second external object 510 operates in response to pre-stored exercise coaching information, and the second virtual character The image 540 may correspond to an image in which the second virtual character 540 corresponding to the body shape of the second external object 510 operates in response to the movement of the second external object 510.

The processor 140 may control the display 140 to display the image of the first virtual character 530 and the image of the second virtual character 540 so as to overlap each other on the display 140.

5E and 5F, the first virtual character 530 image and the second virtual character 540 image are displayed so that the two images are distinguished from each other through different colors, different line processing, or shading processing for a specific image. Can be.

For example, in an embodiment, a portion overlapping each other in the image of the first virtual character 530 and the image of the second virtual character 540 may be displayed in a first color, and a portion that does not overlap may be displayed in a second color.

The processor 110 may calculate a degree of correspondence between the first virtual character 530 image and the second virtual character 540 image, and may output information related to the calculation result to the outside.

For example, the processor 110, based on whether the degree of mutual correspondence (or overlapping) between the image of the first virtual character 530 and the image of the second virtual character 540 is equal to or greater than a preset value, Information about the score can be output to the outside.

For example, the processor 110 may output the information to the outside by using at least one of the sound output unit 160 and the display 150. The user (i.e., the second external object 510) can recognize whether his/her exercise motion matches the motion motion of the first virtual character 530 image through the output information, and exercise with an incorrect posture. If so, it is possible to correct the exercise motion to the correct posture through the images of FIGS.

According to another embodiment, the processor 110 moves the second external object 510 corresponding to the image of the first virtual character 530 based on image information photographing the second external object 510. It is possible to identify whether to perform an operation, count the number of repetitive movements of the identified second external object 510, and output the count result to the outside.

For example, the image of the first virtual character 530 is output on the display 140 of the electronic device 100, and the second external object 510 corresponds to the motion of the image of the first virtual character 530 When the exercise is repeated, the processor 110 identifies the movement of the second external object 510 and the second external object 510 performs the movement operation of the first virtual character 530 image several times. The information can be provided by counting whether or not.

The electronic device 100 not only coaches the user on an accurate exercise method, but also provides information on the amount of exercise previously performed, thereby inspiring the user to exercise motivation and providing an efficient exercise method.

It is apparent to those skilled in the art that the present invention can be embodied in other specific forms without departing from the spirit and essential features of the present invention.

The above detailed description should not be construed as limiting in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention may be included in the scope of the present invention.

Claims (10)

A plurality of cameras including a first camera for photographing an external object in a first direction and a second camera for photographing the external object in a second direction;
display;
Memory; And
Including a processor,
The processor,
Acquiring image information on the external object using the plurality of cameras,
Based on the image information, a first virtual character image corresponding to exercise coaching information stored in the memory is generated,
Based on the image information, a second virtual character image corresponding to the movement of the external object is generated,
The electronic device configured to overlap and display the first virtual character image and the second virtual character image through the display. The method of claim 1,
The processor,
Extracting feature points of the external object from the image information,
Calculate location information and/or spacing information between the feature points,
An electronic device configured to generate the first virtual character image based on the calculated information. The method of claim 2,
The processor,
The electronic device configured to generate the first virtual character image by scaling a virtual character image generated through the exercise coaching information to correspond to the calculated information. The method of claim 1,
Further comprising an input device,
The processor,
Receiving a selection input for a specific exercise among the exercise coaching information through the input device,
Based on the image information, the electronic device configured to generate the first virtual character image using exercise coaching information corresponding to the received selection input. The method of claim 1,
The electronic device, wherein the first camera photographs one side of the external object in the first direction, and the second camera photographs the other side of the external object in the second direction. The method of claim 1,
The processor,
Determine whether to enter the operator mode,
If it is determined that it has entered the operator mode,
Using the plurality of cameras, image information on a first external object is obtained,
Generates exercise coaching information corresponding to the movement of the first external object based on the image information on the first external object,
The electronic device configured to store the generated exercise coaching information in the memory. The method of claim 1,
Further comprising a sound output unit,
The processor,
Calculate a degree of correspondence between the first virtual character image and the second virtual character image,
The electronic device configured to output information related to the calculation result through at least one of the sound output unit and the display. The method of claim 7,
The processor,
Identify the movement of the external object corresponding to the first virtual character image,
Counting the number of repetitive motions of the identified external object movement,
The electronic device configured to output the count result through at least one of the sound output unit and the display. The method of claim 1,
The electronic device, wherein the first virtual character image and the second virtual character image include moving pictures. In the exercise coaching method of an electronic device,
Obtaining image information on the external object by using a plurality of cameras including a first camera photographing an external object in a first direction and a second camera photographing the external object in a second direction;
Generating a first virtual character image corresponding to pre-stored exercise coaching information based on the image information;
Generating a second virtual character image corresponding to the movement of the external object based on the image information; And
And displaying the first virtual character image and the second virtual character image to overlap each other through a display.

Images