KR102608404B1 - Electronic device for providing evaluation information on user posture and controlling method thereof - Google Patents

Abstract

An electronic device and a method for controlling the electronic device are disclosed. In particular, the electronic device according to the present disclosure includes a foot pressure sensor, a camera, a memory that stores at least one instruction, and a processor that executes the at least one instruction. Then, the processor acquires a pressure center value representing the pressure caused by the user's foot through a foot pressure sensor, acquires an image of the user through a camera, and joint angle information representing the angle between the user's joints based on the image. Obtain user posture information representing the user's posture by inputting the center of pressure value and joint angle information into the posture information acquisition model, and input the user posture information into the posture evaluation model to obtain posture evaluation information about the user's posture. Obtain and provide posture evaluation information.

Description

Electronic device for providing evaluation information about user posture and control method thereof {ELECTRONIC DEVICE FOR PROVIDING EVALUATION INFORMATION ON USER POSTURE AND CONTROLLING METHOD THEREOF}

The present disclosure relates to an electronic device and a method of controlling the electronic device, and more specifically, to an electronic device and a method of controlling the electronic device that can provide evaluation information about a user's posture.

Recently, especially with the development of artificial intelligence (AI) technology, research on technology that can guide users' posture in real time while they are doing physical training is receiving attention.

In this regard, as a prior art, there is a technology for obtaining information about the user's posture using various types of sensors, but only a specific sensor accurately obtains information about the user's posture by considering both the user's motion and balance. Accordingly, there is a limitation in that it is difficult to carry out objective posture coaching.

In addition, there are technologies that use multiple sensors/devices such as inertial sensors, electromyography sensors, force plates, and VR glasses for eye tracking, but these require complex algorithms for fusion and calibration of information acquired through multiple sensors/devices. This is necessary, and limitations have been pointed out, such as poor stability in the event of system failure or malfunction, and inconvenience to users due to the installation of multiple sensors.

As an example, Republic of Korea Patent Publication No. 10-1118654 discloses a rehabilitation device and method that can estimate the position and posture of a rehabilitation subject using a pressure sensor board and a camera.

The present disclosure is intended to overcome the limitations of the prior art as described above. The purpose of the present disclosure is to obtain information about the user's posture quickly and conveniently using only a small number of sensors while providing objective and accurate evaluation information about the user's posture. To provide an electronic device that can provide and a control method of the electronic device.

According to an embodiment of the present disclosure for achieving the above-described object, an electronic device includes a foot pressure sensor, a camera, a memory storing at least one instruction, and a processor executing the at least one instruction; , the processor acquires a pressure center value representing the pressure by the user's foot through the foot pressure sensor, acquires an image of the user through the camera, and determines the angle between the user's joints based on the image. Obtain joint angle information representing, input the center of pressure value and the joint angle information into a posture information acquisition model to obtain user posture information representing the user's posture, and input the user posture information into a posture evaluation model. Posture evaluation information about the user's posture is acquired, and the posture evaluation information is provided.

Here, the processor may obtain joint position information indicating a plurality of positions corresponding to the user's joints based on the image, and obtain the joint angle information based on the joint position information.

Meanwhile, the processor acquires user body information including information on the user's body mass index (BMI), and uses the center of pressure value, the joint angle information, and the user body information as the posture information acquisition model. The user posture information can be obtained by inputting .

Meanwhile, the posture evaluation information is calculated based on a score according to the similarity between the user posture information and standard posture information, and the standard posture information includes learning data including center of pressure values and joint angle information for a plurality of users. It can be obtained by inputting it into the posture information acquisition model.

Here, if the score is greater than or equal to a preset threshold, the processor may update the learning data based on the user posture information.

Meanwhile, the posture evaluation information may include coaching information for guiding the user's posture in real time.

According to an embodiment of the present disclosure for achieving the above-described object, a method for controlling an electronic device includes obtaining a center of pressure value representing pressure caused by a user's foot, acquiring an image of the user, Obtaining joint angle information representing the angle between the user's joints based on the image, inputting the center of pressure value and the joint angle information into a posture information acquisition model to obtain user posture information representing the user's posture. It includes the steps of acquiring, inputting the user posture information into a posture evaluation model to obtain posture evaluation information about the user's posture, and providing the posture evaluation information.

1 is a flowchart briefly showing the configuration of an electronic device according to an embodiment of the present disclosure;
2 is a diagram for explaining in detail the operation of the posture information acquisition model and posture evaluation model according to the present disclosure;
3 is a flowchart showing in detail the configuration of an electronic device according to an embodiment of the present disclosure, and
Figure 4 is a flowchart showing a control method of an electronic device according to an embodiment of the present disclosure.

Since these embodiments can be modified in various ways and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives to the embodiments of the present disclosure. In connection with the description of the drawings, similar reference numbers may be used for similar components.

In describing the present disclosure, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, the detailed description thereof will be omitted.

In addition, the following examples may be modified into various other forms, and the scope of the technical idea of the present disclosure is not limited to the following examples. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to completely convey the technical idea of the present disclosure to those skilled in the art.

The terms used in this disclosure are merely used to describe specific embodiments and are not intended to limit the scope of rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present disclosure, expressions such as “have,” “may have,” “includes,” or “may include” refer to the presence of the corresponding feature (e.g., component such as numerical value, function, operation, or part). , and does not rule out the existence of additional features.

In the present disclosure, expressions such as “A or B,” “at least one of A or/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” includes (1) at least one A, (2) at least one B, or (3) it may refer to all cases including both at least one A and at least one B.

Expressions such as “first,” “second,” “first,” or “second,” used in the present disclosure can modify various components regardless of order and/or importance, and can refer to one component. It is only used to distinguish from other components and does not limit the components.

A component (e.g., a first component) is “(operatively or communicatively) coupled with/to” another component (e.g., a second component). When referred to as being “connected to,” it should be understood that any component may be directly connected to the other component or may be connected through another component (e.g., a third component).

On the other hand, when a component (e.g., a first component) is said to be “directly connected” or “directly connected” to another component (e.g., a second component), It may be understood that no other component (e.g., a third component) exists between other components.

The expression “configured to” used in the present disclosure may mean, for example, “suitable for,” “having the capacity to,” depending on the situation. ," can be used interchangeably with "designed to," "adapted to," "made to," or "capable of." The term “configured (or set to)” may not necessarily mean “specifically designed to” in hardware.

Instead, in some contexts, the expression “a device configured to” may mean that the device is “capable of” working with other devices or components. For example, the phrase "processor configured (or set) to perform A, B, and C" refers to a processor dedicated to performing the operations (e.g., an embedded processor), or by executing one or more software programs stored on a memory device. , may refer to a general-purpose processor (e.g., CPU or application processor) capable of performing the corresponding operations.

In an embodiment, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'units' may be integrated into at least one module and implemented with at least one processor, except for 'modules' or 'units' that need to be implemented with specific hardware.

Meanwhile, various elements and areas in the drawing are schematically drawn. Accordingly, the technical idea of the present invention is not limited by the relative sizes or spacing drawn in the attached drawings.

Hereinafter, with reference to the attached drawings, embodiments according to the present disclosure will be described in detail so that those skilled in the art can easily implement them.

1 is a flowchart briefly illustrating the configuration of an electronic device 100 according to an embodiment of the present disclosure. FIG. 2 is a diagram for explaining in detail the operation of the posture information acquisition model 210 and the posture evaluation model 220 according to the present disclosure. Hereinafter, various embodiments of the present disclosure will be described with reference to FIGS. 1 and 2.

As shown in FIG. 1, the electronic device 100 according to the present disclosure may include a foot pressure sensor 110, a camera 120, a memory 130, and a processor 140.

First, the electronic device 100 according to the present disclosure refers to a device that can provide evaluation information about the user's posture. For example, the electronic device 100 may be a device such as a PC, smartphone, digital TV, tablet PC, etc. However, there is no particular limitation on the type of the electronic device 100 according to the present disclosure.

There is no particular limitation on the user's 'posture', which is the subject of evaluation according to the present disclosure. For example, the user's posture may refer to the user's posture while the user is doing physical training such as various types of strength training, stretching, etc., the user's posture while standing, or the user's posture while walking, etc. can do.

The foot pressure sensor 110 may obtain a center of pressure value (COP value) indicating the pressure caused by the user's foot. Here, the 'pressure center value' refers to a value representing the average weight of all pressure points in contact with the ground as a point of the vertical repulsion force vector at the user's foot.

Specifically, the foot pressure sensor 110 may be comprised of a sensor array having a plurality of rows and columns. The foot pressure sensor 110 can obtain information about the distribution of pressure applied to the sensor array by the user's support of one or both feet, and obtain the center of pressure value based on this. There is no particular limitation on the type and number of foot pressure sensors 110 that can be applied to the present disclosure.

The camera 120 may acquire an image for at least one object. Specifically, the camera 120 includes an image sensor, and the image sensor can convert light coming through the lens into an electrical image signal.

In particular, the camera 120 according to an embodiment of the present disclosure may be a stereo camera 120 configured to acquire an image including depth information, but is not limited thereto.

At least one instruction regarding the electronic device 100 may be stored in the memory 130 . Additionally, an operating system (O/S) for driving the electronic device 100 may be stored in the memory 130. Additionally, the memory 130 may store various software programs or applications for operating the electronic device 100 according to various embodiments of the present disclosure. Additionally, the memory 130 may include a semiconductor memory such as flash memory or a magnetic storage medium such as a hard disk.

Specifically, the memory 130 may store various software modules for operating the electronic device 100 according to various embodiments of the present disclosure, and the processor 140 executes various software modules stored in the memory 130. Thus, the operation of the electronic device 100 can be controlled. That is, the memory 130 is accessed by the processor 140, and reading/writing/modifying/deleting/updating data by the processor 140 can be performed.

Meanwhile, in the present disclosure, the term memory 130 refers to the memory 130, a ROM (not shown) within the processor 140, a RAM (not shown), or a memory card (not shown) mounted on the electronic device 100 (e.g. For example, it can be used to mean including micro SD card, memory stick).

In particular, in various embodiments according to the present disclosure, the memory 130 may store various information such as center of pressure value, joint angle information, user body information, user posture information, and posture evaluation information. Additionally, information about the posture information acquisition model 210 and information about the posture evaluation model 220 according to the present disclosure may be stored in the memory 130.

In addition, various information necessary within the scope of achieving the purpose of the present disclosure may be stored in the memory 130, and the information stored in the memory 130 may be updated as it is received from an external device or input by the user. .

The processor 140 controls the overall operation of the electronic device 100. Specifically, the processor 140 is connected to the configuration of the electronic device 100 as shown in FIGS. 1 and 3, and executes at least one instruction stored in the memory 130 as described above, thereby controlling the electronic device ( 100) operations can be controlled overall.

Processor 140 may be implemented in various ways. For example, the processor 140 may include an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM), and a digital signal processor. Processor, DSP). Meanwhile, in the present disclosure, the term processor 140 may be used to include a central processing unit (CPU), a graphics processing unit (GPU), and a main processing unit (MPU).

In particular, the processor 140 can provide posture evaluation information about the user's posture based on information acquired through the foot pressure sensor 110 and the camera 120. Hereinafter, in the control process of the processor 140, Various related embodiments will be described.

The processor 140 may obtain a pressure center value representing the pressure caused by the user's foot through the foot pressure sensor 110. Specifically, the processor 140 may obtain the center of pressure value through the foot pressure sensor 110 based on the pressure applied to the sensor array by the user supporting one or both feet. The center of pressure value may be expressed as multiple values rather than a single value. Meanwhile, the description below will be made on the assumption that the center of pressure value is used, but depending on the embodiment, various information that can be calculated based on information about foot pressure distribution may be used instead of the center of pressure value.

The processor 140 may acquire an image of the user through the camera 120, and obtain joint angle information indicating the angles between the user's joints based on the image of the user. Specifically, when an image of the user is acquired through the camera 120, the processor 140 acquires joint position information (skeleton information) indicating the positions of the user's joints based on the image of the user, and joint position information Joint angle information can be obtained based on .

Here, 'joint angle information' refers to information indicating the angles between a plurality of joints constituting the user's body. The joint angle information may be three-dimensional joint angle information acquired through the stereo camera 120 or the depth camera 120, but depending on the embodiment, two-dimensional joint angle information may be used.

The joint angle information may include a plurality of angle information (eg, 18 pieces) with joints constituting the body of various users as the origin. For example, joint angle information includes information about the angle between the line segment connecting the elbow joint (elbow joint) and shoulder joint (shoulder joint) and the line segment connecting the elbow joint and arm joint (wrist joint), and the knee joint (knee joint) and hip joint. It may include information about the angle between the line segment connecting the (femoral joint) and the line segment connecting the knee joint and the ankle joint (ankle joint).

In one embodiment, the processor 140 may identify a first vector where the first position is a start point and the second position is an end point among a plurality of positions corresponding to the user's joints, , it is possible to identify a second vector where the second position is the starting point and the third position is the end point among the plurality of positions corresponding to the user's joints. Additionally, the processor 140 may obtain first joint angle information indicating the angle between the first vector and the second vector based on the dot product of the first vector and the second vector. Likewise, the processor 140 may generate second joint angle information indicating the angle between a third vector where the second position is the starting point and the third position is the end point, and a fourth vector where the third position is the starting point and the fourth position is the end point. Joint angle information based on various positions can be obtained.

The processor 140 may obtain user posture information representing the user's posture by inputting the center of pressure value and joint angle information into the posture information acquisition model 210.

Referring to FIG. 2, the 'posture information acquisition model 210' refers to a neural network model learned to output joint user posture information when the center of pressure value and joint angle information are input. The posture information acquisition model 210 may calculate user posture information by considering both joint angle information representing the user's motion and the center of pressure value representing the user's balance. Here, 'user posture information' is used as a general term for information representing the user's posture, considering both the user's motion and balance.

In particular, the center of pressure value and joint angle information according to the present disclosure may be acquired at preset time intervals, and the processor 140 combines the sequentially acquired center of pressure value and joint angle information into posture information between information acquired at the same time. By inputting it into the acquisition model 210, user posture information representing the user's posture at the relevant time can be obtained. The posture information acquisition model 210 may be composed of an artificial neural network such as LSTM (Long Short-Term Memory) to process sequentially input pressure center values and joint angle information. However, the type and number of specific artificial neural networks may be special. There are no restrictions.

The processor 140 may input user posture information into the posture evaluation model 220 to obtain posture evaluation information about the user's posture.

Referring to FIG. 2, the 'posture evaluation model 220' refers to a neural network model learned to output posture evaluation information about the user's posture when user posture information is input. Specifically, the posture evaluation model 220 can calculate a score according to the similarity between the user posture information and the standard posture information by comparing the input user posture information with pre-established standard posture information, and based on the calculated score. Thus, posture evaluation information can be obtained. A score according to similarity may be obtained based on probability information indicating the degree to which user posture information is similar to standard posture information. The posture evaluation model 220 may be made of an artificial neural network such as CNN (Convolutional Neural Networks), but is not limited thereto.

'Standard posture information' refers to information indicating correct posture and is already established information. In other words, standard posture information refers to information that serves as an answer sheet for comparison with user posture information.

Standard posture information can be obtained by inputting learning data including center of pressure values and joint angle information for a plurality of users into the posture information acquisition model 210. For example, center of pressure values and joint angle information for a plurality of users with correct posture, such as personal trainers, Pilates instructors, etc., may be input to the posture information acquisition model 210, and the posture information acquisition model 210 may input User posture information corresponding to each set of pressure center values and joint angle information can be output. Additionally, the posture information acquisition model 210 may acquire standard posture information by taking the average of the output user posture information or comparing the output user posture information with label information.

Standard posture information may be augmented through a neural network such as a Generative Adversarial Network (GAN) neural network, and this GAN neural network may be included in the posture information acquisition model 210.

'Postural evaluation information' is used as a general term for information indicating whether the user's posture is correct according to the result of comparison between the user's posture information and standard posture information. Posture evaluation information may include coaching information to guide the user's posture in real time. For example, the posture evaluation information may include various types of coaching information, such as information for guiding the user to change his or her posture, information for guiding additional training time, or additional number of training times.

Meanwhile, if the score calculated through the posture evaluation model 220 is greater than or equal to a preset threshold, the processor 140 may update the learning data based on the user posture information. In other words, if user posture information for an actual user is identified as being very similar to standard posture information, the user posture information may be included in learning data for obtaining standard posture information.

The processor 140 may provide the obtained posture evaluation information. Specifically, posture evaluation information may be provided in image form through a user interface (UI), may be provided in text form through a chatbot, or may be provided in voice form through a speaker. . In addition, the posture evaluation information may be transmitted to an external device, such as a user terminal, and then provided to the user through the external device. Various methods of providing posture evaluation information will be described in more detail with reference to FIG. 3.

Meanwhile, in the above description, the user posture information is acquired based on the center of pressure value and joint angle information. However, according to the disclosure, the information used to acquire the user posture information is limited to the center of pressure value and joint angle information. no.

In one embodiment, the processor 140 may obtain user body information including information on the user's body mass index (BMI) as well as center of pressure value and joint angle information. 'Body mass index' is an index indicating the degree of obesity in humans, and can be calculated according to the relationship between weight and height. 'User body information' is used as a general term for information about the user's body, including body mass index. User body information may not only be directly input by the user, but may also be received from an external device such as a user terminal or server.

When acquiring user posture information using user body information about body mass index, the posture information acquisition model 210 may be trained to output user posture information when the center of pressure value, joint angle information, and user body information are input. , the processor 140 may obtain user posture information by inputting the center of pressure value, joint angle information, and user body information into the posture information acquisition model 210.

According to the above-described embodiment, the electronic device 100 according to the present disclosure quickly and conveniently obtains information about the user's posture using only a small number of sensors and objectively and accurately provides evaluation information about the user's posture. can be provided.

Specifically, according to the present disclosure, since information about the user's posture is acquired using only the foot pressure sensor 110 and the camera 120, information processing speed is fast and stability in the event of a failure or malfunction is excellent. In addition, accurate information about the user's posture can be obtained with only a small amount of data such as joint angle information representing the user's motion and a center of pressure value representing the user's balance, and a posture information acquisition model 210 and a posture evaluation model ( 220), it is possible to provide objective posture evaluation information. In addition, according to the present disclosure, since the user does not need to apply the sensor to the body, separate calibration for the user is not required, and it has the advantage of being conveniently used in various environments.

FIG. 3 is a flowchart illustrating in detail the configuration of the electronic device 100 according to an embodiment of the present disclosure.

As shown in FIG. 3, the electronic device 100 according to the present disclosure includes a foot pressure sensor 110, a camera 120, a memory 130, and a processor 140, as well as a communication unit 150 and an input unit 160. and an output unit 170. However, the configurations shown in FIGS. 1 and 3 are merely exemplary, and in carrying out the present disclosure, new configurations may be added or some configurations may be omitted in addition to the configurations shown in FIGS. 1 and 3. Of course it exists.

The communication unit 150 includes a circuit and can perform communication with an external device. Specifically, the processor 140 can receive various data or information from an external device connected through the communication unit 150, and can also transmit various data or information to the external device.

The communication unit 150 may include at least one of a WiFi module, a Bluetooth module, a wireless communication module, an NFC module, and a UWB module (Ultra Wide Band). Specifically, the WiFi module and the Bluetooth module can each communicate using WiFi and Bluetooth methods. When using a WiFi module or Bluetooth module, various connection information such as SSID is first transmitted and received, and various information can be transmitted and received after establishing a communication connection using this.

Additionally, the wireless communication module can perform communication according to various communication standards such as IEEE, Zigbee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and 5th Generation (5G). In addition, the NFC module can perform communication using the NFC (Near Field Communication) method using the 13.56MHz band among various RF-ID frequency bands such as 135kHz, 13.56MHz, 433MHz, 860~960MHz, 2.45GHz, etc. In addition, through communication between UWB antennas, the UWB module can accurately measure ToA (Time of Arrival), which is the time for a pulse to reach the target, and AoA (Ange of Arrival), which is the angle of arrival of the pulse at the transmitter, and can thus accurately measure indoor Precise distance and position recognition is possible within an error range of several tens of centimeters.

In particular, in various embodiments according to the present disclosure, the processor 140 may control the communication unit 150 to transmit posture evaluation information to an external device such as a user terminal or a server. Additionally, the processor 140 may receive user body information from an external device through the communication unit 150.

Meanwhile, the above description has been made on the assumption that the foot pressure sensor 110 and the camera 120 according to the present disclosure are included in the electronic device 100 as components of the electronic device 100. However, depending on the embodiment, the foot pressure sensor 110 and the camera 120 are included in the electronic device 100. The sensor 110 and the camera 120 are included in one or two external devices, and the processor 140 receives the center of pressure value and/or joint angle information from the external device through the communication unit 150 to obtain the center of pressure value and/or joint angle information. /Or joint angle information may be obtained.

The input unit 160 includes a circuit, and the processor 140 can receive user commands for controlling the operation of the electronic device 100 through the input unit 160. Specifically, the input unit 160 may be comprised of a microphone, a camera 120 (not shown), and a remote control signal receiver (not shown). Additionally, the input unit 160 may be implemented as a touch screen included in the display. In particular, the microphone can receive voice signals and convert the received voice signals into electrical signals.

In particular, in various embodiments according to the present disclosure, the processor 140 may receive a user command for receiving posture evaluation information through the input unit 160, and operate according to the present disclosure based on the received user command. can be performed. Additionally, the processor 140 may receive user body information, that is, information about the user's height and height, directly from the user through the input unit 160.

The processor 140 may receive user feedback on posture evaluation information through the input unit 160, and may use information about the received feedback to learn the posture evaluation model 220.

The output unit 170 includes a circuit, and the processor 140 can output various functions that the electronic device 100 can perform through the output unit 170. And, the output unit 170 may include at least one of a display, a speaker, and an indicator.

The display may output image data under the control of the processor 140. Specifically, the display may output images previously stored in the memory 130 under the control of the processor 140. In particular, the display according to an embodiment of the present disclosure may display a user interface (User Interface) stored in the memory 130. The display can be implemented as a liquid crystal display panel (LCD), organic light emitting diodes (OLED), etc., and in some cases, the display can also be implemented as a flexible display, transparent display, etc. However, the display according to the present disclosure is not limited to a specific type.

The speaker may output audio data under the control of the processor 140, and the indicator may light up under the control of the processor 140.

In particular, in various embodiments according to the present disclosure, the processor 140 controls the display to display a user interface (UI) including posture evaluation information in order to provide posture evaluation information to the user in the form of an image. You can. Additionally, the processor 140 may control the display to display a message including posture evaluation information in order to provide posture evaluation information in text form.

As an example, the processor 140 may provide posture evaluation information to the user in a chatbot manner. Additionally, in order to provide the posture evaluation information in voice form, the processor 140 may convert the posture evaluation information into a voice signal and output it through a speaker. In the process of converting posture evaluation information into a voice signal, a learned text-to-speech (TTS) model can be used.

FIG. 4 is a flowchart showing a control method of the electronic device 100 according to an embodiment of the present disclosure.

Referring to FIG. 4, the electronic device 100 may obtain a center of pressure value representing the pressure caused by the user's foot through a foot pressure sensor (S410). Specifically, the electronic device 100 may obtain the center of pressure value through the foot pressure sensor based on the pressure applied to the sensor array by the user supporting one or both feet.

The electronic device 100 may acquire an image of the user through a camera (S420) and obtain joint angle information indicating the angles between the user's joints based on the image of the user (S430). Specifically, when an image of the user is acquired through a camera, the electronic device 100 acquires joint position information indicating the positions of the user's joints based on the image of the user, and joint angle information based on the joint position information. can be obtained.

When the center of pressure value and joint angle information are acquired, the electronic device 100 may obtain user posture information representing the user's posture by inputting the center of pressure value and joint angle information into the posture information acquisition model (S440). Specifically, the electronic device 100 inputs the sequentially acquired pressure center value and joint angle information into a posture information acquisition model with information acquired at the same time to obtain user posture information representing the user's posture at that time. You can.

Once the user posture information is acquired, the electronic device 100 may obtain posture evaluation information about the user's posture by inputting the user posture information into the posture evaluation model (S450). Specifically, the electronic device 100 can calculate a score based on the similarity between the user posture information and the standard posture information by comparing the user posture information with previously constructed standard posture information through a posture evaluation model, and the calculated score. Based on this, posture evaluation information can be obtained.

When the posture evaluation information is acquired, the electronic device 100 may provide the obtained posture evaluation information (S460). Specifically, posture evaluation information may be provided in image form through a user interface (UI), may be provided in text form through a chatbot, or may be provided in voice form through a speaker. . In addition, the posture evaluation information may be transmitted to an external device, such as a user terminal, and then provided to the user through the external device.

Meanwhile, the control method of the electronic device 100 according to the above-described embodiment may be implemented as a program and provided to the electronic device 100. In particular, a program including a control method for the electronic device 100 may be stored and provided in a non-transitory computer readable medium.

Specifically, in a non-transitory computer-readable recording medium including a program for executing a control method of the electronic device 100, the control method of the electronic device 100 obtains a center of pressure value representing the pressure caused by the user's foot. steps, obtaining an image of the user, obtaining joint angle information representing the angle between the user's joints based on the image, inputting the center of pressure value and joint angle information into the posture information acquisition model to determine the user's It includes obtaining user posture information indicating the posture, acquiring posture evaluation information about the user's posture by inputting the user posture information into a posture evaluation model, and providing posture evaluation information.

In the above, the control method of the electronic device 100 and the computer-readable recording medium including the program for executing the control method of the electronic device 100 have been briefly described. However, this is only to omit redundant description, and the electronic device 100 control method has been briefly described. Of course, various embodiments of the device 100 can also be applied to a control method of the electronic device 100 and a computer-readable recording medium including a program that executes the control method of the electronic device 100.

Meanwhile, functions related to the posture information acquisition model and posture evaluation model as described above may be performed through memory and processor.

The processor may consist of one or multiple processors. At this time, one or more processors may be general-purpose processors such as CPU, AP, GPU, etc. It may be a processor dedicated to graphics, such as a VPU, or a processor dedicated to artificial intelligence, such as an NPU.

One or more processors control input data to be processed according to predefined operation rules or artificial intelligence models stored in non-volatile memory and volatile memory. Predefined operation rules or artificial intelligence models are characterized by being created through learning.

Here, being created through learning means that a predefined operation rule or artificial intelligence model with desired characteristics is created by applying a learning algorithm to a large number of learning data. This learning may be accomplished in the device itself that performs artificial intelligence according to the present disclosure, or may be accomplished through a separate server/system.

An artificial intelligence model may be composed of multiple neural network layers. Each layer has multiple weight values, and layer operations are performed through the operation results of the previous layer and multiple weight values. Examples of neural networks include CNN (Convolutional Neural Network), DNN (Deep Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), BRDNN (Bidirectional Recurrent Deep Neural Network), and GAN. (Generative Adversarial Networks) and Deep Q-Networks (Deep Q-Networks), and the neural network in the present disclosure is not limited to the above examples except as specified.

A learning algorithm is a method of training a target device (eg, a robot) using a large number of learning data so that the target device can make decisions or make predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, and the learning algorithm in the present disclosure is specified. Except, it is not limited to the examples described above.

A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' simply means that it is a tangible device and does not contain signals (e.g. electromagnetic waves). This term refers to cases where data is semi-permanently stored in a storage medium and temporary storage media. It does not distinguish between cases where it is stored as . For example, a 'non-transitory storage medium' may include a buffer where data is temporarily stored.

According to one embodiment, methods according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or via an application store (e.g. Play Store ^TM ) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smartphones) or online. In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) is stored on a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server. It can be temporarily stored or created temporarily.

Each component (e.g., module or program) according to various embodiments of the present disclosure as described above may be composed of a single or multiple entities, and some of the sub-components described above may be omitted. Alternatively, other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity and perform the same or similar functions performed by each corresponding component prior to integration.

According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added. You can.

Meanwhile, the term "unit" or "module" used in the present disclosure includes a unit comprised of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. You can. A “part” or “module” may be an integrated part, a minimum unit that performs one or more functions, or a part thereof. For example, a module may be comprised of an application-specific integrated circuit (ASIC).

Various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage media that can be read by a machine (e.g., a computer). The device calls the stored instructions from the storage medium. And, the device capable of operating according to the called command may include an electronic device (eg, the electronic device 100) according to the disclosed embodiments.

When the instruction is executed by a processor, the processor may perform the function corresponding to the instruction directly or using other components under the control of the processor. Instructions may contain code generated or executed by a compiler or interpreter.

In the above, preferred embodiments of the present disclosure have been shown and described, but the present disclosure is not limited to the specific embodiments described above, and may be used in the technical field to which the disclosure pertains without departing from the gist of the disclosure as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical ideas or perspectives of the present disclosure.

100: Electronic device 110: Foot pressure sensor
120: Camera 130: Memory
140: processor

Claims (7)

In electronic devices,
A foot pressure sensor including a sensor array having a plurality of rows and columns;
camera;
a memory storing at least one instruction; and
a processor executing the at least one instruction; Including,
The processor,
Obtaining a pressure center value representing the pressure caused by the user's foot based on the distribution of pressure applied to the sensor array of the foot pressure sensor,
Obtaining an image of the user through the camera,
Obtain joint position information indicating a plurality of positions corresponding to the user's joints based on the image, and obtain joint angle information indicating angles between the user's joints based on the joint position information,
Input the pressure center value and the joint angle information into a posture information acquisition model to obtain user posture information representing the user's posture,
Input the user posture information into a posture evaluation model to obtain posture evaluation information about the user's posture,
Provide the posture assessment information,
The user posture information is,
An electronic device wherein the user's posture information is calculated based on the center of pressure value and the joint angle information.
delete According to claim 1,
The processor,
Obtaining user body information including information about the user's body mass index (BMI),
An electronic device that acquires the user posture information by inputting the pressure center value, the joint angle information, and the user body information into the posture information acquisition model.
According to claim 1,
The posture evaluation information is calculated based on a score based on the similarity between the user posture information and standard posture information,
The standard posture information is obtained by inputting learning data including pressure center values and joint angle information for a plurality of users into the posture information acquisition model.
According to clause 4,
The processor,
If the score is greater than or equal to a preset threshold, the electronic device updates the learning data based on the user posture information.
According to claim 1,
The posture evaluation information includes coaching information for teaching the user's posture in real time.
In a method of controlling an electronic device,
Obtaining a pressure center value representing the pressure caused by the user's foot based on the distribution of pressure applied to the sensor array of the foot pressure sensor;
Obtaining an image for the user;
Obtaining joint position information indicating a plurality of positions corresponding to the user's joints based on the image, and obtaining joint angle information indicating angles between the user's joints based on the joint position information;
acquiring user posture information representing the user's posture by inputting the center of pressure value and the joint angle information into a posture information acquisition model;
acquiring posture evaluation information about the user's posture by inputting the user posture information into a posture evaluation model; and
providing the posture evaluation information; Including,
The user posture information is,
A control method of an electronic device in which the user's posture information is calculated based on the center of pressure value and the joint angle information.

Images