KR102490430B1 - Control method of electronic device to acquire command for either positive or negative word based on joint tracking - Google Patents

Abstract

The present invention relates to a control method of an electronic device for acquiring a command for any one among a positive word and a negative word based on joint tracking comprising: a step of identifying, by an electronic device, a person object from one image; a step of matching, by the electronic device, a plurality of points for the person object; a step of tracking, by the electronic device, a motion for the person object within the image based on the plurality of points; and a step of identifying, by the electronic device, any one among a positive motion or a negative motion based on a result for tracking. Therefore, the present invention is capable of enabling a user command to be obtained quickly.

Description

Control method of an electronic device obtaining a command for any one of positive and negative words based on joint tracking

The present invention relates to a control method of an electronic device for acquiring a command for any one of positive and negative words based on joint tracking, and more particularly, by identifying each joint of a person included in an image to determine the size of the joint. It is a technology that recognizes motion based on location.

As a method of acquiring a user command for a conventional electronic device, there have been methods such as input through a key button and touch input to a user interface (UI) provided through a display.

In the case of the key button method, the user's fatigue can be created in a series of processes of recognizing and inputting buttons of a limited size, and in the case of the touch input method through the UI, it is difficult to access for the first time for a specific age (ex. the elderly). problems have existed.

Therefore, when essential user commands such as simple positive and negative words are input to an electronic device, there is a need to provide a method of inputting a user command to an electronic device with a simple operation.

On the other hand, the matters described as the background art above are only for improving understanding of the background of the present invention, and should not be taken as an admission that they correspond to prior art already known to those skilled in the art. will be.

Registered Patent Publication No. 10-1304321, 2013.08.30.

The problem to be solved by the present invention is to obtain a simple motion (ex. implementing a circular shape with both arms, implementing a cross shape, etc.) through a camera, and for any one of positive and negative words according to the location of a specific joint. An object of the present invention is to provide a control method of an electronic device that obtains a command for any one of a positive word and a negative word based on joint tracking for obtaining a user command.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

In the control method of an electronic device for obtaining a command for any one of a positive word and a negative word based on joint tracking according to an aspect of the present invention for solving the above problems, the electronic device, a person object from an image Identifying, by an electronic device, matching a plurality of points of a person object, by an electronic device, based on a plurality of points, tracking motion in an image of a person object, and by the electronic device , and identifying one of positive motion and negative motion based on the tracking result.

At this time, the step of matching a plurality of points includes identifying the face of the person object, identifying the neck, left shoulder, left elbow, left wrist, right shoulder, right elbow, and right wrist based on the face; To match the first point, the second point on the left shoulder, the third point on the left elbow, the fourth point on the left wrist, the fifth point on the right shoulder, the sixth point on the right elbow, and the seventh point on the right wrist. Step, generating a first line for the first point, second point, and fifth point, generating a second line by the second to fourth points, and by the fifth to seventh points It may include generating a third line.

In addition, the tracking of the motion in the image of the person object may include tracking the coordinates of each of the first to seventh points, identifying a shape formed by the first to third lines, If the shape is a preset first shape, calculating a first distance that is the distance between the third point and the sixth point based on the coordinates, and the distance between the fourth point and the seventh point based on the coordinates Including calculating two distances, and identifying one of positive motion and negative motion based on the tracking result, if the second distance is less than the first distance and the second distance is less than a certain distance, , it is possible to identify positive motion.

Meanwhile, the step of identifying a shape formed by the first to third lines may include, when the shape is a preset second shape, determining whether there is an intersection with another line among the first to third lines. Step, if there is no intersection, determining whether there is one intersection by the second line and the third line, and if there is one intersection by the second and third lines, the intersection is The step of determining whether the fourth point and the seventh point coincide with at least one of the points, and the step of identifying one of the positive motion and the negative motion based on the result of the tracking comprises: In case of discrepancies for each of the 7 points, it is possible to identify false motion.

At this time, the step of identifying any one of the positive motion and the negative motion based on the tracking result is, when either the positive motion or the negative motion is identified, if the motion is maintained for a certain period of time from the first identified time point, the identification It is possible to obtain user commands for the selected motion.

In addition, in the control method of an electronic device according to an aspect of the present invention, when obtaining a user command for a positive motion, the electronic device determines between a vertical axis coordinate value for a fourth point and a vertical axis coordinate value for a seventh point. Calculating a first difference value, which is a difference value; If the first difference value exceeds a predetermined value, the electronic device calculates a second difference value, which is a difference between the vertical axis coordinate value of the second point and the vertical axis coordinate value of the fifth point; If the second difference value exceeds a predetermined value, identifying, by the electronic device, a point having a smaller value among the ordinate coordinate values of the second point and the ordinate coordinate value of the fifth point, and then, by the electronic device, identifying the left side of the person object. For a shoulder corresponding to a point having a small value identified among the shoulder and the right shoulder, the step of calculating the second difference value includes obtaining a problem joint, and the step of calculating the second difference value comprises: Identifying a point having a larger value among the ordinate coordinate value of the point and the ordinate coordinate value of the fifth point, and the elbow in the direction corresponding to the identified point having the larger value among the left elbow and right elbow of the person object. For this, it may include acquiring a problem joint.

Accordingly, a control method of an electronic device according to an aspect of the present invention includes obtaining, by the electronic device, a diagnosis request from a user corresponding to a person object; identifying, by the electronic device, calculating the number of times acquired as a problem joint for each problem joint; if at least one diagnostic joint acquired with a number exceeding a threshold number among the problem joints is identified, the electronic device, Acquiring, by an electronic device, a point for each of the diagnosed joint and at least one joint adjacent to the diagnosed joint to the diagnostic image; and , providing a diagnostic image matched with points to the user.

Other specific details of the invention are included in the detailed description and drawings.

According to the control method of an electronic device for acquiring a command for any one of a positive word and a negative word based on joint tracking of the present invention, a user's motion can be obtained through a camera and recognized in real time to rapidly obtain a user command. can

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a configuration diagram of an electronic device according to an embodiment of the present disclosure.
2 is a basic flow diagram according to an embodiment of the present disclosure.
3 is an exemplary view of point matching for a joint according to an embodiment of the present disclosure.
4 and 5 are exemplary views of a first shape according to an embodiment of the present disclosure.
6 and 7 are exemplary views of a second shape according to an embodiment of the present disclosure.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only these embodiments are intended to complete the disclosure of the present invention, and are common in the art to which the present invention belongs. It is provided to fully inform the person skilled in the art of the scope of the invention, and the invention is only defined by the scope of the claims.

Terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" does not exclude the presence or addition of one or more other elements other than the recited elements. Like reference numerals throughout the specification refer to like elements, and “and/or” includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first element mentioned below may also be the second element within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

The term "unit" or "module" used in the specification means a hardware component such as software, FPGA or ASIC, and "unit" or "module" performs certain roles. However, "unit" or "module" is not meant to be limited to software or hardware. A “unit” or “module” may be configured to reside in an addressable storage medium and may be configured to reproduce one or more processors. Thus, as an example, a “unit” or “module” may refer to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Functions provided within components and "units" or "modules" may be combined into smaller numbers of components and "units" or "modules" or may be combined into additional components and "units" or "modules". can be further separated.

The spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe a component's correlation with other components. Spatially relative terms should be understood as including different orientations of elements in use or operation in addition to the orientations shown in the drawings. For example, if you flip a component that is shown in a drawing, a component described as "below" or "beneath" another component will be placed "above" the other component. can Thus, the exemplary term “below” may include directions of both below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

In this specification, a computer means any kind of hardware device including at least one processor, and may be understood as encompassing a software configuration operating in a corresponding hardware device according to an embodiment. For example, a computer may be understood as including a smartphone, a tablet PC, a desktop computer, a laptop computer, and user clients and applications running on each device, but is not limited thereto.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram of an electronic device according to an embodiment of the present disclosure.

As shown, the electronic device 100 for acquiring a command for any one of positive and negative words based on joint tracking according to an aspect of the present invention includes a memory 110 and an image including a user's motion. It includes a camera 120 for acquiring, a display 130 for outputting an image, and a processor 140 connected to the memory 110, the camera 120, and the display 130.

In an embodiment, the electronic device 100 includes a server, a smartphone, a tablet personal computer (PC), a mobile phone, a video phone, a laptop PC, and a netbook computer. , a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a wearable device, and a portable medical device.

Meanwhile, the memory 110 may store various programs and data necessary for the operation of the electronic device 100 . The memory 110 may be implemented as a non-volatile memory 110, a volatile memory 110, a flash-memory, a hard disk drive (HDD), or a solid state drive (SSD).

The processor 140 may control overall operations of the electronic device 100 using various programs stored in the memory 110 . The processor 140 may include a RAM, a ROM, a graphic processing unit, a main CPU, first through n interfaces, and a bus. At this time, the RAM, ROM, graphic processing unit, main CPU, first to n interfaces, etc. may be connected to each other through a bus.

RAM stores O/S and application programs. Specifically, when the electronic device 100 is booted, O/S is stored in RAM, and various application data selected by the user may be stored in RAM.

The ROM stores instruction sets for system booting. When a turn-on command is input and power is supplied, the main CPU copies the O/S stored in the memory 110 to the RAM according to the command stored in the ROM, and executes the O/S to boot the system. When booting is completed, the main CPU copies various application programs stored in the memory 110 to RAM, and executes the application programs copied to RAM to perform various operations.

The graphic processing unit uses a calculation unit (not shown) and a rendering unit (not shown) to create a screen including various objects such as items, images, and text. Here, the calculation unit may be configured to calculate attribute values such as coordinate values, shape, size, color, etc. of each object to be displayed according to the layout of the screen by using a control command received from the input unit. And, the rendering unit may be configured to generate screens of various layouts including objects based on the attribute values calculated by the calculation unit. The screen created by the rendering unit may be displayed within the display area of the display 130 .

The main CPU accesses the memory 110 and performs booting using the OS stored in the memory 110 . And, the main CPU performs various operations using various programs, contents, data, etc. stored in the memory 110 .

The first through n interfaces are connected to the various components described above. One of the first through n interfaces may be a network interface connected to an external device through a network.

Meanwhile, the processor 140 may include one or more cores (not shown) and a graphic processor (not shown) and/or a connection path (eg, a bus) for transmitting and receiving signals to and from other components. can

Processor 140 according to one embodiment performs a method described in connection with the present invention by executing one or more instructions stored in memory 110 .

For example, the processor 140 acquires new training data by executing one or more instructions stored in the memory 110, performs a test on the acquired new training data using a learned model, and performs the test As a result, first training data for which the labeled information is obtained with an accuracy equal to or higher than a predetermined first reference value is extracted, the extracted first training data is deleted from the new training data, and the new training data from which the extracted training data is deleted The learned model may be retrained using the training data.

Meanwhile, the processor 140 includes RAM (Random Access Memory, not shown) and ROM (Read-Only Memory) for temporarily and/or permanently storing signals (or data) processed in the processor 140. , not shown) may be further included. In addition, the processor 140 may be implemented in the form of a system on chip (SoC) including at least one of a graphics processing unit, RAM, and ROM.

The memory 110 may store programs (one or more instructions) for processing and control of the processor 140 . Programs stored in the memory 110 may be divided into a plurality of modules according to functions.

The camera 120 is configured to photograph at least one subject, and the processor 140 may recognize at least one object or identify a distance to the object through the camera 120 .

As an embodiment, the camera 120 may be implemented as a depth camera or RGB camera that acquires a 3D image. The depth camera is a stereo camera (stereoscopic camera, stereo camera, binocular camera, two-view camera) that acquires two images at the same time and acquires the subject in three dimensions, and based on the return time and speed of the reflected light on the subject, the background It may be a ToF camera (Time of Flight camera) that obtains 3D image information of the subject.

The display 130 is a configuration for visually outputting various information, and may be implemented as a Liquid Crystal Display (LCD), Plasma Display Panel (PDP), Organic Light Emitting Diodes (OLED), Transparent OLED (TOLED), Micro LED, etc. However, it is not limited thereto, and other types of conventionally known displays may be included.

As an example, the display 130 may be implemented in the form of a touch screen capable of detecting a user's touch manipulation, or may be implemented as a flexible display capable of being folded or bent.

As an example, the electronic device 100 may further include a communication unit and perform communication with an external device (eg, a server, a separate electronic device possessed by a user, another user's electronic device, etc.) through the communication unit.

In particular, the communication unit may include various communication chips such as a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, an NFC chip, and a low energy Bluetooth chip (BLE chip).

At this time, the Wi-Fi chip, the Bluetooth chip, and the NFC chip perform communication in a LAN method, a WiFi method, a Bluetooth method, and an NFC method, respectively. In the case of using a Wi-Fi chip or a Bluetooth chip, various connection information such as an SSID and a session key is first transmitted and received, and various information can be transmitted and received after communication is connected using this.

The wireless communication chip refers to a chip that performs communication according to various communication standards such as IEEE, ZigBee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and 5th Generation (5G).

2 is a basic flow diagram according to an embodiment of the present disclosure.

As shown, in the control method of the electronic device 100 for obtaining a command for any one of a positive word and a negative word based on joint tracking according to an aspect of the present invention, the electronic device 100 includes: The person object 1 is identified from the image (S210), and the plurality of points 11 to 17 are matched with the person object 1 (S220), based on the plurality of points 11 to 17, the person object The motion in the video for is tracked (S230), and based on the tracking result, either positive motion or negative motion is identified (S240).

Specifically, in performing step S210, at least one object included in the image may be identified, and a person object 1 that is an object for a person may be identified among the identified objects.

In this case, when a user's face image is pre-registered in the memory 110, the electronic device 100 may identify the person object 1 based on the user's face image.

As an embodiment, when there are a plurality of people in the image, the electronic device 100 identifies a person object 1 for each person, obtains a face image for each person object 1, and uses the face image as a standard. As a result, steps S220 to S240 may be performed for each of the person objects 1 . For example, when an application for collecting opinions on an agenda is executed through the electronic device 100, any one of positive and negative motions identified for each of the plurality of person objects 1 is identified, and Based on this, it is possible to collect opinions on the corresponding agenda by calculating the number of people who have taken a positive motion and the number of people who have taken a negative motion.

In addition, in performing step S240, when the electronic device 100 identifies any one of the positive motion and the negative motion, if the motion is maintained for a predetermined time from the initially identified time point, the user command (positive motion) for the identified motion any one of a word and a negative word) is obtained.

The user command is input to the electronic device 100 as yes or no.

3 is an exemplary view of point matching for a joint according to an embodiment of the present disclosure.

In performing step S220, the electronic device 100 identifies the face of the person object 1, that is, the face image, and based on the face, the neck, left shoulder, left elbow, and left wrist of the person corresponding to the person object. , right shoulder, right elbow, and right wrist.

Accordingly, as shown in FIG. 3 , the electronic device 100 has a first point 11 on the neck, a second point 12 on the left shoulder, a third point 13 on the left elbow, and a second point 13 on the left wrist. Match the 4 points (14), the 5th point (15) on the right shoulder, the 6th point (16) on the right elbow, and the 7th point (17) on the right wrist.

At this time, in identifying each joint (neck, shoulder, elbow, wrist), the electronic device 100 identifies not only the face of the person object 1 but also various body parts such as the neck, upper body, fingers, and ears. It is possible, but not limited thereto.

Then, the electronic device 100 generates a first line for the first point 11, the second point 12, and the fifth point 15, and the second point 12 to the fourth point 14 A second line may be generated by , and a third line may be generated by the fifth point 15 to the seventh point 17 .

At this time, the first line includes a 1-1 line having the first point and the second point as both vertices and a 1-2 line having the first point and the fifth point as both vertices, and the second line is Line 2-1 having both points 2 and 3 as both vertices, and line 2-2 having both points 3 and 4 as both vertices, and the 3rd line has the 5th and 6th points It is configured to include a 3-1 line having both vertexes, a 3-2 line having the 6th point and the 7th point as both vertexes.

As an example, in identifying the neck, left shoulder, left elbow, left wrist, right shoulder, right elbow, and right wrist of a person corresponding to a person object for a viewpoint in an image, at least one joint among them is When acquired, the electronic device 100 may identify the corresponding joint by identifying different viewpoints at which unacquired joints are respectively obtained.

Meanwhile, in performing step S230, the electronic device 100 tracks the coordinates of each of the first point 11 to the seventh point 17 and identifies a shape formed by the first line to the third line. can

The shape is formed by lines 1-1 to 3-2, and is at least one planar figure obtained by arbitrarily connecting the fourth point and the seventh point, and the first to third lines are different from each other or different from each other. If there is no intersection point for the point of the line, it is identified as a circular first shape by the electronic device 100, and conversely, if the first line to the third line have an intersection point with another line or point of another line , identified by the electronic device 100 as the second X-shaped shape.

4 and 5 are exemplary views of a first shape according to an embodiment of the present disclosure.

As shown in FIGS. 4 and 5 , when the shape is the first shape, the electronic device 100 determines the first distance and the distance between the third point 13 and the sixth point 16 based on the coordinates. A second distance, which is the distance between the fourth point 14 and the seventh point 17, is calculated. At this time, the second distance is the first to third lines that do not have intersections with other lines or points of other lines. 1 Obtained from the shape, which is a non-zero value.

The first distance is the distance between the elbows of the arms, the second distance is the distance between the wrists of the hands, and if the second distance is shorter than the first distance, the second line and the third line are bent toward the outside of the body of the person object, The first to third lines form a circular first shape.

Accordingly, in performing step S240, the electronic device 100 may identify positive motion when the second distance is less than the first distance and the second distance is less than a certain distance.

As an example, when the electronic device 100 identifies the first shape, but the second distance is less than the first distance, but the second distance is less than a certain distance, the electronic device 100 displays the left hand and the right hand on the image. Each hand is identified, and according to the direction in which each identified hand is directed (the direction from the wrist to the fingertip or the direction from the wrist to the end of the fist), the first vector for the direction in which the left hand is directed and the right hand are A second vector for the heading direction can be identified.

In this case, the vector is a vector having a point on the wrist as a starting point and a fingertip as an ending point. Accordingly, when either of the first vector and the second vector is not obtained, the electronic device 100 converts the corresponding motion to the user's motion. It is judged as an invalid motion with no intention and ignored.

Additionally, in the electronic device 100, when the first vector and the second vector are in an outward direction of the first shape, the first vector and the second vector intersect each other, or when each vector extends from an end point, an intersection exists. , identify positive motion.

On the other hand, when the first vector and the second vector are in the inward direction of the first shape, the electronic device 100 determines the corresponding motion as an invalid motion in which the user's intention does not exist and ignores it.

In addition, the electronic device 100 identifies positive motion when the first vector and the second vector are opposite to each other, and when the first vector and the second vector are offset when each vector of the first shape extends from the end point. do.

Additionally, in the electronic device 100, when either one of the first vector and the second vector is in the inner direction of the first shape, the first vector and the second vector are opposite to each other, but the first vector and the second vector are constant. When maintaining a horizontal state less than the distance, if the difference between the vertical axis coordinate value of the fourth point 14 and the vertical axis coordinate value of the seventh point 17 is less than a predetermined value, positive motion is identified.

6 and 7 are exemplary views of a second shape according to an embodiment of the present disclosure.

Meanwhile, when the electronic device 100 identifies a shape formed by the first to third lines, as shown in FIGS. 6 and 7 , when the shape is a preset second shape, the first to third lines Among the three lines, it is determined whether there is an intersection with another line.

If there is no intersection, the electronic device 100 determines whether there is one intersection by the second line and the third line, and if there is one intersection by the second and third lines, It may be determined whether the intersection point coincides with at least one of the fourth point 14 and the seventh point 17 .

Accordingly, in performing step S240, the electronic device 100 determines whether the intersection point is inconsistent with respect to the second shape, the fourth point 14 and the seventh point 17, or the intersection point is the fourth point 14. ) and the seventh point 17, a negative motion is identified.

As an example, in identifying a shape formed by the first to third lines, the electronic device 100 may generate a fourth line having the second point 12 and the fifth point 15 as vertexes. there is.

Accordingly, in performing step S240, the electronic device 100 determines whether the intersection point is inconsistent with respect to the second shape, the fourth point 14 and the seventh point 17, or the intersection point is the fourth point 14. ) and the seventh point 17, it is possible to calculate an angle of at least one of the second line and the third line based on the fourth line.

The electronic device 100 identifies an illegal motion when all of the at least one calculated angle is included in the critical angle range, but when any one of the at least one calculated angle is not included in the critical angle range, the identified second motion It is possible to identify an addition (+, a command indicating addition) word by re-identifying the form as being +-type according to the third form rather than X-type.

Meanwhile, in the control method of the electronic device 100 according to an aspect of the present invention, when the electronic device 100 obtains a user command (positive word) for a positive motion, the vertical axis with respect to the fourth point 14 A first difference value that is a difference value between the coordinate value and the vertical axis coordinate value for the seventh point 17 may be calculated.

Accordingly, when the first difference value exceeds a certain value, the electronic device 100 determines that a problem has occurred regarding the bending angle of the elbow and the height of the shoulder during an operation using the arm, and the ordinate coordinate of the second point 12 A second difference value, which is a difference between the value and the ordinate value of the fifth point 15, is calculated.

At this time, if the second value, which is the height difference between both shoulders, exceeds a certain value, since there is a joint problem with one shoulder, the electronic device 100 determines the ordinate value of the second point 12 and the fifth point ( Among the ordinate coordinate values of 15), by identifying a point having a small value, which is a ordinate coordinate value of a joint estimated to be atrophied due to a problem, the identified small value among the left shoulder and right shoulder of the person object 1 For a shoulder corresponding to a point having , it can be obtained as a problem joint.

On the other hand, if the second value is less than or equal to a certain value, there is a joint problem with one elbow, and therefore, the electronic device 100 selects among the ordinate coordinate values of the second point 12 and the ordinate coordinate value of the fifth point 15. , By identifying a point with a large value, which is the ordinate value of a joint estimated to be not easily folded, to the elbow in the direction corresponding to the point with the identified large value among the left elbow and right elbow of the person object 1. For that, it can be obtained as a problem joint.

In addition, in the control method of the electronic device 100 of the present invention, when the electronic device 100 obtains a diagnosis request from the user corresponding to the person object 1, at least one problem joint pre-registered for the user identify

Accordingly, the electronic device 100 calculates the number of times each problem joint has been acquired as a problem joint, and if at least one diagnosis joint acquired with a number exceeding a threshold number among the problem joints is identified, the diagnosis joint is determined as the problem joint. The acquired image may be acquired as a diagnostic image.

Thereafter, the electronic device 100 visualizes the points for each of the diagnosed joint and at least one joint adjacent to the diagnosed joint and matches them to the diagnostic image, thereby providing the user with a diagnostic image in which the points are matched, so that the problem repeatedly occurs. Allows the user to directly check the movement of the joint.

As an embodiment, when the diagnosed joint corresponds to the elbow, the electronic device 100 identifies two lines that share the point for the diagnosed joint as a vertex, and the identified two lines form an angle with respect to the inner direction of the arm. , based on a plurality of diagnostic images, it is possible to obtain a minimum value and a maximum value.

Accordingly, in providing the diagnostic image to the user, the electronic device 100 outputs the amount of angular change as a graph and provides it to the user, obtains a folding range based on the minimum and maximum values, and sets a pre-registered weight to the folding range. The applied safety range can be calculated and provided together with the amount of angular change.

For example, the weight is a value that reduces the range so as not to strain the arm within the angular range of the arm that can be folded with respect to the diagnostic joint, and the electronic device 100 visualizes and provides a safe range for joint motion in the image. By doing so, the user is instructed to pay attention to the movement of the body based on the visualized numerical value.

At this time, the electronic device 100 obtains a type of exercise that exceeds the safe range based on the safe range and provides it to the user to prevent injury caused by the user performing the corresponding type of exercise. It is possible.

Steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented in a software module executed by hardware, or implemented by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any form of computer readable recording medium well known in the art to which the present invention pertains.

Also, different embodiments of the present invention may complement each other or be combined.

Components of the present invention may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. Components of the present invention may be implemented as software programming or software elements, and similarly, embodiments may include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs, such as C, C++ , Java (Java), assembler (assembler), may be implemented in a programming or scripting language such as Python (Python). Functional aspects may be implemented in an algorithm running on one or more processors.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features of the present invention. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: electronic device
110: memory
120: camera
120: display
130: processor

Claims (7)

In the control method of an electronic device,
identifying, by the electronic device, a person object from one image;
matching, by the electronic device, a plurality of points with respect to the person object;
tracking, by the electronic device, a motion of the person object within the image based on the plurality of points; and
Identifying, by the electronic device, one of positive motion and negative motion based on a result of the tracking;
The step of matching the plurality of points,
identifying a face of the person object;
identifying a neck, a left shoulder, a left elbow, a left wrist, a right shoulder, a right elbow, and a right wrist based on the face;
1st point on the neck, 2nd point on the left shoulder, 3rd point on the left elbow, 4th point on the left wrist, 5th point on the right shoulder, 6th point on the right elbow, 6th point on the right wrist Matching the seventh points to each;
generating a first line for the first point, the second point, and the fifth point;
generating a second line by the second to fourth points; and
Generating a third line by the fifth to seventh points;
Tracking the motion of the person object within the image includes:
Tracking coordinates for each of the first to seventh points;
identifying a shape formed by the first line to the third line;
calculating a first distance that is a distance between the third point and the sixth point based on coordinates when the shape is a preset first shape; and
Calculating a second distance, which is a distance between the fourth point and the seventh point, based on the coordinates;
The step of identifying any one of positive motion and negative motion based on the result of the tracking,
If the second distance is less than the first distance and the second distance is less than a constant distance, identify positive motion;
The step of identifying the shape formed by the first line to the third line,
determining whether there is an intersection with another line among the first line to the third line when the shape is a preset second shape;
if no intersection exists, determining whether there is one intersection between the second line and the third line; and
When there is one intersection point by the second line and the third line, determining whether the intersection point coincides with at least one of the fourth point and the seventh point; including,
The step of identifying any one of positive motion and negative motion based on the result of the tracking,
When the intersection point is inconsistent with each of the fourth point and the seventh point, illegal motion is identified. delete delete delete According to claim 1,
The step of identifying any one of positive motion and negative motion based on the result of the tracking,
A control method of an electronic device, comprising: obtaining a user command for the identified motion if any one of the positive motion and the negative motion is identified and the motion is maintained for a predetermined time from an initially identified time point. According to claim 1,
The control method of the electronic device,
calculating, by the electronic device, a first difference value that is a difference between a ordinate coordinate value for the fourth point and a ordinate coordinate value for the seventh point when a user command for positive motion is acquired;
calculating, by the electronic device, a second difference value, which is a difference between the vertical axis coordinate value of the second point and the vertical axis coordinate value of the fifth point, when the first difference value exceeds a predetermined value;
identifying, by the electronic device, a point having a smaller value among the vertical axis coordinate value of the second point and the vertical axis coordinate value of the fifth point, when the second difference value exceeds a predetermined value; and
Acquiring, by the electronic device, a shoulder corresponding to a point having the identified small value among the left shoulder and the right shoulder of the person object as a problem joint;
In the step of calculating the second difference value,
identifying a point having a larger value among the ordinate coordinate value of the second point and the ordinate coordinate value of the fifth point when the second difference value is less than or equal to a predetermined value; and
Acquiring, among the left elbow and right elbow of the person object, an elbow in a direction corresponding to the point having the identified large value as a problem joint. According to claim 6,
The control method of the electronic device,
obtaining, by the electronic device, a diagnosis request from a user corresponding to the person object;
identifying, by the electronic device, at least one problem joint pre-registered for the user;
calculating, by the electronic device, the number of times acquired as a problem joint for each of the problem joints;
acquiring, by the electronic device, an image obtained as a problem joint as a diagnosis image, when at least one diagnosis joint acquired at a number exceeding a threshold number is identified among the problem joints;
matching, by the electronic device, points for the diagnosis joint and at least one joint adjacent to the diagnosis joint with respect to the diagnosis image; and
The control method of the electronic device, comprising: providing, by the electronic device, a diagnostic image matched with the points to the user.

Images