KR20210075451A - Electronic Device, Method, and System for Diagnosing Musculoskeletal Symptoms - Google Patents

Abstract

Provided, in the present invention, is a system for diagnosing musculoskeletal symptoms comprising: a plurality of image measuring devices disposed at different angles, and acquiring image data for continuously changing postures of a subject; and an electronic device that receives the image data from the image measuring device, extracts feature points corresponding to the designated skeletal joints from the image data, and calculates a score for a musculoskeletal symptom using the extracted feature points. Therefore, the present invention is capable of allowing a degree of the musculoskeletal symptom to be precisely calculated.

Description

Electronic Device, Method, and System for Diagnosing Musculoskeletal Symptoms

Embodiments of the present invention relate to electronic devices, methods, and systems for diagnosing musculoskeletal symptoms.

Workers in heavy industry, manufacturing and distribution industries are more likely to develop musculoskeletal disorders due to repetitive movements. Therefore, it is necessary to manage the health of the musculoskeletal system, such as prevention of musculoskeletal diseases and safety and health education of workers in the workplace. For example, it may be necessary to continuously monitor and evaluate the exercise performance of individual workers through corporate health welfare projects, etc., and provide exercise prescriptions and programs according to the evaluation results. However, it is difficult for a small number of exercise therapy experts to monitor and evaluate all the numerous workers in the workplace for a limited period of time. Therefore, it is difficult for a small number of exercise therapists to select high-risk workers for musculoskeletal disorders who urgently need exercise treatment among many workers, and a lot of time and effort is consumed.

In addition, X-ray or computed tomography (CT), which is one of the existing methods for diagnosing musculoskeletal disorders, is costly and time-consuming, and requires a professional medical team for diagnosis. Therefore, X-rays or CT scans may also be inappropriate for monitoring the exercise performance of a large number of workers at the workplace or monitoring the musculoskeletal health of an unspecified number of subjects. Accordingly, there is a need for a technology that allows an unspecified number of subjects to more conveniently monitor the health of the musculoskeletal system and receive information on the prognosis and prediction of musculoskeletal diseases.

Meanwhile, due to the rapid technological development in the fields of big data, machine learning, and artificial intelligence in recent years, research and development of diagnostic technology linked with artificial intelligence is increasing in the healthcare field as well. For example, there is an increasing interest in self-diagnosis services that can be used directly by general consumers without a doctor's prescription through the connection of smartphones and artificial intelligence with health care. This approach is also required in the field of musculoskeletal health.

Since musculoskeletal disorders are degenerative diseases, early detection is important, so an unspecified number of people, including workers in the workplace, can easily monitor musculoskeletal health, receive information on the prognosis and prediction of musculoskeletal disorders, or self-diagnose musculoskeletal health. The development of technology is required.

The present invention has been devised to improve the above problems, and an object of the present invention is to provide an electronic device, method, and system for diagnosing musculoskeletal symptoms.

In particular, it is an object of the present invention to provide an electronic device, method, and system for allowing consumers to easily self-diagnose musculoskeletal health without a medical professional or an exercise therapist.

In addition, based on the image data and artificial intelligence learning model for various and continuous postures of the user, the degree of musculoskeletal symptoms can be precisely calculated, and the results for multiple musculoskeletal disorders can be calculated at once. , methods, and systems.

However, these problems are exemplary, and the scope of the present invention is not limited thereto.

A system for diagnosing musculoskeletal symptoms according to an embodiment of the present invention includes: a plurality of image measurement devices arranged at different angles and acquiring image data for continuously changing postures of a subject; and an electronic device that receives the image data from the image measurement device, extracts feature points corresponding to designated skeletal joints from the image data, and calculates a score for musculoskeletal symptoms using the extracted feature points. can

According to an embodiment, the continuously changing posture of the subject may include a static posture and a dynamic posture.

According to an embodiment, the image data may include RGB data and depth data.

According to an embodiment, the electronic device generates skeletal data representing a shape of a skeletal shape of the subject by using the RGB data and the depth data, and corresponds to the designated skeletal joints from the generated skeletal data. feature points can be extracted.

According to an embodiment, the electronic device may generate and store 3D coordinate values of feature points corresponding to the designated skeletal joints from the skeletal data.

According to an embodiment, the electronic device may store 3D coordinate values of the feature points that change continuously in response to the continuously changing posture of the subject.

According to an embodiment, the electronic device may calculate a score for a plurality of musculoskeletal symptoms including a first musculoskeletal system symptom and a second musculoskeletal system symptom by using the three-dimensional coordinate values of the continuously changing feature points. .

According to an embodiment, the plurality of musculoskeletal symptoms may include two or more of turtle neck, valgus, varus, and shoulder asymmetry.

According to an embodiment, the electronic device may select first feature points necessary to calculate a score for the first musculoskeletal system symptom and second feature points necessary to calculate a score for the second musculoskeletal system symptom from among the feature points. can be identified.

According to an embodiment, the electronic device calculates the score for the first section and the first musculoskeletal symptom to be used for calculating the score for the first musculoskeletal symptom from among the data representing the feature points that change continuously with time. A second interval to be used for calculation may be identified.

According to an embodiment, the electronic device is configured to identify the first section corresponding to the first posture and the second section corresponding to the second posture based on the posture indicated by the feature points that change continuously with time. can

According to one embodiment, the characteristic points are, pelvis, lumbar spine, thoracic spine, neck, left clavicle, left shoulder, left elbow, left wrist, right clavicle, right shoulder, right elbow, right wrist, left hip, left knee, left ankle, It may include a plurality of feature points selected from feature points corresponding to the left foot, the right hip joint, the right knee, the right ankle, the right foot, the head, the nose, the left eye, the left ear, the right eye, and the right ear.

According to an embodiment, the extracted feature points include a first feature point, a second feature point, a third feature point, and a fourth feature point, and in the electronic device, a first straight line passing through the first feature point and the second feature point and an angle formed by a second straight line passing through the third feature point and the fourth feature point may be calculated, and a score for the musculoskeletal system symptom may be calculated based on the calculated angle.

According to an exemplary embodiment, the extracted feature points include a first feature point and a second feature point, and the electronic device includes the first feature point and the second feature point that change continuously in response to the continuously changing posture of the subject. 3D coordinate values may be generated, and a score for the musculoskeletal system symptom may be calculated based on the elevation change of the first feature point and the elevation change of the second feature point.

An electronic device for diagnosing musculoskeletal symptoms according to an embodiment of the present invention includes a communication unit configured to receive image data from a plurality of image measurement devices arranged at different angles and acquiring image data for continuously changing postures of a subject ; and a processor for extracting feature points corresponding to designated skeletal joints from the image data and calculating a score for musculoskeletal symptoms using the extracted feature points.

A method for diagnosing musculoskeletal symptoms according to an embodiment of the present invention includes: acquiring image data on a continuously changing posture of a subject through a plurality of image measuring devices arranged at different angles; the image data It may include extracting feature points corresponding to designated skeletal joints from the skeletal joint, and calculating a score for musculoskeletal symptoms using the extracted feature points.

A non-transitory computer-readable storage medium according to an embodiment of the present invention is an image of a continuously changing posture of a subject and arranged at different angles An operation of receiving the image data from a plurality of image measurement devices for acquiring data, an operation of extracting feature points corresponding to designated skeletal joints from the image data, and a score for musculoskeletal system symptoms using the extracted feature points One or more programs for executing the calculation operation may be stored.

Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to an embodiment of the present invention made as described above,

The degree of musculoskeletal symptoms can be precisely calculated based on the image data and artificial intelligence learning model for various and continuous postures of the user.

In addition, based on the image data and the artificial intelligence learning model for various and continuous postures of the user, it is possible to calculate the results for a plurality of musculoskeletal disorders at once.

Of course, the scope of the present invention is not limited by these effects.

1 is an example of a functional configuration of a system for diagnosing musculoskeletal symptoms according to an embodiment of the present invention.
2 is an example of an environment of a system for diagnosing musculoskeletal symptoms according to an embodiment of the present invention.
3 is an example of a signal flow of an electronic device and an image measuring device for diagnosing musculoskeletal symptoms according to an embodiment of the present invention.
4 illustrates an example of a plurality of feature points extracted by the electronic device 100 for diagnosing musculoskeletal symptoms according to an embodiment of the present invention.
5 is an example of an operation of the electronic device 100 for diagnosing musculoskeletal symptoms according to an embodiment of the present invention.
6 is an example of an operation of the electronic device 100 for diagnosing musculoskeletal symptoms according to an embodiment of the present invention.
7 shows an example of a situation for diagnosing turtle neck among musculoskeletal symptoms according to an embodiment of the present invention.
8 is an example of an operation of the electronic device 100 for diagnosing musculoskeletal symptoms according to an embodiment of the present invention.
9 shows an example of a situation for diagnosing valgus and varus among musculoskeletal symptoms according to an embodiment of the present invention.
10 shows an example of a situation for diagnosing shoulder asymmetry among musculoskeletal symptoms according to an embodiment of the present invention.
11 is an example of an operation of the electronic device 100 for diagnosing musculoskeletal symptoms according to an embodiment of the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and a method of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when described with reference to the drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted. .

In the following embodiments, terms such as first, second, etc. are used for the purpose of distinguishing one component from another, not in a limiting sense.

In the following examples, the singular expression includes the plural expression unless the context clearly dictates otherwise.

In the following embodiments, terms such as include or have means that the features or components described in the specification are present, and the possibility of adding one or more other features or components is not excluded in advance.

In the drawings, the size of the components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily indicated for convenience of description, the present invention is not necessarily limited to the illustrated bar.

In the following embodiments, when it is said that a part, such as a region, component, part, block, module, etc., is on or on another part, not only when it is directly on the other part, but also another region, component, part in the middle , blocks, modules, etc. are included.

In the following embodiments, when a region, component, part, block, module, etc. are connected, the region, component, part, block, module, etc., as well as the case where the region, component, part, block, and modules are directly connected It includes cases in which other regions, components, parts, blocks, and modules are interposed and indirectly connected between them.

Hereinafter, in this document, 'musculoskeletal symptoms' may refer to 'symptoms of musculoskeletal disorders' or 'musculoskeletal disorders'.

1 is an example of a functional configuration of a system for diagnosing musculoskeletal symptoms according to an embodiment of the present invention. 2 is an example of an environment of a system for diagnosing musculoskeletal symptoms according to an embodiment of the present invention.

1 and 2 , a system for diagnosing musculoskeletal symptoms according to an embodiment of the present invention may include an image measuring device 200 and an electronic device 100 connected to the image measuring device 200 . can The image measuring apparatus 200 and the electronic device 100 may be connected through a wired or wireless communication channel.

The image measuring apparatus 200 may acquire an image of the subject E, that is, an image. According to an embodiment, the image measuring device 200 is a place for monitoring the health of workers in the workplace, or a place that provides a health care service where an unspecified number of consumers can self-diagnose and receive information about musculoskeletal health. can be installed on However, this is only an example, and the present invention is not limited thereto.

The image measuring device 200 may be a device for acquiring an image, such as a camera, and may be installed in plurality. For example, two or more image measuring devices 200 may be installed to photograph one subject E from different angles.

According to an embodiment, the electronic device 100 may be a computer device connected to the image measuring device 200 by wire or wirelessly. For example, the electronic device 100 may be a computer device installed together with the image measuring device 200 at a location providing a musculoskeletal health diagnosis service. For another example, the electronic device 100 may be installed in a different location from the image measuring device 200 , and may be a server or a management device for providing a musculoskeletal system diagnosis service.

According to another embodiment, the electronic device 100 may include a smart phone and a tablet PC connected to the image measuring device 200 through a server (not shown). In this case, the electronic device 100 may be a user personal terminal. However, this is only an example, and the present invention is not limited thereto.

According to an embodiment, the image measuring apparatus 200 may include a sensor unit 210 , a communication unit 230 , and a memory 250 . However, the present invention is not limited thereto, and at least one of these components may be omitted or one or more other components may be added to the image measuring apparatus 200 . The image measuring apparatus 200 may further include a battery (not shown). The image measuring apparatus 200 may further include, for example, an image signal processor (not shown) for various image processing.

The sensor unit 210 includes an image sensor, and may acquire an image of the subject E. For example, the sensor unit 210 may acquire an image corresponding to the subject by converting light transmitted through the lens assembly into an electrical signal.

The image sensor included in the sensor unit 210 may be implemented as, for example, a charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

The sensor unit 210 may continuously (eg, in a predetermined short period) acquire a series image or a stream image of the subject E. This may correspond to the operation of the image measuring device 200 taking a video of the subject E. The image acquired through the sensor unit 210 is transmitted to the electronic device 100 as a digital signal, and may be used to extract a feature point corresponding to the skeletal joint of the subject E, and the subject using the feature point (E) can be used to identify one or more musculoskeletal symptoms.

The sensor unit 210 may include, for example, an RGB sensor, a black and white (BW) sensor, an infrared (IR) sensor, and a depth sensor. According to an embodiment, data obtained through the RGB sensor and the depth sensor may be used to identify a skeleton (or a shape of a limb) of the subject E from a continuous series of images of the subject E. The identification of the shape of the skeleton or limb may be used to extract feature points corresponding to the skeletal joint of the subject E. Here, the identification of the shape of the limb refers to identification of the shape or posture of the entire skeleton including the limbs as well as the limbs.

The sensor unit 210 may remove or filter interference or noise from, for example, an analog signal (eg, an electrical signal) of an acquired image. The filtered analog signal may be converted into a digital signal and transmitted to the electronic device 100 .

According to an embodiment, the sensor unit 210 may further include an inertial measurement unit (IMU).

An inertial measurement unit (IMU) can use a 3-axis accelerometer to sense linear acceleration in the 3-axis direction, and a 3-axis accelerometer (e.g. a gyroscope) can be used to detect the angular velocity or angular acceleration in the 3-axis direction. . The inertial measurement apparatus can measure acceleration in the traveling direction, lateral direction, and height direction, and angular velocities of roll, pitch, and yaw by using the three-axis accelerometer and the three-axis accelerometer.

The sensor unit 210 may increase the accuracy of detecting position data and tracking rotation of an object (eg, the subject E) through the sensing or measurement using the inertial measurement device. For example, the inertial measurement apparatus may minimize an error between a positioning value measured by the positioning system and an actual position value of the object.

Since the sensor unit 210 includes the inertial measurement unit (IMU), it is possible to obtain an accurate image of the posture or movement of the subject E even when the image measurement apparatus 200 is shaken.

According to an embodiment, the image measuring apparatus 200 (eg, the sensor unit 210 ) may include an image signal processor (not shown) in order to remove interference or noise from the obtained image signal.

The image signal processor (not shown) may perform one or more image processing on an image acquired through the sensor unit 210 or an image stored in the memory 250 . The one or more image processing operations may include, for example, generation of a depth map, 3D modeling, feature point extraction, or image compensation (eg, noise reduction, resolution adjustment, brightness adjustment), and the like.

For example, the image signal processor may perform control (eg, exposure time control, etc.) on the sensor unit 210 .

The image processed by the image signal processor may be provided to another component (eg, the memory 250 or the electronic device 100 ).

However, the present invention is not limited thereto, and image processing performed by the image signal processor may be performed by the electronic device 100 (eg, the processor 110 of the electronic device 100 ).

The memory 250 may temporarily store at least a portion of the image acquired through the sensor unit 210 for an image processing operation. For example, the memory 250 may at least temporarily store image data in order to transmit stream data for an image acquired through the sensor unit 210 to the electronic device 100 .

The communication unit 230 establishes a wired or wireless communication channel between the image measurement apparatus 200 and the electronic device 100 (eg, the communication unit 130 of the electronic device 100), and performs communication through the established communication channel. can support The communication unit 230 may transmit a digital signal converted from an analog signal of the acquired image to the electronic device 100 as image data.

The electronic device 100 may include, for example, a computer device, a server, a portable communication device, a tablet PC, or a smart phone, but is not limited thereto.

The electronic device 100 may include a processor 110 , a communication unit 130 , and a memory 150 . The memory 150 identifies the musculoskeletal symptoms of the subject (E) from the image data of the subject (E) received from the image measurement device 200, and for calculating the degree of the musculoskeletal symptoms (eg, score, risk, etc.) The program 160 may be stored. However, the present invention is not limited thereto, and at least one of these components may be omitted, or one or more other components (eg, a display device such as a display, an input device such as a mouse, a keyboard, and a touch screen) may be added.

The communication unit 130 may support establishment of a wired or wireless communication channel with the communication unit 230 of the image measuring apparatus 200 and performing communication through the established communication channel. The communication unit 130 may receive image data of the subject E from the image measuring apparatus 200 .

Processor 110 may include, for example, one or more processors. The processor 110 may execute software or a program 160 to perform various data processing or operations. The data processing or calculation is performed by inputting series image data continuously acquired with respect to the static or dynamic posture of the subject, and identifying the skeleton of the subject (E) included in the series image data, that is, the shape of the extremities, and identification Extracting a feature point corresponding to the skeletal joint of the subject (E) using the shape of the extremity, and using the feature point to identify one or more musculoskeletal symptoms of the subject (E) It may include data processing or calculation.

The processor 110 may load a command or data into the memory 150 (eg, a volatile memory), process the stored command or data, and store the result data in the memory 150 (eg, a non-volatile memory).

The memory 150 may store various data used by at least one component (eg, the processor 110 ) of the electronic device 100 . The data may include, for example, input data or output data for software (eg, the program 160 ), a learning model, and instructions related thereto. The memory 150 may include a volatile memory or a non-volatile memory. According to an embodiment of the present invention, the memory 150 extracts feature points corresponding to the skeletal joints of the subject (E) included in the image data, and uses the feature points to detect one or more musculoskeletal symptoms of the subject (E) It may store software for identification, a program (eg, program 160 ), a learning model, or an algorithm.

The program 160 may include one or more programs. The program 160 may include a learning model learned by artificial intelligence.

For example, the program 160 is a three-dimensional coordinate of a plurality of feature points corresponding to the skeletal joint of the subject E from continuous image data (eg, series image data) for the static or dynamic posture of the subject E. It may include a learning model trained to extract .

The program 160 (eg, artificial intelligence learning model) may generate or obtain a movement of the three-dimensional coordinates of a plurality of feature points from the continuous image data of the subject E, and use the three-dimensional coordinates of the plurality of feature points The severity of multiple musculoskeletal symptoms (eg score, risk, etc.) can be calculated.

Referring to FIG. 2 , in the system for diagnosing musculoskeletal symptoms according to an embodiment, there may be a plurality of image measuring apparatuses 200 .

The subject E may take a plurality of postures (eg, a first posture, a second posture, etc.) among the plurality of image measurement apparatuses 200 . For example, the plurality of postures may include a static posture and a dynamic posture. The static posture may include a sitting posture and a standing posture, and the dynamic posture may include a walking posture that is a walking motion.

The subject E may continuously take a plurality of postures between the image measuring apparatuses 200 . For example, the subject E may continuously perform an operation of sitting on a chair, standing up, walking to the front, and then walking back. The continuous operation may include both the static posture and the dynamic posture described above.

The plurality of image measuring apparatuses 200 may photograph a plurality of postures of the subject E as described above at different angles. For example, the first image measuring device may be installed in front of the subject (E), and the second image measuring device is a reference point (eg, the starting point of the posture of the subject (E)) in front of and behind the subject (E). With respect to the direction, it can be installed at a 45 degree angle.

In this case, the first image measuring apparatus may photograph the front or rear surface of the subject E while the subject E performs a plurality of postures. The second image measuring device may photograph the front side or the rear side of the subject E while the subject E performs a plurality of postures.

However, the present invention is not limited thereto, and according to an embodiment, the plurality of image measuring apparatuses 200 may further include, for example, a third image measuring apparatus. For example, the first, second, and third image measuring apparatuses may be installed at intervals of 120 degrees around a reference point of photographing (eg, a starting point of the posture of the subject E). However, this is only an example, and the present invention is not limited thereto.

The electronic device 100 may receive image data corresponding to the photographed image of the subject E from each of the plurality of image measurement apparatuses 200 .

That is, the electronic device 100 receives the first image data photographed by the first image measurement apparatus from the first image measurement apparatus, and receives the second image data corresponding to the image acquired by the second image measurement apparatus from the second image measurement apparatus. Image data can be received.

The first image data may include RGB data and depth data for a plurality of consecutive postures of the subject E photographed by the first image measuring apparatus. The second image data may also include RGB data and depth data for a plurality of consecutive postures of the subject E photographed by the second image measuring apparatus.

For example, the processor 110 of the electronic device 100 may synchronize the first image data received from the first image measurement apparatus and the second image data received from the second image measurement apparatus. For example, the parameters may be adjusted so that the first image measuring device and the second image measuring device are synchronized with each other.

The processor 110 of the electronic device 100 may identify the skeleton (or the shape of a limb) of the subject E from the first image data and the second image data through the program 160 . For example, the processor 110 identifies the shape or posture of the subject E using RGB data and depth data included in the first image data and the second image data, and a skeleton including the extremities of the subject E The movement of the joint can be identified.

The processor 110 may extract a plurality of feature points representing the skeletal joint of the subject E by using the RGB data and depth data included in the first image data and the second image data by using the learning model. .

The processor 110 extracts the three-dimensional coordinates of the plurality of feature points by using image data taken from multiple angles by a plurality of image measuring devices 200 including a first image measuring device and a second image measuring device. can do. The processor 110 uses image data continuously acquired while the subject E takes a plurality of consecutive postures (eg, a sitting posture, a standing posture, and a walking posture) of the three-dimensional coordinates of the plurality of feature points. It is possible to acquire movement according to time. That is, the processor 110 may generate a path formed by the plurality of feature points in three dimensions according to time. A path formed by the plurality of feature points in three dimensions may correspond to a continuous posture of the subject E.

According to an embodiment of the present invention, by photographing a plurality of continuous postures including a static posture and a dynamic posture of the subject (E), the musculoskeletal symptoms and the degree (or risk) of the subject (E) can be more accurately identified. can As in the conventional method, when the subject (E) takes a specific posture in front of an exercise therapist, or when an X-ray or CT scan of a specific posture is performed, the subject (E) may unconsciously try to take the correct posture, This is because it was difficult to accurately diagnose the severity of musculoskeletal symptoms. On the other hand, according to an embodiment of the present invention, the subject E performs a plurality of continuous postures including a static posture and a dynamic posture (eg, a walking motion), and performs the plurality of image measurement devices 200 at various angles. ) are taken with Through this, it is possible to obtain a natural posture image of the subject E, and accordingly, the musculoskeletal symptoms and the degree (or risk) of the subject E can be accurately identified.

3 is an example of a signal flow of an electronic device and an image measuring device for diagnosing musculoskeletal symptoms according to an embodiment of the present invention.

Referring to FIG. 3 , the image measuring apparatus 200 may acquire an image of the subject E in S101. The image measuring device 200 may include two or more image measuring devices. The two or more image measuring devices may photograph the subject E from different angles, respectively. For example, centering on the reference point of the photographing of the subject (E), the first image measuring device may acquire an image of the subject (E) from the front of the subject (E), and the second image measuring device is the subject (E) An image of the subject (E) can be acquired at a position inclined by 45 degrees with respect to the front and rear direction As another example, the first, second, and third image measuring apparatuses may be disposed at intervals of 120 degrees to acquire an image of the subject E. However, the present invention is not limited thereto.

In S101, the subject E may take a plurality of postures including the first posture and the second posture in succession. The plurality of postures may include a static posture and a dynamic posture. The static posture may include a sitting posture and a standing posture, and the dynamic posture may include a walking posture that is a walking motion. The subject E may continuously perform, for example, a first posture of sitting on a chair, a second posture of standing up from the chair, and a third posture of walking forward and then turning around. The plurality of image measuring apparatuses 200 may successively acquire images of the first, second, and third postures of the subject E.

In S102 , the image measuring apparatus 200 may transmit image data for the acquired image to the electronic device 100 . For example, the plurality of image measurement apparatuses 200 may transmit image data for an image acquired by each image measurement apparatus 200 to the electronic apparatus 100 , respectively. Each image data may include RGB data and depth data. The electronic device 100 may receive image data from each of the plurality of image measurement devices 200 .

In S103, the electronic device 100 may extract a feature point representing a designated skeletal joint from the received image data. A plurality of feature points extracted from image data may correspond to a specific skeletal joint. The types of the plurality of feature points may be predetermined. For example, the learning model implemented in the program 160 may be trained to identify a plurality of feature points corresponding to a designated skeletal joint.

4 illustrates an example of a plurality of feature points extracted by the electronic device 100 for diagnosing musculoskeletal symptoms according to an embodiment of the present invention.

Referring to FIG. 4 , according to an embodiment of the present invention, the electronic device 100 may be set to extract feature points corresponding to 26 designated skeletal joints. 26 designated skeletal joints may be, for example, as shown in Table 1 below.

reference number skeletal joint parent skeletal joint 0 Pelvis (PELVIS) - One Lumbar (SPINE_NAVAL) Pelvis (PELVIS) 2 Thoracic (SPINE_CHEST) Lumbar (SPINE_NAVAL) 3 neck (NECK) Thoracic (SPINE_CHEST) 4 Left clavicle (CLAVICLE_LEFT) Thoracic (SPINE_CHEST) 5 Left shoulder (SHOULDER_LEFT) Left clavicle (CLAVICLE_LEFT) 6 Left Elbow (ELBOW_LEFT) Left shoulder (SHOULDER_LEFT) 7 Left wrist (WRIST_LEFT) Left Elbow (ELBOW_LEFT) 8 Right clavicle (CLAVICLE_RIGHT) Thoracic (SPINE_CHEST) 9 right shoulder (SHOULDER_RIGHT) Right clavicle (CLAVICLE_RIGHT) 10 right elbow (ELBOW_RIGHT) right shoulder (SHOULDER_RIGHT) 11 Right wrist (WRIST_RIGHT) right elbow (ELBOW_RIGHT) 12 Left hip joint (HIP_LEFT) Pelvis (PELVIS) 13 Left knee (KNEE_LEFT) Left hip joint (HIP_LEFT) 14 Left ankle (ANKLE_LEFT) Left knee (KNEE_LEFT) 15 Left foot (FOOT_LEFT) Left ankle (ANKLE_LEFT) 16 Right hip joint (HIP_RIGHT) Pelvis (PELVIS) 17 Right knee (KNEE_RIGHT) Right hip joint (HIP_RIGHT) 18 Right ankle (ANKLE_RIGHT) Right knee (KNEE_RIGHT) 19 Right foot (FOOT_RIGHT) Right ankle (ANKLE_RIGHT) 20 HEAD neck (NECK) 21 Nose HEAD 22 Left eye (EYE_LEFT) HEAD 23 Left ear (EAR_LEFT) HEAD 24 Right eye (EYE_RIGHT) HEAD 25 Right ear (EAR_RIGHT) HEAD

The arrows in FIG. 4 may indicate that the hierarchy extends from the parent skeletal joint to each skeletal joint. However, the plurality of feature points extracted by the electronic device 100 is not limited to FIGS. 4 and 4 , and for example, some feature points may be omitted or other feature points may be further added.

The electronic device 100 may identify the skeleton of the subject E, that is, the shape of a limb, from RGB data and depth data included in each of the image data received from the plurality of image measurement apparatuses 200 . Here, the identification of the shape of the limb refers to identification of the shape or posture of the entire skeleton including the limbs as well as the limbs. Also, identifying the shape of the skeleton or the shape of the limb may include generating skeletal data indicative of the shape of the skeleton.

The electronic device 100 may extract feature points corresponding to a plurality (eg, 26) of skeletal joints as shown in Table 4 from the shape of the skeleton or limb. Here, the operation of extracting the feature points may refer to an operation of acquiring and storing three-dimensional coordinates of each feature point corresponding to the skeletal joint.

Accordingly, the electronic device 100 may acquire the three-dimensional movement of the feature points by using the image data received from the plurality of image measurement apparatuses 200 . In other words, it is possible to generate and store a change with time of the 3D coordinate value of each feature point.

In S104, the electronic device 100 may calculate a score for one or more musculoskeletal symptoms by using the feature points. In other words, the electronic device 100 may calculate a score for one or more musculoskeletal symptoms by using the movement of the three-dimensional coordinates of each feature point. The score for musculoskeletal symptoms may correspond to the severity or risk of musculoskeletal symptoms. Scores for musculoskeletal symptoms may be numeric, for example, but may also represent a scale such as normal, mild, or severe.

According to an embodiment, the electronic device 100 identifies a plurality of musculoskeletal symptoms at a time using the three-dimensional coordinates of the feature points or the movement of the three-dimensional coordinates obtained in S103, and the degree of the plurality of musculoskeletal symptoms (eg, : score, risk) can be calculated at once.

The plurality of musculoskeletal symptoms that the electronic device 100 can identify at once may include, for example, a turtle neck (or straight neck), a valgus knee, a varus knee, and shoulder asymmetry. However, this is only an example, and the present invention is not limited thereto. For example, the types of musculoskeletal symptoms that the electronic device 100 can identify using the three-dimensional coordinates of the feature points may be updated or added through learning. According to an embodiment of the present invention, since the electronic device 100 can obtain three-dimensional coordinates of feature points corresponding to skeletal joints and their movements, an algorithm for identifying other musculoskeletal symptoms using the three-dimensional coordinates is performed. Adding to the program 160 can be simple and efficient.

According to an embodiment of the present invention, the types of feature points used to identify a plurality of musculoskeletal symptoms may be different. For example, among the plurality of acquired feature points, first and second feature points may be used to identify the first musculoskeletal symptom, and third and fourth feature points may be used to identify the second musculoskeletal symptom. However, the present invention is not limited thereto.

5 is an example of an operation of the electronic device 100 for diagnosing musculoskeletal symptoms according to an embodiment of the present invention. The operations of FIG. 5 may be performed by the processor 110 through the program 160 .

Referring to FIG. 5 , in S201 , the electronic device 100 may extract a feature point from image data received from a plurality of image measuring devices 200 . This may correspond to the above-described operation S103. That is, the electronic device 100 may generate three-dimensional coordinate values of the feature points from the image data received from the plurality of image measurement apparatuses 200, and the movement of the feature points according to time (ie, three-dimensional coordinate values). change) can be obtained or created.

In detail, according to the continuous operation taken by the subject E, the characteristic points may also show movement with time. That is, the feature points may also correspond to a plurality of postures.

In S202, the electronic device 100 may calculate a score for the first musculoskeletal system symptom by using the feature points of the first posture during the movement of the feature points over time.

In S203 , the electronic device 100 may calculate a score for the second musculoskeletal system symptom by using the feature points of the second posture during the movement of the feature points over time.

The first posture and the second posture may be a stationary posture or a dynamic posture, and may partially overlap each other.

According to an embodiment, a posture required to identify different first musculoskeletal symptoms (eg, valgus, varus knee) and second musculoskeletal symptoms (eg, shoulder asymmetry) may be different from each other.

For example, in order to identify the valgus and the valgus, the feature points of the front stop posture may be used, the feature points of the lateral posture may be used to identify the turtle neck, and the walking posture, which is a dynamic posture, to identify shoulder asymmetry. The characteristic points of can be used. However, the present invention is not limited thereto, and according to an exemplary embodiment, feature points of a front still posture or a rear stand still posture may be used to identify shoulder asymmetry.

Accordingly, in S202 , the electronic device 100 may calculate a score for the first musculoskeletal system symptoms, such as valgus and varus, using, for example, feature points of the first posture, which is the front stop posture. Also, in S203 , the electronic device 100 may calculate a score for shoulder asymmetry, which is the second musculoskeletal system symptom, by using, for example, feature points of the second posture, which is the walking posture. In this case, for example, the feature points of the second posture (eg, walking posture) may include all feature points moving with time, not feature points corresponding to a single point of view. Therefore, for example, the second musculoskeletal symptom (eg, shoulder asymmetry) may be identified using feature points of the second posture (eg, walking posture) that moves with time.

S202 and S203 may be performed in order or may be performed independently of each other and in parallel.

According to an embodiment, the electronic device 100 calculates a score for a first section and a second musculoskeletal system symptom to be used to calculate a score for the first musculoskeletal system symptom from among data representing feature points that change continuously over time It is possible to identify a second interval to be used for

In this case, the electronic device 100 may identify the first section corresponding to the first posture and the second section corresponding to the second posture based on postures indicated by feature points that change continuously over time. The first interval and the second interval may correspond to a plurality of viewpoints or may correspond to a single viewpoint. That is, according to an embodiment, the electronic device 100 calculates a score for a first time point and a second musculoskeletal system symptom to be used for calculating a score for the first musculoskeletal system symptom among data representing feature points that change continuously with time. It is possible to identify a second time point to be used for

6 is an example of an operation of the electronic device 100 for diagnosing musculoskeletal symptoms according to an embodiment of the present invention. The operations of FIG. 6 may be performed by the processor 110 through the program 160 . 6 to 7 , the first musculoskeletal symptom that can be diagnosed through the lateral resting posture may include, for example, a turtle neck.

Referring to FIG. 6 , in S301 , the electronic device 100 may identify a predetermined characteristic point for diagnosing a first musculoskeletal symptom (eg, turtle neck) from image data for a lateral resting posture.

For example, among the plurality of (eg, 26) characteristic points in the lateral stop posture, predetermined characteristic points for diagnosing the first musculoskeletal symptom (eg, turtle neck) may exist. That is, among the plurality of (eg, 26) characteristic points in the lateral stop posture, not all of the plurality of characteristic points are used to diagnose the first musculoskeletal symptom (eg, turtle neck), among which the specified first and second , only 3 feature points can be used. However, this is not limited to three feature points used.

7 shows an example of a situation for diagnosing turtle neck among musculoskeletal symptoms according to an embodiment of the present invention.

Referring to FIG. 7 , the electronic device 100 includes, for example, first, second, and third feature points for diagnosing a turtle neck, a feature point corresponding to the nose 21 , a feature point corresponding to the right ear 25 , And it is possible to identify a feature point corresponding to the neck (3). For another example, a feature point corresponding to the nose 21, a feature point corresponding to the left ear 23, and a feature point corresponding to the neck 3 may be used as the first, second, and third feature points for diagnosing the turtle neck. is of course

In S302 , the electronic device 100 may calculate an angle A1 formed by the identified first, second, and third feature points.

In S303 , the electronic device 100 may acquire a score for the first musculoskeletal symptom (eg, turtle neck) according to the calculated angle. The score may be a number obtained by converting the angle A1 formed by the first, second, and third feature points according to a predetermined rule. The score may indicate a degree of risk for the first musculoskeletal symptom (eg, turtle neck).

According to an embodiment, the electronic device 100 responds to the first musculoskeletal symptom (eg, turtle neck) when the angle A1 formed by the first, second, and third feature points is less than the first angle, as in situation C2. It can output that it is normal. In addition, the electronic device 100 outputs that the first musculoskeletal symptom (eg, turtle neck) is mild when the angle A1 formed by the first, second, and third feature points is greater than or equal to the first angle and less than the second angle as in situation C1. and, if the angle A1 formed by the first, second, and third feature points is equal to or greater than the second angle, it may be output that the first musculoskeletal symptom (eg, turtle neck) is severe.

The calculated score or risk for the first musculoskeletal system symptom may be output through the display device of the electronic device 100 . For example, the score or risk for the first musculoskeletal system symptom may be calculated by the server and displayed through the application of the subject E's smartphone. The application may guide an appropriate exercise therapy, for example, according to the calculated score or risk for the first musculoskeletal system symptom.

8 is an example of an operation of the electronic device 100 for diagnosing musculoskeletal symptoms according to an embodiment of the present invention. The operations of FIG. 8 may be performed by the processor 110 through the program 160 . 8 to 10 , the second musculoskeletal system symptom that can be diagnosed through the frontal still posture may include, for example, valgus, varus, and shoulder asymmetry.

Referring to FIG. 8 , in S401 , the electronic device 100 may identify a predetermined characteristic point for diagnosing the second musculoskeletal system symptom (eg, valgus, varus, and shoulder asymmetry) from the image data for the front still posture. The front rest posture may include, for example, a front rest posture and/or a rear rest posture.

For example, among a plurality of (eg, 26) characteristic points in the front stop posture, predetermined characteristic points for diagnosing the second musculoskeletal system symptom (eg, valgus, varus, shoulder asymmetry) may exist. That is, among the plurality of (eg, 26) characteristic points in the frontal still posture, not all of the plurality of characteristic points are used to diagnose the second musculoskeletal symptom (eg, valgus, varus, shoulder asymmetry), among which The first, second, third, and fourth feature points may be used. However, this does not limit the number of feature points used to be four.

9 shows an example of a situation for diagnosing valgus and varus among musculoskeletal symptoms according to an embodiment of the present invention.

Referring to FIG. 9 , the electronic device 100 is a feature point for diagnosing, for example, a valgus knee and a varus knee, and among a plurality (eg, 26) feature points, the left hip joint 12, the left knee 13, and the left foot. (14), the right hip joint (16), the right knee (17), the characteristic points corresponding to the right foot (18) can be identified.

10 shows an example of a situation for diagnosing shoulder asymmetry among musculoskeletal symptoms according to an embodiment of the present invention.

Referring to FIG. 10 , the electronic device 100 is a feature point for diagnosing shoulder asymmetry, for example, a left elbow 6 , a right elbow 10 , and a left shoulder from among a plurality (eg, 26) feature points. (5), the characteristic points corresponding to the right shoulder 9, the left ear 23, and the right ear 25 may be identified.

In S402 , the electronic device 100 may calculate an angle formed by a first straight line passing through the identified first and second feature points and a second straight line passing through the third and fourth feature points.

In S303 , the electronic device 100 may acquire a score for the second musculoskeletal system symptom (eg, valgus, varus, shoulder asymmetry) according to the calculated angle. The score may be a number obtained by converting the angle A1 formed by the first and second straight lines according to a predetermined rule. The score may indicate a degree of risk for the second musculoskeletal system symptom.

According to an embodiment, in the electronic device 100 , the angle formed by the first straight line passing through the left hip joint 12 and the left foot 14 and the second straight line passing through the left hip joint 12 and the left knee 13 is If it is less than the first angle, it is a valgus like C3, and if it is more than the second angle, it can output a valgus like C4. The first angle may be negative, and the second angle may be positive. The electronic device 100 may output that the angle between the first straight line and the second straight line is normal when the angle between the first straight line and the second straight line is greater than or equal to the first angle and less than the second angle.

According to another embodiment, in the electronic device 100 , the angle formed by the first straight line passing through the right hip joint 16 and the right knee 17 and the second straight line passing through the left hip joint 12 and the left knee 13 is formed. If is less than the first angle, the valgus, the second angle or more, the valgus, the first angle or more and less than the second angle, it may be output as normal.

Also, according to an embodiment, in the electronic device 100 , the angle formed by the first straight line passing through the left elbow 6 and the right elbow 10 and the second straight line passing through the left shoulder 5 and the right shoulder 9 . , and/or depending on the angle between the third straight line passing through the left ear 23 and the right ear 25 and the second straight line passing through the left shoulder 5 and the right shoulder 9, shoulder asymmetry is severe or It can output that it is mild or that it is normal like C6. According to a predetermined algorithm or equation, the electronic device 100 may output that the shoulder asymmetry is severe as the degree of inclination of the first straight line, the second straight line, and the third straight line to each other increases.

11 is an example of an operation of the electronic device 100 for diagnosing musculoskeletal symptoms according to an embodiment of the present invention. The operations of FIG. 11 may be performed by the processor 110 through the program 160 . The third musculoskeletal symptom that can be diagnosed through the dynamic walking posture of FIG. 11 may include, for example, shoulder asymmetry.

Referring to FIG. 11 , in S501 , the electronic device 100 may identify a predetermined feature point for diagnosing a third musculoskeletal symptom (eg, shoulder asymmetry) from series image data for a dynamic walking posture. The series image data may refer to image data or stream image data acquired continuously (eg, at a predetermined time interval).

For example, among a plurality of (eg, 26) characteristic points in the dynamic walking posture, predetermined characteristic points for diagnosing a third musculoskeletal symptom (eg, shoulder asymmetry) may exist. That is, among the plurality of (eg, 26) characteristic points in the dynamic walking posture, not all of the plurality of characteristic points are used to diagnose the second musculoskeletal symptom (eg, shoulder asymmetry), among which the specified first, Two feature points may be used.

According to an embodiment, as a feature point for diagnosing shoulder asymmetry, the electronic device 100 corresponds to the left wrist 7 and the right wrist 11 among a plurality (eg, 26) feature points of the dynamic walking posture. Feature points can be identified.

In S502 , the electronic device 100 may acquire a change in the position of the first feature point according to time and a change in the position of the second feature point according to time among the movement of the feature points according to time in the dynamic walking posture.

For example, in order to diagnose shoulder asymmetry, the electronic device 100 may change the position of the feature point of the left wrist 7 according to time (eg, a change in the three-dimensional coordinate value) among the movement of the feature points over time in the dynamic walking posture. ) and the position change of the characteristic point of the right wrist 11 can be obtained.

For example, in the dynamic walking posture, the elevation (eg, z coordinate) graph of the feature point of the left wrist 7 over time may be in the form of a sine wave or a cosine wave. In addition, in the dynamic walking posture, the elevation (eg, z-coordinate) graph of the characteristic point of the right wrist 11 over time may be in the form of a sine wave or a cosine wave whose phase is approximately opposite to that of the elevation graph of the characteristic point of the left wrist 7 . .

In S503, the electronic device 100 uses a change in the position of the first characteristic point (eg, the left wrist 7) and the position change of the second characteristic point (eg, the right wrist 11) to the third musculoskeletal symptom (eg, the shoulder). asymmetry). The score may be a number obtained by converting the elevation change of the first feature point and the elevation change of the second feature point according to a predetermined rule. The score may indicate a degree of risk for a third musculoskeletal system symptom (eg, turtle neck).

For example, as the difference between the height difference between the key points of the left wrist 7 (eg, the difference between the average lowest altitude and the average highest altitude) and the height difference between the key points on the right wrist 11 increases, the third musculoskeletal system It is possible to output that the symptom (eg, shoulder asymmetry) is severe, and the smaller the difference, the more normal it may be output.

For another example, the electronic device 100 outputs that the third musculoskeletal symptom (eg, shoulder asymmetry) is severe as the difference between the average height of the left wrist 7 feature point and the average height of the right wrist 11 feature point is greater, The smaller the difference, the more normal it can be output. However, the present invention is not limited thereto.

According to an embodiment of the present invention, the calculated score or risk for musculoskeletal symptoms may be output through the display device of the electronic device 100 . For example, the score or risk for the first, second, and third musculoskeletal symptoms may be calculated in the server and displayed through the application of the subject E's smartphone. The application may guide an appropriate exercise therapy, for example, according to the calculated score or risk for the first, second, and third musculoskeletal symptoms.

According to an embodiment of the present invention, the electronic device 100 may extract three-dimensional movements of feature points corresponding to a plurality of skeletal joints from a continuous motion including a plurality of postures of the subject E, and the feature points One or a plurality of musculoskeletal symptoms can be diagnosed by using the three-dimensional movements.

According to an embodiment, the electronic device 100 may identify a posture and a necessary characteristic point according to a type of a musculoskeletal symptom to be diagnosed, from the three-dimensional movement of the characteristic points.

Various embodiments of the present invention are software including one or more instructions stored in a storage medium (eg, the memory 150 or an external memory) readable by a machine (eg, the electronic device 100). (eg, the program 160). For example, the processor 110 of the electronic device 100 may call one or more instructions stored from a storage medium and execute them. This enables the electronic device 100 to perform a function according to the called commands. A storage medium readable by a device (eg, the electronic device 100 ) may be provided in the form of a non-transitory storage medium.

The method according to various embodiments of the present invention may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)), or distributed through an application store (eg Play Store™) or online (eg, can be downloaded or uploaded).

Although the present invention has been described with reference to one embodiment shown in the drawings, it will be understood that this is merely exemplary, and that those of ordinary skill in the art can make various modifications and variations therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

Claims (17)

a plurality of image measuring devices arranged at different angles and acquiring image data for continuously changing postures of the subject; and
An electronic device for receiving the image data from the image measuring device, extracting feature points corresponding to designated skeletal joints from the image data, and calculating a score for musculoskeletal symptoms using the extracted feature points;
A system for diagnosing musculoskeletal symptoms. According to claim 1,
The continuously changing posture of the subject includes a static posture and a dynamic posture,
A system for diagnosing musculoskeletal symptoms. According to claim 1,
The image data includes RGB data and depth data,
A system for diagnosing musculoskeletal symptoms. 4. The method of claim 3,
The electronic device is
using the RGB data and the depth data to generate skeleton data representing the shape of the skeleton of the subject;
Extracting feature points corresponding to the designated skeletal joints from the generated skeletal data,
A system for diagnosing musculoskeletal symptoms. 5. The method of claim 4,
The electronic device is
Generating and storing three-dimensional coordinate values of feature points corresponding to the specified skeletal joints from the skeletal data,
A system for diagnosing musculoskeletal symptoms. According to claim 1,
The electronic device is
storing three-dimensional coordinate values of the feature points that change continuously in response to the continuously changing posture of the subject;
A system for diagnosing musculoskeletal symptoms. 7. The method of claim 6,
The electronic device is
calculating scores for a plurality of musculoskeletal symptoms including a first musculoskeletal system symptom and a second musculoskeletal system symptom by using the three-dimensional coordinate values of the continuously changing feature points,
A system for diagnosing musculoskeletal symptoms. 8. The method of claim 7
The plurality of musculoskeletal symptoms are
Containing two or more of turtle neck, valgus, varus knee, and shoulder asymmetry,
A system for diagnosing musculoskeletal symptoms. 8. The method of claim 7,
The electronic device is
Among the above features
identifying first feature points necessary for calculating a score for the first musculoskeletal system symptom and second feature points necessary for calculating a score for the second musculoskeletal system symptom;
A system for diagnosing musculoskeletal symptoms. 8. The method of claim 7,
The electronic device is
Among the data representing the feature points that change continuously with time, a first section to be used for calculating a score for the first musculoskeletal symptom and a second section to be used for calculating a score for the first musculoskeletal symptom are identified,
A system for diagnosing musculoskeletal symptoms. 11. The method of claim 10,
The electronic device is
identifying the first section corresponding to the first posture and the second section corresponding to the second posture based on the posture indicated by the feature points that change continuously with time,
A system for diagnosing musculoskeletal symptoms. According to claim 1,
The characteristic points are
Pelvis, lumbar spine, thoracic spine, neck, left clavicle, left shoulder, left elbow, left wrist, right clavicle, right shoulder, right elbow, right wrist, left hip, left knee, left ankle, left foot, right hip, right knee, right ankle , including a plurality of feature points selected from among feature points corresponding to the right foot, head, nose, left eye, left ear, right eye, and right ear,
A system for diagnosing musculoskeletal symptoms. According to claim 1,
The extracted feature points include a first feature point, a second feature point, a third feature point, and a fourth feature point,
The electronic device is
calculating an angle formed by a first straight line passing through the first feature point and the second feature point and a second straight line passing through the third feature point and the fourth feature point;
Calculating the score for the musculoskeletal system symptoms based on the calculated angle,
A system for diagnosing musculoskeletal symptoms. According to claim 1,
The extracted feature points include a first feature point and a second feature point,
The electronic device is
generating three-dimensional coordinate values of the first feature point and the second feature point that continuously change in response to the continuously changing posture of the subject;
calculating the score for the musculoskeletal symptoms based on the elevation change of the first feature point and the elevation change of the second feature point,
A system for diagnosing musculoskeletal symptoms. a communication unit arranged at different angles and receiving the image data from a plurality of image measurement devices for acquiring image data for continuously changing postures of the subject; and
A processor for extracting feature points corresponding to designated skeletal joints from the image data, and calculating a score for musculoskeletal symptoms using the extracted feature points;
Electronic device for diagnosing musculoskeletal symptoms. Acquiring image data for a continuously changing posture of a subject through a plurality of image measuring devices arranged at different angles;
extracting feature points corresponding to designated skeletal joints from the image data;
Comprising the step of calculating a score for musculoskeletal symptoms using the extracted feature points,
Methods for diagnosing musculoskeletal symptoms. A non-transitory computer-readable storage medium comprising:
An operation of receiving the image data from a plurality of image measurement devices arranged at different angles and acquiring image data for continuously changing postures of the subject;
an operation of extracting feature points corresponding to designated skeletal joints from the image data;
A non-transitory computer-readable storage medium for storing one or more programs for executing an operation of calculating a score for a musculoskeletal symptom using the extracted feature points.

Images