KR20230168095A - Method, system and non-transitory computer-readable recording medium for analyzing nail by using artificial intelligence model - Google Patents

Abstract

According to one aspect of the present invention, there is provided a method for analyzing fingernails using an artificial intelligence model, comprising: recognizing at least one nail by type from a captured image using a recognition model; and attaching a fingernail to the recognized at least one nail. A method is provided that includes inputting a corresponding image into an analysis model.

Description

Method, system, and non-transitory computer-readable recording medium for analyzing nails using an artificial intelligence model {METHOD, SYSTEM AND NON-TRANSITORY COMPUTER-READABLE RECORDING MEDIUM FOR ANALYZING NAIL BY USING ARTIFICIAL INTELLIGENCE MODEL}

The present invention relates to a method, system, and non-transitory computer-readable recording medium for analyzing fingernails using an artificial intelligence model.

As IT technology develops, many changes are occurring in the healthcare field. In particular, an attempt to determine or diagnose a person's health status using data that can be collected on a daily basis, breaking away from the traditional method of analyzing a person's health status using biomarkers such as blood and genes or cumbersome imaging methods such as MRI. is being done.

As an example, many technologies have been introduced to determine or diagnose a person's health status by analyzing images of body parts such as eyes, lips, and skin using artificial intelligence models such as artificial neural networks. However, images taken in an uncontrolled environment during daily life have limitations in that they are difficult to analyze because their quality and format vary, and the characteristics and changes that appear in body parts vary from person to person, making it difficult to accurately diagnose a specific person's health status. There was also.

Accordingly, the present inventor(s) have developed a technology to accurately and easily determine or diagnose a person's health status using images taken of fingernails.

The purpose of the present invention is to solve all the problems of the prior art described above.

In addition, the present invention recognizes at least one nail by type from a captured image using a recognition model, and inputs the image corresponding to the recognized at least one nail into an analysis model, thereby identifying the nail using an artificial intelligence model. Another purpose is to analyze and further determine or diagnose the health status of the subject being measured.

A representative configuration of the present invention to achieve the above object is as follows.

According to one aspect of the present invention, there is provided a method for analyzing fingernails using an artificial intelligence model, comprising: recognizing at least one nail by type from a captured image using a recognition model; and attaching a fingernail to the recognized at least one nail. A method is provided that includes inputting a corresponding image into an analysis model.

According to another aspect of the present invention, there is provided a system for analyzing nails using an artificial intelligence model, comprising: a recognition management unit that recognizes at least one nail by type from a captured image using a recognition model; and the recognized at least one nail. A system including an analysis management unit that inputs images corresponding to an analysis model is provided.

In addition, another method for implementing the present invention, another system, and a non-transitory computer-readable recording medium for recording a computer program for executing the method are further provided.

According to the present invention, it is possible to determine or diagnose the health status of a subject using fingernail images that can be easily captured on a daily basis.

In addition, according to the present invention, nails can be accurately recognized by type in an image using a recognition model learned separately for each type of nail, so the characteristics or changes that appear in the nail can be analyzed for each nail, and furthermore, the nail image It is possible to increase the accuracy of the health status analysis model using .

In addition, according to the present invention, not only images of fingernails, but also images of other body parts other than fingernails, the health status of the subject, and various data about the environment to which the subject belongs are used as learning data to create an analysis model. Since it can be learned, the reliability of the health status analysis model can be increased.

In addition, according to the present invention, a health status analysis model can be learned using time series data captured at multiple viewpoints, so changes in the health status of the subject are tracked based on changes appearing in the nail image, and the changes are derived from the changes. It is possible to determine or diagnose the health status of the person being measured.

Figure 1 is a diagram showing a schematic configuration of an entire system for analyzing fingernails using an artificial intelligence model according to an embodiment of the present invention.
Figure 2 is a diagram illustrating in detail the internal configuration of an analysis system according to an embodiment of the present invention.
Figure 3 is a diagram illustrating a situation in which a hand (nail) image is captured according to an embodiment of the present invention.
Figure 4 is a diagram illustrating a situation in which fingernails are recognized in an image of a hand according to an embodiment of the present invention.

The detailed description of the present invention described below refers to the accompanying drawings, which show by way of example specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different from one another but are not necessarily mutually exclusive. For example, specific shapes, structures and characteristics described herein may be implemented with changes from one embodiment to another without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description described below is not to be taken in a limiting sense, and the scope of the present invention should be taken to encompass the scope claimed by the claims and all equivalents thereof. Like reference numbers in the drawings indicate identical or similar elements throughout various aspects.

Hereinafter, several preferred embodiments of the present invention will be described in detail with reference to the attached drawings in order to enable those skilled in the art to easily practice the present invention.

Configuration of the entire system

Figure 1 is a diagram showing the schematic configuration of an entire system for analyzing fingernails using an artificial intelligence model according to an embodiment of the present invention.

As shown in FIG. 1, the entire system according to an embodiment of the present invention may include a communication network 100, an analysis system 200, and a device 300.

First, the communication network 100 according to an embodiment of the present invention can be configured regardless of communication mode, such as wired communication or wireless communication, and can be used as a local area network (LAN) or a metropolitan area network (MAN). It can be composed of various communication networks such as (Network) and Wide Area Network (WAN). Preferably, the communication network 100 referred to in this specification may be the known Internet or World Wide Web (WWW). However, the communication network 100 is not necessarily limited thereto and may include at least a portion of a known wired or wireless data communication network, a known telephone network, or a known wired or wireless television communication network.

For example, the communication network 100 is a wireless data communication network, including radio frequency (RF) communication, WiFi communication, cellular (LTE, etc.) communication, and Bluetooth communication (more specifically, low-power Bluetooth (BLE) communication. ; Bluetooth Low Energy)), infrared communication, ultrasonic communication, etc. may be implemented at least in part.

Next, the analysis system 200 according to an embodiment of the present invention recognizes at least one nail by type from the captured image using a recognition model, and uses the image corresponding to the recognized at least one nail as an analysis model. By inputting the fingernails using an artificial intelligence model, it is possible to perform the function of determining or diagnosing the health status of the subject being measured.

The configuration and function of the analysis system 200 according to an embodiment of the present invention will be discussed in detail through the detailed description below.

Next, the device 300 according to an embodiment of the present invention is a digital device that includes a function that can communicate after connecting to the analysis system 200, such as a smartphone, tablet, smart watch, smart patch, or smart device. Any digital device equipped with a memory means and equipped with a microprocessor, such as a band, smart glass, desktop computer, laptop computer, workstation, PDA, web pad, mobile phone, etc., and has computing power can be used as the device 300 according to the present invention. It can be adopted as. Additionally, the device 300 according to an embodiment of the present invention may be a digital device in the form of a kiosk that can be installed in a place where people regularly visit, such as a business or nail salon.

In particular, the device 300 according to an embodiment of the present invention may include a sensing means (eg, a camera, etc.) for obtaining an image of the body (nails, face, etc.) of the person being measured.

Meanwhile, the device 300 according to an embodiment of the present invention may include an application (not shown) that supports a user to receive a service according to the present invention from the analysis system 200. Such an application may be downloaded from the analysis system 200 or an external application distribution server (not shown). Meanwhile, the nature of this application may be generally similar to the recognition management unit 210, analysis management unit 220, communication unit 230, and control unit 240 of the analysis system 200, which will be described later. Here, at least part of the application may be replaced with a hardware device or firmware device that can perform substantially the same or equivalent functions as necessary.

Configuration of analysis system

Below, we will look at the internal configuration of the analysis system 200 and the function of each component, which performs important functions for implementing the present invention.

FIG. 2 is a diagram illustrating in detail the internal configuration of the analysis system 200 according to an embodiment of the present invention.

As shown in FIG. 2, the analysis system 200 according to an embodiment of the present invention may include a recognition management unit 210, an analysis management unit 220, a communication unit 230, and a control unit 240. According to one embodiment of the present invention, at least some of the recognition management unit 210, analysis management unit 220, communication unit 230, and control unit 240 of the analysis system 200 are connected to an external system (not shown). It may be a program module that communicates with. These program modules may be included in the analysis system 200 in the form of an operating system, application program module, or other program module, and may be physically stored in various known storage devices. Additionally, these program modules may be stored in a remote memory device capable of communicating with the analysis system 200. Meanwhile, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform specific tasks or execute specific abstract data types according to the present invention.

Meanwhile, although the analysis system 200 has been described as above, this description is illustrative and at least some of the components or functions of the analysis system 200 may be stored in the device 300 or server (not shown) as needed. It is obvious to those skilled in the art that it may be realized in or included in an external system (not shown).

First, the recognition management unit 210 according to an embodiment of the present invention may recognize at least one fingernail by type from a captured image using a recognition model.

Specifically, according to one embodiment of the present invention, images of the hands (nails) of a person being measured can be collected through a photography device provided at home, a nail salon, a hospital, a pharmacy, etc., and the image data collected in this way is It may be stored in association with the identification information or account information of the person being measured.

Figure 3 is a diagram illustrating a situation in which a hand (nail) image is captured according to an embodiment of the present invention.

Figure 4 is a diagram illustrating a situation in which fingernails are recognized in an image of a hand according to an embodiment of the present invention.

In addition, according to an embodiment of the present invention, an image of a hand (fingernail) may be captured multiple times over multiple viewpoints, and the images collected by being photographed over multiple viewpoints for a specific subject may be time series data. It can be stored as and used in the recognition model and analysis model described later.

The recognition model according to an embodiment of the present invention may include a Convolutional Neural Network (CNN) model, a Recurrent Neural Network (RNN) model, a Deep Belief Network (DBN) model, or an artificial neural network model combining the above models. . However, the artificial neural network model according to an embodiment of the present invention is not limited to those listed above, and may be changed in various ways within the scope of achieving the purpose of the present invention. For example, the recognition model according to an embodiment of the present invention may be an artificial intelligence model specialized for object detection and recognition, such as YOLOv5.

In addition, according to one embodiment of the present invention, the recognition model can be learned to recognize fingernails separately for each finger type, and for this purpose, the fingernail images constituting the learning data can be constructed separately for each finger type. By doing this, the recognition model can recognize the fingernails of each of the five fingers separately in the hand image, thereby eliminating the need to separately photograph or recognize the fingernails for each finger.

In fact, because signs indicating a person's health condition do not appear in the same way on all nails of the ten fingers, but often appear only on the nails of a specific finger, in order to accurately diagnose the health condition using a nail image, it is necessary to use a nail image. There is a need to recognize and collect them separately by type (by finger), and furthermore, there is a need to analyze nail images separately by type (by finger).

According to one embodiment of the present invention, the recognition model is learned based on first learning data including data about at least one of finger shape, finger position, finger length, finger thickness, fingernail shape, fingernail position, and finger joint wrinkles. It can be. Additionally, not only data collected from the general public but also data collected from people with diseases may be included in the first learning data.

For example, the recognition management unit 210 according to an embodiment of the present invention may specify the thumb with reference to the shape and location of the thumb that is significantly differentiated from other fingers, and may specify the thumb, such as a nail detected in the thumb portion. By using data on features such as shape and number of joint wrinkles as learning data, the recognition model can be trained so that it can accurately recognize the thumb nail.

In addition, for example, the recognition management unit 210 according to an embodiment of the present invention may collect a large number of fingernail images labeled for each finger and learn a recognition model using data about the shape of the fingernail appearing in the fingernail images. there is.

In addition, for example, the recognition management unit 210 according to an embodiment of the present invention uses the index finger, middle finger, ring finger, and small finger in that order based on the location of the thumb and whether it is left or right hand, which is relatively easily recognized in the hand image. It can be recognized as it is, and the nail part of each finger can be recognized by further referring to the joint position of each finger, and the recognition model can be retrained using the nail images recognized and collected.

In addition, for example, the recognition management unit 210 according to an embodiment of the present invention can recognize the fingernail part from the hand image by referring to data about the length and thickness of the finger, and the state in which the subject is measuring with his or her hand open. It is possible to determine whether a person is in a state of being aware or holding a hand, and apply different recognition standards based on the length and thickness of the fingers depending on the state. And, the recognition model can be retrained using the fingernail images recognized and collected in this way.

Next, according to an embodiment of the present invention, an image corresponding to at least one fingernail recognized by the above recognition model may be input into the analysis model. In addition, the analysis model according to an embodiment of the present invention can output data related to the determination or diagnosis of the health status of the subject using an image of the subject's fingernails.

Here, the analysis model according to an embodiment of the present invention includes a Convolutional Neural Network (CNN) model, a Recurrent Neural Network (RNN) model, a Deep Belief Network (DBN) model, or an artificial neural network model combining the above models. You can. However, the artificial neural network model according to an embodiment of the present invention is not limited to those listed above, and may be changed in various ways within the scope of achieving the purpose of the present invention.

According to an embodiment of the present invention, the analysis model may be learned based on second learning data including images of the fingernails of the subject and data on the health status of the subject.

For example, the color of the fingernail meniscus of a subject may have a significant correlation with the subject's health status, and an analysis model may be learned based on learning data supporting this correlation. Specifically, the training data supports the association between blue nail meniscus color and the health condition silver poisoning, and the training data supporting the association between red nail meniscus color and the health condition heart failure. An example would be training data that supports the association between the color of the fingernail meniscus being yellow and the state of health being on antibiotics.

In addition, according to an embodiment of the present invention, the analysis model includes data on at least one of the image of the subject's fingernails, images of other body parts of the subject, the health status of the subject, and the surrounding environment of the subject. It may be learned based on second learning data included.

In fact, signs that indicate a person's health status often appear not only in the nails but also in other body parts, such as the subject's face. Additionally, a person's health condition is greatly influenced by the surrounding environment surrounding the person. Therefore, according to the present invention, an analysis model is learned by complexly using highly correlated data such as images of the subject's fingernails, images of other body parts of the subject, the health status of the subject, and the surrounding environment of the subject. By doing so, both the accuracy and reliability of the analysis model can be improved.

Additionally, according to an embodiment of the present invention, the second learning data used to learn the analysis model may include time series data collected at at least two points in time, and the analysis model may be learned based on the time series data. Therefore, the analysis model according to the present invention can determine or diagnose changes in the health status of the subject that appears over time.

Specifically, according to an embodiment of the present invention, the analysis model determines the health status of a specific subject by referring to at least one of images of nails collected for a specific subject and images of other body parts of the subject. You can output related data.

For example, it can be assumed that an analysis model is learned based on learning data that supports the fact that a round face and round hands are associated with good health. In this case, in response to the fact that features indicating a sharp face, such as an angular jawline, sharp cheekbones, and protruding eyebrow bones, are detected in the subject's face data, and features indicating round hands, such as chubbyness, are detected in the subject's hand data, , the analysis model according to an embodiment of the present invention can output a determination or diagnosis result to the effect that the subject has a lung or kidney abnormality. In addition, the analysis model according to an embodiment of the present invention can track changes in the health status of the subject over time by analyzing time series data. Here, the jaw line, cheek angle, and eyebrow bone protrusion shown in the face data can be quantified, and the plumpness and joint bone protrusion shown in the hand data can also be quantified.

As another example, suppose that an analysis model is trained based on training data that supports the association between Terry's nail symptoms and jaundice in the face or eyes in cases of liver disease (such as cirrhosis). You can. In this case, in response to features indicating jaundice symptoms being detected in the subject's face data and features indicating terry nails being detected in the subject's fingernail data, the analysis model according to an embodiment of the present invention is used to determine whether the subject is being measured. It is possible to output a determination or diagnosis result indicating that the patient may have a liver disease such as cirrhosis.

The analysis system 200 according to the present invention can be used in all situations where it is necessary to routinely determine or diagnose a person's health condition.

As an example, it can be assumed that the analysis system 200 according to the present invention is applied to a workplace where a chemically hazardous environment exists. In this case, hand images including fingernails can be periodically captured for workers working at the relevant workplace, and fingernails can be recognized and analyzed from the captured images. And, as a result of the analysis, (1) in response to the fact that analysis results indicating health conditions are common among several workers, a notification to the effect that the workplace environment is likely to be highly hazardous can be provided; , (2) In response to analysis results showing that only one worker has a health condition, a notification may be provided to encourage the worker to undergo a health examination.

As another example, it can be assumed that the analysis system 200 according to the present invention is applied to a nail salon. In this case, hand images including fingernails can be captured at regular intervals for customers visiting the nail salon, and fingernails can be recognized and analyzed from the captured images. Additionally, in response to an analysis result indicating that a customer's health condition is abnormal, a notification recommending that the customer not receive nail art treatment may be provided. This is because nails with nail art cannot be used to determine or diagnose health status. By doing this, the customer can be encouraged to continuously receive health status analysis services based on nail analysis.

Next, the communication unit 230 according to an embodiment of the present invention may perform a function that enables data transmission and reception to/from the recognition management unit 210 and the analysis management unit 220.

Lastly, the control unit 240 according to an embodiment of the present invention may perform a function of controlling the flow of data between the recognition management unit 210, the analysis management unit 220, and the communication unit 230. That is, the control unit 240 according to the present invention controls the data flow to/from the outside of the analysis system 200 or the data flow between each component of the analysis system 200, thereby controlling the recognition management unit 210, the analysis management unit ( 220) and the communication unit 230 can each be controlled to perform their own functions.

The embodiments according to the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable by those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. A hardware device can be converted into one or more software modules to perform processing according to the invention and vice versa.

In the above, the present invention has been described in terms of specific details, such as specific components, and limited embodiments and drawings, but this is only provided to facilitate a more general understanding of the present invention, and the present invention is not limited to the above embodiments. Anyone with ordinary knowledge in the technical field to which the invention pertains can make various modifications and changes from this description.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the patent claims described below as well as all scopes equivalent to or equivalently changed from the scope of the claims are within the scope of the spirit of the present invention. It will be said to belong to

100: communication network
200: analysis system
210: Awareness management department
220: Analysis management department
230: Department of Communications
240: control unit
300: device

Claims (13)

As a method for analyzing nails using an artificial intelligence model,
Recognizing at least one fingernail by type from a captured image using a recognition model, and
Including the step of inputting an image corresponding to the recognized at least one fingernail into an analysis model.
method. According to paragraph 1,
The recognition model is learned based on first learning data including data on at least one of finger shape, finger position, finger length, finger thickness, fingernail shape, fingernail position, and finger joint wrinkles.
method. According to paragraph 1,
In the recognition step, the nails of the remaining fingers are recognized by referring to the position of the thumb nail.
method. According to paragraph 1,
The analysis model is learned based on second learning data including data about at least one of an image of the subject's fingernails, images of other body parts of the subject, the health status of the subject, and the surrounding environment of the subject. felled
method. According to paragraph 4,
The second learning data includes time series data collected at at least two points in time.
method. According to paragraph 1,
The analysis model outputs data on the health status of a specific subject by referring to images of nails collected for the subject and images of other body parts of the subject.
method. A non-transitory computer-readable recording medium recording a computer program for executing the method according to claim 1. A system for analyzing fingernails using an artificial intelligence model,
A recognition management unit that recognizes at least one fingernail by type from a captured image using a recognition model, and
Comprising an analysis management unit that inputs the image corresponding to the recognized at least one fingernail into an analysis model.
system. According to clause 8,
The recognition model is learned based on first learning data including data on at least one of finger shape, finger position, finger length, finger thickness, fingernail shape, fingernail position, and finger joint wrinkles.
system. According to clause 8,
The recognition management unit recognizes the nails of the remaining fingers by referring to the position of the thumb nail.
system. According to clause 8,
The analysis model is learned based on second learning data including data about at least one of an image of the subject's fingernails, images of other body parts of the subject, the health status of the subject, and the surrounding environment of the subject. felled
system. According to clause 11,
The second learning data includes time series data collected at at least two points in time.
system. According to clause 8,
The analysis model outputs data on the health status of a specific subject by referring to images of nails collected for the subject and images of other body parts of the subject.
system.

Images