KR20220053799A - Parkinson's disease determination method based on contour of clock and hands of clock - Google Patents

Abstract

Disclosed is a Parkinson's disease determination method of an electronic device. The Parkinson's disease determination method comprises: a step of extracting a contour of a clock from a test image generated based on user input; a step of acquiring a first score based on the extracted contour; a step of extracting at least one hand of the clock included in the test image; a step of acquiring a second score based on the extracted hand of the clock; and a step of identifying whether a user has Parkinson's disease based on the first score and the second score.

Description

PARKINSON'S DISEASE DETERMINATION METHOD BASED ON CONTOUR OF CLOCK AND HANDS OF CLOCK }

The present disclosure relates to an electronic device for diagnosing Parkinson's disease, and more particularly, to an electronic device for determining whether or not Parkinson's disease is present based on a watch outline and/or a watch hand drawn according to a clock drawing test (CDT)

A clock drawing test (CDT) is used as one of the methods for diagnosing degenerative brain diseases such as Parkinson's disease and dementia.

In the CDT, the test subject draws a watch, and scores the accuracy of the watch drawn by the subject based on various items (eg, whether numbers are written, the accuracy of the spatial composition of the watch, the accuracy of the hour and minute hands, etc.). Then, based on the score, it is to diagnose whether the subject has Parkinson's disease.

However, when the evaluator directly grades the subject's picture based on each item, it takes a long time. In particular, when the number of test subjects is larger than the number of evaluators, this problem is further exacerbated.

In addition, in the sense that some evaluation items may be somewhat subjective, there was a problem that the score may vary depending on the evaluator even if the subject encounters the same picture of the clock drawn by the subject.

Registered Patent Publication No. 10-2014-0015708 (2014.11.19)

The present disclosure provides a method for determining Parkinson's disease of an electronic device that automatically determines whether or not Parkinson's disease is present based on an outline of a watch and a watch hand included in a test image generated according to a user input.

A method of determining Parkinson's disease of an electronic device according to an embodiment of the present disclosure includes extracting a contour of a watch from a test image generated based on a user input, acquiring a first score based on the extracted contour; extracting at least one watch hand included in the test image, obtaining a second score based on the extracted watch hand, and determining whether the user has Parkinson's disease based on the first score and the second score identifying the step.

In this case, the obtaining of the first score may include calculating the first score according to at least one of whether the contour is a closed curve and a slope for each section of the contour.

On the other hand, the Parkinson's disease determination method inputs at least one sample image drawn by a normal person to an artificial intelligence model trained to extract the outline of the watch, and the position of the outline of the watch included in the sample image in the sample image It may further include the step of identifying. In this case, the acquiring of the first score may include acquiring the first score based on a degree of matching between a position of the extracted contour in the test image and a position of the identified contour in the sample image.

In addition, the method for determining Parkinson's disease includes: acquiring sensing data based on coordinates within the touch screen of the user input received on a touch screen; generating the test image based on the acquired sensing data may include more. In this case, the step of extracting the clock hand may include identifying, among the sensed data, sensing data included in a time period in which a rate of change of coordinates in the touch screen according to the user input is equal to or greater than a preset threshold, and within the test image. may extract the hour hand or the minute hand corresponding to the identified sensing data.

Also, the method for determining Parkinson's disease may further include identifying a plurality of numbers included in the test image based on the extracted contour. In this case, the step of obtaining the second score may include identifying the time indicated by the clock hand based on the plurality of numbers, and obtaining the second score according to whether the identified time and a preset time match. can

In this case, the step of obtaining the second score may include extending the watch hand so that the watch hand approaches the extracted outline, identifying the time indicated by the extended watch hand based on the plurality of numbers, and The second score may be obtained according to whether the identified time matches the preset time.

On the other hand, the Parkinson's disease determination method inputs at least one sample image drawn by a normal person to an artificial intelligence model trained to extract the clock hand, and identifies the position in the sample image of the clock hand included in the sample image It may include further steps. Here, the obtaining of the second score may include obtaining the second score based on a degree of matching between a position in the test image of the extracted watch hand and a position in the sample image of the identified watch hand. .

The electronic device according to the present disclosure has an effect of automatically and quickly scoring the accuracy of a test image by extracting an outline and/or a clock hand from a test image drawn by a user.

The electronic device and the control method thereof according to some embodiments of the present disclosure have an advantage that the CDT test can be performed through an application or remote communication that can be executed through two or more devices.

1 is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure;
2 is a view for explaining an example in which an electronic device generates a test image according to a user input according to an embodiment of the present disclosure;
3 is a flowchart illustrating a method for diagnosing Parkinson's disease of an electronic device according to an embodiment of the present disclosure;
4A to 4B are diagrams for explaining an operation of comparing a contour extracted through an artificial intelligence model with a contour in a test image in a method for diagnosing Parkinson's disease according to an embodiment of the present disclosure;
5 is a view for explaining an operation of determining a number indicated by a clock hand in a method for diagnosing Parkinson's disease according to an embodiment of the present disclosure;
6 is a view for explaining an operation of determining a number indicated by a clock hand in a method for diagnosing Parkinson's disease according to an embodiment of the present disclosure;
7 is a view for explaining an operation of determining a number indicated by a clock hand in a test image drawn in a state where an outline and number points are given in the method for diagnosing Parkinson's disease according to an embodiment of the present disclosure;
8 is a view for explaining an operation of comparing a watch hand extracted through an artificial intelligence model with a watch hand in a test image in a method for diagnosing Parkinson's disease according to an embodiment of the present disclosure;
9A to 9C are diagrams for explaining specific examples in which an electronic device extracts numbers from a test image according to an embodiment of the present disclosure;
10 is an algorithm for explaining a method for diagnosing Parkinson's disease of an electronic device according to an embodiment of the present disclosure;
11 is a block diagram illustrating a configuration and operation of an electronic device according to various embodiments of the present disclosure;

Prior to describing the present disclosure in detail, a description will be given of the description of the present specification and drawings.

First, terms used in the present specification and claims have been selected in consideration of functions in various embodiments of the present disclosure. However, these terms may vary depending on the intention or legal or technical interpretation of a person skilled in the art, and the emergence of new technology. Also, some terms are arbitrarily selected by the applicant. These terms may be interpreted in the meanings defined herein, and in the absence of specific definitions, they may be interpreted based on the general content of the present specification and common technical common sense in the art.

Also, the same reference numerals or reference numerals in each drawing attached to this specification indicate parts or components that perform substantially the same functions. For convenience of description and understanding, the same reference numbers or reference numerals are used in different embodiments. That is, even though all components having the same reference number are illustrated in a plurality of drawings, the plurality of drawings do not mean one embodiment.

In addition, in this specification and claims, terms including an ordinal number such as “first” and “second” may be used to distinguish between elements. This ordinal number is used to distinguish the same or similar elements from each other, and the meaning of the term should not be construed as limited due to the use of the ordinal number. As an example, the use order or arrangement order of components combined with such an ordinal number should not be limited by the number. If necessary, each ordinal number may be used interchangeably.

In this specification, the singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprises" or "consisting of" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and are intended to indicate that one or more other It should be understood that this does not preclude the possibility of addition or presence of features or numbers, steps, operations, components, parts, or combinations thereof.

In an embodiment of the present disclosure, terms such as “module”, “unit”, “part”, etc. are terms for designating a component that performs at least one function or operation, and such component is hardware or software. It may be implemented or implemented as a combination of hardware and software. In addition, a plurality of "modules", "units", "parts", etc. are integrated into at least one module or chip, except when each needs to be implemented in individual specific hardware, and thus at least one processor. can be implemented as

In addition, in an embodiment of the present disclosure, when it is said that a certain part is connected to another part, this includes not only a direct connection but also an indirect connection through another medium. In addition, the meaning that a certain part includes a certain component means that other components may be further included, rather than excluding other components, unless otherwise stated.

1 is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 1 , the electronic device 100 includes a memory 110 and a processor 120 .

The electronic device 100 may be implemented as various devices such as a smartphone, a tablet PC, a notebook PC, a desktop PC, a kiosk, and a TV. In addition, the electronic device 100 may be implemented as a server, and details will be described later with reference to FIG. 10 .

The memory 110 is a configuration for storing an operating system (OS) for controlling overall operations of the components of the electronic device 100 and at least one instruction or data related to the components of the electronic device 100 . .

The memory 110 may include non-volatile memory such as ROM and flash memory, and may include volatile memory such as DRAM. Also, the memory 120 may include a hard disk, a solid state drive (SSD), or the like.

The processor 120 controls the overall operation of the electronic device 100 . Specifically, the processor 120 may be connected to the memory 110 to control the electronic device 100 .

To this end, the processor 120 may include a central processing unit (CPU), a graphic processing unit (GPU), a neural processing unit (NPU), etc. in hardware, Control-related operations and data processing can be performed.

The processor 120 may be implemented as a micro processing unit (MPU) or may correspond to a computer in which random access memory (RAM) and read only memory (ROM) are connected to a CPU or the like through a system bus.

The processor 120 may control one or more software modules included in the electronic device 100 as well as hardware components included in the electronic device 100 , and the result of controlling the software modules by the processor 120 is It may be derived from the operation of hardware configurations.

The processor 120 may perform a method for determining Parkinson's disease to be described later by executing at least one instruction stored in the memory 110 , which will be described below with reference to the drawings.

2 is a diagram for explaining an example in which an electronic device generates a test image according to a user input according to an embodiment of the present disclosure;

FIG. 2 assumes that the electronic device 100 is implemented as a tablet PC including a display configured as a touch screen.

In addition, Fig. 2 assumes a situation in which a guide to proceed with the CDT test (ex. Draw the outline of the watch. Draw the numbers inside the watch. Draw a watch indicating 11:20 o'clock, etc.) is given to the user. .

In this case, a guide for performing the CDT test may be provided visually through the display of the electronic device 100 or may be provided aurally through the speaker of the electronic device 100 .

Referring to FIG. 2 , the electronic device 100 may receive an input of the user 1 who touches the touch screen of the electronic device 100 using a touch pen or a finger.

Specifically, the processor 120 may acquire sensing data based on coordinates within the touch screen of a user input received on the touch screen.

In addition, the processor 120 may generate the test image 10 based on the acquired sensing data.

Meanwhile, the electronic device 100 may communicate with at least one external device to receive a test image from the corresponding external device.

For example, the electronic device 100 implemented as a server may receive a test image drawn according to a user input received through the smartphone from the smartphone.

Meanwhile, although it is assumed in FIG. 2 that the outline, numbers, and hands of the watch are all drawn directly according to the user input, in contrast to this, at least one of the outline, numbers, and hands of the watch is drawn in the test image. In the state, it is also possible to add the remaining elements according to user input.

For example, in a state in which the outline and numbers of the watch are already drawn, the electronic device 100 may provide a guide for the user to draw the watch hands (eg, draw a watch hand indicating 11:20). In this case, according to a user input, a clock hand may be additionally drawn in the test image.

3 is a flowchart illustrating a method for diagnosing Parkinson's disease by an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 3 , the processor 120 may extract the outline (: border) of the watch from the test image (S310).

As an example, when a guidance of “draw the outline of the watch” is provided, the processor 120 may acquire the coordinates of the user input received on the touch screen after the guidance, and acquire the outline of the watch composed of the coordinates. there is.

As another example, the processor 120 may use at least one artificial intelligence model trained to extract the contour of the watch from the image. The present artificial intelligence model may be a model trained to identify circles and/or closed curves drawn by humans.

Then, the processor 120 may obtain a first score based on the extracted contour (S320).

In this case, the processor 120 may calculate the first score according to at least one of whether the extracted contour of the watch is a closed curve and a slope for each section of the contour.

As an example, the processor 120 may calculate the first score according to the criteria of Table 1 below.

2 points Display without total distortion 1 point incomplete or some distortion 0 points absent or completely inappropriate

As a specific example, the processor 120 may divide the extracted outline of the watch into a plurality of sections, and obtain inclinations of each of the divided sections.

If the amount of change in inclination for each section, such as the contour of the watch being sharply bent (ex. sharp shape) or not bent (ex. straight line), is out of a preset threshold range for drawing a circle, the processor 120 performs the first A score of 0 (no or completely inappropriate) can be calculated.

As another example, if the amount of change in the inclination of the watch outline for each section does not deviate from a preset threshold range, but the watch outline is not a closed curve, the processor 120 calculates the first score as 1 point (: incomplete or partially distorted) can do.

However, when the contour of the watch is a closed curve and a change in inclination within a critical range is maintained without a sharp bend section, the processor 120 may calculate the first score as 2 points (displayed without total distortion).

Meanwhile, the processor 120 may obtain the first score by comparing the outline of the watch extracted from the test image with the outline of the watch extracted from a picture of a normal person (a person not suffering from Parkinson's disease).

4A to 4B are diagrams for explaining an operation of comparing a contour extracted through an artificial intelligence model with a contour in a test image in a method for determining Parkinson's disease according to an embodiment of the present disclosure.

According to this embodiment, the processor 120 inputs at least one sample image drawn by a normal person to the artificial intelligence model trained to extract the outline of the watch, and determines the position in the sample image of the outline of the watch included in the sample image. can be identified.

In relation to this, FIG. 4A shows an example of an artificial intelligence model trained to output a map of a specific part (eg, the outline of a watch) within an image when an image is input. However, although the artificial intelligence model implemented in the U-net form is used in FIG. 4A, it goes without saying that artificial intelligence models according to various forms and training methods may be used.

This artificial intelligence model, for example, may be trained through a large number of images (input) including a watch and a map (output) of the outline of the watch included in each of the images. Here, the map of the outline of the watch indicates the position of the outline of the watch included in each sample image.

The processor 120 may input one or more sample images (a clock picture) drawn by a normal person to the artificial intelligence model as shown in FIG. 4A . As a result, a map indicating the position of the outline of the field of view in the sample image may be output.

Here, referring to FIG. 4B , the processor 120 matches the position (coordinate) in the test image of the contour (sensor) extracted from the test image and the position (coordinate) in the sample image of the segmented contour extracted from the sample image. A first score may be obtained based on the degree.

In this case, the position in the sample image of the segmented contour extracted from the sample image may correspond to an average of the position of the contour indicated by the plurality of maps of the watch contour extracted from the plurality of sample images.

Referring to FIG. 4B , when the matching degree is 75% or more, the processor 120 may calculate the first score as 2 points. In addition, the processor 120 may calculate the first score as 1 point when the matching degree is 50 percent or more and less than 75 percent, and may calculate the first score as 0 points when the matching degree is less than 50 percent. However, the specific percentage threshold is not limited to the embodiment of FIG. 4B .

Meanwhile, referring to FIG. 3 , the processor 120 may extract a watch hand included in the test image ( S330 ).

As an example, the processor 120 may extract at least one watch hand located inside the outline of the watch extracted in step S310 (using coordinates on the touch screen) and having a straight line length equal to or greater than a threshold value.

On the other hand, in general, before and after the user draws the hour or minute hand, the user input for touching the touch screen is cut off or the speed of changing the touched coordinates is often slowed down, and the speed is relatively fast while the hour or minute hand is being drawn. Since there are many cases, the hour hand or the minute hand may be identified using the rate of change of coordinates in the touch screen according to continuous user input.

As a related embodiment, it may be assumed that the guidance "Please draw the hour and minute hands" is provided.

In this case, the processor 120 detects a user input (: touch), among the sensing data obtained by sensing the user input, the sensing data included in the time period in which the rate of change of the coordinates in the touch screen according to the user input is maintained at or above a preset threshold. can be identified.

Here, the processor 120 may extract the clock hands by using coordinates matching the sensed data within the corresponding time period.

When there are a plurality of time sections, one watch hand may be extracted for each time section, and the processor 120 may identify the length of the watch hand using the speed of change of coordinates for each section and the time length of the time section.

For example, when two watch hands are extracted, the processor 120 may identify the long watch hand as the minute hand and the short watch hand as the hour hand.

The processor 120 may obtain a second score based on the extracted watch hands (S340).

Specifically, the processor 120 is configured based on the number of extracted clock hands, a difference in length between the extracted clock hands (whether the hour hand and the minute hand are distinguished), and the angle of the extracted clock hands (time pointed by the clock hand), etc. A second score may be obtained.

Table 2 below corresponds to an example of a criterion for calculating the second score.

4 points Needles in correct position, length difference 3 points Error in needle sequence or configuration 2 points The main error of needle position 1 point poor expression of the needle 0 points no needle

As a specific example, when the watch hands are not drawn, a hand whose length is equal to or greater than a threshold value may not be identified in the outline of the watch included in the test image. In this case, the processor 120 may calculate the second score as 0 (no needle).

As another example, when only one watch hand is identified, the processor 120 may calculate the second score as 1 point (ie, poor hand expression).

As another example, if both the hour and minute hands are identified in the test image, but the time indicated by the corresponding hour and minute hands is different from the previously provided guidance (eg, please draw a clock indicating 11:20), the processor 120 A second score can be calculated as 2 points (: major error in needle position).

Here, the processor 120 may identify the time indicated by the clock in the test image according to the distance between the clock hands (hour and minute hands) and each number.

In relation to this, FIG. 5 is a diagram for explaining an operation of determining a time indicated by a clock hand in a method for diagnosing Parkinson's disease according to an embodiment of the present disclosure.

5 shows a test image generated according to a user input. Referring to FIG. 5 , the processor 120 may identify the outline 510 of the watch and the watch hands 531 and 532 from the test image according to the above-described embodiments.

Also, the processor 120 may identify the plurality of numbers 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, and 12 from the test image.

Specifically, the processor 120 may identify a plurality of numbers located therein based on the extracted contour 610 .

In this case, the processor 120 may use optical character recognition (OCR) or at least one artificial intelligence model trained to identify a number included in an image.

Here, the processor 120 may identify the time indicated by the clock hands 531 and 532 based on the identified plurality of numbers.

For example, the processor 120 may identify a point closer to the outline 510 among both ends of the clock hand 531 that is the hour hand, and identify at least one number “11” closest to the identified point.

Likewise, the processor 120 may identify the number “2” that is closest to the minute hand 632 .

Here, the time indicated by the hour hand 531 and the minute hand 532 included in the test image is 11:10.

In this case, the processor 120 may calculate the second score according to the degree of matching between the time indicated by the hour hand 531 and the minute hand 532 in the test image and a preset time. Here, the preset time may be a time according to a guide provided to the user in advance.

For example, if the guide provided in advance is "Please draw a clock that shows 11:20", the preset time is 11:20.

In this case, since the time indicated by the clock in the test image (11:10) and the preset time (11:20) do not match with each other, the processor 120 sets the second score as 3 points according to Table 2 above. can be calculated.

Meanwhile, unlike FIG. 5 , the processor 120 may identify the nearest number by extending the length of each clock hand 531 and 532 .

In relation to FIG. 6 , the processor 120 may extend the watch hand 531 so that the watch hand 531 approaches the outline 510 . In this case, the processor 120 may extend the watch hand 531 while maintaining the inclination of the watch hand 531 .

In addition, the processor 120 may identify the number “11” that is the closest to the extended clock hand 531 ′.

On the other hand, although in the case of FIGS. 5 and 6, it is assumed that all numbers in the test image match 1 to 12, as well as a situation in which all numbers in the test image are drawn according to relatively accurate positions and arrangement order. There may also be situations where the composition does not correspond to 1 to 12, or the position and arrangement of numbers are not correct.

Accordingly, the processor 120 may identify the time indicated by the clock hand by using the virtual number point.

As a specific example, as shown in FIG. 7 , the processor 120 generates virtual number points 720-1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12) can be generated (it is of course also possible that the virtual numbers 1-12, not the virtual number point, are generated).

Then, the processor 120 identifies the time (: 11:10) indicated by the test image according to the number points 720-11 and 2 closest to the clock hands 531 and 532 among the generated virtual number points. can do. Then, the processor 120 may identify whether the identified time matches a preset time (guide).

On the other hand, as another example, two clock hands are identified in the test image, and the numbers pointed to by the clock hands match the time according to the guide provided in advance (eg, please draw a clock indicating 11:20), but the clock hands It can be assumed that there is no difference in length between them.

In this case, the processor 120 may calculate the second score as 3 points (error in needle order or configuration).

As another example, when the hour hand and the minute hand are clearly distinguished and their positions (: time) also match the guidance, the processor 120 may calculate the second score as 4 points.

On the other hand, according to an embodiment of the present disclosure, the processor 120 inputs at least one sample image drawn by a normal person to the artificial intelligence model trained to extract the watch hand, and the sample image of the watch hand included in the sample image. I can identify my location.

In addition, the processor 120 may obtain a second score based on the degree of matching between the extracted position in the test image of the watch hand and the identified position in the sample image of the watch hand.

In relation to this, FIG. 8 is a diagram for explaining an operation in which an electronic device of the present disclosure compares a watch hand extracted through an artificial intelligence model with a watch hand in a test image.

Here, the artificial intelligence model may be configured and trained in a U-net form as in FIG. 4A described above, and may be a model trained to extract at least one clock hand from an image.

According to this embodiment, the processor 120 inputs at least one sample image drawn by a normal person to the artificial intelligence model trained to extract the watch hands, and identifies the position in the sample image of the watch hands included in the sample image. can

And, as shown in Figure 8, the processor 120 is based on the degree of matching between the position in the test image of the watch hand (sensor) extracted from the test image and the position in the sample image of the watch hand (segmented) extracted from the sample image to obtain a second score.

As a specific example, the processor 120 calculates the second score as 4 points when the degree of overlap between the watch hand (sensor) and the watch hand (segmented) is 80% or more, and when the overlapping degree is 70% or more and less than 80%, the second score A score is calculated as 3 points, and when the degree of overlap is 60 percent or more and less than 70 percent, the second score is calculated as 2 points, and when the degree of overlap is 50 percent or more and less than 60 percent, the second score is calculated as 1 point, and the degree of overlap is calculated as 1 point. If is less than 50 percent, the second score may be calculated as 0 points. However, specific numerical values are not limited to the above-described examples.

When the first score and the second score are calculated according to the above-described various embodiments, the processor 120 may identify whether the user (whether the test image is drawn) has Parkinson's disease based on the first score and the second score ( S350).

As a specific example, when the sum of the first score and the second score is equal to or greater than a threshold (eg, 5), the processor 120 may identify that the user does not have Parkinson's disease.

However, when the summed score is less than the threshold, the processor 120 may identify the user as having Parkinson's disease. Here, it can be identified that the severity of Parkinson's disease is greater as the summed score is smaller.

Meanwhile, it is of course possible for the processor 120 to determine whether or not Parkinson's disease is present by using only one of the above-described first score and second score.

For example, in the CDT test, it may be assumed that a user input is started in a state in which an outline is drawn in a test image from the beginning. In this case, the processor 120 may calculate the above-described second score using the clock hand drawn according to the user input, and determine whether or not Parkinson's disease is present based on the second score.

Meanwhile, the processor 120 may obtain a (summed) score by considering additional factors in addition to the above-described first score (outline) and second score (clock hands).

According to an embodiment, the processor 120 may extract numbers included in the watch in addition to the outline and the watch hands of the watch from within the test image.

For example, the processor 120 may identify a number included in the test image through optical character recognition (OCR) technology.

As another example, the processor 120 may use at least one artificial intelligence model trained to identify a number.

This artificial intelligence model may be a model trained to identify digit numbers based on number images drawn by various people (ex. MNIST dataset), and may include a Convolutional Neural Network (CNN).

The processor 120 may identify at least one single digit by inputting at least a part of the drawn test image according to the user input to the corresponding AI model.

In relation to this, FIGS. 9A to 9C are diagrams for explaining a first embodiment in which an electronic device according to the present disclosure extracts numbers from a test image.

9A illustrates a test image drawn through a user input.

Referring to FIG. 9A , the processor 120 may extract the outline 950 of the watch. To this end, the processor 120 may use at least one artificial intelligence model trained to extract the outline of the watch when a watch picture is input. This AI model may be a model trained to identify circles or closed curves drawn by humans.

In addition, the processor 120 may identify an area including at least one character drawn within a preset distance from the extracted outline 950 .

Here, the processor 120 may define each region constituting one character to which touch coordinates according to a user input are connected.

For example, referring to FIG. 9A , since all of the (touch) coordinates constituting “1” are connected, the region 901 including “1” is identified as one region. However, since the coordinates constituting "1" and the coordinates constituting "2" are spaced apart from each other without a connection point, the area 902 including "2" is the area 901 including "1" and are separated

In this case, the processor 120 may identify each rectangular area including all of the coordinates connected to each other, but the shape of the area need not be limited to only the rectangular area.

In this way, a plurality of regions 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911, 912, 913, 914, 915, 916 can be identified as shown in FIG. 9A. there is.

Then, the processor 120 inputs the identified plurality of regions into the artificial intelligence model trained to identify the above-described single-digit number, thereby providing a plurality of numbers (1, 2, 3, 4, 5, 6, 7, 0, 0, 9, 1, 0, 1, 1, 1, 2) can be identified.

However, as an example, when the distance between regions corresponding to “0” is less than the first threshold, the processor 120 may identify one “8” including the corresponding regions. Here, the distance between the regions may correspond to a minimum distance between the regions or a distance between centers of gravity of the regions (eg, an average of coordinate values within the region).

As a specific example, referring to FIG. 9B , when the minimum distance 961 between the regions 908 and 909 corresponding to “0” is less than a predetermined first threshold (ex. 3 mm), the processor 120 controls the corresponding region It can be identified that the region 921 including the fields 908 and 909 corresponds to “8”.

As a result, the composition of the identified (single-digit) plurality of numbers is 1, 2, 3, 4, 5, 6, 7, 8, 9, 1, 0, 1, 1, 1, 2.

On the other hand, although the coordinates constituting “8” are not entirely connected in FIG. 9A, it is assumed that “8” is not identified from the beginning and is identified as “0” and “0”, but unlike FIG. 9A, “8” If the (touch) coordinates constituting " "are drawn to be connected to each other, the processor 120 may identify the area corresponding to "8" from the beginning.

In addition, the processor 120 identifies a two-digit number consisting of two numbers whose distance between each other is less than a predetermined second threshold (ex. 1.8 cm) among the plurality of (single-digit) numbers identified in FIG. 9A . can do. Here, the distance between the numbers may be at least one of a minimum distance between regions including each number or a distance between centers of gravity of the regions, but is not limited thereto.

Specifically, referring to FIG. 9C , when the minimum distance 962 between the region 915 including “1” and the region 916 including “2” is less than a second preset threshold, the processor 120 A region 922 including the corresponding regions 915 and 916 may be identified as corresponding to a two-digit number “12”.

Likewise, in FIG. 9A , the two-digit number “11” can be identified that includes an area 913 and an area 914 . Also, in FIG. 9A , a two-digit number “10” including an area 911 and an area 912 can be identified.

As a result, the processor 120 includes a plurality of numbers (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12) can be identified.

Meanwhile, unlike the above-described first embodiment, according to the second embodiment, the processor 120 applies multi-size patches within a preset range inside the outline 950 to correspond to various sizes and locations (coordinates). regions can be extracted.

Specifically, the processor 120 may extract box regions of various sizes with centers of gravity at points located inside the contour 950 and having a distance from the contour 950 less than a preset threshold.

Then, the processor 120 may input the extracted regions to the model trained to identify the above-described single-digit number.

As a result, multiple numbers corresponding to a single digit (1, 2, 3, 4, 5, 6, 0, 7, 8, 0, 0, 9, 0, 1, 0, 1, 1, 1, 2) can be identified.

At this time, not only 6 is identified, but also a circle included in 6 may be identified as 0, 8 may be identified as well as two circles constituting 8 may be identified as additional 0s, respectively, and 9 may be identified as well The circle contained in 9 may be identified as an additional zero.

However, this is only an example, and if the circles included in 6, 8, and 9 are not clear, 0 may not be additionally identified and only 6, 8, and 9 may be identified.

Here, the processor 120 may erase all 0's constituting 6, 8, or 9. That is, the processor 120 may remove 0, which consists only of touch coordinates constituting 6, 8, or 9, from the list of identified numbers.

Accordingly, the configuration of the identified plurality of numbers may be 1, 2, 3, 4, 5, 6, 7, 8, 9, 1, 0, 1, 1, 1, 2.

In addition, the processor 120 may identify two-digit numbers 10 , 11 , and 12 including single-digit numbers having a distance between each other less than the second threshold, similar to the above-described FIG. 9C .

As a result, the configuration of the identified plurality of numbers may be 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12.

When a plurality of numbers are extracted based on the above-described first or second exemplary embodiment, the processor 120 may obtain a third score based on the extracted plurality of numbers.

Specifically, the processor 120 may obtain the third score according to at least one of the configuration, position, and arrangement order of the plurality of extracted numbers.

As an example, the third score may be calculated according to the criteria of Table 3 below.

4 points Everything is displayed in the correct order 3 points All but an error in the collating sequence 2 points Count is correct, but there is an error in the composition or position of the number 1 point count added 0 points not enough number

In general, the numbers included in the watch correspond to 1 to 12. Accordingly, 1 to 12 may be preset as numbers included in the watch (however, an embodiment in which only 3, 6, 9, and 12 are preset is of course possible).

In addition, the processor 120 may calculate a third score based on whether a plurality of numbers extracted from the test image match each of the numbers 1 to 12 that are preset to be included in the watch.

Here, the processor 120 may calculate the third score based on whether the number of the extracted plurality of numbers matches the predetermined number of numbers to be included in the watch.

As an example, when less than 12 numbers are extracted from the test image, the processor 120 may calculate the third score as 0 points in Table 3 (the number is insufficient).

As another example, when more than 12 numbers are extracted, the processor 120 may calculate the third score as 1 point (the number is added).

Also, the processor 120 may calculate a third score according to whether each of the plurality of extracted numbers matches each of the preset numbers.

As an example, when the number of extracted plurality of numbers is 12, but not all of the numbers 1 to 12 are included, the processor 120 sets the third score to 2 points (the number is correct but the number configuration or position is correct). in error) can be calculated.

Meanwhile, the processor 120 may calculate a third score based on whether the positions and the arrangement order of the plurality of numbers in the test image match the preset positions and the preset arrangement order, respectively.

The preset position is, for example, in a situation in which a horizontal line and a vertical line drawn based on the center of gravity of the outline of the watch (eg, the average of the coordinate values of all coordinates within the outline) are defined, from the upper part of the vertical line based on the center of gravity. A number is drawn within a set distance, a number is drawn within a preset distance from the lower part of the vertical line, a number is drawn within a preset distance from the left part of the horizontal line based on the center of gravity, and a right part of the horizontal line It may correspond to a situation in which one number is drawn within a preset distance from .

In addition, in the preset position, two numbers are drawn between the upper part of the vertical line and the right part of the horizontal line, two numbers are drawn between the right part of the horizontal line and the lower part of the vertical line, and two numbers are the left side of the horizontal line Drawn between the part and the lower part of the vertical line, two numbers may correspond to the situation drawn between the left part of the horizontal line and the upper part of the vertical line.

Also, the preset position may correspond to a situation in which all numbers are located inside the outline of the watch, all numbers are located within a preset distance from the outline of the watch, and the distance between adjacent numbers is equal to or greater than a threshold value.

The preset arrangement order is, for example, in a situation in which a horizontal line and a vertical line drawn based on the center of gravity of the outline of the watch are defined, 12 is drawn within a predetermined distance from the upper part of the vertical line based on the center of gravity, and the lower side of the vertical line It may correspond to a situation in which 6 is drawn within a predetermined distance from the part, 9 is drawn within a predetermined distance from the left part of the horizontal line based on the center of gravity, and 3 is drawn within a predetermined distance from the right part of the horizontal line.

In addition, as for the preset arrangement order, 1 and 2 are drawn in order between 12 and 3, 4 and 5 are drawn in order between 3 and 6, 7 and 8 are drawn in order between 6 and 9, It may correspond to a situation where 10 and 11 are drawn in sequence between 9 and 12.

For example, if the number of numbers is 12, but the composition of the numbers is not 1 to 12, and the positions of the numbers do not match the preset positions, the processor 120 sets the third score to 2 points (the number is correct, but the numbers are configuration or location is not correct).

As another example, if the number of numbers is 12, there are all of them from 1 to 12, and the positions of the numbers also match the preset positions, but the arrangement order is different from the preset arrangement order, the processor 120 assigns the third score to 3 points ( All of them, but there is an error in the arrangement order).

As another example, when the number of numbers is 12, there are all of them from 1 to 12, and the position and arrangement order of the numbers match the preset position and the preset arrangement order, the processor 120 assigns the third score to 4 points (all are displayed in the correct order).

Meanwhile, in calculating the third score, the processor 120 may additionally consider at least one of the above-described distances 961 and 962 between the regions obtained in FIGS. 9B and 9C .

As an example, the processor 120 may deduct the third score according to the distance 962 of FIG. 9C . Specifically, as the minimum distance 962 between the region 915 and the region 916 corresponding to single-digit numbers constituting one two-digit number increases, the degree of deduction of the third score may increase.

As a specific example, when the positions and arrangement order of numbers ( 1 to 12 in FIG. 9A ) in the test image match the above-described preset positions and preset arrangement order, the processor 120 calculates the third score as 4 points. can do. However, here, when the minimum distance 962 between the region 915 and the region 916 exceeds a threshold (eg, 1 cm) by 0.1 mm, the processor 120 deducts the third score by 0.1 points from 4 points to 3.9 points can be calculated. Alternatively, if the minimum distance 962 between the region 915 and the region 916 exceeds the threshold by 0.2 mm, the processor 120 may calculate the third score as 3.8 points by deducting 0.2 points from 4 points. there is.

That is, as the distance between single-digit numbers (“1” and “2”) constituting the two-digit number “12” increases, the processor 120 may lower the third score.

When the third score is calculated according to the above-described various embodiments, the processor 120 may determine whether or not Parkinson's disease is present by adding up the third score to the first and second scores.

As a specific example, when the sum of the first score, the second score, and the third score is 9 or more, the processor 120 may identify that the user does not have Parkinson's disease.

As another example, when the summed score is 7 or more and less than 9, the processor 120 may identify that the user needs additional diagnosis and/or prevention of Parkinson's disease.

As another example, when the summed score is less than 7, the processor 120 may identify the user as having Parkinson's disease. In this case, the processor 120 may identify that the lower the summed score, the higher the severity.

Meanwhile, according to an embodiment of the present disclosure, in determining whether Parkinson's disease is present, the processor 120 may additionally consider a secondary test image drawn according to a separate user input.

Specifically, the processor 120 may generate the secondary test image based on an additional user input of the same user.

In addition, the processor 120 may extract the outline of the watch, the hands of the watch, a plurality of numbers, and the like from the secondary test image.

In addition, the processor 120 may update the (summed) score by comparing the field of view included in the first generated (primary) test image with the field of view included in the secondary test image.

As an example, the processor 120 may compare the configuration (hour hand, hour hand) and time of the clock hands included in the first test image with the configuration and time of the clock hands included in the secondary test image.

If the configuration of the watch hands is changed or the time indicated by the watch hands is changed, the processor 120 may deduct a score.

For example, when the number of clock hands is changed, the processor 120 may deduct two points. As another example, when the number of clock hands is the same but the time indicated is different, the processor 120 may deduct one point.

Alternatively, the processor 120 may compare the configuration, position, and arrangement order of the plurality of numbers in the first test image with the configuration, position, and arrangement order of the plurality of numbers in the second test image.

And, when at least one of a configuration, a position, and an arrangement order is changed, the processor 120 may perform a deduction of points on the score. Here, as the degree of change increases, the degree of point deduction may also increase.

For example, the processor 120 may calculate the correlation coefficient R according to correlation by comparing the arrangement order of the plurality of numbers in the first test image with the arrangement order of the plurality of numbers in the second test image.

Here, the correlation coefficient (R) may be a rank correlation coefficient, and a Spearman correlation coefficient, a Kendall rank correlation, etc. may be used. In addition, various conventionally known correlation coefficients may be used.

For example, if R is 0.75 or more, the processor 120 may not deduct the score. On the other hand, when R is less than 0.75, the processor 120 may deduct the score. In this case, as R is smaller, the degree of deduction of the score may increase.

In addition, the processor 120 may determine whether or not the user has Parkinson's disease and the severity of the user's Parkinson's disease according to the score updated (eg, deducted or maintained) through the secondary test image.

As a result of the additional consideration of the secondary test image, the user's cognitive ability can be evaluated more systematically by considering the consistency of the user who did not accurately draw the clock in the primary test image.

In relation to this, FIG. 10 is an algorithm for explaining a method for diagnosing Parkinson's disease of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 10 , the processor 120 may obtain a first test image according to a user input (S1010), and may obtain a score based on the first test image (S1020).

Specifically, the processor 120 may add up the first score and the second score calculated according to the outline and/or the watch hand extracted from the first test image. In this case, the processor 120 may additionally add the third score calculated according to the configuration, position, arrangement order, etc. of the plurality of numbers.

If the score of the first test image is equal to or greater than the threshold (S1030 - Y), the processor 120 may determine that the user does not have Parkinson's disease (S1040).

On the other hand, when the score of the first test image is less than the threshold (S1030 - N), the processor 120 may acquire the second test image based on a separate user input (S1050).

To this end, the processor 120 may provide an additional guide to draw an additional watch figure (second test image).

Then, the processor 120 may update the score by comparing the first test image with the second test image (S1060).

Specifically, the processor 120 may compare the outline and/or watch hands of the watch included in the first test image with the outline and/or watch hands of the watch included in the second test image.

Also, the processor 120 may compare the plurality of numbers included in the first test image with the plurality of numbers included in the secondary text image.

In this case, as the difference according to the comparison result is greater, the processor 120 may deduct a greater score.

In addition, the processor 120 may determine whether or not Parkinson's disease is present based on the updated score and also identify the severity ( S1070 ).

Meanwhile, FIG. 11 is a block diagram for explaining the configuration and operation of an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 11 , the electronic device 100 may further include at least one of a display 130 , a user input unit 140 , and a communication unit 150 in addition to the memory 110 and the processor 120 .

The memory 110 may include at least one of the aforementioned artificial intelligence models.

The artificial intelligence model included in the memory 110 may include, for example, a plurality of neural network layers. Each of the plurality of neural network layers may have a plurality of weight values, and a neural network operation is performed through an operation result of a previous layer and a plurality of weight values. The plurality of weights of the plurality of neural network layers may be optimized by the training result of the artificial intelligence model.

Examples of artificial intelligence models included in the memory 110 include Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), BRDNN (Bidirectional Recurrent Deep Neural Network), GAN (Generateve Adhesive Network), and deep Q-networks, etc., but are not limited to the above-described example.

The artificial intelligence model included in the memory 110 may be trained through the processor 120 or trained in at least one external device.

For example, the processor 120 may train at least one artificial intelligence model using a learning algorithm such as supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

The display 130 is configured to display various contents or various user interfaces provided through the electronic device 100 .

The processor 120 may visually provide various guides for performing the CDT test through the display 130 .

The processor 120 may display at least one user interface (UI) screen for the user to draw a watch through the display 130 . In this case, the UI screen may be a blank screen, a screen on which the outline of the watch is drawn, a screen on which the outline of the watch and number dots are drawn, or a screen on which the outline and numbers of the watch are drawn, but is not limited thereto.

When a user input is received on the UI screen, the processor 120 may display at least a portion of a test image generated according to the user input through the display 130 .

As a result, a test image drawn in real time according to a received user input on the UI screen may be displayed.

The display 130 may be implemented as an LED, a liquid crystal display (LCD), a plasma display panel (PDP), an organic light emitting diode (OLED), a transparent OLED (TOLED), a micro LED, or the like, but is not limited thereto.

The user input unit 140 is configured to receive a user command or a user command. The user input unit 140 may be implemented as a touch sensor, a button, a camera, a microphone, or the like. The touch sensor may be implemented in various forms, such as a pressure-sensitive type, a capacitive type, an infrared type, and an ultrasonic type.

For example, the touch sensor of the user input unit 140 may constitute a touch screen included in the display 130 , and a user's input (ex. touch) to the touch screen is converted into coordinates within the touch screen, and the converted A result test image in which a point is drawn at each of the coordinates may be drawn.

Meanwhile, the user input unit 140 may include a hovering sensor, a proximity sensor, and the like, and a user input for drawing a test image may be received through the corresponding sensors.

The communication unit 150 may include a circuit as a configuration for the electronic device 100 to communicate with at least one external device.

Communication unit 150 is TCP/IP (Transmission Control Protocol/Internet Protocol), UDP (User Datagram Protocol), HTTP (Hyper Text Transfer Protocol), HTTPS (Secure Hyper Text Transfer Protocol), FTP (File Transfer Protocol), SFTP ( A communication protocol (protocol) such as Secure File Transfer Protocol) and MQTT (Message Queuing Telemetry Transport) may be used to transmit/receive various information to and from one or more external electronic devices.

To this end, the communication unit 150 may be connected to an external device based on a network implemented through wired communication and/or wireless communication. In this case, the communication unit 150 may be directly connected to an external device, or may be connected to an external electronic device through one or more external servers (eg, Internet Service Providers (ISPs)) that provide a network.

The network may be a personal area network (PAN), a local area network (LAN), a wide area network (WAN), etc. depending on the area or scale, and depending on the openness of the network, an intranet, It may be an extranet or the Internet.

Wireless communication includes long-term evolution (LTE), LTE Advance (LTE-A), 5th generation (5G) mobile communication, code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), WiBro (Wireless Broadband), GSM (Global System for Mobile Communications), DMA (Time Division Multiple Access), WiFi (Wi-Fi), WiFi Direct, Bluetooth, NFC (near field communication), at least one of the communication methods such as Zigbee may include

Wired communication may include at least one of communication methods such as Ethernet, optical network, USB (Universal Serial Bus), and ThunderBolt. Here, the communication unit 150 may include a network interface or a network chip according to the above-described wired/wireless communication method. Meanwhile, the communication method is not limited to the above-described example, and may include a communication method newly appearing according to the development of technology.

As an example, it is assumed that the electronic device 100 implemented as a server is connected to a user's smartphone.

In this case, while the application for the CDT test is executed through the smartphone, the electronic device 100 may provide a web page constituting at least a part of the application to the smartphone.

In this case, a test image may be generated according to the user's touch received through the touch screen of the smart phone, and the electronic device 100 as the server performs the processes of S310 to S350 described above to draw the test image by the user of the smart phone. can determine whether a person has Parkinson's disease.

In addition, the electronic device 100 may transmit information including a result of determining whether or not Parkinson's disease is present to the corresponding smart phone or at least one other external device.

Meanwhile, the various embodiments described above may be implemented in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof.

According to the hardware implementation, the embodiments described in the present disclosure are ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays) ), processors, controllers, micro-controllers, microprocessors, and other electrical units for performing other functions may be implemented using at least one.

In some cases, the embodiments described herein may be implemented by the processor itself. According to the software implementation, embodiments such as the procedures and functions described in this specification may be implemented as separate software modules. Each of the above-described software modules may perform one or more functions and operations described herein.

On the other hand, the computer instructions or computer program for performing the processing operation in the electronic device 100 according to various embodiments of the present disclosure described above is a non-transitory computer-readable medium. can be stored in When the computer instructions or computer program stored in the non-transitory computer-readable medium are executed by the processor of the specific device, the specific device performs the processing operation in the electronic device 100 according to various embodiments described above.

The non-transitory computer-readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, a cache, a memory, and can be read by a device. Specific examples of the non-transitory computer-readable medium may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In the above, preferred embodiments of the present disclosure have been illustrated and described, but the present disclosure is not limited to the specific embodiments described above, and is generally used in the technical field belonging to the present disclosure without departing from the gist of the present disclosure as claimed in the claims. Various modifications may be made by those having the knowledge of

100: electronic device 110: memory
120: processor

Claims (9)

In the method for determining Parkinson's disease in an electronic device,
extracting the outline of the watch from the test image generated based on the user input;
obtaining a first score based on the extracted contour;
extracting at least one watch hand included in the test image;
obtaining a second score based on the extracted clock hands; and
Recognizing whether the user has Parkinson's disease based on the first score and the second score; According to claim 1,
The step of obtaining the first score comprises:
and calculating the first score according to at least one of whether the contour is a closed curve and a slope for each section of the contour. According to claim 1,
The method for determining Parkinson's disease,
The method further comprising: inputting at least one sample image drawn by a normal person into an artificial intelligence model trained to extract the contour of the watch, and identifying a position in the sample image of the contour of the watch included in the sample image;
The step of obtaining the first score comprises:
and obtaining the first score based on a degree of matching between a position in the test image of the extracted contour and a position in the sample image of the identified contour. According to claim 1,
The Parkinson's disease determination method,
acquiring sensing data based on coordinates in the touch screen of the user input received on the touch screen; and
Further comprising; generating the test image based on the acquired sensing data;
The step of extracting the watch hands,
Identifies, among the sensed data, sensed data included in a time period in which a rate of change of coordinates in the touch screen according to the user input is equal to or greater than a preset threshold,
A method for determining Parkinson's disease in an electronic device, wherein the hour hand or minute hand corresponding to the identified sensing data is extracted from the test image. According to claim 1,
The method for determining Parkinson's disease,
Further comprising; identifying a plurality of numbers included in the test image based on the extracted contour;
Obtaining the second score comprises:
Identifies the number closest to the distance to the clock hand among the plurality of numbers,
A method for determining Parkinson's disease in an electronic device, wherein the second score is obtained according to whether the identified number matches a preset time. 6. The method of claim 5,
Obtaining the second score comprises:
extending the watch hand so that the watch hand approaches the extracted contour;
Identifies the number closest to the distance to the extended clock hand among the plurality of numbers,
The method for determining Parkinson's disease in an electronic device is to acquire the second score according to whether the identified number matches the preset time. According to claim 1,
The method for determining Parkinson's disease,
The method further comprising: inputting at least one sample image drawn by a normal person into an artificial intelligence model trained to extract a watch hand, and identifying a position of a watch hand included in the sample image in the sample image;
Obtaining the second score comprises:
and obtaining the second score based on a degree of matching between the extracted position in the test image of the watch hand and the identified position in the sample image of the watch hand. a memory storing one or more instructions; and
A processor that performs the method of claim 1 for determining Parkinson's disease by executing the one or more instructions. In the computer program stored in a computer-readable medium,
A computer program executed by a processor of an electronic device to cause the electronic device to perform the method of claim 1 for determining Parkinson's disease.

Images