WO2021039923A1

WO2021039923A1 - Abnormal state estimation device, sole state estimation device, system, and program

Info

Publication number: WO2021039923A1
Application number: PCT/JP2020/032439
Authority: WO
Inventors: 裕竹村; 里樹築地原; 光平渡邉; 啓也小森; 聖浦周; 美貴子原口
Original assignee: 学校法人東京理科大学; 医療法人社団誠馨会
Priority date: 2019-08-28
Filing date: 2020-08-27
Publication date: 2021-03-04
Also published as: JPWO2021039923A1

Abstract

According to the present invention, a sole imaging device (50) captures two sole images via a transparent support surface (52) on which the sole of the subject is placed while making the support surface (52) have different hardness at each time of the capturing. This sole state estimation device compares the two sole images to estimate the abnormal state of the subject or the hardness state of the sole.

Description

Abnormal condition estimator, sole condition estimator, system, and program

This disclosure relates to an abnormal state estimation device, a sole state estimation device, a system, and a program.

Conventionally, a diabetic foot inspection device has been known (Japanese Patent Publication No. 2005-533543). Further, a system for inspecting the skin condition of the sole of the foot is known (Japanese Unexamined Patent Publication No. 2013-090928).

Also, the "nerve conduction test DPN (diabetic peripheral neuropathy) check" manufactured by Fukuda Colin Co., Ltd. is known. In general, a method of measuring nerve conduction velocity is useful for determining neuropathy.

The device used for the above "nerve conduction test DPN (diabetic peripheral neuropathy) check" is a device that measures the nerve conduction velocity, specializing in the sural nerve of the foot. However, there is a problem that it is invasive and takes time and effort.

The present disclosure has been made in view of the above circumstances, and is a non-invasive and abnormal state estimation device, a sole state estimation device, a system, which can easily estimate the abnormal state or the sole state of the subject. And the purpose of providing the program.

The first aspect of the present disclosure is an abnormal state estimation device, which is an image taken through the support surface when the sole of the subject is placed on each of two transparent support surfaces having different hardness. It is configured to include an image acquisition unit that acquires two foot sole images, and an abnormal state estimation unit that compares the two foot sole images and estimates the abnormal state of the subject.

A second aspect of the present disclosure is an abnormal state estimation system, which has a transparent support surface on which the sole of a subject is placed, and the hardness of the support surface is made different, and the support surface is passed through the support surface. It is configured to include a sole imaging device for capturing one foot image and the above-mentioned abnormal state estimation device.

A third aspect of the present disclosure is an abnormal state estimation program, in which a computer is photographed through the support surface when the sole of the subject is placed on each of two transparent support surfaces having different hardness. This is a program for functioning as an image acquisition unit that acquires two foot images, which are the images obtained, and an abnormal state estimation unit that estimates the abnormal state of the subject by comparing the two foot images.

According to the present disclosure, the image acquisition unit is an image of two feet taken through the support surface when the sole of the subject is placed on each of the two transparent support surfaces having different hardness. Get the back image. Then, the abnormal state estimation unit compares the two foot sole images and estimates the abnormal state of the subject.

In this way, when the sole of the subject is placed on each of the two transparent support surfaces having different hardness, the two sole images, which are images taken through the support surface, are compared. The abnormal state of the subject is estimated. As a result, the abnormal state of the subject can be easily estimated in a non-invasive manner.

A fourth aspect of the present disclosure is a foot sole state estimation device, which is photographed through the support surface when the sole of the subject is placed on each of two transparent support surfaces having different hardness. It is configured to include an image acquisition unit that acquires two foot images, which are images, and a sole state estimation unit that compares the two foot images and estimates the hardness state of the sole of the subject. Has been done.

A fifth aspect of the present disclosure is a foot sole state estimation system, which has a transparent support surface on which the sole of a subject is placed, and has different hardness of the support surface through the support surface. It is configured to include a sole photographing device for capturing two sole images and the sole state estimating device.

A sixth aspect of the present disclosure is a foot condition estimation program, in which a computer is placed through the support surface when the sole of the subject is placed on each of two transparent support surfaces having different hardness. It functions as an image acquisition unit that acquires two foot images that are captured images, and a foot condition estimation unit that estimates the hardness state of the sole of the subject by comparing the two foot images. It is a program for.

According to the present disclosure, the image acquisition unit is an image of two feet taken through the support surface when the sole of the subject is placed on each of the two transparent support surfaces having different hardness. Get the back image. Then, the foot sole state estimation unit compares the two sole images and estimates the hardness state of the sole of the subject.

In this way, when the sole of the subject is placed on each of the two transparent support surfaces having different hardness, the two sole images, which are images taken through the support surface, are compared. The hardness state of the sole of the subject is estimated. Thereby, the hardness state of the sole of the subject can be easily estimated in a non-invasive manner.

As described above, according to one aspect of the present disclosure, it is possible to obtain an effect that the abnormal state or the sole state of the subject can be easily estimated in a non-invasive manner.

It is a figure for demonstrating the difference of the contact area of the skin on a hard floor. It is a figure for demonstrating the difference of the contact area of the skin on a soft floor. It is a block diagram which shows the abnormal state estimation system which concerns on 1st Embodiment of this disclosure. It is a front view which shows the structure of the foot sole photographing apparatus which concerns on 1st to 3rd Embodiment of this disclosure. It is a side view which shows the structure of the foot sole photographing apparatus which concerns on 1st to 3rd Embodiment of this disclosure. It is a schematic block diagram of an example of the computer functioning as the abnormal state estimation device and the sole state estimation device according to the first to third embodiments of the present disclosure. It is a functional block diagram which shows the abnormal state estimation apparatus which concerns on 1st and 2nd Embodiment of this disclosure. It is a figure which shows an example of the sole image. It is a figure which shows an example of a binarized image. It is a figure which shows an example of the binarized image after noise removal. It is a figure for demonstrating the method of dividing a binarized image into three. It is a figure for demonstrating the contact area of the skin on a hard floor, and the contact area of the skin on a soft floor. It is a flowchart which shows the content of the abnormality state estimation processing routine of the abnormality state estimation apparatus which concerns on 1st Embodiment of this disclosure. It is a functional block diagram which shows the foot sole state estimation apparatus which concerns on 3rd Embodiment of this disclosure. It is a flowchart which shows the content of the sole state estimation processing routine of the foot sole state estimation apparatus which concerns on 3rd Embodiment of this disclosure. It is a graph which shows the average value of the area change rate of each subject group. It is a graph which shows the correlation of the estimated value and the measured value of the severity of diabetes when only the diabetic patient is analyzed. It is a graph which shows the correlation of the estimated value and the measured value of the severity of diabetes when the whole subject is analyzed.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The following is an example of applying the present disclosure to an abnormal condition estimation device that estimates the presence or absence of diabetes and the presence or absence of neuropathy in diabetes of a subject, or estimates the severity of diabetes and the degree of neuropathy in diabetes of a subject. Explain to.

<Outline of Embodiments of the present disclosure>
In the embodiment of the present disclosure, the hardness of the skin is detected from the contact area with the floor. Specifically, as shown in FIG. 1, since the skin is crushed on a hard floor, the harder the skin, the smaller the contact area. On the other hand, as shown in FIG. 2, on a soft floor, the floor is crushed, so that the difference in the ground contact area due to the hardness of the skin is small.

From the above, the harder the sole of the foot, the greater the increase or decrease in the ground contact area due to the change in floor hardness. For this reason, diabetic neuropathy patients with high sole hardness have a large area change rate due to the difference in floor hardness.

In addition, since skin hardening due to neuropathy often worsens from one leg, diabetic neuropathy patients have a large difference in area change rate due to the difference in floor hardness.

Also, with the progression of neuropathy in diabetes, there is a tendency for calluses (bench / octopus) and corns (cagan / corn) to form on the soles of the feet. In addition, octopus and corns due to neuropathy often deteriorate from one foot, octopus and corns occur in the anterior or posterior part of the sole, and the hardness of the octopus or corns is significantly increased. As a result, the hardness of the sole of the foot becomes harder, the standing position changes, and the center of gravity of the subject deviates from the center, so that a difference appears in the ground contact area in the front, back, left, and right. That is, in diabetic neuropathy patients, the difference between the left and right of the area change rate due to the difference in the hardness of the front and rear floors of the sole is large.

Therefore, in the embodiment of the present disclosure, diabetes or diabetic neuropathy is estimated by comparing foot images in a stationary standing state taken through transparent floor surfaces having different hardness.

Specifically, by standing the subject on a transparent acrylic plate and using a mirror, the sole of the foot is photographed through the acrylic plate and the mirror. At this time, the acrylic plate is used as a hard floor, and the soft floor is reproduced with a transparent silicone sheet. A transparent glass plate may be used instead of the acrylic plate.

Also, the captured sole image is binarized and the contact area of the sole is calculated. Based on the left-right difference in the area change rate due to the hard and soft floor, the left-right difference in the area change rate of the entire sole, the left-right difference in the area change rate in the front part of the sole, and the left-right difference in the area change rate in the rear part of the sole. Make estimates for diabetes and diabetic neuropathy. At this time, multiple logistic regression analysis is used to estimate the presence or absence of diabetes and the presence or absence of diabetic neuropathy, and multiple regression analysis is used to estimate the severity of diabetes and the degree of diabetic neuropathy.

[First Embodiment]
<Structure of the abnormal state estimation system according to the first embodiment of the present disclosure>
As shown in FIG. 3, the abnormal state estimation system 100 of the first embodiment of the present disclosure includes an abnormal state estimation device 10 and a foot sole photographing device 50. The abnormal state estimation device 10 and the foot sole photographing device 50 are connected by wire or wirelessly. The abnormal state estimation device 10 and the foot sole photographing device 50 may be connected to each other via a network such as a LAN (Local Area Network) or the Internet.

As shown in FIGS. 4A and 4B, the foot sole photographing device 50 has a transparent support surface 52 on which the sole of the subject is placed, and is provided by the camera 54 through the mirror 56 and the support surface 52. An image is taken and transmitted to the abnormal state estimation device 10. The support surface 52 is provided with a light source 58 so as to irradiate the inside of the support surface 52. Instead of shooting from below through the mirror 56, the sole of the foot may be scanned by a line sensor of a thin device such as a scanner to capture an image of the sole of the foot. LED lighting may be used as the light source 58.

The camera 54 captures an image of the sole of the foot when the transparent silicone sheet 60 is placed on the support surface 52, and the camera 54 captures the foot when the silicone sheet 60 is not placed on the support surface 52. Take a back image. As a result, the hardness of the support surface 52 can be made different, and two foot sole images can be taken through the support surface 52.

It should be noted that two types of silicone sheets 60 having different hardness may be prepared, the hardness of the support surface 52 may be different, and two foot sole images may be taken through the support surface 52.

Further, instead of switching the presence or absence of the silicone sheet 60, the foot sole photographing device 50 in which the silicone sheet 60 is placed on the support surface 52 and the foot sole photographing device 50 in which the silicone sheet 60 is not placed on the support surface 52. The two devices of and may be provided.

FIG. 5 is a block diagram showing a hardware configuration of the abnormal state estimation device 10 according to the first embodiment.

As shown in FIG. 5, the abnormal state estimation device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and communication. It has an interface (I / F) 17. Each configuration is communicably connected to each other via a bus 19.

The CPU 11 is a central arithmetic processing unit that executes various programs and controls each part. That is, the CPU 11 reads the program from the ROM 12 or the storage 14, and executes the program using the RAM 13 as a work area. The CPU 11 controls each of the above configurations and performs various arithmetic processes according to the program stored in the ROM 12 or the storage 14. In the present embodiment, the ROM 12 or the storage 14 stores an abnormal state estimation program for estimating the abnormal state of the subject. The abnormal state estimation program may be one program, or may be a program group composed of a plurality of programs or modules.

ROM 12 stores various programs and various data. The RAM 13 temporarily stores a program or data as a work area. The storage 14 is composed of an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various programs including an operating system and various data.

The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

The display unit 16 is, for example, a liquid crystal display and displays various types of information. The display unit 16 may adopt a touch panel method and function as an input unit 15.

The communication interface 17 is an interface for communicating with other devices including the foot sole photographing device 50, and for example, standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark) are used.

Next, the functional configuration of the abnormal state estimation device 10 will be described. FIG. 6 is a block diagram showing an example of the functional configuration of the abnormal state estimation device 10.

Functionally, the abnormal state estimation device 10 includes an image acquisition unit 30, a binarized image creation unit 32, and an abnormal state estimation unit 34, as shown in FIG.

The image acquisition unit 30 acquires two foot sole images taken by the foot sole photographing device 50.

The binarized image creation unit 32 binarizes each of the two acquired foot sole images and removes noise.

Specifically, the binarized image creation unit 32 binarizes each pixel with respect to the sole image as shown in FIG. 7A by using a threshold value, so that the binarized image as shown in FIG. 7B is obtained. To create. Then, the binarized image creation unit 32 acquires a binarized image as shown in FIG. 7C by removing noise.

Further, the binarized image creation unit 32 divides the binarized images obtained for each of the two sole images into three in the length direction of the sole as shown in FIG. 8, and the sole is divided into three. A binarized image of the anterior part and a binarized image of the posterior part of the sole are obtained.

The abnormal state estimation unit 34 estimates the presence or absence of diabetes and the presence or absence of neuropathy in diabetes as the abnormal state of the subject based on the rate of change in the area of the sole region in the two sole images.

Specifically, the abnormal state estimation unit 34 uses the binarized image to determine the total area of the sole region, the area of each sole region of the right foot and the left foot, and the front of each sole region of the right foot and the left foot. The area of the part and the area of the rear part of each sole area of the right foot and the left foot are calculated. In addition, the abnormal state estimation unit 34
Rate of change in the total area of the sole area in the two sole images,
Difference between the rate of change in the area of the sole area of the right foot in the two sole images and the rate of change in the area of the sole area of the left foot in the two sole images,
The difference between the rate of change in the area of the front part of the sole area of the right foot in the two sole images and the rate of change in the area of the front part of the sole area of the left foot in the two sole images, and the two sole images. Based on the difference between the rate of change in the posterior area of the sole region of the right foot and the rate of change in the posterior area of the sole region of the left foot in the two sole images, the presence or absence of diabetes and the presence or absence of neuropathy in diabetes To estimate.

For example, the rate of change A of the area of the entire sole region in the two sole images is calculated by the following formula.

A = (SH) / (S + H)

However, S is the area of the entire sole region (see the thin dot portion in FIG. 9) obtained from the sole image taken when the hardness of the support surface 52 is soft. H is the area of the entire sole region (see the dark dot portion in FIG. 9) obtained from the sole image taken when the hardness of the support surface 52 is hard.

Further, the laterality D, which is the difference between the rate of change RA of the area of the sole region of the right foot in the two sole images and the rate of change LA of the area of the sole region of the left foot in the two sole images, is as follows. Obtained by the formula.

D = | RA-LA |
RA = (RS-RH) / (RS + RH)
LA = (LS-LH) / (LS + LH)

However, RS is the area of the entire sole region of the right foot obtained from the sole image taken when the hardness of the support surface 52 is soft. RH is the area of the entire sole region of the right foot obtained from the sole image taken when the hardness of the support surface 52 is hard. LS is the area of the entire sole region of the left foot obtained from the sole image taken when the hardness of the support surface 52 is soft. LH is the area of the entire sole region of the left foot obtained from the sole image taken when the hardness of the support surface 52 is hard.

In addition, the difference between the rate of change RF of the front area of the sole region of the right foot in the two sole images and the rate of change LF of the area of the front part of the sole region of the left foot in the two sole images is left and right. The difference DF is calculated by the following formula.

DF = | RF-LF |
RF = (RFS-RFH) / (RFS + RFH)
LF = (LFS-LFH) / (LFS + LFH)

However, RFS is the area of the front part of the sole region of the right foot obtained from the sole image taken when the hardness of the support surface 52 is soft. RFH is the area of the front part of the sole region of the right foot obtained from the sole image taken when the hardness of the support surface 52 is hard. LFS is the area of the front part of the sole region of the left foot obtained from the sole image taken when the hardness of the support surface 52 is soft. LFH is the area of the front part of the sole region of the left foot obtained from the sole image taken when the hardness of the support surface 52 is hard.

In addition, the laterality DB, which is the difference between the rate of change RB of the area of the rear part of the sole area of the right foot in the two sole images and the rate of change LB of the area of the rear part of the sole area of the left foot in the two sole images. Is calculated by the following formula.

DB = | RB-LB |
RB = (RBS-RBH) / (RBS + RBH)
LB = (LBS-LBH) / (LBS + LBH)

However, the RBS is the area of the rear part of the sole region of the right foot obtained from the sole image taken when the hardness of the support surface 52 is soft. RBH is the area of the rear part of the sole region of the right foot obtained from the sole image taken when the hardness of the support surface 52 is hard. The LBS is the area of the rear part of the sole region of the left foot obtained from the sole image taken when the hardness of the support surface 52 is soft. LBH is the area of the rear part of the sole region of the left foot obtained from the sole image taken when the hardness of the support surface 52 is hard.

Then, using the above-mentioned rate of change A, laterality D, laterality DF, and laterality DB as explanatory variables, and using the regression equation of multiple logistic regression shown in the following equation, the presence or absence of diabetes and the presence or absence of neuropathy in diabetes To estimate.

y is the objective variable and is the probability of taking 1. x ₁ , ..., X _p are explanatory variables. b ₀ , ···, b _p are coefficients. _{The coefficients b 0} , ..., B _p of the regression equation for determining the presence or absence of diabetes are obtained in advance from the data (change rate A, left-right difference D, left-right difference DF, left-right difference DB) in which the presence or absence of diabetes is known. deep. In addition, the coefficient left-right difference of the regression equation for determining the presence or absence of neuropathy in diabetes is obtained in advance from data (change rate A, left-right difference D, left-right difference DF, left-right difference DB) in which the presence or absence of neuropathy in diabetes is known. Keep it.

<Operation of abnormal state estimation system>
Next, the operation of the abnormal state estimation system 100 according to the first embodiment will be described.

In the foot sole imaging device 50, when the subject puts his / her foot on the support surface 52 with the silicone sheet 60 placed on the support surface 52 and is in a stationary standing state, the foot is taken by the camera 54. The sole image is taken, and the sole image is transmitted to the abnormal state estimation device 10.

Further, in the foot sole photographing apparatus 50, when the subject puts his / her foot on the support surface 52 without placing the silicone sheet 60 on the support surface 52 and becomes a stationary standing state, the camera 54 , The sole image is taken, and the sole image is transmitted to the abnormal state estimation device 10.

At this time, the abnormal state estimation device 10 executes the abnormal state estimation processing routine shown in FIG.

First, in step S100, the image acquisition unit 30 acquires two foot sole images received from the foot sole photographing device 50.

In step S102, the binarized image creation unit 32 binarizes each of the two acquired sole images and removes noise. Further, the binarized image creation unit 32 divides the binarized images obtained for each of the two sole images into three in the length direction of the sole.

In step S104, the abnormal state estimation unit 34 uses the binarized image to determine the total area of the sole region, the area of each sole region of the right foot and the left foot, and the front portion of each sole region of the right foot and the left foot. And the area of the rear part of each sole area of the right foot and the left foot.

In step S106, the abnormal state estimation unit 34
Rate of change in the total area of the sole area in the two sole images,
Difference between the rate of change in the area of the sole area of the right foot in the two sole images and the rate of change in the area of the sole area of the left foot in the two sole images,
The difference between the rate of change in the area of the front part of the sole area of the right foot in the two sole images and the rate of change in the area of the front part of the sole area of the left foot in the two sole images, and the two sole images. Based on the difference between the rate of change in the posterior area of the sole region of the right foot and the rate of change in the posterior area of the sole region of the left foot in the two sole images, the presence or absence of diabetes and the presence or absence of neuropathy in diabetes To estimate. The abnormal state estimation unit 34 displays the estimation result on the display unit 16 and ends the abnormal state estimation processing routine.

As described above, according to the abnormal state estimation system according to the first embodiment of the present disclosure, the foot sole of the subject is supported when the sole of the subject is placed on each of the two transparent support surfaces having different hardness. The presence or absence of diabetes in the subject and the presence or absence of neuropathy in diabetes are estimated by comparing the two sole images, which are images taken through the surface. Thereby, the presence or absence of diabetes and the presence or absence of neuropathy in diabetes can be easily estimated in a non-invasive manner.

[Second Embodiment]
Next, the abnormal state estimation system according to the second embodiment will be described. Since the configuration is the same as that of the abnormal state estimation system according to the first embodiment, the same reference numerals are given and the description thereof will be omitted.

The abnormal state estimation unit 34 of the abnormal state estimation device 10 estimates the severity of diabetes and the degree of neuropathy in diabetes as the abnormal state of the subject based on the rate of change in the area of the sole region in the two sole images. To do.

Specifically, the abnormal state estimation unit 34 uses the binarized image obtained from each of the two sole images, as in the first embodiment, and uses the area of the entire sole region, the right foot and the left foot. The area of each sole area of the foot, the area of the front part of each sole area of the right foot and the left foot, and the area of the rear part of each sole area of the right foot and the left foot are obtained. Further, the abnormal state estimation unit 34 obtains the rate of change A, the laterality D, the laterality DF, and the laterality DB, as in the first embodiment. Then, the abnormal state estimation unit 34 uses the above-mentioned rate of change A, laterality D, laterality DF, and laterality DB as explanatory variables, and uses the regression equation of the multiple regression analysis shown in the following equation to determine the severity of diabetes. And estimate the degree of neuropathy in diabetes.

y is the objective variable and is the probability of taking 1. x ₁ , ..., X _p are explanatory variables. b ₀ , ···, b _p are coefficients. Coefficient b ₀ of the regression formula for determining the severity of _diabetes, · · ·, b _p is severity known data diabetes (rate of change A, laterality D, laterality DF, laterality DB) in advance from the I'll ask for it. The coefficient b ₀ of the regression formula for determining the degree of neuropathy in _diabetes, · · ·, b _p is degree known data (the change rate A of neuropathy in diabetes, laterality D, laterality DF, It is obtained in advance from the left-right difference DB).

Since other configurations and operations of the abnormal state estimation system according to the second embodiment are the same as those of the first embodiment, the description thereof will be omitted.

As described above, according to the abnormal state estimation system according to the second embodiment of the present disclosure, the foot sole of the subject is supported when the sole of the subject is placed on each of the two transparent support surfaces having different hardness. The severity of diabetes in the subject and the degree of neuropathy in diabetes are estimated by comparing the two foot images, which are images taken through the surface. This makes it possible to estimate the severity of diabetes and the degree of neuropathy in diabetes of a subject in a non-invasive manner and easily.

[Third Embodiment]
Next, the foot sole state estimation system according to the third embodiment will be described. The parts having the same configuration as the abnormal state estimation system according to the first embodiment are designated by the same reference numerals and the description thereof will be omitted.

<Overview>
The third embodiment is different from the first embodiment in that the hardness state of the sole of the subject is estimated.

The foot sole state estimation system of the third embodiment of the present disclosure includes the foot sole state estimation device 310 shown in FIG. 11 and the foot sole imaging device 50. The foot sole state estimation device 310 and the foot sole photographing device 50 are connected by wire or wirelessly. The foot sole state estimation device 310 and the foot sole photographing device 50 may be connected via a network such as LAN or the Internet.

As shown in FIG. 5, the sole state estimation device 310 includes a CPU 11, a ROM 12, a RAM 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each configuration is communicably connected to each other via a bus 19.

The CPU 11 is a central arithmetic processing unit that executes various programs and controls each part. That is, the CPU 11 reads the program from the ROM 12 or the storage 14, and executes the program using the RAM 13 as a work area. The CPU 11 controls each of the above configurations and performs various arithmetic processes according to the program stored in the ROM 12 or the storage 14. In the present embodiment, the ROM 12 or the storage 14 stores a foot sole state estimation program for estimating the foot sole state of the subject. The sole state estimation program may be one program, or may be a program group composed of a plurality of programs or modules.

Next, the functional configuration of the sole state estimation device 310 will be described. FIG. 11 is a block diagram showing an example of the functional configuration of the sole state estimation device 310.

Functionally, as shown in FIG. 11, the foot sole state estimation device 310 includes an image acquisition unit 30, a binarized image creation unit 32, and a foot sole state estimation unit 334.

The sole state estimation unit 334 estimates the hardness state of the sole of the subject based on the rate of change in the area of the sole region in the two sole images.

Specifically, the sole state estimation unit 334 uses the binarized image obtained from each of the two sole images to determine the total area of the sole region, the area of each sole region of the right foot and the left foot, and the area of each sole region of the right foot and the left foot. The area of the front part of each sole area of the right foot and the left foot, and the area of the rear part of each sole area of the right foot and the left foot are calculated. In addition, the sole state estimation unit 334 estimates the hardness state of the entire sole of the subject based on the rate of change in the area of the entire sole region in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the sole of the right foot of the subject based on the rate of change in the area of the sole region of the right foot in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the sole of the left foot of the subject based on the rate of change in the area of the sole region of the left foot in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the front part of the sole of the right foot of the subject based on the rate of change in the area of the front part of the sole region of the right foot in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the front part of the sole of the left foot of the subject based on the rate of change in the area of the front part of the sole region of the left foot in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the rear part of the sole of the right foot of the subject based on the rate of change in the area of the rear part of the sole region of the right foot in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the rear part of the left foot of the subject based on the rate of change in the area of the rear part of the sole region of the left foot in the two sole images.

In the above estimation of the hardness state, it is estimated that the larger the rate of change, the harder the sole of the foot.

<Operation of sole condition estimation system>
Next, the operation of the foot sole state estimation system according to the third embodiment will be described.

In the foot sole imaging device 50, when the subject puts his / her foot on the support surface 52 with the silicone sheet 60 placed on the support surface 52 and is in a stationary standing state, the foot is taken by the camera 54. The sole image is taken, and the sole image is transmitted to the sole state estimation device 310.

Further, in the foot sole photographing apparatus 50, when the subject puts his / her foot on the support surface 52 without placing the silicone sheet 60 on the support surface 52 and becomes a stationary standing state, the camera 54 , The sole image is taken, and the sole image is transmitted to the sole state estimation device 310.

At this time, the sole state estimation device 310 executes the sole state estimation processing routine shown in FIG. The same processing as the abnormal state estimation processing routine in the first embodiment is designated by the same reference numerals, and detailed description thereof will be omitted.

First, in step S100, the image acquisition unit 30 acquires two foot sole images taken by the foot sole photographing device 50.

In step S104, the sole state estimation unit 334 uses the binarized image to determine the total area of the sole region, the area of each sole region of the right foot and the left foot, and the front of each sole region of the right foot and the left foot. The area of the part and the area of the rear part of each sole area of the right foot and the left foot are calculated.

In step S300, the sole state estimation unit 334 estimates the hardness state of the entire sole of the subject based on the rate of change in the area of the entire sole region in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the sole of the right foot of the subject based on the rate of change in the area of the sole region of the right foot in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the sole of the left foot of the subject based on the rate of change in the area of the sole region of the left foot in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the front part of the sole of the right foot of the subject based on the rate of change in the area of the front part of the sole region of the right foot in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the front part of the sole of the left foot of the subject based on the rate of change in the area of the front part of the sole region of the left foot in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the rear part of the sole of the right foot of the subject based on the rate of change in the area of the rear part of the sole region of the right foot in the two sole images. In addition, the sole state estimation unit 334 estimates the hardness state of the rear part of the left foot of the subject based on the rate of change in the area of the rear part of the sole region of the left foot in the two sole images. The sole state estimation unit 334 displays the estimation result on the display unit 16 and ends the foot sole state estimation processing routine.

As described above, according to the foot condition estimation system according to the third embodiment of the present disclosure, when the sole of the subject is placed on each of two transparent support surfaces having different hardness. The hardness state of the sole of the subject is estimated by comparing the two sole images, which are images taken through the support surface. Thereby, the hardness state of the sole of the subject can be easily estimated in a non-invasive manner.

<Experimental example 1>
In order to explain the effectiveness of the method described in the first embodiment described above, the results of comparing the area change rates of each subject group will be described. As shown in FIG. 13, for each of the subject group consisting of healthy subjects, the subject group consisting of diabetic patients without neuropathy, and the subject group consisting of diabetic patients with neuropathy.
The average value of the rate of change in the area of the entire sole area in the two sole images,
The average value of the laterality of the rate of change in the area of the sole area of the right foot and the left foot in the two sole images,
The mean value of the laterality of the lateral difference in the area of the anterior part of the sole area of the right foot and the left foot in the two sole images, and the rate of change of the area of the posterior part of the sole area of the right foot and the left foot in the two sole images. Shows the result of comparing the average value of the left-right difference. As a result of the T-test, the left-right difference in the area change rate of the entire sole area in the two sole images, the lateral difference in the area change rate of the right foot and left foot sole areas in the two sole images, and the two soles. A significant difference was observed between the subject groups in the laterality of the rate of change in the area of the posterior part of the sole region of the right foot and the left foot in the image. Therefore, it can be seen that the presence or absence of diabetes and the presence or absence of neuropathy in diabetes can be estimated by multiple logistic regression using the above rate of change and laterality.

<Experimental example 2>
In order to explain the effectiveness of the method described in the second embodiment described above, the result of examining the correlation between the estimated value of the severity of diabetes and the measured value will be described. Specifically, the method described in the second embodiment described above was used for the estimated value of the severity of diabetes, and the actual diagnosis result was used for the measured value of the severity of diabetes. In addition, the analysis target is only diabetic patients, the objective variable is the scoring result of severity (0 to 12 points), and the explanatory variables are BMI, age, area change rate of the entire sole region, and sole region. The correlation between the estimated value of the severity of diabetes and the measured value was investigated as the laterality of the area change rate and the laterality of the area change rate of the posterior part of the sole region. A significant positive correlation was found as shown in FIG. 14 (correlation coefficient is 0.581).

In addition, the correlation between the estimated value of the severity of diabetes and the measured value was examined in the same way for all subjects to be analyzed. A significant positive correlation was found as shown in FIG. 15 (correlation coefficient is 0.624).

From this, it can be seen that the severity of diabetes and the degree of neuropathy in diabetes can be estimated by multiple regression analysis using the above rate of change and laterality.

Note that this disclosure is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of this disclosure.

<Modification example 1>
In the above embodiment, the case of estimation using multiple logistic regression analysis and multiple regression analysis has been described as an example, but the present invention is not limited to this. For example, the evaluation value may be obtained based on the following formula and estimated according to the evaluation value.

Evaluation value = rate of change in the area of the entire sole area A + score 1 + score 2 + score 3 + score 4

Here, the score 1 is the difference between the rate of change RA of the area of the sole region of the right foot in the two sole images and the rate of change LA of the area of the sole region of the left foot in the two sole images. When D is 0.6 or more, it becomes 0.1, and in other cases, it becomes 0. Left-right difference DF, which is the difference between the rate of change RF of the area of the front part of the sole area of the right foot in the two sole images and the rate of change LF of the area of the front part of the sole area of the left foot in the two sole images. However, if it is 0.2 or more, the score 2 becomes 0.1, and in other cases, the score 2 becomes 0. The laterality DB, which is the difference between the rate of change RB of the area of the rear part of the sole area of the right foot in the two sole images and the rate of change LB of the area of the rear part of the sole area of the left foot in the two sole images, is If it is 0.1 or more, the score 3 becomes 0.1, and in other cases, the score 3 becomes 0. The score 4 is 0.5 if it is Charcot's foot and 0 if it is not Charcot's foot. In the case of Charcot's foot, scores 1 to 3 are set to 0. Here, Charcot's foot is a flat foot caused by severe neuropathy in diabetes and destruction of the arch joint.

<Modification 2>
In addition, using two foot images as inputs, a neural network that estimates the presence or absence of diabetes, the presence or absence of neuropathy in diabetes, the severity of diabetes, or the degree of neuropathy in diabetes is used to determine the presence or absence of diabetes and nerves in diabetes. The presence or absence of disability, the severity of diabetes, or the degree of neuropathy in diabetes may be estimated.

<Modification example 3>
In addition, the case of estimating diabetes or neuropathy in diabetes as an abnormal state of the subject has been described as an example, but the present invention is not limited to this. If the abnormal state can be estimated by comparing the two sole images, the abnormal state of the subject other than diabetes and neuropathy in diabetes may be estimated.

<Modification example 4>
Further, the case where the system includes one foot sole imaging device and one abnormal state estimation device or foot sole state estimation device has been described as an example, but the present invention is not limited to this. For example, it may be a system including a plurality of foot sole imaging devices and one abnormal state estimation device or foot sole state estimation device configured as a server. In this case, if the thickness and hardness of the transparent silicone sheet differ depending on the foot sole imaging device, the sole imaging device may transmit the thickness and hardness of the transparent silicone sheet together with the sole image. Good.

<Modification 5>
In addition, when the rate of change A, laterality D, laterality DF, and laterality DB are used to estimate the presence or absence of diabetes, the presence or absence of neuropathy in diabetes, the severity of diabetes, or the degree of neuropathy in diabetes. Although explained as an example, the present invention is not limited to this. For example, at least one of rate of change A, laterality D, laterality DF, and laterality DB can be used to determine the presence or absence of diabetes, the presence or absence of neuropathy in diabetes, the severity of diabetes, or the degree of neuropathy in diabetes. You may try to estimate.

<Modification 6>
Further, in the above-described first embodiment, the case of estimating the presence or absence of diabetes and the presence or absence of neuropathy in diabetes has been described as an example, but the present invention is not limited to this. Either the presence or absence of diabetes and the presence or absence of neuropathy in diabetes may be estimated. Further, in the second embodiment described above, the case of estimating the severity of diabetes and the degree of neuropathy in diabetes has been described as an example, but the present invention is not limited thereto. Either the severity of diabetes or the degree of neuropathy in diabetes may be estimated.

<Modification 7>
Further, in the above-described first embodiment, the case where LED lighting is used as the light source 58 has been described as an example, but the present invention is not limited to this. For example, illumination having a specific wavelength such as near infrared may be used as the light source 58. For example, a specific wavelength at which the presence or absence of diabetes and the presence or absence of neuropathy in diabetes are more prominent may be obtained in advance, and the illumination of the specific wavelength may be used as the light source 58.

Various processors other than the CPU may execute various processes executed by the CPU reading software (program) in the above embodiment. In this case, the processors include PLD (Programmable Logic Device) whose circuit configuration can be changed after the manufacture of FPGA (Field-Programmable Gate Array), and ASIC (Application Specific Integrated Circuit) for executing ASIC (Application Special Integrated Circuit). An example is a dedicated electric circuit or the like, which is a processor having a circuit configuration designed exclusively for the purpose. Further, the abnormal state estimation process or the sole state estimation process may be executed by one of these various processors, or a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs, etc.). And a combination of CPU and FPGA, etc.). Further, the hardware structure of these various processors is, more specifically, an electric circuit in which circuit elements such as semiconductor elements are combined.

Further, in each of the above embodiments, the mode in which the abnormal state estimation program or the sole state estimation program is stored (installed) in the storage 14 in advance has been described, but the present invention is not limited to this. The program is a non-temporary storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versailles Disk Online Memory), and a USB (Universal Serial Bus) memory. It may be provided in the form. Further, the program may be downloaded from an external device via a network.

Regarding the above embodiments, the following additional notes will be further disclosed.

(Appendix 1)
With memory
With at least one processor connected to the memory
Including
The processor
When the sole of the subject was placed on each of the two transparent support surfaces having different hardness, two sole images, which are images taken through the support surface, were acquired.
An abnormal state estimation device that estimates the abnormal state of the subject by comparing the two foot sole images.

(Appendix 2)
A non-temporary storage medium that stores a program that can be executed by a computer to perform anomalous state estimation processing.
The abnormal state estimation process is
When the sole of the subject was placed on each of the two transparent support surfaces having different hardness, two sole images, which are images taken through the support surface, were acquired.
A non-temporary storage medium that estimates the abnormal state of the subject by comparing the two foot sole images.
(Appendix 3)
With memory
With at least one processor connected to the memory
Including
The processor
When the sole of the subject was placed on each of the two transparent support surfaces having different hardness, two sole images, which are images taken through the support surface, were acquired.
A foot sole state estimation device that estimates the hardness state of the sole of the subject by comparing the two sole images.

(Appendix 4)
A non-temporary storage medium that stores a program that can be executed by a computer to execute the sole state estimation process.
The sole state estimation process is
When the sole of the subject was placed on each of the two transparent support surfaces having different hardness, two sole images, which are images taken through the support surface, were acquired.
A non-temporary storage medium for estimating the hardness state of the sole of the subject by comparing the two images of the sole of the foot.

The entire disclosure of Japanese application 2019-155407 is incorporated herein by reference in its entirety.

All documents, patent applications, and technical standards described herein are to the same extent as if the individual documents, patent applications, and technical standards were specifically and individually stated to be incorporated by reference. Incorporated herein by reference.

Claims

An image acquisition unit that acquires two foot images, which are images taken through the support surface when the sole of the subject is placed on each of the two transparent support surfaces having different hardness.
An abnormal state estimation unit that estimates the abnormal state of the subject by comparing the two foot sole images,
Abnormal state estimator including.
The abnormal state estimation device according to claim 1, wherein the abnormal state estimation unit estimates the abnormal state of the subject based on the rate of change in the area of the sole region in the two sole images.
The abnormal state estimation unit is based on the difference between the rate of change in the area of the sole region of the right foot in the two sole images and the rate of change in the area of the sole region of the left foot in the two sole images. The abnormal state estimation device according to claim 2, wherein the abnormal state of the subject is estimated.
The abnormal state estimation unit includes the rate of change in the area of the front part of the sole region of the right foot in the two sole images and the rate of change in the area of the front part of the sole region of the left foot in the two sole images. And at least the difference between the rate of change in the area of the rear part of the sole area of the right foot in the two sole images and the rate of change in the area of the rear part of the sole area of the left foot in the two sole images. The abnormal state estimation device according to claim 3, which estimates the abnormal state of the subject based on one of them.
The abnormal state estimation device according to any one of claims 1 to 4, wherein the abnormal state estimation unit estimates the presence or absence of diabetes or the presence or absence of neuropathy in diabetes as the abnormal state of the subject.
The abnormal state estimation device according to any one of claims 1 to 4, wherein the abnormal state estimation unit estimates the severity of diabetes or the degree of neuropathy in diabetes as the abnormal state of the subject.
A foot sole imaging device having a transparent support surface on which the sole of the subject is placed and taking two sole images through the support surface by varying the hardness of the support surface.
The abnormal state estimation device according to any one of claims 1 to 6,
Abnormal state estimation system including.
An image acquisition unit that acquires two foot images, which are images taken through the support surface when the sole of the subject is placed on each of the two transparent support surfaces having different hardness.
A foot condition estimation unit that estimates the hardness state of the foot sole of the subject by comparing the two foot sole images, and a foot condition estimation unit.
Sole condition estimator including.
A foot sole imaging device having a transparent support surface on which the sole of the subject is placed and taking two sole images through the support surface by varying the hardness of the support surface.
The sole state estimation device according to claim 8 and
Sole condition estimation system including.
Computer,
An image acquisition unit that acquires two foot images, which are images taken through the support surface when the sole of the subject is placed on each of the two transparent support surfaces having different hardness, and the above. An abnormal state estimation program for comparing two foot sole images and functioning as an abnormal state estimation unit for estimating the abnormal state of the subject.
Computer,
An image acquisition unit that acquires two foot images, which are images taken through the support surface when the sole of the subject is placed on each of the two transparent support surfaces having different hardness, and the above. A sole condition estimation program for comparing two foot sole images and functioning as a sole condition estimation unit for estimating the hardness state of the sole of the subject.