WO2021187272A1 - 自動排泄処理装置、管理システム、判定方法、およびプログラム - Google Patents

自動排泄処理装置、管理システム、判定方法、およびプログラム Download PDF

Info

Publication number
WO2021187272A1
WO2021187272A1 PCT/JP2021/009542 JP2021009542W WO2021187272A1 WO 2021187272 A1 WO2021187272 A1 WO 2021187272A1 JP 2021009542 W JP2021009542 W JP 2021009542W WO 2021187272 A1 WO2021187272 A1 WO 2021187272A1
Authority
WO
WIPO (PCT)
Prior art keywords
cup
captured image
determination unit
image
state
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/009542
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
正樹 関野
靖広 岡田
雄貴 須永
登美夫 足立
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Libertysolution Co Ltd
University of Tokyo NUC
Original Assignee
Libertysolution Co Ltd
University of Tokyo NUC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Libertysolution Co Ltd, University of Tokyo NUC filed Critical Libertysolution Co Ltd
Priority to US17/912,010 priority Critical patent/US20230172744A1/en
Priority to JP2021570165A priority patent/JP7093980B2/ja
Priority to CN202180021213.0A priority patent/CN115279309A/zh
Priority to KR1020227032085A priority patent/KR20220142497A/ko
Publication of WO2021187272A1 publication Critical patent/WO2021187272A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F5/00—Orthopaedic methods or devices for non-surgical treatment of bones or joints; Nursing devices ; Anti-rape devices
    • A61F5/44—Devices worn by the patient for reception of urine, faeces, catamenial or other discharge; Colostomy devices
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B10/00—Instruments for taking body samples for diagnostic purposes; Other methods or instruments for diagnosis, e.g. for vaccination diagnosis, sex determination or ovulation-period determination; Throat striking implements
    • A61B10/0038—Devices for taking faeces samples; Faecal examination devices
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B10/00—Instruments for taking body samples for diagnostic purposes; Other methods or instruments for diagnosis, e.g. for vaccination diagnosis, sex determination or ovulation-period determination; Throat striking implements
    • A61B10/0045—Devices for taking samples of body liquids
    • A61B10/007—Devices for taking samples of body liquids for taking urine samples
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration or pH-value ; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid or cerebral tissue
    • A61B5/14507—Measuring characteristics of blood in vivo, e.g. gas concentration or pH-value ; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid or cerebral tissue specially adapted for measuring characteristics of body fluids other than blood
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • the present invention relates to an automatic excretion processing device, a management system, a determination method, and a program.
  • the present application claims priority based on 62 / 991,082 filed in the United States on March 18, 2020, the contents of which are incorporated herein by reference.
  • the care worker inspects the condition of the object excreted from the care recipient and manages the physical condition of the care recipient.
  • the technique described in Patent Document 1 includes an imaging unit that acquires an image of overflowing water in a drainage pipe of a toilet bowl, a calculation unit that calculates the amount of a liquid object excreted from the body based on the image, and a calculation unit. It has.
  • Patent Document 1 estimates the amount of a liquid object, and does not determine the state of a solid object.
  • the inventors have been diligently researching an automatic excretion treatment device that automatically determines the state of a solid object in order to reduce the burden on care workers.
  • One aspect of the present invention includes a cup that is attached to the body and formed so as to receive an object excreted from the body, a processing unit that transfers the object in the cup to the outside of the cup, and the cup.
  • a processing unit that transfers the object in the cup to the outside of the cup, and the cup.
  • An automatic excretion processing apparatus including a determination unit that extracts features of the object and classifies the state of the object.
  • the acquisition unit acquires a captured image of the object excreted in the cup, it is not necessary for the care worker to inspect the state of the object, the burden on the care worker is reduced, and labor is performed.
  • the environment can be improved.
  • the acquired captured image is analyzed by the determination unit to determine the state of the object.
  • the determination unit has been repeatedly machine-learned in advance to determine the state of the object. Therefore, the determination unit can recognize the object in the captured image. Further, the determination unit can recognize the characteristics of the object in the captured image.
  • the determination unit is configured to recognize the characteristics of the object by learning corresponding to the patterns classified in advance. When the determination unit recognizes the characteristics of the object, the determination unit classifies the state of the object according to the characteristics.
  • the classification result of the judgment unit can be used by the care worker to manage the physical condition of the care recipient.
  • the determination unit of the present invention may monitor the change over time of the object in the cup based on the captured image and classify the state of the object.
  • the determination unit captures the change of the object by monitoring the change of the object with time in the cup. be able to.
  • the determination unit of the present invention may classify the state of the object based on the degree of deformation of the object in the process of reaching the inside of the cup and deforming the object based on the captured image. good.
  • the viscosity of a solid object differs depending on the amount of water contained. Deformation occurs when the object is excreted from the body and reaches the wall surface inside the cup. Since the degree of deformation over time differs depending on the viscosity of the object, the determination unit can classify the object from the degree of deformation by referring to the learning result of the degree of deformation over time of the object.
  • the determination unit of the present invention may classify the physical symptoms by comparing the hue of the object with the reference based on the captured image.
  • the object can be used as a judgment material for capturing changes in the physical condition of the care recipient not only by classification by viscosity but also by hue.
  • the determination unit can determine, for example, the feature that the object contains blood based on the hue of the captured image. Furthermore, since the color of blood changes with time, the bleeding site can be predicted according to the color depth of blood, and the symptoms of the care recipient can be classified. In addition, the determination unit can also determine the feature that the undigested substance contained in the object is contained based on the hue of the captured image.
  • the determination unit of the present invention may classify the physical symptoms by comparing the hue of the object with the reference based on the captured image, and determine a coping method according to the symptoms.
  • the care worker when the symptom of the care-requiring person is classified based on the hue of the object, the care worker can administer medication to the care-requiring person by determining the coping method corresponding to the symptom. Treatment can be done early.
  • the determination unit of the present invention may extract the characteristics of the object by the iterative learning that performs deep learning using a convolutional neural network and classify the state of the object.
  • the judgment unit can extract the features of the object based on the captured image by performing deep learning using a convolutional neural network that is strong in image classification in machine learning.
  • the determination unit of one aspect of the present invention may classify the physical symptoms by comparing the number of appearances of the object within a predetermined period with the criteria.
  • the determination unit can manage the physical condition of the long-term care recipient by grasping the number of excretion of the long-term care recipient.
  • One aspect of the present invention includes a cup that is attached to the body and formed to receive an object excreted from the body, a processing unit that transfers the object in the cup to the outside of the cup, and the cup.
  • the said Management including a management device that refers to the result of iterative learning using the captured images of the objects having different states based on the captured images, extracts the features of the objects, and classifies the states of the objects. It is a system.
  • a plurality of automatic excretion processing devices for acquiring captured images can be connected to a network to acquire a large amount of captured images required for learning. Since the captured image can be acquired as the operation period of the management system becomes longer, the determination unit can perform supervised learning. Further, the determination unit may start the operation by unsupervised learning, or may increase the learning opportunity as the operation period of the management system becomes longer to improve the determination accuracy.
  • One aspect of the present invention includes a cup that is attached to a body and is formed so as to receive an object excreted from the body, and a processing unit that transfers the object in the cup to the outside of the cup.
  • the inside of the cup is imaged in the automatic excretion processing device, an image of the object is acquired, and the result of repeated learning using the image of the object in a different state is referred to, and the acquired image of the object is imaged.
  • This is a determination method in which a computer executes a process of extracting features of the object and classifying the state of the object based on an image.
  • the determination unit can recognize the characteristics of the object by learning corresponding to the pre-classified patterns and classify the state of the object according to the characteristics.
  • One aspect of the present invention includes a cup that is attached to a body and is formed so as to receive an object excreted from the body, and a processing unit that transfers the object in the cup to the outside of the cup.
  • the inside of the cup is imaged in the automatic excretion processing device, an image of the object is acquired, and the result of repeated learning using the image of the object in a different state is referred to, and the acquired image of the object is imaged.
  • It is a program that causes a computer to execute a process of extracting features of the object and classifying the state of the object based on an image.
  • the program of the present invention it is possible to recognize the characteristics of an object by learning corresponding to a pre-classified pattern and classify the state of the object according to the characteristics.
  • an automatic excretion processing device capable of determining the state of an object by extracting the characteristics of the captured image of the object.
  • the automatic excretion processing device of the present invention is a device that automatically determines the state of a mainly solid object excreted from the body of a care recipient when the excretion processing is automatically performed.
  • the automatic excretion processing device 1 takes an image of the inside of the cup, the cup 2 attached to the body, the processing unit 3 for transferring the object in the cup to the outside of the cup 2, and the object. It is provided with an acquisition unit 4 for acquiring the captured image of the above, a determination unit 5 for determining the state of the object based on the captured image, a storage unit 6 for storing various data, and a display unit 7 for displaying the determination result.
  • the cup 2 is formed so as to receive an object excreted from the body of the care recipient.
  • the processing unit 3 transfers the object received in the cup 2 to the outside of the cup, and performs a process of cleaning the inside of the cup 2 and the body of the care recipient.
  • the processing unit 3 supplies washing water and air into the cup 2.
  • the processing unit 3 transfers the object, the washing water, and the air from the inside of the cup 2 to the outside of the cup 2 by suction.
  • the processing unit 3 temporarily stores the object transferred from the cup 2.
  • the processing unit 3 may drain the object into a drain pipe such as a sewer pipe.
  • the acquisition unit 4 includes a camera that images the inside of the cup 2.
  • the camera is attached to the cup 2 so as to image the inside of the cup 2, for example, and continuously acquires an electronically captured image.
  • the camera is, for example, an endoscopic camera including a light source.
  • the acquisition unit 4 may include an infrared camera having no light source in addition to the endoscopic camera.
  • the acquisition unit 4 may also have other sensors such as an ultrasonic sensor for generating an ultrasonic image.
  • the acquisition unit 4 stores the captured image in the storage unit 6.
  • the determination unit 5 reads out the captured image acquired by the acquisition unit 4 and stored in the storage unit 6.
  • the determination unit 5 has repeatedly performed learning using other captured images of the object in advance, and refers to the learning result based on the captured image acquired by the acquisition unit 4 to extract the features of the object. ..
  • the determination unit 5 classifies the state of the object based on the extracted features.
  • the classification is the Bristol stool scale (hereinafter, appropriately referred to as BSFS) used in the medical field (see Non-Patent Document 1).
  • BSFS Bristol stool scale
  • the Bristol stool scale is a medical diagnostic tool that classifies the condition of an object into seven categories.
  • the determination unit 5 is realized by, for example, a processor such as a CPU (Central Processing Unit) executing a program stored in the program memory. Further, a part or all of the determination unit 5 may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array), or software. And hardware may work together.
  • a processor such as a CPU (Central Processing Unit) executing a program stored in the program memory.
  • a part or all of the determination unit 5 may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array), or software. And hardware may work together.
  • the storage unit 6 is, for example, an HDD (Hard Disc Drive), a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), or a RAM (Random Access Memory), or a hybrid type using a plurality of these. It is realized by a storage device. Further, the storage unit 14 stores various programs such as firmware and application programs, processing results by various functional units, and the like.
  • the display unit 7 is a video display device using a liquid crystal display, an organic EL display, or the like.
  • the determination unit 5, the storage unit 6, and the display unit 7 may be composed of a personal computer, a tablet terminal, or a smartphone separate from the automatic excretion processing device.
  • teacher data of the object was created in order to verify the determination accuracy of the determination unit 5.
  • a pseudo sample was prepared using miso, cake flour, and chocolate powder to reproduce various states and shapes.
  • a teacher dataset 700 sheets
  • a test data set (171 sheets) were created to classify the shape of the object.
  • the shape rule for each type was determined, and the sample was fine-tuned using miso and cake flour to reproduce the texture.
  • the state of the object was integrated as class 4, and the state in which nothing was shown was defined as class 0, and was reclassified into a total of 7 classes.
  • the ratio of miso and flour is adjusted to be constant, so the water content is also constant.
  • the reason why the water content is kept constant is that there is a correlation between the shape of the object and the water content contained in the object as an index of BSFS (see, for example, Non-Patent Document 2). Therefore, it is considered that the water content of the sample of each object can be estimated by classifying the objects based on the shape.
  • the image data captured the change over time in the state of the sample when the prepared sample was dropped from the position corresponding to the anus portion in the cup 2.
  • the determination unit 5 monitors the change over time of the solid object in the cup based on the captured image, and classifies the state of the object based on the Bristol stool scale. In the classification, the elements and criteria that need to be analyzed in order to analyze the captured image and estimate the health condition are set. In this embodiment, an image of the object is collected by a camera installed in the cup. In the determination of the determination unit 5, the analysis was performed based on the following factors that are usually confirmed in the nursing care facility where the automatic excretion processing device is used. (1) Shape classification of objects and their changes (2) The color of the object and the blood and mixture contained in it (3) Frequency of excretion and volume of object
  • the determination unit 5 extracts the characteristics of the object and classifies the state of the object based on iterative learning that performs deep learning using, for example, a convolutional neural network (CNN) (see Non-Patent Document 3).
  • CNN convolutional neural network
  • the CNN executed by the determination unit 5 learns, for example, a captured image of an object classified and created in advance as teacher data. In this embodiment, two types of CNN models were compared to verify the determination accuracy.
  • FIG. 5 schematically shows the processing flow of the Simple model, which is the first CNN model.
  • the Simple model is, for example, a model having a basic configuration for performing a process of extracting a shape based on a captured image.
  • the Simple model includes a 6-layer convolution layer that performs a process of extracting a shape in a captured image, and an avg pooling layer that compresses information of input data after the process of the convolution layer.
  • FIG. 6 schematically shows the processing flow of the transfer learning model (Resnet18) (see, for example, Non-Patent Document 3), which is the second CNN model.
  • Resnet18 is a general transfer learning model.
  • the transfer learning model is a pre-learned image classification network.
  • the network of transfer learning models is pre-learned to extract features from images. In pre-learning, learning is performed in advance to extract powerful and information-rich features based on a large amount of natural images.
  • Resnet18 is configured to process a pre-trained 18-layer convolutional neural network that extracts features based on captured images using large datasets.
  • Fine-tuning the parameters in the transfer learning model can be applied to various datasets or applications.
  • Resnet18 was used and all parameters were changed (full-fine tuning).
  • the Simple model Compared to Resnet18, the Simple model has a shallower layer and less expressive power, but because there are fewer parameters, the calculation cost is reduced and it can be applied to 3DCNN and the like.
  • the cross entropy was used as the classification loss, and the stochastic gradient descent method with an attenuation learning rate of 0.001 was used as the gradient method.
  • data expansion is performed to increase the number of image data used as teacher data.
  • Data augmentation is a common technique that increases the variability of teacher data and reduces the algorithmic bias of the dataset.
  • Data expansion is applied at the stage of inputting teacher data.
  • processing such as rotation, inversion, cropping, and zooming of image data is generally performed, and the number of original data can be increased.
  • two preprocessings of gamma correction and histogram flattening were performed on the input image.
  • the histogram of the pixel values of the entire input image is flattened. No, the frequencies of all pixel values (0 to 255) are converted to the average number of pixels.
  • the contrast of the image is improved and the light source environment can be adjusted.
  • the pixel value of the input image is I (x, y)
  • the frequency thereof is H (x, y)
  • the total number of pixels of the input image is S
  • the pixel value I'(x, y) of the output image y) can be calculated by the equation (2).
  • CNN has abundant features captured in each treatment in a plurality of convolutional layers.
  • Grad-CAM Gradient-weighted Class Activation Mapping
  • Grad-CAM uses the gradient at the time of back propagation to calculate the weight of the feature diagram based on the following equation (3).
  • Alpha c k of the formula (3) is a partial derivative with respect to the probability y c output characteristic diagram A k of CNN is determined in class c.
  • Gradient shown in equation (3) is, i in the feature map A k ij, to changes in the j position pixel, represents the influence of the probability of being determined in the class c. It has been shown that this gradient is the same as that of the polymerizer (feature diagram by CAM: Class Activation Mapping) of the feature diagram of the final layer (see Non-Patent Document 4). Unlike the conventional CAM, this method does not need to change the network of the model, so that it can be applied to various models.
  • Figure 7 shows the comparison result of the correct answer rate of the test data based on Resnet 18 and the simple model.
  • Resnet18 and simple were trained under the conditions of preprocessing and data expansion for teacher data.
  • HV and rot mean inverted and rotated (maximum 45 °) image data, and his and gam mean histogram averaging and ⁇ conversion, respectively.
  • mono means a grayscale processed image.
  • the correct answer rate of the simple model and Resnet18 is about 0.2 higher in the transfer learning by Resnet18 than in the simple model. This is because Resnet18 is more sophisticated than the simple model because it already has a "seeing eye” due to pre-learning by ImageNet in addition to the depth of the learning layer.
  • the judgment result based on the grayscale image has higher performance when the data is not expanded than the judgment result based on the image in the RGB color space.
  • the histogram of the image is averaged, the performance is higher than when the image is handled in the RGB color space.
  • the correct answer rate increased more when the histogram was averaged than when the image was gamma-corrected.
  • the correct answer rate for the Simple model was 82.3%, which was highly accurate.
  • the correct answer rate for transfer learning of Resnet18 was 98.8%, which was much higher accuracy than the Simple model.
  • the simple model and the Resnet18 model are trained based on the data-expanded teacher data, and each uses Grad-CAM to output a feature diagram that serves as a basis for judgment.
  • FIG. 8 shows the judgment basis for the sample images of classes 1 to 6 based on the Resnet18 model.
  • the order (gradient) of the output class of the feature diagram surrounded by the black frame matches the correct label.
  • the output value under the image is output to the output class.
  • the determination unit 5 determines the entire image of the object as a determination basis and outputs an output value (0.995).
  • the determination unit 5 determines the entire image of the object as the basis for determination with respect to the feature diagrams of Class 2 to 6, and outputs the maximum value to the correct answer label of the "output class". From the above, the Resnet18 model can classify the captured image of the object according to the order of BSFS classification in the process of determining the feature amount in the image of the object.
  • FIG. 9 shows the judgment basis based on the Simple model.
  • the order (gradient) of the output feature diagram is shown non-linearly without corresponding to the order of class.
  • the feature amount in the image needs to be extracted based on the influence of the surroundings of the feature portion. Therefore, it can be seen that the simple model cannot learn the ability to "view the image as an image” in the determination process, and as a result, shows the gradient of the feature diagram non-linearly.
  • Resnet18 already has the ability to extract features from images by pre-learning
  • the simple model has a learning ability to recognize as an image in a dataset of images of ubiquitous objects compared to Resnet18. And decrease.
  • Resnet18 there is a big difference in accuracy between Resnet18 and the simple model.
  • the layer of the simple model is shallow compared to Resnet18.
  • the other is that the number of data sets is small in extracting features as an image. Therefore, a model with the same structure as Resnet18 and a simple model are trained in advance using the sample image data set cifar-10, and then retrained using the image data set of the object to improve the performance of the model. Compared.
  • FIG. 10 shows the correct answer rate of the judgment result when the pre-learning is applied to each judgment method.
  • the preprocessing of the image data of cifar-10 includes left / right inversion and trimming
  • the preprocessing of the image data of the object includes up / down / left / right inversion, rotation, and flattening of the histogram.
  • the cup 2 attached to the body of the care recipient is formed to have a small capacity.
  • the automatic excretion processing device 1 is configured to detect that an object has fallen into the cup 2 and at the same time suck the object out of the cup 2.
  • the determination unit 5 proposes a 3D model that classifies the state of the object by machine learning the change over time of the object based on the moving image data at the time of excretion.
  • the determination unit 5 determines the viscosity based on the moving image, the viscosity can also be taken into consideration from the diffusion rate of the object. Therefore, even if the determination unit 5 determines based on the data obtained by capturing a part of the object, the object can be determined. It can be classified with high accuracy. Based on the above experimental results, a simple model or a transfer-learned CNN model is applied to the determination unit 5.
  • the transfer-learned CNN model has higher accuracy than the Simple model. However, the transfer-learned CNN model has a longer calculation time than the Simple model. Therefore, when applying the Simple model to the determination unit 5, it is desirable that the Simple model is extended so as to improve the classification accuracy.
  • the determination unit 5 classifies the object based on the time course of the object recorded in the captured image continuously captured in the cup 2 attached to the body of the care subject using the Simple model or the transfer-learned CNN. I do.
  • the determination unit 5 determines the state of the object in the cup 2 based on the change over time.
  • the determination unit 5 refers to the result of iterative learning using other captured images of objects in different states, extracts the features of the object based on the captured images acquired by the acquisition unit 4, and extracts the features of the object. Classify the state.
  • the determination unit 5 monitors the change over time of the object in the cup based on the captured image, and classifies the state of the object.
  • the determination unit 5 monitors, for example, a state in which an object appears outside the body, reaches the inner wall of the cup 2, and is deformed by using a continuous captured image. At this time, the determination unit 5 classifies the state of the object based on the degree of deformation of the object with time. The properties of the object change depending on the water content. Therefore, the state of the object can be classified by monitoring the change in the degree of deformation of the object over time.
  • the determination unit 5 may classify the physical symptoms by comparing the hue of the object with the reference based on the captured image.
  • the determination unit 5 extracts the hue characteristics included in the object based on the analysis result of the pixels of the captured image.
  • the determination unit 5 extracts, for example, changes in the hue of the entire object, blood contained in the object, undigested material, and the like.
  • the determination unit 5 determines, for example, the hue of blood when blood is extracted.
  • the oxygenation rate of blood changes with the passage of time, and the hue changes.
  • the determination unit 5 compares the reference of the hue of blood with the hue of blood extracted from the captured image, and determines the degree of passage of time of blood. When the determination unit 5 determines that the passage of time of blood is low, it determines that there is a bleeding site on the downstream side of the digestive organ, and extracts the physical symptom of bleeding on the downstream side of the digestive organ. When the determination unit 5 determines that the degree of passage of blood time is high, it determines that there is a bleeding site on the upstream side of the digestive organ, and extracts the physical symptom of bleeding on the upstream side of the digestive organ.
  • the determination unit 5 may extract the undigested substance contained in the object based on the hue of the object. When the undigested substance contained in the object is extracted, the determination unit 5 extracts the physical symptom containing the undigested substance. By the above processing, the determination unit 5 can classify the physical symptoms by comparing the hue of the object with the standard. Further, the determination unit 5 may classify the symptom and determine a coping method according to the symptom.
  • the determination unit 5 may classify physical symptoms by comparing the number of appearances of the object within a predetermined period with the criteria. For example, when the number of appearances of the object is smaller than the standard, the determination unit 5 extracts symptoms related to constipation and abnormalities of the digestive organs.
  • FIG. 11 shows a flow chart of the process related to the determination method for classifying the state of the object.
  • Objects in different states are imaged, and teacher data is created based on the captured images (step S10).
  • the teacher data is processed by rotation, inversion, cropping, zooming, etc., and data expansion is performed to increase the number of original data.
  • the determination unit 5 performs iterative learning to classify the object by deep learning using a convolutional neural network using the captured image of the object (step S12).
  • the acquisition unit 4 images an object with a camera (step S14).
  • the acquisition unit 4 acquires an captured image of the object and stores it in the storage unit 6 (step S16).
  • the determination unit 5 refers to the result of iterative learning using the captured images of the objects having different states, extracts the features of the object based on the acquired captured images of the objects, and classifies the states of the objects. (Step S18).
  • the state of the object excreted by the long-term care subject is automatically classified based on the captured image of the inside of the cup 2 attached to the body of the long-term care subject. be able to.
  • the automatic excretion processing device 1 even if the space inside the cup 2 is small and only a part of the object is recorded in the captured image, the change over time of the object is determined based on the determination method based on machine learning. By doing so, the state of the object can be accurately classified.
  • the automatic excretion processing device 1 it is possible to classify the physical symptoms of the long-term care target person by comparing the state of the object with the standard, and it is possible to easily perform the health management of the long-term care target person.
  • the determination unit 5 executes the process in the automatic excretion processing device 1.
  • a management system 100 that comprehensively manages the care recipient in the management device 20 may be configured.
  • the same names and reference numerals will be used for the same configurations as those in the first embodiment, and duplicate description will be omitted as appropriate.
  • the management system 100 includes one or more automatic excretion processing devices 1 connected to the network NW, a management device 20 connected to the network NW, and a terminal device 40 connected to the network NW. And.
  • the management device 20 includes, for example, an acquisition unit 21 that acquires an captured image via a network NW, a determination unit 22 that determines the properties of an object based on the captured image, and a storage unit 23 that stores various information.
  • a display unit 24 for displaying the determination result in the determination unit 22 is provided.
  • the management device 20 manages, for example, a plurality of automatic excretion processing devices 1 provided for each building such as a nursing care facility.
  • the management device 20 may manage a plurality of automatic excretion processing devices 1 for each of a plurality of buildings.
  • the determination unit 22 determines and classifies the state of the object based on the captured image acquired via the network NW.
  • the captured image is stored as teacher data in the storage unit 23 after the plurality of automatic excretion processing devices 1 are started to operate. For example, a captured image of an object created by classifying in advance is learned as teacher data.
  • the determination unit 5 starts learning without teacher data, repeats learning based on captured images acquired from a plurality of automatic excretion processing devices 1 connected to the network NW, and improves the determination accuracy with the lapse of the operating period. You may do so.
  • the determination result by the determination unit 22 may be displayed on the terminal device 40 separate from the management device 20.
  • the terminal device 40 is used, for example, by a care worker.
  • the care worker confirms the symptom of the care recipient and the coping method displayed on the terminal device 40, and takes appropriate measures for the care recipient.
  • the terminal device 40 includes a display unit 41 on which a determination result is displayed and a control unit 42 that controls the display unit 41.
  • the terminal device 40 is, for example, a portable information terminal device such as a tablet terminal or a smartphone.
  • the determination unit 22 in the management device 20 by aggregating the captured images transmitted from one or more automatic excretion processing devices 1 into the management device 20, the determination unit 22 in the management device 20 repeatedly learns based on the aggregated captured images. Can be performed to improve the determination accuracy.
  • the determination result in the determination unit 22 is centrally managed by the management device 20, and the health state of the care subject who uses the automatic excretion processing device 1 can be centrally managed.
  • the data regarding the health condition of the care target person is transmitted to the terminal device 40, and the care worker can take appropriate measures for the care target person.
  • the health condition of the care recipient can be automatically managed, and the burden on the care worker in the care can be significantly reduced.
  • the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added. For example, the configurations described in each of the above-described embodiments and examples may be combined. Further, the determination unit 5 not only learns with supervised learning shown in the embodiment, but also starts learning from a state without teacher data, repeats learning based on the captured image acquired from the acquisition unit 4, and determines the determination accuracy. You may try to improve it.
  • a program for realizing these functions is recorded on a computer-readable recording medium. It may be realized by loading a program recorded on a recording medium into a computer system and executing the program.
  • the term "computer system” as used herein includes hardware such as an OS and peripheral devices.
  • the "computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage unit such as a hard disk built in a computer system.
  • a “computer-readable recording medium” is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. It may also include a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client in that case.
  • the above-mentioned program may be a program for realizing a part of the above-mentioned functions, and may be a program for realizing the above-mentioned functions in combination with a program already recorded in the computer system.
  • 1 automatic excretion processing device 2 cup 2 processing unit, 4 acquisition unit, 5 judgment unit, 6 storage unit, 7 display unit, 20 management device, 21 acquisition unit, 22 judgment unit, 23 storage unit, 24 display unit, 40 terminals Device, 41 display unit, 42 control unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • General Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Pathology (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Computing Systems (AREA)
  • Surgery (AREA)
  • Databases & Information Systems (AREA)
  • Nursing (AREA)
  • Orthopedic Medicine & Surgery (AREA)
  • Epidemiology (AREA)
  • Vascular Medicine (AREA)
  • Biophysics (AREA)
  • Hematology (AREA)
  • Biochemistry (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Immunology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Optics & Photonics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Biodiversity & Conservation Biology (AREA)
PCT/JP2021/009542 2020-03-18 2021-03-10 自動排泄処理装置、管理システム、判定方法、およびプログラム Ceased WO2021187272A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US17/912,010 US20230172744A1 (en) 2020-03-18 2021-03-10 Automatic excretion-processing device, management system, determination method, and program
JP2021570165A JP7093980B2 (ja) 2020-03-18 2021-03-10 自動排泄処理装置、管理システム、判定方法、およびプログラム
CN202180021213.0A CN115279309A (zh) 2020-03-18 2021-03-10 自动排泄处理装置、管理系统、判定方法以及程序
KR1020227032085A KR20220142497A (ko) 2020-03-18 2021-03-10 자동배설처리장치、관리시스템、판정방법 및 프로그램

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202062991082P 2020-03-18 2020-03-18
US62/991,082 2020-03-18

Publications (1)

Publication Number Publication Date
WO2021187272A1 true WO2021187272A1 (ja) 2021-09-23

Family

ID=77768275

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/009542 Ceased WO2021187272A1 (ja) 2020-03-18 2021-03-10 自動排泄処理装置、管理システム、判定方法、およびプログラム

Country Status (5)

Country Link
US (1) US20230172744A1 (https=)
JP (1) JP7093980B2 (https=)
KR (1) KR20220142497A (https=)
CN (1) CN115279309A (https=)
WO (1) WO2021187272A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023171035A (ja) * 2022-05-20 2023-12-01 キヤノン株式会社 画像認識方法、プログラム、記録媒体及び画像認識装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA3137612A1 (en) * 2021-11-05 2023-05-05 Pulsemedica Corp. Hybrid classifier training for feature extraction
US12539015B2 (en) * 2023-09-05 2026-02-03 Midea Group Co., Ltd. Dishwasher with personalized utensil detection and scanning aids therefor

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005013244A (ja) * 2002-05-16 2005-01-20 Masanobu Kujirada 排尿・排便の検知及び自動吸引処理装置
JP2015136386A (ja) * 2014-01-20 2015-07-30 株式会社国際電気通信基礎技術研究所 排泄管理システム、排泄管理プログラム、排泄管理方法および排泄管理装置
JP2016004005A (ja) * 2014-06-18 2016-01-12 関根　弘一 大便色検知装置
JP2017137707A (ja) * 2016-02-04 2017-08-10 株式会社Ｌｉｘｉｌ 便器装置
JP2017533423A (ja) * 2014-10-13 2017-11-09 コリア インスティテュート オブ インダストリアル テクノロジーＫｏｒｅａ Ｉｎｓｔｉｔｕｔｅ Ｏｆ Ｉｎｄｕｓｔｒｉａｌ Ｔｅｃｈｎｏｌｏｇｙ 排泄物形状測定装置及び方法
WO2019187018A1 (ja) * 2018-03-30 2019-10-03 株式会社ファーストスクリーニング 健康補助システム、センサー、及び健康補助方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101312559B1 (ko) * 2011-08-09 2013-09-30 장황진 건강을 진단하는 다기능 변기시스템
CN106651883B (zh) * 2016-12-30 2019-12-17 四川沃文特生物技术有限公司 基于机器视觉的粪便形态识别方法
WO2019245359A1 (en) * 2018-06-21 2019-12-26 N.V. Nutricia Method and system for characterizing stool patterns of young infants

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005013244A (ja) * 2002-05-16 2005-01-20 Masanobu Kujirada 排尿・排便の検知及び自動吸引処理装置
JP2015136386A (ja) * 2014-01-20 2015-07-30 株式会社国際電気通信基礎技術研究所 排泄管理システム、排泄管理プログラム、排泄管理方法および排泄管理装置
JP2016004005A (ja) * 2014-06-18 2016-01-12 関根　弘一 大便色検知装置
JP2017533423A (ja) * 2014-10-13 2017-11-09 コリア インスティテュート オブ インダストリアル テクノロジーＫｏｒｅａ Ｉｎｓｔｉｔｕｔｅ Ｏｆ Ｉｎｄｕｓｔｒｉａｌ Ｔｅｃｈｎｏｌｏｇｙ 排泄物形状測定装置及び方法
JP2017137707A (ja) * 2016-02-04 2017-08-10 株式会社Ｌｉｘｉｌ 便器装置
WO2019187018A1 (ja) * 2018-03-30 2019-10-03 株式会社ファーストスクリーニング 健康補助システム、センサー、及び健康補助方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023171035A (ja) * 2022-05-20 2023-12-01 キヤノン株式会社 画像認識方法、プログラム、記録媒体及び画像認識装置

Also Published As

Publication number Publication date
JP7093980B2 (ja) 2022-07-01
US20230172744A1 (en) 2023-06-08
KR20220142497A (ko) 2022-10-21
JPWO2021187272A1 (https=) 2021-09-23
CN115279309A (zh) 2022-11-01

Similar Documents

Publication Publication Date Title
US10860930B2 (en) Learning method, image recognition device, and computer-readable storage medium
JP7297081B2 (ja) 画像分類方法、画像分類装置、医療用電子機器、画像分類機器、及びコンピュータプログラム
CN109376636B (zh) 基于胶囊网络的眼底视网膜图像分类方法
JP7093980B2 (ja) 自動排泄処理装置、管理システム、判定方法、およびプログラム
CN113012173A (zh) 基于心脏mri的心脏分割模型和病理分类模型训练、心脏分割、病理分类方法及装置
US11087462B2 (en) System and method for determining a colonoscopy image
CN111340819A (zh) 图像分割方法、装置和存储介质
US20230255467A1 (en) Diagnostic imaging device, diagnostic imaging method, diagnostic imaging program, and learned model
CN110008971A (zh) 图像处理方法、装置、计算机可读存储介质及计算机设备
WO2019184851A1 (zh) 图像处理方法和装置及神经网络模型的训练方法
CN113889267A (zh) 基于眼象识别的糖尿病诊断模型的构建方法和电子设备
Naskar et al. An efficient, cost-effective and reliable non-invasive anaemia detection method by analysing palm pallor
Rajasekhar et al. An Improved Machine Learning and Deep Learning based Breast Cancer Detection using Thermographic Images
CN116012324A (zh) 一种医学影像质量控制方法、装置和电子装置
CN114972297A (zh) 口腔健康监测方法及装置
Mayya AI-Driven Stitching of Retinal Smartphone-Based Images for Automated Fundoscopy
TWI863845B (zh) 醫療輔助診斷系統以及醫療輔助診斷胃炎的方法
Pathak Deep learning approach to early detection of rheumatoid arthritis
CN119991597A (zh) 一种肠镜患者肠道清洁等级识别方法及系统
CN113222985B (zh) 一种图像处理方法、装置、计算机设备和介质
Mhedbi et al. A Convolutional Neural Network based system for classifying malignant and benign skin lesions using mobile-device images
CN117132521B (zh) 心脏瓣膜图像分割方法、电子设备和存储介质
TWI885336B (zh) 用以判斷心臟鈣化等級的系統及方法
CN119723119B (zh) 一种基于内镜图像序列分析小肠动力系统的方法及装置
Vaish et al. Smartphone based automatic organ validation in ultrasound video

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21771557

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021570165

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20227032085

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21771557

Country of ref document: EP

Kind code of ref document: A1