JP2020187691A - Determination device, determination method, and program - Google Patents

Abstract

To provide a determination device capable of reducing determination errors in an analysis relating to excrement using machine learning.SOLUTION: In a determination system 1A, a determination device 10a includes: an image information acquiring unit for acquiring image information on an object image which is to be determined for determination items related to feces and is acquired by photographing an internal space of a toilet bowl after excretion; a preprocessing unit for generating a difference image showing the difference between a reference image obtained by photographing the internal space of the toilet bowl before excretion and the object image; an analysis unit for performing estimation on determination items about the difference image by inputting the difference image into the already learned model obtained by learning the correspondence relation between the difference image for learning showing the difference between images indicating before and after excretion and a determination result of the determination items by machine leaning using a neutral network; and a determination unit for performing determination on the determination items about the object image based on an estimation result by the analysis unit.SELECTED DRAWING: Figure 8

Description

The present invention relates to a determination device, a determination method, and a program.

There is an attempt to grasp the status of excretion of the living body. For example, a technique for photographing excrement with a camera and analyzing the image is disclosed (see, for example, Patent Document 1).

In addition, it is common practice to use machine learning to analyze human facial expressions. In the method using machine learning, for example, a trained model is created by executing machine learning using learning data in which a feature amount of a human facial expression and an emotion corresponding to the facial expression are associated with each other. By inputting the features of human facial expressions into this trained model, it is possible to estimate the emotions that the facial expressions mean, and it is possible to analyze facial expressions.

JP-A-2007-252805

When the technique as described in Patent Document 1 is used, it cannot be said that the accuracy of the analysis is sufficient. That is, since the analysis is performed using a table that associates the stool discharge rate, hardness or size created in advance with the stool classification (hard stool, watery stool, etc.), the target that is not set in the table is correct. There is a concern that the classification cannot be obtained.

On the other hand, it is conceivable to apply the above-mentioned machine learning method to the analysis of excrement. For example, a trained model is created by executing machine learning using learning data in which an image of the internal space of a toilet bowl after excretion is associated with the results of classification and determination. By inputting an image to be analyzed into this trained model, the desired analysis result in excrement can be estimated.

However, when an image of the internal space of the toilet bowl after excretion is used as the learning data, various things are reflected in the image. For example, the shadow of the light reflected on the toilet bowl itself, the surface of the water pool, or the drain trap, the shadow created by the user blocking the indoor light, etc. are imaged together with the excrement. In such a case, there is a possibility that a judgment different from the originally desired result may be made, such as erroneously recognizing the shadow as a stool.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a determination device, a determination method, and a program capable of reducing determination errors in analysis of excrement using machine learning. It is to be.

In order to solve the above-mentioned problems, one embodiment of the present invention is a target image for determining determination items related to stool, and acquires image information of the target image obtained by capturing the internal space of the toilet bowl after excretion. The difference between the image information acquisition unit, the reference image that captures the internal space of the toilet bowl before excretion, the preprocessing unit that generates a difference image showing the difference between the target image, and the image showing before and after excretion. By inputting the difference image into a trained model learned by machine learning using a neural network, the correspondence between the learning difference image showing the above and the judgment result of the judgment item is input to the judgment item. It is a determination device including an estimation unit that estimates the image, and a determination unit that determines the determination item for the target image based on the estimation result by the estimation unit.

Further, one embodiment of the present invention is the above-mentioned determination device, and the preprocessing unit corresponds to color information indicating the color of a predetermined pixel in the reference image and the predetermined pixel in the pixel of the target image. Based on the difference between the pixel and the color information, the color information of the pixel corresponding to the predetermined pixel in the difference image is determined.

Further, one embodiment of the present invention is the above-mentioned determination device, in which the reference image, the target image, and the difference image are RGB images, and the preprocessing unit is a predetermined pixel in the reference image. The RGB value of the pixel corresponding to the predetermined pixel in the difference image is determined based on the difference between the RGB value and the RGB value of the pixel corresponding to the predetermined pixel in the target image.

Further, one embodiment of the present invention is the above-mentioned determination device, in which the reference image, the target image, and the difference image are RGB images, and the preprocessing unit is a predetermined pixel in the reference image. Based on the difference between the color ratio indicating the ratio of R value, G value, and B value and the color ratio of the pixel corresponding to the predetermined pixel in the target image, the pixel corresponding to the predetermined pixel in the difference image Determine the RGB value.

Further, in one embodiment of the present invention, the image information acquisition unit acquires the image information of the target image obtained by capturing the internal space of the toilet bowl after excretion, which is the target image for which the image information acquisition unit determines the determination items related to stool. , The preprocessing unit generates a difference image showing the difference between the reference image of the internal space of the toilet bowl before excretion and the target image, and the estimation unit calculates the difference between the images before and after excretion. By inputting the difference image into a trained model learned by machine learning using a neural network, the correspondence between the difference image for learning shown and the judgment result of the judgment item is related to the judgment item. This is a determination method in which estimation is performed and the determination unit determines the determination item for the target image based on the estimation result by the estimation unit.

Further, in one embodiment of the present invention, the computer of the determination device is a target image for determining a determination item related to stool, and image information of the target image obtained by capturing the internal space of the toilet bowl after excretion is acquired. Image information acquisition means, preprocessing means for generating a difference image showing the difference between the reference image of the internal space of the toilet bowl before excretion and the target image, learning showing the difference between the images before and after excretion. By inputting the difference image into a trained model learned by machine learning using a neural network, the correspondence relationship between the difference image for use and the judgment result of the judgment item is estimated for the judgment item. It is a program for functioning as an estimation means to perform, and a determination means for determining the determination item with respect to the target image based on the estimation result by the estimation means.

As described above, according to the present invention, it is possible to reduce determination errors in the analysis of excrement using machine learning.

It is a block diagram which shows the structure of the determination system 1 to which the determination apparatus 10 which concerns on 1st Embodiment is applied. It is a block diagram which shows the structure of the trained model storage part 16 which concerns on 1st Embodiment. It is a figure explaining the image to be determined by the determination apparatus 10 which concerns on 1st Embodiment. It is a flowchart which shows the whole flow of the process performed by the determination apparatus 10 which concerns on 1st Embodiment. It is a flowchart which shows the flow of the determination process performed by the determination apparatus 10 which concerns on 1st Embodiment. It is a flowchart which shows the flow of the determination process of the cleaning method performed by the determination apparatus 10 which concerns on 1st Embodiment. It is a figure explaining the determination apparatus 10A which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the determination system 1A to which the determination apparatus 10A which concerns on 2nd Embodiment is applied. It is a figure explaining the process performed by the pre-processing unit 19 which concerns on 2nd Embodiment. It is a flowchart which shows the flow of the process performed by the determination apparatus 10A which concerns on 2nd Embodiment. It is a figure explaining the process performed by the pre-processing unit 19 which concerns on the modification 1 of the 2nd Embodiment. It is a flowchart which shows the flow of the process performed by the determination apparatus 10A which concerns on the modification 1 of the 2nd Embodiment. It is a figure explaining the process performed by the pre-processing unit 19 which concerns on modification 2 of the 2nd Embodiment. It is a flowchart which shows the flow of the process performed by the determination apparatus 10A which concerns on the modification 2 of the 2nd Embodiment. It is a block diagram which shows the structure of the determination apparatus 10B which concerns on 3rd Embodiment. It is a figure explaining the process performed by the analysis unit 12B which concerns on 3rd Embodiment. It is a flowchart which shows the flow of the process performed by the determination apparatus 10B which concerns on 3rd Embodiment.

Hereinafter, the determination device of the embodiment will be described with reference to the drawings.

(First Embodiment)
First, the first embodiment will be described.
FIG. 1 is a block diagram showing a configuration of a determination system 1 to which the determination device 10 according to the first embodiment is applied. The determination system 1 includes, for example, a determination device 10 and a learning device 20.

The determination device 10 determines the excretion based on the target image (hereinafter, also simply referred to as an image) to be determined. The target image is an image related to excretion, for example, an image of the internal space 34 (same as above) of the toilet bowl 32 (see FIG. 3) after excretion. The post-excretion is an arbitrary time point after the user excretes and before the toilet bowl is washed, for example, when the user sitting on the toilet bowl 30 (same as above) is unseat. Judgment regarding excretion is a judgment item regarding excretion behavior and situation, and cleaning of excrement. For example, presence / absence of excretion, presence / absence of urine, presence / absence of stool, properties of stool, presence / absence of use of paper (toilet paper). , How to clean the toilet bowl 30 after excretion based on information such as the amount of paper used, and the status of excretion. The property of the stool may be information indicating the state of the stool such as "hard stool", "ordinary stool", "loose stool", "muddy stool", "watery stool" indicating the shape of the stool, or "hard" or "hard". It may be information indicating properties or conditions such as "soft". The shape of the stool is labeled (evaluated) from the viewpoints of splashing into the toilet bowl, how the stool in the reservoir melts out, how it becomes turbid, and the characteristics inside the reservoir (underwater) and above the water surface (in the air). )I do. Further, the property of the stool may be information indicating the amount of stool, or information indicating the classification of the amount of stool into two such as large or small, or large, normal or small. It may be information that indicates the amount of stools numerically. Further, the property of the stool may be information indicating the color of the stool. The color of the stool may be, for example, ocher to brown as the color of the normal stool, and may be information indicating whether or not the color of the stool is the color of the normal stool. In particular, the color of the stool is black (so-called tar stool). It may be information indicating whether or not it is (color). The washing method of the toilet bowl 30 after excretion includes the amount of washing water used for washing, the water pressure, the number of washings, and the like.

The determination device 10 includes, for example, an image information acquisition unit 11, an analysis unit 12, a determination unit 13, an output unit 14, an image information storage unit 15, a learned model storage unit 16, and a determination result storage unit 17. , Equipped with. Here, the analysis unit 12 is an example of the “estimation unit”.

The image information acquisition unit 11 acquires image information of an image (target image) of the internal space 34 of the toilet bowl 32, which is captured for excretion. The image information acquisition unit 11 outputs the acquired image information to the analysis unit 12 and stores it in the image information storage unit 15. The image information acquisition unit 11 is connected to the toilet bowl device 3 and the image pickup device 4 (see FIG. 3).

The analysis unit 12 analyzes an image (target image) corresponding to the image information obtained from the image information acquisition unit 11. The analysis by the analysis unit 12 is to estimate the items of determination regarding excretion based on the image related to excretion.

The analysis unit 12 estimates using, for example, a trained model corresponding to the determination item of the determination unit 13. The trained model is, for example, a model stored in the trained model storage unit 16 and is a model in which the correspondence between the image related to excretion and the evaluation result regarding excretion is learned.

For example, the analysis unit 12 uses the output obtained from the learned model that learned the correspondence between the image and the presence or absence of urine as the estimation result of estimating the presence or absence of urine. Further, the analysis unit 12 uses the output obtained from the learned model that learned the correspondence between the image and the presence or absence of stool as the estimation result of estimating the presence or absence of stool. In addition, the analysis unit 12 uses the output obtained from the trained model that learned the correspondence between the image and the stool properties as the estimation result of estimating the stool properties. Further, the analysis unit 12 uses the output obtained from the trained model that has learned the correspondence between the image and the presence / absence of paper as the estimation result of estimating the presence / absence of paper. Further, the analysis unit 12 uses the output obtained from the trained model that has learned the correspondence between the image and the paper usage as the estimation result of estimating the paper usage.

The analysis unit 12 may perform estimation using a trained model that estimates a plurality of items from an image. For example, the analysis unit 12 may perform estimation using a trained model that has learned the correspondence between the image and the presence / absence of each of urine and stool. When it is estimated from the trained model that there is neither urine nor stool in the image, the analysis unit 12 estimates that there is no excretion.

The determination unit 13 makes a determination regarding excretion using the analysis result obtained from the analysis unit 12. For example, the determination unit 13 uses the presence or absence of urine estimated from the image as a determination result for determining the presence or absence of urine in the image. Further, the determination unit 13 uses the presence / absence of stool estimated from the image as the determination result of determining the presence / absence of stool in the image. In addition, the determination unit 13 uses the stool properties estimated from the image as the determination result for determining the stool properties in the image. In addition, the determination unit 13 determines whether or not paper is used in the image as a determination result of determining whether or not paper is used in the image. Further, the determination unit 13 uses the amount of paper used estimated from the image as the determination result for determining the amount of paper used in the image.

The determination unit 13 may make a determination regarding excretion using a plurality of estimation results. For example, the method of cleaning the toilet bowl 30 after excretion may be determined based on the properties of the stool estimated from the image and the amount of paper used.

The output unit 14 outputs the determination result by the determination unit 13. For example, the output unit 14 may transmit the determination result to the terminal of the user who has performed the excretion behavior. As a result, the user can recognize his / her excretory behavior and the determination result of the situation.
The image information storage unit 15 stores the image information acquired by the image information acquisition unit 11.
The trained model storage unit 16 stores the trained model corresponding to each of the determination items.
The determination result storage unit 17 stores the determination result by the determination unit 13.

The trained model stored in the trained model storage unit 16 is created, for example, by using a deep learning (DL) method. DL is a machine learning method using a deep neural network (DNN) composed of a multi-layer neural network. DNN is realized by a network inspired by the principle of predictive coding in neuroscience, and is constructed by a function that mimics the neurotransmission network. By using the DL method, the trained model can be made to automatically recognize the feature amount inherent in the image as if it were a human thought. That is, it is possible to estimate directly from the image by letting the trained model learn the data itself without performing the work of extracting the feature amount.

In this embodiment, the case where the trained model is created by using the DL method will be described as an example, but the present invention is not limited to this. The trained model may be a model created by training the training data in which the image data and the evaluation result of convenience are associated with each other, at least without extracting the feature amount from the image data. The image data here are various images of the internal space 34 of the toilet bowl 32.

FIG. 2 is a block diagram showing the configuration of the trained model storage unit 16 according to the first embodiment. The trained model storage unit 16 includes, for example, a urine presence / absence estimation model 161, a stool presence / absence estimation model 162, a stool property estimation model 163, a paper use / absence estimation model 165, and a paper usage estimation model 166.

The urine presence / absence estimation model 161 is a learned model in which the correspondence relationship between the image and the presence / absence of urine is learned, and the learning data in which the image related to excretion is associated with the information indicating the presence / absence of urine judged from the image. Created by learning.
The stool presence / absence estimation model 162 is a learned model in which the correspondence between the image and the presence / absence of stool is learned, and the learning data in which the image related to excretion is associated with the information indicating the presence / absence of stool judged from the image is used. Created by learning.

The stool property estimation model 163 is a learned model in which the correspondence between the image and the stool property is learned, and the learning data in which the information indicating the stool property determined from the image is associated with the image related to excretion is provided. Created by learning.

The paper use estimation model 165 is a learned model that learns the correspondence between the image and the use of paper, and the image related to excretion corresponds to the information indicating the presence or absence of the used paper judged from the image. It is created by training the attached learning data.
The paper usage estimation model 166 is a trained model that has learned the correspondence between the image and the paper usage, and the image related to excretion is associated with information indicating the paper usage judged from the image. It is created by training the training data. The amount of paper used may be two, which is large or small, or three, which is large, normal, or small, and may be information that indicates the amount of paper numerically.
As a method of determining the presence or absence of excretion from the image, for example, the person in charge of creating the learning data may determine.

FIG. 3 is a diagram for explaining an image to be determined by the determination device 10 according to the first embodiment. FIG. 3 schematically shows the positional relationship between the toilet bowl device 3 and the image pickup device 4.

The toilet device 3 includes, for example, a toilet 30 having a toilet bowl 32. The toilet device 3 is configured to be able to supply the washing water S to the opening 36 provided in the internal space 34 of the toilet bowl 32. The toilet device 3 uses a functional unit (not shown) provided in the toilet 30 to allow the user of the toilet device 3 to sit or leave, start using local cleaning, and perform cleaning of the toilet bowl 32 after excretion. Detected. The toilet device 3 transmits the detection result by the functional unit to the determination device 10.

In the following description, when the user of the toilet bowl device 3 is seated on the toilet bowl 30, the front side of the user is referred to as "front side" and the rear side is referred to as "rear side". Further, when the user of the toilet bowl device 3 is seated on the toilet bowl 30, the left side of the user is referred to as "left side" and the right side is referred to as "right side". Further, the side away from the floor surface on which the toilet bowl device 3 is installed is referred to as "upper side", and the side close to the floor surface is referred to as "lower side".

The imaging device 4 is provided so that the contents related to the excretory behavior can be imaged. The image pickup device 4 is installed above the toilet bowl 30, for example, inside the rear edge of the toilet bowl 32 so that the lens faces the internal space 34 of the toilet bowl 32. For example, the image pickup device 4 takes an image according to the instruction of the determination device 10, and transmits the image information of the captured image to the determination device 10. In this case, the determination device 10 transmits control information indicating an imaging instruction to the imaging device 4 via the communication unit 18.

Here, the processing performed by the determination device 10 according to the first embodiment will be described with reference to FIGS. 4 to 6. FIG. 4 is a flowchart showing the entire flow of processing performed by the determination device 10 according to the first embodiment, and FIG. 5 is a flowchart showing a flow of determination processing performed by the determination device 10. FIG. 6 is a flowchart showing the flow of the determination process of the cleaning method performed by the determination device 10.

First, the entire flow of the process performed by the determination device 10 will be described with reference to FIG.
The determination device 10 determines whether or not the user of the toilet device 3 is seated on the toilet bowl 30 by communication with the toilet bowl device 3 (step S10), and when it is determined that the user is seated on the toilet bowl 30, an image. Acquire information (step S11). The image information is image information of an image related to excretion. The determination device 10 transmits the control signal instructing the image pickup to the image pickup device 4, causes the image pickup device 4 to image the internal space 34 of the toilet bowl 32, and transmits the image information of the captured image, thereby transmitting the image information. get.
In the flowchart shown in FIG. 4, the case where the determination result of determining the seating by the user is used as a trigger for acquiring image information has been described as an example, but the present invention is not limited to this. As a trigger for acquiring image information, a judgment result of other contents may be used, or when a complex condition is satisfied by using both the judgment result of seating judgment and the judgment result of other contents. , Image information may be acquired. The determination result of other contents is, for example, the detection result of the human body detection sensor that detects the existence of the human body using infrared rays or the like. In this case, for example, when the human body detection sensor detects that the user has approached the toilet bowl 30, image acquisition is started.

Next, the determination device 10 performs a determination process (step S12). The content of the determination process will be described with reference to FIG. The determination device 10 stores the determination result in the determination result storage unit 17 (step S13).
Next, the determination device 10 determines whether or not the user of the toilet device 3 has left the toilet by communicating with the toilet device 3 (step S14), and if it determines that the user has left the toilet, the process is performed. finish. On the other hand, when the determination device 10 determines that the user has not left the seat, the determination device 10 waits for a certain period of time (step S15) and returns to step S11.

Next, the flow of the determination process performed by the determination device 10 will be described with reference to FIG.
The determination device 10 estimates the presence or absence of urine in the image using the urine presence / absence estimation model 161 (step S122).

Further, the determination device 10 estimates the presence or absence of stool in the image using the stool presence / absence estimation model 162 (step S123), and determines the presence / absence of stool based on the estimation result (step S124).

When the determination device 10 estimates that there is stool in step S124 (step S124, YES), the determination device 10 estimates the stool properties using the stool property estimation model 163 (step S125).

Further, the determination device 10 estimates whether or not paper is used in the image by using the paper use or absence estimation model 165 (step S126).

When the determination device 10 estimates that paper is used in step S126 (step S127, YES), the determination device 10 estimates the amount of paper used using the paper usage estimation model 166 (step S128).
The determination device 10 determines the cleaning method of the toilet bowl 30 after use (step S129).

The content of the process in which the determination device 10 determines the cleaning method will be described with reference to FIG. In the flowchart shown in FIG. 6, the determination device 10 exemplifies and describes a case where the cleaning method is determined to be one of four, "large", "medium", "small", or "none". "Large", "medium", and "small" in the cleaning method mean that the cleaning intensity is small in the order of "large", "medium", and "small". The washing strength is the degree of strength for washing the toilet bowl 32. For example, the smaller the strength, the smaller the amount of washing water S, and the higher the strength, the larger the amount of washing water S. Alternatively, the smaller the strength, the smaller the number of washings, and the higher the strength, the larger the number of washings. If the cleaning method is "none", it means that the toilet bowl 32 is not washed.

The determination device 10 determines whether or not paper is used (step S130), and when it is determined that paper is used, determines whether or not the amount of paper used is large (step S131). The determination device 10 determines that the amount of paper used is large when the amount of paper estimated in step S126 is equal to or greater than a predetermined threshold value, and determines that the amount of paper used is small when the amount is less than the predetermined threshold value. .. When the determination device 10 determines that the amount of paper used is large (step S131, YES), the determination device 10 determines that the cleaning method is “large” (step S132).

When the determination device 10 determines that the amount of paper used is small (step S131, NO), the determination device 10 determines whether or not there is stool (step S133). The determination device 10 determines the presence or absence of stool according to the estimation result of the presence or absence of stool estimated in step S123. When the determination device 10 determines that there is stool (step S133, YES), the determination device 10 determines whether or not the amount of stool is large (step S134). In the stool properties estimated in step S125, the determination device 10 determines that the amount of stool is large when the amount of stool is estimated to be equal to or more than a predetermined threshold value, and determines that the amount of stool is less than the predetermined threshold value. It is judged that the amount of is small. When the determination device 10 determines that the amount of stool is large (step S134, YES), the determination device 10 determines that the cleaning method is “large” (step S132).

When the determination device 10 determines that the amount of stool is small (step S134, NO), the determination device 10 determines whether or not the shape of the stool is other than watery stool (step S135). When the determination device 10 estimates that the shape of the stool is not watery stool (that is, any of hard stool, normal stool, loose stool, and muddy stool) in the properties of the stool estimated in step S125. In addition, it is determined that the shape of the stool is other than watery stool, and when the shape of the stool is estimated to be watery stool, the shape of the stool is determined to be watery stool. When the determination device 10 determines that the shape of the stool is other than watery stool (step S135, YES), the determination device 10 determines the cleaning method as "medium" (step S136). On the other hand, when the determination device 10 determines that the shape of the stool is watery stool (step S135, NO), the determination device 10 determines that the cleaning method is "small" (step S138).

When the determination device 10 determines in step S133 that there is no stool (step S133, NO), the determination device 10 determines whether or not there is urine (step S137). The determination device 10 determines the presence or absence of urine according to the estimation result of the presence or absence of urine estimated in step S122. When the determination device 10 determines that there is urine (step S137, YES), the determination device 10 determines that the washing method is "small" (step S138). On the other hand, when the determination device 10 determines that there is no urine (step S137, NO), the determination device 10 determines that the cleaning method is "absent" (step S139).

As in the example of the flowchart shown in FIG. 6, the determination device 10 determines the cleaning method according to the combination of the results of estimating the presence / absence of urine, the presence / absence of stool, and the presence / absence of paper. When it is estimated that there is stool, the determination device 10 can finely control the amount of washing water by determining the washing method using the estimation result of the property of the stool, and can perform sufficient washing while performing sufficient washing. It is possible to control wasteful spending and save water appropriately.

In addition, the determination device 10 uses the estimation result of estimating the presence or absence of urine shown in step S122 and the estimation result of estimating the presence or absence of stool shown in step S123 to determine whether or not the user has excreted. May determine whether or not the patient has excreted. In this case, the determination device 10 determines that the user is not excreting when it is estimated that there is no urine and there is no stool.

As described above, the determination device 10 of the first embodiment includes an image information acquisition unit 11, an analysis unit 12, and a determination unit 13. The image information acquisition unit 11 acquires the image information of the image (target image) obtained by capturing the internal space 34 of the toilet bowl 32. The analysis unit 12 estimates the judgment items related to excretion of the target image by inputting the image information into the trained model. The determination unit 13 determines the determination items for the image based on the estimation result. The trained model is a model trained using the DL method. When learning is performed using the DL method, it is only necessary to label (associate) the result of judging the judgment items such as the presence or absence of excretion in the image, so it is necessary to extract the feature amount from the image and create the learning data. There is no. Therefore, there is no need for time to consider what kind of feature quantity should be extracted by what kind of method. That is, the determination device 10 of the first embodiment can reduce the time required for development in the analysis of the excretory behavior using machine learning.

Further, in the determination device 10 of the first embodiment, the target image is an image of the internal space 34 of the toilet bowl 32 after excretion. Thereby, for example, it is possible to reduce the number of images as compared with the case where hundreds of images obtained by continuously shooting images in which excrement is falling are targeted for determination. Therefore, the load required for estimation and determination can be reduced, and the time required for development can be reduced.

Further, in the determination device 10 of the first embodiment, the determination item includes at least one of the presence / absence of urine, the presence / absence of stool, and the properties of stool. As a result, the determination device 10 of the first embodiment can determine the excrement.

Further, in the determination device 10 of the first embodiment, the determination items include the presence or absence of paper used for excretion and the amount of paper used when paper is used. As a result, the determination device 10 of the first embodiment can determine the use of paper in excretion, and can be used as an index for determining, for example, a cleaning method of the toilet bowl 30 using the determination result. It becomes.

Further, in the determination device 10 of the first embodiment, the determination unit 13 determines a cleaning method for cleaning the toilet bowl 30 in the situation shown in the target image. As a result, the determination device 10 of the first embodiment can determine not only the excrement but also the cleaning method of the toilet bowl 30.

Further, in the determination device 10 of the first embodiment, the determination item includes at least one of the stool properties and the amount of paper used for excretion, and the analysis unit 12 determines the stool properties in the target image. And at least one of the amount of paper used in excretion is estimated, and the determination unit 13 determines a cleaning method for cleaning the toilet bowl 30 in the situation shown in the target image using the estimation result by the analysis unit 12. .. As a result, the determination device 10 of the first embodiment can determine an appropriate cleaning method according to the amount of excrement and paper used.

In this embodiment, the case where the determination of excrement and the determination of the cleaning method are performed has been described as an example, but only the determination of excrement or the determination of the cleaning method may be performed.

Further, in the determination device 10 of the first embodiment, the determination item includes determination of whether or not excretion has been performed. Thereby, for example, in the case of watching over the elderly in a facility for the elderly, it is possible to grasp whether or not the elderly have excreted with the toilet bowl device 3. It is also possible to examine the content of long-term care based on whether or not the elderly were excreted by themselves when they were guided to the toilet.
In addition, the health condition of the user may be determined using the determination result regarding excrement.

(Second Embodiment)
Next, the second embodiment will be described. This embodiment is different from the above-described embodiment in that the presence or absence of the use of paper is not the subject of the determination, and only the properties of the stool are the subject of the determination. Further, it differs from the above-described embodiment in that the target image is preprocessed. The preprocessing here is a process performed on the image for learning before the model is machine-learned for the image for learning. Further, the pre-processing is a process performed on the unlearned image before the trained model is input with the unlearned image. In the following description, only configurations different from the above-described embodiment will be described, and the same reference numerals will be given to configurations equivalent to the above-described embodiments, and the description thereof will be omitted.

FIG. 7 is a diagram for explaining the determination performed by the determination device 10 of the second embodiment. FIG. 7 shows a conceptual diagram when trying to classify a specific object into three types, types A, B, and C.
In general, when trying to classify things such as stools that can have various properties into three types A, B, and C based on the properties, it is difficult to clearly classify all the things. That is, there are many cases where types A, B, and C are mixed with each other. For example, as shown in FIG. 7, a region E1 that can be clearly classified as type A, a region E2 that is a mixture of types A and B classified as type A or B, and a region E2 that can be clearly classified as type B. Region E3, region E4 where types B and C classified as type B or C are mixed, region E5 which can be clearly classified as type C, region where type C and A classified as type C or A are mixed The region of E6 is generated.

When a trained model that classifies stool properties into three types, types A, B, and C, is constructed by DL, the accuracy of estimation may decrease in the region where types A, B, and C are mixed. Conceivable. In particular, when watery stool falls on the washing water S (reservoir surface) stored in the toilet bowl 32, the color of the stool is transferred from the dropped watery stool to the washing water S and diffuses. As a result, even if there is a stool with a different property from the watery stool excreted before the watery stool, the color of the stool with a different property from the watery stool and the color of the washed water S that has been transferred There is almost no difference. In this case, the trained model cannot recognize the properties of stools that have properties different from those of watery stools, and even though there are stools that have properties different from those of watery stools, it is estimated to be watery stools. , It is possible that an estimation error will occur. If there is an error in the estimation by the trained model, an error will occur in the determination of the target image.

As a countermeasure, in the present embodiment, the pretreatment removes elements (hereinafter referred to as noise components) that may cause an estimation error, such as turbidity of the washing water S. As a result, it is possible to reduce the estimation error by the trained model and reduce the determination error of the target image.

FIG. 8 is a block diagram showing a configuration of a determination system 1A to which the determination device 10A according to the second embodiment is applied. The determination system 1A includes, for example, a determination device 10A. The determination device 10A includes an image information acquisition unit 11A, an analysis unit 12A, a determination unit 13A, and a preprocessing unit 19.

The image information acquisition unit 11A captures image information of the internal space 34 of the toilet bowl 32 before excretion (reference image) and an image of the internal space 34 of the toilet bowl 32 after excretion (target image). To get. The pre-excretion is an arbitrary time point before the user of the toilet bowl device 3 excretes, for example, when the user enters a private room of the toilet or when the user sits on the toilet bowl 30.

The preprocessing unit 19 generates a difference image using the image information of the reference image and the target image. The difference image is an image showing the difference between the reference image and the target image. The difference here is the content that is captured in the target image and not captured in the reference image. That is, the difference image is an image expressing excrement, which is captured in the target image after excretion and not captured in the reference image before excretion.

The preprocessing unit 19 outputs the image information of the generated difference image to the analysis unit 12A. Further, the preprocessing unit 19 may store the image information of the generated difference image in the image information storage unit 15. The analysis unit 12A estimates the properties of the stool in the difference image using the trained model. The trained model used by the analysis unit 12A for estimation is a model in which the correspondence between the learning image showing the difference between the images before and after excretion and the result of evaluation of the stool properties is learned. Here, the image used for learning when creating the trained model (the image for learning showing the difference between the images before and after excretion) is an example of the "difference image for learning".

The determination unit 13A determines the properties of the stool shown in the target image based on the properties of the stool estimated by the analysis unit 12A. Further, the determination unit 13A may determine the excretion status of the user based on the stool properties estimated by the analysis unit 12A. The method by which the determination unit 13A determines the excretion status of the user will be described with reference to the flowchart of the present embodiment described later.

Here, regarding the method for generating the difference image, the preprocessing unit 19 is an RGB image in which the colors of the reference image, the target image, and the difference image are expressed by R (Red) G (Green) B (Blue). This case will be described as an example. However, each image is not limited to an image whose color is expressed by RGB, and even if it is an image other than the RGB image (for example, a Lab image or a YCbCr image), it can be generated by the same method. Here, the RGB value is information indicating the color of the image, and is an example of "color information".

The preprocessing unit 19 is based on the difference between the RGB value of the predetermined pixel in the reference image and the RGB value of the pixel corresponding to the predetermined pixel in the target image, and the preprocessing unit 19 of the pixel corresponding to the predetermined pixel in the difference image. Determine the RGB value. The pixel corresponding to a predetermined pixel is a pixel having the same or near position coordinates in the image. The difference here indicates the difference in color between the two pixels, and is determined based on, for example, the difference in RGB values. For example, the preprocessing unit 19 determines that there is no difference when the RGB values show the same color, and there is a difference when the RGB values do not show the same color.

For example, in the preprocessing unit 19, the RGB value of the predetermined pixel in the reference image is (255, 255, 0) (yellow), and the RGB value of the predetermined pixel in the target image is (255, 255, 0) (yellow). In some cases, since there is no color difference between the two pixels, mask processing is performed so that the RGB values of the predetermined pixels in the difference image are set to a predetermined color (for example, white) indicating that there is no difference.

On the other hand, in the preprocessing unit 19, the RGB value of the predetermined pixel in the reference image is (255, 255, 0) (yellow), and the RGB value of the predetermined pixel in the target image is (255, 0, 0) (red). In some cases, since there is a color difference between the two pixels, the RGB value of the predetermined pixel in the difference image is set to the RGB value (255, 0, 0) (red) of the predetermined pixel in the target image.

Alternatively, if there is a color difference between the two pixels, the preprocessing unit 19 may use a predetermined color (for example, black) indicating that fact.

Alternatively, if there is a color difference between the two pixels, the preprocessing unit 19 may use a predetermined color determined in advance according to the degree of the difference. The degree of difference is, for example, a value calculated according to the vector distance of the RGB values in the color space. In this case, the preprocessing unit 19 classifies the color difference between the two pixels into a plurality of colors according to the degree of the difference. For example, when the degree of difference is classified into three types (large, medium, and small), the preprocessing unit 19 sets the RGB value of the pixel in the difference image to black and the degree of difference for the pixel having a large degree of difference. For pixels with a medium value, the RGB value of the pixel in the difference image may be gray, and for pixels with a small degree of difference, the RGB value of the pixel in the difference image may be light gray to generate the difference image. ..

Here, it is conceivable that the amount of light irradiating the subject (internal space 34 of the toilet bowl 32) changes depending on the influence of the user's seating. When the amount of light changes, the shade of color may change even in places where there is no change before and after excretion. In such a case, it is conceivable that the preprocessing unit 19 determines that the change in color shading has a color difference.

As a countermeasure, the preprocessing unit 19 may determine the color of the predetermined pixel in the difference image according to the color ratio of the predetermined pixel in the reference image and the color ratio of the predetermined pixel in the target image. .. The color ratio is the ratio of each color of RGB, and is shown, for example, as a ratio with respect to a predetermined reference value. That is, the color ratio of the RGB values (R, G, B) is (R / L: G / L: B / L). Here, L indicates a predetermined reference value. The predetermined reference value L may be any value. Further, the predetermined reference value L may be a fixed value regardless of the RGB value, or may be a value that varies depending on the RGB value (for example, the R value of the RGB value).

For example, in the preprocessing unit 19, the predetermined pixels in the reference image are gray (RGB values (128, 128, 128)), and the predetermined pixels in the target image are light gray (RGB values (192, 192, 192)). In this case, since the color ratios of the two pixels are the same, it is determined that there is no color difference between the two pixels.

On the other hand, in the preprocessing unit 19, when the predetermined pixel in the reference image is yellow (RGB value (255, 255, 0)) and the predetermined pixel in the target image is red (RGB value (255, 0, 0)). , Since the color ratios of the two pixels are not the same, it is determined that there is a color difference between the two pixels.

FIG. 9 is a diagram illustrating a process performed by the pretreatment unit 19 according to the second embodiment. The left side of FIG. 9 shows the image G1 as an example of the reference image, the center shows the image G2 as an example of the target image, and the right side shows the image G3 as an example of the difference image.
As shown in the image G1 of FIG. 9, in the reference image, the internal space 34 before excretion is imaged, and the state in which the washing water S is stored in the opening 36 substantially in the center of the internal space 34 is imaged. Has been done.
As shown in the image G2 of FIG. 9, in the target image, the internal space 34 after excretion is imaged, and the excrement T1 in the direction between the front side and the rear side of the internal space 34 and the upper side of the washing water S, The appearance of T2 is imaged.
As shown in the image G3 of FIG. 9, the excrement T1 and T2, which are the differences between the reference image and the target image, are represented in the difference image.

Here, the process performed by the determination device 10A according to the second embodiment will be described with reference to FIG. FIG. 10 is a flowchart showing a flow of processing performed by the determination device 10A according to the second embodiment. Of the flowcharts shown in FIG. 10, steps S20, S22, S25 to S27, and S29 are the same as S10, S11, S12 to S14, and S15 in the flowchart of FIG. 4, and thus the description thereof will be omitted.

In step S21, the determination device 10A generates a reference image when it is determined that the user is seated on the toilet bowl 30. The reference image is an image showing the internal space 34 of the toilet bowl 32 before excretion. When the determination device 10A determines that the user is seated on the toilet bowl 30, the determination device 10A acquires the image information of the reference image by transmitting a control signal instructing the image pickup to the image pickup device 4.

In step S23, the determination device 10A performs mask processing using the reference image and the target image. The mask processing is a process of setting a predetermined color (for example, white) to pixels for which there is no difference between the reference image and the target image.
In step S24, the determination device 10A generates a difference image. The difference image is, for example, an image in which mask processing is performed on pixels for which there is no difference between the reference image and the target image, and the pixel value (RGB value) of the target image is reflected for pixels for which there is a difference between the reference image and the target image. is there.

In step S28, when the determination device 10A determines that the user has left the toilet bowl 30, the determination device 10A discards the image information of the reference image, the target image, and the difference image. Specifically, the determination device 10A erases the image information of the reference image, the target image, and the difference image stored in the image information storage unit 15. As a result, it is possible to prevent the storage capacity from becoming tight.

It has been explained that the determination process shown in step S25 of FIG. 10 is the same as the process shown in step S12 of FIG. 4, but in the present embodiment, at least the determination process using the properties of the stool as the determination item is performed. You just have to be.

Further, in step S25 of FIG. 10, the determination unit 13A determines the excretion status of the user by using the estimation result of the stool property in the difference image. For example, the determination unit 13A determines that the user's defecation status tends to be constipation when the shape of the stool is hard. The determination unit 13A determines that the defecation condition of the user is good when the shape of the stool is a normal stool. When the shape of the stool is loose, the determination unit 13A determines that the defecation status of the user is a condition requiring observation. When the shape of the stool is muddy stool or watery stool, the determination unit 13A determines that the defecation status of the user tends to be diarrhea. Alternatively, the determination unit 13A may determine the health condition of the user from the defecation status.

As described above, in the determination device 10A of the second embodiment, the preprocessing unit 19 generates a difference image showing the difference between the reference image and the target image. As a result, in the determination device 10A of the second embodiment, since the portion having a difference before and after excretion can be shown in the difference image, the property of the excrement can be grasped more accurately and the property can be more accurately grasped. It becomes possible to judge.

Further, in the determination device 10A of the second embodiment, the preprocessing unit 19 determines the difference between the color information indicating the color of the predetermined pixel in the reference image and the color information of the pixel corresponding to the predetermined pixel in the pixel of the target image. Based on, the color information of the pixel corresponding to the predetermined pixel in the difference image is determined. As a result, in the determination device 10A of the second embodiment, the portion having a color difference can be shown in the difference image before and after excretion, so that the same effect as the above-mentioned effect can be obtained.

Further, in the determination device 10A of the second embodiment, the preprocessing unit 19 sets the difference between the RGB value of the predetermined pixel in the reference image and the RGB value of the pixel corresponding to the predetermined pixel in the target image in the difference image. It is the RGB value of the pixel corresponding to the predetermined pixel. As a result, in the determination device 10A of the second embodiment, the difference in color before and after excretion can be recognized as the difference in RGB values, so that the color difference can be quantitatively calculated, and the same as the above-mentioned effect. It has a great effect.

Further, in the determination device 10A of the second embodiment, the preprocessing unit 19 corresponds to the color ratio indicating the ratio of the R value, the G value, and the B value of the predetermined pixel in the reference image and the predetermined pixel in the target image. The RGB value of the pixel corresponding to the predetermined pixel in the difference image is determined based on the difference from the color ratio of the pixel. As a result, in the determination device 10A of the second embodiment, even if there is a difference in the background color due to a difference in the amount of light applied to the subject before and after excretion, the difference is mistaken for excrement. The properties of excrement can be extracted without recognizing it, and the same effect as described above can be obtained.

In the above description, the case where the image information acquisition unit 11A acquires the image information of the reference image has been described as an example, but the present invention is not limited to this. For example, the image information of the reference image may be acquired by an arbitrary functional unit, or may be stored in the image information storage unit 15 in advance.

(Modification 1 of the second embodiment)
Next, a modification 1 of the second embodiment will be described. This modification differs from the above-described embodiment in that a divided image obtained by dividing the target image is generated as a preprocessing. In the following description, only configurations different from the above-described embodiment will be described, and the same reference numerals will be given to configurations equivalent to the above-described embodiments, and the description thereof will be omitted.

Generally, the toilet bowl 32 is formed so as to incline downward from the edge of the toilet bowl 32 toward the opening 36. Therefore, when there are a plurality of stools that have fallen into the toilet bowl 32, it is considered that the one that has fallen first moves downward along the inclined surface of the toilet bowl 32 so as to be pushed by the one that has fallen later. That is, there is a property that the one that has fallen first moves to the front side of the opening 36.

Utilizing this property, in this modified example, estimation is performed in consideration of the time series of excrement discharge. Specifically, the target image is divided into a front side and a rear side. Then, the excrement captured in the image obtained by dividing the front side of the target image (front divided image) is referred to as old stool, and the excrement captured in the image obtained by dividing the rear side of the target image (rear divided image) is referred to as new stool. Determining the properties of the stool. As a result, it is possible to determine the defecation status of the user based on the stool that is close to the current state by determining the old stool.

In this modification, the preprocessing unit 19 generates a divided image. The divided image is an image including a part of a region of the target image, and is, for example, a front divided image and a rear divided image. The boundary for dividing the target image into the front side divided image and the rear side divided image may be arbitrarily set. For example, in the left-right direction (left side) passing through the center of the water storage surface of the washing water S stored in the toilet bowl 32. It is divided by a line (in the direction connecting the right side).

The divided image is not limited to the above-mentioned front-side divided image and rear-side divided image. The divided image may be an image that includes at least a part of the target image. The target image may be an image divided into three regions in the front-rear direction (the direction connecting the front side and the rear side), or the front-side divided image may be an image further divided into a plurality of regions in the left-right direction. There may be. Further, one divided image may be generated from the target image, or a plurality of divided images may be generated. Further, when a plurality of divided images are generated from the target image, the region in which the regions shown in the plurality of divided images are combined may be the entire region shown in the target image, or may be a part of the region. You may.

The preprocessing unit 19 outputs the image information of the generated divided image to the analysis unit 12A. Further, the preprocessing unit 19 may store the image information of the generated divided image in the image information storage unit 15. The analysis unit 12A estimates the properties of the stool in the divided image using the trained model. The trained model used for estimation by the analysis unit 12A is a model that learns the correspondence between the training image obtained by dividing the image of the internal space 34 of the toilet bowl 32 in excretion and the result of evaluation of the stool properties. is there.

The determination unit 13A determines the excretion status of the user in the situation shown in the target image based on the properties of the stool in the divided image estimated by the analysis unit 12A. When there are a plurality of divided images generated from the target image, the determination unit 13A comprehensively looks at the estimation results of each of the divided images and determines the excretion status of the user. The method by which the determination unit 13A comprehensively determines the excretion status of the user will be described with reference to the flowchart of this modification described later.

Here, an image used for learning when creating a trained model (an image for learning obtained by dividing an image of the internal space 34 of the toilet bowl 32 in excretion) will be described. The divided image as the learning image in this modification is an example of the “learning divided image”. The divided image as the image for learning is an image obtained by dividing a part of various images of the internal space 34 of the toilet bowl 32, which was taken during the past excretion. The method of dividing the image by the preprocessing unit 23 may be arbitrary, but it is desirable that the method is the same as the method of dividing the image by the preprocessing unit 19. By using the same method, it is expected that the accuracy of estimation using the trained model will be improved. In addition, since the trained model is trained in a part of the target image, that is, a region narrower than the target image, the trained model can be trained more accurately than when the trained model is trained in the entire target image. It can be a model for estimating the state.

FIG. 11 is a diagram illustrating a process performed by the pretreatment unit 19 according to the first modification of the second embodiment. Image G4 is shown on the left side of FIG. 11 as an example of the target image, image G5 is shown as an example of the front split image in the center, and image G6 is shown as an example of the rear split image on the right side.
As shown in the image G4 of FIG. 11, in the target image, the internal space 34 is imaged, and the internal space 34 includes a state in which the washing water S is stored in the opening 36 substantially in the center of the internal space 34. The whole of is imaged.
As shown in the image G5 of FIG. 11, the front region of the internal space 34 is extracted from the front split image, and the boundary in the left-right direction passing through the center of the pool surface in which the wash water S is stored in the opening 36. The area in front of the line is extracted.
As shown in the image G6 of FIG. 11, the rear region of the internal space 34 is extracted from the rear split image, and the region posterior to the lateral boundary line passing through the center of the water reservoir surface is extracted. ing.

Here, the process performed by the determination device 10A according to the first modification of the second embodiment will be described with reference to FIG. FIG. 12 is a flowchart showing a flow of processing performed by the determination device 10A according to the first modification of the second embodiment. Of the flowcharts shown in FIG. 12, steps S30, S31, S33, S37, and S42 are the same as S10, S11, S14, S15, and S13 in the flowchart of FIG. 4, and thus the description thereof will be omitted.

In step S32, the determination device 10A generates a divided image using the target image. The divided image is, for example, a front divided image showing a region on the front side of the region captured in the target image, and a rear divided image showing the region on the rear side.

In step S34, the determination device 10A performs determination processing for each of the front-side divided image and the rear-side divided image. Since the content of this determination process is the same as the process shown in step S25 in the flowchart of FIG. 10, the description thereof will be omitted.

In step S35, if the determination device 10A does not determine that the user has left the toilet bowl 30 (steps S33, NO), the determination device 10A determines whether or not the local cleaning operation on the toilet bowl 30 has been performed, and determines whether or not the local cleaning operation on the toilet bowl 30 has been performed. When the cleaning operation is performed, the process shown in step S34 is performed.
In step S36, when the determination device 10A does not determine that the local cleaning operation on the toilet bowl 30 has been performed (step S35, NO), the determination device 10A determines whether or not the toilet bowl cleaning operation on the toilet bowl 30 has been performed, and the toilet bowl 30 When the toilet bowl cleaning operation is performed, the process shown in step S34 is performed.

In step S38, the determination device 10A determines whether or not there is a determination result in both the front-side divided image and the rear-side divided image. The fact that both sides have a judgment result means that both sides have an image of a stool and that each stool image has a judgment result regarding the properties.
In step S39, when the determination device 10A has a determination result in both the front divided image and the rear divided image, the determination result of the front divided image is the determination result of the old flight, and the determination result of the rear divided image is the determination result of the new flight. It is the judgment result.

In step S40, the determination device 10A performs a determination process by the determination unit 13A. The confirmation process is a process for determining the excretion status of the user by using the determination result of the new stool and the determination result of the old stool. The determination device 10A determines the excretion status by assuming that the old stool reflects the current excretion status, for example. In the confirmation process, for example, when the property of the old stool is determined to be hard stool and the property of the new stool is determined to be normal stool, the determination unit 13A discharges the hard stool remaining in the large intestine during defecation. It is determined that the user's excretion status tends to be constipation. On the other hand, in the confirmation process, when the property of the old stool is determined to be normal stool and the property of the new stool is determined to be muddy stool, for example, the determination unit 13A has a good excretion condition of the user. Is determined.

In step S41, when the determination unit 13A has a determination result of only one of the front side divided image and the rear side divided image, the determination device 10A determines whether or not there is a determination result of the front side divided image. If there is a determination result of the front split image, the process shown in step S40 is performed using the determination result of the front split image. If there is no determination result of the front split image, the process shown in step S40 is performed using the determination result of the rear split image. The case where there is no determination result of the front side divided image is, for example, a case where excrement is not imaged in the front side divided image and the property of stool cannot be determined.

As described above, in the determination device 10A according to the first modification of the second embodiment, the preprocessing unit 19 generates a divided image including a part of the target image. As a result, in the determination device 10A according to the first modification of the second embodiment, a part of the target image can be the target of the determination, which is narrower than the case where the entire target image is the target of the determination. The area can be determined in detail, and the determination can be made more accurately.

Further, in the determination device 10A according to the first modification of the second embodiment, the preprocessing unit 19 generates a front side divided image showing at least the front side region of the toilet bowl in the target image. As a result, in the determination device 10A according to the first modification of the second embodiment, when the new stool and the old stool are imaged in the target image, the region considered to be the image of the new stool is regarded as the divided image. can do. Further, even when the new stool and the old stool are not imaged in the target image, the region where the stool is likely to be imaged can be a divided image, and the same effect as the above-mentioned effect can be obtained.

Further, in the determination device 10A according to the first modification of the second embodiment, the preprocessing unit 19 generates a front-side divided image and a rear-side divided image, and the analysis unit 12A relates to the determination items for the front-side divided image. In addition to performing the estimation, the determination items are estimated for the rear divided image, and the determination unit 13A uses the estimation results for the front divided image and the estimation results for the rear divided image to determine the determination items for the target image. Make a judgment about. As a result, the determination device 10A according to the first modification of the second embodiment can comprehensively determine the excretion status of the user by using the estimation results of the front-side divided image and the rear-side divided image. This makes it possible to make a more accurate determination than when using the estimation result of either the front-side divided image or the rear-side divided image.

Further, in the determination device 10A according to the first modification of the second embodiment, the preprocessing unit 19 sets the estimation result for the front split image to be older than the estimation result for the rear split image in time series. As a result, the target image is determined regarding the determination items. As a result, in the determination device 10A according to the first modification of the second embodiment, the estimation result for the front split image is regarded as the estimation result for the old stool, and the estimation result for the rear split image is regarded as the estimation result for the new stool. , It is possible to make a determination in consideration of the time series of excretion, and it is possible to make an accurate determination of the excretion status of the user closer to the current state.
Since the direction in which the previously dropped stool moves changes depending on the shape of the toilet bowl 32, the time-series relationship associated with the front-side divided image and the rear-side divided image may be reversed. That is, in the above, the front split image has been described as being older than the rear split image in time series, but the present invention is not limited to this, and the front split image is newer in time series than the rear split image. Assuming that, the process related to the determination (confirmation) may be performed.

(Modification 2 of the second embodiment)
Next, a modification 2 of the second embodiment will be described. This modification differs from the above-described embodiment in that, as preprocessing, an entire image showing the entire target image and a partial image obtained by cutting out a part of the target image are generated. In the following description, only configurations different from the above-described embodiment will be described, and the same reference numerals will be given to configurations equivalent to the above-described embodiments, and the description thereof will be omitted.

In general, if a machine learning method is used to estimate detailed judgment contents from the entire image, high computing power is required and the device cost increases. For example, if the number of DNN layers used in the model is increased, the number of operations required for one trial increases because the number of nodes increases, and the processing load increases. Further, in order for the model to be able to estimate the detailed contents (that is, the error between the output of the model and the output of the training data with respect to the input of the training data is minimized), the weight W and the bias component b are changed. However, it is necessary to repeat the trials many times, and in order to converge such repeated trials in a realistic time, a device capable of processing a huge amount of operations at high speed is required. That is, if it is attempted to analyze the entire target image in detail, a high-performance device is required, and the device cost increases.

By the way, the target image is an image obtained by capturing the entire internal space 34 of the toilet bowl 32. That is, the target image has a region in which excrement is imaged and a region in which excrement is not imaged. Therefore, a method is conceivable in which a specific region (for example, a region near the opening 36) where excrement is likely to fall is cut out from the target image, and detailed determination contents are estimated for the cut out region. .. As a result, the area of the image to be analyzed can be narrowed, and an increase in device cost can be suppressed.

However, it is originally unknown where in the toilet bowl 32 the excrement falls. In addition, the properties of stool change depending on the physical condition of the user. Therefore, even if the area where the excrement falls is often a specific area of the toilet bowl 32, the excrement does not always fall only in the specific area and may be scattered around. obtain. If the judgment is made using only the image of a specific area without using the surrounding image even though it is scattered in the surroundings, the judgment may be different from the actual situation.

As a countermeasure for this, in this modification, a whole image showing the entire target image and a partial image obtained by cutting out a part of the target image are generated by preprocessing.

For the entire image, the increase in equipment cost is suppressed by making a global judgment that is not detailed. The global determination is an overall (global) determination as compared with determining the properties of the stool, and is, for example, determining the presence or absence of stool scattering. Since the presence or absence of scattered stools does not determine the properties of the scattered stools, it can be said that the determination is relatively rough and easy as compared with the case of determining the properties of the stools. Here, the determination regarding the presence or absence of stool scattering performed on the entire image is an example of the "first determination item".

On the other hand, for the partial image, a judgment is made for more detailed judgment items than the judgment for the whole image. The detailed determination item is, for example, to determine the properties of the stool. By targeting a partial image in which the area of the image is narrowed, detailed judgment can be performed without using a high-performance device, and an increase in device cost can be suppressed. Here, the determination regarding the properties of the stool performed on the partial image is an example of the "second determination item".

In this modification, the preprocessing unit 19 generates an entire image and a partial image. The whole image is an image showing the whole of the target image, for example, the target image itself. The partial image is an image obtained by cutting out a part of a region of the target image, for example, an image obtained by cutting out a region near the opening 36 from the target image. As a partial image, which region of the target image to be cut out may be arbitrarily set, and is, for example, a fixed region determined at the time of shipment according to the shape of the toilet bowl 30.

The preprocessing unit 19 outputs the image information of the generated whole image and partial image to the analysis unit 12A. Further, the preprocessing unit 19 may store the generated image information of the entire image and the partial image in the image information storage unit 15.

The analysis unit 12A estimates the presence or absence of stool scattering in the entire image using the trained model. Here, the estimation of the presence or absence of stool scattering in the entire image is an example of the "first estimation".

In addition, the analysis unit 12A estimates the properties of the stool in the partial image using the trained model. Here, the estimation of the stool properties in the partial image is an example of the "second estimation".

The determination unit 13A determines the excretion status of the user in the situation shown in the target image based on the presence or absence of stool scattering in the entire image estimated by the analysis unit 12A and the properties of the stool in the partial image. The method by which the determination unit 13A determines the excretion status of the user based on the estimation result in the whole image and the estimation result in the partial image will be described in the flowchart of this modification described later.

Here, the training data to be trained by the trained model used in this modification will be described.
The trained model used for estimating the whole image is a model that learns the correspondence between the whole image for learning that images the entire internal space 34 of the toilet bowl 32 in excretion and the evaluation result that evaluates the presence or absence of stool scattering. Is. The whole image for learning is various images showing the whole internal space 34 of the toilet bowl 32 taken during the past excretion. The whole image for learning (an image for learning that captures the entire internal space 34 of the toilet bowl 32 in excretion) is an example of the “whole image for learning”.
The trained model used for estimating the partial image is a correspondence between the learning partial image obtained by cutting out a part of the entire internal space 34 of the toilet bowl 32 in excretion and the evaluation result of evaluating the properties of the stool. It is a model that learned the relationship. The partial image for learning is an image obtained by cutting out a part of the whole image. The learning partial image (a learning image obtained by cutting out a part of the entire internal space 34 of the toilet bowl 32 in excretion) is an example of the “learning partial image”.
The method for generating the whole image for learning and the partial image for learning may be arbitrary, but it is desirable that the method is the same as the method for generating the whole image and the partial image by the preprocessing unit 19. By using the same method, it is expected that the accuracy of estimation using the trained model will be improved.

FIG. 13 is a diagram illustrating a process performed by the pretreatment unit 19 according to the second modification of the second embodiment. The left side of FIG. 13 shows an image G7 as an example of a target image, the center shows an image G8 as an example of an entire image, and the right side shows an image G9 as an example of a partial image.
As shown in the image G7 of FIG. 13, in the target image, the internal space 34 is imaged, and the internal space 34 includes a state in which the washing water S is stored in the opening 36 substantially in the center of the internal space 34. The whole of is imaged.
As shown in the image G8 of FIG. 13, the entire target image is shown in the overall image. The whole image may be the target image itself, or the whole image may be extracted from the target image.
As shown in the image G9 of FIG. 13, in the partial image, a region near the opening 36 at the substantially center of the internal space 34 is extracted, and the pool surface in which the wash water S is stored and the region around the pool surface are extracted. It has been extracted.

Here, the process performed by the determination device 10A according to the second modification of the second embodiment will be described with reference to FIG. FIG. 14 is a flowchart showing a flow of processing performed by the determination device 10A according to the second modification of the second embodiment. Of the flowcharts shown in FIG. 14, steps S50, S51, S53, S57, and S62 are the same as S10, S11, S14, S15, and S13 in the flowchart of FIG. 4, and thus the description thereof will be omitted. Further, among the flowcharts shown in FIG. 14, steps S55 and S56 are the same as those of the flowcharts S35 and S36 of FIG. 12, so the description thereof will be omitted.

In step S52, the determination device 10A generates a whole image and a partial image using the target image. The whole image is, for example, an image showing the whole area captured by the target image. The partial image is, for example, an image showing a specific part of a region captured in the target image.

In step S54, the determination device 10A performs determination processing for each of the entire image and the partial image.
The determination device 10A performs a global determination on the entire image, for example, a determination of the presence or absence of stool scattering. The determination device 10A estimates the presence or absence of stool scattering in the entire image using the trained model, and uses the estimated result as the determination result for determining the presence or absence of stool scattering in the entire image. The trained model here is a model created by learning using learning data in which the entire image for learning and the determination result of determining the presence or absence of stool scattering are associated with each other.
Further, the determination device 10A performs detailed determination on the partial image, for example, determination of the property of the stool. The determination device 10A estimates the stool properties in the partial image using the trained model, and uses the estimated result as the determination result for determining the stool properties in the partial image. The trained model here is a model created by learning using learning data in which a partial image for learning and a determination result of determining the properties of stool are associated with each other.

In step S58, the determination device 10A determines whether or not there is a determination result in both the entire image and the partial image. The fact that both have determination results means that the presence or absence of stool scattering is determined in the entire image, and the stool properties are determined in the partial image.

In step S59, when the determination unit 13A has a determination result in both the entire image and the partial image, the determination device 10A corrects the determination result of the partial image by using the determination result of the entire image. Correcting the determination result of the partial image means changing or supplementing the determination result of the partial image by using the determination result of the whole image.
For example, the determination unit 13A determines that the condition of defecation is diarrhea when it is determined that the stool is loose and the stool is scattered from the determination result of the whole image as the determination result of the partial image. to correct. On the other hand, the determination unit 13A does not correct the defecation status as the determination result of the partial image when it is determined from the determination result of the entire image that there is no scattering of stool.

In step S60, the determination device 10A performs a determination process by the determination unit 13A. The confirmation process is a process of determining the defecation status of the user by using the determination result of the entire image and the determination result of the portion.

In step S61, when the determination unit 13A has no determination result in both the whole image and the partial image, the determination device 10A determines whether or not there is a determination result of the partial image. When there is a determination result of the partial image, the process shown in step S60 is performed using the determination result of the partial image. If there is no determination result of the partial image, the process shown in step S60 is performed using the determination result of the entire image. The case where there is no determination result of the partial image is, for example, a case where the excrement is not imaged in the partial image and the property of the stool cannot be determined.

As described above, in the determination device 10A according to the second modification of the second embodiment, the preprocessing unit 19 generates a whole image and a partial image from the target image. The analysis unit 12A makes a global estimation (first estimation) from the whole image using the trained model, and makes a detailed estimation (second estimation) from the partial image using another trained model. As a result, in the determination device 10A according to the second modification of the second embodiment, the overall image having a large number of pixels is used to perform a relatively easy global estimation, so that the details are relatively difficult to understand from the overall image. It is possible to reduce the load of arithmetic processing and suppress an increase in device cost as compared with the case of performing a simple estimation. Further, by performing detailed estimation using a partial image having a relatively small number of pixels, the load of arithmetic processing can be reduced and the device cost can be reduced as compared with the case of performing detailed estimation from the entire image having a relatively large number of pixels. Can be suppressed from increasing.

Further, in the determination device 10A according to the second modification of the second embodiment, the preprocessing unit 19 generates a partial image including at least the opening 36 of the toilet bowl 32 in the target image. As a result, in the determination device 10A according to the second modification of the second embodiment, the region where the excrement is likely to fall can be cut out as a partial image, and the detailed determination regarding the excrement is made using the partial image. It becomes possible to do.

Further, in the determination device 10A according to the second modification of the second embodiment, the preprocessing unit 19 estimates the presence or absence of stool scattering as a global estimation (first estimation), and makes a detailed estimation (second estimation). ) To estimate the properties of the stool. As a result, the determination device 10A according to the second modification of the second embodiment can estimate the presence or absence of scattering together with the properties of the stool, and can make a more accurate determination using the estimation results of both. It becomes.

Further, in the determination device 10A according to the second modification of the second embodiment, the estimation result of the global estimation (first estimation) is used to correct the estimation result of the detailed estimation (second estimation). As a result, the determination device 10A according to the second modification of the second embodiment can correct the detailed estimation and can perform the determination more accurately.

(Third Embodiment)
Next, a third embodiment will be described. This embodiment differs from the above-described embodiment in that a target region in the target image is extracted. The target area is an area to be determined in the present embodiment, and is an area to be determined for the properties of excrement. That is, the determination region is a region where excrement is presumed to be imaged in the target image. In the following description, only configurations different from the above-described embodiment will be described, and the same reference numerals will be given to configurations equivalent to the above-described embodiments, and the description thereof will be omitted.

FIG. 15 is a block diagram showing a configuration of the determination device 10B according to the third embodiment. The determination device 10B includes an analysis unit 12B and a determination unit 13B. Here, the analysis unit 12B is an example of the “extraction unit”.

The analysis unit 12B uses a region of a color close to the assumed color as a determination region based on the difference (color difference) between the color of the image in the target image and the predetermined color assumed for excrement (hereinafter referred to as the assumed color). Extract. The analysis unit 12B determines whether or not the color in the target image is close to the assumed color based on the distance in the color space of both colors (hereinafter, referred to as spatial distance). When the spatial distance between the two colors is small, it indicates that the color difference is small, indicating that the two colors are close to each other. On the other hand, when the spatial distance is large, it indicates that the color difference is large, and that the two colors are distant from each other. Here, the spatial distance is an example of "characteristics of assumed color".

Here, a method in which the analysis unit 12B calculates the spatial distance will be described. Hereinafter, the case where the target image is an RGB image and the assumed color is a color indicated by an RGB value will be described as an example, but the present invention is not limited to this. The same method is used even when the target image is an image other than the RGB image (for example, a Lab image or a YCbCr image) or the assumed color is indicated by a color other than the RGB value (for example, a Lab value or a YCbCr value). It is possible to extract the determination area. Further, in the following, the case where the assumed color is the color of stool will be described as an example, but the present invention is not limited to this. The assumed color may be any color expected for excrement, and may be, for example, the color of urine.

The analysis unit 12B calculates, for example, the Euclidean distance in the color space as the spatial distance. The analysis unit 12B calculates the Euclidean distance using the following equation (1). In the equation (1), Z1 is the Euclidean distance, ΔR is the difference in R value between the predetermined pixel X and the assumed color Y in the target image, ΔG is the difference in G value between the pixel X and the assumed color Y, and ΔB is the pixel X. It is the difference between the B value and the assumed color Y. The RGB values of the predetermined pixels X in the target image are (red, green, blue), and the RGB values of the assumed color Y are (Rs, Gs, Bs).

Z1 = (ΔR ^ 2 + ΔG ^ 2 + ΔB ^ 2) ^ (1/2)… (1)
However,
ΔR = red-Rs
ΔG = green-Gs
ΔB = blue-Bs

Further, the analysis unit 12B may perform weighting when calculating the spatial distance. The weighting is for emphasizing the difference between specific elements that make up a color, for example, by multiplying the R, G, and B elements that make up a color by different coefficients (weighting coefficients). Do. By weighting, the color difference from the assumed color can be emphasized according to the element.

The analysis unit 12B can calculate the weighted Euclidean distance by using, for example, the following equation (2). In the equation (2), Z2 is the weighted Euclidean distance, R_COEF is the weighting coefficient of the R element, G_COEF is the coefficient of the G element, and B_COEF is the weighting coefficient of the B element. ΔR is the difference in R value between the pixel X and the assumed color Y, ΔG is the difference in the G value between the pixel X and the assumed color Y, and ΔB is the difference in the B value between the pixel X and the assumed color Y. The RGB values of the predetermined pixels X in the target image are (red, green, blue), and the RGB values of the assumed color Y are (Rs, Gs, Bs).

Z2 = (R_COEF × ΔR ^ 2
+ G_COEF × ΔG ^ 2
+ B_COEF × ΔB ^ 2) ^ (1/2)… (2)
However,
R_COEF>G_COEF> B_COEF
ΔR = red-Rs
ΔG = green-Gs
ΔB = blue-Bs

Here, as for the characteristics of the color of the stool as the assumed color Y, the R element tends to be stronger than the G element, and the G element tends to be stronger than the B element. Based on the characteristics of each element constituting such a color, the analysis unit 12B sets the weighting coefficient of the R element to a value larger than the weighting coefficient of the G element. That is, in the equation (2), the relationship of R_COEF> G_COEF> B_COEF is established for the coefficient R_COFE, the coefficient G_COFE, and the coefficient B_COFE.

By the way, it is conceivable that the amount of light irradiating the subject (internal space 34 of the toilet bowl 32) changes depending on the influence of the user's seating. When the amount of light changes, even excrement of the same color may be imaged as if the shade of the color is different. In such a case, the spatial distance is calculated as a different distance even though the colors are the same.

As a countermeasure, the analysis unit 12B may calculate the Euclidean distance of the ratio of each element constituting the color (hereinafter referred to as the color ratio) as the spatial distance. The color ratio is determined by, for example, dividing the values of the other elements by the values of the reference element among the R value, the G value, and the B value. By using the color ratio, it is possible to calculate the spatial distance so as not to reflect the difference due to the shade of color.

The element to be used as a reference when deriving the color ratio may be arbitrarily determined, but for example, it can be considered to be the dominant element of the color. For example, in the color of stool, the R element is dominant. Therefore, in the present embodiment, the color ratio is created by dividing each of the R value, the G value, and the B value by the R value.

For example, the color ratio of pixel X (RGB values (red, green, blue)) is (red / red, green / red, blue / red), that is, (1, green / red, blue / red). Further, the color ratio of the assumed color Y (RGB values (Rs, Gs, Bs)) is (Rs / Rs, Gs / Rs, Bs / Rs), that is, (1, Gs / Rs, Bs / Rs). ..

The analysis unit 12B can calculate the Euclidean distance of the color ratio by using the following equation (3). In equation (3), Z3 is the Euclidean distance of the color ratio, ΔRp is the difference between the R elements between the color ratio of the pixel X and the color ratio of the assumed color Y, and ΔGp is the color ratio of the pixel X and the color ratio of the assumed color Y. The difference of the G element, ΔBp, is the difference of the B element between the color ratio of the pixel X and the color ratio of the assumed color Y. GR_RATE is the ratio of the G element in the color ratio of the assumed color Y, and BR_RATE is the ratio of the B element in the color ratio of the assumed color Y. The RGB values of the predetermined pixels X in the target image are (red, green, blue), and the RGB values of the assumed color Y are (Rs, Gs, Bs).

Z3 = (ΔRp ^ 2 + ΔGp ^ 2 + ΔBp ^ 2) ^ (1/2)
= (ΔGp ^ 2 + ΔBp ^ 2) ^ (1/2)… (3)
However,
ΔRp = red / red-Rs / Rs = 0 (zero)
ΔGp = green / red-GR_RATE
ΔBp = blue / red-BR_RATE
GR_RATE = Gs / Rs
BR_RATE = Bs / Rs
1>GR_RATE>BR_RATE> 0

The characteristic of the color of the stool as the assumed color Y tends to be that the R element is stronger than the G element (Rs> Gs) and the G element is stronger than the B element (Gs> Bs). The ratio GR_RATE and the ratio BR_RATE are both values between 0 (zero) and 1. Further, the ratio BR_RATE is smaller than the ratio GR_RATE. That is, in the equation (3), the relationship of 1> GR_RATE> BR_RATE> 0 is established between the ratio GR_RATE and the ratio BR_RATE.

Further, the analysis unit 12B may weight a specific element constituting the color ratio when calculating the Euclidean distance of the color ratio. The analysis unit 12B can calculate the weighted Euclidean distance of the color ratio by using the following equation (4). In equation (4), Z4 is the Euclidean distance weighted by the color ratio. ΔRp is the difference between the R values of the pixel X and the assumed color Y, ΔGp is the difference between the G values of the pixel X and the assumed color Y, and ΔBp is the difference between the B values of the pixel X and the assumed color Y. GR_COEF is a weighting coefficient of the difference ΔGp, and BR_COEF is a weighting coefficient of the difference ΔBp. The RGB values of the predetermined pixels X in the target image are (red, green, blue), and the RGB values of the assumed color Y are (Rs, Gs, Bs).

Z4 = (GR_COEF × ΔGp ^ 2
+ BR_COEF × ΔBp ^ 2) ^ (1/2)… (4)
However,
GP_COEF> BP_COEF
ΔGp = green / red-GR_RATE
ΔBp = blue / red-BR_RATE
GR_RATE = Gs / Rs
BR_RATE = Bs / Rs
1>GR_RATE>BR_RATE> 0

In the equation (4), the relationship of GP_COEF> BP_COEF is established for the coefficients GR_COFE and the coefficient BR_COFE in the same manner as the relationship of the coefficient G_COFE and the coefficient B_COFE in the equation (2). Further, in the equation (4), the relationship of 1> GR_RATE> BR_RATE> 0 is established between the ratio GR_RATE and the ratio BR_RATE, as in the equation (3). For example, the ratio GR_RATE = 0.7, the ratio BR_RATE = 0.3, the coefficient GR_COFE = 40, and the coefficient BR_COFE = 1 are set.

The analysis unit 12B creates an image (grayscale target image) in which the spatial distance calculated for each pixel of the target image is grayscale. For example, the analysis unit 12B adjusts the scale of the spatial distance by using the equation (5) and converts it into a gray scale value. In equation (5), Val is a grayscale value, AMP is a coefficient for scale adjustment, and Z is a spatial distance. The spatial distance Z may be any of RGB value Euclidean distance Z1, RGB value weighted Euclidean distance Z2, color ratio Euclidean distance Z3, and color ratio weighted Euclidean distance Z4. Further, Z_MAX is the maximum value of the spatial distance calculated for each pixel of the target image, and Val_MAX is the maximum value of the gray scale value.

Val = AMP × Z… (5)
However,
AMP = Val_MAX / Z_MAX

For example, when the grayscale gradation is shown in 256 steps from 0 to 255 in the grayscale target image, the maximum value Val_MAX of the grayscale value is 255. In this case, the spatial distance Z is converted to the grayscale value Val so that the maximum value X_MAX of the spatial distance becomes the maximum value Val_MAX (255) of the grayscale according to the equation (5). As a result, the analysis unit 12B creates a grayscale target image in which the spatial distance from the assumed color is expressed by a grayscale value of 0 (white) to 255 (black).

Here, a method in which the analysis unit 12B extracts the determination region will be described with reference to FIG. FIG. 16 is a diagram illustrating a process performed by the analysis unit 12B according to the third embodiment. In FIG. 16, the grayscale axis is shown in the left-right direction, and it is shown that the grayscale value increases as the grayscale axis moves from left to right.

As shown in FIG. 16, in a grayscale target image, pixels having a small spatial distance are represented by a small grayscale value. That is, a color closer to the color of the stool as the assumed color is represented by a small gray scale value, and a region having a small gray scale value can be regarded as a region in which the stool is imaged. On the other hand, in the grayscale target image, pixels having a large spatial distance are represented by a large grayscale value. That is, a color distant from the color of the stool as the assumed color is expressed by a large gray scale value, and a region having a large gray scale value can be regarded as a “non-stool” region in which the stool is not imaged.

Utilizing this property, the analysis unit 12B extracts a determination region. Specifically, the analysis unit 12B sets a region in which the grayscale value of the pixel in the grayscale target image is less than a predetermined first threshold value (threshold value 1) as a region containing excrement, and has the excrement as a determination region. Extract the area. The first threshold value is a grayscale value corresponding to a boundary that distinguishes the color of the washing water S stored in the toilet bowl 32 from the color of the watery stool.

Further, when comparing the colors of hard stool and watery stool, it is considered that the watery stool is thinner than the hard stool because it is dissolved in water (washing water S). In this case, the grayscale value corresponding to the color of watery stool is expressed in dark gray indicating that the color of the stool is farther from the assumed color than the grayscale value corresponding to hard stool.

Utilizing this property, the analysis unit 12B extracts the watery stool region and the hard stool region separately from the determination region. Specifically, the analysis unit 12B sets the region where the grayscale value is less than a predetermined second threshold value (threshold value 2) as the hard stool region and the region where the grayscale value is equal to or higher than the second threshold value among the determination regions in the grayscale target image. Area of water-soluble stool. The second threshold value is set to a value smaller than the first threshold value. Here, the region of water-soluble stool is an example of a "determination region". The hard stool area is an example of a "judgment area".

Alternatively, when the determination region contains a watery stool region and a hard stool region, the analysis unit 12B distinguishes and extracts the two regions (watery stool region and hard stool region). It may be. When two areas are mixed in the judgment area, the grayscale range that the pixels included in the judgment area can take is the combination of the grayscale range that water-soluble stool can take and the grayscale range that hard stool can take. Because it is a wide range, it is a relatively wide range. On the other hand, when only one region (watery stool region or hard stool region) exists in the determination region, the grayscale range that the pixels included in the determination region can take is a relatively narrow range. ..

Utilizing this property, the analysis unit 12B determines whether or not the region of watery stool and the region of hard stool are mixed in the determination region according to the grayscale range of the pixels included in the determination region. To do. For example, the analysis unit 12B sets the difference between the maximum value and the minimum value of the gray scale in the pixels included in the determination region as the range of the gray scale. When the grayscale range in the determination region is less than a predetermined difference threshold value, the analysis unit 12B does not mix the watery stool region and the hard stool region in the determination region (that is, the determination region is water. (Only the area of grayscale or the area of hard stool). When the grayscale range in the determination region is equal to or greater than a predetermined difference threshold value, the analysis unit 12B includes a watery stool region and a hard stool region in the determination region. The difference threshold corresponds to, for example, a wide range, a narrow range, or a representative value of two ranges, depending on the gray scale range that water-soluble stool can take and the gray scale range that hard stool can take. Is set to the value. The representative value may be any of commonly used representative values such as a simple addition mean value, a weighted mean value, and a median value in two ranges.

The analysis unit 12B outputs the information of the image (extracted image) indicating the extracted determination region to the determination unit 13B. In this case, when the determination region contains a watery stool region and a hard stool region, the analysis unit 12B determines that the determination region is an image showing the watery stool region (water-like region extracted image). The information of the image (hard part extraction image) showing the hard stool area in the area is output to the determination unit 13B. On the other hand, when the determination region does not include the watery stool region and the hard stool region, the analysis unit 12B has an image showing the watery stool region as the determination region (watery stool extraction image) or the determination region. The information of the image showing the region of hard stool (hard stool extraction image) is output to the determination unit 13B.

Returning to FIG. 15, the determination unit 13B determines the determination items based on the extracted image acquired by the analysis unit 12B. Specifically, the determination unit 13B determines the properties of the watery stool using the watery stool extract image. The determination unit 13B determines the properties of hard stool using the hard stool extract image. The determination unit 13B determines the properties of watery stool using the image extracted from the watery part. The determination unit 13B determines the properties of hard stool using the hard part extracted image.

The determination unit 13B may determine the properties of the stool using the estimation result by machine learning, as in the other embodiments described above. In this case, the analysis unit 12B may perform the estimation by machine learning, or another functional unit may perform the estimation. In addition, the determination unit 13B may determine the properties of the stool by using another image analysis method. In this case, the determination device 10B can omit the learned model storage unit 16.

The determination unit 13B does not need to analyze the entire target image because the determination area is extracted by the analysis unit 12B and the image whose range is narrowed down can be the target of analysis. Further, since the determination unit 13B can analyze an image in which watery stool or hard stool is distinguished, a case where an image in which watery stool or hard stool is not distinguished is targeted for analysis. Compared with, the process of determining the properties becomes easier.

Here, the process performed by the determination device 10B of the third embodiment will be described with reference to FIG. FIG. 17 is a flowchart showing a flow of processing performed by the determination device 10B according to the third embodiment. This flowchart shows the flow of processing after the processing for acquiring image information is performed. The process of acquiring image information is a process corresponding to step S11 of the flowchart shown in FIG. 4, and corresponds to the process described as “camera image” in this flowchart.

The analysis unit 12B grayscales the target image to create a grayscale target image (step S70).
The analysis unit 12B determines whether or not the grayscale value for each pixel in the grayscale target image is less than the first threshold value (threshold value 1) (step S71).
The analysis unit 12B calculates the difference D between the maximum value and the minimum value of the gray scale value for the pixel group in the determination region where the gray scale value is less than the first threshold value (threshold value 1) in the gray scale target image (step). S72).

The analysis unit 12B determines whether or not the difference D is less than the predetermined difference threshold value a (step S73), and if the difference D is less than the predetermined difference threshold value a, the determination region includes the region of watery stool and the hardness. It is determined that the stool area is not mixed, and the process proceeds to the process shown in step S74. In step S74, the analysis unit 12B determines whether or not the grayscale value for each pixel in the determination region is less than the second threshold value (threshold value 2).
When the grayscale value for each pixel in the determination region is less than the second threshold value (threshold value 2), the analysis unit 12B outputs the hard stool extraction image to the determination unit 13B (step S75). The determination unit 13B determines the properties of stools containing only hard stools (specializing in hard stools) based on the hard stool extract image (step S82).
When the grayscale value for each pixel in the determination region is equal to or greater than the second threshold value (threshold value 2), the analysis unit 12B outputs the watery stool extract image to the determination unit 13B (step S76). The determination unit 13B determines the properties of stools containing only watery stools (specializing in watery stools) based on the extracted image of watery stools (step S83).

On the other hand, when the difference D is equal to or greater than the predetermined difference threshold value a (step S73, NO), the analysis unit 12B determines that the watery stool region and the hard stool region are mixed in the determination region (step). S77). The analysis unit 12B determines whether or not the grayscale value for each pixel in the determination region is less than the second threshold value (threshold value 2) (step S78).
When the grayscale value for each pixel in the determination region is less than the second threshold value (threshold value 2), the analysis unit 12B outputs the region as a hard portion image to the determination unit 13B (step S79). The determination unit 13B determines the stool properties of the hard portion (the region of the hard stool in the mixed state) based on the hard portion extracted image (step S84).
When the grayscale value for each pixel in the determination region is equal to or greater than the second threshold value (threshold value 2), the analysis unit 12B outputs the region as a watery portion extracted image to the determination unit 13B (step S76). The determination unit 13B determines the stool properties of the watery portion (region of the watery stool in a mixed state) based on the image extracted from the watery portion (step S85).

The determination unit 13B comprehensively determines the stool properties in the target image by using the results of determining the stool properties in steps S82 to S85 (step S86).

In step S71, the pixel group in which the grayscale value for each pixel in the grayscale target image is equal to or higher than the first threshold value (threshold value 1) is determined to be an image other than stool and excluded from the determination area (step S81). ..

As described above, in the determination device 10B of the third embodiment, the analysis unit 12B extracts the determination region from the target image based on the color characteristics of the assumed color Y. As a result, the determination device 10B of the third embodiment can extract a region containing excrement from the target image. Since the area for determining the properties of the stool can be narrowed down, it is possible to reduce the processing load required for the determination as compared with the case where the entire target image is analyzed. By reducing the processing load, it is possible to perform processing even in an apparatus that does not have high computing power, so that an increase in apparatus cost can be suppressed. Further, since the determination region can be extracted based on the color characteristics of the assumed color Y of the excrement to be determined, the determination region is compared with the region extracted regardless of the color of the assumed color Y. Judgment becomes easy.

Further, in the determination device 10B of the third embodiment, the analysis unit 12B calculates the spatial distance Z in the color space with the assumed color Y for each pixel color in the target image, and the calculated spatial distance Z is predetermined. A set of pixels that is less than the threshold value of is extracted as a determination region. As a result, in the determination device 10B of the third embodiment, the color difference (color difference) from the assumed color Y can be calculated from the space distance Z, and the region where the color difference from the assumed color Y is small can be determined, which is quantitative. It is possible to extract a determination area based on a specific index.

Further, in the determination device 10B of the third embodiment, the analysis unit 12B uses a value obtained by weighting the difference between the color of each pixel in the target image and the assumed color Y for each color element in the color space. Calculate the space distance above. Thereby, in the determination device 10B of the third embodiment, it is possible to calculate the spatial distance in which the element (for example, R element) in which the difference from the assumed color Y is likely to appear is emphasized. This makes it possible to extract the determination region with high accuracy.

Further, in the determination device 10B of the third embodiment, the target image is an RGB image, the assumed color is a color indicated by an RGB value, and the analysis unit 12B has an R value, a G value, and a G value for each pixel in the target image. Using a weighted value of the difference in the ratio in the R element, the difference in the ratio in the G element, and the difference in the ratio in the B element between the color ratio indicating the ratio of the B value and the color ratio in the assumed color Y, Calculate the spatial distance on the color space. As a result, in the determination device 10B of the third embodiment, it is possible to calculate the spatial distance without being affected by the difference in the shade of color due to the difference in the amount of light irradiating the subject. This makes it possible to extract the determination region with high accuracy.

Further, in the determination device 10B of the third embodiment, the analysis unit 12B creates a grayscale target image, and the grayscale value of the pixels in the grayscale target image is less than a predetermined first threshold value (threshold value 1). The area is defined as the area where the excrement is present, and the area where the excrement is present is extracted as the determination area. As a result, in the determination device 10B of the third embodiment, the determination region can be extracted by an easy method of comparing the grayscale value for each pixel in the grayscale target image with the threshold value.

Further, in the determination device 10B of the third embodiment, in the analysis unit 12B, the grayscale value of the pixels in the grayscale target image is less than the first threshold value and equal to or more than a predetermined second threshold value smaller than the first threshold value. The region is defined as the region where watery stool is shown, the region where the grayscale value of the pixels in the grayscale target image is less than the second threshold value is defined as the region where hard stool is shown, and the determination region is the watery stool. The area where stool is shown and the area where the hard stool is shown are extracted. As a result, in the determination device 10B of the third embodiment, the region where watery stool is shown and the hard stool are shown by an easy method of comparing the grayscale value for each pixel in the grayscale target image with the threshold value. The determination area can be extracted separately from the determined area, and the determination area can be extracted more accurately. Further, by extracting the determination region by distinguishing the region showing watery stool and the region showing hard stool, the processing load of the determination by the determination unit 13B can be reduced as compared with the case where the determination region is not distinguished. Is possible.

In the above description, the case where the analysis unit 12B extracts the determination region using one grayscale target image has been described as an example, but the present invention is not limited to this. The analysis unit 12B may extract a determination region using a plurality of different grayscale target images. For example, the analysis unit 12B may only perform the process of extracting the determination region according to the first threshold value by using the grayscale target image obtained by converting the Euclidean distance Z4 weighted by the color ratio into grayscale. Then, the analysis unit 12B may only perform the process of distinguishing and extracting the regions of watery stool and hard stool according to the second threshold value using the grayscale target image obtained by converting the Euclidean distance Z1 into grayscale. Good.

Further, although a plurality of embodiments have been described above, the configuration in each embodiment is not limited to the configuration of the embodiment, and may be used as the configuration of another embodiment. For example, in the process of determining the properties of stool in the first embodiment, a difference image, a divided image, or a whole image and a partial image according to the second embodiment and its modified examples may be used. Further, the grayscale target image in the third embodiment may be used for the difference image or the like according to the second embodiment and the modified example thereof. Further, the difference image, the divided image, or the whole image and the partial image according to the second embodiment and the modified example thereof may be used for the process of determining the property of the stool of the third embodiment.

All or part of the processing performed by the determination device 10 (10A, 10B) in the above-described embodiment may be realized by a computer. In that case, the program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. It may also include a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client in that case. Further, the above program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system. It may be realized by using a programmable logic device such as FPGA.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs and the like within a range that does not deviate from the gist of the present invention.

1 Judgment system 10 Judgment device 11 Image information acquisition unit 12 Analysis unit (estimation unit) (extraction unit)
13 Judgment unit 14 Output unit 15 Image information storage unit 16 Learned model storage unit 17 Judgment result storage unit 18 Communication unit 20 Learning device 21 Communication unit 22 Learning unit 3 Toilet bowl device 30 Toilet bowl 32 Toilet bowl 34 Internal space 36 Opening S Cleaning Water 4 imager

Claims (6)

An image information acquisition unit that acquires image information of the target image that is a target image for determining judgment items related to stool and that captures the internal space of the toilet bowl after excretion.
A preprocessing unit that generates a reference image that captures the internal space of the toilet bowl before excretion and a difference image that shows the difference between the target image and the target image.
The difference image is input to a trained model in which the correspondence between the learning difference image showing the difference between the images showing before and after excretion and the judgment result of the judgment item is learned by machine learning using a neural network. The estimation unit that estimates the determination items for the difference image and the estimation unit
Based on the estimation result by the estimation unit, a determination unit that determines the determination items for the target image, and a determination unit.
A determination device comprising. The preprocessing unit has the predetermined value in the difference image based on the difference between the color information indicating the color of the predetermined pixel in the reference image and the color information of the pixel corresponding to the predetermined pixel in the pixel of the target image. Determining the color information of a pixel corresponding to a pixel,
The determination device according to claim 1. The reference image, the target image, and the difference image are RGB images.
The preprocessing unit is based on the difference between the RGB value of the predetermined pixel in the reference image and the RGB value of the pixel corresponding to the predetermined pixel in the target image, and the pixel corresponding to the predetermined pixel in the difference image. Determine the RGB value,
The determination device according to claim 1 or 2. The reference image, the target image, and the difference image are RGB images.
The preprocessing unit is based on the difference between the color ratio indicating the ratio of the R value, the G value, and the B value of the predetermined pixel in the reference image and the color ratio of the pixel corresponding to the predetermined pixel in the target image. , Determines the RGB value of the pixel corresponding to the predetermined pixel in the difference image.
The determination device according to any one of claims 1 to 3. The image information acquisition unit acquires the image information of the target image that is the target image for determining the judgment items related to stool and that captures the internal space of the toilet bowl after excretion.
The preprocessing unit generates a difference image showing the difference between the reference image obtained by capturing the internal space of the toilet bowl before excretion and the target image.
The estimation unit applies the correspondence between the learning difference image showing the difference between the images showing before and after excretion and the judgment result of the judgment item to the trained model learned by machine learning using a neural network. By inputting the difference image, the judgment items are estimated for the difference image.
The determination unit makes a determination regarding the determination item for the target image based on the estimation result by the estimation unit.
Judgment method. Judgment device computer,
An image information acquisition means for acquiring image information of a target image that is a target image for determining a judgment item related to stool and is an image of the internal space of the toilet bowl after excretion.
A preprocessing means for generating a difference image showing a difference between a reference image of the internal space of a toilet bowl before excretion and the target image.
The difference image is input to a trained model in which the correspondence between the learning difference image showing the difference between the images showing before and after excretion and the judgment result of the judgment item is learned by machine learning using a neural network. An estimation means that estimates the determination items for the difference image.
A determination means that determines the determination items for the target image based on the estimation result by the estimation means.
A program to function as.

Images