JP7262301B2 - Determination device, determination method, and program - Google Patents

Description

The present invention relates to a determination device, determination method, and program.

Attempts have been made to grasp the state of excretion in living organisms. For example, a technique of photographing excrement with a camera and analyzing the image has been disclosed (see Patent Document 1, for example).

Machine learning is generally used to analyze human facial expressions. In the technique using machine learning, for example, a learned model is created by executing machine learning using learning data in which human facial expressions and emotions corresponding to the facial expressions are associated with each other. By inputting a human facial expression into this trained model, it is possible to estimate the emotion that the facial expression means, and to analyze the facial expression.

JP 2007-252805 A

It is difficult to say that the accuracy of the analysis is sufficient when the technique disclosed in Patent Document 1 is used. In other words, since analysis is performed using a table that associates the excretion speed, hardness or size of stool with the classification of stool (hard stool, watery stool, etc.), which is not set in the table, correct There is concern that a classification cannot be obtained.

On the other hand, it is conceivable to apply the above-described machine learning method to the analysis of excrement. For example, a learned model is created by executing machine learning using learning data in which an image of the internal space of the toilet bowl after excretion is associated with the results of evaluating the classification and the situation. By inputting an image to be analyzed into this trained model, it is possible to estimate a desirable analysis result of the excreta.

However, when attempting to estimate detailed determination contents from the entire image using a machine learning method, high computing power is required, which causes the problem of increased device cost.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a determination device, a determination method, and a program that can reduce the increase in device cost in the analysis of excrement using machine learning. is to provide

In order to solve the above-described problems, one embodiment of the present invention acquires image information of a target image, which is a target image for determining items related to feces, and captures the internal space of a toilet bowl after excretion. an image information acquisition unit; a preprocessing unit that generates an entire image showing the entire target image and a partial image showing a partial area of the target image; and an image showing the entire internal space of the toilet bowl after excretion. and the overall image for learning is input to a trained model that has learned the correspondence relationship between the overall image for learning and the determination result of the first overall determination item among the determination items by machine learning using a neural network. By doing so, a first estimation regarding the first determination item is performed for the entire image, and a learning partial image that is a partial area of the learning entire image and the first determination item among the determination items are obtained in more detail than the first determination item. A second estimation regarding the second determination item is performed for the partial image by inputting the partial image to a trained model that has learned the correspondence relationship with the second determination item by machine learning using a neural network. The determination device includes an estimation unit, and a determination unit that determines the determination item for the target image based on the estimation result of the estimation unit.

Moreover, one embodiment of the present invention is the above-described determination device, wherein the preprocessing unit generates a partial image including at least the opening of the toilet bowl in the target image.

Moreover, one embodiment of the present invention is the above-described determination device, wherein the estimation unit estimates the presence or absence of scattering of stool as the first estimation, and estimates the nature of the stool as the second estimation.

Moreover, one embodiment of the present invention is the above-described determination device, wherein the determination unit corrects the estimation result of the second estimation using the estimation result of the first estimation.

Further, in one embodiment of the present invention, the image information acquisition unit acquires image information of a target image that is a target image for determining items related to stool, and that is an image of the internal space of the toilet bowl after excretion. , the preprocessing unit generates a whole image as the target image and a partial image that is a partial area of the target image, and the estimation unit is an image showing the entire internal space of the toilet bowl after excretion. By inputting the whole image to a learned model that has learned the correspondence relationship between the whole image for learning and the determination result of the first overall judgment item among the judgment items by machine learning using a neural network. and performing a first estimation on the first determination item for the whole image, and performing a learning partial image that is a partial area of the learning whole image and a first determination item that is more detailed than the first determination item among the determination items. 2, by inputting the partial image into a trained model that has learned a correspondence relationship with the second judgment item by machine learning using a neural network, performing a second estimation regarding the second judgment item for the partial image, and making a judgment. A determination method in which a unit determines the determination item for the target image based on an estimation result by the estimation unit.

In one embodiment of the present invention, the computer of the determination device acquires image information of a target image that is a target image for determining a determination item related to stool, and that is an image of the internal space of the toilet bowl after excretion. image information acquisition means; preprocessing means for generating an entire image as the target image; and a partial image that is a partial area of the target image; By inputting the whole image to a trained model that has learned the correspondence relationship between the whole image and the judgment result of the first big judgment item among the judgment items by machine learning using a neural network, performing a first estimation on the first determination item for the entire image, and performing a learning partial image that is a partial area of the learning entire image and a second determination that is more detailed than the first determination item among the determination items; estimating means for performing a second estimation on the second determination item for the partial image by inputting the partial image to a trained model that has learned a correspondence relationship with the item by machine learning using a neural network; A program for functioning as determination means for performing determination regarding the determination item for the target image based on an estimation result by the estimation means.

An increase in device cost can be reduced in the analysis of excreta using machine learning.

1 is a block diagram showing the configuration of a determination system 1 to which a determination device 10 according to a first embodiment is applied; FIG. 3 is a block diagram showing the configuration of a learned model storage unit 16 according to the first embodiment; FIG. FIG. 3 is a diagram illustrating an image to be determined by the determination device 10 according to the first embodiment; FIG. 4 is a flowchart showing the overall flow of processing performed by the determination device 10 according to the first embodiment; 4 is a flowchart showing the flow of determination processing performed by the determination device 10 according to the first embodiment; 4 is a flow chart showing the flow of cleaning method determination processing performed by the determination device 10 according to the first embodiment. It is a figure explaining 10 A of determination apparatuses which concern on 2nd Embodiment. It is a block diagram which shows the structure of 1 A of determination systems to which 10 A of determination apparatuses which concern on 2nd Embodiment are applied. FIG. 10 is a diagram illustrating processing performed by a preprocessing unit 19 according to the second embodiment; FIG. It is a flow chart which shows a flow of processing which 10 A of judgment devices concerning a 2nd embodiment perform. FIG. 10 is a diagram illustrating processing performed by a preprocessing unit 19 according to Modification 1 of the second embodiment; 10 is a flow chart showing the flow of processing performed by a determination device 10A according to Modification 1 of the second embodiment. It is a figure explaining the process which the pre-processing part 19 based on the modification 2 of 2nd Embodiment performs. 10 is a flow chart showing the flow of processing performed by a determination device 10A according to modification 2 of the second embodiment; FIG. 11 is a block diagram showing the configuration of a determination device 10B according to a third embodiment; FIG. It is a figure explaining the process which the analysis part 12B which concerns on 3rd Embodiment performs. 13 is a flow chart showing the flow of processing performed by a determination device 10B according to the third embodiment;

A determination device according to an embodiment will be described below with reference to the drawings.

(First embodiment)
First, the first embodiment will be described.
FIG. 1 is a block diagram showing the configuration of a determination system 1 to which a determination device 10 according to the first embodiment is applied. The determination system 1 includes, for example, a determination device 10 and a learning device 20 .

The determination device 10 performs a determination regarding excretion based on a target image (hereinafter simply referred to as an image) to be determined. The target image is an image related to excretion, for example, an image of the internal space 34 (same) of the toilet bowl 32 (see FIG. 3) after excretion. "After excretion" means any time after the user has excreted and before the toilet bowl is washed, for example, when the user who has been seated on the toilet bowl 30 (same) leaves the seat. Determination regarding excretion refers to determination items related to excretion behavior and conditions, and excrement washing, such as presence or absence of excretion, presence or absence of urine, presence or absence of stool, properties of stool, and whether or not paper (toilet paper) is used. , the method of cleaning the toilet bowl 30 after excretion, the condition of excretion, and the like, based on information such as the amount of paper used. The stool properties may be information indicating the condition of the stool, such as "hard stool," "normal stool," "soft stool," "muddy stool," and "watery stool," indicating the shape of the stool. It may be information indicating properties or states such as "soft". The shape of the stool is labeled (evaluated) from the perspective of scattering to the toilet bowl, how the stool dissolves in the pooled water, how it becomes cloudy, and the characteristics inside the pooled water (underwater) and above the water surface (in the air). )I do. Further, the stool properties may be information indicating the amount of stool, or may be information indicating the classification of the amount of stool, such as two, such as large or small, or three, such as large, normal, or small. Alternatively, it may be information indicating the amount of stool numerically. Further, the stool properties may be information indicating the color of the stool. The color of the stool may be, for example, ocher to brown as the normal stool color, and may be information indicating whether the stool is the normal color. color). The method of washing the toilet bowl 30 after excretion includes the amount of washing water used for washing, the water pressure, the number of times of washing, and the like.

The determination device 10 includes, for example, an image information acquisition unit 11, an analysis unit 12, a determination unit 13, an output unit 14, an image information storage unit 15, a trained model storage unit 16, and a determination result storage unit 17. , provided. Here, the analysis unit 12 is an example of an "estimation unit".

The image information acquisition unit 11 acquires image information of an image (target image) of the internal space 34 of the toilet bowl 32 captured regarding excretion. The image information acquisition unit 11 outputs the acquired image information to the analysis unit 12 and stores it in the image information storage unit 15 . The image information acquisition unit 11 is connected to the toilet device 3 and the imaging device 4 (see FIG. 3).

The analysis unit 12 analyzes an image (target image) corresponding to the image information obtained from the image information acquisition unit 11 . The analysis by the analysis unit 12 means estimating the determination items related to excretion based on the image related to excretion.

The analysis unit 12 performs estimation using, for example, a learned model corresponding to the determination item of the determination unit 13 . The learned model is, for example, a model stored in the learned model storage unit 16, and is a model that has learned a correspondence relationship between an image related to excretion and an evaluation result related to excretion.

For example, the analysis unit 12 uses the output obtained from the trained model that has learned the correspondence relationship between the image and the presence or absence of urine as the estimation result of estimating the presence or absence of urine. In addition, the analysis unit 12 uses the output obtained from the learned model that has learned the correspondence relationship between the image and the presence or absence of stool as the estimation result of estimating the presence or absence of stool. Further, the analysis unit 12 uses the output obtained from the trained model that has learned the correspondence relationship between the image and the properties of the stool as the estimation result of estimating the properties of the stool. Further, the analysis unit 12 uses the output obtained from the trained model that has learned the correspondence relationship between the image and the use/non-use of paper as the estimation result of the use/non-use of paper. In addition, the analysis unit 12 uses the output obtained from the trained model that has learned the correspondence relationship between the image and the amount of paper used as the estimation result of the amount of paper used.

The analysis unit 12 may perform estimation using a trained model for estimating a plurality of items from an image. For example, the analysis unit 12 may perform estimation using a learned model that has learned the correspondence relationship between an image and the presence or absence of urine and feces. The analysis unit 12 estimates that there is no excretion when it is estimated from the learned model that neither urine nor feces is present in the image.

The determination unit 13 uses the analysis result obtained from the analysis unit 12 to determine excretion. For example, the determining unit 13 uses the presence/absence of urine estimated from the image as the determination result of determining the presence/absence of urine in the image. Further, the determination unit 13 uses the presence/absence of stool estimated from the image as the determination result of determining the presence/absence of stool in the image. Further, the determination unit 13 uses the properties of stool estimated from the image as the determination result of the properties of stool in the image. Further, the determination unit 13 uses the presence/absence of paper usage estimated from the image as the determination result of the determination of the presence/absence of paper usage in the image. Further, the determination unit 13 uses the amount of paper used estimated from the image as the determination result of the amount of paper used in the image.

The determination unit 13 may use a plurality of estimation results to determine excretion. For example, the method for cleaning the toilet bowl 30 after excretion may be determined based on the properties of stool estimated from the image and the amount of paper used.

The output unit 14 outputs the result of determination by the determination unit 13 . You may make it the output part 14 transmit a determination result to the terminal of the user who performed the excretion action, for example. This allows the user to recognize his/her own excretion behavior and the determination result of the situation.
The image information storage unit 15 stores image information acquired by the image information acquisition unit 11 .
The learned model storage unit 16 stores a learned model corresponding to each of the judgment items.
The determination result storage unit 17 stores the determination result by the determination unit 13 .

A trained model stored in the trained model storage unit 16 is created using, for example, a deep learning (DL) technique. DL is a machine learning method using a deep neural network (DNN) composed of multilayer neural networks. A DNN is realized by a network inspired by the principle of predictive coding in neuroscience, and is constructed by a function modeled after a neural transmission network. By using the DL method, it is possible to make the learned model automatically recognize the feature quantity inherent in the image, just like human thinking. In other words, it is possible to perform estimation directly from the image by allowing the trained model to learn the data itself without performing the work of extracting the feature amount.

In this embodiment, a case in which a trained model is created using the DL technique will be described as an example, but the present invention is not limited to this. The trained model may be a model created by learning learning data in which image data and convenience evaluation results are associated with each other without extracting feature amounts from at least image data. The image data here are various images of the internal space 34 of the toilet bowl 32 .

FIG. 2 is a block diagram showing the configuration of the learned model storage unit 16 according to the first embodiment. The learned model storage unit 16 includes, for example, a urine presence/absence estimation model 161, a stool presence/absence estimation model 162, a stool properties estimation model 163, a paper usage/non-use estimation model 165, and a paper usage estimation model 166.

The urine presence/absence estimation model 161 is a trained model that has learned the correspondence relationship between an image and the presence/absence of urine. Created by learning.
The stool presence/absence estimation model 162 is a trained model that has learned the correspondence relationship between images and the presence/absence of stool. Created by learning.

The stool properties estimation model 163 is a trained model that has learned the correspondence relationship between images and stool properties. Created by learning.

The paper usage/absence estimation model 165 is a trained model that has learned the correspondence relationship between images and the usage/nonuse of paper. It is created by learning the attached learning data.
The paper usage estimation model 166 is a trained model that has learned the correspondence relationship between images and paper usage, and information indicating the paper usage determined from the image is associated with an image related to excretion. Created by learning the learning data. The information may be information indicating two types of paper usage, or three types, such as high, normal, and low, or information indicating the amount of paper as a numerical value.
In addition, as a method of determining the presence or absence of excretion from the image, for example, it is conceivable that the person in charge of creating the learning data makes the determination.

FIG. 3 is a diagram illustrating an image to be determined by the determination device 10 according to the first embodiment. FIG. 3 schematically shows the positional relationship between the toilet device 3 and the imaging device 4. As shown in FIG.

The toilet device 3 includes, for example, a toilet bowl 30 having a toilet bowl 32 . The toilet device 3 is configured to be able to supply flush water S to an opening 36 provided in the internal space 34 of the toilet bowl 32 . The toilet device 3 is operated by a functional unit (not shown) provided in the toilet bowl 30 to allow the user of the toilet device 3 to sit or leave the seat, start using local cleansing, and clean the toilet bowl 32 after excretion. detected. The toilet device 3 transmits the result of detection by the functional unit to the determination device 10 .

In the following description, when the user of the toilet device 3 sits on the toilet bowl 30, the front side of the user will be referred to as the "front side", and the rear side will be referred to as the "rear side". Further, when the user of the toilet device 3 sits on the toilet bowl 30, the left side of the user is called the "left side" and the right side is called the "right side". The side away from the floor on which the toilet device 3 is installed is called the "upper side", and the side closer to the floor is called the "lower side".

The imaging device 4 is provided so as to be able to capture the contents of the excretion behavior. The imaging device 4 is installed above the toilet bowl 30 , for example, inside the edge of the rear side of the toilet bowl 32 so that the lens faces toward the inner space 34 of the toilet bowl 32 . For example, the imaging device 4 takes an image according to an instruction from the determination device 10 and transmits image information of the captured image to the determination device 10 . In this case, the determination device 10 transmits control information indicating an instruction for imaging to the imaging device 4 via the communication unit 18 .

Here, processing performed by the determination device 10 according to the first embodiment will be described with reference to FIGS. 4 to 6. FIG. FIG. 4 is a flowchart showing the overall flow of processing performed by the determination device 10 according to the first embodiment, and FIG. 5 is a flow chart showing the flow of determination processing performed by the determination device 10. As shown in FIG. FIG. 6 is a flow chart showing the flow of cleaning method determination processing performed by the determination device 10 .

First, with reference to FIG. 4, the overall flow of processing performed by the determination device 10 will be described.
The determination device 10 determines whether or not the user of the toilet device 3 is seated on the toilet 30 by communicating with the toilet device 3 (step S10). Information is acquired (step S11). The image information is image information of an image related to excretion. The determination device 10 transmits a control signal instructing imaging to the imaging device 4, causes the imaging device 4 to capture an image of the internal space 34 of the toilet bowl 32, and transmits image information of the captured image, thereby obtaining image information. get.
In the flowchart shown in FIG. 4, the case where the determination result of determining whether the user is seated is used as a trigger for acquiring image information has been described as an example, but the present invention is not limited to this. As a trigger for acquiring image information, the determination result of other content may be used, and when multiple conditions are satisfied using both the determination result of determining the seating and the determination result of other content , the image information may be acquired. The determination result of other content is, for example, the detection result of a human body detection sensor that detects the presence of a human body using infrared rays or the like. In this case, for example, when the human body detection sensor detects that the user has approached the toilet bowl 30, image acquisition is started.

Next, the determination device 10 performs determination processing (step S12). Details of the determination process will be described with reference to FIG. The determination device 10 stores the determination result in the determination result storage unit 17 (step S13).
Next, the determination device 10 determines whether or not the user of the toilet device 3 has left the seat by communicating with the toilet device 3 (step S14). finish. On the other hand, when determining that the user has not left the seat, the determination device 10 waits for a certain period of time (step S15) and returns to step S11.

Next, the flow of determination processing performed by the determination device 10 will be described with reference to FIG.
The determination device 10 uses the urine presence/absence estimation model 161 to estimate the presence/absence of urine in the image (step S122).

The determination device 10 also estimates the presence or absence of stool in the image using the stool presence/absence estimation model 162 (step S123), and determines the presence or absence of stool based on the estimation result (step S124).

If determination device 10 estimates that there is stool in step S124 (step S124, YES), it estimates stool properties using stool properties estimation model 163 (step S125).

The determination device 10 also estimates whether or not paper is used in the image using the paper usage/non-use estimation model 165 (step S126).

If it is estimated that paper is used in step S126 (step S127, YES), the determination device 10 estimates the amount of paper used using the used paper amount estimation model 166 (step S128).
The determination device 10 determines a method for cleaning the toilet bowl 30 after use (step S129).

The contents of the processing for determining the cleaning method by the determination device 10 will be described with reference to FIG. 6 . In the flowchart shown in FIG. 6, the determination device 10 exemplifies a case where the determination device 10 determines the cleaning method to be one of four, "large", "medium", "small", or "none". "High", "medium" and "small" in the washing methods mean that the strength of washing decreases in the order of "high", "medium" and "small". The strength of washing is the degree of strength with which the toilet bowl 32 is washed. For example, the lower the strength, the smaller the amount of washing water S, and the higher the strength, the larger the amount of washing water S. Alternatively, the lower the strength, the less the number of times of washing, and the higher the strength, the more times of washing. If the washing method is "no", it means that the toilet bowl 32 is not washed.

The determination device 10 determines whether or not paper is used (step S130), and if it is determined that paper is used, determines whether the amount of paper used is large (step S131). The determination device 10 determines that the amount of paper used is large when the amount of paper estimated in step S126 is equal to or greater than a predetermined threshold, and determines that the amount of paper used is small when the amount is less than the predetermined threshold. . When determining that the amount of paper used is large (step S131, YES), the determining device 10 determines that the cleaning method is "large" (step S132).

When determining that the amount of paper used is small (step S131, NO), the determination device 10 determines whether or not there is a flight (step S133). The determination device 10 determines the presence/absence of flights according to the estimation result of the presence/absence of flights estimated in step S123. When determining that there is a flight (step S133, YES), the determination device 10 determines whether or not there is a large amount of flight (step S134). The determination device 10 determines that the amount of stool is large when the amount of stool is estimated to be equal to or greater than a predetermined threshold in the properties of the stool estimated in step S125, and determines that the amount of stool is large when the amount of stool is estimated to be less than the predetermined threshold. It is determined that the amount of is small. When determining that the amount of stool is large (step S134, YES), the determination device 10 determines the washing method to be "large" (step S132).

When determining that the amount of stool is small (step S134, NO), the determination device 10 determines whether or not the shape of the stool is other than watery stool (step S135). When the determination device 10 estimates that the shape of the stool is not watery stool (that is, hard stool, normal stool, soft stool, or muddy stool) in the stool properties estimated in step S125. First, the shape of the stool is determined to be other than watery stool, and when the shape of the stool is estimated to be watery stool, the shape of the stool is determined to be watery stool. When determining that the shape of the stool is other than watery stool (step S135, YES), the determination device 10 determines the washing method to be "medium" (step S136). On the other hand, when determining that the shape of the stool is watery stool (step S135, NO), the determination device 10 determines the washing method to be "small" (step S138).

When determining that there is no stool in step S133 (step S133, NO), the determination device 10 determines whether or not there is urine (step S137). The determination device 10 determines the presence or absence of urine according to the estimation result of the presence or absence of urine estimated in step S122. When determining that there is urine (step S137, YES), the determining device 10 determines that the cleaning method is "small" (step S138). On the other hand, when determining that there is no urine (step S137, NO), the determination device 10 determines that the cleaning method is "no" (step S139).

As in the example of the flowchart shown in FIG. 6, the determination device 10 determines the cleaning method according to the combination of the results of estimating the presence or absence of urine, the presence or absence of feces, and the presence or absence of paper, thereby finely controlling the amount of cleaning water. It is possible to suppress water waste and appropriately save water while performing sufficient washing.

Note that the determination device 10 determines whether the user has excreted using the estimation result of estimating the presence or absence of urine shown in step S122 and the estimation result of estimating the presence or absence of stool shown in step S123. It may be determined whether or not has excreted. In this case, the determination device 10 determines that the user has not excreted when it is estimated that there is no urine and no feces.

As described above, the determination device 10 of the first embodiment includes the image information acquisition section 11 , the analysis section 12 and the determination section 13 . The image information acquisition unit 11 acquires image information of an image (target image) obtained by imaging the internal space 34 of the toilet bowl 32 . The analysis unit 12 inputs image information to the learned model, thereby estimating determination items related to excretion for the target image. The determination unit 13 determines determination items for the image based on the estimation result. A trained model is a model that has been trained using the DL technique. When learning using the DL method, it is only necessary to label (correspond) the result of judging judgment items such as the presence or absence of excretion in the image, so it is necessary to extract the feature amount from the image and create learning data. There is no Therefore, there is no need to spend time considering what kind of feature quantity should be extracted by what kind of method. That is, the determination device 10 of the first embodiment can reduce the time required for development in the analysis of excretion behavior using machine learning.

Further, in the determination device 10 of the first embodiment, the target image is an image of the internal space 34 of the toilet bowl 32 after excretion. This makes it possible to reduce the number of images compared to, for example, several hundreds of images obtained by continuously photographing images of falling excrement. Therefore, the load required for estimation and determination can be reduced, and the time required for development can be reduced.

Further, in the determination device 10 of the first embodiment, the items to be determined include at least one of the presence or absence of urine, the presence or absence of stool, and the properties of stool. As a result, the determination device 10 of the first embodiment can make a determination regarding excrement.

Further, in the determination device 10 of the first embodiment, determination items include whether or not paper is used in excretion, and the amount of paper used when paper is used. As a result, the judgment device 10 of the first embodiment can judge the use of paper in excretion, and the judgment result can be used as an index for judging the cleaning method of the toilet bowl 30, for example. becomes.

Further, in the determination device 10 of the first embodiment, the determination unit 13 determines a cleaning method for cleaning the toilet bowl 30 in the situation indicated by the target image. As a result, the judgment device 10 of the first embodiment can judge not only excrement but also the cleaning method of the toilet bowl 30 .

In addition, in the determination device 10 of the first embodiment, the items to be determined include at least one of the properties of the stool and the amount of paper used in excretion, and the analysis unit 12 determines the properties of the stool in the target image, and the amount of paper used in excretion, and the determination unit 13 determines a cleaning method for cleaning the toilet bowl 30 in the situation shown in the target image using the estimation result by the analysis unit 12. . As a result, the determination device 10 of the first embodiment can determine an appropriate cleaning method according to the amount of excrement and paper used.

In the present embodiment, the case where the determination of the excrement and the determination of the cleaning method are performed has been described as an example, but only the determination of the excrement or the determination of the cleaning method may be performed.

Moreover, in the determination apparatus 10 of 1st Embodiment, determination items include determination of whether excretion was performed. As a result, for example, when watching over an elderly person in a facility for the elderly, it is possible to ascertain whether or not the elderly person has excreted using the toilet device 3 . It is also possible to examine the content of nursing care based on whether or not the elderly person excreted by himself or not when guided to the toilet.
It should be noted that the user's health condition may be determined using the determination result regarding excrement.

(Second embodiment)
Next, a second embodiment will be described. This embodiment differs from the above-described embodiments in that it does not determine whether or not paper is used, but only the properties of stool. Moreover, it differs from the above-described embodiments in that preprocessing is performed on the target image. The preprocessing here is processing performed on the learning image before the model undergoes machine learning on the learning image. Also, preprocessing is processing performed on an unlearned image before inputting the unlearned image to the trained model. In the following description, only configurations different from the above-described embodiment will be described, and configurations equivalent to those of the above-described embodiment will be denoted by the same reference numerals, and descriptions thereof will be omitted.

FIG. 7 is a diagram for explaining the determination made by the determination device 10 of the second embodiment. FIG. 7 shows a conceptual diagram when trying to classify a specific object into three types of types A, B, and C. As shown in FIG.
In general, when trying to classify substances such as stool that can have various properties into three types of types A, B, and C based on their properties, it is difficult to clearly classify all substances. That is, there are many cases where types A, B, and C are mixed. For example, as shown in FIG. 7, an area E1 that can be clearly classified as type A, an area E2 that is a mixture of types A and B classified as type A or B, and an area that can be clearly classified as type B. Area E3, area E4 where type B and C are mixed and classified as type B or C, area E5 which can be clearly classified as type C, area where type C and A are mixed and classified as type C or A A region of E6 occurs.

When building a trained model that classifies stool properties into three types, A, B, and C, using DL, estimation accuracy may decrease in areas where types A, B, and C are mixed. Conceivable. In particular, when watery stool falls into the wash water S (water surface) stored in the toilet bowl 32, the color of the dropped watery stool transfers to the wash water S and spreads. As a result, even if there is stool having a different nature from the watery stool excreted before the watery stool, the color of the stool having a different nature from the watery stool and the color-transferred wash water S is improved. The difference is almost gone. In this case, the trained model cannot recognize the properties of stool that have properties different from watery stool, and even though there is stool with properties different from watery stool, it is estimated to be watery stool. , an estimation error may occur. If there is an error in the estimation by the trained model, an error will occur in the determination of the target image.

As a countermeasure, in the present embodiment, elements (hereinafter referred to as noise components) that may cause estimation errors, such as turbidity of the cleaning water S, are removed by preprocessing. As a result, it is possible to reduce the estimation error by the trained model and the judgment error of the target image.

FIG. 8 is a block diagram showing the configuration of a determination system 1A to which a determination device 10A according to the second embodiment is applied. The determination system 1A includes, for example, a determination device 10A. The determination device 10A includes an image information acquisition section 11A, an analysis section 12A, a determination section 13A, and a preprocessing section 19. FIG.

The image information acquisition unit 11A obtains image information of an image (reference image) of the internal space 34 of the toilet bowl 32 before excretion and an image of the internal space 34 of the toilet bowl 32 after excretion (target image). to get “Before excretion” means any time before the user of the toilet device 3 excretes, for example, when the user enters a private toilet room or sits on the toilet bowl 30 .

The preprocessing unit 19 uses the image information of the reference image and the target image to generate a difference image. A difference image is an image showing the difference between the reference image and the target image. The difference here refers to the content captured in the target image and not captured in the reference image. That is, the difference image is an image representing the excrement that is captured in the target image after excretion and is not captured in the reference image before excretion.

The preprocessing unit 19 outputs image information of the generated difference image to the analysis unit 12A. Also, the preprocessing unit 19 may store the image information of the generated difference image in the image information storage unit 15 . 12 A of analysis parts estimate the property of the stool in a difference image using a learned model. The learned model used for estimation by the analysis unit 12A is a model that has learned the correspondence relationship between the learning image showing the difference between the images before and after excretion and the evaluation result of the stool properties. Here, an image used for learning when creating a trained model (a learning image showing a difference between images before and after excretion) is an example of a "learning difference image."

The determination unit 13A determines the properties of the stool shown in the target image based on the properties of the stool estimated by the analysis unit 12A. Further, the determination unit 13A may determine the state of excretion of the user based on the stool properties estimated by the analysis unit 12A. The method by which the determination unit 13A determines the state of excretion of the user will be described later with reference to the flowchart of the present embodiment.

Here, regarding the method for generating the differential image by the preprocessing unit 19, the color of each image of the reference image, the target image, and the differential image is represented by R (Red), G (Green), and B (Blue). A case will be described as an example. However, each image is not limited to an image in which colors are represented by RGB, and an image other than an RGB image (for example, a Lab image or a YCbCr image) can be generated by the same method. Here, the RGB value is information indicating the color of the image, and is an example of "color information".

Based on the difference between the RGB value of a predetermined pixel in the reference image and the RGB value of the pixel corresponding to the predetermined pixel in the target image, the preprocessing unit 19 calculates the value of the pixel corresponding to the predetermined pixel in the difference image. Determine the RGB values. A pixel corresponding to a predetermined pixel is a pixel located at the same or neighboring position coordinates in the image. The difference here indicates the color difference between two pixels, and is determined based on the difference in RGB values, for example. For example, the preprocessing unit 19 determines that there is no difference when the RGB values indicate the same color, and that there is a difference when the RGB values do not indicate the same color.

For example, the preprocessing unit 19 determines that the RGB values of a predetermined pixel in the reference image are (255, 255, 0) (yellow) and the RGB values of the predetermined pixel in the target image are (255, 255, 0) (yellow). In some cases, since there is no color difference between two pixels, mask processing is performed to set the RGB values of a predetermined pixel in the difference image to a predetermined color (for example, white) indicating that there is no difference.

On the other hand, the preprocessing unit 19 determines that the RGB value of the predetermined pixel in the reference image is (255, 255, 0) (yellow) and the RGB value of the predetermined pixel in the target image is (255, 0, 0) (red). In some cases, since there is a color difference between two pixels, the RGB values of a given pixel in the difference image are set to the RGB values (255, 0, 0) (red) of the given pixel in the target image.

Alternatively, if there is a color difference between two pixels, the preprocessing unit 19 may set a predetermined color (for example, black) indicating that fact.

Alternatively, when there is a color difference between two pixels, the preprocessing unit 19 may set a predetermined color according to the degree of the difference. The degree of difference is, for example, a value calculated according to the vector distance of RGB values in the color space. In this case, the preprocessing unit 19 classifies the color difference between two pixels into a plurality of categories according to the degree of the difference. For example, when the degree of difference is classified into three types (large, medium, and small), the preprocessing unit 19 converts pixels with a large degree of difference to black in the RGB value of the pixel in the difference image, A difference image may be generated by setting the RGB values of the pixels in the difference image to gray for pixels with a medium degree of difference, and setting the RGB values of the pixels in the difference image to light gray for pixels with a small degree of difference. .

Here, it is conceivable that the amount of light illuminating the subject (the inner space 34 of the toilet bowl 32) changes due to the influence of the user's seating position. When the amount of light changes, the shade of color may change even in areas where there is no change before and after excretion. In such a case, it is possible that the preprocessing unit 19 determines that there is a color difference in the change in color density.

As a countermeasure, the preprocessing unit 19 may determine the color of the predetermined pixel in the difference image according to the ratio of the color of the predetermined pixel in the reference image and the ratio of the color of the predetermined pixel in the target image. . The color ratio is the ratio of RGB colors, and is indicated, for example, as a ratio to a predetermined reference value. That is, the color ratio of RGB values (R, G, B) is (R/L:G/L:B/L). Here, L indicates a predetermined reference value. The predetermined reference value L may be any value. Further, the predetermined reference value L may be a fixed value regardless of the RGB values, or may be a value that varies depending on the RGB values (for example, the R value of the RGB values).

For example, the preprocessing unit 19 determines that a predetermined pixel in the reference image is gray (RGB values (128, 128, 128)) and a predetermined pixel in the target image is light gray (RGB values (192, 192, 192)). In this case, since the two pixels have the same color ratio, it is determined that there is no color difference between the two pixels.

On the other hand, if the predetermined pixel in the reference image is yellow (RGB values (255, 255, 0)) and the predetermined pixel in the target image is red (RGB values (255, 0, 0)), the preprocessing unit 19 , it is determined that there is a difference in color between the two pixels because the ratios of colors in the two pixels are not the same.

FIG. 9 is a diagram illustrating processing performed by the preprocessing unit 19 according to the second embodiment. An image G1 as an example of a reference image is shown on the left side of FIG. 9, an image G2 as an example of a target image is shown in the center, and an image G3 is shown as an example of a difference image on the right side.
As shown in the image G1 of FIG. 9, the reference image is an image of the internal space 34 before excretion, and the state in which the cleansing water S is stored in the opening 36 located substantially in the center of the internal space 34 is imaged. It is
As shown in the image G2 of FIG. 9, the target image shows the internal space 34 after excretion. A state with T2 is imaged.
As shown in the image G3 of FIG. 9, excretions T1 and T2, which are the differences between the reference image and the target image, are expressed in the difference image.

Here, processing performed by the determination device 10A according to the second embodiment will be described with reference to FIG. FIG. 10 is a flow chart showing the flow of processing performed by the determination device 10A according to the second embodiment. In the flowchart shown in FIG. 10, steps S20, S22, S25-S27, and S29 are the same as S10, S11, S12-S14, and S15 in the flowchart of FIG.

In step S21, when determining that the user is seated on the toilet bowl 30, the determination device 10A generates a reference image. The reference image is an image showing the internal space 34 of the toilet bowl 32 before excretion. When determining that the user is seated on the toilet bowl 30, the determination device 10A acquires the image information of the reference image by transmitting a control signal instructing the imaging device 4 to perform imaging.

In step S23, the determination device 10A performs mask processing using the reference image and the target image. The masking process is a process of making pixels having no difference between the reference image and the target image a predetermined color (for example, white).
In step S24, the determination device 10A generates a difference image. The difference image is, for example, an image in which pixels having no difference between the reference image and the target image are masked, and pixel values (RGB values) of the target image are reflected for pixels having a difference between the reference image and the target image. be.

In step S28, when determining that the user has left the toilet bowl 30, the determination device 10A discards the image information of the reference image, the target image, and the difference image. Specifically, the determination device 10A deletes the image information of the reference image, target image, and difference image stored in the image information storage unit 15 . As a result, it is possible to prevent the storage capacity from becoming tight.

It has been explained that the determination process shown in step S25 of FIG. 10 is the same as the process shown in step S12 of FIG. I wish I could.

In addition, in step S25 of FIG. 10, the determination unit 13A determines the state of excretion of the user using the result of estimating the properties of stool in the difference image. For example, when the shape of the stool is hard stool, the determination unit 13A determines that the user's bowel movements tend to be constipated. The determination unit 13A determines that the user's defecation condition is good when the shape of the stool is normal stool. The determining unit 13A determines that the user's defecation condition requires observation when the shape of the stool is loose stool. The determining unit 13A determines that the user's defecation is prone to diarrhea when the shape of the stool is muddy or watery stool. Alternatively, the determination unit 13A may determine the health condition of the user from the state of defecation.

As described above, in the determination device 10A of the second embodiment, the preprocessing unit 19 generates a difference image indicating the difference between the reference image and the target image. As a result, in the determination device 10A of the second embodiment, since a difference image can show a portion where there is a difference before and after excretion, the properties of the excrement can be grasped more accurately, and the properties can be more accurately determined. can be determined.

Further, in the determination device 10A of the second embodiment, the preprocessing unit 19 calculates the difference between the color information indicating the color of the predetermined pixel in the reference image and the color information of the pixel corresponding to the predetermined pixel in the target image. determines the color information of the pixel corresponding to the predetermined pixel in the differential image. As a result, in the determination device 10A of the second embodiment, the portion having a different color before and after excretion can be shown in the difference image, and thus the same effects as those described above can be obtained.

Further, in the determination device 10A of the second embodiment, the preprocessing unit 19 calculates the difference between the RGB value of a predetermined pixel in the reference image and the RGB value of the pixel corresponding to the predetermined pixel in the target image as RGB values of the pixel corresponding to the predetermined pixel. As a result, in the determination device 10A of the second embodiment, the difference in color before and after excretion can be recognized as a difference in RGB values, so the color difference can be quantitatively calculated. effect.

In addition, in the determination device 10A of the second embodiment, the preprocessing unit 19 includes a color ratio indicating the ratio of the R value, the G value, and the B value of a predetermined pixel in the reference image, and a color ratio corresponding to the predetermined pixel in the target image. RGB values of the pixel corresponding to the predetermined pixel in the difference image are determined based on the difference from the color ratio of the pixel. As a result, in the determination device 10A of the second embodiment, even if there is a difference in the background color due to, for example, a difference in the amount of light applied to the subject before and after excretion, the difference is mistaken for excrement. The properties of the excrement can be extracted without recognition, and the same effects as those described above can be obtained.

In addition, although the case where the image information acquisition unit 11A acquires the image information of the reference image has been described above as an example, the present invention is not limited to this. For example, the image information of the reference image may be acquired by an arbitrary functional unit, or may be stored in the image information storage unit 15 in advance.

(Modification 1 of the second embodiment)
Next, Modification 1 of the second embodiment will be described. This modification differs from the above-described embodiment in that divided images obtained by dividing the target image are generated as preprocessing. In the following description, only configurations different from the above-described embodiment will be described, and configurations equivalent to those of the above-described embodiment will be denoted by the same reference numerals, and descriptions thereof will be omitted.

In general, the toilet bowl 32 is formed so as to slope downward from the edge of the toilet bowl 32 toward the opening 36 . For this reason, when there are a plurality of stools that have fallen into the toilet bowl 32, it is considered that the stools that have fallen earlier move downward along the inclined surface of the stool bowl 32 so as to be pushed by the ones that have fallen later. That is, there is a property that the object that falls first moves to the front side of the opening 36 .

Utilizing this property, in this modified example, estimation is performed in consideration of the time series of excrement discharge. Specifically, the target image is divided into a front side and a back side. Then, the excrement imaged in the image obtained by dividing the front side of the target image (front divided image) is regarded as old stool, and the excrement imaged in the image obtained by dividing the rear side of the target image (rear divided image) is regarded as new stool. and determine the properties of the stool. As a result, by judging old stools, it is possible to make a judgment based on stools that are close to the current state of the user's bowel movements.

In this modified example, the preprocessing unit 19 generates divided images. A divided image is an image including a partial area of the target image, and is, for example, a front divided image and a rear divided image. The boundary dividing the target image into the front divided image and the rear divided image may be set arbitrarily. It is divided by a line in the direction connecting to the right side).

Note that the divided images are not limited to the front divided image and the rear divided image. A divided image may be an image that includes at least a partial area of the target image. The target image may be an image divided into three regions in the front-back direction (direction connecting the front side and the back side), or an image in which the front-side divided image is further divided into a plurality of regions in the left-right direction. There may be. Also, one divided image may be generated from the target image, or a plurality of divided images may be generated. Further, when generating a plurality of divided images from a target image, the region obtained by combining the regions shown in the plurality of divided images may be the entire region shown in the target image, or may be a part of the region. may

The preprocessing unit 19 outputs image information of the generated divided images to the analysis unit 12A. Also, the preprocessing unit 19 may store the image information of the generated divided images in the image information storage unit 15 . 12 A of analysis parts estimate the property of the feces in a division image using a learned model. The learned model used for estimation by the analysis unit 12A is a model that has learned the correspondence relationship between learning images obtained by dividing an image of the internal space 34 of the toilet bowl 32 during excretion and evaluation results of fecal properties. be.

The determination unit 13A determines the state of excretion of the user in the state shown in the target image based on the properties of the stool in the divided images estimated by the analysis unit 12A. When there are a plurality of divided images generated from the target image, the determination unit 13A determines the state of excretion of the user by comprehensively viewing the estimation results of the respective divided images. The method by which the determination unit 13A comprehensively determines the state of excretion of the user will be described later with reference to the flowchart of this modified example.

Here, images used for learning when creating a trained model (images for learning obtained by dividing an image of the internal space 34 of the toilet bowl 32 during excretion) will be described. A divided image as a learning image in this modified example is an example of a “learning divided image”. The divided images as images for learning are images obtained by dividing partial regions of various images of the internal space 34 of the toilet bowl 32 captured during excretion in the past. The method of dividing the image by the preprocessing unit 23 may be arbitrary, but it is desirable to use the same method as the division by the preprocessing unit 19 . Using a similar method can be expected to improve the accuracy of estimation using a trained model. In addition, rather than having the trained model learn the entire target image, we let the trained model learn a part of the target image, that is, an area that is narrower than the target image. It can be a model that estimates the state.

FIG. 11 is a diagram illustrating processing performed by the preprocessing unit 19 according to Modification 1 of the second embodiment. In FIG. 11, an image G4 as an example of a target image is shown on the left, an image G5 as an example of a front split image is shown in the center, and an image G6 is shown as an example of a rear split image on the right.
As shown in the image G4 in FIG. 11, the target image captures the internal space 34, and the internal space 34 includes a state in which the cleansing water S is stored in the opening 36 located substantially in the center of the internal space 34. is imaged in its entirety.
As shown in the image G5 of FIG. 11, the front side area of the internal space 34 is extracted in the front side divided image, and the boundary in the left-right direction passing through the center of the pooled water surface where the cleansing water S is stored in the opening 36. A region on the front side of the line is extracted.
As shown in the image G6 of FIG. 11, the rear side area of the internal space 34 is extracted in the rear side divided image, and the area behind the horizontal boundary line passing through the center of the stagnant water surface is extracted. ing.

Here, processing performed by the determination device 10A according to Modification 1 of the second embodiment will be described with reference to FIG. FIG. 12 is a flow chart showing the flow of processing performed by the determination device 10A according to Modification 1 of the second embodiment. In the flowchart shown in FIG. 12, steps S30, S31, S33, S37, and S42 are the same as steps S10, S11, S14, S15, and S13 in the flowchart of FIG. 4, so description thereof will be omitted.

In step S32, the determination device 10A generates divided images using the target image. The divided images are, for example, a front divided image indicating the front area of the area captured in the target image, and a rear divided image indicating the rear area.

In step S34, the determination device 10A performs determination processing on each of the divided images of the front divided image and the rear divided image. Since the content of this determination process is the same as the process shown in step S25 in the flowchart of FIG. 10, the description thereof will be omitted.

In step S35, if the determination device 10A does not determine that the user has left the toilet bowl 30 (step S33, NO), it determines whether or not the operation of washing the private parts of the toilet bowl 30 has been performed. If the cleaning operation has been performed, the process shown in step S34 is performed.
In step S36, if the determination device 10A does not determine that the operation of cleaning the private parts of the toilet 30 has been performed (step S35, NO), the determination device 10A determines whether or not the operation of cleaning the toilet 30 has been performed. If the operation of flushing the toilet bowl has been performed, the processing shown in step S34 is performed.

In step S38, the determination device 10A determines whether or not there are determination results for both the front divided image and the rear divided image. Both have determination results means that there are images of stool on both sides, and that each image of stool has a determination result of properties.
In step S39, when there are determination results for both the front divided image and the rear divided image, the determination device 10A sets the determination result of the front divided image to the determination result of the old flight, and the determination result of the rear divided image to the new flight. be the judgment result.

In step S40, the determination unit 13A of the determination device 10A performs confirmation processing. The determination process is a process of determining the state of excretion of the user using the determination result of new stool and the determination result of old stool. The determination device 10A determines the excretion situation by, for example, considering that older stools reflect the current excretion situation. In the determination process, for example, when the property of the old stool is determined to be hard stool and the property of the new stool is determined to be normal stool, the determination unit 13A removes the hard stool remaining in the large intestine during defecation. Then, it is determined that the user's excretion situation is constipation prone. On the other hand, in the determination process, for example, if the determination unit 13A determines that the old stool is normal stool and the new stool is muddy stool, the user's excretion condition is good. I judge.

In step S41, the determination device 10A determines whether or not there is a determination result for the front divided image when the determination unit 13A has determined only one of the front divided image and the rear divided image. If there is a determination result for the front divided image, the process shown in step S40 is performed using the determination result for the front divided image. If there is no determination result for the front split image, the process shown in step S40 is performed using the determination result for the rear split image. A case where there is no determination result for the front divided image is, for example, a case where excrement is not imaged in the front divided image and the nature of stool cannot be determined.

As described above, in the determination device 10A according to Modification 1 of the second embodiment, the preprocessing unit 19 generates divided images including partial regions of the target image. As a result, in the determination device 10A according to Modification 1 of the second embodiment, a partial area of the target image can be used as a determination target. It is possible to determine the area in detail, and to perform determination with higher accuracy.

In addition, in the determination device 10A according to Modification 1 of the second embodiment, the preprocessing unit 19 generates a front divided image showing at least the front region of the toilet bowl in the target image. As a result, in the determination device 10A according to Modification 1 of the second embodiment, when a new flight and an old flight are imaged in the target image, an area in which the new flight is assumed to be imaged is treated as a divided image. can do. Moreover, even when the new flight and the old flight are not imaged in the target image, a region where there is a high possibility that the image of the flight is likely to be imaged can be set as a divided image, and the same effects as those described above can be obtained.

Further, in the determination device 10A according to the modified example 1 of the second embodiment, the preprocessing unit 19 generates a front divided image and a rear divided image, and the analysis unit 12A relates to determination items for the front divided image. The determination unit 13A estimates the determination items for the rear divided image, and determines the determination items for the target image using the estimation results for the front divided image and the estimation results for the rear divided image. make a judgment about As a result, the determination device 10A according to Modification 1 of the second embodiment can comprehensively determine the state of excretion of the user using the estimation results of the front divided image and the rear divided image. Therefore, it is possible to perform a more accurate determination than when using either the front split image or the back split image.

Further, in the determination device 10A according to the modification 1 of the second embodiment, the preprocessing unit 19 sets the estimation result for the front divided image to the estimation result that is chronologically older than the estimation result for the rear divided image. As a result, determination regarding the determination item is performed for the target image. As a result, in the determination device 10A according to Modification 1 of the second embodiment, the estimation result for the front divided image is regarded as the estimation result for the old flight, and the estimation result for the rear division image is regarded as the estimation result for the new flight. , it is possible to make a judgment considering the time series of excretion, and it is possible to make an accurate judgment that is closer to the current state of the user's excretion situation.
Since the direction in which the stool that fell first moves changes depending on the shape of the toilet bowl 32, the time-series relationship associated with the front divided image and the rear divided image may be reversed. That is, in the above description, the front divided image is chronologically older than the rear divided image, but the present invention is not limited to this, and the front divided image is chronologically newer than the rear divided image. It may be regarded that the processing related to determination (confirmation) may be performed.

(Modification 2 of the second embodiment)
Next, Modification 2 of the second embodiment will be described. This modification differs from the above-described embodiment in that, as preprocessing, an entire image representing the entire target image and a partial image obtained by cutting out a part of the target image are generated. In the following description, only configurations different from the above-described embodiment will be described, and configurations equivalent to those of the above-described embodiment will be denoted by the same reference numerals, and descriptions thereof will be omitted.

In general, if a machine learning technique is used to estimate detailed judgment content from the entire image, a high computing power is required, which increases the cost of the device. For example, if the number of DNN layers used in the model is increased, the number of calculations required for one trial increases due to the increase in the number of nodes, increasing the processing load. Also, in order for the model to be able to estimate detailed contents (that is, to minimize the error between the output of the model with respect to the input of the training data and the output of the training data), the weight W and the bias component b are changed. However, it is necessary to repeat trials many times, and in order to converge such repeated trials in a realistic time, a device that can process a huge amount of calculation at high speed is required. In other words, if an attempt is made to analyze the entire target image in detail, a high-performance device is required, which increases the cost of the device.

By the way, the target image is an image in which the entire internal space 34 of the toilet bowl 32 is captured. That is, the target image includes an area in which excrement is imaged and an area in which excrement is not imaged. For this reason, a method is conceivable in which a specific region (for example, a region near the opening 36) where excrement is likely to fall is cut out from the target image, and detailed determination contents are estimated for the cut out region. . As a result, the area of the image to be analyzed can be narrowed, and an increase in apparatus cost can be suppressed.

However, it is essentially unknown to which region of the toilet bowl 32 the excrement falls. In addition, the properties of stool change depending on the physical condition of the user. Therefore, even if the area where the excrement falls is often a specific area of the toilet bowl 32, the excrement does not always fall only in that specific area and may scatter around. obtain. If determination is made using only the image of a specific region without using the image of the surroundings, the determination may be made differently from the actual situation.

As a countermeasure, in this modified example, a whole image showing the entire target image and a partial image obtained by cutting out a part of the target image are generated by preprocessing.

As for the whole image, by performing a global determination that is not detailed, an attempt is made to suppress an increase in the apparatus cost. Comprehensive determination is overall (global) determination compared to determination of the properties of stool, and for example, determination of presence or absence of scattering of stool. Since the presence or absence of scattering of stool does not determine the properties of scattered stool, it can be said to be a relatively rough and easy determination compared to the case of determining the properties of stool. Here, the determination regarding the presence or absence of scattering of stool, which is performed on the entire image, is an example of the “first determination item”.

On the other hand, for the partial image, more detailed determination items than those for the entire image are determined. A detailed determination item is, for example, determining the properties of stool. By using a partial image obtained by narrowing an image area as a determination target, detailed determination can be performed without using a high-performance device, and an increase in device cost can be suppressed. Here, the determination related to the properties of stool performed on the partial image is an example of the "second determination item".

In this modified example, the preprocessing unit 19 generates a full image and partial images. The whole image is an image showing the entire target image, for example, the target image itself. A partial image is an image obtained by cutting out a partial area of the target image, for example, an image obtained by cutting out the vicinity of the opening 36 from the target image. An area of the target image to be cut out as a partial image may be arbitrarily set.

The preprocessing unit 19 outputs the generated image information of the entire image and partial images to the analysis unit 12A. Also, the preprocessing unit 19 may store the generated image information of the entire image and the partial image in the image information storage unit 15 .

12 A of analysis parts estimate the presence or absence of scattering of feces in a whole image using a trained model. Here, the estimation of presence/absence of scattering of stool in the entire image is an example of the "first estimation".

Also, the analysis unit 12A estimates the properties of stool in the partial image using the learned model. Here, the estimation of the properties of stool in the partial image is an example of the "second estimation".

The determination unit 13A determines the state of excretion of the user in the state shown in the target image based on the presence or absence of scattering of feces in the entire image estimated by the analysis unit 12A and the properties of feces in the partial images. The method by which the determination unit 13A determines the state of excretion of the user based on the estimation result of the whole image and the estimation result of the partial image will be described later with reference to the flowchart of this modified example.

Here, the learning data to be learned by the trained model used in this modified example will be described.
The trained model used for estimating the overall image is a model that has learned the correspondence relationship between the overall image for learning that captures the entire interior space 34 of the toilet bowl 32 during excretion and the evaluation result of evaluating the presence or absence of scattering of stool. is. The whole image for learning is various images showing the entire interior space 34 of the toilet bowl 32 captured during excretion in the past. A whole image for learning (an image for learning that captures the entire internal space 34 of the toilet bowl 32 during excretion) is an example of a "whole image for learning."
The learned model used for estimating the partial image is a correspondence between a partial image for learning obtained by extracting a part from an image of the entire internal space 34 of the toilet bowl 32 during excretion and an evaluation result of evaluating the properties of stool. It is a model that has learned relationships. A partial image for learning is an image obtained by cutting out a part of the whole image. A partial image for learning (an image for learning obtained by cutting out a part of an image capturing the entire internal space 34 of the toilet bowl 32 during excretion) is an example of a "partial image for learning."
The method of generating the whole image for learning and the partial image for learning may be arbitrary, but it is desirable to use the same method as the method for generating the whole image and the partial image by the preprocessing unit 19 . Using a similar method can be expected to improve the accuracy of estimation using a trained model.

FIG. 13 is a diagram illustrating processing performed by the preprocessing unit 19 according to Modification 2 of the second embodiment. In FIG. 13, an image G7 as an example of a target image is shown on the left side, an image G8 as an example of a whole image is shown in the center, and an image G9 is shown as an example of a partial image in the right side.
As shown in the image G7 of FIG. 13, the target image captures the internal space 34, and the internal space 34 includes a state in which the cleaning water S is stored in the opening 36 located substantially in the center of the internal space 34. is imaged in its entirety.
As shown in the image G8 in FIG. 13, the entire image is shown in the entire target image. The entire image may be the target image itself, or may be an image obtained by extracting the entirety of the target image.
As shown in the image G9 of FIG. 13, the partial image includes an area in the vicinity of the opening 36 in the approximate center of the internal space 34, and the water surface where the cleansing water S is accumulated and the surrounding area. extracted.

Here, processing performed by the determination device 10A according to Modification 2 of the second embodiment will be described with reference to FIG. FIG. 14 is a flow chart showing the flow of processing performed by the determination device 10A according to Modification 2 of the second embodiment. Steps S50, S51, S53, S57, and S62 in the flowchart shown in FIG. 14 are the same as S10, S11, S14, S15, and S13 in the flowchart of FIG. Also, in the flowchart shown in FIG. 14, steps S55 and S56 are the same as S35 and S36 in the flowchart of FIG. 12, so description thereof will be omitted.

In step S52, the determination device 10A generates a whole image and partial images using the target image. A whole image is, for example, an image showing the entire area captured in the target image. A partial image is, for example, an image showing a specific part of the area captured in the target image.

In step S54, the determination device 10A performs determination processing on each of the entire image and the partial image.
The determination device 10A performs a global determination of the entire image, for example, determination of presence or absence of scattering of stool. The determination device 10A estimates the presence or absence of scattering of stool in the entire image using the learned model, and uses the estimated result as the determination result of determining the presence or absence of scattering of stool in the entire image. The learned model here is a model created by learning using learning data in which the whole image for learning and the judgment result of judging the presence or absence of stool scattering are associated with each other.
Further, the determination device 10A performs detailed determination of the partial image, for example, determination of the properties of stool. The determination device 10A estimates the properties of the stool in the partial image using the learned model, and uses the estimated results as the determination results of the properties of the stool in the partial image. The trained model here is a model created by learning using learning data in which a partial image for learning and a determination result of determining the properties of stool are associated with each other.

In step S58, the determination device 10A determines whether or not there are determination results for both the entire image and the partial image. The presence of determination results in both means that the presence or absence of scattering of stool has been determined in the entire image, and that the properties of stool have been determined in the partial image.

In step S59, if the determination unit 13A has determination results for both the entire image and the partial images, the determination device 10A corrects the determination results for the partial images using the determination results for the entire image. Correcting the judgment result of the partial image means changing or supplementing the judgment result of the partial image using the judgment result of the whole image.
For example, when the determination result of the partial image indicates that the stool is loose stool, and the determination result of the entire image indicates that the stool is scattered, the determination unit 13A determines that the defecation situation is likely to have diarrhea. to correct. On the other hand, when it is determined that there is no scattering of feces from the determination result of the whole image, the determination unit 13A does not correct the state of defecation as the determination result of the partial image.

In step S60, the determination unit 13A of the determination device 10A performs confirmation processing. The determination process is a process of determining the user's defecation status, etc., using the determination result of the entire image and the determination result of the part.

In step S61, the determination device 10A determines whether or not there is a determination result for the partial image when the determining unit 13A does not have a determination result for both the entire image and the partial image. If there is a judgment result of the partial image, the process shown in step S60 is performed using the judgment result of the partial image. If there is no determination result for the partial image, the process shown in step S60 is performed using the determination result for the entire image. A case where there is no determination result for the partial image is, for example, a case where excrement is not imaged in the partial image and the nature of stool cannot be determined.

As described above, in the determination device 10A according to Modification 2 of the second embodiment, the preprocessing unit 19 generates the entire image and the partial images from the target image. The analysis unit 12A performs global estimation (first estimation) from the entire image using the trained model, and performs detailed estimation (second estimation) from the partial image using another trained model. As a result, the determination device 10A according to Modification 2 of the second embodiment uses the entire image with a large number of pixels to perform relatively easy global estimation, thereby obtaining relatively difficult details from the overall image. It is possible to reduce the load of arithmetic processing and suppress an increase in apparatus cost, as compared with the case of performing a simple estimation. In addition, by performing detailed estimation using a partial image with a relatively small number of pixels, it is possible to reduce the load of arithmetic processing and reduce the device cost compared to performing detailed estimation from an entire image with a relatively large number of pixels. can be suppressed.

Further, in the determination device 10A according to Modification 2 of the second embodiment, the preprocessing unit 19 generates a partial image including at least the opening 36 of the toilet bowl 32 in the target image. As a result, the determination device 10A according to the modification 2 of the second embodiment can cut out a region in which the excrement is likely to fall as a partial image, and use the partial image to make a detailed determination of the excrement. It is possible to do

Further, in the determination device 10A according to the modification 2 of the second embodiment, the preprocessing unit 19 estimates the presence or absence of flight scattering as a global estimation (first estimation), and performs a detailed estimation (second estimation). ) to estimate the properties of the stool. As a result, the determination device 10A according to Modification 2 of the second embodiment can estimate the presence or absence of scattering as well as the properties of stool, and can perform determination with higher accuracy using both estimation results. becomes.

Further, the determination device 10A according to Modification 2 of the second embodiment corrects the estimation result of detailed estimation (second estimation) using the estimation result of global estimation (first estimation). As a result, the determination device 10A according to Modification 2 of the second embodiment can correct detailed estimation, and can perform determination with higher accuracy.

(Third Embodiment)
Next, a third embodiment will be described. This embodiment differs from the above-described embodiments in that a target region in a target image is extracted. The target area is an area to be determined in the present embodiment, and is an area to be determined for the property of excrement. That is, the determination region is a region in which excrement is estimated to be captured in the target image. In the following description, only configurations different from the above-described embodiment will be described, and configurations equivalent to those of the above-described embodiment will be denoted by the same reference numerals, and descriptions thereof will be omitted.

FIG. 15 is a block diagram showing the configuration of a determination device 10B according to the third embodiment. The determination device 10B includes an analysis section 12B and a determination section 13B. Here, the analysis unit 12B is an example of an "extraction unit".

Based on the difference (color difference) between the color of the image in the target image and the predetermined color assumed for the excrement (hereinafter referred to as the assumed color), the analysis unit 12B determines an area having a color close to the assumed color as the determination area. Extract. The analysis unit 12B determines whether or not the color in the target image is close to the assumed color based on the distance in the color space (hereinafter referred to as spatial distance) between the two colors. A small spatial distance between two colors indicates a small color difference and indicates that the two colors are close to each other. On the other hand, when the spatial distance is large, it indicates that the color difference is large and that the two colors are far from each other. Here, the spatial distance is an example of "characteristics of assumed colors."

Here, a method for calculating the spatial distance by the analysis unit 12B will be described. A case where the target image is an RGB image and the assumed colors are colors indicated by RGB values will be described below, but the present invention is not limited to this. Even if the target image is an image other than an RGB image (for example, Lab image or YCbCr image), or if the assumed color is indicated by colors other than RGB values (for example, Lab value or YCbCr value), the same method is used. It is possible to extract the judgment region. Moreover, although the case where the assumed color is the color of stool will be exemplified below, the present invention is not limited to this. The assumed color may be any color assumed for excrement, for example, the color of urine.

The analysis unit 12B calculates, for example, the Euclidean distance in the color space as the spatial distance. The analysis unit 12B calculates the Euclidean distance using the following formula (1). In equation (1), Z1 is the Euclidean distance, ΔR is the difference in R value between a predetermined pixel X in the target image and assumed color Y, ΔG is the difference in G value between pixel X and assumed color Y, and ΔB is pixel X. and the assumed color Y. The RGB values of the predetermined pixel X in the target image are (red, green, blue), and the RGB values of the assumed color Y are (Rs, Gs, Bs).

Z1=(ΔR̂2+ΔĜ2+ΔB̂2)̂(1/2) (1)
however,
ΔR = red - Rs
ΔG = green - Gs
ΔB = blue - Bs

Further, the analysis unit 12B may perform weighting when calculating the spatial distance. Weighting is for emphasizing the difference of specific elements that make up the color. For example, by multiplying the R, G, and B elements that make up the color conduct. By weighting, the color difference from the assumed color can be emphasized according to the element.

The analysis unit 12B can calculate the weighted Euclidean distance using the following equation (2), for example. In equation (2), Z2 is the weighted Euclidean distance, R_COEF is the weighting factor of the R element, G_COEF is the weighting factor of the G element, and B_COEF is the weighting factor of the B element. ΔR is the difference in R value between pixel X and assumed color Y, ΔG is the difference in G value between pixel X and assumed color Y, and ΔB is the difference in B value between pixel X and assumed color Y. The RGB values of the predetermined pixel X in the target image are (red, green, blue), and the RGB values of the assumed color Y are (Rs, Gs, Bs).

Z2=(R_COEF×ΔR^2
+G_COEF×ΔG^2
+B_COEF×ΔB̂2)̂(1/2) …(2)
however,
R_COEF>G_COEF>B_COEF
ΔR = red - Rs
ΔG = green - Gs
ΔB = blue - Bs

Here, regarding the characteristics of the color of feces as the assumed color Y, the R element tends to be stronger than the G element, and the G element tends to be stronger than the B element. Based on the characteristics of each element that constitutes such a color, the analysis unit 12B sets the weighting factor of the R element to a value larger than the weighting factor of the G element. That is, in equation (2), the coefficient R_COFE, the coefficient G_COFE, and the coefficient B_COFE have a relationship of R_COEF>G_COEF>B_COEF.

By the way, it is conceivable that the amount of light illuminating the subject (the inner space 34 of the toilet bowl 32) changes due to the influence of the user's seating position. When the amount of light changes, even excrement of the same color may be imaged as if the shade of color is different. In such a case, even though the colors are the same, the spatial distances are calculated as different distances.

As a countermeasure against this, the analysis unit 12B may calculate the Euclidean distance of the ratio of each element constituting the color (hereinafter referred to as color ratio) as the spatial distance. The color ratio is determined, for example, by dividing the values of the other elements by the reference element values among the R value, G value, and B value. By using the color ratio, it is possible to calculate the spatial distance that does not reflect the difference caused by the shade of color.

The element used as a reference when deriving the color ratio may be arbitrarily determined, but it is conceivable, for example, to use the dominant element of that color. For example, in stool color, the R element is dominant. Therefore, in this embodiment, the color ratio is created by dividing each of the R value, G value, and B value by the R value.

For example, the color ratio of pixel X (RGB values (red, green, blue)) is (red/red, green/red, blue/red), or (1, green/red, blue/red). The color ratio of the assumed color Y (RGB values (Rs, Gs, Bs)) is (Rs/Rs, Gs/Rs, Bs/Rs), that is, (1, Gs/Rs, Bs/Rs). .

The analysis unit 12B can calculate the Euclidean distance of the color ratio using the following equation (3). In equation (3), Z3 is the Euclidean distance of the color ratio, ΔRp is the difference in R factor between the color ratio of pixel X and the assumed color Y, and ΔGp is the color ratio of pixel X and assumed color Y. , and ΔBp is the B-element difference between the color ratio of the pixel X and the assumed color Y. In FIG. GR_RATE is the ratio of the G element in the assumed color Y color ratio, and BR_RATE is the B element ratio in the assumed color Y color ratio. The RGB values of the predetermined pixel X in the target image are (red, green, blue), and the RGB values of the assumed color Y are (Rs, Gs, Bs).

Z3=(ΔRp̂2+ΔGp̂2+ΔBp̂2)̂(1/2)
=(ΔGp̂2+ΔBp̂2)̂(1/2) (3)
however,
ΔRp=red/red−Rs/Rs=0 (zero)
ΔGp=green/red−GR_RATE
ΔBp=blue/red-BR_RATE
GR_RATE = Gs/Rs
BR_RATE=Bs/Rs
1>GR_RATE>BR_RATE>0

Regarding the characteristics of the color of feces as the assumed color Y, the R element tends to be stronger than the G element (Rs>Gs), and the G element tends to be stronger than the B element (Gs>Bs). Both the ratio GR_RATE and the ratio BR_RATE are values between 0 (zero) and 1. Also, the ratio BR_RATE is smaller than the ratio GR_RATE. That is, in equation (3), the relationship 1>GR_RATE>BR_RATE>0 is established between the ratio GR_RATE and the ratio BR_RATE.

Further, when calculating the Euclidean distance of the color ratio, the analysis unit 12B may weight specific elements constituting the color ratio. The analysis unit 12B can calculate the Euclidean distance weighted by the color ratio using the following equation (4). In equation (4), Z4 is the weighted Euclidean distance of the color ratio. ΔRp is the difference in R value between pixel X and assumed color Y, ΔGp is the difference in G value between pixel X and assumed color Y, and ΔBp is the difference in B value between pixel X and assumed color Y. GR_COEF is a weighting factor for the difference ΔGp, and BR_COEF is a weighting factor for the difference ΔBp. The RGB values of the predetermined pixel X in the target image are (red, green, blue), and the RGB values of the assumed color Y are (Rs, Gs, Bs).

Z4=(GR_COEF×ΔGp̂2
+BR_COEF×ΔBp^2)^(1/2) (4)
however,
GP_COEF > BP_COEF
ΔGp=green/red−GR_RATE
ΔBp=blue/red-BR_RATE
GR_RATE=Gs/Rs
BR_RATE=Bs/Rs
1>GR_RATE>BR_RATE>0

In the equation (4), the coefficient GR_COFE and the coefficient BR_COFE have a relationship of GP_COEF>BP_COEF, similar to the relationship between the coefficient G_COFE and the coefficient B_COFE in the equation (2). In addition, in expression (4), the relationship 1>GR_RATE>BR_RATE>0 is established between the ratio GR_RATE and the ratio BR_RATE, as in expression (3). For example, the ratio GR_RATE=0.7, the ratio BR_RATE=0.3, the coefficient GR_COFE=40, and the coefficient BR_COFE=1.

The analysis unit 12B creates an image (grayscale target image) in which the spatial distance calculated for each pixel of the target image is converted into a grayscale. For example, the analysis unit 12B uses equation (5) to adjust the scale of the spatial distance and convert it into a grayscale value. In equation (5), Val is a grayscale value, AMP is a coefficient for scale adjustment, and Z is a spatial distance. The spatial distance Z may be any of the Euclidean distance Z1 of RGB values, the weighted Euclidean distance Z2 of RGB values, the Euclidean distance Z3 of color ratios, and the weighted Euclidean distance Z4 of color ratios. Z_MAX is the maximum spatial distance calculated for each pixel of the target image, and Val_MAX is the maximum grayscale value.

Val=AMP×Z (5)
however,
AMP = Val_MAX/Z_MAX

For example, the maximum value Val_MAX of the grayscale value is 255 when the grayscale gradation is indicated in 256 steps from 0 to 255 in the grayscale target image. In this case, the spatial distance Z is converted into a grayscale value Val by equation (5) so that the maximum spatial distance value X_MAX becomes the maximum grayscale value Val_MAX (255). As a result, the analysis unit 12B creates a grayscale target image in which the spatial distance from the assumed color is represented by grayscale values from 0 (white) to 255 (black).

Here, a method for extracting the determination region by the analysis unit 12B will be described with reference to FIG. 16 . FIG. 16 is a diagram illustrating processing performed by the analysis unit 12B according to the third embodiment. FIG. 16 shows the grayscale axis in the left-to-right direction, indicating that grayscale values increase as the grayscale axis moves from left to right.

As shown in FIG. 16, in the grayscale target image, pixels with small spatial distances are expressed with small grayscale values. In other words, a color closer to the color of feces as an assumed color is represented by a small grayscale value, and the region with a small grayscale value can be regarded as the region where feces is imaged. On the other hand, in the grayscale target image, pixels with large spatial distances are expressed with large grayscale values. In other words, a color that is different from the color of feces as an assumed color is represented by a large grayscale value, and an area with a large grayscale value can be regarded as an "inconvenient" area in which feces are not imaged.

Using this property, the analysis unit 12B extracts the determination region. Specifically, the analysis unit 12B defines a region in which the grayscale value of pixels in the grayscale target image is less than a predetermined first threshold value (threshold value 1) as a region where excrement exists, and determines that the excrement exists as a determination region. Extract regions. The first threshold is a grayscale value corresponding to a boundary that distinguishes between the color of the wash water S stored in the toilet bowl 32 and the color of watery feces.

Further, when the color of hard stool and watery stool is compared, watery stool is dissolved in water (rinsing water S), so it is considered to be lighter than hard stool. In this case, the grayscale value corresponding to the color of watery stool is expressed in darker gray, which indicates that the color is farther from the assumed color of stool than the grayscale value corresponding to hard stool.

Using this property, the analysis unit 12B distinguishes and extracts a watery stool area and a hard stool area from the determination area. Specifically, among the determination regions in the grayscale target image, the analysis unit 12B determines that a region in which the grayscale value is less than a predetermined second threshold value (threshold value 2) is a hard stool region, and a region in which the grayscale value is equal to or greater than the second threshold value. It is an area of watery stool. The second threshold is set to a value smaller than the first threshold. Here, the region of watery stool is an example of the “determination region”. Also, the hard stool region is an example of the “determination region”.

Alternatively, when the determination region includes both the watery stool region and the hard stool region, the analysis unit 12B extracts the two regions (the watery stool region and the hard stool region) separately. can be When two areas are mixed in the determination area, the grayscale range that the pixels included in the determination area can take is a combination of the grayscale range that can be taken by watery stool and the grayscale range that can be taken by hard stool. This is a relatively wide range. On the other hand, when there is only one area (a watery stool area or a hard stool area) in the determination area, the possible grayscale range of the pixels included in the determination area is a relatively narrow range. .

Using this property, the analysis unit 12B determines whether or not a watery stool region and a hard stool region are mixed in the determination region according to the grayscale range of the pixels included in the determination region. do. The analysis unit 12B, for example, takes the difference between the maximum value and the minimum value of the grayscale in the pixels included in the determination region as the range of the grayscale. When the grayscale range in the determination region is less than the predetermined difference threshold, the analysis unit 12B determines that the determination region does not include a watery stool region and a hard stool region (that is, the determination region is a watery stool region). It is determined that there is only an area of stool that is difficult or hard stool). When the grayscale range in the determination region is equal to or greater than a predetermined difference threshold, the analysis unit 12B determines that the determination region includes both a watery stool region and a hard stool region. The difference threshold corresponds to, for example, whichever is the wider range, whichever the narrower range, or the representative value of the two ranges, depending on the grayscale range that can be taken by watery stool and the range of grayscale that can be taken by hard stool. set to the value The representative value may be any commonly used representative value such as a simple addition average value, a weighted average value, or a median value of two ranges.

The analysis unit 12B outputs information of an image (extracted image) indicating the extracted determination region to the determination unit 13B. In this case, when the determination region includes a mixture of the watery stool region and the hard stool region, the analysis unit 12B generates an image (watery part extraction image) showing the watery stool region in the determination region and the determination Information of an image (hard part extraction image) indicating a hard stool area in the area is output to the determination unit 13B. On the other hand, when the determination region does not include the watery stool region and the hard stool region, the analysis unit 12B selects an image (watery stool extraction image) indicating the watery stool region as the determination region, or , the information of the image (hard stool extraction image) indicating the region of hard stool is output to the determination unit 13B.

Returning to FIG. 15, the determination unit 13B performs determination regarding determination items based on the extracted image acquired by the analysis unit 12B. Specifically, the determination unit 13B determines the properties of watery stool using the watery stool extraction image. The determination unit 13B determines the properties of hard stool using the hard stool extraction image. The determination unit 13B determines the properties of watery stool using the watery part extraction image. The determination unit 13B determines the properties of hard stool using the hard part extraction image.

The determination unit 13B may determine the properties of stool using an estimation result obtained by machine learning, as in the other embodiments described above. In this case, the analysis unit 12B may perform the estimation by machine learning, or another functional unit may perform the estimation. Further, the determination unit 13B may determine the properties of stool using another image analysis method. In this case, the determination device 10B can omit the learned model storage unit 16 .

Since the determination unit 13B can analyze the image whose range has been narrowed down by extracting the determination region by the analysis unit 12B, it is not necessary to analyze the entire target image. Further, since the determination unit 13B can analyze an image in which watery stool or hard stool is distinguished, when an image in which watery stool or hard stool is not distinguished is set as an analysis target. Compared to , the process of determining properties becomes easier.

Here, processing performed by the determination device 10B of the third embodiment will be described with reference to FIG. 17 . FIG. 17 is a flow chart showing the flow of processing performed by the determination device 10B according to the third embodiment. This flowchart shows the flow of processing after the processing for acquiring image information is performed. The process of acquiring the image information is the process corresponding to step S11 in the flowchart shown in FIG. 4, and corresponds to the process described as "camera image" in this flowchart.

The analysis unit 12B grayscales the target image to create a grayscale target image (step S70).
The analysis unit 12B determines whether the grayscale value of each pixel in the grayscale target image is less than a first threshold (threshold 1) (step S71).
In the grayscale target image, the analysis unit 12B calculates the difference D between the maximum value and the minimum value of the grayscale value for the pixel group of the determination region whose grayscale value is less than the first threshold value (threshold value 1) (step S72).

The analysis unit 12B determines whether or not the difference D is less than a predetermined difference threshold value a (step S73). It is determined that the area of the flight is not mixed, and the process proceeds to step S74. In step S74, the analysis unit 12B determines whether the grayscale value of each pixel in the determination region is less than the second threshold (threshold 2).
When the grayscale value of each pixel in the determination region is less than the second threshold (threshold 2), the analysis unit 12B outputs the hard stool extraction image to the determination unit 13B (step S75). Based on the hard stool extraction image, the determining unit 13B determines the properties of only hard stool (specialized for hard stool) (step S82).
When the grayscale value of each pixel in the determination region is equal to or greater than the second threshold (threshold 2), the analysis unit 12B outputs the watery stool extraction image to the determination unit 13B (step S76). Based on the watery stool extraction image, the determining unit 13B determines the properties of only watery stool (specific to watery stool) (step S83).

On the other hand, when the difference D is equal to or greater than the predetermined difference threshold value a (step S73, NO), the analysis unit 12B determines that the determination region includes both the watery stool region and the hard stool region (step S77). The analysis unit 12B determines whether the grayscale value of each pixel in the determination region is less than the second threshold (threshold 2) (step S78).
When the grayscale value of each pixel in the determination region is less than the second threshold (threshold 2), the analysis unit 12B outputs the region to the determination unit 13B as a hard part image (step S79). The determining unit 13B determines the properties of the stool of the hard part (region of hard feces in a mixed state) based on the hard part extraction image (step S84).
When the grayscale value of each pixel in the determination region is equal to or greater than the second threshold (threshold 2), the analysis unit 12B outputs the region to the determination unit 13B as a watery part extraction image (step S76). The determining unit 13B determines the stool properties of the watery part (area of watery stool in a mixed state) based on the watery part extraction image (step S85).

The determination unit 13B comprehensively determines the properties of feces in the target image using the results of the determination of the properties of feces in steps S82 to S85 (step S86).

Note that in step S71, a pixel group in which the grayscale value of each pixel in the grayscale target image is equal to or greater than the first threshold value (threshold value 1) is determined to be an image other than feces and excluded from the determination region (step S81). .

As described above, in the determination device 10B of the third embodiment, the analysis unit 12B extracts a determination region from the target image based on the color characteristics of the assumed color Y. FIG. As a result, the determination device 10B of the third embodiment can extract a region containing excrement from the target image. Since the region for determining the nature of stool can be narrowed down, it is possible to reduce the processing load required for determination as compared with the case where the entire target image is analyzed. By reducing the processing load, it is possible to perform processing even with a device that does not have high computing power, so an increase in device cost can be suppressed. In addition, since the judgment area can be extracted based on the color characteristics of the assumed color Y of the excrement to be judged, the judgment area is compared with the area extracted regardless of the assumed color Y. Judgment becomes easier.

Further, in the determination device 10B of the third embodiment, the analysis unit 12B calculates the spatial distance Z on the color space between the assumed color Y and the color of each pixel in the target image, and the calculated spatial distance Z is a predetermined value. A set of pixels below the threshold of is extracted as a determination region. As a result, the determination device 10B of the third embodiment can calculate the color difference (color difference) from the assumed color Y using the spatial distance Z, and determine an area with a small color difference from the assumed color Y. It is possible to extract the determination region based on the relevant index.

Further, in the determination device 10B of the third embodiment, the analysis unit 12B uses a value obtained by weighting the difference of each color element from the assumed color Y for the color of each pixel in the target image to determine the color space Calculate the spatial distance above. As a result, with the determination device 10B of the third embodiment, it is possible to calculate a spatial distance in which an element (for example, an R element) that is likely to differ from the assumed color Y is emphasized. This makes it possible to extract the determination region with high accuracy.

Further, in the determination device 10B of the third embodiment, the target image is an RGB image, the assumed colors are colors represented by RGB values, and the analysis unit 12B calculates the R value, G value, Using a value obtained by weighting the ratio difference in the R element, the ratio difference in the G element, and the ratio difference in the B element between the color ratio indicating the ratio of the B value and the color ratio in the assumed color Y, Calculate the spatial distance on the color space. As a result, the determination device 10B of the third embodiment can calculate the spatial distance without being affected by the difference in color density due to the difference in the amount of light illuminating the subject. This makes it possible to extract the determination region with high accuracy.

Further, in the determination device 10B of the third embodiment, the analysis unit 12B creates a grayscale target image, and the grayscale value of a pixel in the grayscale target image is less than a predetermined first threshold (threshold 1). The region is defined as a region containing excrement, and the region containing excrement is extracted as a determination region. As a result, the determination device 10B of the third embodiment can extract the determination region by a simple method of comparing the grayscale value of each pixel in the grayscale target image with the threshold value.

Further, in the determination device 10B of the third embodiment, the analysis unit 12B determines that the grayscale value of the pixel in the grayscale target image is less than the first threshold and is equal to or greater than a predetermined second threshold that is smaller than the first threshold. A region is defined as a region where watery stool is shown, and a region where the grayscale value of pixels in the grayscale target image is less than the second threshold is defined as a region where hard stool is shown. A region showing stool and a region showing hard stool are extracted. As a result, in the determination device 10B of the third embodiment, by a simple method of comparing the grayscale value of each pixel in the grayscale target image with a threshold value, the region showing watery stool and the region showing hard stool are detected. Therefore, it is possible to extract the determination area by distinguishing it from the area that has been detected, and it is possible to extract the determination area with higher accuracy. In addition, by distinguishing between the region showing watery stool and the region showing hard stool and extracting the judgment region, the processing load of the judgment by the judgment unit 13B can be reduced compared to the case where no distinction is made. is possible.

In the above description, the case where the analysis unit 12B extracts the determination region using one grayscale target image has been described as an example, but the present invention is not limited to this. The analysis unit 12B may extract the determination region using a plurality of different grayscale target images. For example, the analysis unit 12B may use a grayscale target image obtained by converting the color-ratio-weighted Euclidean distance Z4 into a grayscale, and perform only the process of extracting the determination region based on the first threshold. Then, the analysis unit 12B uses the grayscale target image obtained by converting the Euclidean distance Z1 into grayscale, and performs only the process of distinguishing and extracting the regions of watery stool and hard stool according to the second threshold. good.

Moreover, although a plurality of embodiments have been described above, the configuration of each embodiment is not limited to the configuration of the embodiment, and may be used as the configuration of other embodiments. For example, the difference image, divided image, or whole image and partial image according to the second embodiment and its modification may be used for the process of determining the properties of stool in the first embodiment. Further, the grayscale target image in the third embodiment may be used for the difference image and the like according to the second embodiment and its modification. Further, the difference image, the divided image, or the whole image and the partial image according to the second embodiment and its modification may be used for the process of determining the properties of stool of the third embodiment.

All or part of the processing performed by the determination device 10 (10A, 10B) in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read into a computer system and executed. It should be noted that the "computer system" referred to here includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible discs, magneto-optical discs, ROMs and CD-ROMs, and storage devices such as hard discs incorporated in computer systems. Furthermore, "computer-readable recording medium" means a medium that dynamically retains a program for a short period of time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include something that holds the program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or client in that case. Further, the program may be for realizing a part of the functions described above, or may be capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be implemented using a programmable logic device such as FPGA.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design and the like are included within the scope of the gist of the present invention.

1 determination system 10 determination device 11 image information acquisition unit 12 analysis unit (estimation unit) (extraction unit)
13 Determination unit 14 Output unit 15 Image information storage unit 16 Learned model storage unit 17 Determination result storage unit 18 Communication unit 20 Learning device 21 Communication unit 22 Learning unit 3 Toilet device 30 Toilet bowl 32 Toilet bowl 34 Internal space 36 Opening S Cleaning Water 4 Imaging device

Claims (6)

an image information acquisition unit that acquires image information of a target image that is a target image for determining a determination item related to stool and that is an image of the internal space of the toilet bowl after excretion;
a preprocessing unit that generates an entire image showing the entire target image and a partial image showing a partial area of the target image;
Machine learning using a neural network determines the correspondence relationship between the overall image for learning, which is an image showing the entire internal space of the toilet bowl after excretion, and the determination result of the first overall determination item among the determination items. By inputting the whole image to the learned model that has been trained, a first estimation regarding the first determination item is performed for the whole image, and a learning partial image that is a partial area of the learning whole image is obtained. , by inputting the partial image to a trained model trained by machine learning using a neural network for the correspondence relationship between the second judgment item, which is more detailed than the first judgment item, among the judgment items, an estimation unit that performs a second estimation regarding the second determination item for the image;
a determining unit that determines the determination item for the target image based on the estimation result of the estimating unit;
A determination device comprising: The preprocessing unit generates a partial image including at least an opening of a toilet bowl in the target image.
The determination device according to claim 1. The estimation unit estimates the presence or absence of scattering of stool as the first estimation, and estimates the nature of the stool as the second estimation.
The determination device according to claim 1 or 2. The determination unit corrects the estimation result of the second estimation using the estimation result of the first estimation.
The determination device according to any one of claims 1 to 3. An image information acquisition unit acquires image information of a target image that is a target image for determining a judgment item related to stool, and that is an image of the internal space of the toilet bowl after excretion,
a preprocessing unit that generates a whole image as the target image and a partial image that is a partial area of the target image;
The estimating unit uses a neural network to determine the correspondence relationship between the overall learning image, which is an image showing the entire internal space of the toilet bowl after excretion, and the determination result of the first overall determination item among the determination items. By inputting the whole image to a trained model trained by machine learning, a first estimation regarding the first determination item is performed for the whole image, and a learning that is a part of the learning whole image inputting the partial image to a trained model that has learned the corresponding relationship between the partial image for use and a second determination item that is more detailed than the first determination item among the determination items by machine learning using a neural network; performs a second estimation regarding the second determination item for the partial image,
a determining unit, based on the estimation result by the estimating unit, determines the determination item for the target image;
judgment method. The computer of the judgment device,
Image information acquisition means for acquiring image information of a target image that is a target image for determining a judgment item related to stool, and that is an image of the internal space of the toilet bowl after excretion;
Preprocessing means for generating a whole image as the target image and a partial image as a partial area of the target image;
Machine learning using a neural network determines the correspondence relationship between the overall image for learning, which is an image showing the entire internal space of the toilet bowl after excretion, and the determination result of the first overall determination item among the determination items. By inputting the whole image to the learned model that has been trained, a first estimation regarding the first determination item is performed for the whole image, and a learning partial image that is a partial area of the learning whole image is obtained. , by inputting the partial image to a trained model trained by machine learning using a neural network for the correspondence relationship between the second judgment item, which is more detailed than the first judgment item, among the judgment items, estimation means for performing a second estimation regarding the second determination item for the image;
determination means for determining the determination item for the target image based on the estimation result by the estimation means;
A program to function as

Images