JP7164135B2 - Information processing device, evaluation system, machine learning device, information processing method, and program - Google Patents

Description

The present invention relates to a technique for inspecting the condition of nails.

Human fingernails grow daily and need to be trimmed regularly. Dirt tends to accumulate on the inside of overgrown nails, so it is required that nails are always trimmed to an appropriate length at food manufacturing sites and medical sites. In addition, false nails and nails with nail art such as manicure are not preferable in terms of cleanliness, contamination, and risk of infection in the medical field.

In order to prevent these situations, methods have been taken such as encouraging each person to take care of their personal appearance through enthusiastic guidance, and thoroughly enforcing the rule of having others double-check when entering and leaving a room. However, this method cannot completely prevent human error, and it is necessary to stop the work of the surrounding people and have them check it when another person double-checks it. I had a problem.

Japanese Patent Application Laid-Open No. 2017-185102

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and it is an object of the present invention to provide a technique for inspecting the state of a nail, which can easily determine whether the state of the nail is acceptable without human visual confirmation. do.

According to the technique described in Patent Literature 1, it is possible to select recommended sports hand protectors based on the measured three-dimensional shape of the hand. However, there has been no technology that solves the problem of simply determining whether the nail condition is acceptable or not without human visual confirmation.

In order to solve the above problems, an information processing apparatus according to an aspect of the present invention includes an acquisition unit that acquires image data including a nail as a subject; and an evaluation unit that generates information indicating an evaluation result by evaluating the sanitary condition.

The information processing apparatus according to each aspect of the present invention may be implemented by a computer. In this case, the information processing apparatus is implemented by the computer by operating the computer as each part (software element) provided in the information processing apparatus. A control program for an information processing apparatus realized by a computer and a computer-readable recording medium recording it are also included in the scope of the present invention.

Advantageous Effects of Invention According to the present invention, it is possible to provide a technique capable of easily determining whether a nail condition is acceptable or not without human visual confirmation.

1 is a configuration diagram of a nail condition inspection device (information processing device) according to Embodiment 1 of the present invention; FIG. 1 is an explanatory diagram of an imaging device according to Embodiment 1 of the present invention; FIG. 1 is an example of a schematic configuration of a nail condition inspection device (evaluation system) according to Embodiment 1 of the present invention; 1 is an example of a schematic configuration of a nail condition inspection device (evaluation system) according to Embodiment 1 of the present invention; It is a block diagram which shows the structure of the evaluation system which concerns on Embodiment 2 of this invention. 6 is a flowchart showing the flow of an information processing method performed by an information processing device included in the evaluation system of FIG. 5; FIG. 7 is a schematic diagram illustrating an example of processing in an evaluation step shown in the flowchart of FIG. 6; 6 is a diagram showing an example of "information indicating an evaluation result" output by an output device on the subject's side included in the evaluation system of FIG. 5; FIG. 6 is a diagram showing an example of "information indicating an evaluation result" output by an output device on the side of a manager included in the evaluation system of FIG. 5; FIG. 6 is a perspective view showing the configuration of an imaging device included in the evaluation system of FIG. 5; FIG. FIG. 11 is a top view showing an example of the configuration of a guide mechanism provided in the imaging device of FIG. 10; 6 is a flow chart showing the flow of a machine learning method performed by a machine learning device included in the evaluation system of FIG. 5; 6 is a block diagram showing the hardware configuration of an information processing device included in the evaluation system of FIG. 5; FIG. 6 is a block diagram showing the hardware configuration of a machine learning device included in the evaluation system of FIG. 5; FIG. It is a block diagram which shows the structure of the evaluation system which concerns on Embodiment 3 of this invention.

[Embodiment 1]
EMBODIMENT OF THE INVENTION Hereinafter, with reference to an accompanying drawing, the form (henceforth an embodiment) for implementing this invention is demonstrated in detail. The same numbers are given to the same elements throughout the description of the embodiment.

(Configuration of Embodiment 1)
As shown in FIG. 1, a nail condition inspection device (evaluation system) 1 according to the first embodiment includes at least an imaging device 10, an arithmetic device (information processing device) 20, and an input/output device 30 as hardware. The computing device (information processing device) 20 has a trained neural network (evaluation model) 21 .

The arithmetic device (information processing device) 20 and the input/output device 30 can be implemented by, for example, a computer with a stand-alone configuration (for example, a personal computer) or an online configuration that combines a server and a computer.

The nail condition inspection device (evaluation system) 1 can be used for determination for any purpose, and can be used particularly for determining whether or not a person can enter or leave the room (indoor) R by the above determination. Furthermore, it is preferable that the room (indoor) R is a food manufacturing site or a medical site where hand hygiene is required, and a room (indoor) R of a food factory or a food processing plant R is particularly preferable. .

The computing device (information processing device) 20 includes a trained neural network (evaluation model) 21 trained with an image H of a human hand. The image H of the human hand may be obtained by the imaging device 10, or may be obtained by using another imaging device. Those obtained from the apparatus are preferred. Also, the image of the human hand includes at least the image of the back side of the human hand to inspect the nails.

The learned neural network (evaluation model) 21 can be learned from nail information such as the length of the nail and the color tone of the entire nail, and judgment result information by visual observation. From the side image, the condition of the nail can be determined.

Further, the learned neural network (evaluation model) 21 can learn based on the color tone of the nail as a whole, so that it is possible to determine whether or not there is a deposit on the nail. Examples of attachments include nail art, scalps, nail tips, installation of rhinestones, manicures, gels, and the like.

The input/output device 30 can use an output device (output device) such as a display, an indicator lamp, and an acoustic device such as a buzzer. Furthermore, input devices such as a mouse, keyboard, and touch panel may be provided.

As shown in FIG. 2, an imaging unit (imaging device) 10 according to the present embodiment has an imaging device 101 installed at the top, a bottom surface 103 installed at the bottom, and a light shielding wall 102 . The bottom surface 103 is preferably monochromatic. The light shielding wall 102 and the bottom surface 103 can also be configured as a continuous body so that light from the outside does not easily hit the bottom surface 103 . Furthermore, as shown in FIG. 10, which will be described later, an imaging box 105, which is a box-shaped continuous body formed by adding a top surface 104, can be configured. It prevents light from entering and can be installed at the entrance/exit of a room (indoor). A subject P can obtain an image by inserting the back of his or her hand into the imaging device.

FIG. 3 shows a configuration example in which the nail condition inspection device (evaluation system) 1 has a stand-alone configuration. In this case, the computing device (information processing device) 20 and the input/output device 30 may be configured as one personal computer. The subject P acquires a hand image H (see FIG. 7, which will be described later) using the imaging device 10 and transmits the hand image H to the arithmetic device (information processing device) 20 . The computing device 20 judges the image H of the hand by means of a trained neural network (evaluation model) 21 . It may be possible to output the determination result to the input/output device 30 .

The administrator S may be able to confirm the stored information of the subject P using the personal computer. At that time, information on the physical condition of the subject P, such as body temperature, breath alcohol information, whether or not there is a wound, is linked to information uniquely set for the individual, such as the employee code of the subject P. Information such as whether or not the hair is sticking out of the hat may be displayed in a list form.

FIG. 4 shows a configuration example in which the entrance/exit inspection device (evaluation system) 1 has an online configuration. In this case, the subject P and the administrator S may be able to use separate input/output devices 30 . The subject P acquires a hand image H (see FIG. 7, which will be described later) using the imaging device 10 and transmits the hand image H to the arithmetic device (information processing device) 20 . The computing device (information processing device) 20 judges the image H of the hand using a trained neural network (evaluation model) 21 . The determination result may be output to the input/output device 30 on the subject side and/or the administrator side.

The administrator S may be able to check the stored information on the subject P using the input/output device on the administrator side. At that time, information on the physical condition of the subject P, such as body temperature, breath alcohol information, and whether or not there is a wound, is linked to the information uniquely set for the individual, such as the employee code of the subject P, and is separately measured or input. , whether or not the hair is sticking out of the hat, etc., may be displayed in a list form.

[Embodiment 2]
Other embodiments of the invention are described below. For convenience of description, members having the same functions as the members described in the first embodiment are denoted by the same reference numerals, and description thereof will not be repeated.

(Configuration of evaluation system 1a)
An evaluation system 1a according to Embodiment 2 of the present invention will be described with reference to FIG. FIG. 5 is a diagram showing the configuration of the evaluation system 1a.

Evaluation system 1a is a system for evaluating hygiene based on nail length. The evaluation system 1a includes an imaging device 10a, an information processing device 20a, an output device 30a, and a machine learning device 40, as shown in FIG. The input/output device 30 is an example of the output device described in the claims. The information processing device 20a is communicably connected to each of the imaging device 10a, the machine learning device 40, and the output device 30a.

The imaging device 10a is a device for capturing an image of a human body part including a nail and generating image data including the nail as a subject (hereinafter, sometimes simply referred to as "image data"). The imaging device 10a differs from the imaging device 10 according to the first embodiment at least in that an imaging box 105 is provided. The details of the configuration of the imaging device 10a will be described later with reference to a different drawing. The output device 30a is an output device included in the input/output device 30 in the first embodiment.

The information processing device 20a differs from the computing device (information processing device) 20 in the first embodiment in that it uses a first evaluation model 21a and a second evaluation model 21b instead of the trained neural network 21 (FIG. 1). At least different. The information processing device 20a is a device for implementing the information processing method M1. The information processing method M1 is a method of evaluating nail hygiene using the first evaluation model 21a and the second evaluation model 21b based on the image data provided from the imaging device 10a. Details of the configuration of the information processing apparatus 20a and the flow of the information processing method M1 will be described later with reference to different drawings.

Here, as the first evaluation model 21a and the second evaluation model 21b, for example, machine learning algorithms using neural networks, such as CNN (Convolution Neural Network) and GAN (Generative Adversarial Network), may be used. can be done. As the first evaluation model 21a and the second evaluation model 21b, machine learning algorithms that do not use neural networks, such as support vector machines, random forests, and gradient boosting trees, can be used. However, algorithms for realizing the first evaluation model 21a and the second evaluation model 21b are not limited to those described above. Algorithms for realizing the first evaluation model 21a and the second evaluation model 21b may be the same or different.

The first evaluation model 21a is constructed by machine learning so as to input image data and output first information indicating the evaluation result of the sanitary condition based on the length of the nail. Here, the "nail length" is, for example, the length of the tip of the nail. The "toe" is a white portion that protrudes from the nail bed, and the length of the toe corresponds to the length from the yellow line of the nail to the tip of the nail. The "yellow line" is the boundary between the nail plate and the toe, the "nail plate" is the area of the nail above the nail bed, and the "nail bed" is the subcutaneous tissue below the nail plate. is. However, the first evaluation model 21a is not machine-learned using the length of the nail (for example, the length of the toe described above) itself as correct data. The first evaluation model 21a is constructed by machine learning using evaluation results obtained by an evaluator from the viewpoint of nail length (for example, the length of the toe described above) as correct answer data. Note that the "length of the nail" is not limited to the length of the tip of the nail described above. For example, the "length of the nail" may be the length of the entire nail or the ratio of the length of the tip to the length of the entire nail. In addition, the "length of the nail" is a comprehensive viewpoint including part or all of the length of the nail, the length of the entire nail, the ratio of the length of the nail to the length of the entire nail, and other aspects. may be In this case, the first evaluation model 21a is constructed by machine learning using the evaluation result of the evaluator's evaluation of the sanitary condition from a comprehensive viewpoint of the nail length as the correct answer data. A method of constructing the first evaluation model 21a by machine learning will be described later.

The second evaluation model 21b is constructed by machine learning so as to input the image data and output the second information indicating the evaluation result of the sanitary condition based on the nail color. Here, the "nail color" is the color tone, hue, or the like of the nail. Further, the "nail color" may be the color of at least a part of the nail, but is preferably the color of the entire nail from the viewpoint of hygiene. A method of constructing the second evaluation model 21b by machine learning will be described later.

The relationship between the image data including the nail as the subject, which is the input of the first evaluation model 21a, and the sanitary condition based on the length of the nail, which is the output of the first evaluation model 21a, is explicitly specified as a relational expression. It is known that there is a certain relationship, although it is difficult to Similarly, there is a certain relationship between the image data including nails as an object, which is the input of the second evaluation model 21b, and the hygiene state based on the nail color, which is the output of the second evaluation model 21b. It has been known. Therefore, by using the first evaluation model 21a whose input is image data including nails as subjects, it is possible to accurately evaluate the sanitary condition based on the length of the nails. Further, by using the second evaluation model 21b that receives image data including nails as an object, it is possible to accurately evaluate the sanitary condition based on the color of the nails.

The machine learning device 40 is a device for implementing the machine learning method M2. The machine learning method M2 creates a learning data set using the learning image data, and builds the first evaluation model 21a and the second evaluation model 21b by machine learning using the learning data data set. It is a method for The configuration of the machine learning device 40 and the details of the flow of the machine learning method M2 will be described later with reference to a different drawing.

The evaluation system 1a goes through a preparation phase, a trial phase, and then a practical use phase. A brief description of the contents of the preparation phase, the trial phase, and the practical use phase is as follows.

(1) Preparation Phase In the preparation phase, the machine learning device 40 constructs the first evaluation model 21a and the second evaluation model 21b. In the preparation phase, an evaluator performs an evaluation based on the length of the nail and an evaluation based on the color of the nail included as a subject in each of the plurality of image data for learning. The machine learning device 40 generates a learning data set composed of teacher data in which the evaluation result by the evaluator is labeled for each learning image data, and performs the first evaluation by machine learning using the learning data set. A model 21a and a second evaluation model 21b are constructed. The constructed first evaluation model 21a and second evaluation model 21b are transferred from the machine learning device 40 to the information processing device 20a.

(2) Trial Phase In the trial phase, the evaluation system 1a is installed at the entrance of a room (for example, a room such as a room in a food factory). When a worker who should enter the room inserts his/her fingernail into the imaging box 105 (see FIG. 10 described later) of the imaging device 10a, the information processing device 20a acquires image data including the fingernail as an object from the imaging device 10a. to assess the hygiene of the nail. In the trial phase, the evaluator performs an evaluation based on the length of the nail and an evaluation based on the color of the nail for the image data to be evaluated by the information processing device 20a. The evaluation by the evaluator may be performed each time the information processing device 20a performs the evaluation, or the evaluation by the information processing device 20a may be performed for a predetermined period (for example, one week, one month, or one year) or after a predetermined period of time. After being performed a number of times (for example, 100 times, 1000 times, or 10000 times), it may be performed collectively. The evaluator compares the evaluation result by the evaluator and the evaluation result by the information processing device 20a, and evaluates the evaluation accuracy of the information processing device 20a. If the evaluation accuracy is insufficient, return to the preparation phase. If the evaluation accuracy is sufficient, proceed to the practical use phase. Note that evaluation accuracy may be confirmed using a part of the learning data set after the preparation phase is finished. In this case, the trial phase can be omitted.

(3) Practical Phase In the practical phase, the information processing device 20a evaluates nail hygiene based on the length and color of the nail. The first evaluation model 21a and the second evaluation model 21b used by the information processing apparatus 20a in the practical phase were confirmed to have sufficient estimation accuracy in the trial phase. In the practical phase, the visual assessment of hygiene based on nail length and color can be omitted when the subject enters the room of the food factory. Therefore, the subject is freed from the trouble of evaluating the hygiene state based on the length and color of the nails, and the hygiene state based on the length and color of the nails can be easily and accurately evaluated without human visual confirmation. evaluation can be made.

(Configuration of information processing device 20a)
The configuration of the information processing device 20a will be described with reference to FIG. The information processing device 20 a includes a control section 22 and a storage section 23 . The control unit 22 comprehensively controls the information processing device 20a. In the information processing device 20 a , the control section 22 includes an acquisition section 221 and an evaluation section 222 . Acquisition unit 221 executes acquisition step M11 of information processing method M1. The evaluation unit 222 executes the evaluation step M12 and the output step M13 of the information processing method M1. The details of the flow of the information processing method M1 will be described later. The storage unit 23 stores a first evaluation model 21a and a second evaluation model 21b.

(Flow of information processing method M1)
The flow of the information processing method M1 will be described with reference to FIG. FIG. 6 is a flow chart showing the flow of the information processing method M1.

The information processing method M1 includes an acquisition step M11, an evaluation step M12, and an output step M13.

(Acquisition step M11)
Acquisition step M11 is a step in which the acquisition unit 221 acquires image data including a nail as a subject. In acquisition step M11, the acquisition unit 221 acquires image data including a nail as a subject generated by the imaging device 10a.

(Evaluation step M12)
The evaluation step M12 is a step in which the evaluation unit 222 evaluates the sanitary condition based on the length and color of the nail using the first evaluation model 21a and the second evaluation model 21b. In the evaluation step M<b>12 , the evaluation unit 222 evaluates nail hygiene based on at least two fingernail regions included in the image data acquired by the acquisition unit 221 . Specifically, the evaluation unit 222 inputs the image data including the nail as the subject acquired by the acquisition unit 221 to the first evaluation model 21a, and acquires the first information output from the first evaluation model 21a. . In addition, the evaluation unit 222 inputs the image data including the nail as a subject acquired by the acquisition unit 221 to the second evaluation model 21b, and acquires the second information output from the second evaluation model 21b. Also, the evaluation unit 222 generates information indicating evaluation results based on the first information and the second information. Note that the evaluation unit 222 writes information indicating the evaluation result into the storage unit 23 .

FIG. 7 is a schematic diagram illustrating an example of the processing of the evaluation step M12. For example, the evaluation step M12 includes a process of extracting partial image data, a first evaluation process, a second evaluation process, and a process of generating information indicating evaluation results. These processes will be described with reference to FIG.

(Extraction processing of partial image data)
The evaluation unit 222 extracts the “nail” regions of each of the five fingers from the image data H obtained by the acquisition unit 221 and including the nail as the subject, and converts them into partial image data IMG-1, IMG-2, IMG-3, and IMG. -4, extracted as IMG-5. Partial image data IMG-1 is the thumb nail area, partial image data IMG-2 is the index finger nail area, partial image data IMG-3 is the middle finger nail area, and partial image data IMG-1 is the nail area of the index finger. 4 is the area of the nail of the ring finger, and the partial image data IMG-5 is the area of the nail of the little finger. Hereinafter, partial image data IMG1 to IMG5 are also simply referred to as partial image data IMG-i (i=1, 2, 3, 4, 5) when there is no particular need to distinguish between them. In other words, partial image data IMG-i includes one nail as a subject.

A known object detection algorithm can be used as a method for extracting the nail region from the image data. Examples of such object detection algorithms include, but are not limited to, YOLO V3 (You Only Look Once V3), MASK R-CNN (MASK Regions with CNN features), and the like.

Also, the number of partial image data IMG-i extracted from the image data H by the evaluation unit 222 is not limited to five, but is preferably plural. For example, the evaluation unit 222 continues the process when n (n is an integer equal to or greater than 2) pieces of partial image data IMG-i are extracted, and when less than n pieces of partial image data IMG-i are extracted, may interrupt processing and output error information. For example, by setting n to 2 or more and less than 5, if all five nails are not included in the image data H due to the position of the hand at the time of shooting, or if the orientation of the finger at the time of shooting, etc. It is possible to flexibly deal with a case where some nails are not clearly included in the image data H due to the above.

(First evaluation process)
The evaluation unit 222 inputs each of the extracted n pieces of partial image data IMG-i to the first evaluation model 21a. As a result, the first score is output from the first evaluation model 21a. Here, the first score is an example of the first information indicating the evaluation result of the sanitary condition based on the nail length. The first score is a numerical value in the range of 0 or more and 1 or less, and indicates the possibility of good hygiene based on the length of the nail. That is, the closer the first score is to 1, the higher the possibility that the sanitary condition based on the nail length is good, and the closer the first score is to 0, the higher the possibility that the hygiene condition is not good. If the average value of the n first scores obtained from the n partial image data IMG-i is equal to or greater than a predetermined threshold, the evaluation unit 222 evaluates the sanitary condition based on the nail length of the subject P. Evaluate as "Pass". On the other hand, if the average value of the first scores is less than the predetermined threshold value, the evaluation unit 222 evaluates the sanitary condition based on the nail length of the subject P as "failed". The evaluation unit 222 stores, in the storage unit 23, information indicating whether the length of the nail is acceptable in association with the identification information (employee code, etc.) of the subject P. FIG.

In the first evaluation process, the method of determining whether the sanitary condition is acceptable or not based on the length of the nails of the subject P by referring to the n first scores is not limited to the one described above. For example, the evaluation unit 222 may make a determination based on whether various statistical values such as the maximum value, minimum value, median value, etc. of the n first scores are equal to or greater than a threshold. For example, if a minimum value is used, in other words if there is even one nail with a bad nail length, it is determined to be rejected. Also, for example, when the maximum value is used, in other words, if there is even one nail with a good length, it is determined to be acceptable.

(Second evaluation process)
The evaluation unit 222 inputs each of the extracted n pieces of partial image data IMG-i to the second evaluation model 21b. As a result, a second score is output from the second evaluation model 21b. Here, the second score is an example of the second information indicating the evaluation result of the sanitary condition based on the nail color. The second score is a numerical value in the range of 0 or more and 1 or less, and indicates the possibility of good sanitary conditions based on nail color. That is, the closer the second score is to 1, the higher the possibility that the sanitary condition based on the nail color is good, and the closer the second score is to 0, the higher the possibility that it is not good. If the average value of the n second scores obtained from the n partial image data IMG-i is equal to or greater than a predetermined threshold, the evaluation unit 222 evaluates the sanitary condition based on the nail color of the subject P as " evaluated as "passed". On the other hand, if the average value of the second scores is less than the predetermined threshold value, the evaluation unit 222 evaluates the sanitary condition based on the nail color of the subject P as "failed". The evaluation unit 222 stores, in the storage unit 23, information indicating whether the nail color is acceptable in association with the subject's identification information.

In the second evaluation process, the method for determining whether the sanitary condition is acceptable based on the color of the nail of the subject P with reference to the n second scores is not limited to the above. For example, the evaluation unit 222 may make a determination based on whether various statistical values such as the maximum value, minimum value, median value, etc. of the n second scores are equal to or greater than a threshold. For example, if the minimum value is used, in other words, if there is even one nail with a poor nail color, it is determined to be rejected. Also, for example, when the maximum value is used, in other words, if there is even one nail with a good nail color, it is determined to be acceptable. Also, the execution order of the first evaluation process and the second evaluation process is random. In other words, the evaluation unit 222 may perform the second evaluation process after the first evaluation process, or may perform the first evaluation process after the second evaluation process. Evaluation processing may be performed in parallel.

(Generation processing of information indicating evaluation results)
Based on the first information and the second information, the evaluation unit 222 generates information indicating whether the user can enter or leave the room. For example, the evaluation unit 222 determines that (i) the number n of extracted partial images is 2 or more, and (ii) the hygiene evaluation result based on the nail length and the hygiene evaluation result based on the nail color. are both "passed", information is generated indicating that the subject P can enter and leave the room. In addition, when neither of (i) nor (ii) is satisfied, the evaluation unit 222 generates information indicating that the subject P cannot enter or leave the room. The evaluation unit 222 stores, in the storage unit 23, information indicating whether or not entry/exit is possible in association with the identification information of the subject P. FIG. Note that the evaluation unit 222 associates the identification information of the subject P with the evaluation date and time when the evaluation step M12 was performed, and stores the nail length as a record uniquely identified by the identification information of the subject P and the evaluation date and time. Information indicating acceptance/rejection, nail color acceptance/rejection, and entrance/exit acceptance/rejection may be stored. Here, "indoor" is the interior of any building, for example, any room within the building. The “building” is not particularly limited, but is preferably a food manufacturing site building (e.g., food processing factory, etc.) or a medical site building (e.g., hospital, etc.) where hand hygiene is required. .

From a sanitary point of view, the evaluation unit 222 preferably evaluates the sanitary conditions of both hands of the subject P based on the length and color of the nails. The image data of both hands of the subject P may be one image data, or may be individual image data obtained by imaging each hand.

From a hygiene point of view, the evaluation unit 222 evaluates the hygiene condition based on the nail length and color based on at least two fingernail regions (partial image data IMG-i) included in the image data. should be evaluated. Of the nail partial image data recognized as one nail by the object detection algorithm, two or more nail partial image data may be used as input data, and which finger nail partial image data is selected as input data. There is no particular limitation as to whether to When evaluating the sanitary conditions of both hands of the subject P based on the length and color of the nails, partial image data of two or more nails for each hand may be used as input data.

The output step M13 is a step of outputting information indicating the evaluation result of the nail hygiene evaluated in the evaluation step M12. In this output step M13, the information processing device 20a reads out the information indicating the evaluation result from the storage unit 23 and outputs it to the output device 30a to present it to the user.

FIG. 8 is a diagram showing an example of a screen output by the output device 30a included in the evaluation system 1a of FIG. A screen example 500 shown in FIG. 8 is typically a suitable screen example when the output device 30a is installed so that the subject P can visually recognize it. A screen example 500 includes information 501 indicating whether each subject is allowed to enter or leave. The information 501 indicating whether entry/exit is permitted or not is output after reading the information stored in the storage unit 23 by the evaluation unit 222 in the evaluation step M12. By visually recognizing the example screen 500, the subject P can easily confirm whether or not the subject P can enter or exit the room. Note that the example screen 500 may include other additional information in addition to the information 501 indicating whether or not to enter indoors. For example, when the example screen 500 includes information 501 indicating that indoors is not allowed to enter, the reasons for the evaluation of being not allowed (for example, nail length, nail color, etc.) are also output. good too.

FIG. 9 is a diagram showing an example of a screen output by the output device 30a included in the evaluation system 1a of FIG. A screen example 600 shown in FIG. 9 is typically a suitable screen example when the output device 30a is installed so that the manager S can visually recognize it. Screenshot 600 includes information 601 indicating evaluation results for each subject. The information 601 indicating the evaluation result includes information indicating the employee code, which is the identification information of the subject P, the date and time of the evaluation, the acceptance/rejection of the nail length, the acceptance/rejection of the nail color, and the entry/exit permission/rejection, respectively. . The information stored in the storage unit 23 by the evaluation unit 222 in the evaluation step M12 is read and output as these pieces of information. By visually recognizing the example screen 601, the administrator S can confirm the information indicating the evaluation result of the nail hygiene of each subject P in a list.

(Configuration of imaging device 10a)
The configuration of the imaging device 10a will be described with reference to FIG. FIG. 10 is a perspective view showing the configuration of an imaging device 10a included in the evaluation system 1a of FIG. The imaging device 10 a includes an imaging box 105 . In the imaging apparatus 10a, the imaging device 101 is installed inside the imaging box 105. As shown in FIG. The imaging box 105 has a bottom surface 103, a side surface 102 as a light shielding wall, and a top surface 104 on which the imaging device 101 is installed. The imaging device 10a is arranged in the three-dimensional coordinate space shown in FIG. The bottom surface 103 of the imaging box 105 of the imaging device 10a is arranged on the xy plane, and of the normal directions perpendicular to the xy plane, the direction toward the zenith is defined as the z-axis direction.

One of the four side surfaces 102 of the imaging box 105 has an opening 107 into which a part of the human body including nails of the subject P can be inserted. A side surface having this opening 107 is specifically referred to as a front surface 106 .

The front surface 106 is made of a translucent material so that the inside of the imaging box 105 can be seen from the outside of the imaging box 105 and the light irradiation inside the imaging box 105 is constant. Thereby, the subject P can insert his or her hand into the imaging box 105 while visually confirming the state inside the imaging box 105 from the outside of the imaging box 105 . The material forming the front surface 106 is not particularly limited as long as it is a translucent material. For example, it can be formed from a translucent material such as resin or glass. Here, the term “semi-transparent” means that the light environment inside the imaging box 105 is less likely to interfere with light from the outside during imaging, and that the subject P can see his or her hand inside the imaging box 105 from the outside of the imaging box 105. It is sufficient if it is transparent enough to be visually recognized.

The size of the opening 107 is not particularly limited, but from the viewpoint of preventing light from outside the imaging box 105 from entering the imaging box 105, the subject P's hand can be inserted into the imaging box 105. is preferably the smallest possible size. Further, it is preferable that the opening 107 is provided on the lower side of the front surface 106 so that the hand inserted into the inside of the imaging box 105 can be easily placed on the bottom surface 103 .

The bottom surface 103 functions as an alignment table that enables horizontal alignment of the hand of the subject P with respect to the imaging device 101 . The imaging device 101 is installed so that the imaging range includes the bottom surface 103 of the imaging box 105 as an alignment table. The bottom surface 103 preferably has a monochromatic surface facing the imaging device 101, and preferably has a color such as black or blue that has a high contrast with the color of the subject's P hand. By selecting a color having a large contrast with the color of the hand of the subject P as the color of the bottom surface 103, the information processing device 20a recognizes the nail in the "image data including the nail as a subject" generated by the imaging device 101. Accuracy can be improved.

The image pickup device 10a preferably includes a guide mechanism that enables photographing of a human body part including the nail while the adjacent nails are separated from each other. FIG. 11 is a top view showing an example of the configuration of the guide mechanism provided in the imaging apparatus 10a of FIG. (Modification 1) is described, and reference numeral 1102 describes a configuration (Modification 2) in which the bottom surface 103b of the imaging box 105 is provided with a hand-shaped guide line 109b as a guide mechanism.

In Modified Example 1, a plurality of protrusions 109a as a guide mechanism protrude from the bottom surface 103a of the imaging box 105 in the positive Z-axis direction. A plurality of protrusions 109a are arranged at predetermined intervals on the bottom surface 103a. After inserting the hand into the imaging box 105, the subject P should place the hand on the bottom surface 103a so that the protrusions 109a are positioned between the fingers. Since the imaging apparatus 10a is provided with a plurality of projections 109a as a guide mechanism, it is possible to guide the subject P to place his or her hand on the bottom surface 103a in a state suitable for imaging. It becomes possible to photograph parts of the human body including the nails while they are separated from each other. As a result, it is possible to improve the recognition accuracy of the nail by the information processing device 20a in the “image data including the nail as a subject” generated by the imaging device 101 .

When the guide mechanism is the hand-shaped guide wire 109b as in Modification 2, the subject P inserts his/her hand into the imaging box 105 and then opens the guide wire 109b to match the shape. A hand can be placed on the bottom surface 103b in this state. Also in Modification 2, as in Modification 1, the subject P can be guided to place his or her hand on the bottom surface 103b in a state suitable for imaging. It is possible to photograph parts of the human body, including fingernails. As a result, it is possible to improve the recognition accuracy of the nail by the information processing device 20a in the “image data including the nail as a subject” generated by the imaging device 101 . In addition, since the front surface 106 is made of a translucent material, the subject P can visually confirm the position of the hand in the imaging box 105 through the front surface 106 while placing the hand at an appropriate position on the bottom surface 103b. can be placed.

The configuration of the guide mechanism included in the imaging device 10a is not limited to the above-described modification. For example, the bottom surface 103 may be uneven, a rod-shaped object may be provided on a part of the surface of the bottom surface 103, or a pattern using a plurality of colors may be used. Further, for example, in the "image data including a nail as a subject" generated by the imaging device 10a, the fingers of the subject P are not sufficiently opened, and the information processing device 20a recognizes them as one nail. is not possible, the information processing device 20a causes the output device 30a to display a message prompting the subject P to place the hand again on the bottom surface 103 with the adjacent fingers separated from each other. can be configured to

The imaging device 10a includes a plurality of LEDs (light emitting diodes) 108 as illumination inside an imaging box 105 . Thereby, the entire surface of the bottom surface 103 can be uniformly irradiated with light. The intensity of the light emitted by the plurality of LEDs 108 may be appropriately adjusted so that the reflection of the light emitted from the nail does not become too strong and does not affect the evaluation accuracy of the sanitary condition based on the color of the nail.

(Configuration of machine learning device 40)
A configuration of the machine learning device 40 will be described with reference to FIG. The machine learning device 40 includes a storage section 41 and a control section 42 . The control unit 42 comprehensively controls the machine learning device 40 . In the machine learning device 40 , the control section 42 includes a constructing section 421 . The constructing unit 421 executes the machine learning method M2.

(Flow of machine learning method M2)
The flow of machine learning method M2 will be described with reference to FIG. FIG. 12 is a flow chart showing the flow of the machine learning method M2.

The machine learning method M2 includes a learning data set building step M21 and a learned model building step M22.

(Learning data set construction step M21)
The learning data set constructing step M21 is a step in which the processor 401 creates a first learning data set and a second learning data set.

(Processing for creating first learning data set)
The first learning data set is a set of a plurality of first learning data (teacher data) for constructing the first evaluation model 21a.

Each first learning data includes teacher image data including one nail as a subject and correct answer information. The teacher image data may be captured by the imaging device 10a, or may be captured by another imaging device. Further, the teacher image data may be partial image data extracted from image data including a plurality of nails as subjects. Further, the teacher image data may be public image data published on a network, or may be partial image data extracted from the public image data. The correct answer information is information indicating the evaluation result of the nail length by the evaluator.

In the learning data set construction step M21, the construction unit 421 acquires teacher image data and displays it on a display device (not shown). In addition, the constructing unit 421 acquires correct information indicating the evaluation result of the sanitary condition based on the nail length by the evaluator via an input device (not shown). For example, the evaluator may view the teacher image data displayed on the display device and input the evaluation result of "pass" or "fail" from the input device. Also, the evaluator may view the teacher image data displayed on the display device and input the score of the sanitary condition from the input device.

The construction unit 421 also generates first learning data including teacher image data and correct answer information. The construction unit 421 also stores the generated first learning data in the storage unit 41 . The construction unit 421 repeats the above process for a plurality of pieces of teacher image data to create the first learning data set.

(Processing for creating second learning data set)
The second learning data set is a set of a plurality of second learning data (teacher data) for constructing the second evaluation model 21b.

Each second learning data includes teacher image data including one nail as a subject and correct answer information. The teacher image data is as described in the process of creating the first learning data set. Part or all of the teacher data used in the process of creating the second learning data set may be the same as that used in creating the first learning data set. The correct answer information is information indicating the evaluation result of the nail color by the evaluator.

In the learning data set construction step M21, the construction unit 421 acquires teacher image data and displays it on a display device (not shown). In addition, the construction unit 421 acquires correct information indicating the evaluation result of the sanitary condition based on the nail color by the evaluator via an input device (not shown). A specific example of the correct answer information input method by the evaluator is as described in the first learning data set creation process.

The construction unit 421 also generates second learning data including teacher image data and correct answer information. The construction unit 421 also stores the generated second learning data in the storage unit 41 . The construction unit 421 repeats the above process for a plurality of pieces of teacher image data to create a second learning data set.

(Learned model construction step M22)
The learned model construction step M22 is a step in which the construction unit 421 constructs the first evaluation model 21a and the second evaluation model 21b. In the trained model construction step M22, the construction unit 421 constructs the first evaluation model 21a by supervised learning using the first learning data set. Also, the construction unit 421 constructs the second evaluation model 21b by supervised learning using the second learning data set. The constructed first evaluation model 21 a and second evaluation model 21 b are stored in the storage unit 41 .

(Other Aspect 1 of Embodiment 2)
As the second embodiment, the number of evaluation models used by the evaluation unit 222 to evaluate the hygiene state of the nails is two. However, the number of evaluation models used by the evaluation unit 222 is not limited to two. For example, the evaluation unit 222 may evaluate the nail hygiene using either one of the first evaluation model 21a and the second evaluation model 21b. In addition to the first evaluation model 21a and the second evaluation model 21b, the evaluation unit 222 may evaluate the nail hygiene using another evaluation model. In this case, the other evaluation model is constructed by machine learning so as to input image data including nails as an object and output information indicating the evaluation result of the sanitary condition based on another viewpoint of the nails.

(Other Aspect 2 of Embodiment 2)
As the second embodiment, the input of the first evaluation model 21a and the second evaluation model 21b is the partial image data IMG-i including one nail as an object. However, the input of the first evaluation model 21a and the second evaluation model 21b may be image data H including two or more fingernails as subjects.

In this case, the construction unit 421 performs first learning including image data for learning including a plurality of nails as subjects, and correct information indicating the evaluation result of the evaluator's evaluation of the sanitary condition based on the length of the plurality of nails. Generate data for Thereby, the first evaluation model 21a is machine-learned using the first learning data set composed of such first learning data. The construction unit 421 also includes second learning data including learning image data including a plurality of nails as subjects, and correct information indicating an evaluation result of an evaluator's evaluation of the sanitary condition based on the colors of the plurality of nails. to generate Thereby, the second evaluation model 21b is machine-learned using the second learning data set composed of such second learning data.

In this case, the evaluation unit 222 omits the process of extracting the partial image data in the evaluation step M12, and uses the image data H acquired by the acquisition unit 221 as the first evaluation model 21a and the second evaluation model 21b. can be entered in

When the input of the first evaluation model 21a and the second evaluation model 21b is the partial image data IMG-i, the first evaluation model 21a and the second evaluation model 21a and the second evaluation model 21a are obtained using a relatively small amount of image data for learning. It is possible to construct the model 21b with high accuracy. On the other hand, when the input of the first evaluation model 21a and the second evaluation model 21b is the image data H, there is an advantage that the processing cost of the evaluation unit 222 can be reduced.

(Other Aspect 3 of Embodiment 2)
In Embodiment 2, the evaluation system 1a may include a subject terminal, a subject-side output device, an administrator terminal, and an administrator-side output device instead of the output device 30a. In this case, the information processing device 20a is communicably connected to each of the subject terminal and the administrator terminal via a network. For example, the subject terminal, the subject side output device, and the imaging device 10a are installed near the entrance of a food factory. Also, for example, the administrator terminal and the administrator's output device are installed in the administrator's office.

In this aspect, in the output step M13, the evaluation unit 222 outputs information for the subject as shown in FIG. 8 to the output device on the side of the subject, and outputs information for the administrator as shown in FIG. You may output to the output device of the administrator side. Further, the evaluation unit 222 executes the process of outputting the information for the administrator to the output device of the administrator in response to the administrator's request instead of or in addition to the execution in the output step M13. good too. Requests by the administrator are input through an input device (not shown) of the administrator terminal.

(Other Aspect 4 of Embodiment 2)
As the second embodiment, a mode in which the first evaluation model 21a and the second evaluation model 21b are stored in the storage unit 23 of the information processing device 20a has been described. However, the information processing device 20a may not include the first evaluation model 21a and the second evaluation model 21b. For example, the first evaluation model 21a and the second evaluation model 21b may be built in another computer (for example, a computer constituting a cloud server) connected to the information processing device 20a via a network. In this case, the evaluation unit 222 of the information processing device 20a uses the first evaluation model 21a and the second evaluation model 21b via the network to generate the hygiene evaluation result based on the nail length and color. do it.

(Effect of Embodiment 2)
This embodiment uses image data as an input to evaluate nail hygiene using AI (artificial intelligence), so image analysis is not required, and nail hygiene is evaluated based on the analysis results. It has the advantage that it is not necessary to set evaluation rules in advance. This can be said to be the same effect as the evaluation system 1 of the first embodiment using AI. In addition, in the present embodiment, the evaluation unit 222 evaluates the sanitary condition based on the length and color of the nails. Accurate and multifaceted evaluation is possible. Furthermore, in the present embodiment, the evaluation unit 222 uses two types of evaluation models, the first evaluation model 21a and the second evaluation model 21b. It is possible to evaluate the nail hygiene condition more accurately than in the case of evaluating the hygiene condition based on.

(Hardware Configuration Example of Information Processing Device 20a)
The information processing device 20a may have a hardware configuration as shown in FIG. 13, for example. FIG. 13 is a block diagram showing an example of the hardware configuration of the information processing device 20a.

The information processing device 20 a is implemented using a general-purpose computer, and includes a processor 201 , a primary memory 202 , a secondary memory 203 , an input/output interface 204 , a communication interface 205 and a bus 206 . Processor 201 , primary memory 202 , secondary memory 203 , input/output interface 204 and communication interface 205 are interconnected via bus 206 .

The secondary memory 203 stores an information processing program and one or more evaluation models. The secondary memory 203 is an example of a configuration that implements the storage unit 23 . The processor 201 loads the information processing program and one or more evaluation models stored in the secondary memory 203 onto the primary memory 202 . The processor 201 functions as the control unit 22 by executing each step included in the information processing method M1 according to instructions included in the information processing program developed on the primary memory 202 . One or more evaluation models (for example, the first evaluation model 21a and the second evaluation model 21b) developed on the primary memory 202 are used when the processor 201 executes the evaluation step M12 of the information processing method M1. be done. The fact that the information processing program is stored in the secondary memory 203 means that the source code or an executable file obtained by compiling the source code is stored in the secondary memory 203 . Further, the statement that one or more evaluation models are stored in the secondary memory 203 means that the parameters defining the one or more evaluation models are stored in the secondary memory 203 .

Devices that can be used as the processor 201 include, for example, CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating point number Processing Unit), PPU (Physics Processing Unit), a microcontroller, or a combination thereof.

A device that can be used as the primary memory 202 is, for example, a RAM (Random Access Memory). Primary memory 202 is sometimes referred to as "main memory." Devices that can be used as the secondary memory 203 include, for example, flash memories, HDDs (Hard Disk Drives), SSDs (Solid State Drives), ODDs (Optical Disk Drives), and combinations thereof. . Secondary memory 203 is sometimes called an "auxiliary storage device." The secondary memory 203 may be built in the information processing device 20a, or may be another computer connected to the information processing device 20a via the input/output interface 204 or the communication interface 205 (for example, a cloud server). computer). In this embodiment, the storage in the information processing device 20a is implemented by two memories (primary memory 202 and secondary memory 203), but the present invention is not limited to this. That is, the storage in the information processing device 20a may be realized by one memory. In this case, for example, a certain storage area of the memory may be used as the primary memory 202 and another storage area of the memory may be used as the secondary memory 203 .

An input device and/or an output device are connected to the input/output interface 204 . Examples of the input/output interface 204 include interfaces such as USB (Universal Serial Bus), ATA (Advanced Technology Attachment), SCSI (Small Computer System Interface), and PCI (Peripheral Component Interconnect). An example of an input device connected to the input/output interface 204 is the imaging device 10a. Image data acquired from the imaging device 10a in the information processing method M1 is stored in the primary memory 202 . Input devices connected to the input/output interface 204 include, for example, a keyboard, mouse, touch pad, microphone, or a combination thereof. Output devices connected to the input/output interface 204 include, for example, an output device 30a (display, projector, printer, speaker, headphone, or a combination thereof). Information provided to the user in the information processing method M1 is output from the information processing apparatus 20a via these output devices. The information processing apparatus 20a may incorporate a keyboard functioning as an input device and a display functioning as an output device, like a laptop computer. Alternatively, the information processing device 20a may incorporate a touch panel that functions as both an input device and an output device, like a tablet computer.

Other computers are wired or wirelessly connected to the communication interface 205 via a network. Examples of the communication interface 205 include interfaces such as Ethernet (registered trademark) and Wi-Fi (registered trademark). Available networks include, for example, PAN (Personal Area Network), LAN (Local Area Network), CAN (Campus Area Network), MAN (Metropolitan Area Network), WAN (Wide Area Network), and GAN (Global Area Network). , or an internetwork comprising these networks. An internetwork may be an intranet, an extranet, or the Internet. Data (e.g., one or a plurality of evaluation models) acquired by the information processing device 20a from another computer (e.g., machine learning device 40) in the information processing method M1, and Data to be provided to computers is sent and received via these networks.

In this embodiment, a single processor (processor 201) is used to execute the information processing method M1, but the present invention is not limited to this. That is, a configuration in which the information processing method M1 is executed using a plurality of processors may be employed. For example, the obtaining step M11 may be performed in a first processor and the evaluating step M12 may be performed in a second processor. In this case, the plurality of processors that cooperate to execute the information processing method M1 may be provided in a single computer and configured to communicate with each other via a bus, or may be distributed among a plurality of computers. may be provided and configured to be able to communicate with each other via a network. As an example, a processor built in a computer that constitutes a cloud server and a processor built in a computer owned by a user of the cloud server cooperate to execute the information processing method M1.

Further, in this embodiment, a configuration is adopted in which the evaluation model is stored in a memory (secondary memory 203) built in the same computer as the processor (processor 201) that executes the information processing method M1. is not limited to this. That is, a configuration may be adopted in which one or more evaluation models are stored in a memory incorporated in a computer different from the processor that executes the information processing method M1. In this case, a computer having a built-in memory storing one or more evaluation models and a computer having a built-in processor executing the information processing method M1 are configured to communicate with each other via a network. As an example, one or more evaluation models are stored in a memory built into a computer that constitutes a cloud server, and a processor built into a computer owned by a user of the cloud server executes the information processing method M1. can be considered.

In addition, although this embodiment adopts a configuration in which one or more evaluation models are stored in a single memory (secondary memory 203), the present invention is not limited to this. That is, a configuration in which one or a plurality of evaluation models are distributed and stored in a plurality of memories may be employed. In this case, a plurality of memories storing one or more evaluation models are stored in a single computer (which may or may not be a computer containing a processor that executes the information processing method M1). It may be provided, or it may be distributed and provided in a plurality of computers (which may or may not include a computer in which a processor that executes the information processing method M1 is included). . As an example, a configuration is conceivable in which one or more evaluation models are distributed and stored in a memory built into each of a plurality of computers that constitute a cloud server.

(Hardware configuration of machine learning device 40)
The machine learning device 40 may have a hardware configuration as shown in FIG. 14, for example. FIG. 14 is a block diagram showing an example of the hardware configuration of the machine learning device 40. As shown in FIG.

Machine learning device 40 is implemented using a general-purpose computer, and includes processor 401 , primary memory 402 , secondary memory 403 , input/output interface 404 , communication interface 405 , and bus 406 . Processor 401 , primary memory 402 , secondary memory 403 , input/output interface 404 and communication interface 405 are interconnected via bus 406 .

A secondary memory 403 stores a machine learning program and a learning data set. The secondary memory 403 is an example of a configuration that implements the storage unit 41 . A learning data set is a set of multiple pieces of learning data. The processor 401 expands the machine learning program stored in the secondary memory 403 onto the primary memory 402 . The processor 401 functions as the control unit 42 by executing each step included in the machine learning method M2 according to instructions included in the machine learning program expanded on the primary memory 402 . The learning data set stored in the secondary memory 403 is created in the learning data set building step M21 of the machine learning method M2, and used in the learned model building step M22 of the machine learning method M2. The secondary memory 403 also stores one or more evaluation models constructed in the learned model construction step M22 of the machine learning method M2. Note that storing the machine learning program in the secondary memory 403 means that the source code or an executable file obtained by compiling the source code is stored in the secondary memory 403 . Further, the statement that one or more evaluation models are stored in the secondary memory 403 means that the parameters defining one or more evaluation models are stored in the secondary memory 403 .

Specific examples of devices that can be used as the processor 401, the primary memory 402, or the secondary memory 403 are as described in the hardware configuration of the information processing apparatus 20a. Note that the secondary memory 403 may be built in the machine learning device 40, or may be another computer connected to the machine learning device 40 via the input/output interface 404 or the communication interface 405 (for example, a cloud server). computer). Note that in the present embodiment, storage in the machine learning device 40 is realized by two memories (the primary memory 402 and the secondary memory 403), but it is not limited to this. That is, the storage in the machine learning device 40 may be realized by one memory. In this case, for example, a certain storage area of the memory may be used as the primary memory 402 and another storage area of the memory may be used as the secondary memory 403 .

An input device and/or an output device are connected to the input/output interface 404 . A specific example of the input/output interface 404 is as described in the description of the hardware configuration of the information processing device 20a. An example of an input device connected to the input/output interface 404 is the imaging device 10a. Data acquired from the imaging device 10 a in the machine learning method M2 is input to the machine learning device 40 and stored in the primary memory 402 . A specific example of the input device connected to the input/output interface 404 is as described in the description of the hardware configuration of the information processing apparatus 20a. Data obtained from the user in machine learning method M2 is input to machine learning apparatus 40 via these input devices and stored in primary memory 402 . A specific example of the output device connected to the input/output interface 404 is as described in the description of the hardware configuration of the information processing apparatus 20a. Information provided to the user in the machine learning method M2 is output from the machine learning device 40 via these output devices. Machine learning device 40 may incorporate a keyboard functioning as an input device and a display functioning as an output device, like a laptop computer. Alternatively, the machine learning device 40 may incorporate a touch panel that functions as both an input device and an output device, like a tablet computer.

Other computers are wired or wirelessly connected to the communication interface 405 via a network. A specific example of the communication interface 405 is as described in the description of the hardware configuration of the information processing device 20a. An internetwork may be an intranet, an extranet, or the Internet. Data (eg, one or more evaluation models) provided by the machine learning device 40 to another computer (eg, the information processing device 20a) is transmitted and received via these networks.

Note that although the present embodiment employs a configuration in which a single processor (processor 401) is used to execute the machine learning method M2, the present invention is not limited to this. That is, a configuration may be adopted in which a plurality of processors are used to execute the machine learning method M2. For example, a configuration may be employed in which the learning data set constructing step M21 is executed by a first processor, and the learned model constructing step M22 is executed by a second processor. In this case, the plurality of processors that cooperate to execute the machine learning method M2 may be provided in a single computer and configured to be able to communicate with each other via a bus, or distributed to a plurality of computers. may be provided and configured to be able to communicate with each other via a network. As an example, a mode in which a processor built in a computer that constitutes a cloud server and a processor built in a computer owned by a user of the cloud server cooperate to execute the machine learning method M2 can be considered.

Further, in this embodiment, a configuration is adopted in which the learning data set is stored in a memory (secondary memory 403) built in the same computer as the processor (processor 401) that executes the machine learning method M2. The invention is not limited to this. That is, a configuration may be adopted in which the learning data set is stored in a memory built in a computer different from the processor that executes the machine learning method M2. In this case, the computer containing the memory storing the learning data set is configured to be able to communicate with the computer containing the processor executing the machine learning method M2 via the network. As an example, a learning data set is stored in a memory built into a computer that constitutes a cloud server, and a processor built into a computer owned by a user of the cloud server executes the machine learning method M2. be done.

Further, in this embodiment, a configuration is adopted in which the learning data set is stored in a single memory (secondary memory 403), but the present invention is not limited to this. That is, a configuration may be adopted in which the learning data sets are distributed and stored in a plurality of memories. In this case, the plurality of memories storing the learning data sets are provided in a single computer (which may or may not be a computer with a built-in processor that executes the machine learning method M2). Alternatively, it may be distributed among a plurality of computers (which may or may not include a computer with a processor that executes the machine learning method M2). As an example, a configuration is conceivable in which learning data sets are distributed and stored in memories built into each of a plurality of computers that constitute a cloud server.

In addition, although the present embodiment adopts a configuration in which the information processing method M1 and the machine learning method M2 are executed using different processors (the processor 201 and the processor 401), the present invention is not limited to this. That is, the information processing method M1 and the machine learning method M2 may be executed using the same processor. In this case, one or more evaluation models are stored in the memory contained in the same computer as this processor by executing the machine learning method M2. This processor will use one or more evaluation models stored in this memory when executing the information processing method M1.

[Embodiment 3]
(Configuration of evaluation system 1b)
An evaluation system 1b according to Embodiment 3 of the present invention will be described with reference to FIG. FIG. 15 is a block diagram showing the configuration of the evaluation system 1b. The evaluation system 1a includes, as shown in FIG. 15, an imaging device 10a, an information processing device 20b, and an output device 30a. The imaging device 10a and the output device 30a are as described in the second embodiment.

The information processing device 20b is a device for executing the information processing method M1b. The information processing apparatus 20b is different from the second embodiment at least in that it evaluates the nail hygiene without using an evaluation model constructed by machine learning.

(Configuration of information processing device 20b)
The configuration of the information processing device 20b will be described with reference to FIG. The information processing device 20b includes a control section 22b and a storage section 23b. The control unit 22b comprehensively controls the information processing device 20b. In the information processing device 20b, the control unit 22b includes an acquisition unit 221 and an evaluation unit 222b. Acquisition unit 221 executes acquisition step M11 of information processing method M1b. The evaluation unit 222b executes the evaluation step M12b and the output step M13 of the information processing method M1b.

(Flow of information processing method M1b)
The information processing method M1b differs from the information processing method M1 according to the second embodiment in that an evaluation step M12b is included instead of the evaluation step M12. Only the difference will be described below.

(Evaluation step M12b)
In the evaluation step M12b, the evaluation unit 222b measures the length of the nail in the image data including the nail as the subject acquired by the acquisition unit 221 using a known image analysis technique. In addition, the evaluation unit 222b refers to the information indicating the correspondence relationship between the length of the nail and the sanitary condition, which is stored in advance in the storage unit 23b. is evaluated, and data indicating the evaluation result is written in the storage unit 23b. For example, the evaluation unit 222 may measure the relative length of the tip of the nail with respect to the length of the entire nail area in the image data including the nail as the subject, as the “length of the nail”. In this case, the storage unit 23b stores information indicating the correspondence between the relative nail length and the sanitary condition.

Similarly, the evaluation unit 222b uses a known image analysis technique to measure the color of the nail in the image data obtained by the acquisition unit 221 and including the nail as an object. By referring to the information indicating the correspondence relationship between the nail color and the hygiene condition stored in advance in the storage unit 23b, the hygiene condition based on the nail color is evaluated from the measurement result of the nail color, and the evaluation result is indicated. Data is written in the storage unit 23b.

(Effect of Embodiment 3)
In this embodiment, the information processing device 20b does not require a machine learning device because the information processing device 20b evaluates nail hygiene without using an evaluation model constructed by machine learning. Therefore, there is an advantage that the construction cost of the evaluation system 1b can be reduced.

[Example of hardware implementation]
In each of the above-described embodiments, the control blocks of the information processing devices 20, 20a and 20b (especially the control units 22 and 22b) are realized by reading the programs stored in the secondary memory 203 and executing them by the processor 201. Aspects have been described. Not limited to this, part or all of the control blocks of the information processing devices 20, 20a and 20b may be realized by logic circuits (hardware) formed in integrated circuits (IC chips) or the like. Also, in the second embodiment, the control block (especially the control unit 42) of the machine learning device 40 reads the program stored in the secondary memory 403 and the processor 401 executes it. Not limited to this, some or all of the control blocks of the machine learning device 40 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various modifications or improvements can be made to the above embodiments. Also, it is obvious to those skilled in the art that such changes or improvements can be made. In addition, it is clear from the description of the scope of the claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

〔summary〕
The present invention
(1) An acquisition unit that acquires image data including a nail as an object, and generates information indicating an evaluation result by referring to the image data and evaluating the sanitary condition of the nail based on the length of the nail. an information processing device comprising an evaluation unit,
(2) The information processing device according to (1), wherein the evaluation unit generates information including whether or not the person with the fingernail can enter or leave the room as the information indicating the evaluation result.
(3) The information processing device of (2), wherein the indoor is a room in a food factory or a food processing plant,
(4) Any one of (1) to (3), wherein the evaluation unit evaluates the sanitary condition of the nail based on at least two fingernail regions included in the image data. two information processing devices,
(5) The information processing device according to any one of (1) to (4), wherein the evaluation unit further evaluates the sanitary condition of the nail based on the color of the nail with reference to the image data;
(6) using a first evaluation model constructed by machine learning so as to output first information indicating the evaluation result of the sanitary condition based on the length of the nails, with the image data as input; , the information processing device according to any one of (1) to (5), which generates information indicating the evaluation result based on the first information,
(7) using a second evaluation model constructed by machine learning so as to output second information indicating the evaluation result of the hygiene state based on the color of the nail, with the image data as input; The information processing device according to (6), which generates information indicating the evaluation result based on the first information and the second information,
(8) The information processing device according to any one of (1) to (7), an imaging device that images a part of the human body including the nail and generates the image data, and outputs information indicating the evaluation result. an output device; and a rating system, including:
(9) The imaging device includes an imaging device, an alignment table that enables alignment of the part with respect to the imaging device, and a light shielding wall that blocks light entering the alignment table from the outside. , the imaging equipment is installed so as to include the alignment table in an imaging range, the alignment table has a monochromatic surface facing the imaging equipment, and the light shielding wall is installed around the alignment table. The evaluation system of (8),
(10) The imaging device is installed inside an imaging box, and the imaging box has a bottom surface as the alignment table, a side surface as the light shielding wall, and a top surface on which the imaging device is installed. The evaluation system of (9), wherein the side surface has an opening that allows the site to be inserted inside the imaging box,
(11) The evaluation system of (9) or (10), wherein the imaging device includes a guide mechanism that enables imaging of the site with the adjacent claws separated from each other;
(12) By machine learning using the learning data set, the building unit is provided for building a first evaluation model that evaluates the hygiene state based on the length of the nail, and the input of the first evaluation model is a machine learning device, wherein the image data includes a nail as an object, and the output of the first evaluation model is first information indicating the evaluation result of the sanitary condition based on the length of the nail;
(13) The construction unit further constructs a second evaluation model that evaluates the sanitary condition based on the color of the nail by machine learning using the learning data set, and the input of the second evaluation model is the nail as a subject, and the output of the second evaluation model is the second information indicating the evaluation result of the hygiene state based on the color of the nail, the machine learning device of (12),
(14) An obtaining step of obtaining image data including a nail as an object, and referring to the image data to evaluate the sanitary condition of the nail based on the length of the nail, thereby generating information indicating an evaluation result. an information processing method comprising an evaluation step;
(15) A program for causing a computer to function as an information processing device according to any one of (1) to (7),
is.

1 Nail condition inspection device (evaluation system)
1a, 1b evaluation system 10 (10') imaging device (imaging terminal)
10a imaging device (imaging terminal)
20 Arithmetic device (information processing device)
20a, 20b information processing device 21 trained neural network 21a first evaluation model 21b second evaluation model 22, 22b control unit 23, 23b storage unit 30 (30') input/output device (input/output terminal) (output device)
30a output device 40 machine learning device 101 imaging device (imaging device)
102 light shielding wall (side)
103, 103a, 103b bottom (alignment table)
104 top surface 105 imaging box 106 front surface 107 opening 108 LED
109a Protrusion (guide mechanism)
109b guide wire (guide mechanism)
201 processor 202 primary memory 203 secondary memory 204 input/output interface 205 communication interface 206 bus 221 acquisition unit 222, 222b evaluation unit 401 processor 402 primary memory 403 secondary memory 404 input/output interface 405 communication interface 406 bus 500 screen example 501 indoors 600 Screen example 601 Information indicating that entry to the room is prohibited R Room (indoor)
H Hand image P Examinee S Administrator

Claims (12)

an acquisition unit that acquires image data including a nail as an object;
an evaluation unit that generates information indicating an evaluation result by referring to the image data and evaluating the sanitary condition of the nail based on the length of the nail;
with
The evaluation unit outputs information indicating whether the hygiene condition of the nails based on the length of the nails satisfies a standard, included in the information indicating the evaluation results,
The evaluation unit extracts partial image data of each nail from the image data, and when the number of extracted partial image data is n or more, continues the processing by referring to the n or more partial image data. and, if the number is less than n, the processing is interrupted, and n is set to 2 or more and less than 5 for one hand . 2. The information processing apparatus according to claim 1, wherein said evaluation unit generates, as information indicating said evaluation result, information including whether or not said person having nails can enter and exit indoors. 3. The information processing apparatus according to claim 2, wherein said indoor space is a room of a food factory or a food processing plant. The information processing apparatus according to any one of claims 1 to 3 , wherein the evaluation unit further evaluates the sanitary condition of the nail based on the color of the nail by referring to the image data. Using a first evaluation model constructed by machine learning so as to output first information indicating the evaluation result of the sanitary condition based on the length of the nail with the image data as input,
The information processing apparatus according to any one of claims 1 to 4 , wherein said evaluation unit generates information indicating said evaluation result based on said first information. Using a second evaluation model constructed by machine learning so as to output second information indicating the evaluation result of the hygiene state based on the color of the nail with the image data as input,
6. The information processing apparatus according to claim 5 , wherein said evaluation unit generates information indicating said evaluation result based on said first information and said second information. an information processing apparatus according to any one of claims 1 to 6 ;
an imaging device that images a part of a human body including the nail and generates the image data;
an output device that outputs information indicating the evaluation result;
Rating system including. The imaging device is
an imaging device;
an alignment table that enables alignment of the region with respect to the imaging device;
a light blocking wall that blocks light entering the alignment table from the outside;
and
The imaging device is installed so as to include the alignment table in an imaging range,
the alignment table has a monochromatic surface facing the imaging device;
8. The evaluation system according to claim 7 , wherein said light shielding wall is installed around said alignment table. The imaging device is installed inside an imaging box,
The imaging box is
a bottom surface as the alignment table;
a side surface as the light shielding wall;
a top surface on which the imaging device is installed,
9. The evaluation system according to claim 8 , wherein said side surface has an opening through which said site can be inserted inside said imaging box. 10. The evaluation system according to claim 8 or 9 , wherein said imaging device comprises a guide mechanism capable of imaging said part with adjacent claws separated from each other. An information processing method executed by a computer including a processor,
an acquisition step in which the processor acquires image data including a nail as an object;
an evaluation step in which the processor refers to the image data and evaluates the hygiene of the nail based on the length of the nail to generate information indicative of an evaluation result;
including
In the evaluation step, the processor outputs information indicating whether the nail hygiene based on the length of the nail satisfies the standard in the information indicating the evaluation result , and
In the evaluation step, the processor extracts partial image data of each nail from the image data, and when the number of extracted partial image data is n or more, refers to the n or more partial image data. An information processing method that continues processing, interrupts processing when the number is less than n, and sets n to 2 or more and less than 5 for one hand . A program for causing a computer to function as the information processing apparatus according to any one of claims 1 to 6 .

Images