JP6644281B2 - True / false judgment system - Google Patents

Description

  The present invention relates to a correctness determination system capable of improving the accuracy of the accuracy determination performed by the computer.

  At present, there is known an inspection apparatus which provides a learning model to a computer having artificial intelligence and improves the accuracy of judgment and prediction made by the computer through machine learning.

  For example, an image processing apparatus disclosed in Patent Document 1 below includes a user input unit, an image feature label learning unit, an image feature label learning parameter storage unit, an image feature amount extraction unit, a lesion suspected area malignancy learning unit, The system includes a suspected lesion area grade estimation parameter storage unit, a suspected lesion area grade estimation unit, an image feature label learning update unit, a suspected lesion area grade learning update unit, and a display unit.

  The image feature label learning unit receives information on the suspected lesion area image and the corresponding image feature label from the medical image DB, performs machine learning (CNN method) for classifying the image feature label, and generates a learning parameter thereof. . Thereafter, the image feature amount extracting unit receives the diagnostic image and the lesion suspicious area image, receives image feature label learning parameters further from the image feature label learning parameter storage unit, the input of the lesion suspicious area image using the CNN network Extract image features. Finally, the display unit receives the diagnostic image, the suspected lesion area image, and the corresponding suspected lesion malignancy, and displays the result as the diagnosis result of the image processing apparatus.

  In addition, the lesion suspected area malignancy learning update unit receives the correct / incorrect information of the malignancy corresponding to the displayed lesion suspected area image input by the user from the user input unit, and calculates a new suspected area malignancy estimation parameter. Perform machine learning (re-learning). Thus, the suspected lesion area grade estimating unit re-estimates the grade of the suspected lesion area regarding the suspected lesion area image using the suspected lesion area grade estimation parameter updated by the suspected lesion area learning update unit. (Paragraphs 0019, 0029 to 0035, FIGS. 1 and 9).

JP 2018-61771 A

  However, in the image processing apparatus disclosed in Patent Literature 1, input of correct / incorrect information is complicated for the user, and can be used without the user inputting correct / incorrect information. There is. In that case, the image processing apparatus has a problem that machine learning based on the user's correctness information is not performed, and the accuracy of correctness determination does not improve.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a true / false determination system that can improve the accuracy of a true / false determination performed by a computer.

A first invention is a right / wrong judgment system having a judgment device that makes a predetermined judgment when input information is given to a computer that performs machine learning and answers the result, wherein the judgment device displays various information. A display unit, a determination unit for determining correctness, a storage unit for storing a device determination result of the determination unit, and a fee calculation unit for calculating a usage fee for the determination device, wherein the display unit includes the device Yes and apparatus determination result display screen for displaying the determination result, the user determination result input screen for inputting the user determination result performed by the user, and a basis display screen for the user to obtain a rationale to the device determination result The determination unit determines whether the device determination result matches the user determination result, and the storage unit determines whether or not the device determination result matches the user determination result. Stored, the charge calculation unit, by the correctness of grounds of the apparatus determination result obtained by the user, to change the weighting for determining the charge for the determination device, before SL-size constant information stored in the storage unit Based on the above, the rate of increase in the usage fee of the determination device is reduced or reduced.

  In the present invention, the result of the device determination performed by the determining device is displayed on the device determination result display screen. Further, when the user inputs the user determination result to the user determination result input screen, the determination unit determines whether the device determination result matches the user determination result. It can be determined whether or not they match. The determination information on the match or mismatch between the device determination result and the determination result is stored in the storage unit of the determination device.

Then, the fee calculation unit of the determination device, for example, reduces the price increase of the usage fee of the determination device (including zero increase) or reduces the price based on the mismatch determination information. Therefore, if the user cannot agree with the device determination result displayed on the display unit, the user is provided with an incentive to actively input a mismatched user determination result. As described above, according to the right / wrong judgment system of the present invention, the judgment device can surely perform machine learning based on the judgment information, so that the accuracy of right / wrong judgment can be improved.
Basically, the charge calculation unit of the determination device reduces the price increase (including zero increase) or decreases the price increase of the usage charge of the determination device based on the determination information, regardless of the match or the mismatch. For example, if the user lowers the usage fee as the user inputs the user determination result, the user is provided with an incentive to actively input the user fee even when the user agrees with the device determination result. The determination device obtains determination information by a user input, and machine learning proceeds based on the determination information, so that the determination accuracy can be improved.
Further, the display unit has a judgment basis display screen, and the user can confirm the basis of the apparatus judgment result performed by the judgment apparatus. For this reason, it is possible to easily grasp whether or not the device determination result is appropriate, and to take an inappropriate determination, for example, adopt the user determination result.
Further, the user confirms the correctness of the basis of the device determination result from the determination basis display screen, and if the basis of the device determination result is the same as the actual (correct), the determination of the determination device is excellent, and the device is excellent. If the basis of the judgment result is different from the actual one (erroneous), even if the judgment of the match or the mismatch is correct, the evaluation is not performed. Based on such information, by changing the weighting fee calculation unit determines the charge for determination device can determine the appropriate usage fee.

  In the true / false determination system of the first invention, it is preferable that the determination unit determines a match when the user does not input the user determination result.

  If the user agrees with the device determination result displayed on the display unit, the user is not given an incentive to input because the user fee cannot be reduced. However, in this type of right / wrong determination system, it is desirable for the user that the determination accuracy is high, and therefore, there is a low possibility that a false user determination result that reduces the determination accuracy is intentionally input. Therefore, when the user determination result is not input, the determination information can be processed by positively determining that the device determination result and the user determination result match, regardless of whether they match or not. Accordingly, the determination device can perform machine learning based on the determination information even when the user determination result is not input.

  Further, in the true / false determination system of the first invention, it is preferable that the display unit has a determination threshold setting screen in which a user sets a threshold for determining a determination criterion performed by the determination device.

  According to this configuration, the user can set the threshold value of the judgment performed by the judgment device on the judgment threshold value input screen. The user can set the judgment strictly or loosely according to the level of the threshold, and use it properly according to the purpose.

  Further, in the right / wrong judgment system of the first invention, it is preferable that the display unit has a factor entry screen in which a user enters a factor for which the judgment made by the judgment device is a match or a mismatch.

  According to this configuration, the user receives the match / mismatch determination information and enters the factor from the factor entry screen. As a result, the determination result data relating to the current true / false determination is reinforced, so that machine learning can be performed based on the determination result, and the determination accuracy of the determination device can be improved.

According to a second aspect of the present invention, when input information is given to a computer that performs machine learning, a right / wrong determination is made of a determining device that makes a predetermined determination and answers the result and a fee calculating device that calculates a usage fee of the determining device In the system, the determination device, a display unit that displays various information, a determination unit that determines the correctness, and a storage unit that stores the device determination result of the determination unit, the display unit, A device determination result display screen for displaying the device determination result, a user determination result input screen for inputting a user determination result performed by a user, and a determination ground display screen for a user to obtain the basis for the device determination result. The determination unit determines whether the device determination result matches the user determination result, and the storage unit determines whether the device determination result matches the user determination result. Storing information, the price calculation device, the correctness of grounds of the apparatus determination result obtained by the user, to change the weighting for determining the charge for the determination device, provided from the storage unit of the determining device based on the prior SL-size constant information, characterized in that it or reduce markdown price increases the width of the usage fee of the judgment device.

  In the true / false determination system of the present invention, since the fee calculation device is configured separately from the determination device, the determination unit of the determination device determines a match, the determination information is stored in the storage unit, and the determination information is stored in the fee calculation device. Provided to

The fee calculation device calculates the usage fee of the determination device based on the mismatch determination information, for example, but reduces the price increase of the usage fee (including the increase of zero) or reduces the price. At this time, if the user cannot agree with the device determination result, the user has an incentive to actively input a mismatched user determination result. Accordingly, the correctness / incorrectness determination system of the present invention allows the determination device to reliably perform machine learning based on the determination information, thereby improving the accuracy of correctness determination.

Basically, the fee calculation device reduces or increases the price increase (including zero increase) of the usage fee of the determination device based on the determination information irrespective of the match or mismatch provided by the determination device. For example, if the user lowers the usage fee as the user inputs the user determination result, the user is provided with an incentive to actively input the user fee even when the user agrees with the device determination result. The determination device obtains determination information by a user input, and machine learning proceeds based on the determination information, so that the determination accuracy can be improved.
Further, the display unit has a judgment basis display screen, and the user can confirm the basis of the apparatus judgment result performed by the judgment apparatus. For this reason, it is possible to easily grasp whether or not the device determination result is appropriate, and it is possible to take measures such as adopting the user determination result with respect to a determination that is considered inappropriate.
Further, the user confirms the correctness of the basis of the device determination result from the determination basis display screen, and if the basis of the device determination result is the same as the actual one (correct), the determination of the determination device is excellent, and the device is excellent. If the basis of the judgment result is different from the actual one (erroneous), even if the judgment of the match or the mismatch is correct, the evaluation is not performed. Based on such information, the charge calculating device changes the weight for determining the usage fee of the determination device, so that an appropriate usage fee can be determined.

FIG. 1 is a schematic diagram of a true / false judgment system according to a first embodiment of the present invention. FIG. 2 is a diagram illustrating a hardware configuration of a determination device (first embodiment). 5 is a flowchart of a right / wrong determination process. 7 shows an example of a judgment result display screen and a user judgment input screen. 9 is an example of a determination threshold setting screen. Example of a judgment result list screen. An example of a judgment grounds display screen. Example of factor entry screen. The schematic diagram of the right / wrong judgment system of a 2nd embodiment of the present invention.

  Hereinafter, an embodiment of a true / false judgment system of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a schematic diagram of a true / false determination system 1 according to a first embodiment of the present invention. The right / wrong judgment system 1 mainly includes a judgment device 2 equipped with artificial intelligence (the “computer” of the present invention) and installed with an application for making a right / wrong judgment.

  In the following, the right / wrong determination system 1 will be described by taking, as an example, a judgment made when a construction machine is returned in a rental company that lends construction equipment (hereinafter, referred to as a construction machine) such as a crane truck, an excavator car, or a forklift. When the construction machine is returned, the determination device 2 makes a determination of “repair” or “discard” (which may include “no abnormality”).

  Although details will be described later, the storage unit 2a of the determination device 2 stores a learning model DB (database) and a construction machine information DB. The determination device 2 (control unit 2b) determines “repair” or “discard” by using a learning model DB in which past result information and the like provided in the initial stage are stored. At this time, the use period and repair history of the construction machine stored in the construction machine information DB are also referred to. Although the determination device 2 has a certain degree of determination accuracy from the time of introduction, result information is accumulated by use of the user, and the determination accuracy is improved by performing machine learning described below.

  When a certain construction machine (for example, construction machine A or the like) is returned and the information of the return list L is input to the judgment device 2, the judgment device 2 makes a judgment of "repair" or "discard", and the judgment result is It is displayed on the display unit 2c (display X). In addition, the user P, who is an employee of the rental agent, judges the same event independently of the judgment device 2 and inputs the judgment result. Here, it is assumed that the user P is a person who has knowledge and information regarding the construction machine and has a judgment ability.

  On the other hand, when the user P makes the same determination independently of the determination device 2 and inputs the determination result, the determination result of the user P is also displayed on the display unit 2c (display Y). In FIG. 1, the determination result of the determination device 2 and the determination result of the user P are displayed on the same screen. However, a separate screen may be used so that the user P is not affected by the determination result of the determination device 2.

  The determination device 2 determines “correct” when the determination result of the device and the determination result of the user P match, and determines “incorrect” when they do not match. The determination results of "correct answer" and "incorrect answer" are stored in the storage unit 2a (a determination result DB described later) of the determination device 2, and are later incorporated into the learning model DB to become data for machine learning.

  As the machine learning of the judging device 2 progresses, the accuracy of correct / incorrect judgment is improved by the above-described series of processing (for example, 90% to 95%), so that the usage fee of the judging device 2 may be increased (for example, there is a case where the usage fee is increased). From 100,000 yen / month to 120,000 yen / month). In this regard, although the utilization rate is higher, not necessarily every time, there is no need to be responsible for the decision to the skilled person, because the burden is reduced, larger rental companies of advantage. Thus, based on the determination precision determining unit 2, tariff M is reviewed periodically.

  However, the above machine learning is performed based on the user P actively inputting the determination result. In particular, when the determination result of the determination device 2 and the determination result of the user P do not match (“incorrect answer”), it is highly necessary to cause the determination device 2 to re-learn (iterative machine learning). Therefore, if the user P inputs a determination result that does not match the determination result of the determination device 2 and cooperates in improving the determination accuracy, the rate of increase in the usage fee is reduced (for example, 100,000 yen / month). From 110,000 yen / month) or lower the usage fee. Note that reducing the price increase of the usage fee includes reducing the price increase to zero, that is, leaving the usage fee unchanged.

  Next, with reference to FIG. 2, illustrating the hardware configuration of the judgment apparatus 2.

  As shown in the drawing, the determination device 2 includes a storage unit 2a, a control unit 2b, and a display unit 2c. The storage unit 2a is a storage device such as a ROM and a RAM, and includes at least a learning model DB 21, a construction machine information DB 22, and a determination result DB 23. The storage unit 2a may be a storage device such as an external hard disk independent of the determination device 2.

  The learning model DB 21 stores basic criterion for correctness, past determination result data and related information, and provides various information for machine learning to a machine learning unit 25 described later. By the machine learning, the accuracy of the determination device 2 for correctness is gradually increased.

  The construction machine information DB 22 stores data such as service periods and repair histories of all construction machines owned by the rental company. These data are referred to when the construction machine is returned and the determination device 2 determines “repair” or “discard”.

  The judgment result DB 23 matches not only the judgment result made by the judgment device 2 (“apparatus judgment result” of the present invention), but also the judgment result and the judgment result made by the user P (“user judgment result” of the present invention). , Mismatch determination result data is stored. The determination result data is later taken into the learning model DB 21 and used for machine learning of the determination device 2.

  Next, the control unit 2b is mainly the CPU of the determination device 2, and includes a machine learning unit 25, a determination unit 26, a display control unit 27, a determination threshold setting unit 28, a user determination reception unit 29, It comprises a factor receiving unit 30, a charge calculating unit 31, and an alarm unit 32.

  When the information of the learning model DB 21 and the construction machine information DB 21 is input, the machine learning unit 25 performs machine learning for determination. The machine learning unit 25 performs “supervised learning” that prepares data in which an input and a correct output are set in advance and performs learning so that a correct determination can be made when a new input is given. It is to be noted that “unsupervised learning” in which learning is performed while extracting patterns and rules in data by giving only input may be performed.

  The determination unit 26 makes a predetermined determination when new input data (the return list L in FIG. 1) is given. In the present embodiment, the determining unit 26 determines “repair” or “discard” for the returned construction machine. The user P can set the threshold value for the determination (whether the determination is made strictly or loosely). Therefore, the information of the determination threshold value setting unit 28 is input to the determination unit 26.

  In addition, the determination unit 26 compares the determination result of the determination device 2 with the determination result of the user P, and determines that they are “correct” when they match, and determines that they are not correct when they do not match. It is determined as "incorrect". Matching whether results (determination result data) is stored in the determination result DB 23, are accumulated.

  The display control unit 27 is provided with the result from the determination unit 26, and outputs a control signal for performing the display to the display unit 2c. In addition, the display control unit 27 is provided with the determination result of the user P from the user determination reception unit 29, and outputs a control signal for performing the display to the display unit 2c. Thereby, the determination result of the determination device 2 and the determination result of the user P can be displayed on the display unit 2c.

  The display control unit 27 also receives information from the true / false factor reception unit 30, and outputs a control signal for displaying the information to the display unit 2c. The correct / wrong factor receiving unit 30 receives correct / wrong factor (correct / wrong factor) data of the determination result entered by the user P.

  The fee calculation unit 31 calculates the fee of the determination device 2 based on the determination result data of the determination result DB 23. The calculation of the fee may be performed in consideration of the judgment accuracy from the judgment result data for 10 to 20 times, or may be performed based on the judgment accuracy after summarizing the judgment result data for 3 to 6 months. Good.

  The warning unit 32 warns the content by voice, display, mail to the user P, or the like, particularly when the answer is “incorrect” in the right / wrong determination. Depending on the type of "incorrect", it may change the voice or display of the content.

  Finally, the display unit 2c is a liquid crystal display of the decision device 2. The display unit 2c displays various screens of the application installed in the determination device 2. Details of the display screen will be described later.

  Although the above description has been made on the assumption that the determination device 2 is a desktop PC installed in an office, the same true / false determination can be made with a portable terminal such as a notebook PC, a tablet terminal, or a smartphone. In such a case, the learning model DB 21 and the like are preferably stored in a cloud server.

  Next, with reference to FIG. 3, a flowchart of the right / wrong judgment processing of the right / wrong judgment system 1 will be described. Further, appropriately in the flowchart description, with reference to FIGS. 4 to 6, illustrating the various screens of the application.

  First, input data is provided to the determination device 2 (step S01). In this embodiment, the input data is a construction machine return list L (see FIG. 1). For example, the user P inputs the name of the returned construction machine. Thereafter, the process proceeds to step S02.

  In step S02, a determination process is performed. This is a process in which the determining unit 26 of the determining device 2 determines whether the returned construction machine corresponds to “repair” or “discard”. FIG. 4 includes an apparatus determination result display screen X by the determination unit 26, and the construction machine A is determined to be “〇 (discard)”. Thereafter, the process proceeds to step S03.

  Here, the user P can set in advance whether to make the judgment of the judging device 2 stricter or looser. FIG. 5 is an example of a screen on which the user P determines a determination threshold of the determination device 2. On the determination threshold setting screen Z, the vertical axis P indicates a determination criterion.

  For example, “〇 (discard)” of the determination device 2 includes “○” with a probability of 40% and “○” with a probability of 80%. At present, the threshold value is set to “0.6”, so that the determination device 2 regards “○” with a probability of 60% or more as “discard”, and “」 ”with a probability of less than 60% as“ △ ” ) ". The user P can change the judgment threshold by dragging the threshold bar b on the judgment threshold setting screen Z.

  Next, in step S03, the result of the user's right / wrong determination is input. A pull-down menu Y 'is prepared on the user determination result input screen Y shown in FIG. 4, and "repair" or "discard" can be selected. Judgment results, determined by the user to operate the "determination" button. Thereafter, the process proceeds to step S04.

  In step S04, it is determined whether the two determinations match. This is a process in which the determination unit 26 determines whether or not the determination result of the determination device 2 matches the determination result of the user. If the two judgments match, the process proceeds to step S05, and the previous judgment is counted as “correct answer”. On the other hand, if the two determinations do not match, the process proceeds to step S06, and the previous determination is counted as “incorrect”. Thereafter, the process proceeds to step S07.

  In step S07, the determination result data is stored. Specifically, the previous determination result is stored in the determination result DB 23. On the judgment result list screen R (May 2018) shown in FIG. 6, "returned objects" and "judgments" stored by this processing are displayed in a list. It should be noted that the judgment date and additional information regarding the judgment may be displayed together. Thereafter, the process proceeds to step S08.

  In step S08, the learning model DB is updated. Specifically, the learning model DB 21 is updated by transferring the determination result of the determination result DB 23 added in step S07 to the learning model DB 21. This processing does not need to be performed every time a new determination result is added, and may be performed when 10 to 20 determination results are accumulated, or collectively for one month. Thereafter, the process proceeds to step S09.

  In step S09, the correct answer rate (judgment accuracy) is calculated. The judgment result list screen R in FIG. 6 is an example in which the judgment results are managed for each month, but each time a judgment result is added, the correct answer rate is calculated, and the final correct answer rate is determined at the end of the month. I do. Thereafter, the process proceeds to step S10.

  Finally, in step S10, the usage fee is reviewed based on the correct answer rate. This processing need not be performed each time a new determination result is added, but the outline is as follows. For example, if the correct answer rate of the judgment apparatus 2 in the last month (April 2018) is 90% and the correct answer rate of this month is 92% (see FIG. 6), the usage fee of the judgment apparatus 2 will be increased. However, the burden on the rental agency (user P) is reduced.

  On the other hand, when the user P inputs the determination result of “incorrect” and cooperates with the improvement of the determination accuracy, the rate of increase in the usage fee is reduced. In addition, the usage fee of the determination device 2 may be reduced. In this way, the usage fee (charge table M) is reviewed every time or periodically based on the correct answer rate of the determination device 2. After that, the right / wrong judgment processing is ended.

  In the right / wrong judgment process, the user P may not input the judgment result. In this case, when the predetermined time has elapsed, for example, a notification may be given to the user P who has logged in to the right / wrong determination system 1. Nevertheless, if the user P does not enter a judgment result advances the subsequent processing of the correctness determination process.

  In particular, when the user P agrees with the determination result of the determination device 2 displayed on the display unit 2c, the incentive to input to the user P does not work because reduction of the usage fee cannot be expected. However, since it is desirable that the accuracy of the correctness determination system 1 be higher, it is unlikely that the user P intentionally inputs a false determination result that lowers the determination accuracy. Therefore, when the user P does not input the determination result, the determination result is set to “match (correct answer)” regardless of whether the match is made or not. Thereby, even when the user P does not input the determination result, the current determination result data is obtained, and the determination result DB 23 is updated.

  In addition, in the right / wrong judgment processing, there are two patterns in the case where the judgment result is “incorrect”. The first is a pattern in which the user P makes a negative determination, such that the determination device 2 determines “repair” and the user P determines “discard”. If the user P does not input the determination result and later decides to “discard”, the decision of the decision device 2 is adopted, and the construction machine remains without being discarded. Since "discarding" can be performed later, no major problem arises in this case.

  The other is a pattern in which the judging device 2 makes a negative judgment such that the judging device 2 judges “discard” and the user P judges “repair”. If the user P does not input the determination result and determines “repair” later, the determination of the determination device 2 is adopted, and the construction machine may be already disposed. In this case, an irreversible possibility may occur, so that an alarm is issued by the alarm unit 32 described above.

  When the threshold is set to a relatively low value (for example, “0.5” or less) on the determination threshold setting screen Z (see FIG. 4), the determination device 2 makes a strict determination and makes a negative determination. I often do it. Therefore, it is preferable that the threshold is set to a value between “0.6” and “0.8”.

  Even if the threshold value is set to an appropriate value, if the determination device 2 makes a negative determination and obtains an “incorrect answer”, weighting is performed when storing the determination result data in the determination result DB 23. It may be changed. Then, in addition to the correct answer rate (judgment accuracy), the usage fee can be reviewed based on the judgment result data in which the weight is considered.

  The judgment device 2 of the right / wrong judgment system 1 also has the following functions. 7, to the determination result of the determination device 2 was carried out, shows a Basis display screen S for the user P obtains its grounds.

  When the user P is interested in the judgment result of the judging device 2 or wants to confirm why the judging device 2 made the judgment, the user P displays the judgment ground display screen S on the display unit 2c. In the judgment basis display screen S, a document serving as a basis for making a decision on “discard” or “repair” for the returned “construction machine B” is displayed.

  Specifically, the usage period, operating time, market price, market condition, repair history, and the like of “construction machine B” are displayed. For example, if the use period is relatively short, but the repair history of a certain part is recorded, when the determination device 2 determines “repair”, the user P will suspect that the same part is repaired.

  If the part determined as “repair” by the determination device 2 matches the part to be actually repaired, the result of the determination by the determination device 2 is very good. On the other hand, if the part determined as “repair” by the determination device 2 does not match the part to be actually repaired, the determination result merely becomes “correct” by chance.

  Such information is also stored in the determination result DB 23 together with the determination result data, and the weight may be changed at that time. Then, this weighting may affect the usage fee. As described above, since the user P can obtain the basis of the determination result of the determination device 2, it can verify whether the determination of the determination device 2 is really correct.

  FIG. 8 shows a factor entry screen T in which the user P enters factors for which the determination made by the determination device 2 is correct or incorrect.

  The factor entry screen T is composed of a combination of information on a returned product and a text input window. “Construction machine A” and “Construction machine B” are cases where the determination result is “correct answer”, and the user P enters the cause of the correct answer in the text input window. The comment of the user P is stored in the determination result DB 23 together with the determination result data. When the judgment result is "correct answer", there is almost no matter to be entered, and the text input window is made smaller.

  Furthermore, "Construction Machinery C", for example, determines judging device 2 is the "repair", the user P is determined to "discard", the determination result is an occurrence of a "wrong answer". In this case, the user P fills in the grounds on which the user P has determined to be “discarded”. When the determination result is "incorrect", there are many entries to be entered from the viewpoint of investigating the cause and the like, and the text input window is made large. Information on how the final disposal of “construction machine C” was made may also be entered.

  As described above, the determination result data regarding the current true / false determination is reinforced, and such information is finally taken into the learning model DB 21 and can be used for machine learning. Thereby, the determination accuracy of the determination device 2 can be improved.

  As described above, in the right / wrong determination system 1 according to the first embodiment of the present invention, the determination device 2 includes a display unit 2c that displays various information, a determination unit 26 that determines correctness, and a device determination result. It includes a determination result DB 23 for storing the determination result, and a fee calculation unit 31 for calculating a usage fee. The display unit 2c has a device determination result display screen X for displaying the result of the device determination performed by the determination device 2, and a user determination result input screen Y for inputting the user determination result performed by the user P. Then, the charge calculation unit 31 based on the determination information mismatch stored in the determination result DB 23, to or reductions reduce the price increase of the usage fee determination unit 2. As described above, the incentive for inputting the determination result is given to the user P, and the accuracy of the determination performed by the determination device 2 can be improved.

  It is considered that the determination accuracy of the determination device 2 is gradually increased when the determination device 2 is continuously used, and the usage fee is increased in a long term. Therefore, in order to suppress an increase in the usage fee, the fee calculation unit 31 reduces or decreases the price increase of the usage fee based on the determination information irrespective of the match or mismatch stored in the determination result DB 23. You may.

  For example, if the user P is cooperating with the input regardless of “correct answer” or “incorrect answer”, the usage fee is reduced. Since a reduction in the usage fee is a strong incentive for the user P to make an input, a system in which the usage fee can be suppressed by the input of the user P can be realized even if the usage fee increases due to the improvement in the accuracy of the determination device 2. Becomes

  In addition, when the user P performs an input, determination information is generated, and machine learning of the determination device 2 proceeds to improve determination accuracy. If the user P agrees with the device determination result of the determination device 2 and inputs “correct answer”, the accuracy of the determination made by the determination device 2 is also significant.

[Second embodiment]
FIG. 9 is a schematic diagram of a true / false determination system 10 according to the second embodiment of the present invention. In the following, the same configuration as the first embodiment are denoted by the same reference numerals and will not be part of the description.

  The right / wrong judgment system 10 includes a judgment device 20 equipped with artificial intelligence and installed with an application for making a right / wrong judgment, and a fee calculation device 5. Since the fee calculating device 5 is a device separate from the judging device 20, it is not necessary to install the fee calculating device 5 in a construction equipment rental company.

  As illustrated, the determination device 2 includes a storage unit 2a, a control unit 2b ', and a display unit 2c. Storage unit 2a, ROM, is a storage device such as a RAM, and includes at least learning model DB 21, the construction machine information DB 22, and a determination result DB 23.

  Next, the control unit 2b ′ is mainly a CPU of the determination device 2, and includes a machine learning unit 25, a determination unit 26, a display control unit 27, a determination threshold setting unit 28, a user determination reception unit 29, It comprises a true / false factor receiving unit 30 and an alarm unit 32.

  The judging unit 26 compares the judgment result of the judging device 20 with the judgment result of the user P. If the two agree, the judgment is “correct”. Correct answer ". The result of the match (judgment result data) is stored and accumulated in the judgment result DB 23.

  Price calculation device 5 is provided a judgment result data from the judgment result DB23 determination device 20 is a computer which calculates the price for the determination device 20 based on the judgment result data. Since the determination device 20 is a computer, the determination result data can be provided to the fee calculation device 5 via a telecommunication line or the like.

  Here, it is assumed that the judging device 20 is installed in a rental company of construction equipment, and the fee calculating device 5 is installed in an equipment lending company that lent the judging device 20. At this time, the judgment result data of the judgment result DB 23 can be transferred from the rental company of the construction equipment to the fee calculation device 5 of the equipment lending company through a storage medium such as a portable hard disk. Also, so as to store the determination result data to the cloud server, price calculation device 5 may acquire the determination result data by accessing the cloud server.

  For example, after the judgment result data for 10 to 20 times or for 3 to 6 months is put together, the judgment result data is automatically transmitted from the judging device 20 to the fee calculating device 5, and the fee calculating device 5 calculates the usage fee. it may be so. It should be noted that the charge calculation result and the revised charge table are notified from the charge calculation device 5 to the determination device 20 by e-mail or the like.

  As described above, the right / wrong determination system 10 according to the second embodiment of the present invention includes the determination device 20 and the fee calculation device 5. The determination device 20 includes a display unit 2c that displays various information, a determination unit 26 that determines correctness, and a determination result DB 23 that stores the device determination result and the determination result. It has a device determination result display screen X for displaying the result of the device determination performed, and a user determination result input screen Y for inputting the user determination result performed by the user P. Then, the fee calculation device 5 reduces or reduces the price increase of the usage fee of the determination device 20 based on the mismatch determination information provided from the determination result DB 23. In this way, by giving the user P an incentive to input the determination result, the accuracy of the determination performed by the determination device 20 can be improved.

  The above description is only a part of the embodiment of the present invention, and various other embodiments can be considered. In the true / false determination system 10 of the second embodiment, the fee calculation device 5 determines whether to reduce the price increase of the usage fee of the determination device 20 based on the determination information provided from the determination result DB 23 regardless of the match or the mismatch. Price cuts can be made. As a result, a system in which the usage fee is reduced as the user P cooperates with the input can be realized.

  In addition, various factors can be added to the review of the usage fee (charge table M) in addition to the correct answer rate (judgment accuracy). For example, when the construction machine is returned and the determination device 2 makes a decision of “repair”, an estimated amount of repair is calculated, and a repair estimation screen of the construction machine to be repaired is displayed on the display unit 2c. Thereafter, if the user P also makes a determination of “repair”, the construction machine is actually repaired. Here, the estimated amount of the determination device 2 is compared with the actual estimated amount to change the weight, so that the weight affects the usage fee. Also by such a method, it is possible to determine an appropriate usage fee for the determination device 2.

  In the right / wrong judgment process (see FIG. 3), the order in which the user P judges the same event after the judgment process of the judgment device 2 is not limited to this. The user P may input the determination result before the determination process of the determination device 2. Further, when the user P inputs the determination result, the device determination result display screen X may not be visible. Thereby, the user P can make a fair judgment without being affected by the judgment result of the judgment device 2.

  1,10 ... correctness judgment system, 2,20 ... judgment device, 2a ... storage unit, 2b, 2b '... control unit, 2c ... display unit, 5 ... charge calculation device, 21 ... learning model DB, 22 ... construction machine information DB, 23: determination result DB, 25: machine learning unit, 26: determination unit, 27: display control unit, 28: determination threshold setting unit, 29: user determination reception unit, 30: true / false factor reception unit, 31: charge calculation part, 32 ... alarm section, L ... return list, M ... price list, P ... user, R ... the determination result list screen, S ... basis display screen, T ... factor fill the screen, X ... apparatus determination result display screen, Y ... user judgment result input screen, Z ... judgment threshold setting screen.

Claims (5)

When given input information to the computer for machine learning, performs predetermined decision, a right or wrong decision system having a decision device to answer the results,
The determination device, a display unit that displays various information, a determination unit that determines the correctness, a storage unit that stores the device determination result of the determination unit, a fee calculation unit that calculates the usage fee of the determination device With
The display unit is a device determination result display screen that displays the device determination result, a user determination result input screen that inputs a user determination result performed by a user, and a user acquires a basis for the device determination result. A judgment basis display screen ,
The determination unit determines whether the device determination result matches the user determination result,
The storage unit stores determination information of a match or mismatch between the device determination result and the user determination result,
The charge calculation unit, by the correctness of grounds of the apparatus determination result obtained by the user, to change the weighting for determining the charge for the determination device, based on the previous SL-size constant information stored in the storage unit, A true / false judgment system, wherein the price increase of the usage fee of the judgment device is reduced or reduced. The right / wrong judgment system according to claim 1,
The true / false determination system, wherein the determination unit determines a match when the user does not input the user determination result. In correctness determination system according to claim 1 or 2,
The true / false judgment system, wherein the display unit has a judgment threshold setting screen in which a user sets a threshold for determining a judgment criterion performed by the judgment device. In the right / wrong judgment system according to any one of claims 1 to 3 ,
The right / wrongness determination system, wherein the display unit has a factor entry screen in which a user enters a factor in which the determination made by the determination device is matched or mismatched. When input information is given to a computer that performs machine learning, a predetermined determination is performed, and a correctness determination system including a determination device that answers the result and a fee calculation device that calculates a usage fee of the determination device,
The determination device includes a display unit that displays various information, a determination unit that determines correctness, and a storage unit that stores a device determination result of the determination unit,
The display unit is a device determination result display screen that displays the device determination result, a user determination result input screen that inputs a user determination result performed by a user, and a user acquires a basis for the device determination result. A judgment basis display screen ,
The determination unit determines whether the device determination result matches the user determination result,
The storage unit stores determination information of a match or mismatch between the device determination result and the user determination result,
The price calculation device, the correctness of grounds of the apparatus determination result obtained by the user, to change the weighting for determining the charge for the determination device, before SL-format provided from the storage unit of the determination apparatus Teijoho A true / false judgment system characterized in that the price increase of the usage fee of the judgment device is reduced or reduced based on the following.