WO2023042301A1 - Dispositif de traitement d'informations, procédé de traitement d'informations et support d'enregistrement - Google Patents

Dispositif de traitement d'informations, procédé de traitement d'informations et support d'enregistrement Download PDF

Info

Publication number
WO2023042301A1
WO2023042301A1 PCT/JP2021/033932 JP2021033932W WO2023042301A1 WO 2023042301 A1 WO2023042301 A1 WO 2023042301A1 JP 2021033932 W JP2021033932 W JP 2021033932W WO 2023042301 A1 WO2023042301 A1 WO 2023042301A1
Authority
WO
WIPO (PCT)
Prior art keywords
prediction
data
input data
abnormal values
absence
Prior art date
Application number
PCT/JP2021/033932
Other languages
English (en)
Japanese (ja)
Inventor
紗和子 梅津
Original Assignee
日本電気株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日本電気株式会社 filed Critical 日本電気株式会社
Priority to JP2023548001A priority Critical patent/JPWO2023042301A5/ja
Priority to PCT/JP2021/033932 priority patent/WO2023042301A1/fr
Publication of WO2023042301A1 publication Critical patent/WO2023042301A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • This disclosure relates to monitoring machine learning models.
  • Patent Literature 1 discloses a method of generating an inspector model for detecting deterioration in accuracy of an operational model and using this to detect changes in output results of the operational model due to temporal changes in data trends.
  • One object of the present disclosure is to provide an information processing device capable of creating and presenting diagnostic information regarding a plurality of factors affecting prediction accuracy for a model in operation.
  • an information processing device includes: input data acquisition means for acquiring a plurality of input data; input data evaluation means for detecting outliers and abnormal values in the input data; prediction data acquisition means for acquiring prediction data generated from the input data using the trained model; prediction data evaluation means for detecting abnormal values in the prediction data; prediction accuracy acquisition means for acquiring prediction accuracy by the trained model; display means for displaying diagnostic information including the presence or absence of outliers and abnormal values in the input data, the presence or absence of abnormal values in the prediction data, and the prediction accuracy; Prepare.
  • an information processing method includes: get multiple input data, detecting outliers and abnormal values in the input data; Obtaining prediction data generated from the input data using the trained model, detecting abnormal values in the predicted data; Obtaining the prediction accuracy of the trained model, Diagnosis information including presence/absence of outliers and abnormal values in the input data, presence/absence of abnormal values in the predicted data, and the prediction accuracy is displayed.
  • the recording medium comprises get multiple input data, detecting outliers and abnormal values in the input data; Obtaining prediction data generated from the input data using the trained model, detecting abnormal values in the predicted data; Obtaining the prediction accuracy of the trained model, A program for causing a computer to execute a process of displaying diagnostic information including the presence or absence of outliers and abnormal values in the input data, the presence or absence of abnormal values in the predicted data, and the prediction accuracy is recorded.
  • FIG. 1 is a block diagram showing the overall configuration of a monitoring system 1 according to the first embodiment.
  • the monitoring system 1 is a system that monitors the state of a machine learning model that has been learned in advance and is in operation.
  • the monitoring system 1 includes a prediction device 2 and a monitoring device 100 .
  • the prediction device 2 is a device that makes predictions using a prediction model.
  • a prediction model is an example of a machine learning model to be monitored by the monitoring system 1, and is a model that has already been trained using learning data.
  • the prediction device 2 makes a prediction based on the input data D1, generates prediction data D2 as a prediction result, and outputs the prediction data D2 to the monitoring device 100.
  • FIG. 1 A prediction model is an example of a machine learning model to be monitored by the monitoring system 1, and is a model that has already been trained using learning data.
  • the prediction device 2 makes a prediction based on the input data D1, generates prediction data D2 as a prediction result, and outputs the prediction data D2 to the monitoring device 100.
  • the monitoring device 100 evaluates whether the input data D1 is normal and evaluates whether the prediction data D2 generated by the prediction device 2 is normal. Also, the monitoring device 100 is input with performance data D3. The performance data D3 is data corresponding to the input data D1 and data actually obtained in the real world. The monitoring device 100 evaluates the prediction accuracy of the prediction model by calculating the error rate between the prediction data D2 generated by the prediction device 2 and the performance data D3. Then, the monitoring device 100 generates display data including the input data D1, the prediction data D2, and the prediction accuracy evaluation result. By looking at the displayed data, the user can know the operation status of the prediction model and consider the necessity of re-learning the prediction model.
  • FIG. 2 is a block diagram showing the hardware configuration of the monitoring device 100. As shown in FIG. As illustrated, the monitoring device 100 includes an interface (I/F) 11, a processor 12, a memory 13, a recording medium 14, a database (DB) 15, a display device 16, and an input device 17. Prepare.
  • the interface 11 performs data input/output with an external device. Specifically, the input data D1, the prediction data D2, and the performance data D3 are input to the monitoring device 100 through the interface 11. FIG.
  • the processor 12 is a computer such as a CPU (Central Processing Unit), and controls the entire monitoring device 100 by executing a program prepared in advance.
  • the processor 12 may be a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array).
  • the processor 12 executes monitoring processing, which will be described later.
  • the memory 13 is composed of ROM (Read Only Memory), RAM (Random Access Memory), and the like. Memory 13 is also used as a working memory during execution of various processes by processor 12 .
  • the recording medium 14 is a non-volatile, non-temporary recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the monitoring device 100 .
  • the recording medium 14 records various programs executed by the processor 12 .
  • DB15 memorize
  • the display device 16 is, for example, a liquid crystal display device, etc., and displays the monitoring results generated by the monitoring device 100 .
  • the input device 17 is, for example, a mouse, a keyboard, etc., and is used by the user to make necessary instructions and inputs in the monitoring process.
  • FIG. 3 is a block diagram showing the functional configuration of the monitoring device 100 of the first embodiment.
  • the monitoring device 100 functionally includes an input data evaluation unit 21 , a prediction data evaluation unit 22 , a prediction accuracy evaluation unit 23 , and a display data generation unit 24 .
  • Input data D1 is input to the input data evaluation unit 21 .
  • the input data evaluation unit 21 detects outliers and abnormal values in the input data D1.
  • the "outlier” is a value within a predetermined range that is defined as an impossible value for the input data D1.
  • an "abnormal value” is a value within a predetermined range that deviates from the normal value when the value of the input data with which the prediction model can make an appropriate prediction is assumed to be the normal value.
  • the input data evaluation unit 21 adds information indicating an outlier or an abnormal value to the input data D1 and outputs the data to the display data generation unit 24 .
  • Prediction data D2 which is the result of prediction by the prediction device 2, is input to the prediction data evaluation unit 22.
  • the predicted data evaluation unit 22 detects abnormal values in the predicted data D2.
  • the "abnormal value” is a value within a predetermined range outside the normal value when the value of the prediction data output when the prediction model makes a proper prediction is assumed to be the normal value.
  • the prediction data evaluation unit 22 adds information indicating an abnormal value to the prediction data D2 and outputs the data to the display data generation unit 24 .
  • the prediction accuracy evaluation unit 23 receives the prediction data D2 and the performance data D3.
  • the prediction accuracy evaluation unit 23 calculates an error rate, which is an index indicating prediction accuracy, based on the prediction data D2 and the performance data D3.
  • the error rate is a value that indicates the deviation of the prediction result from the actual value, and is used as an index that indicates the prediction accuracy of the prediction model.
  • the prediction accuracy evaluation section 23 outputs the calculated error rate to the display data generation section 24 .
  • the prediction accuracy evaluation unit 23 uses the error rate as an index indicating the prediction accuracy. may be evaluated.
  • the display data generation unit 24 uses the input data D1 input from the input data evaluation unit 21, the prediction data D2 input from the prediction data evaluation unit 22, and the error rate input from the prediction accuracy evaluation unit 23. , to generate the display data.
  • the display data is data for displaying to the user the presence/absence of outliers and abnormal values in the input data D1, the presence/absence of abnormal values in the prediction data D2, and the prediction accuracy, and is output to the display device 16.
  • the display device 16 displays the input display data on the screen. This allows the user to easily obtain information on the performance and status of the prediction model currently in operation.
  • FIG. 4 is a flow chart of processing by the monitoring device. This processing is realized by executing a program prepared in advance by the processor 12 shown in FIG. 2 and operating as each element shown in FIG.
  • the input data evaluation unit 21 acquires the input data D1, the prediction data evaluation unit 22 acquires the prediction data D2, and the prediction accuracy evaluation unit 23 acquires the prediction data D2 and the performance data D3 (step S11).
  • the input data evaluation unit 21 evaluates the input data D1, adds information indicating an outlier or an abnormal value to the input data D1, and outputs the evaluation result to the display data generation unit 24 (step S12).
  • the prediction data evaluation unit 22 evaluates the prediction data, adds information indicating an abnormal value to the prediction data D2, and outputs the evaluation result to the display data generation unit 24 (step S13).
  • the prediction accuracy evaluation unit 23 calculates prediction accuracy (error rate) based on the prediction data D2 and the performance data D3, and outputs the evaluation result to the display data generation unit 24 (step S14).
  • the display data generation unit 24 generates display data using the evaluation results of the input data evaluation unit 21, the prediction data evaluation unit 22, and the prediction accuracy evaluation unit 23, and outputs the display data to the display device 16 (step S15).
  • the display device 16 displays the display data (step S16). Then the process ends.
  • the display data includes diagnostic reports and, if necessary, analysis result screens. That is, as the monitoring result regarding the prediction model, only the above diagnostic report may be presented to the user, or the following analysis result screen may be presented in addition to the diagnostic report.
  • the person in charge at Company A displays the diagnostic report and the analysis result screen, continues detailed monitoring of the forecast model delivered to Company B, and presents only the diagnostic report to Company B.
  • a report on the state of the predictive model may be made.
  • FIG. 5 shows an example of a diagnosis report included in display data.
  • the diagnostic report is created based on the evaluation results by the input data evaluation unit 21, the prediction data evaluation unit 22, and the prediction accuracy evaluation unit 23, and other various data obtained by monitoring the prediction model, and is used to describe the current state of the prediction model. This report briefly describes the situation of
  • the diagnostic report is a report on 10 diagnostic items.
  • the diagnostic report includes an ID, description, evaluation result, countermeasures, and priority of countermeasures for each diagnostic item.
  • ID is the identification information of each diagnostic item reported by the diagnostic report
  • Delivery is the description of the diagnostic item.
  • the “evaluation result” is the diagnosis result for each diagnostic item, and the “countermeasure” is the action proposed to improve the condition when the evaluation result of the diagnostic item is not good.
  • "Response priority” indicates the order of priority of each countermeasure when a plurality of countermeasures are proposed.
  • the diagnostic item 001 “existence of outlier in input data” indicates whether or not the input data is an outlier.
  • An “outlier” is a value that cannot exist in reality. For example, in the real estate rent prediction model described later, "required time from the station” is used as input data. Negative values are set as outliers for "required time from station”. In the example of FIG. 5, since the evaluation result of the diagnostic item 001 is "none", the diagnostic report indicates that there are no outliers in the input data D1.
  • Presence or absence of abnormal value in input data indicates whether or not the input data is an abnormal value.
  • the "abnormal value” here is a value that indicates that the tendency of the input data has changed, and specifically refers to a value that does not belong to the range of the input data used during learning of the prediction model. If the input data value corresponds to an abnormal value, it is determined that the input data value input to the prediction model has changed during actual prediction. In the example of FIG. 5, since the evaluation result of the diagnostic item 002 is "none", the diagnostic report indicates that there were no abnormal values in the input data.
  • Presence or absence of abnormal value in prediction data indicates whether or not the prediction data generated by the prediction model is an abnormal value.
  • the "abnormal value” here refers to a value that does not belong to the range of the actually obtained performance data or the correct data used during the learning of the prediction model. If the predicted data value corresponds to an abnormal value, it is suspected that the input data is an outlier or an abnormal value, or that the prediction accuracy of the prediction model is low. In the example of FIG. 5, since the evaluation result of the diagnostic item 003 is "none", the diagnostic report indicates that there is no abnormal value in the prediction data.
  • the "prediction accuracy” of the diagnosis item 004 indicates the accuracy of prediction by the prediction model, that is, the reliability of the model, and is indicated by the above-mentioned error rate, for example.
  • the evaluation result of the diagnostic item 004 is "40% lower than the initial”, and the diagnostic report shows that the accuracy of the prediction model currently in use has decreased by 40% compared to when it was first used. It is shown that. For this reason, the diagnosis report proposes "re-learning" as a countermeasure, and the priority of this countermeasure is set to "1", which is the highest.
  • the diagnostic item 005 "whether exceptions occurred in data processing" indicates whether or not any problem occurred in the processing when the input data was processed before being input to the prediction model.
  • Data processing means that, for example, instead of inputting certain input data as it is to a prediction model, when calculating and inputting the average value of a predetermined number or the moving average value of the last 7 days, those values means to calculate
  • the defect includes the case where the value obtained by the data processing corresponds to an outlier of the input data.
  • the evaluation result of the diagnostic item 005 is "none", so the diagnostic report indicates that no exception occurred during data processing.
  • Model creation time in diagnostic item 006 indicates the time required to create a prediction model, that is, to learn a prediction model using learning data.
  • the creation of the model here includes updating (relearning) of the model in addition to the creation of the initial model.
  • the time required to create a prediction model can be predicted to some extent according to the amount of learning data to be used, conditions for terminating the learning process, and the like. Therefore, if the prediction model creation time is much shorter or longer than usual, it is doubtful whether the learning process was performed correctly. Therefore, it is diagnosed whether or not the model creation time was appropriate.
  • the evaluation result of the diagnostic item 006 is "within the regulation", so the diagnostic report indicates that the model creation time was appropriate.
  • Diagnosis item 007 "cause of fluctuation in prediction accuracy" indicates an item presumed to be the cause when the diagnosis item 004 diagnoses that the prediction accuracy has deteriorated. For example, in the case of the forecast model for real estate rents mentioned above, factors such as ⁇ stagnation of economic activity due to the epidemic of infectious diseases'' and ⁇ construction of luxury condominiums and shopping malls nearby'' can be considered as causes of fluctuations in forecast accuracy. be done. Note that the cause of the prediction accuracy variation may be estimated using an algorithm for evaluating a prediction model or an evaluation model, or may be estimated by humans. In the example of FIG. 5, the evaluation result of diagnostic item 007 is presumed to be "because XX has occurred and the tendency has changed". In addition, regarding this, the diagnostic report proposes a countermeasure to add YY to the explanatory variable, and the priority of the countermeasure is set to "2.”
  • the diagnostic item 008 "whether or not appropriate system output is implemented" indicates whether or not the prediction data is correctly output from the prediction device using the prediction model. Prediction results using prediction models are usually input to other related systems and used. Therefore, it is diagnosed whether or not the prediction device is correctly outputting prediction data to another system. In the example of FIG. 5, the evaluation result of the diagnostic item 008 is "implemented", so the diagnostic report indicates that the prediction data is correctly output from the prediction device.
  • "Response time to system" of diagnostic item 009 indicates whether or not the prediction data is correctly exchanged between the prediction device using the prediction model and another system that receives and uses the prediction data. . Specifically, if the prediction device is designed to transmit prediction data to another system, the response time to the system is the response from the other system indicating receipt after the prediction device has transmitted the prediction data to the other system. It is time to receive. In addition, in the case of specifications in which another system requests prediction data from the prediction device, the response time to the system is the time from when the prediction device receives the request from the other system to when it transmits the prediction data to the other system. It's time. It is diagnosed whether or not these response times are within the specified time. In the example of FIG. 5, the evaluation result of the diagnostic item 009 is "within specification", and the diagnostic report indicates that the prediction data is correctly transmitted and received.
  • the diagnostic item 010 "display of caution information" displays various alerts such as alerts related to input data, prediction data and prediction accuracy, which will be described later, and alerts output when prediction data is not output from the prediction device. 16 indicates whether or not it is displayed correctly. In the example of FIG. 5, the evaluation result of the diagnostic item 010 is "displayed", and the diagnostic report indicates that various alerts are correctly displayed.
  • diagnostic items 001 to 010 are diagnostic items mainly related to the state of the prediction model
  • diagnostic items 005 to 010 are the overall operation of the prediction device that performs prediction using the prediction model. It is an item related to the situation.
  • the diagnostic report displays a list of diagnostic results for predetermined diagnostic items, so the overall status of the prediction model can be easily grasped.
  • the diagnosis item for which the diagnosis result is determined to be abnormal is highlighted by, for example, changing the color of the characters or the background, so as to alert the user that there is an abnormality. good too.
  • the analysis result screen displays the diagnostic results by graphically showing the numerical grounds for the diagnostic items 001 to 004, which are particularly related to the prediction model, among the diagnostic items described in the diagnostic report.
  • the prediction model will predict real estate rents. Specifically, the prediction model predicts the monthly rent of a rental property based on input data such as the size of the property, the floor plan, and the time required from the station. Note that the prediction model is a trained model that has been trained using input data such as the size of the actual rental property, the floor plan, and the time required from the station, and the actual rent of the rental property as correct data.
  • Fig. 6 shows an example of the analysis result screen.
  • the analysis result screen 200 includes input data, prediction data, and an error rate indicating prediction accuracy as display items. As described above, there are multiple pieces of input data such as the size of the property, the floor plan, and the time required from the station. can be made Note that the example of FIG. 6 shows analysis results in a steady operation state, that is, in a state in which all of the input data, prediction data, and error rate are normal.
  • the analysis result screen 200 includes graphs 201 and 204 regarding input data, graphs 202 and 205 regarding predicted data, and graphs 203 and 206 regarding error rates.
  • a graph 201 is a box plot showing transition of input data.
  • the horizontal axis indicates the date when the input data was input, and the vertical axis indicates the required time from the station, which is one of the input data.
  • the graph 201 is a box plot showing the minimum, maximum, median, average, and interquartile range (25% to 75% of the total) of the required time from the station for each day when the input data is entered. distribution of values belonging to ).
  • the graph 202 is a boxplot of forecast data, with the horizontal axis showing the date when the forecast data was obtained and the vertical axis showing the rent as the forecast result.
  • a graph 202 shows the minimum value, maximum value, median value, average value, interquartile range, etc. of the rent for each day on which forecast data is obtained by means of a box plot.
  • the graph 203 is a boxplot of the error rate, the horizontal axis indicates the date when the forecast data was obtained, and the vertical axis indicates the error rate of the forecast data.
  • a graph 203 shows the minimum value, maximum value, median value, average value, interquartile range, etc. of the error rate that indicates the prediction accuracy for each day on which the prediction is made, using a box plot.
  • a graph 204 is a histogram of the required time from the station, which is one of the input data, and shows the distribution of the required time from the station over a certain period of time, for example, the period shown in the graph 201 .
  • the horizontal axis indicates the required time from the station, and the vertical axis indicates the frequency.
  • diagonally hatched bins indicate the distribution of values used as input data when training the prediction model
  • gray bins indicate the distribution of values input as input data when making predictions using the prediction model.
  • the value "8.03" in the figure is the average value of input data during learning, and the value "5.91" is the average value of input data during prediction.
  • Graph 205 is a histogram of forecast data, and shows the distribution of rent over a certain period of time, for example, the period shown in graph 202.
  • the horizontal axis indicates rent, and the vertical axis indicates frequency.
  • obliquely hatched bins indicate the distribution of rents used as correct data when learning the prediction model
  • gray bins indicate the distribution of rents obtained as prediction results when making predictions using the prediction model. Similar to the graph 204, the average value of correct data during learning is "90,364" and the average value of prediction data during prediction is "70,036".
  • a graph 206 is a histogram of error rates, showing the distribution of error rates over a certain period of time, for example, the period shown in graph 203 .
  • the horizontal axis indicates the error rate, and the vertical axis indicates the frequency.
  • obliquely hatched bins indicate the distribution of error rates calculated during prediction model learning
  • gray bins indicate the distribution of error rates calculated during prediction using the prediction model. Similar to the graph 204, the average value of the error rate during learning is “24.977” and the average value of the error rate during prediction is “13.15”.
  • the analysis result screen shows the analysis results of input data, prediction data, and error rate (prediction accuracy) for a certain period of time in graphs such as box plots and histograms.
  • the state of the error rate can be grasped based on a concrete numerical value.
  • graphs 204 to 206 for the input data, prediction data, and error rate, by displaying the values at the time of learning of the prediction model and the values at the time of prediction in the same graph, It is possible to visualize the extent and trend of changes in each data.
  • Fig. 7 shows an example of the analysis result screen on another day.
  • the analysis result screen 210 includes boxplots 211-213 and histograms 214-216 for input data, predicted data and error rate, as in the example of FIG.
  • the view of each graph is the same as in the example of FIG.
  • FIG. 7 shows analysis results when input data includes outliers.
  • "a value less than 0" is set as an outlier for the input data "required time from the station”.
  • the minimum value of the required time from the station on August 13 is "-1", which corresponds to an outlier. Therefore, in the graph 211, a circle AL1 is displayed as an alert.
  • the minimum value of the required time from the station is "-1".
  • an alert indicating that fact is displayed.
  • Fig. 8 shows an example of the analysis result screen on yet another day.
  • the analysis result screen 220 includes boxplots 221-223 and histograms 224-226 for input data, prediction data and error rate, as in the example of FIG.
  • the view of each graph is the same as in the example of FIG.
  • the input data is an abnormal value.
  • the case where "the difference between the average value at the time of learning and the average value at the time of prediction is 3 or more” is set as an abnormal value.
  • the difference between the average value “8.03” during learning of the input data “required time from the station” and the average value “14.13” during prediction is 3 or more.
  • the data are judged to be outliers. Therefore, an arrow AL2 is displayed in the graph 224 as an alert to that effect.
  • the predicted data is an abnormal value.
  • the prediction data is set as an abnormal value when it is "260,000 or more" or "the difference between the average value at the time of learning and the average value at the time of prediction is 30,000 or more".
  • the maximum value of forecast data for September 12 exceeds 260,000, and a circle AL3 is displayed as an alert to that effect.
  • the error rate is an abnormal value.
  • an error rate of "40% or more” is set as an abnormal value.
  • FIG. 8 as shown in graph 223, there are days when the error rate exceeds 40%, and a rectangle AL4 is displayed as an alert to that effect.
  • the analysis result screen displays alerts for outliers and abnormal values in the input data, abnormal values in the predicted data, and abnormal values in the error rate (prediction accuracy). Abnormal state of the error rate can be easily known.
  • the alerts (circle AL3, rectangle AL4) related to the prediction data and the error rate are displayed in the boxplot graphs 222 and 223, but these alerts are displayed in the histogram graphs 225 and 226. You may That is, alerts regarding input data, predicted data, and error rates may be displayed on one or more of the multiple graphs.
  • FIGS. 6 to 8 the input data, predicted data, and error rate are shown by box plots and histograms, but line graphs may be used instead of box plots. 9 to 11 show examples of displaying input data, predicted data, and error rates using line graphs and histograms.
  • line graphs 201a to 203a are displayed instead of the boxplots 201 to 203 shown in FIG.
  • line graphs 211a to 213a are displayed instead of the boxplots 211 to 213 shown in FIG.
  • a circle AL5 is displayed as an alert indicating an outlier in the input data.
  • line graphs 221a to 223a are displayed instead of the boxplots 221 to 223 shown in FIG.
  • an arrow AL6 is displayed as an alert in the graph 224 as in FIG.
  • a circle AL7 is displayed as an alert indicating an abnormal value of predicted data
  • a rectangle AL8 is displayed as an alert indicating an abnormal value of the error rate.
  • the input data, prediction data and error rate for one prediction model are displayed on one analysis result screen, but if there are multiple prediction models , input data, prediction data and error rates for a plurality of prediction models may be simultaneously displayed on one analysis result screen.
  • graphs of input data, prediction data, and error rates as shown in FIG. 6 may be prepared for each model and displayed side by side on one analysis result screen.
  • the values of the two models may be displayed simultaneously in different colors and superimposed on each graph of the input data, prediction data, and error rate. In this case, it may be determined arbitrarily whether to use a box plot, a line graph, or a histogram.
  • FIG. 12 is a block diagram showing the functional configuration of the information processing apparatus according to the second embodiment.
  • the information processing device 70 includes input data acquisition means 71 , input data evaluation means 72 , prediction data acquisition means 73 , prediction data evaluation means 74 , prediction accuracy acquisition means 75 , and display means 76 .
  • FIG. 13 is a flowchart of processing by the information processing apparatus of the second embodiment.
  • the input data obtaining means 71 obtains a plurality of input data (step S71).
  • the input data evaluation means 72 detects outliers and abnormal values in the input data (step S72).
  • the prediction data acquisition unit 73 acquires prediction data generated from the input data using the trained model (step S73).
  • the predicted data evaluation means 74 detects abnormal values in the predicted data (step S74).
  • the prediction accuracy acquisition unit 75 acquires the prediction accuracy of the learned model (step S75). Note that steps S71 to S72, steps S73 to S74, and step S75 may be performed in a different order from the above, or may be performed in parallel in terms of time.
  • the display means 76 displays diagnostic information including the presence or absence of outliers and abnormal values in the input data, the presence or absence of abnormal values in the prediction data, and the prediction accuracy (step S76).
  • the information processing device 70 of the second embodiment it is possible to create and present diagnostic information regarding multiple factors that affect the prediction accuracy for the model in operation.
  • the diagnostic information includes: a diagnostic report describing the presence or absence of outliers and abnormal values in the input data, the presence or absence of abnormal values in the prediction data, and the prediction accuracy; an analysis result screen displaying the input data, the prediction data, and the prediction accuracy on a graph, respectively;
  • the information processing device comprising:
  • appendix 3 The information according to appendix 2, wherein the analysis result screen displays the presence or absence of outliers and abnormal values in the input data, the presence or absence of abnormal values in the prediction data, and the prediction accuracy, respectively, on time-series graphs. processing equipment.
  • Appendix 4 The information processing according to appendix 2 or 3, wherein the analysis result screen displays the presence or absence of outliers and abnormal values in the input data, the presence or absence of abnormal values in the prediction data, and the prediction accuracy, respectively, on histograms. Device.
  • the analysis result screen includes a plurality of graphs showing the presence or absence of outliers and abnormal values in the input data for each of the plurality of trained models, and the presence or absence of abnormal values in the prediction data for each of the plurality of trained models. 6.
  • the information processing apparatus according to any one of appendices 2 to 5, wherein a plurality of graphs and a plurality of graphs showing the prediction accuracy for each of the plurality of trained models are displayed side by side.
  • the analysis result screen includes one graph that simultaneously displays the presence or absence of outliers and abnormal values in the input data for a plurality of trained models, and the presence or absence of abnormal values in the prediction data for each of the plurality of trained models. 6.
  • the information processing apparatus according to any one of Appendices 2 to 5, including one graph that simultaneously displays , and one graph that simultaneously displays the prediction accuracy for each of a plurality of trained models.
  • the diagnostic report includes countermeasures and corresponding countermeasures. 8.
  • the information processing apparatus according to any one of Appendices 2 to 7, including a description of the priority of measures.
  • a recording medium recording a program for causing a computer to execute processing for displaying diagnostic information including the presence or absence of outliers and abnormal values in the input data, the presence or absence of abnormal values in the predicted data, and the prediction accuracy.
  • prediction device 12 processor 16 display device 21 input data evaluation unit 22 prediction data evaluation unit 23 prediction accuracy evaluation unit 24 display data generation unit 100 monitoring device 200, 200a, 210, 210a, 220, 220a Analysis result screen

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

Moyen d'acquisition de données d'entrée qui acquiert une pluralité d'éléments de données d'entrée, et moyen d'évaluation de données d'entrée qui détecte des valeurs aberrantes et des valeurs anormales dans les données d'entrée. Moyen d'acquisition de données de prédiction qui acquiert des données de prédiction générées à partir des données d'entrée à l'aide d'un modèle formé, et moyen d'évaluation de données de prédiction qui détecte des valeurs anormales dans les données de prédiction. Moyen d'acquisition de précision de prédiction qui acquiert une précision de prédiction à partir d'un modèle formé. Moyen d'affichage qui affiche des informations de diagnostic comprenant le fait de savoir si les données d'entrée présentent des valeurs aberrantes ou des valeurs anormales, si les données de prédiction présentent des valeurs anormales, et la précision de prédiction.
PCT/JP2021/033932 2021-09-15 2021-09-15 Dispositif de traitement d'informations, procédé de traitement d'informations et support d'enregistrement WO2023042301A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023548001A JPWO2023042301A5 (ja) 2021-09-15 情報処理装置、情報処理方法、及び、プログラム
PCT/JP2021/033932 WO2023042301A1 (fr) 2021-09-15 2021-09-15 Dispositif de traitement d'informations, procédé de traitement d'informations et support d'enregistrement

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/033932 WO2023042301A1 (fr) 2021-09-15 2021-09-15 Dispositif de traitement d'informations, procédé de traitement d'informations et support d'enregistrement

Publications (1)

Publication Number Publication Date
WO2023042301A1 true WO2023042301A1 (fr) 2023-03-23

Family

ID=85602005

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/033932 WO2023042301A1 (fr) 2021-09-15 2021-09-15 Dispositif de traitement d'informations, procédé de traitement d'informations et support d'enregistrement

Country Status (1)

Country Link
WO (1) WO2023042301A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017217050A1 (fr) * 2016-06-16 2017-12-21 ソニー株式会社 Dispositif de traitement d'informations, procédé de traitement d'informations et support de stockage
JP2020014799A (ja) * 2018-07-27 2020-01-30 コニカミノルタ株式会社 Ｘ線画像物体認識システム
US20210097433A1 (en) * 2019-09-30 2021-04-01 Amazon Technologies, Inc. Automated problem detection for machine learning models

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017217050A1 (fr) * 2016-06-16 2017-12-21 ソニー株式会社 Dispositif de traitement d'informations, procédé de traitement d'informations et support de stockage
JP2020014799A (ja) * 2018-07-27 2020-01-30 コニカミノルタ株式会社 Ｘ線画像物体認識システム
US20210097433A1 (en) * 2019-09-30 2021-04-01 Amazon Technologies, Inc. Automated problem detection for machine learning models

Also Published As

Publication number Publication date
JPWO2023042301A1 (fr) 2023-03-23

Similar Documents

Publication Publication Date Title
Alaswad et al. A review on condition-based maintenance optimization models for stochastically deteriorating system
JP6756374B2 (ja) プロセスの異常状態診断装置および異常状態診断方法
JP2010506258A (ja) 分散型プロセス制御システムにおけるオンライン多変量解析
JP2007249997A (ja) 工業プロセスの監視方法及び監視システム
JP2018112903A (ja) プラント運転支援装置、プラント運転支援方法、プラント運転支援プログラム及び記録媒体
US20120116827A1 (en) Plant analyzing system
JP6711323B2 (ja) プロセスの異常状態診断方法および異常状態診断装置
US20220187813A1 (en) Hybrid ensemble approach for iot predictive modelling
JP2016539425A (ja) プロセスユニットの中の全プロセスセグメントの状態を自動的に監視し決定するための、コンピュータで実装される方法およびシステム
CN107992410A (zh) 软件质量监测方法、装置、计算机设备和存储介质
JP2012164314A (ja) 現象における変数の影響を決定する方法
US20210333768A1 (en) Control support apparatus, control support method, computer readable medium with control support program recorded thereon and control system
JPWO2018073960A1 (ja) 表示方法、表示装置、および、プログラム
JP2022035686A (ja) 診断装置及びパラメータ調整方法
JP6702297B2 (ja) プロセスの異常状態診断方法および異常状態診断装置
JP7318612B2 (ja) 監視装置、監視方法、および監視プログラム
Chien et al. Bayesian decision analysis for optimizing in-line metrology and defect inspection strategy for sustainable semiconductor manufacturing and an empirical study
WO2023042301A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et support d'enregistrement
WO2020066124A1 (fr) Dispositif d'analyse d'état de traitement et procédé d'analyse d'état de traitement
US20220147039A1 (en) Event analytics in modular industrial plants
WO2020262709A1 (fr) Dispositif de surveillance de système de traitement, procédé de système de traitement, et programme
JP7223947B2 (ja) 製造条件計算装置、製造条件計算方法及び製造条件計算プログラム
JP2022127958A (ja) 業務改善支援装置、プログラムおよびプログラムを格納した記憶媒体
US20240019827A1 (en) Energy-saving prediction method for factory electricity consumption and electronic apparatus
JP7052914B1 (ja) 異常診断システム、異常診断装置、異常診断方法、及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21957483

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023548001

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE