US20250053750A1 - Computer system and method for generating interpretation sentence - Google Patents

Computer system and method for generating interpretation sentence Download PDF

Info

Publication number
US20250053750A1
US20250053750A1 US18/791,514 US202418791514A US2025053750A1 US 20250053750 A1 US20250053750 A1 US 20250053750A1 US 202418791514 A US202418791514 A US 202418791514A US 2025053750 A1 US2025053750 A1 US 2025053750A1
Authority
US
United States
Prior art keywords
feature
relationship
computer system
sentence
inference
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US18/791,514
Other languages
English (en)
Inventor
Kohei Matsushita
Masaki Hamamoto
Masayoshi Mase
Masashi Egi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hitachi Ltd
Original Assignee
Hitachi Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Ltd filed Critical Hitachi Ltd
Assigned to HITACHI, LTD. reassignment HITACHI, LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: EGI, MASASHI, HAMAMOTO, MASAKI, MASE, MASAYOSHI, MATSUSHITA, KOHEI
Publication of US20250053750A1 publication Critical patent/US20250053750A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G06F40/35—Discourse or dialogue representation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/55—Rule-based translation
    • G06F40/56—Natural language generation

Definitions

  • the present disclosure relates to a computer system and a method for generating an interpretation sentence for an inference result of a machine learning model.
  • PTL 1 discloses a system that presents data useful for a user to interpret a basis of a prediction result output by AI.
  • PTL 1 discloses a “computer system that stores interpretation factor conversion information for managing an interpretation factor interpreting a basis of a prediction result for input data including values of a plurality of features, the interpretation factor being determined by the value of each of the plurality of features included in the input data and contribution values of the plurality of features included in the input data, calculates a prediction result for evaluation target data when the evaluation target data is input, calculates a contribution value of each of the plurality of features included in the evaluation target data, specifies a corresponding interpretation factor by referring to the interpretation factor conversion information based on the value and the contribution value of each of the plurality of features included in the evaluation target data, and generates and outputs display information for presenting the specified interpretation factor” (for example, see Abstract).
  • the present invention provides a system and a method that use a language model to present a business interpretation without requiring knowledge of an expert.
  • a representative example of the invention disclosed in this application is as follows. That is, a computer system, which is connected to an inference system that receives input data including a plurality of features and performs an inference using an inference model, stores a large-scale language model that receives an instruction sentence as an input and outputs an interpretation sentence for interpreting a result of the inference, and first text template information that stores first template data in which a characteristic of a contribution value indicating a magnitude of a contribution to the result of the inference of a feature in a group constituted with one or more features is associated with a template of the instruction sentence, calculates the contribution value of each of the plurality of features using the input data, the result of the inference, and the inference model, generates a plurality of groups each constituted with one or more of the features, acquires, for each of the plurality of groups, the first template data by referring to the first text template information based on the characteristic of the contribution value of the feature included in the group, generates, based on the acquired first template data and
  • the computer system can present an interpretation sentence for an inference result of an inference model. Problems, configurations, and effects other than those described above will be clarified by the description of the following embodiments.
  • FIG. 1 is a diagram showing an example of a hardware configuration of a computer system according to Embodiment 1;
  • FIG. 2 A is a diagram showing an example of a software configuration of an inference device according to Embodiment 1;
  • FIG. 2 B is a diagram showing an example of a software configuration of an interpretation sentence generation device according to Embodiment 1;
  • FIG. 3 is a diagram showing an example of a data structure of input data according to Embodiment 1;
  • FIG. 4 is a diagram showing an example of a data structure of contribution value information according to Embodiment 1;
  • FIG. 5 is a diagram showing an example of a data structure of feature information according to Embodiment 1;
  • FIG. 6 is a diagram showing an example of a data structure of a feature list according to Embodiment 1;
  • FIG. 7 is a diagram showing an example of a data structure of group information according to Embodiment 1;
  • FIG. 8 A is a diagram showing an example of a data structure of first text template information according to Embodiment 1;
  • FIG. 8 B is a diagram showing the example of the data structure of the first text template information according to Embodiment 1;
  • FIG. 9 is a diagram showing an example of a data structure of prompt information according to Embodiment 1;
  • FIG. 10 is a diagram showing an example of a data structure of analysis information according to Embodiment 1;
  • FIG. 11 is a diagram showing an example of a data structure of second text template information according to Embodiment 1;
  • FIG. 12 is a diagram showing an example of a data structure of relationship document information according to Embodiment 1;
  • FIG. 13 is a diagram showing an example of a data structure of a table for managing interpretation sentences according to Embodiment 1;
  • FIG. 14 is a flowchart illustrating an outline of processing executed by the interpretation sentence generation device according to Embodiment 1;
  • FIG. 15 is a flowchart illustrating an example of feature selection processing executed by the interpretation sentence generation device according to Embodiment 1;
  • FIG. 16 is a flowchart illustrating an example of prompt generation processing executed by the interpretation sentence generation device according to Embodiment 1;
  • FIG. 17 is a flowchart illustrating an example of relationship document generation processing executed by the interpretation sentence generation device according to Embodiment 1;
  • FIG. 18 is a flowchart illustrating an example of filtering processing executed by the interpretation sentence generation device according to Embodiment 1;
  • FIG. 19 is a diagram showing an example of a screen displayed by a feature presentation unit according to Embodiment 1;
  • FIG. 20 is a diagram showing an example of a screen displayed by a prompt presentation unit according to Embodiment 1;
  • FIG. 21 is a diagram showing an example of a screen displayed by an analysis result presentation unit according to Embodiment 1;
  • FIG. 22 is a diagram showing an example of a screen displayed by a result presentation unit according to Embodiment 1.
  • a system of one embodiment of the present specification may be a physical computer system (one or more physical computers) or may be a system built on a calculation resource group (plurality of calculation resources) such as a cloud infrastructure.
  • the computer system or the calculation resource group may include one or more interface devices (including, for example, a communication device and an input and output device), one or more storage devices (including, for example, a memory (main memory) and an auxiliary storage device), and one or more calculation devices.
  • a function When a function is implemented by the calculation device executing a program including an instruction code, predetermined processing is executed appropriately using the storage device and/or the interface device or the like, and thus the function may be at least a part of the calculation device. Processing described with the function as a subject may be processing executed by the calculation device or a system including the calculation device.
  • the program may be installed from a program source.
  • the program source may be, for example, a program distribution computer or a computer-readable storage medium (for example, a computer-readable non-transitory storage medium).
  • a description of each function is an example, and a plurality of functions may be integrated into one function, or one function may be divided into a plurality of functions.
  • FIG. 1 is a diagram showing an example of a hardware configuration of a computer system according to Embodiment 1.
  • the computer system shown in FIG. 1 includes an inference device 110 , an interpretation sentence generation device 100 , and a user terminal 120 .
  • the devices are connected to each other via a network 130 .
  • Any type of the network 130 is possible, and for example, a wide area network (WAN), a local area network (LAN), and the like are possible.
  • a connection method of the network 130 may be wired or wireless.
  • the inference device 110 performs various inferences using an inference model.
  • an inference model for predicting a fire probability of a property will be described as an example.
  • the inference device 110 includes a CPU 111 , a memory 112 , an auxiliary storage device 113 , and a network interface 114 as hardware.
  • the hardware elements communicate with each other via an internal bus.
  • the CPU 111 executes a program stored in the memory 112 .
  • the memory 112 stores a program executed by the CPU 111 and information necessary for the program.
  • the memory 112 includes a work area temporarily used by the program.
  • the auxiliary storage device 113 permanently stores data.
  • the auxiliary storage device 113 is considered to be a storage medium such as a hard disk drive (HDD) or a solid state drive (SSD), a non-volatile memory, or the like.
  • the program and the information stored in the memory 112 may be stored in the auxiliary storage device 113 .
  • the CPU 111 reads the program and the information from the auxiliary storage device 113 , loads the program and the information into the memory 112 , and executes the program loaded in the memory 112 .
  • the network interface 114 is connected to other devices via a network.
  • the interpretation sentence generation device 100 generates an interpretation sentence reflecting business knowledge to which an inference is applied from an inference result, and presents the interpretation sentence to a user who uses the inference device 110 .
  • the interpretation sentence generation device 100 includes a CPU 101 , a memory 102 , an auxiliary storage device 103 , and a network interface 104 as hardware.
  • the hardware elements communicate with each other via an internal bus or the like.
  • the CPU 101 , the memory 102 , the auxiliary storage device 103 , and the network interface 104 are hardware elements similar to the CPU 111 , the memory 112 , the auxiliary storage device 113 , and the network interface 114 .
  • the user terminal 120 is a terminal used by the user.
  • the user terminal 120 receives a user input for generating an interpretation sentence of an inference result, and presents the generated interpretation sentence to the user.
  • the user terminal 120 includes a CPU 121 , a memory 122 , an auxiliary storage device 123 , a network interface 124 , an input device 125 , and an output device 126 as hardware.
  • the hardware elements communicate with each other via an internal bus.
  • the CPU 121 , the memory 122 , the auxiliary storage device 123 , and the network interface 124 are hardware elements similar to the CPU 111 , the memory 112 , the auxiliary storage device 113 , and the network interface 114 .
  • the input device 125 is a device for inputting data and the like, and includes a keyboard, a mouse, a touch panel, and the like.
  • the output device 126 is a device for outputting data and the like, and includes a display, a touch panel, and the like.
  • the CPU executes processing according to the program to operate as a functional unit having a predetermined function.
  • processing when processing is described with the program as a subject, it indicates that the CPU or a device on which the CPU is mounted executes the program for implementing the functional unit.
  • different computers execute tasks of operation management, decision-making support, and user interface, respectively. In other examples, a combination of all or some of these tasks may be executed by one computer.
  • the inference device 110 and the interpretation sentence generation device 100 may be implemented as a virtual computer operating on one computer.
  • the computer system may be implemented by one or more computers including one or more calculation devices and one or more storage devices including a non-transitory storage medium.
  • a memory, an auxiliary storage device, or a combination thereof is the storage device.
  • the CPU is an example of the calculation device.
  • the calculation device may be implemented by a single processing unit or a plurality of processing units, and may include a single or a plurality of calculation units or a plurality of processing cores.
  • the calculation device may be implemented as one or more central processing units, a microprocessor, a microcomputer, a microcontroller, a digital signal processor, a state machine, a logic circuit, a graphics processing device, a chip-on-system, and/or any device for operating a signal based on a control instruction.
  • FIG. 2 A is a diagram showing an example of a software configuration of the inference device 110 according to Embodiment 1.
  • the inference device 110 includes a prediction unit 201 that performs prediction using the inference model.
  • the prediction unit 201 executes processing using the inference model on input data 202 , and outputs a prediction value 203 .
  • FIG. 2 B is a diagram showing an example of a software configuration of the interpretation sentence generation device 100 according to Embodiment 1.
  • the interpretation sentence generation device 100 includes an explainable artificial intelligence (XAI) unit 211 , a selection unit 212 , a feature presentation unit 213 , a group generation unit 221 , a prompt generation unit 222 , a prompt presentation unit 223 , an analysis unit 231 , an analysis result presentation unit 232 , a relationship document generation unit 233 , an interpretation sentence generation unit 241 , a filtering unit 242 , and a result presentation unit 243 .
  • XAI explainable artificial intelligence
  • the XAI unit 211 acquires the input data 202 and the prediction value 203 from the inference device 110 , and calculates a contribution value of each feature of the input data 202 for the prediction value 203 .
  • the contribution value of each feature is output as contribution value information 214 .
  • the XAI unit 211 calculates the contribution value using a known algorithm. For example, Shapley additive explanation (SHAP), local interpretable model-agnostic explanations (LIME), integrated gradient, or the like may be used.
  • SHAP Shapley additive explanation
  • LIME local interpretable model-agnostic explanations
  • integrated gradient or the like may be used.
  • the selection unit 212 acquires the contribution value information 214 from the XAI unit 211 , refers to feature information 215 , verbalizes the feature, and selects a feature used for generating an interpretation sentence based on the contribution value of each feature.
  • the selected verbalized features are output as a feature list 216 .
  • a feature that is verbalized is referred to as a verbalized feature.
  • the feature presentation unit 213 displays the verbalized feature selected by the selection unit 212 on a screen to the user terminal 120 .
  • the group generation unit 221 acquires the feature list 216 from the selection unit 212 and generates a plurality of groups having the verbalized features as elements.
  • the group of the verbalized features is constituted with at least one verbalized feature.
  • the generated group is output as group information 224 .
  • the prompt generation unit 222 acquires the group information 224 from the group generation unit 221 , refers to first text template information 225 , and generates a prompt for each group.
  • the prompt is an instruction sentence to be input to a large-scale language model to be described later.
  • the prompt generation unit 222 outputs prompt information 226 that stores the prompt of each group.
  • the prompt presentation unit 223 displays the prompt generated by the prompt generation unit 222 on a screen to the user terminal 120 .
  • the analysis unit 231 acquires the feature list 216 from the selection unit 212 , and analyzes a relationship between a value of the feature and the contribution value.
  • the analysis unit 231 uses the XAI unit 211 to generate, as an analysis result, a graph representing a relationship between a value of a selected feature and a contribution value for a prediction value (fire probability).
  • the analysis result presentation unit 232 displays the analysis result of the analysis unit 231 on a screen to the user terminal 120 .
  • the user checks the screen, and inputs information related to the relationship as necessary using the user terminal 120 .
  • the analysis unit 231 outputs the information input via the user terminal 120 and the analysis result as analysis information 234 .
  • the relationship document generation unit 233 acquires the analysis information 234 from the analysis unit 231 , refers to second text template information 235 , and generates one or more relationship documents by verbalizing the relationship included in the analysis information 234 .
  • the relationship document generation unit 233 outputs relationship document information 236 that stores one or more relationship documents.
  • the interpretation sentence generation unit 241 acquires the prompt information 226 from the prompt generation unit 222 , and generates an interpretation sentence by inputting the prompt to the large-scale language model.
  • the large-scale language model for example, generative pre-trained transformer-4 (GPT-4), pathways language model (PaLM), or the like may be used.
  • the invention is not limited to a type of the large-scale language model.
  • the filtering unit 242 acquires the relationship document information 236 from the relationship document generation unit 233 , and excludes, from interpretation sentences 244 to be output, the interpretation sentence 244 including a fact inconsistent with a fact expressed by the relationship document.
  • the result presentation unit 243 displays the interpretation sentence 244 not excluded by the filtering unit 242 on a screen.
  • FIG. 3 is a diagram showing an example of a data structure of the input data 202 according to Embodiment 1.
  • the input data 202 includes a plurality of entries each including a feature ID 301 and a value 302 .
  • One entry corresponds to one feature.
  • the feature ID 301 is a field for storing an ID of the feature.
  • the value 302 is a field for storing a value of the feature.
  • the input data 202 is, for example, property data, and includes an age of a building, a construction method, and the like as features.
  • FIG. 4 is a diagram showing an example of a data structure of the contribution value information 214 according to Embodiment 1.
  • the contribution value information 214 includes a plurality of entries each including a feature ID 401 , a value 402 , and a contribution value 403 .
  • One entry corresponds to one feature.
  • the feature ID 401 and the value 402 are the same fields as the feature ID 301 and the value 302 , respectively.
  • the contribution value 403 is a field for storing a contribution value.
  • FIG. 5 is a diagram showing an example of a data structure of the feature information 215 according to Embodiment 1.
  • the feature information 215 stores entries each including a feature ID 501 , a feature name 502 , a value 503 , and a verbalized template 504 .
  • the feature ID 501 is the same field as the feature ID 301 .
  • the feature name 502 is a field for storing a name of the feature.
  • the value 503 is a field for storing a value of the feature.
  • One feature includes as many rows as the number of values that the feature can take.
  • the verbalized template 504 is a field for storing a template for verbalizing the value of the feature.
  • the template is a text in which the value of the feature is expressed in a language, and is set by the user based on the value and meaning of the feature.
  • FIG. 6 is a diagram showing an example of a data structure of the feature list 216 according to Embodiment 1.
  • the feature list 216 stores entries each including a feature ID 601 , a value 602 , a feature name 603 , a verbalized feature 604 , and a contribution value 605 .
  • One entry corresponds to one feature.
  • the feature ID 601 and the value 602 are the same fields as the feature ID 301 and the value 302 , respectively.
  • the feature name 603 is the same field as the feature name 502 .
  • the verbalized feature 604 is a field for storing a verbalized feature.
  • the contribution value 605 is the same field as the contribution value 403 .
  • FIG. 7 is a diagram showing an example of a data structure of the group information 224 according to Embodiment 1.
  • the group information 224 stores entries each including a group ID 701 , a feature ID list 702 , a first verbalized feature list 703 , a second verbalized feature list 704 , and a contribution value sum 705 .
  • One entry corresponds to one group.
  • the group ID 701 is a field for storing an ID of the group.
  • the feature ID list 702 is a field for storing a list of IDs of features that constitute the group.
  • the first verbalized feature list 703 stores the verbalized feature whose contribution value is positive.
  • the second verbalized feature list 704 stores the verbalized feature whose contribution value is negative.
  • the contribution value sum 705 stores a sum of the contribution values of the features constituting the group.
  • FIGS. 8 A and 8 B are diagrams showing an example of a data structure of the first text template information 225 according to Embodiment 1.
  • the first text template information 225 includes a template table 800 that stores templates each corresponding to a preceding sentence of the prompt, and a template table 810 that stores templates each corresponding to a main sentence of the prompt.
  • the template table 800 stores entries each including a contribution value sum type 801 and a text template 802 .
  • the contribution value sum type 801 is a field for storing a type of the sum of the contribution values of the group.
  • the text template 802 is a field for storing a template of the preceding sentence corresponding to the type of the sum of the contribution values of the group.
  • the template table 810 stores entries each including a contribution value sum 811 and a text template 812 .
  • the contribution value sum 811 is a field for storing a range of the sum of the contribution values of the group.
  • the text template 812 is a field for storing a template of the main sentence of the prompt corresponding to the sum of the contribution values. The main sentence is set in advance by the user.
  • the main sentence is a question sentence, but is not limited thereto.
  • an interpretation generation model may generate the following document as “a reason why is there a high possibility of a fire”.
  • a classification of the range of the contribution value shown in FIG. 8 B is an example, and is not limited thereto.
  • FIG. 9 is a diagram showing an example of a data structure of the prompt information 226 according to Embodiment 1.
  • the prompt information 226 stores entries each including a group ID 901 and a prompt 902 . One entry corresponds to one group.
  • the group ID 901 is the same field as the group ID 701 .
  • the prompt 902 is a field for storing a prompt generated based on the preceding sentence and the main sentence.
  • the prompt is an input of the interpretation sentence generation unit 241 .
  • FIG. 10 is a diagram showing an example of a data structure of the analysis information 234 according to Embodiment 1.
  • the analysis information 234 stores entries each including a feature ID 1001 , a feature name 1002 , a relationship 1003 , and a remark 1004 .
  • the feature ID 1001 is the same field as the feature ID 301 .
  • the feature name 1002 is a field for storing a name of the feature.
  • the relationship 1003 is a field for storing information indicating a relationship between the value of the feature and the contribution value. Possible relationships include monotonically increasing, monotonically decreasing, rapidly increasing, and the like.
  • the remark 1004 is a field for storing information related to the relationship. For example, a value of a feature whose contribution value increases rapidly is stored.
  • FIG. 11 is a diagram showing an example of a data structure of the second text template information 235 according to Embodiment 1.
  • the second text template information 235 stores entries each including a relationship 1101 and a text template 1102 .
  • the relationship 1101 is a field for storing a relationship between the value of the feature and the contribution value.
  • the text template 1102 is a field for storing a template of a text in which the relationship is verbalized.
  • FIG. 12 is a diagram showing an example of a data structure of the relationship document information 236 according to Embodiment 1.
  • the relationship document information 236 stores entries each including a feature ID 1201 , a feature name 1202 , and a relationship document 1203 . One entry corresponds to one relationship document.
  • the feature ID 1201 and the feature name 1202 are the same fields as the feature ID 1001 and the feature name 1002 , respectively.
  • the relationship document 1203 is a field for storing a document in which a relationship between the value of the feature and the contribution value is verbalized.
  • FIG. 13 is a diagram showing an example of a data structure of a table for managing the interpretation sentences 244 according to Embodiment 1.
  • a table 1300 stores entries each including a group ID 1301 , a prompt 1302 , and an interpretation sentence 1303 .
  • One entry corresponds to one group.
  • the group ID 1301 and the prompt 1302 correspond to the group ID 901 and the prompt 902 , respectively.
  • the interpretation sentence 1303 is a field for storing the interpretation sentence 244 obtained by inputting the prompt.
  • the interpretation sentence generation device 100 generates a business interpretation for the prediction value 203 and presents the business interpretation to the user.
  • FIG. 14 is a flowchart illustrating an outline of the processing executed by the interpretation sentence generation device 100 according to Embodiment 1.
  • an interpretation sentence for the prediction of the fire probability will be described as an example.
  • the XAI unit 211 of the interpretation sentence generation device 100 calculates a contribution value of each feature of the input data 202 by using the input data 202 and the prediction value 203 (step S 101 ).
  • SHAP As a method of calculating the contribution value, for example, SHAP can be used.
  • the SHAP uses a base line serving as a reference for calculating the contribution value.
  • the XAI unit 211 determines the contribution value of each feature of the property data based on a relative value of the feature with a value of the base line as a reference. In this example, the XAI unit 211 calculates the contribution value of each feature of the property data (input data 202 ) input for calculating the fire probability.
  • the interpretation sentence generation device 100 executes feature selection processing for selecting a feature to be used for generating a prompt (step S 102 ).
  • FIG. 15 is a flowchart illustrating an example of the feature selection processing executed by the interpretation sentence generation device 100 according to Embodiment 1.
  • the selection unit 212 acquires the contribution value information 214 (step S 201 ).
  • the selection unit 212 refers to the feature information 215 and verbalizes a value of the feature (step S 202 ).
  • the selection unit 212 refers to an entry with a matching feature ID for each feature, and acquires the verbalized template 504 of a row corresponding to the value 503 .
  • the selection unit 212 may use the verbalized template as it is, or may modify the template according to the value of the feature as necessary.
  • the selection unit 212 selects the feature based on the contribution value of each feature (step S 203 ).
  • the selection unit 212 selects a predetermined number of features in descending order of an absolute value of the contribution value. Accordingly, it is possible to efficiently select features that are considered important by the inference model.
  • the selection unit 212 generates ranking information for storing entries each including a verbalized feature and the contribution value of the selected feature. The entries of the ranking information are sorted in descending order of the absolute value of the contribution value.
  • the feature presentation unit 213 presents the selected feature (step S 204 ). That is, the ranking information is presented.
  • FIG. 19 is a diagram showing an example of a screen displayed by the feature presentation unit 213 according to Embodiment 1.
  • the feature presentation unit 213 displays a screen 1900 on the user terminal 120 .
  • the screen 1900 includes sections 1901 and 1903 , ranking information 1902 , and an operation button 1904 .
  • the section 1901 is a section that displays the prediction value 203 .
  • the section 1903 is a section that displays the number of patterns of an interpretation sentence generated based on the selected feature. The number of patterns of the interpretation sentence corresponds to the number of groups.
  • the ranking information 1902 stores entries each including a check box for the user to select a feature to be used, a ranking, a verbalized feature, and a contribution value.
  • the selection unit 212 selects the feature to be used based on a magnitude of the absolute value of the contribution value, and adds a check to the check box of the selected feature.
  • the user can also operate the check box to select the feature.
  • the user can acquire information for generating an interpretation sentence that the user wants.
  • the number of patterns of the interpretation sentence displayed in the section 1903 is large, the user can reduce the number of features to be used.
  • the selection unit 212 generates and outputs the feature list 216 based on an operation on the screen 1900 (step S 205 ).
  • the interpretation sentence generation device 100 executes prompt generation processing using the selected feature (step S 103 ).
  • FIG. 16 is a flowchart illustrating an example of the prompt generation processing executed by the interpretation sentence generation device 100 according to Embodiment 1.
  • the group generation unit 221 acquires the feature list 216 (step S 301 ).
  • the group generation unit 221 generates a group by combining the features registered in the feature list 216 (step S 302 ).
  • the group generation unit 221 registers information on the generated group in the group information 224 .
  • groups are generated as many as the number of possible combinations of features. As to be described later, a prompt is generated for each group. Therefore, a variety of interpretation sentences 244 can be generated.
  • the prompt generation unit 222 refers to the first text template information 225 and generates a prompt for each group (step S 303 ).
  • the prompt generation unit 222 refers to the template table 800 in FIG. 8 A and generates a preceding sentence based on the sum of the contribution values of each group.
  • the prompt generation unit 222 refers to the template table 810 in FIG. 8 B and generates a question sentence based on the sum of the contribution values of each group.
  • the prompt generation unit 222 generates, as the prompt, a document in which the preceding sentence and the question sentence are combined.
  • the first text template information 225 according to Embodiment 1 is an example, and may include, for example, only one table or may include three or more tables. A table in which characteristics of the contribution values other than the sum of the contribution values are associated with the templates may be used.
  • the first text template information 225 can generate a different prompt for each group. For example, by using the template table 800 , it is possible to include an adversative conjunction corresponding to the positive or negative of the contribution value in the preceding sentence. This information represents a result of XAI, and the prompt can include information on the target inference model.
  • the prompt presentation unit 223 presents the prompt generated by the prompt generation unit 222 (step S 304 ).
  • FIG. 20 is a diagram showing an example of a screen displayed by the prompt presentation unit 223 according to Embodiment 1.
  • the prompt presentation unit 223 displays a screen 2000 on the user terminal 120 .
  • the screen 2000 includes a table 2001 corresponding to the feature list 216 , a table 2002 corresponding to the prompt information 226 , and an operation button 2003 .
  • the table 2001 stores entries each including, for example, a name of a feature, a verbalized feature, and a contribution value.
  • the entry may include a field for storing a value of the feature.
  • the table 2002 stores entries each including, for example, a check box and a prompt.
  • the entry may include a field for storing the sum of the contribution values of each group and the like.
  • the check box is used to select a prompt to be input.
  • an interpretation sentence is generated for every prompt, but the user may select a prompt to be used by using the check box.
  • An input format may be used in which the prompt can be corrected.
  • the prompt presentation unit 223 generates and outputs the prompt information 226 based on an operation on the screen 2000 (step S 305 ).
  • the interpretation sentence generation device 100 determines whether a feature to be analyzed is included in the selected feature (step S 104 ). For example, in the prediction of the fire probability, an age of a building, a floor area, or the like is the feature to be analyzed. The feature to be analyzed is specified in advance by the user.
  • the interpretation sentence generation device 100 acquires the prompt information 226 and generates the interpretation sentence 244 (step S 105 ). Specifically, the interpretation sentence generation unit 241 generates the interpretation sentence 244 by inputting the prompt to the large-scale language model.
  • the interpretation sentence generation device 100 executes relationship document generation processing (step S 106 ).
  • FIG. 17 is a flowchart illustrating an example of the relationship document generation processing executed by the interpretation sentence generation device 100 according to Embodiment 1.
  • the analysis unit 231 acquires the feature list 216 (step S 401 ).
  • the analysis unit 231 starts loop processing of the feature (step S 402 ). Specifically, the analysis unit 231 selects one feature from the feature list 216 .
  • the analysis unit 231 determines whether the selected feature is the feature to be analyzed (step S 403 ).
  • step S 406 the analysis unit 231 proceeds to step S 406 .
  • the analysis unit 231 analyzes a relationship between the value of the feature and the contribution value (step S 404 ). Specifically, the analysis unit 231 uses the XAI to analyze a change in a contribution value when a value of a target feature is changed, and depicts the change in a graph.
  • the analysis result presentation unit 232 presents an analysis result (step S 405 ).
  • FIG. 21 is a diagram showing an example of a screen displayed by the analysis result presentation unit 232 according to Embodiment 1.
  • the analysis result presentation unit 232 displays a screen 2100 on the user terminal 120 .
  • the screen 2100 includes a graph 2101 , input sections 2102 and 2103 , and operation buttons 2104 and 2105 .
  • the graph 2101 is a graph showing a relationship between the value of the feature and the contribution value.
  • a horizontal axis indicates the value of the feature, and a vertical axis indicates the contribution value.
  • the graph 2101 clearly shows the value of the feature in the input data 202 to be described. For example, the value of the feature is shown by a dotted line in FIG. 21 .
  • the input sections 2102 and 2103 are sections for the user to confirm the graph and input the relationship.
  • the input section 2102 is used to input a global relationship. For example, the user selects a corresponding relationship from options such as monotonically increasing, monotonically decreasing, and none.
  • the input section 2103 is used to input a local relationship. When there are a plurality of local relationships, the input section 2103 is added by pressing the operation button 2104 . A change amount or the like may be added to the input section 2103 .
  • the analysis unit 231 determines whether the processing is completed for all the features (step S 406 ).
  • the analysis unit 231 returns to step S 402 and selects a new feature.
  • the analysis unit 231 When it is determined that the processing is completed for all the features (S 406 : YES), the analysis unit 231 outputs the analysis information 234 (step S 407 ).
  • the relationship document generation unit 233 refers to the second text template information 235 and verbalizes the relationship included in the analysis information 234 to generate and output the relationship document information 236 (step S 408 ).
  • step S 107 The processing of step S 107 is the same as the processing of step S 105 .
  • the filtering unit 242 executes filtering processing of the interpretation sentence 244 using the relationship document (step S 108 ).
  • FIG. 18 is a flowchart illustrating an example of the filtering processing executed by the interpretation sentence generation device 100 according to Embodiment 1.
  • the filtering unit 242 acquires the interpretation sentence 244 and the relationship document information 236 (step S 501 ).
  • the filtering unit 242 starts loop processing of the interpretation sentence 244 (step S 502 ). Specifically, the filtering unit 242 selects one interpretation sentence 244 from the interpretation sentences 244 .
  • the filtering unit 242 starts loop processing of the relationship document (step S 503 ). Specifically, the filtering unit 242 selects one relationship document from the relationship document information 236 . Only a relationship document indicating a global relationship may be selected.
  • the filtering unit 242 determines whether the selected interpretation sentence 244 includes a feature that is a target of the selected relationship document (step S 504 ).
  • the filtering unit 242 proceeds to step S 507 .
  • the filtering unit 242 determines whether the interpretation sentence 244 includes a fact inconsistent with the relationship document (step S 505 ). For example, by performing recognizing textual entailment (RTE) using BERT, which is one of language models, it is possible to determine whether there is an entailment or inconsistence between two documents.
  • RTE recognizing textual entailment
  • BERT which is one of language models
  • the filtering unit 242 proceeds to step S 507 .
  • the filtering unit 242 excludes the interpretation sentence 244 from an output target (step S 506 ). Then, the filtering unit 242 ends the loop on the relationship document and proceeds to step S 508 .
  • the relationship document represents the relationship between the value of the selected feature that is considered important by the inference model to be described and the contribution value. Since a behavior for the inference result of the target feature is not input to the interpretation generation model, there is a possibility that the interpretation sentence 244 including the fact inconsistent with the relationship document is output. Such the interpretation sentence 244 is not suitable for the interpretation sentence 244 for the inference result of the inference model, and thus is excluded in advance.
  • step S 507 the filtering unit 242 determines whether the processing is completed for all the relationship documents (step S 507 ).
  • the filtering unit 242 When it is determined that the processing is not completed for all the relationship documents (S 507 : NO), the filtering unit 242 returns to step S 503 and selects a new relationship document.
  • the filtering unit 242 proceeds to step S 508 .
  • step S 508 the filtering unit 242 determines whether the processing is completed for all the interpretation sentences 244 (step S 508 ).
  • the filtering unit 242 When it is determined that the processing is not completed for all the interpretation sentences 244 (S 508 : NO), the filtering unit 242 returns to step S 502 and selects a new interpretation sentence 244 .
  • the filtering unit 242 ends the filtering processing.
  • the above is the description of the filtering processing. Returning to the description of FIG. 14 .
  • the result presentation unit 243 acquires and outputs the interpretation sentence 244 (step S 109 ).
  • FIG. 22 is a diagram showing an example of a screen 2200 displayed by the result presentation unit 243 according to Embodiment 1.
  • the result presentation unit 243 displays the screen 2200 on the user terminal 120 .
  • the screen 2200 displays prompts and interpretation sentences.
  • the interpretation sentence generation device 100 can generate and present an interpretation sentence reflecting business knowledge without receiving an input of specialized knowledge from a user.
  • the invention is not limited to the above-described embodiment, and includes various modifications.
  • the above-described embodiment has been described in detail to facilitate understanding of the invention, and the invention is not necessarily limited to those including all the described configurations.
  • a part of a configuration in each embodiment can be added to, deleted from, or replaced with another configuration.
  • a part or all of the configurations, functions, processing units, processing methods, and the like described above may be implemented by hardware by, for example, designing with an integrated circuit.
  • the invention can also be implemented by a program code of software for implementing the functions of the embodiment.
  • a storage medium storing the program code is provided to a computer, and a processor provided in the computer reads the program code stored in the storage medium.
  • the program code read from the storage medium implements the functions of the embodiment described above by itself, and the program code itself and the storage medium storing the program code constitute the invention.
  • Examples of the storage medium for supplying such a program code include a flexible disk, a CD-ROM, a DVD-ROM, a hard disk, a solid state drive (SSD), an optical disk, a magneto-optical disk, a CD-R, a magnetic tape, a nonvolatile memory card, and a ROM.
  • a flexible disk a CD-ROM, a DVD-ROM, a hard disk, a solid state drive (SSD), an optical disk, a magneto-optical disk, a CD-R, a magnetic tape, a nonvolatile memory card, and a ROM.
  • program code for implementing the functions described in the present embodiment can be implemented in a wide range of programs or script languages such as assembler, C/C++, Perl, Shell, PHP, Python, and Java (registered trademark).
  • program code of software for implementing the functions of the embodiment may be distributed via a network to be stored in a storage unit such as a hard disk or a memory of a computer or a storage medium such as a CD-RW or a CD-R, and a processor provided in the computer may read and execute the program code stored in the storage unit or the storage medium.
  • a storage unit such as a hard disk or a memory of a computer or a storage medium such as a CD-RW or a CD-R
  • a processor provided in the computer may read and execute the program code stored in the storage unit or the storage medium.
  • Control lines and information lines considered to be necessary for description are shown in the embodiment described above, and not all control lines and information lines in a product are necessarily shown. All the configurations may be connected to one another.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Machine Translation (AREA)
US18/791,514 2023-08-10 2024-08-01 Computer system and method for generating interpretation sentence Pending US20250053750A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2023131099A JP2025025887A (ja) 2023-08-10 2023-08-10 計算機システム及び解釈文の生成方法
JP2023-131099 2023-08-10

Publications (1)

Publication Number Publication Date
US20250053750A1 true US20250053750A1 (en) 2025-02-13

Family

ID=94482095

Family Applications (1)

Application Number Title Priority Date Filing Date
US18/791,514 Pending US20250053750A1 (en) 2023-08-10 2024-08-01 Computer system and method for generating interpretation sentence

Country Status (2)

Country Link
US (1) US20250053750A1 (https=)
JP (1) JP2025025887A (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120822624A (zh) * 2025-09-18 2025-10-21 山东高速集团有限公司 一种面向高速公路场景的可解释分层推理多模态方法和系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080120257A1 (en) * 2006-11-20 2008-05-22 Yahoo! Inc. Automatic online form filling using semantic inference
US11386342B2 (en) * 2018-04-20 2022-07-12 H2O.Ai Inc. Model interpretation
US20240394481A1 (en) * 2023-05-22 2024-11-28 Sage Global Services Limited Prompt Generation

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080120257A1 (en) * 2006-11-20 2008-05-22 Yahoo! Inc. Automatic online form filling using semantic inference
US11386342B2 (en) * 2018-04-20 2022-07-12 H2O.Ai Inc. Model interpretation
US20240394481A1 (en) * 2023-05-22 2024-11-28 Sage Global Services Limited Prompt Generation

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120822624A (zh) * 2025-09-18 2025-10-21 山东高速集团有限公司 一种面向高速公路场景的可解释分层推理多模态方法和系统

Also Published As

Publication number Publication date
JP2025025887A (ja) 2025-02-21

Similar Documents

Publication Publication Date Title
US11551818B2 (en) Computer system and method of presenting information related to basis of predicted value output by predictor
US11487972B2 (en) Reward function generation method and computer system
US11132385B2 (en) Information processing device and information processing method
CN112000330B (zh) 建模参数的配置方法、装置、设备和计算机存储介质
CN113378007B (zh) 数据回溯方法及装置、计算机可读存储介质、电子设备
CN107169534A (zh) 模型训练方法及装置、存储介质、电子设备
JP2022529178A (ja) 人工知能推奨モデルの特徴処理方法、装置、電子機器、及びコンピュータプログラム
US20230186092A1 (en) Learning device, learning method, computer program product, and learning system
CN111656453A (zh) 用于信息提取的层次实体识别和语义建模框架
US20250053750A1 (en) Computer system and method for generating interpretation sentence
CN115469849B (zh) 一种业务处理系统、方法、电子设备和存储介质
US20250378127A1 (en) Method and system for navigation control to select a target page from possible target pages
CN106484389A (zh) 动作流分段管理
CN118860392A (zh) 基于Flutter组件的界面编辑方法、装置、设备以及存储介质
US20220308841A1 (en) Enabling custom software development by domain experts
JP2023079183A (ja) 情報処理システム、情報処理方法及び情報処理プログラム
CN119960731A (zh) 基于大模型的汽车软件开发方法、装置、设备及存储介质
CN119719138A (zh) 数据库语句生成方法、装置、设备、介质和程序产品
JP5600826B1 (ja) 非構造化データ処理システム、非構造化データ処理方法およびプログラム
CN119166526A (zh) 基于智能体的测试方法、装置、电子设备及存储介质
US20230213897A1 (en) Information processing device, information processing method, and computer program product
JP6869082B2 (ja) テストケースを選択する計算機及びテストケースの選択方法
US12032937B2 (en) Programming support program for preventing work efficiency from lowering as a result of conversion
JP2014048745A (ja) ユースケースシナリオ作成支援装置、ユースケースシナリオ作成支援方法、およびユースケースシナリオ作成支援プログラム
CN113190154A (zh) 模型训练、词条分类方法、装置、设备、存储介质及程序

Legal Events

Date Code Title Description
AS Assignment

Owner name: HITACHI, LTD., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MATSUSHITA, KOHEI;HAMAMOTO, MASAKI;MASE, MASAYOSHI;AND OTHERS;SIGNING DATES FROM 20240702 TO 20240723;REEL/FRAME:068148/0446

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

Free format text: NON FINAL ACTION COUNTED, NOT YET MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION COUNTED, NOT YET MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED