JP2021002250A - Determination method, device, and program - Google Patents

Abstract

To provide a determination method, a device, and a program that optimize a difficulty level assigned to a word or a word string for determining a user skill.SOLUTION: A change unit sets, according to a click rate when a message including a descriptive phrase ranked according to a difficulty level is presented to a first user corresponding to a rank thereof, a temporary rank, and a determination unit determines, according to a temporary rank click rate when a message including the descriptive phrase is presented to a second user corresponding to a temporary rank, whether or not to determine the temporary rank. When a rank of the description phrase is updated, the determination unit updates a rank of terms, which are included in the descriptive phrase, used to determine a user skill.SELECTED DRAWING: Figure 2

Description

The disclosed technique relates to a determination method, a determination device, and a determination program.

Conventionally, in an IT (Information Technology) system used by a user, when a system failure or the like occurs, the user inquires at a support desk or the like, and a support member directly talks with the user to deal with the problem.

IT systems are becoming more complex every day, and users manage multiple systems consisting of systems from multiple vendors, an OS (Operating System), and software. Therefore, for example, even in the work of applying a patch, it is necessary to check a plurality of manuals, which imposes a heavy burden on the corresponding support staff.

Therefore, it is conceivable to respond to inquiries from users by automatically responding with a chatbot or the like. Here, the level of IT skill varies depending on the customer, and when the support staff responds, the answer is given according to the level of the user. However, when automatically responding with a chatbot or the like, it is difficult to answer according to the level of the user's IT skill, and the presented solution may not be understood, or conversely, the explanation may be redundant. is assumed.

As a technology related to the above, a program for assisting an operator who answers an inquiry from a customer has been proposed. This program extracts keywords from customer inquiries, determines the customer's knowledge level using the extracted keywords and the keyword DB, and adjusts the determined customer's knowledge level using the adjustment DB. Then, this program creates an answer to the customer's inquiry based on the adjusted customer's knowledge level and the answer content DB, and displays the created answer to the operator.

In addition, an explanation device for explaining terms according to the knowledge level of the user has been proposed. This device determines the knowledge level of the user based on the knowledge level given to the term for which the explanation is requested, and creates an explanation according to the knowledge level of the user based on this judgment.

Japanese Unexamined Patent Publication No. 2004-355108 Japanese Unexamined Patent Publication No. 2000-250693

In the prior art, the skill of the customer or the user is judged based on the keyword extracted from the inquiry from the customer or the level of the term requested to be explained by the user. However, it can be difficult to properly determine the level of keywords and terms. In addition, if the level of keywords and terms is not properly determined, the skill of the user cannot be judged appropriately, and it may not be possible to respond appropriately to inquiries.

As one aspect, the technique of disclosure aims to optimize the difficulty assigned to a word or word sequence to determine a user's skill.

In one aspect, the technique of disclosure comprises a word or word sequence selected from a plurality of words or word strings ranked according to difficulty according to the first user's understanding of the word or word string. The message is presented to the first user. In this case, the disclosed technique changes the rank of each of the plurality of words or word strings to a tentative rank according to the frequency with which the description of the word or word string is referred to by the first user. Then, the disclosed technique presents to the second user a message including the word or word string of the tentative rank selected according to the degree of understanding of the second user. In this case, the disclosed technique determines whether or not to determine the tentative rank based on the frequency with which the second user refers to the description of the word or word sequence.

As one aspect, it has an effect that the difficulty level given to a word or a word string for determining a user's skill can be optimized.

It is a functional block diagram of the determination device which concerns on this embodiment. It is a figure which shows an example of the consensus word DB. It is a figure which shows an example of the skill determination DB. It is a figure which shows an example of a user skill DB. It is a figure which shows an example of the message of the inquiry and the answer of rank 3. It is a figure which shows an example of the message of the inquiry and the answer of rank 2. It is a figure which shows an example of the message of the inquiry and the answer of rank 1. It is a figure which shows an example of the format of the answer pattern. It is a block diagram which shows the schematic structure of the computer which functions as the determination device which concerns on this embodiment. It is a flowchart which shows an example of a setting process. It is a flowchart which shows an example of a presentation process. It is a flowchart which shows an example of a determination process.

Hereinafter, an example of the embodiment according to the disclosed technology will be described with reference to the drawings.

As shown in FIG. 1, the determination device 10 according to the present embodiment functionally includes a setting unit 12, a presentation unit 14, a change unit 16, and a determination unit 18. Further, the consensus word DB (Database) 22, the skill determination DB 24, and the user skill DB 26 are stored in the predetermined storage area of the determination device 10.

The consensus word DB 22 stores an explanatory phrase used for answering an inquiry from the user. FIG. 2 shows an example of the consensus word DB22. In the example of FIG. 2, each line (each record) of the consensus word DB 22 corresponds to information about one explanatory phrase. Information about the explanation phrase includes "Explanation No.", "GR_No.", "Explanation phrase", "Rank", "Click rate", "Temporary rank", "Temporary rank click rate", and "Last rank confirmation date and time". Each item of "is included.

"Explanation No." is an identification number of each explanatory phrase. "GR_No." Is an identification number assigned to the group of explanatory phrases. Explanatory phrases included in the same group describe the same content. The "rank" is a value indicating the difficulty level of each explanatory phrase, and is a rank given to a word or a word string (hereinafter, referred to as "term") included in the explanatory phrase.

An "explanatory phrase" is a word, phrase, or sentence used to answer an inquiry from a user. The "previous rank confirmation date and time" is the date and time when the initial value of the rank is stored, or the date and time when the rank update or deferment is confirmed by the determination process described later. The "click rate", "temporary rank", and "temporary rank click rate" will be described later.

In the skill determination DB 24, a plurality of terms ranked according to the difficulty level are stored in order to determine the skill of the user. FIG. 3 shows an example of the skill determination DB 24. In the example of FIG. 3, each line (each record) of the skill determination DB 24 corresponds to information related to one term. The information about the term includes each item of "term No.", "GR_No.", "Term", "number of hits", and "rank".

"Term No." is an identification number of each term. "GR_No." Is an identification number of the group to which the explanatory phrase including each term belongs. "Rank" is a value indicating the difficulty level of each term. In the present embodiment, a case where three ranks of rank 1 indicating that the difficulty level is low, rank 2 indicating that the difficulty level is medium, and rank 3 indicating that the difficulty level is high are used will be described as an example. .. The "number of hits" will be described later.

The user skill DB 26 stores a rank indicating the skill of the user specified by the presentation unit 14 described later. FIG. 4 shows an example of the user skill DB 26. As shown in FIG. 4, the "user name" and the "rank" are stored in association with each other in the user skill DB. In addition, instead of the user name, other information that can identify the user, such as a user ID and an address of an information processing device used by the user, may be used.

The setting unit 12 sets the initial value of the rank of each term. Specifically, the setting unit 12 extracts a term for skill determination from each explanatory phrase stored in the consensus word DB 22, and sets the term No. Is given and stored in the skill determination DB 24. The term for skill determination can be extracted by morphologically analyzing each explanatory phrase stored in the consensus word DB22, collating it with a word dictionary or the like according to the purpose, and extracting the target term. The method for extracting terms is not limited to the above example, and the result extracted manually may be accepted by the setting unit 12.

Further, the setting unit 12 searches for each term stored in the skill determination DB 24 from a general site or the like with an existing search engine, and stores the number of hits in the search result in the "hits" column of the skill determination DB 24. To do. Further, the setting unit 12 has the same GR_No. For each group to which is given, a rank indicating that the difficulty level is high is given in ascending order of the number of hits of the terms included in the group, and is stored in the "rank" column of the skill determination DB 24. This is because a term with a large number of hits is a general term and has a low difficulty level, and a term with a small number of hits is a technical term and is judged to have a high difficulty level.

For example, when three ranks are used as in the present embodiment, the setting unit 12 ranks 3 for the top 1/3 term and rank 2 for the top 1/3 to 2/3 terms in descending order of the number of hits. , Rank 1 can be given to the lower 1/3 term.

When the rank of each term is set, the setting unit 12 stores the rank set for the term included in the explanatory phrase in the "rank" column of each explanatory phrase in the synonymous word DB 22. The method of determining the rank is not limited to the case where the number of hits of the search is used as described above, and the rank set manually may be accepted by the setting unit 12.

When the presentation unit 14 receives an inquiry from a user, the presentation unit 14 generates a message using an explanatory phrase of a rank according to the skill of the user and presents the message to the user.

Specifically, the presentation unit 14 extracts a term that matches the term stored in the skill determination DB 24 from the inquiry from the user. The presentation unit 14 specifies the rank of the extracted term with reference to the skill determination DB 24. Then, the presentation unit 14 calculates the average value of the ranks of all the terms extracted from the inquiry, and specifies it as a rank indicating the skill of the user. In addition, instead of the average value, among the ranks of the extracted terms, the highest rank or the lowest rank may be specified as a rank indicating the skill of the user.

For example, when an inquiry as shown in the upper figure of FIG. 5 is received, the presentation unit 14 uses the skill determination DB 24 shown in FIG. 3 to obtain the “Windows (registered trademark) logo”, “LCD”, and “ The term "D-Sub15" is extracted. The presentation unit 14 specifies the rank of each term as 3, 3, 3, calculates the average value as 3, and specifies the rank as 3.

Further, for example, when an inquiry as shown in the upper figure of FIG. 6 is received, the presentation unit 14 uses the skill determination DB 24 shown in FIG. 3 to obtain a “Windows (registered trademark) mark” and a “liquid crystal display”. , And the term "VGA terminal" is extracted. The presentation unit 14 specifies the rank of each term as 2, 2, 2, calculates the average value as 2, and specifies the rank as 2.

Further, for example, when an inquiry as shown in the upper figure of FIG. 7 is received, the presentation unit 14 uses the skill determination DB 24 shown in FIG. 3 to display the “four-color flag pattern”, the “screen”, and the screen. Extract the term "connector". The presentation unit 14 specifies the rank of each term as 1, 1, 1, calculates the average value as 1, and specifies the rank as 1.

If the average value has a fraction, the average value may be rounded off to specify the rank. The presentation unit 14 stores the rank indicating the skill of the specified user in the user skill DB 26 in association with the user name.

The presentation unit 14 analyzes the inquiry from the user and selects the answer pattern corresponding to the inquiry from the plurality of answer patterns prepared in advance. Since existing methods can be applied to the analysis of inquiries and the selection of corresponding response patterns, detailed description thereof will be omitted here. Further, the answer pattern is in a format in which an answer message can be generated by embedding an explanatory phrase stored in the consensus word DB 22. For example, as shown in FIG. 8, in each answer pattern, GR_No. Of the explanatory phrase embedded in the answer pattern. Is defined in the place to be embedded (the place <<>> in FIG. 8).

The presentation unit 14 has GR_No. Of the explanatory phrase defined in the selected answer pattern. And, based on the rank according to the skill of the specified user, the explanatory phrase to be embedded in the answer pattern is acquired from the consensus word DB22. For example, as shown in the lower figure of FIG. 5, when the rank indicating the skill of the user is 3, the response pattern << GR_No. The "BIOS menu" is embedded in the place of 3 >>.

The presentation unit 14 acquires an explanatory phrase in which the rank indicating the skill of the user and the temporary rank match with respect to the explanatory phrase in which the “temporary rank” of the consensus word DB 22 is stored. Therefore, in the above example, GR_No. Regarding the explanation phrase of 3, the "setting menu for computer time and CPU operation mode" in which provisional rank 3 is set is << GR_No. It is embedded in the place of 3 >>.

The user of the skill corresponding to the original rank of the explanation phrase is an example of the "first user" of the disclosed technology, and the user of the skill corresponding to the provisional rank of the explanation phrase is the "second user" of the disclosed technology. This is an example.

As shown in the lower figure of FIGS. 5 to 7, the presentation unit 14 embeds the acquired explanatory phrase in the selected answer pattern to generate an answer message. Then, the presentation unit 14 sets a link to a pop-up screen or the like in which the explanation phrase embedded in the answer pattern is displayed with the explanation phrase one rank lower than the rank of the explanation phrase. The presentation unit 14 sets a link so that a warning such as "Please contact a support staff" is displayed when the rank of the embedded explanation phrase is the lowest. As a result, when the user clicks on any of the explanatory phrases included in the presented message, a simpler explanatory phrase of the explanatory phrase is referred to.

The presentation unit 14 transmits the generated message to the information processing terminal used by the user who made the inquiry.

Further, when the presentation unit 14 detects that the user has clicked any of the explanatory phrases included in the response message, the presentation unit 14 adds 1 to the number of clicks counted since the "previous rank confirmation date and time" of the clicked explanatory phrase. to add. In addition, the presentation unit 14 calculates the click rate for the explanatory phrase included in the presented message, and updates the "click rate" column of the consensus word DB 22. The click rate is the number of clicks relative to the number of times each explanatory phrase was included in the message and presented to the user. The click rate is an example of the frequency with which the explanation of the word or word string of the disclosed technique is referred to.

However, for the explanatory phrase in which the provisional rank is stored in the "provisional rank" of the synonym DB22, the presentation unit 14 calculates the click rate after the provisional rank is set (details will be described later), and the "provisional rank" of the synonym DB22 Update the "Temporary rank click rate" column.

Among the explanatory phrases stored in the consensus word DB 22, the changing unit 16 changes the rank of the explanatory phrase to a provisional rank according to the "click rate" for the explanatory phrase for which a predetermined time has passed from the "previous rank confirmation date and time". change.

Specifically, when the click rate is the threshold value TH1 (for example, 30%) or less, the change unit 16 indicates that the difficulty level of the explanation phrase is lower than the current rank, that is, the rank demoted by one rank. Is set as a temporary rank. A low click-through rate means that the explanatory phrase may be less difficult for the user who is presented with the message, so the provisional rank is demoted by one rank.

Further, when the click rate is the threshold value TH3 (for example, 70%) or more, the change unit 16 temporarily ranks the rank indicating that the explanation phrase is more difficult than the current rank, that is, the rank promoted by one rank. Set as. A high click rate means that the explanatory phrase may be difficult for the user who is presented with the message, so that the provisional rank is promoted by one rank.

As a result, if a tentative rank that has been demoted or promoted is set after a certain explanatory phrase is presented to the first user, the explanatory phrase is one rank lower or one rank higher than the first user. Will be presented to.

The changing unit 16 stores the set temporary rank in the “temporary rank” of the synonym DB22. If the click rate exceeds the threshold value TH1 and is less than the threshold value TH3, the current rank is deferred and the provisional rank is not set.

The determination unit 18 determines whether or not to determine the provisional rank based on the "provisional rank click rate" at the timing when a predetermined time has elapsed since the provisional rank was set for each explanatory phrase for which the provisional rank was set. To do.

It is assumed that the explanation phrase presented in the original rank was not clicked because the explanation phrase is too verbose or appropriate. Therefore, a low click-through rate for an explanatory phrase presented in the original rank does not mean that the rank can be demoted immediately. In addition, if the click rate of the explanation phrase presented in the original rank is high and the rank is promoted, the difficulty level may still be high even for a user with a high skill rank. Therefore, the determination unit 18 determines whether or not to determine the provisional rank by using the provisional rank click rate as follows.

Specifically, the determination unit 18 determines the provisional rank that has been set if the click rate of the provisional rank is TH2 (for example, 10%) or less for the explanatory phrase for which the provisional rank that has been demoted by one rank is set. Is confirmed as the official rank. On the other hand, when the provisional rank click rate exceeds the threshold value TH2, the determination unit 18 returns the provisional rank to the original rank.

In this case, if the presented explanatory phrase is hardly clicked, it means that even the user of the skill one rank lower has understood the explanatory phrase, so that the rank is demoted. On the other hand, if the presented explanation phrase is clicked at a certain rate, it means that the explanation phrase was difficult for the user of the skill one rank lower, so the original rank is maintained. is there.

Further, if the tentative rank click rate is equal to or less than the threshold value TH4 (for example, 10%) for the explanatory phrase in which the tentative rank promoted by one rank is set, the determination unit 18 officially sets the tentative rank that has been set. Confirmed as a rank. On the other hand, when the provisional rank click rate exceeds the threshold value TH4, the determination unit 18 outputs a warning prompting the review of the consensus word DB22.

In this case, if the presented explanation phrase is rarely clicked, it means that the user of the skill one rank higher has understood the explanation phrase, so the rank is promoted. is there. On the other hand, when the presented explanatory phrase is clicked at a certain rate, it means that even a user with a skill one rank higher could not understand the explanatory phrase, and the consensus word DB22 needs to be reviewed. It is judged to exist.

When the tentative rank is confirmed as the official rank, the determination unit 18 updates the "rank" and "previous rank confirmation date and time" of the consensus word DB 22, and also updates the "click rate", "temporary rank", and "temporary rank click". Clear the "Rate" column. On the other hand, when the tentative rank is not confirmed, the determination unit 18 updates the "previous rank confirmation date and time" while leaving the "rank" column deferred, and also updates the "click rate", "temporary rank", and "temporary rank click". Clear the "Rate" column.

In addition, the determination unit 18 also updates the rank of the term included in the explanation phrase whose rank has been updated to the same rank as the explanation phrase in the skill determination DB 24. This allows the user's skill to be identified using the updated rank.

The determination device 10 can be realized by, for example, the computer 40 shown in FIG. The computer 40 includes a CPU (Central Processing Unit) 41, a memory 42 as a temporary storage area, and a non-volatile storage unit 43. Further, the computer 40 includes an input / output device 44 such as an input unit and a display unit, and an R / W (Read / Write) unit 45 that controls reading and writing of data to the storage medium 49. Further, the computer 40 includes a communication I / F (Interface) 46 connected to a network such as the Internet. The CPU 41, the memory 42, the storage unit 43, the input / output device 44, the R / W unit 45, and the communication I / F 46 are connected to each other via the bus 47.

The storage unit 43 can be realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. The storage unit 43 as a storage medium stores a determination program 50 for causing the computer 40 to function as the determination device 10. The determination program 50 has a setting process 52, a presentation process 54, a change process 56, and a determination process 58. In addition, the storage unit 43 has an information storage area 60 in which information constituting each of the consensus word DB 22, the skill determination DB 24, and the user skill DB 26 is stored.

The CPU 41 reads the determination program 50 from the storage unit 43, expands it into the memory 42, and sequentially executes the processes included in the determination program 50. The CPU 41 operates as the setting unit 12 shown in FIG. 1 by executing the setting process 52. Further, the CPU 41 operates as the presentation unit 14 shown in FIG. 1 by executing the presentation process 54. Further, the CPU 41 operates as the change unit 16 shown in FIG. 1 by executing the change process 56. Further, the CPU 41 operates as the determination unit 18 shown in FIG. 1 by executing the determination process 58. Further, the CPU 41 reads information from the information storage area 60, and expands each of the consensus word DB 22, the skill determination DB 24, and the user skill DB 26 into the memory 42. As a result, the computer 40 that executes the determination program 50 functions as the determination device 10. The CPU 41 that executes the program is hardware.

The function realized by the determination program 50 can also be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit) or the like.

Next, the operation of the determination device 10 according to the present embodiment will be described. First, the determination device 10 executes the setting process shown in FIG. Then, when the inquiry from the user is received, the determination device 10 executes the presentation process shown in FIG. Further, the determination device 10 executes the determination process shown in FIG. 12 for the explanatory phrase for which a predetermined time has elapsed from the “previous rank determination date and time”. The setting process, presentation process, and determination process are examples of determination methods of the disclosed technology.

First, the setting process shown in FIG. 10 will be described.

In step S12, the setting unit 12 extracts a term for skill determination from each explanatory phrase stored in the consensus word DB22, and sets the term No. Is given and stored in the skill determination DB 24.

Next, in step S14, the setting unit 12 searches for each term stored in the skill determination DB 24 from a general site or the like with an existing search engine, and determines the number of hits in the search result as the “hits” of the skill determination DB 24. Store in the "Number" column.

Next, in step S16, the setting unit 12 has the same GR_No. For each group to which is given, a rank indicating that the difficulty level is high is given in ascending order of the number of hits of the terms included in the group, and is stored in the "rank" column of the skill determination DB 24.

Next, in step S18, the setting unit 12 stores the rank set for the term included in the explanation phrase in the "rank" column of each explanation phrase in the synonym DB 22, and the setting process ends.

Next, the presentation process shown in FIG. 11 will be described.

In step S32, the presentation unit 14 extracts a term that matches the term stored in the skill determination DB 24 from the inquiry from the user, and identifies a rank indicating the skill of the user based on the rank of the extracted term. To do.

Next, in step S34, the presentation unit 14 analyzes the inquiry from the user. Then, the presenting unit 14 makes an inquiry from a plurality of prepared answer patterns in a format capable of generating an answer message by embedding an explanatory phrase stored in the consensus word DB 22, for example, as shown in FIG. Select the answer pattern corresponding to.

Next, in step S36, the presentation unit 14 acquires the explanatory phrase to be embedded in the answer pattern selected in step S34 from the consensus word DB 22. At this time, if the explanation phrase for which the provisional rank is set exists in the group of the explanation phrases, the presentation unit 14 acquires the explanation phrase of the provisional rank that matches the rank corresponding to the skill of the user. .. If there is no description phrase for which a temporary rank is set, the description phrase of the rank that matches the rank corresponding to the user's skill is acquired. Then, the presentation unit 14 embeds the acquired explanatory phrase in the selected answer pattern, and generates an answer message in which a link for displaying the explanatory phrase of the lower rank is set in the explanatory phrase.

Next, in step S38, the presenting unit 14 transmits the message generated in step S36 to the information processing terminal used by the user who made the inquiry. As a result, a message is displayed on the display of the information processing device used by the user.

Next, in step S40, it is determined whether or not the user has clicked any explanatory phrase included in the answer message. If it is clicked, the process proceeds to step S42, and if it is not clicked, the process proceeds to step S44.

In step S42, the presentation unit 14 adds 1 to the number of clicks counted since the "previous rank confirmation date and time" of the clicked explanatory phrase.

In step S44, it is determined whether or not the message has been confirmed by the user by determining whether or not a command indicating that the message from the user's information processing terminal has been confirmed, an instruction to close the message screen, or the like has been received. If the message is confirmed, the process proceeds to step S46, and if not confirmed, the process returns to step S40.

In step S46, the presentation unit 14 calculates the click rate for each of the explanatory phrases included in the presented message. The presentation unit 14 calculates the tentative rank click rate for the explanatory phrase for which the tentative rank is set. Then, the presentation unit 14 updates the "click rate" or "temporary rank click rate" column of the consensus word DB 22 with the calculated click rate or provisional rank click rate, and the presentation process ends.

Next, the determination process shown in FIG. 12 will be described. The determination process is executed by selecting one by one the explanatory phrases for which a predetermined time has elapsed from the "previous rank confirmation date and time" in the synonym DB22.

In step S62, the changing unit 16 determines that the "click rate" of the target explanatory phrase is below the threshold value TH1 (for example, 30%), above the threshold value TH3 (for example, 70%), or exceeds the threshold value TH1. Determine if it is less than TH3. When the threshold value is TH1 or less, the process proceeds to step S64, when the threshold value is TH3 or more, the process proceeds to step S72, and when the threshold value TH1 is exceeded and the threshold value is less than TH3, the determination process ends. To do.

In step S64, the changing unit 16 sets a rank indicating that the difficulty level of the target explanation phrase is lower than the current rank, that is, a rank demoted by one rank as a provisional rank, and sets the provisional rank as a synonym. It is stored in the "temporary rank" of DB22.

Next, after waiting for a predetermined time, the process proceeds to step S66, and the determination unit 18 determines whether or not the provisional rank click rate is the threshold value TH2 (for example, 10%) or less. When the threshold value is TH2 or less, the process proceeds to step S68, and when the threshold value TH2 is exceeded, the process proceeds to step S70.

In step S68, the determination unit 18 determines the set provisional rank as the official rank, updates the "rank" and "previous rank confirmation date and time" of the consensus word DB22, and also updates the "click rate" and "temporary rank". , And "Temporary rank click rate" fields are cleared. Further, the determination unit 18 updates the rank of the term included in the explanation phrase whose rank has been updated to the same rank as the explanation phrase in the skill determination DB 24. Then, the determination process ends.

On the other hand, in step S70, the determination unit 18 updates the "previous rank confirmation date and time" while leaving the "rank" column unchanged, and also updates the "click rate", "temporary rank", and "temporary rank click rate" columns. By clearing, the temporary rank is canceled. Then, the determination process ends.

In step S72, the changing unit 16 sets a rank indicating that the difficulty level of the target explanatory phrase is higher than the current rank, that is, a rank promoted by one rank as a provisional rank.

Next, after waiting for a predetermined time, the process proceeds to step S74, and the determination unit 18 determines whether or not the provisional rank click rate is the threshold value TH4 (for example, 10%) or less. When the threshold value is TH4 or less, the process proceeds to step S76, and when the threshold value TH4 is exceeded, the process proceeds to step S78.

In step S76, the determination unit 18 determines the set provisional rank as a formal rank, and the determination process ends, as in step S68.

On the other hand, in step S78, the determination unit 18 outputs a warning prompting the review of the consensus word DB 22, and the determination process ends.

By executing the above determination process, the number of explanatory phrases corresponding to each rank is biased or the number of explanatory phrases corresponding to a certain rank becomes 0 for the explanatory phrases in the same group stored in the consensus word DB 22. It may become. Even in that case, a warning prompting the review of the consensus word DB 22 regarding the corresponding explanatory phrase may be output.

As described above, the determination device according to the present embodiment responds to the click rate when a message including an explanatory phrase ranked according to the difficulty level is presented to the first user corresponding to the rank. , Set a temporary rank. Then, the determination device determines whether or not to determine the provisional rank according to the provisional rank click rate when the message including the explanatory phrase is presented to the second user corresponding to the provisional rank. Then, when the rank of the explanatory phrase is updated, the rank of the word or word string included in the explanatory phrase and used to determine the skill of the user is also updated. As a result, the difficulty level assigned to the word or word string for determining the skill of the user can be optimized.

In addition, this makes it possible to take an appropriate response according to the skill of the user, and it becomes possible to realize an automatic response by a chatbot or the like.

In addition, the difficulty level of a word or word string changes depending on changes in the environment such as the passage of time, but as in the present embodiment, the difficulty level of a word or word string can be reviewed as needed to change the environment. Can be accommodated.

In the above embodiment, the case where characters are displayed on the display as an example of presenting a message to the user has been described, but the present invention is not limited to this. For example, the message may be output by voice. In this case, instead of clicking in the above embodiment, when a question about the explanatory phrase included in the presented message is input, such as "What is the boot sector?", It is determined that the explanatory phrase is referred to. do it.

Further, in the above embodiment, the mode in which the determination program is stored (installed) in the storage unit in advance has been described, but the present invention is not limited to this. The program according to the disclosed technology can also be provided in a form stored in a storage medium such as a CD-ROM, a DVD-ROM, or a USB memory.

The following additional notes will be further disclosed with respect to the above embodiments.

(Appendix 1)
When a message including a word or a word string selected according to the understanding level of the first user for the word or the word string is presented to the first user from a plurality of words or word strings ranked according to the difficulty level. In addition, the rank of each of the plurality of words or word strings is changed to a provisional rank according to the frequency with which the description of the word or word string is referred to by the first user.
When a message including the word or word string of the provisional rank selected according to the understanding level of the second user is presented to the second user, the explanation of the word or word string is referred to by the second user. A determination method in which a computer executes a process including determining whether or not to determine the provisional rank based on the frequency.

(Appendix 2)
When the frequency with which the explanation of the word or word string is referred to by the first user is equal to or less than the first threshold value, the rank indicating that the difficulty level is lower than the current rank is defined as the provisional rank.
When the frequency with which the explanation of the word or word string is referred to by the second user is less than or equal to the second threshold value, the provisional rank is determined, and when the second user exceeds the second threshold value, the provisional rank is returned to the current rank. The determination method described in Appendix 1.

(Appendix 3)
When the frequency with which the explanation of the word or word string is referred to by the first user is equal to or higher than the third threshold value, the rank indicating that the difficulty level is higher than the current rank is defined as the provisional rank.
When the frequency with which the explanation of the word or word string is referred to by the second user is less than or equal to the fourth threshold value, the provisional rank is determined, and when the frequency exceeds the fourth threshold value, each of the plurality of words or word strings is used. The determination method described in Appendix 1 or Appendix 2 that outputs a warning prompting the review of the ranking.

(Appendix 4)
The message is presented to the first user or the second user by displaying a document containing the word or word string with a link to the description.
In the displayed document, when the word or word string is selected, the explanation is displayed based on the link set for the selected word or word string. Any one of Supplementary notes 1 to 3. Judgment method described in the section.

(Appendix 5)
The message is a response to an inquiry from the first user or the second user, and specifies the degree of understanding of the first user or the second user based on the rank of a word or a word string included in the inquiry. The determination method according to any one of Supplementary note 1 to Supplementary note 4.

(Appendix 6)
Any of Appendix 1 to Appendix 5 that presents a word or word string of a rank indicating that the difficulty level is lower than the rank of the referenced word or word string when the explanation of the word or word string is referred to. The determination method according to item 1.

(Appendix 7)
The frequency with which the explanation of the word or word string is referred to is any of Appendix 1 to Appendix 6 which is the number of times the word or word string is referred to with respect to the number of times a message containing the word or word string is presented. The determination method described in item 1.

(Appendix 8)
When a message including a word or a word string selected according to the understanding level of the first user for the word or the word string is presented to the first user from a plurality of words or word strings ranked according to the difficulty level. In addition, a change unit that changes the rank of each of the plurality of words or word strings to a provisional rank according to the frequency with which the explanation of the word or word string is referred to by the first user.
When a message including the word or word string of the provisional rank selected according to the understanding level of the second user is presented to the second user, the explanation of the word or word string is referred to by the second user. A determination unit that determines whether or not to determine the provisional rank based on the frequency of
Judgment device including.

(Appendix 9)
When the frequency with which the explanation of the word or word string is referred to by the first user is equal to or less than the first threshold value, the change unit sets the rank indicating that the difficulty level is lower than the current rank as the provisional rank.
The determination unit determines the tentative rank when the frequency with which the explanation of the word or word string is referred to by the second user is equal to or less than the second threshold value, and determines the tentative rank when the frequency exceeds the second threshold value. Return to the current rank The determination device according to Appendix 8.

(Appendix 10)
When the frequency with which the explanation of the word or word string is referred to by the first user is equal to or greater than the third threshold value, the change unit sets the rank indicating that the difficulty level is higher than the current rank as the provisional rank.
The determination unit determines the provisional rank when the frequency with which the explanation of the word or word string is referred to by the second user is less than or equal to the fourth threshold value, and when the frequency exceeds the fourth threshold value, the plurality of words or The determination device according to Appendix 8 or Appendix 9, which outputs a warning prompting a review of the ranking of each word string.

(Appendix 11)
The message includes a presenting unit that presents the message to the first user or the second user by displaying a document containing the word or word sequence to which a link to the description is set.
When the word or word string is selected in the displayed document, the presentation unit displays the explanation based on the link set for the selected word or word string. The determination device according to any one of 10.

(Appendix 12)
The message is a response to an inquiry from the first user or the second user.
The determination device according to Appendix 11, wherein the presentation unit specifies the degree of understanding of the first user or the second user based on the rank of a word or word string included in the inquiry.

(Appendix 13)
When the explanation of the word or word string is referred to, the presenting unit presents a word or word string having a rank indicating that the difficulty level is lower than the rank of the referred word or word string. The determination device according to Appendix 12.

(Appendix 14)
The frequency with which the explanation of the word or word string is referred to is any of Appendix 8 to Appendix 13 which is the number of times the word or word string is referred to with respect to the number of times a message containing the word or word string is presented. The determination device according to item 1.

(Appendix 15)
When a message including a word or a word string selected according to the understanding level of the first user for the word or the word string is presented to the first user from a plurality of words or word strings ranked according to the difficulty level. In addition, the rank of each of the plurality of words or word strings is changed to a provisional rank according to the frequency with which the description of the word or word string is referred to by the first user.
When a message including the word or word string of the provisional rank selected according to the understanding level of the second user is presented to the second user, the explanation of the word or word string is referred to by the second user. A determination program for causing a computer to execute a process including determining whether or not to determine the provisional rank based on the frequency.

(Appendix 16)
When the frequency with which the explanation of the word or word string is referred to by the first user is equal to or less than the first threshold value, the rank indicating that the difficulty level is lower than the current rank is defined as the provisional rank.
When the frequency with which the explanation of the word or word string is referred to by the second user is less than or equal to the second threshold value, the provisional rank is determined, and when the second user exceeds the second threshold value, the provisional rank is returned to the current rank. The determination program according to Appendix 15.

(Appendix 17)
When the frequency with which the explanation of the word or word string is referred to by the first user is equal to or higher than the third threshold value, the rank indicating that the difficulty level is higher than the current rank is defined as the provisional rank.
When the frequency with which the explanation of the word or word string is referred to by the second user is less than or equal to the fourth threshold value, the provisional rank is determined, and when the frequency exceeds the fourth threshold value, each of the plurality of words or word strings is used. The determination program according to Appendix 15 or Appendix 16 that outputs a warning prompting a review of the ranking.

(Appendix 18)
The message is presented to the first user or the second user by displaying a document containing the word or word string with a link to the description.
In the displayed document, when the word or word string is selected, the explanation is displayed based on the link set for the selected word or word string. Any one of Appendix 15 to Appendix 17 The judgment program described in the section.

(Appendix 19)
The message is a response to an inquiry from the first user or the second user, and specifies the degree of understanding of the first user or the second user based on the rank of a word or a word string included in the inquiry. The determination program according to any one of Supplementary note 15 to Supplementary note 18.

(Appendix 20)
When a message including a word or a word string selected according to the understanding level of the first user for the word or the word string is presented to the first user from a plurality of words or word strings ranked according to the difficulty level. In addition, the rank of each of the plurality of words or word strings is changed to a provisional rank according to the frequency with which the description of the word or word string is referred to by the first user.
When a message including the word or word string of the provisional rank selected according to the understanding level of the second user is presented to the second user, the explanation of the word or word string is referred to by the second user. A storage medium that stores a determination program for causing a computer to execute a process including determining whether or not to determine the provisional rank based on the frequency.

10 Judgment device 12 Setting unit 14 Presentation unit 16 Change unit 18 Judgment unit 22 Consent word DB
24 Skill judgment DB
26 User Skill DB
40 computer 41 CPU
42 Memory 43 Storage unit 49 Storage medium 50 Judgment program

Claims (7)

When a message including a word or a word string selected according to the understanding level of the first user for the word or the word string is presented to the first user from a plurality of words or word strings ranked according to the difficulty level. In addition, the rank of each of the plurality of words or word strings is changed to a provisional rank according to the frequency with which the description of the word or word string is referred to by the first user.
When a message including the word or word string of the provisional rank selected according to the understanding level of the second user is presented to the second user, the explanation of the word or word string is referred to by the second user. A determination method in which a computer executes a process including determining whether or not to determine the provisional rank based on the frequency. When the frequency with which the explanation of the word or word string is referred to by the first user is equal to or less than the first threshold value, the rank indicating that the difficulty level is lower than the current rank is defined as the provisional rank.
When the frequency with which the explanation of the word or word string is referred to by the second user is less than or equal to the second threshold value, the provisional rank is determined, and when the second user exceeds the second threshold value, the provisional rank is returned to the current rank. The determination method according to claim 1. When the frequency with which the explanation of the word or word string is referred to by the first user is equal to or higher than the third threshold value, the rank indicating that the difficulty level is higher than the current rank is defined as the provisional rank.
When the frequency with which the explanation of the word or word string is referred to by the second user is less than or equal to the fourth threshold value, the provisional rank is determined, and when the frequency exceeds the fourth threshold value, each of the plurality of words or word strings is used. The determination method according to claim 1 or 2, which outputs a warning prompting a review of the ranking. The message is presented to the first user or the second user by displaying a document containing the word or word string with a link to the description.
Any of claims 1 to 3, which, when the word or word string is selected in the displayed document, displays the explanation based on the link set in the selected word or word string. The determination method according to item 1. The message is a response to an inquiry from the first user or the second user, and specifies the degree of understanding of the first user or the second user based on the rank of a word or a word string included in the inquiry. The determination method according to any one of claims 1 to 4. When a message including a word or a word string selected according to the understanding level of the first user for the word or the word string is presented to the first user from a plurality of words or word strings ranked according to the difficulty level. In addition, a change unit that changes the rank of each of the plurality of words or word strings to a provisional rank according to the frequency with which the explanation of the word or word string is referred to by the first user.
When a message including the word or word string of the provisional rank selected according to the understanding level of the second user is presented to the second user, the explanation of the word or word string is referred to by the second user. A determination unit that determines whether or not to determine the provisional rank based on the frequency of
Judgment device including. When a message including a word or a word string selected according to the understanding level of the first user for the word or the word string is presented to the first user from a plurality of words or word strings ranked according to the difficulty level. In addition, the rank of each of the plurality of words or word strings is changed to a provisional rank according to the frequency with which the description of the word or word string is referred to by the first user.
When a message including the word or word string of the provisional rank selected according to the understanding level of the second user is presented to the second user, the explanation of the word or word string is referred to by the second user. A determination program for causing a computer to execute a process including determining whether or not to determine the provisional rank based on the frequency.