JP2024066418A - Program, method, and system - Google Patents

Abstract

[Problem] To provide a system that can visualize customer feedback on a website and contribute to website improvement. [Solution] A program disclosed herein causes a processor to execute the steps of acquiring text information entered by a user into a text input form launched from at least one of a plurality of pages displayed on a customer terminal, referring to the user's operation history and identifying a related page among the plurality of pages that is related to the input operation of the text information, and referring to the operation history and displaying the acquired text information in association with a portion of the related page that corresponds to the input operation. [Selected Figure] Figure 10

Description

This disclosure relates to a program, method, and system for executing processing using a text input interface.

2. Description of the Related Art Conventionally, there has been known an automatic response system that automatically responds to inquiries made by text input.
For example, Patent Literature 1 discloses an automatic response system equipped with a machine learning model that has been machine-learned using a plurality of question patterns prepared for each category as learning data.
In this automatic response system, a question input by a user is classified into a specific category by a machine learning model, and an answer pattern linked to the classified category is output to the user as an answer.

JP 2019-219737 A

Incidentally, the feedback from customers who use a service can provide useful hints for improving the customer experience with that service, and customer feedback is extremely useful information. However, conventional automated response systems left room for improvement in the analysis of customer feedback.

Therefore, the purpose of this disclosure is to provide a system that can visualize customer feedback about a site and contribute to improving the site.

A program in one aspect of the present disclosure is a program to be executed by a computer having a processor and a memory, and the program causes the processor to execute the steps of acquiring text information input by a user into a text input form launched from at least one of a plurality of pages displayed on a customer terminal, referring to the user's operation history and identifying a related page among the plurality of pages that is related to the input operation of the text information, and referring to the operation history and displaying the acquired text information in association with a portion of the related page that corresponds to the input operation.

This disclosure makes it possible to visualize customer feedback about a site, which can help improve the site.

FIG. 1 is a diagram illustrating an overview of a service page management system according to the present invention. FIG. 2 is a diagram illustrating a hardware configuration of the customer terminal illustrated in FIG. 1 . 2 is a diagram illustrating a hardware configuration of the administrator terminal illustrated in FIG. 1 . FIG. 2 is a diagram illustrating a hardware configuration of the question and answer server illustrated in FIG. 1 . FIG. 5 is a diagram showing a functional configuration of the question and answer server shown in FIG. 4 . FIG. 4 is a diagram showing a specific example of a user database stored in a storage unit. 4 is a diagram showing a specific example of a question and answer database stored in a storage unit; FIG. 11 is a diagram illustrating a specific example of an operation log database stored in a storage unit. FIG. FIG. 11 is a diagram illustrating the flow of an automatic response process by the service page management system. 11 is a diagram illustrating a flow of a visualization process of customer opinions by a service page management system. FIG. FIG. 13 is a diagram showing an example of a screen of a customer terminal in the service page management system. FIG. 11 is a diagram showing an example of a first screen of an administrator terminal in the service page management system. FIG. 13 is a diagram showing a second example of a screen of the administrator terminal in the service page management system.

A service page management system 1 (hereinafter simply referred to as system 1) according to an embodiment of the present disclosure will be described below with reference to the drawings. Note that the embodiment described below does not unduly limit the contents of the present disclosure described in the claims. Furthermore, not all of the configurations described in the embodiments are necessarily essential components of the present disclosure. Furthermore, in all figures describing the embodiments, common components are given the same reference numerals, and repeated explanations will be omitted.

＜1. Overview＞
FIG. 1 is a diagram for explaining an overview of a system 1 according to the present invention.
As shown in FIG. 1, a system 1 according to this embodiment is a system that functions as a response system installed for user support on a site such as an EC (Electronic Commerce) site.
Specifically, the system 1 functions as a response system with a dialogue-style text input interface, using a database in which question patterns and answer patterns are stored.
Furthermore, the system 1 has a function of visualizing customer feedback on a site that functions as a response system.

That is, the system 1 estimates the location on a service page (hereinafter simply referred to as a page) that constitutes the site where a question was asked by the user by associating the content of the question from the user (customer) viewing the site with the operation history of the user.
The system 1 then displays the content of the user's question at the location where the user has presumably asked the question, thereby making customer feedback visible to the site administrator.

For example, if it is found that similar questions are frequently asked by multiple users viewing the same page on a certain site, this suggests that the page is unclear in terms of the perspective asked by the multiple users.
By obtaining such suggestions from the system 1, the site administrator can improve each page of the site that he or she manages so that it is clearer for users, which is expected to improve the convenience of customers browsing the site.
In the following description, the configuration of the system 1 will be first described, and then the automatic response process and the process of visualizing customer comments using the system 1 will be described in detail.

<2. Overall structure＞
The overall configuration of a system 1 according to this embodiment will be described.
As shown in FIG. 1, the system 1 includes a plurality of customer terminals 10 (customer terminals 10A, 10B, and 10C used by users, hereinafter collectively referred to as "customer terminals 10" in FIG. 1) and an administrator terminal 20. and a question and answer server 30.

The customer terminal 10, the administrator terminal 20, and the question and answer server 30 are connected to each other so that they can communicate with each other via a network 80. The network 80 is configured as a wired or wireless network.

The customer terminal 10 is a device operated by each user who browses the site on which the system 1 is installed. The customer terminal 10 is realized by, for example, a mobile terminal such as a smartphone or tablet compatible with a mobile communication system. The customer terminal 10 may also be a stationary PC (Personal Computer), a laptop PC, etc. In the example of FIG. 1, the customer terminal 10 is described as a smartphone.

The customer terminal 10 is connected to the administrator terminal 20 and the question and answer server 30 via the network 80 so that they can communicate with each other. The customer terminal 10 is connected to the network 80 by communicating with communication devices such as a wireless base station 81 that supports communication standards such as 5G and LTE (Long Term Evolution) and a wireless LAN router 82 that supports wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.11.

The administrator terminal 20 is a device operated by an administrator who manages the site on which the system 1 is installed. The administrator may be the entity providing the service related to the site, or may be a person in charge of maintaining the site or the system 1 in response to a request from the entity providing the service.

The administrator terminal 20 is realized by, for example, a mobile terminal such as a smartphone or tablet compatible with a mobile communication system. The administrator terminal 20 may also be a stationary PC (Personal Computer), a laptop PC, etc. In the example of FIG. 1, the administrator terminal 20 is described as a laptop PC.

The administrator terminal 20 is connected to the customer terminal 10 and the question and answer server 30 via the network 80 so that it can communicate with them. The administrator terminal 20 is connected to the network 80 by communicating with communication devices such as a wireless base station 81 that supports communication standards such as 5G and LTE (Long Term Evolution) and a wireless LAN router 82 that supports wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.11.

The question and answer server 30 is a server device that realizes the functions of system 1 for the site on which system 1 is installed. The question and answer server 30 is managed by an administrator.

<2-1. Hardware configuration of customer terminal 10>
FIG. 2 is a diagram showing the hardware configuration of the customer terminal 10. As shown in FIG.
As shown in FIG. 2 , the customer terminal 10 includes a communication IF (Interface) 12 , an input device 13 , an output device 14 , a memory 15 , a storage unit 16 , and a processor 19 .

The communication IF 12 is an interface for inputting and outputting signals so that the customer terminal 10 can communicate with external devices.
The input device 13 is a device for receiving input operations from a user (for example, a keyboard, a touch panel, a touch pad, a pointing device such as a mouse, etc.).

The output device 14 is a device (such as a display and a speaker) for presenting information to a user.
The memory 15 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

The storage unit 16 is a storage device for saving data, such as a flash memory or a hard disk drive (HDD).
The processor 19 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, peripheral circuits, and the like.

<2-2. Hardware configuration of the administrator terminal 20>
FIG. 3 is a diagram showing the hardware configuration of the administrator terminal 20. As shown in FIG.
As shown in FIG. 3, the administrator terminal 20 includes a communication IF (Interface) 22 , an input device 23 , an output device 24 , a memory 25 , a storage unit 26 , and a processor 29 .

The communication IF 22 is an interface for inputting and outputting signals so that the manager terminal 20 can communicate with external devices.
The input device 23 is a device (for example, a keyboard, a touch panel, a touch pad, a pointing device such as a mouse, etc.) for receiving input operations from the administrator.

The output device 24 is a device (such as a display and a speaker) for presenting information to the user.
The memory 25 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

The storage unit 26 is a storage device for saving data, such as a flash memory or a hard disk drive (HDD).
The processor 29 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, peripheral circuits, and the like.

2-3. Hardware configuration of the question and answer server 30
FIG. 4 is a diagram showing the hardware configuration of the question and answer server 30. As shown in FIG.
As shown in FIG. 5, the question and answer server 30 includes a communication IF (Interface) 32 , a memory 35 , a storage unit 36 , and a processor 39 .

The communication IF 32 is an interface for inputting and outputting signals so that the customer terminal 10 can communicate with external devices.
The memory 25 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

The storage unit 26 is a storage device for saving data, such as a flash memory or a hard disk drive (HDD).
The processor 29 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, peripheral circuits, and the like.

3. Functional configuration of question and answer server 30
5 is a diagram showing the functional configuration of the question and answer server 30 constituting the system 1. As shown in FIG. 5, the question and answer server 30 fulfills the functions of a communication unit 301, a storage unit 302, and a control unit 303.

The communication unit 301 performs processing for the question and answer server 30 to communicate with external devices.

The storage unit 302 stores data and programs used by the question and answer server 30. The storage unit 302 stores, for example, the following data:
User database 3021 (user DB3021)
Question and answer database 3022 (question and answer DB3022)
Operation log database 3023 (operation log DB3023)

<3-1. Database structure>
Here, each database stored in the storage unit 302 will be described in detail.

<3-1-1. User DB 3021>
The user DB 3021 is a database that stores information about users. FIG. 6 is a diagram showing a specific example of the user DB 3021 stored in the storage unit 302.
6, the user DB 3021 includes the items “user ID”, “name”, “personal information”, “membership information”, and “terminal information”. A new record is recorded in the user DB 3021, for example, when a user registers as a member of the site.

The "User ID" item stores user identification information that can identify individual users.

The "Name" item stores information about the user's name corresponding to the user ID.

The "Personal Information" item stores personal attribute information about the user corresponding to the user ID. Examples of personal attribute information include gender, age, place of residence, nationality, occupation, family structure, place of origin, and native language.

The "member information" item stores attribute information of the user corresponding to the user ID as a member who uses system 1. Attribute information of a member includes, for example, the type of membership, usage history, contract details, etc.

The item "terminal information" stores attribute information about the customer terminal 10 used by the user corresponding to the user ID. The attribute information as a terminal includes terminal identification information for identifying a tablet or a smartphone.
6 is merely an example, and other items may be managed. Also, the storage unit 302 does not have to store the user DB 3021.

<3-1-2. Question and Answer DB3022>
The question and answer DB 3022 shown in Fig. 5 is a database that stores question patterns and answer patterns in association with each other. Fig. 7 is a diagram showing a specific example of the question and answer DB 3022 stored in the storage unit 302.
7, in the question and answer DB 3022, for each category, a title and one answer pattern are associated with each other, and one or more question patterns expected for each category are stored. In other words, an answer pattern and a question pattern are linked to a category. The question and answer DB 3022 is stored in advance in the storage unit 302, for example, before the operation of the system 1 is started.

The question and answer DB 3022 includes an item "category", an item "answer pattern", and an item "question pattern".
The item "category" stores a title indicating the category of the question. The category title is, for example, information that succinctly indicates the content of the question.

Also, there is no need to have only one question pattern for one answer pattern. For example, if a user of an EC site equipped with the system 1 wants to know how to change his/her address, there are various ways to ask, such as "I want to change my address," or "I entered the wrong address," so it is useful to register multiple question patterns for one category (title: how to change address).
The question and answer DB 3022 may have a structure in which category titles are managed hierarchically.

<3-1-3. Operation log DB 3023>
The operation log DB 3023 shown in Fig. 5 is a database that stores the operation logs of the user. Fig. 8 is a diagram showing a specific example of the operation log DB 3023 stored in the storage unit 302.
8, the operation log DB 3023 includes an item “operation log ID”, an item “user ID”, an item “operation information”, an item “operation target”, and an item “operation date and time”. Every time an operation is accepted from a user, a new record is recorded in the operation log DB 3023.

The item "operation log ID" stores identification information for identifying the user's operation log acquired by the question and answer server 30.

The "User ID" item stores the user ID of the user who performed the operation corresponding to the operation log ID.

The item "operation information" stores operation information corresponding to the operation log ID. The operation information is time-series information, and all operations performed by the user, such as login operation, page transition operation, page stay operation, product selection operation by clicking, etc., are recorded.
Furthermore, the "operation information" includes various types of text information including the question entered by the user.
The text information entered from the text input form may be recorded in a database different from the operation log DB 3023. For example, a chatbot history database that stores the history of chatbot usage may be provided separately, and information regarding chatbot usage may be accumulated in the chatbot history database. In this case, the information regarding chatbot usage may include, for example, the following items:
・User ID
- Text information about the entered question - The category into which the question was classified (including the likelihood of each category)
- Results of categorization errors - Answers given to questions - Contents of survey results from users regarding answers

The "Operation target" item stores the URL of the site where the operation corresponding to the operation log ID was performed.

The item "operation date and time" stores the date and time of the operation corresponding to the operation log ID.
It should be noted that the structure of the operation log DB 3023 shown in FIG. 8 is merely an example, and other items may be managed.

<3-2. Functions of the control unit>
The control unit 303 shown in FIG. 5 fulfills the functions of various functional modules by the processor 39 of the question and answer server 30 performing processes according to the programs.
The various functional modules include a transmission/reception control module 3031 , a category selection module 3032 , a response output module 3033 , an operation history acquisition module 3034 , and a visualization module 3035 .

<3-2-1. Transmission/reception control module 3031>
The transmission/reception control module 3031 controls the process in which the question and answer server 30 transmits and receives signals from external devices in accordance with a communication protocol.

<3-2-2. Category Selection Module 3032>
The category selection module 3032 sets, for text information related to a question, a corresponding category from among a plurality of categories stored in the question and answer DB 3022. The category selection module 3032 uses a question classification model to select a category to which the text information related to a question belongs.

The question classification model is obtained by having a machine learning model perform machine learning based on the learning data in accordance with a model learning program. In this embodiment, the question classification model is trained to output the likelihood for each category in response to an input question sentence, for example. At this time, the learning data has multiple question patterns as input data, and the categories to which each of the multiple question patterns corresponds are set as correct answer output data.

The question classification model is, for example, a parameterized composite function in which multiple functions are combined. The parameterized composite function is defined by a combination of multiple adjustable functions and parameters. The question classification model may be any parameterized composite function that meets the above requirements, but is assumed to be a multi-layered network model (hereinafter referred to as a multi-layered network). A question classification model that uses a multi-layered network has an input layer that inputs questions, an output layer that outputs likelihoods, and at least one intermediate layer or hidden layer that is provided between the input layer and the output layer. The question classification model is expected to be used as a program module that is part of artificial intelligence software.

As the multi-layered network according to this embodiment, for example, a deep neural network (DNN), which is a multi-layered neural network that is the subject of deep learning, may be used. As the DNN, for example, a recurrent neural network (RNN) that targets time-series information, etc. may be used. Also, a network with a self-attention mechanism that targets natural language may be used.

In calculating the likelihood, the question classification model compares the question sentence with question patterns stored in the question and answer DB 3022, and calculates the likelihood based on the similarity.
The similarity can be calculated, for example, by natural language processing. Specifically, it is possible to extract keywords from the question pattern and the question sentence, and use weights for each keyword to determine the degree of agreement between the keywords as the similarity. Alternatively, the question pattern and the question sentence can be converted into vectors, and the similarity can be calculated using cosine similarity. In this way, by calculating which question pattern group the question sentence is similar to using the similarity, it becomes possible to select an answer pattern even if it is not the same as the question pattern stored in the question and answer DB 3022.

<3-2-3. Answer output module 3033>
The answer output module 3033 outputs an answer pattern linked to the category selected by the category selection module 3032 to the customer terminal 10. The answer output module 3033 refers to the question and answer DB 3022, identifies an answer pattern linked to the category selected by the category selection module 3032, and outputs text information of the answer pattern to the customer terminal 10 as an answer to the question sentence.
The answer output module 3033 may output the top n answer sentences with the highest likelihood together with the categories to the customer terminal 10, and allow the user to select the most appropriate answer.

<3-2-4. Operation History Acquisition Module 3034>
The operation history acquisition module 3034 acquires a log of operations performed by the user on the customer terminal 10. The operation history acquisition module 3034 can continuously acquire all operation logs related to operations of the user browsing the site as the operation log. The operation history acquisition module 3034 acquires, for example, the URLs of multiple browsed pages that transition between screens in chronological order, the duration of stay on each browsed page, the amount of scrolling within the browsed page, the operation contents, and the like as the operation log.

The operation history acquisition module 3034 also acquires, as an operation log, text information that the user inputs into the text input form using the customer terminal 10. Furthermore, when inputting text information into the text input form, the operation history acquisition module 3034 acquires the user's operation history regarding browsing of related pages that are related to the input operation of the text information among the multiple pages that make up the site.
The operation history acquisition module 3034 records the acquired operation log in the operation log DB 3023 .

<3-2-5. Visualization module 3035>
The visualization module 3035 visualizes user (customer) feedback for multiple pages that make up the site in response to instructions from the administrator. Here, customer feedback refers to text information related to text input from a user who has input a question or other text while viewing the page.

The visualization module 3035 refers to the operation log DB and identifies text information related to the question from the operation log in which the question was asked.
The visualization module 3035 also identifies a related page related to the input operation of the text information related to the identified question. The related page related to the input operation of the text information is a page that the user viewed shortly before the question was asked, and is presumed to have caused the user to have doubts about the question.

The visualization module 3035 identifies, for example, any of the following as a page related to the input operation of text information.
- The viewed page that was displayed when the question text was input - At least one of the multiple transition pages identified by tracing back the screen transitions in the operation history chronologically from the viewed page that was displayed when the question text was input - Of the multiple transition pages, a page with a long operation time or a long amount of operation, identified from the operation history - Of the multiple transition pages, a viewed page that is highly related to the acquired text information Specific examples of such methods of identifying related pages using the visualization module 3035 will be described later.

The visualization module 3035 also identifies a portion of the identified related page that is related to the operation of inputting the question message. The portion of the identified related page that is related to the operation of inputting the question message is the portion of the page viewed by the user that was displayed when the user asked the question, and is the portion of the related page that is presumed to have given the user a question that should be resolved by the question.

The visualization module 3035 identifies, for example, any of the following as a portion related to the input operation of the question sentence:
- A portion of a related page that is identified based on the amount of scrolling in the user's browsing operation. - A portion of a transition page that is identified based on the amount of scrolling in the user's browsing operation immediately before the transition operation to the next page in a screen transition in the operation history. Specific examples of such a method of identifying a portion related to the input operation of a question sentence by the visualization module 3035 will be described later.

The visualization module 3035 also displays text information related to the question in association with a portion related to the input operation of the identified question sentence. Examples of display modes when displaying text information in association are as follows.
The acquired text information is displayed in a partitioned area on the related page so as to be superimposed on the related page.
- Display the acquired text information in the margin of the related page.
A specific example of how the text information is displayed on the browse page by the visualization module 3035 will be described later.

The visualization module 3035 may also display additional information related to the input operation of the question sentence alongside the text information. Examples of the additional information include the following information:
Information about the attributes of the input operation Information about the operation history Information about the user's attributes Type of input text (attribute indicating the content)
- Whether the answer resolved the user's question (e.g., positive evaluation of the survey results)
- Whether or not the response resulted in a conversion (subsequent use of the service, etc.)

The information on the attributes of the input operation includes a question category related to the input operation of the question sentence.
The information on the operation history includes the type of operation that was performed shortly before the input operation of the question text and is presumed to have caused the question to be asked. For example, the information on the attributes of the input operation includes a page browsing operation, a login operation, a product purchase operation, a membership registration operation, a cancellation operation, a payment method change operation, etc.

The information on user attributes includes information on various users linked to the user ID in the user DB 3021 .
Regarding the type of input text, the attributes indicating the content of the input text information include items such as "question" or "complaint", for example.

<4. Processing of System 1>
Next, a description will be given of the processing of the system 1. In this description, first, a processing related to an automatic response by the system 1 will be described, and then a processing related to visualization of customer comments by the system 1 will be described.

<4-1. Automatic response processing>
The system 1 executes processing as a response system using a dialogue-style text input interface. That is, the system 1 plays a role as a chatbot on a user support inquiry screen using a dialogue-style text input interface.

An example of a response system using an interactive text input interface is a chatbot that pops up and appears in the lower right corner of an inquiry screen on an EC site.
In a response system with a dialogue-style text input interface, the first step in preparation for system setup is to register a conversation card to be used as a response system with a dialogue-style text input interface. The conversation card contains a message to prompt the user to enter a question in text. This completes the preparations for providing message guidance to users who access the inquiry screen.

FIG. 9 is a diagram for explaining the flow of the automatic response process by the system 1.
9, in the automatic response process, first, the user inputs the question content as text from the keyboard, which is the output device 14 of the customer terminal 10 (step S101). The control unit 170 of the customer terminal 10 transmits the input text information to the question and answer server 30 via the transmission/reception unit 172 in accordance with the processing of the data processing unit 173.

After step S101, the question and answer server 30 acquires a question sentence (step S301).
Specifically, the transmission/reception control module 3031 of the question and answer server 30 acquires text information related to the question sent from the customer terminal 10 .

After step S301, the question and answer server 30 selects a category to which the question should be answered (step S302).
Specifically, the category selection module 3032 of the question and answer server 30 inputs text information related to the question to the question classification model, and calculates the likelihood for each of a plurality of categories, thereby selecting an appropriate category.

After step S302, the question and answer server 30 outputs the answer sentence (step S303).
Specifically, the answer output module 3033 of the question and answer server 30 outputs an answer sentence corresponding to the category selected by the category selection module 3032 to the customer terminal 10 as an answer to the question.

After step S303, the customer terminal 10 displays the received reply text (step S102).
Specifically, the control unit 170 of the customer terminal 10 outputs text information about the answer to the display, which is the output device 14. As a result, the answer to the input question from the question and answer server 30 is presented to the user. The user checks the presented answer and attempts to resolve any questions that the user may have had while browsing the page.
In addition, in the above steps S301 to S303, the operation history acquisition module 3034 continuously acquires an operation log relating to operations by the user.
This completes the automatic response process by the system 1.

As another function, the question and answer server 30 may output a questionnaire to the user regarding the answer text that has been answered. The questionnaire may include the following contents.
- Whether the answer resolved your question (whether the problem was solved)
- Whether the answer (category) was appropriate or not. If the answer was inappropriate, the reason why.

<4-2. Visualization of customer feedback>
Next, a process for visualizing customer feedback by the system 1 will be described.
When performing maintenance on the site, the administrator of the site equipped with system 1 causes system 1 to execute a process for visualizing customer feedback in order to reflect customer feedback and make the site more convenient for customers.
FIG. 10 is a diagram illustrating a flow of a process for visualizing customer opinions by the system 1.

10 , in the visualization process of customer opinions, first, as a preparation stage, the question and answer server 30 identifies text information related to the question to be visualized (step S311). The question and answer server 30 refers to the operation log DB 3023, and at preset processing intervals, such as one day intervals, targets questions newly stored in the operation log DB 3023 for visualization processing, and identifies the corresponding text information.
Specifically, the visualization module 3035 of the question and answer server 30 refers to the operation log DB, searches for operation logs related to the input of the corresponding question sentence, and identifies text information related to the input of the corresponding question sentence.

After step S311, the question and answer server 30 identifies, from among the multiple pages, a page that is related to the input operation of the text information related to the identified question sentence (step S312).
Specifically, the visualization module 3035 of the question and answer server 30 refers to the operation log DB 3023 and identifies, as a related page, the viewed page that was displayed when the input operation of the question text related to the identified question was being performed.

After step S312, the question and answer server 30 identifies a portion of the related page that corresponds to the input operation of the question sentence (step S313).
Specifically, the visualization module 3035 of the question and answer server 30 refers to the operation log DB 3023 and identifies the part that the user was checking when the user performed the operation to input the question sentence, based on the amount of scrolling of the user's browsing operation. The processes from step S311 to step S313 up to this point are performed as a preparatory process for visualization, and the process results are continuously accumulated in the storage unit 302 of the question and answer server 30 at preset processing intervals.

Next, when the manager checks the customer's voice, he/she operates the manager terminal 20 to input an instruction to visualize the customer's voice. The manager terminal 20 accepts the input instruction to visualize the customer's voice (step S201).
The input instruction to visualize the customer's voice is transmitted to the question and answer server 30 .

After step S201, the question and answer server 30 receives an instruction to visualize the customer's voice (step S314).
Here, the instruction to visualize the customer's voice may include the following information as evaluation conditions.
- Information regarding the period to be evaluated - Information regarding the site to be evaluated - Information regarding the pages to be evaluated - Information regarding the attributes of the customers to be evaluated

After step S314, the question and answer server 30 displays text information related to the question on the related page (step S315).
Specifically, the visualization module 3035 of 30 displays text information related to the question in a portion of the identified related page that corresponds to the input operation of the identified question. The display mode at this time will be described later using a screen example. If an evaluation condition is specified, a process is executed to display text information related to the question on the related page only for the question that matches the evaluation condition.

After step S315, the administrator terminal 20 displays a page on which the customer's voice is visualized (step S202).
Specifically, a related page of the site showing text information related to the question is displayed on the display, which is the output device 24 of the administrator terminal 20. By checking this information, the administrator can understand what part of the page the user had a question about, and can contribute to improving the site.
This completes the process of visualizing customer feedback in the system 1.
The process of receiving the visualization instruction by the administrator terminal 20 (step S201) may be used as a trigger to execute the process of identifying the text information related to the question by the question and answer server (step S311) and subsequent processes.

<5. Screen example>
Next, a description will be given of an example of a screen in the system 1. First, a description will be given of an example of a screen in the customer terminal 10. FIG.
As shown in Fig. 11, a site page is displayed on the customer terminal 10. In the illustrated example, explanations to the customer regarding the use of the site are displayed. When the input form indicated by the symbol A in these explanations is clicked, an input field for inputting requests regarding the contents of the order is displayed. Such an input form also corresponds to the text input form of the present invention.

In addition, when the chatbot startup icon shown by symbol B is clicked, a chatbot with an automatic response function is started and a question input window pops up. By inputting a question into the input window, the user can check the reply from the chatbot. Such an input form for using a chatbot also corresponds to the text input form of the present invention.

<5-1. Example of a first screen of the administrator terminal 20>
Next, a description will be given of an example of a screen on the manager terminal 20. In the example of the screen on the manager terminal 20, a display form when customer comments are visualized on the page shown in FIG.
FIG. 12 is a diagram showing a first example of a screen of the administrator terminal 20 in the system 1, and shows a first example of a display form when customer feedback is visualized on a page.

As shown in FIG. 12, in the first screen example, multiple pieces of text information related to the question are displayed so as to be superimposed on the page. The text information related to the question is displayed in a partitioned area on the page. The partition in which the text information is displayed is located in a location identified as a location corresponding to the input operation of the question text.

In the illustrated example, text information is displayed in rectangular sections S1 to S4. By performing a mouse-over operation to place the mouse pointer over each of sections S1 to S4, the section is enlarged and additional information related to the question may be newly displayed.

Here, sections S1 to S3 each display a question entered by the customer on input form A. The display positions of sections S1 to S3 are set based on the amount of scrolling of the viewed page when input is made on input form A.

Meanwhile, in section S4, the operation content immediately before the question was asked using the chatbot and a link to detailed information about the question are displayed. Clicking on the link to the detailed information about the question displays the detailed information about the question. The detailed information about the question may include, for example, the following information:
- Text information of the question - Answered content - Type of input text, including question category - Results of a survey of users regarding their satisfaction with the answer (e.g., whether their question was resolved)
- Attribute information of the user who asked the question - Information on whether the answer resulted in a conversion (use of the service) The color of the sections S1 to S4 may be changed according to the content of the text information. For example, the color of the section may be light blue if the content is related to a question, and orange if the content is related to a complaint.

<5-2. Example of a second screen of the administrator terminal 20>
Next, another display mode of the page in which the customer's voice is visualized will be described.
FIG. 13 is a diagram showing a second example of a screen of the administrator terminal 20 in the system 1, and shows a second example of the display form of a page in which customer feedback is visualized and displayed on the administrator terminal 20.

13, in the display mode according to the second example, visualized information is displayed in spaces S1 to S4 outside the margins of a page. In the illustrated example, spaces S1 to S4 are provided on the right side of the page margin.
The positions of the comment icons C1 to C4 displayed on the page enable the user to determine which of the spaces S1 to S4 correspond to the input operation of the question. That is, the position of the comment icon C1 indicates the position corresponding to the input operation of the question written in the space S1. Similarly, the comment icons C2 to C4 also indicate the positions corresponding to the input operation of the question written in the spaces S2 to S4, respectively.
The positions of the spaces S1 to S4 are not limited to the right side of the page, but can be changed as desired.

<6. Summary >
As described above, in the system 1, the acquired text information is displayed in association with a portion of a related page that is related to the input operation of the text information. It is possible to visualize what questions customers have and what their opinions are of the site, which can be used to improve the site.

In addition, in system 1, the related page on which customer opinions are visualized corresponds to the viewed page, which is the page displayed when the input operation was performed among multiple pages. The location corresponding to the input operation corresponds to a location on the viewed page that is identified based on the amount of scrolling of the user's viewing operation. Therefore, the page that the user was viewing when the input operation of the question text was performed can be estimated as the page on which a question related to the question arose.

Furthermore, the system 1 displays the acquired text information in a partitioned area on the relevant page so as to be superimposed on the relevant page. This makes it possible to clearly indicate which part of the page caused the user to have a certain question.

In addition, system 1 displays the acquired text information in the margin of the related page. Therefore, compared to a configuration in which the information is superimposed on the page, it is possible to ensure the visibility of both the page contents and the visualized text information.

In addition, in system 1, at least one of information related to the attributes of the input operation, information related to the operation history, and information related to the user's attributes is displayed in association with the location corresponding to the input operation alongside the text information. Therefore, by allowing the administrator to check the supplementary information related to the question in addition to the content of the question as customer feedback, a higher resolution analysis can be performed on the visualized customer feedback.

7. Modifications
The processing performed by the system 1 may include other processing.
For example, in the above-mentioned visualization process of customer opinions, the visualization module 3035 identifies the viewed page displayed when the question input operation is performed as the related page, but this is not limited to this. The visualization module 3035 may identify at least one of a plurality of transition pages identified by tracing back the screen transitions in the operation history in chronological order from the viewed page displayed when the question input operation is performed as the related page.

For example, depending on the site configuration, it may take several screen transitions before the chatbot's inquiry form can be launched on the page where the user has a question. In such cases, it is possible that the user may not have had a question on the page that was displayed when the question was entered.

In this case, the visualization module 3035 can further identify, from among the multiple transition pages, a page with a large amount of operation time or operation amount, as identified from the operation history, as a related page. In other words, the visualization module 3035 can refer to the operation history and estimate that the user had a question about the question text on a viewed page with a large amount of stay time or a page on which some operation (e.g., a login operation) was attempted many times.

Furthermore, the visualization module 3035 may identify, from among a plurality of transition pages, a viewed page that is highly related to the acquired text information.
That is, it can be assumed that the user has a question about the question on the viewed page on which content related to the content of the text information related to the question is described.
For example, if the question is about a login method and one of the multiple transition pages includes a login screen, the transition page including the login screen can be identified as a related page.

The visualization module 3035 can also identify a location among multiple transition pages based on the amount of scrolling of the user's browsing operation immediately before the transition operation to the next page in the operation history, as a location related to the input operation of the question text on the identified related page.

In this way, in system 1 according to the modified example, at least one of a plurality of transition pages identified by tracing back the screen transitions in the operation history in chronological order from the viewed page, which is the page displayed when the input operation was performed, can be identified as a related page. The location of the transition page that corresponds to the input operation corresponds to a location identified based on the amount of scrolling of the viewing operation by the user immediately before the transition operation to the next page in the screen transitions in the operation history. Therefore, one of the pages that was viewed shortly before the input operation of the question text was performed can be estimated as the page on which a question related to the question arose.

Furthermore, in the system 1 according to the modified example, among the multiple transition pages, a page with a large amount of operation time or operation amount, as determined from the operation history, can be identified as a related page. Therefore, if several pages have been transitioned before the operation of inputting a question is performed, the viewed page with a particularly large amount of operation time and operation amount can be estimated as the page on which a question related to the question has arisen.

Furthermore, in the system 1 according to the modified example, among multiple transition pages, pages that are highly related to the acquired text information can be identified as related pages. Therefore, if several pages have been transitioned before the input operation of the question text is performed, it is possible to accurately estimate the page on which a question related to the question arose.

8. Other Modifications
As another modification, the question and answer DB 3022 may be realized by a server that installs a program on a computer. The components constituting the question and answer server 30 may be distributed and realized on multiple computers. In fact, the service pages on which the system 1 is used are not limited to pages on a website displayed by a web browser, but also include pages on an app that is displayed by executing application software.

Furthermore, in the visualization process of customer opinions, the question and answer server 30 may refer to the results of answers to questions to select question sentences to be subject to the visualization process.
Specifically, the visualization module 3035 of the question and answer server 30 may exclude the following questions from the visualization process:
・Questions whose content is unclear and cannot be categorized ・Questions whose likelihood of categorizing does not meet a certain threshold ・Questions for which there is no corresponding category ・Questions for which the user has given a negative response in the questionnaire after answering the question ・Questions whose content is, for example, strongly related to the user's personal circumstances and cannot be generalized to the extent that it can contribute to improving the site

Furthermore, when a large number of similar questions are confirmed, the question and answer server 30 may omit some of the text information relating to the questions, rather than displaying all of the text information relating to the questions on the relevant related page.
For example, when multiple questions that fall into the same category are found, the visualization module 3035 of the question and answer server 30 may display only the most representative question sentence on the related page, and may also display the number of similar questions. This ensures visibility, and the number of questions can suggest the scale of customers who actually have doubts. In this case, the visualization module 3035 may select the question sentences to be visualized by random sampling.

In the above embodiment, the system 1 is provided with both a function as a chatbot and a function as a customer voice visualization system, but this is not limited to the above. The system 1 does not need to have a function as a chatbot. In other words, it is sufficient for the system 1 to have an input form that can acquire the user's voice as text information. Examples of such input forms other than chatbots include the following.
・Inquiry forms by email, etc. ・Inquiry forms by voice input (converting voice input by the user into text information)
・Interpersonal chat system without automatic responses

The above describes the embodiments of the present disclosure, but various modifications and combinations that may be necessary due to design convenience or other factors are intended to be included in the scope of the invention described in the claims or the invention corresponding to the specific examples described in the embodiments of the invention.

<9. Notes>
The contents of the present invention are described below.

(Appendix 1)
A program for causing a computer having a processor and a memory to execute the program, the program causing the processor to:
A step of acquiring text information input by a user into a text input form activated from at least one of a plurality of pages displayed on a customer terminal (step S301);
A step of referring to a user's operation history and identifying a relevant page related to the input operation of the text information from among the plurality of pages;
and a step of referencing the operation history and displaying the obtained text information in association with a portion of the related page corresponding to the input operation (step S314).

(Appendix 2)
What are related pages?
This refers to the viewed page, which is the page that was displayed when the input operation was performed among multiple pages.
The parts that correspond to input operations are:
The program according to claim 1, which refers to a portion of a viewed page that is specified based on a scroll amount of a user's viewing operation.

(Appendix 3)
What are related pages?
Among the multiple pages, at least one of multiple transition pages identified by tracing screen transitions in the operation history back in chronological order from a viewed page that was displayed when the input operation was performed,
The parts that correspond to input operations are:
The program according to claim 1, which refers to a part of a transition page that is specified based on a scroll amount of a browsing operation by a user immediately before a transition operation to a next page in a screen transition in an operation history.

(Appendix 4)
What are related pages?
The program according to claim 3, which specifies, among a plurality of transition pages, a page that has a large operation time or a large amount of operation, as identified from the operation history.

(Appendix 5)
What are related pages?
The program according to claim 3, which indicates a page among a plurality of transition pages that is highly relevant to the acquired text information.

(Appendix 6)
The processor further
The program described in Appendix 1 executes a step (step S314) of displaying, alongside the text information, at least one of information regarding the attributes of the input operation, information regarding the operation history, information regarding the user's attributes, and the type of input text in association with the location corresponding to the input operation.

(Appendix 7)
In the step of associating and displaying the acquired text information (step S314),
2. The program according to claim 1, which displays the acquired text information on the page so as to be superimposed on the related page.

(Appendix 8)
In the step of associating and displaying the acquired text information (step S314),
The program according to claim 1, which displays the acquired text information in a margin on a related page.

(Appendix 9)
A computer-implemented method having a processor and a memory, the method comprising the processor:
A step of acquiring text information input by a user into a text input form activated from at least one of a plurality of pages displayed on a customer terminal (step S301);
A step of referring to a user's operation history and identifying a relevant page related to the input operation of the text information from among the plurality of pages;
and a step of referencing the operation history and displaying the obtained text information in association with a portion of the related page corresponding to the input operation (step S314).

(Appendix 10)
A system having a computer including a processor and a memory, the system comprising:
A means for acquiring text information input by a user into a text input form activated from at least one of a plurality of pages displayed on a customer terminal;
A means for referring to a user's operation history and identifying a relevant page related to the input operation of the text information among a plurality of pages;
and a means for referencing the operation history and displaying the acquired text information in association with a portion of the related page that corresponds to the input operation.

1 Service page management system 10 Customer terminal 20 Administrator terminal 30 Question and answer server 302 Storage unit 3021 User database (user DB)
3022 Question and Answer Database (Question and Answer DB)
3023 Operation log database (operation log DB)
303 Control unit 3031 Transmission/reception control module 3032 Answer output module 3033 Question acquisition module 3034 Operation history acquisition module 3035 Visualization module

Claims (10)

A program to be executed by a computer having a processor and a memory,
The program causes the processor to:
acquiring text information input by a user into a text input form activated from at least one of a plurality of pages displayed on a customer terminal;
referring to an operation history of a user, and identifying a relevant page related to the input operation of the text information among the plurality of pages;
and referencing the operation history, and displaying the acquired text information in association with a portion of the related page that corresponds to the input operation. The related pages are:
A browsed page is a page that was displayed when the input operation was performed among the plurality of pages.
The portion corresponding to the input operation is
The program according to claim 1 , wherein the location of the viewed page is specified based on an amount of scrolling of a viewing operation by the user. The related pages are:
Among the plurality of pages, at least one of a plurality of transition pages identified by tracing back screen transitions in the operation history in chronological order from a viewed page that is a page that was displayed when the input operation was performed,
The portion corresponding to the input operation is
The program according to claim 1 , wherein the location of the transition page is identified based on a scroll amount of a browsing operation by the user immediately before a transition operation to a next page in a screen transition in the operation history. The related pages are:
The program according to claim 3 , wherein the page number indicates a page, among the plurality of transition pages, that has a large operation time or an operation amount, identified from the operation history. The related pages are:
The program according to claim 3 , wherein the transition page indicates a page that is highly related to the acquired text information among the plurality of transition pages. The processor further comprises:
The program according to claim 1, further comprising a step of displaying at least one of information regarding attributes of the input operation, information regarding the operation history, information regarding attributes of the user, and a type of input text in association with a location corresponding to the input operation, alongside the text information. In the step of displaying the acquired text information in association with the text information,
The program according to claim 1 , further comprising: displaying the acquired text information on the related page so as to be superimposed on the related page. In the step of displaying the acquired text information in association with the text information,
2. The program according to claim 1, further comprising: displaying the acquired text information in a margin of the related page. A method implemented by a computer having a processor and a memory, comprising:
The method further comprises the processor:
acquiring text information input by a user into a text input form activated from at least one of a plurality of pages displayed on a customer terminal;
referring to an operation history of a user, and identifying a relevant page related to the input operation of the text information among the plurality of pages;
and displaying the acquired text information in association with a portion of the related page corresponding to the input operation by referring to the operation history. A system having a computer including a processor and a memory,
The system further comprises:
A means for acquiring text information input by a user into a text input form activated from at least one of a plurality of pages displayed on a customer terminal;
a means for referring to a user's operation history and identifying a relevant page related to the input operation of the text information among the plurality of pages;
means for referring to the operation history and displaying the acquired text information in association with a portion of the related page corresponding to the input operation.