Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/90—Details of database functions independent of the retrieved data types

Definitions

• This disclosure relates to an automatic response system, an automatic response method, and a program that generate answers to questions.
• Patent Document 1 discloses a route guidance system for use in places like shopping malls and event venues.
• a display device shows the route, and the robot explains the route to the destination to the user using voice and body movements.
• Patent Document 1 can guide users, on behalf of employees, on routes to a specified destination. However, users have a wide variety of needs and situations. For this reason, the technology described in Patent Document 1 may not be able to provide information that is appropriate for the user.
• This disclosure has been made in light of the above, and aims to provide an automatic response system that can provide facility users with more appropriate responses.
• the automatic response system disclosed herein comprises a question acquisition unit that acquires questions about facility use from facility users who are users of the facility; a situation acquisition unit that acquires additional information indicating the situation of the facility user who asked the question; an answer generation unit that generates an answer to the question using the question, the additional information, and facility data that is data about the facility; and an output unit that outputs the answer.
• the automated response system disclosed herein has the advantage of being able to provide facility users with more appropriate responses.
• FIG. 1 is a diagram illustrating a configuration example of an automatic response system according to a first embodiment.
• FIG. 1 is a diagram showing an example of the configuration of a response generation unit according to a first embodiment;
• FIG. 1 is a diagram for explaining an automatic response according to the first embodiment.
• FIG. 1 is a diagram for explaining an automatic response according to the first embodiment.
• 1 is a flowchart showing an example of an automatic response processing procedure in the automatic response system according to the first embodiment. 5.
• FIG. 1 is a diagram showing an example of dynamic facility data according to the first embodiment;
• FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment.
• FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment.
• FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment.
• FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment.
• FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment.
• FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment.
• FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the first embodiment.
• FIG. 10 is a diagram showing a specific example of a response
• FIG. 10 is a diagram showing another specific example of a response that takes the situation into consideration in the automatic response system of the first embodiment.
• FIG. 1 is a diagram showing an example of the configuration of a computer system that realizes each of the information processing units according to the first embodiment.
• FIG. 10 is a diagram illustrating a configuration example of an automatic response system according to a second embodiment.
• FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the second embodiment.
• FIG. 10 is a diagram showing a specific example of a response that takes into account the situation in the automatic response system of the second embodiment.
• FIG. 10 is a diagram illustrating a configuration example of an automatic response system according to a third embodiment.
• FIG. 1 is a diagram illustrating an example of the configuration of an automatic answering system according to an embodiment.
• the automatic answering system 10 of the present embodiment accepts questions from facility users regarding facility use and outputs information related to facility guidance as a response to the questions.
• the facility may be, for example, a train station, a hospital, a commercial facility, a hotel, a building, an event venue, a conference center, a theme park, a railway facility including multiple stations (including the interior of a railway car), a school, a business office, a factory, or the like, but is not limited thereto as long as it is available to multiple users.
• the facility may be a facility available to an unspecified number of people, such as a train station, a facility available to specific people, such as a school or business office, or a facility available to both specific and unspecified people.
• the automatic response system 10 includes a data management unit 11, a question acquisition unit 12, a situation acquisition unit 13, an answer generation unit 14, and an output unit 15.
• the data management unit 11 and the answer generation unit 14 constitute an information processing unit 20, which is an information processing device.
• the automatic response system 10 is installed, for example, within a facility, but at least some of the components that make up the automatic response system 10 may be installed outside the facility.
• the data management unit 11 manages facility data, which is data related to facilities.
• the facility data is data used by the answer generation unit 14 when generating answers, and may include not only data related to the facility itself, but also data about the surrounding area of the facility and data related to the facility outside the facility.
• facility data includes, but is not limited to, at least one of the following: map information within the facility, facility status information related to the status of the facility, map information around the facility, facility information issued by at least one of the facility manager and facility employees, and weather information for the area including the facility.
• the map information includes information that enables users to determine routes to potential locations within the facility that they will visit, such as stores and conference rooms.
• the map information may include information represented by a graph in which links represent corridors within the facility and the potential locations represent nodes.
• the map information may also include attributes of the corridors, such as whether the corridors include stairs, slopes, elevators, or escalators.
• the map information within the facility may include image data showing predetermined routes for each potential location that the user will visit.
• the map information may include image data for each corridor attribute, such as whether the corridors include slopes, elevators, or escalators.
• Facility status information includes, but is not limited to, congestion information indicating the degree of congestion at the facility, information such as road closures within the facility, information indicating whether equipment within the facility is broken, and road conditions indicating the condition of surrounding roads.
• Facility-generated information is information that facility managers or employees want to convey to users, such as "It is raining, so the facility is slippery" or "The venue for event X starting at 10:00 today has been changed from X to Y.”
• Weather information includes, but is not limited to, at least one of the following: weather (sunny, rainy, strong winds, etc.), temperature, and humidity.
• the facility data may include timetables for trains and other railway vehicles, and if there is a store within the facility, it may include at least one of the following: information about the products sold by the store, information about the food and drink served at the store, information indicating the store's main target demographic, information about the store's recommended products, and store information transmitted by store employees.
• Store information is information that store employees want to convey to customers, such as "Today, Shop A is having a rainy day sale,” "Shop B is currently running a limited-time sale,” or "Restaurant K has a wide selection of cold drinks available that are perfect for hot days.”
• Facility data may include static data that does not change over a short period of time, such as map information within the facility or information indicating the store's main target demographic.
• Facility data may also include dynamic data that may change over a short period of time, such as facility status information, facility transmission information, and weather information.
• Facility data may also include both static and dynamic data.
• the data management unit 11 includes a data acquisition unit 111, a pre-processing unit 112, and a data storage unit 113.
• the data acquisition unit 111 acquires facility data and outputs the acquired facility data to the pre-processing unit 112.
• the data acquisition unit 111 may acquire static facility data by accepting data input from, for example, a facility manager, operator, or store employee, or by receiving the data from another device (not shown).
• the data acquisition unit 111 may receive map information about the area around the facility from a device that manages the map information.
• a terminal device that can be operated by a facility manager, operator, store employee, etc. may accept input of facility data from the facility manager, operator, store employee, etc., and the data acquisition unit 111 may receive the facility data from the terminal device.
• the data acquisition unit 111 may acquire dynamic data from facility data by, for example, receiving sensor information acquired by a sensor not shown in FIG. 1 , such as a surveillance camera or microphone, and calculating the dynamic data based on the received sensor information.
• the dynamic data may include data acquired by a sensor installed inside or around the facility.
• the data acquisition unit 111 may calculate congestion information based on video that is sensor information acquired by a surveillance camera.
• the data acquisition unit 111 may output sensor information to the preprocessing unit 112, and the preprocessing unit 112 may calculate the dynamic data from the sensor information.
• another device not shown may calculate the dynamic data based on sensor information, and the data acquisition unit 111 may receive the dynamic data from the other device.
• the data acquisition unit 111 may acquire facility transmission information and store transmission information as text data by accepting input from the sender, or may accept input of facility transmission information and store transmission information by voice.
• information broadcast within the facility may be acquired as facility transmission information and store transmission information from a microphone used for in-facility broadcasting at the facility.
• microphones may be installed near the location where in-facility broadcasting is made, and information broadcast within the facility may be acquired from the microphone as facility transmission information and store transmission information.
• the preprocessing unit 112 performs preprocessing on the facility data received from the data acquisition unit 111.
• the preprocessing unit 112 stores the preprocessed facility data in the data storage unit 113 as a database.
• Preprocessing is, for example, a process of converting the data into a data format accessible by the answer generation unit 14.
• the preprocessing unit 112 vectorizes the facility data and stores the vectorized facility data in the data storage unit 113.
• Vectorization includes, for example, a process of converting words in natural language processing into vectors, which are distributed representations according to the meaning of the words.
• the content of the preprocessing performed by the preprocessing unit 112 is not limited to the example described above, and may be determined depending on the processing method used by the answer generation unit 14. Depending on the processing method used by the answer generation unit 14, the preprocessing unit 112 may not be provided, and the facility data may be stored directly in the data storage unit 113.
• the data storage unit 113 stores the facility data.
• the question acquisition unit 12 acquires questions about facility use from facility users (hereinafter also referred to as facility users). More specifically, the question acquisition unit 12 accepts questions about the facility from facility users and outputs the accepted questions to the answer generation unit 14.
• the question acquisition unit 12 may acquire the questions as voice data, or may acquire the questions as text data by accepting input through operation by the facility user. That is, the question acquisition unit 12 may be equipped with a microphone, or may be equipped with input means such as a touch panel that accepts input through user operation, or may be equipped with both a microphone and input means.
• the question acquisition unit 12 may also accept image input. In this case, for example, the image input is accepted by capturing an image with a camera. For example, a facility user may present a photo of a product, store, etc.
• the camera may be a camera used as the situation acquisition unit 13 described below.
• the question acquisition unit 12 receives a question using an image, it outputs the image data together with the question to the answer generation unit 14.
• the situation acquisition unit 13 acquires additional information indicating the situation of the facility user who asked the question and outputs the acquired additional information to the answer generation unit 14.
• the situation acquisition unit 13 is equipped with a sensor such as a camera or an infrared sensor, and acquires the detection results of the sensor as additional information (auxiliary information).
• the additional information includes video data of the questioner.
• the situation acquisition unit 13 may acquire the additional information indicating the facility user's situation as text data by accepting input through operation by the facility user, or may acquire the additional information indicating the facility user's situation as audio data.
• the situation acquisition unit 13 acquires the additional information as text data or audio data, for example, a question inquiring about the situation is presented to the facility user by the output unit 15, and the situation acquisition unit 13 acquires the answer to the question as additional information.
• the question acquisition unit 12 may have the function of the situation acquisition unit 13.
• the microphone used by the situation acquisition unit 13 to acquire audio data may be shared with the question acquisition unit 12.
• the additional information is, for example, situation information indicating the situation of the facility user asking the question.
• the additional information may be situation information indicating the situation of the facility user asking the question, as well as attribute information indicating the attributes of the facility user asking the question.
• video data acquired by a camera can be used to understand both the situation and attributes of a facility user.
• the situation acquisition unit 13 may present the facility user with a question asking about their attributes in addition to their situation via the output unit 15, and the situation acquisition unit 13 may acquire the answer to the question as additional information.
• the additional information may include situation information and attribute information.
• the situation information and attribute information may be acquired by different means, such as acquiring video data as situation information and acquiring attribute information as a response to a question asking about their attributes.
• the method of acquiring the attributes of a facility user is not limited to acquiring them from video data acquired by a camera.
• the attributes of an individual may be identified using an individual's identification results.
• the answer generation unit 14 generates an answer regarding facility guidance using the question received from the question acquisition unit 12, the additional information received from the situation acquisition unit 13, and the facility data stored in the data storage unit 113, and outputs the generated answer to the output unit 15. Specifically, the answer generation unit 14 understands the question and generates an answer using the facility data, taking into account the situation indicated by the additional information.
• the answer generation unit 14 is capable of generating an answer using data in multiple formats, including, for example, audio, text, and images, as input, and is capable of outputting answers in multiple formats. That is, for example, the answer generation unit 14 is capable of, but is not limited to, performing multimodal output in response to multimodal input such as audio data, text data, and video data.
• performing multimodal output in response to multimodal input will also be referred to as multimodal processing. Details of the answer generation unit 14 will be described later.
• the output unit 15 presents the answer to the facility user who asked the question by outputting the answer received from the answer generation unit 14.
• the answer may be output as audio, text data, an image, video, or a combination of two or more of these. Therefore, the output unit 15 includes, for example, at least one of a speaker and a display device such as a display or a touch panel.
• the output unit 15 outputs the answer according to the format of the answer received from the answer generation unit 14. For example, if the answer received from the answer generation unit 14 is audio data, the output unit 15 outputs the answer as audio. If the answer received from the answer generation unit 14 is at least one of video data and text data, the output unit 15 outputs the answer by display.
• the output unit 15 includes a touch panel, the touch panel may be used as an input means for the question acquisition unit 12 and as an input means for the situation acquisition unit 13.
• FIG. 2 is a diagram showing an example of the configuration of the answer generation unit 14 in this embodiment.
• the answer generation unit 14 includes a situation recognition unit 141 and a generation unit 142.
• the situation recognition unit 141 recognizes the situation based on the additional information received from the situation acquisition unit 13 and outputs the recognized situation to the generation unit 142.
• the situation recognition unit 141 recognizes the facility user who asked the question. However, if the facility user is accompanied by someone, such as a family or couple, hereinafter, not only the facility user who actually asked the question but also the accompanying person of the facility user will be treated as the facility user who asked the question.
• the situation recognized by the situation recognition unit 141 includes, but is not limited to, at least one of the following: what the questioner is wearing (such as the questioner's clothing or accessories worn by the questioner), whether the questioner is using an assistive device such as a wheelchair or cane (whether or not the assistive device is being used), the number of questioners, the questioner's emotions, the size of luggage the questioner is carrying, and the direction of movement of the questioner.
• the situation recognition unit 141 uses the additional information to recognize the situation and attributes of the questioner, and outputs the recognized situation and attributes to the generation unit 142.
• the attributes of the questioner may include, for example, at least one of gender, age, generation, presence or absence of a disability, and occupation.
• the generation unit 142 generates an answer regarding facility guidance using the situation (or situation and attributes) received from the situation recognition unit 141, the question received from the question acquisition unit 12, and facility data stored in the data storage unit 113, and outputs the generated answer to the output unit 15.
• the answer generation unit 14 generates answers that reflect the questioner's situation. For example, if a questioner using a wheelchair asks how to get to a destination, the answer generation unit 14 can generate an answer that shows a route that does not have stairs and uses ramps or elevators.
• FIGS. 3 and 4 are diagrams illustrating the automatic response of this embodiment.
• the automatic response system 10 is equipped with a camera as the situation acquisition unit 13, and a touch panel and speaker as the output unit 15.
• a keyboard software keyboard
• the automatic response system 10 is further equipped with a microphone as the question acquisition unit 12.
• the answer generation unit 14 of the automatic response system 10 recognizes the situation that the questioner 30 is carrying large luggage from video data received from a camera, which is an example of a sensor used as the situation acquisition unit 13, and generates an answer using the recognized situation, the audio data of the question received from the question acquisition unit 12, and the facility data.
• the situation recognized in the example shown in Figure 3 is not limited to “carrying large luggage,” but may also be “both hands are full” or “difficulty walking.”
• the answer generation unit 14 since the questioner 30 is carrying large luggage, the answer generation unit 14 generates a route to the station platform, i.e., a route using an elevator, as shown at the bottom of Figure 3, based on the map information within the facility in the facility data, rather than a route using stairs or escalators. Note that there are no particular restrictions on the method of generating the route, and methods such as searching for the shortest route within a range that meets the conditions can be used.
• the answer generation unit 14 generates a diagram (image) showing the route and text data explaining the diagram as an answer, and outputs the generated answer to the output unit 15, which then outputs the answer.
• the answer is displayed as text and a diagram showing the route on the touch panel of the output unit 15, but the output method is not limited to this.
• the answer may also be output as audio by a speaker of the output unit 15, or the answer may be output as audio by a speaker instead of text.
• the automatic response system 10 is composed of a camera, which is an example of a sensor used as the situation acquisition unit 13, and a main body 40.
• the main body 40 includes an information processing unit 20, a question acquisition unit 12, and an output unit 15, all of which are not shown in FIG. 4.
• the sensor used as the situation acquisition unit 13 may be provided separately from the main body 40.
• the automatic response system 10 may also include multiple sensors as the situation acquisition unit 13; for example, sensors may be provided both on the main body 40 and outside the main body 40.
• the questioner 30 is using a wheelchair and asks the automatic response system 10, "Where is the exit?"
• the microphone which is the question acquisition unit 12, receives the question.
• the answer generation unit 14 generates an answer that includes a slope with no steps as a route to the exit, and the output unit 15 displays the answer.
• the answer generation unit 14 assumes that the questioner will be leaving the facility, and if it is raining based on the facility data, the answer generation unit 14 may add a sentence that corresponds to the outside weather, such as "It's raining outside, so please be careful on your way home.”
• the question is about an exit, but by including map information about the area around the facility in the facility data, for example, if a questioner asks, "Please tell me the way to the department store" when the facility is inside a train station, the automatic response system 10 may be able to provide guidance on the way from the station to the department store. In this way, the answer generation unit 14 may respond with information about the area around the facility as information related to facility guidance.
• the facility data may include information indicating the train lines that stop at the station, the stations on those lines, the nearest stores and tourist attractions to the station, etc., so that a question such as, "Which train line should I take to get to VV?" can be answered by indicating which train line to take and at which station to get off.
• the automatic response system 10 is realized as a single device, and in the example shown in Figure 4, it is realized by a camera, which is an example of a sensor used as the situation acquisition unit 13, and the main body 40, but the configuration of the device as hardware for the automatic response system 10 is not limited to these examples.
• the information processing device which is the information processing unit 20 shown in Figure 1
• the input/output device including the question acquisition unit 12, situation acquisition unit 13, and output unit 15 may be provided in separate locations.
• the input/output device and the information processing device may each be provided with a transceiver unit for communication, and information may be exchanged via the transceiver unit.
• Figures 3 and 4 are merely examples, and the situations considered by the automatic response system 10 and the answers presented by the automatic response system 10 are not limited to these examples.
• the answer generation unit 14 may be, for example, generative AI (Artificial Intelligence).
• the generative AI may be, for example, multimodal AI such as Gemini (registered trademark) or GPT (Generative Pre-trained Transformer)-4V, image generation AI such as Stable Diffusion, large-scale language model such as GPT-3, or speech generation AI such as Murf. AI.
• multimodal AI is used for the answer generation unit 14, the answer generation unit 14 has the functions of a situation recognition unit 141 and a generation unit 142.
• the answer generation unit 14 Before accepting a question from a facility user, such as when the automatic response system 10 is started, an instruction such as "Please consider the situation of the facility user who asked the question and generate video data, text data, and audio data as necessary to provide an answer" is input to the answer generation unit 14.
• the answer generation unit 14 is also instructed to use facility data as appropriate.
• the answer generation unit 14 takes into account the situation of the questioner 30, references the facility data, and generates an answer regarding facility guidance using at least one of video data, text data, and audio data.
• the specific circumstances to be considered may be input to the answer generation unit 14, or data indicating a definition of the circumstances may be included in the facility data.
• data indicating a definition of the circumstances may be included in the facility data.
• an instruction such as "The circumstances of the facility user who asked the question include whether the questioner is using an assistive device such as a wheelchair or a cane, the number of people asking, the emotions of the questioner, the clothing of the questioner, the items worn by the questioner, the size of luggage the questioner is carrying, and the direction of movement of the questioner" may be input to the answer generation unit 14.
• the format of the answer will be determined by the answer generation unit 14.
• instructions indicating those rules can be input to the answer generation unit 14, or data indicating the rules can be included in the facility data.
• typical examples, norms, guidelines, etc. of answers corresponding to questions can be stored in the data storage unit 113 as facility data or separately from the facility data, and the answer generation unit 14 can refer to those rules when creating an answer.
• pre-learning can be performed to learn these rules.
• facility data is used to generate answers, but part of the facility data may be used as is as part of the answer. For example, if the facility data includes images of products sold by the store, the answer generation unit 14 may include those images in the answer.
• the answer generation unit 14 may input the situation of the questioner 30 to be considered and the question into the multimodal AI of the answer generation unit 14 in advance, check whether the desired answer is obtained, and if the desired answer is not obtained, instruct the multimodal AI to provide a correct answer corresponding to the situation and question.
• the correct answer corresponding to the situation and question may also be included in the facility data. For example, as shown in FIG. 3, when a questioner 30 carrying large luggage asks about the route to their destination, if the answer generation unit 14 generates an answer that includes stairs, the answer generation unit 14 may be given an instruction such as "If the questioner has large luggage, use the elevator instead of the stairs," or the instruction may be included in the facility data.
• the instruction may be broken down into information such as "It is difficult for the questioner to walk if they are carrying large luggage or have both hands full” and information such as "If it is difficult to walk, use the elevator instead of the stairs.”
• instructions for generating appropriate answers for each situation and attribute may be given to the answer generation unit 14, or these instructions may be included in the facility data.
• the question acquisition unit 12 may accept the question as text data, and the output unit 15 may output the answer as text data.
• the question acquisition unit 12 may accept the question as audio, and the question acquisition unit 12 or the answer generation unit 14 may convert the audio data of the question into text data.
• a large-scale language model may function as the generation unit 142, and a situation recognition model that recognizes the situation from video data may be used as the situation recognition unit 141.
• the situation recognition model may be, for example, a trained model that has been trained by machine learning to infer the situation from video data.
• An example of machine learning used to generate the trained model is supervised learning such as a neural network, but it may also be unsupervised learning, reinforcement learning, etc., and is not limited to these.
• the situation recognition model may also be a model that recognizes attributes.
• the situation recognition model may be a combination of multiple models, such as a combination of a general emotion recognition model that recognizes emotions from video data and an image recognition model that recognizes the presence or absence of assistive devices, the size of luggage, clothing, etc. from video data.
• the generation unit 142 generates an answer as text data using, for example, text data indicating the situation (or the situation and attributes) received from the situation recognition unit 141 and the question.
• the generation unit 142 may further include a voice generation AI that converts the answer generated by the large-scale language model into voice data, or the generation unit 142 may further include an image generation AI that converts the answer generated by the large-scale language model into image data.
• output may be performed in all supported output formats, or rules for selecting the output format may be determined in advance and the output format may be determined in accordance with the rules.
• a rule may be established that associates the situation (or situation and attributes) of the asker 30 with the output format, or a rule may be established that outputs in the same format as the format of the question asked by the asker 30.
• a rule may be established that associates the type of question content (for example, a question about the route to a destination, a question asking about recommended shops, etc.) with the output format.
• the rules for determining the output format are not limited to these examples.
• the facility data is vectorized by the preprocessing unit 112 and stored in the data storage unit 113, allowing the answer generation unit 14 to use the facility data as is. Therefore, the facility data can be reflected in the answers of the answer generation unit 14 without the need for additional learning.
• a general automatic conversation program such as a chatbot may be used for the generation unit 142.
• a situation recognition model may be used as the situation recognition unit 141, just as when a large-scale language model is used for the generation unit 142, and text data indicating the situation (or situation and attributes) recognized by the situation recognition unit 141 may be input to the generation unit 142.
• the automatic conversation program may be a rule-based (scenario-based) program in which response rules are defined in advance, or a machine learning program.
• a rule-based automatic conversation program for example, answers corresponding to the content of questions for each situation (or situation and attributes) are determined in advance and set in the automatic conversation program.
• the machine learning in a machine learning automatic conversation program may be supervised learning, unsupervised learning, or reinforcement learning.
• supervised learning is used in the automatic conversation program, a trained model is generated using multiple training datasets containing questions for each situation (or situation and attributes) and corresponding answers, which are correct answer data, and the generation unit 142 generates an answer by inputting the situation (or situation and attributes) and question corresponding to the questioner 30 into the trained model.
• the answer generated by the automatic conversation program may be converted into voice data by a voice generation AI or into image data by an image generation AI.
• output may be performed in all supported output formats, or rules for selecting the output format may be defined in advance and the output format may be determined in accordance with the rules.
• the questioner 30 may not only be an unspecified person visiting the facility, but may also be a specific person, such as an employee, manager, or student if the facility is a school. That is, facility users may include specific people that have been specified in advance. For example, if the facility is intended for specific people, only specific people may be considered as questioners 30. In this case, for example, a facial photograph of the specific person may be included in the facility data, the situation acquisition unit 13 may acquire camera video data as additional information, and the answer generation unit 14 may identify the questioner 30 based on the facial photograph and generate an answer based on the identification result.
• This video data is an example of personal authentication information for identifying an individual.
• personal identification information For specific people who may use the facility, their identification information (hereinafter also referred to as personal identification information) may be associated with a facial photograph and stored in the data storage unit 113 as facility data. Furthermore, for each piece of personal identification information, individual information about the person corresponding to that personal identification information may be stored in the data storage unit 113 as facility data. Individual information is information relating to a specific person's individual use of the facility, such as, but not limited to, one or more of the following: schedule, accessible locations, and the scope of information that can be obtained (authority to view information).
• the facility is a school
• information indicating the classes each student is taking is included in the individual information of the facility data
• class location information indicating the time and location (classroom, etc.) of each class is included in the facility data.
• the answer generation unit 14 can determine the next class the student will take based on the individual information of the student, and the time and location of the class using the class location information.
• the answer generation unit 14 generates an answer that reflects the information it has determined, such as "The next class is class Y, which will be held in classroom X in building 3." At this time, the answer may also include an image showing the route from the current location to classroom X in building 3. When generating an answer, for example, the situation is taken into consideration, as described above.
• the facility data may include individual information not only for students but also for school staff.
• the individual information may include information indicating the schedule of classes taught by staff members, the time and location of the staff members' exam supervision, etc.
• a response such as, "The next exam supervision will be in classroom Z in building 2, starting at 11:00" is generated.
• the content of the response described above is merely illustrative, and the content of the response is not limited to the above example.
• a passcode assigned to each individual may be included in the facility data, and the automatic response system 10 may accept input of the passcode when a question is asked, thereby identifying the individual.
• the automatic response system 10 may be equipped with a device that performs biometric authentication other than facial recognition, and the device may be used to identify the individual.
• the personal authentication information may be a passcode, biometric authentication information, etc.
• attribute-related information is information related to facility use for each attribute, and includes schedules, accessible locations, the range of information that can be obtained (information viewing authority), and contact information.
• the attributes may include, for example, status (student, professor, assistant professor, teacher, office worker, etc.) and affiliation (faculty, department, selection).
• the attributes may include job title, affiliation (department, faculty, division), qualifications held, etc.
• the attributes may include at least one of status, affiliation, and job title.
• the response generation unit 14 may identify individuals using a method similar to that for identifying individuals described above, understand their attributes based on the individual information, and generate a response according to their attributes.
• the answer generation unit 14 identifies the department to which the student belongs as an attribute of the student (questioner 30). The answer generation unit 14 also identifies the class corresponding to the identified attribute based on the attribute-related information, and generates an answer indicating the location of the class based on the class location information. For example, if accessible locations are specified for each attribute, when the questioner 30 asks how to get to a destination, the unit 14 obtains a route that travels within the range of accessible locations, and generates the obtained route as the answer.
• the unit 14 generates an answer for the questioner 30 using information within the range that is permitted to be viewed. For example, if the attribute-related information includes contact information for each attribute, the unit 14 may generate an answer by adding the contact information to a direct answer corresponding to the question, regardless of the content of the question.
• the contact information may include a notice of a change in class.
• the questioner 30 may be both an unspecified person, such as a customer of the facility or a visitor to the university, and a specific person, such as an employee or student of the facility.
• the answer generation unit 14 may change the content of the answer depending on whether the questioner 30 is an unspecified person or a specific person.
• the attributes of the questioner 30 may include, for example, whether the questioner is a specific person, and rules for generating answers for each attribute, such as accessible locations, the range of information that can be obtained (information viewing authority), and contact information, may be defined in the attribute-related information, as in the above example.
• the attribute of whether the questioner is a specific person is determined in the same way as when identifying the individual specific person described above, and is achieved, for example, by including facial information or the like for the specific person in the facility data in advance.
• the attribute-related information may include information that distinguishes between facility data used to generate answers for specific people and facility data used to generate answers for unspecified people.
• attributes such as affiliation and position may be further included as attributes, as in the above example, and answers may be generated according to these attributes.
• the answer generation unit 14 will generate, as an answer, information indicating the warehouse where product P is stored based on the facility data if the questioner 30 is an employee, and will generate an answer indicating that the question cannot be answered if the questioner 30 is not an employee. At this time, for example, the answer generation unit 14 will further generate an answer depending on the situation, as described above.
• the answer generation unit 14 may cause the output unit 15 to output information to the questioner 30 asking for the missing information. That is, for example, if the answer generation unit 14 of the automatic response system 10 cannot recognize the question or if there is insufficient information to generate an answer, the answer generation unit 14 may generate an answer to ask the questioner 30 to repeat the question.
• the answer generation unit 14 may output at least one of text data and audio data such as "Please repeat the question again" to the output unit 15, and the output unit 15 may output at least one of text and audio.
• the answer generation unit 14 may output to the output unit 15 at least one of text data and voice data for identifying the platform, such as "Is this the K line platform or the L line platform?", and the output unit 15 may output at least one of text and voice.
• the answer generation unit 14 may output at least one of text data and voice data for identifying the number of people, such as "How many people are there?", and the output unit 15 may output at least one of text and voice.
• FIG. 5 is a flowchart showing an example of the automatic response processing procedure in the automatic response system 10 of this embodiment.
• the automatic response system 10 acquires facility data (step S1).
• the data acquisition unit 111 acquires the facility data and outputs it to the preprocessing unit 112.
• the facility data may be static data as described above, dynamic data, or both.
• the automatic response system 10 stores the facility data (step S2).
• the preprocessing unit 112 performs preprocessing on the facility data, and stores the preprocessed facility data in the data storage unit 113.
• Preprocessing is, for example, vectorization as described above, but is not limited to this, and may involve conversion to a data format that can be accessed by the answer generation unit 14, or a data format that can be accessed quickly by the answer generation unit 14. Also, as described above, preprocessing does not necessarily have to be performed.
• the automatic response system 10 acquires the question content (step S3).
• the question acquisition unit 12 acquires the question from the questioner 30 as voice or text data, thereby acquiring the question content, and outputs the acquired question to the answer generation unit 14.
• the automatic response system 10 acquires the situation (step S4).
• the situation acquisition unit 13 acquires additional information indicating the situation of the questioner 30 and outputs the additional information to the answer generation unit 14.
• the situation acquisition unit 13 may acquire sensor information acquired by a sensor such as a camera as the additional information, or may acquire the additional information through input by voice or text data from the questioner 30.
• the automatic response system 10 generates a response (step S5).
• the response generation unit 14 grasps the situation (the situation of the questioner 30) from the additional information, generates a response to the question based on the situation, and outputs the generated response to the output unit 15.
• the automatic response system 10 may also grasp attributes (the attributes of the questioner 30) from the additional information, and generate a response to the question based on the situation and attributes.
• the automatic response system 10 outputs the response (step S6) and ends the automatic response process. More specifically, in step S6, the output unit 15 outputs the response.
• the output unit 15 outputs the response by performing at least one of audio output and display, depending on the format of the response generated by the response generation unit 14 (whether the response is audio data, text data, or image data).
• FIG 6 is a flowchart showing an example of the answer generation processing procedure in the answer generation unit 14 shown in step S5 of Figure 5.
• the answer generation unit 14 recognizes the situation and the question (step S11).
• the situation recognition unit 141 recognizes the situation based on the accompanying information and outputs it to the generation unit 142.
• the generation unit 142 recognizes the situation acquired from the situation recognition unit 141 and the question received from the question acquisition unit 12 by converting them into a format required for answer generation processing.
• the generation unit 142 understands the meaning of the situation and the question.
• the answer generation unit 14 will understand that "you are being asked about the route to the platform,” and if the recognized situation is “you are carrying large luggage,” it will understand that “it would be better to take a route that avoids stairs as much as possible.”
• the answer generation unit 14 references the facility data (step S12).
• the generation unit 142 reads out the facility data required to answer the question from the data storage unit 113. For example, in the example above where the question is "Where is the platform?" and the situation is "I have large luggage,” map information within the station is read out as facility data.
• the answer generation unit 14 generates an answer (step S13) and ends the answer generation process. More specifically, in step S13, the generation unit 142 uses the referenced facility data to generate an answer to the question that takes the situation into consideration, and outputs the generated answer to the output unit 15. For example, in the example where the question is "Where is the platform?" and the situation is "I have large luggage," the unit uses map information within the station to find a route to the platform that uses an elevator and does not require stairs, and generates a diagram showing the route obtained through the search, along with an explanation of the diagram.
• the automatic response system 10 of this embodiment can generate an answer that takes into account the situation of the questioner 30. This allows the automatic response system 10 to provide a more appropriate answer to the facility user than when simply answering a specific question from the facility user.
• the automatic response system 10 may also generate an answer based on attributes in addition to the situation, and in this case, it is possible to provide the facility user with a more appropriate answer that suits the attributes.
• dynamic data dynamic facility data
• FIG. 7 is a diagram showing an example of dynamic facility data in this embodiment.
• the facility is a railway facility (including the inside of a railway vehicle) including a station premises or multiple stations.
• a questioner 30 asks, "Which car is empty?" In order to generate an answer to this question, it is necessary to know the degree of congestion in each car.
• the data acquisition unit 111 acquires, as dynamic facility data, video data acquired by cameras 50 installed in each railway vehicle.
• a timetable indicating when each railway vehicle will arrive at the station is also stored as facility data in the data storage unit 113.
• the dynamic facility data is updated to the latest data when new data is acquired, for example, but is not limited to this; past dynamic facility data may also be stored in the data storage unit 113.
• the preprocessing unit 112 may store video data of the interior of the vehicles in the data storage unit 113, or may determine the degree of congestion in each vehicle based on the video data and store the determined result in the data storage unit 113 as facility data.
• the degree of congestion may be indicated, for example, in two stages: whether or not there is a mixture, or in three stages: crowded, normal, and empty, or by a congestion rate, or in some other way.
• the answer generation unit 14 recognizes the degree of congestion based on the video data and uses it to generate an answer.
• the answer generation unit 14 identifies the next arriving train car based on the timetable, and can identify which cars are empty by using the degree of congestion calculated from video data captured inside each car of that train car. This makes it possible to generate an appropriate answer that takes into account the actual situation, such as "Car 2 is empty,” in response to the question, "Which car is empty?"
• FIG. 7 The example shown in FIG.
• the automatic response system 10 is installed on a station platform, and the answer generation unit 14 generates an image showing the position of car 2 and the current location as an answer, and the output unit 15 displays the generated image along with the text answer, "Car 2 is empty.”
• the situation (or the situation and attributes) may also be taken into consideration when generating this answer. For example, in the example shown in Figure 7, if the asker 30 does not have large luggage, a response may be generated to guide them to the emptiest available vehicle, and if the asker 30 has large luggage, a response may be generated to guide them to the nearest available vehicle (even if it is not the emptiest vehicle).
• the dynamic facility data is not limited to the example shown in Figure 7, but may be information indicating the degree of congestion in restrooms within the facility, information indicating the degree of congestion in a store, or, as described above, facility-originated information, store-originated information, etc.
• the situation of the questioner 30 may include, for example, the number of questioners 30.
• the automatic response system 10 will select a quiet restaurant based on the facility data, generate a response indicating that the selected restaurant is recommended, and output the response.
• the automatic response system 10 will select a restaurant that is suitable for families, such as a family restaurant, generate a response indicating that the selected restaurant is recommended, and output the response.
• Figures 8 to 14 are diagrams showing specific examples of responses that take the situation into consideration in the automatic response system 10 of this embodiment.
• a middle-aged or elderly woman wearing elegant clothing and accessories is asking the question, "Do you know of any clothing stores?"
• the automatic response system 10 recognizes that the situation of the questioner 30 is that she is wearing accessories and elegant clothing, and based on this recognition result, the question, and information in the facility data indicating the products sold by each store and the target audience of each store, it determines that Shop A and Shop B are recommended clothing stores for middle-aged or elderly people.
• the automatic response system 10 then generates information directing the user to Shop A and Shop B as a response to the question and displays it on the output unit 15. In this way, a response that takes the situation of the questioner 30 into consideration is generated.
• the automatic response system 10 generates an answer including basis information indicating the basis for the answer based on the situation used to generate the answer.
• the automatic response system 10 generates basis information of "mother generation” as the basis for deciding to guide the customer to Shop A and Shop B, based on the situation that the customer is a middle-aged or elderly woman wearing elegant clothing and accessories.
• the automatic response system 10 not only the suggested store names "Shop A” and “Shop B” but also the text "How about a fashionable clothing store popular with the mother generation?" are displayed.
• touching the "Shop A” and "Shop B" parts on the display screen shown in FIG. 8 may display detailed information about each store. Note that the specific content of the sentence (text) in the answer is not limited to the example shown in FIG. 8.
• a young woman in a uniform asks, "Do you know of any clothing stores?"
• the automatic response system 10 recognizes that the questioner 30 is in a situation where she is a young woman in a uniform, and based on this recognition result, the question, and information in the facility data indicating the products sold by each store and the target audience of each store, it determines that Shop C and Shop D are recommended clothing stores for female high school students (high school girls).
• the automatic response system 10 then generates information directing users to Shop C and Shop D as an answer to the question, and displays this on the output unit 15.
• the basis information is also displayed. Specifically, the text "high school girl" is generated as the basis information, and an answer including the basis information is displayed.
• a middle-aged man wearing a classic hat and classic clothing asks, "Do you know of any clothing stores?"
• the automatic response system 10 recognizes that the situation of the questioner 30 is that of a middle-aged man wearing a classic hat and classic clothing. Based on this recognition result, the question, and the information in the facility data indicating the products sold by each store and the target audience of each store, the automatic response system 10 determines that an e-shop is a recommended clothing store for middle-aged men who like classic fashion. The automatic response system 10 then generates information providing directions to the e-shop as a response to the question and displays it on the output unit 15. In the example shown in FIG. 10, similar to the examples shown in FIGS. 8 and 9, the basis information is also displayed.
• the text "dandy" is generated as the basis information, and an answer including the basis information is displayed.
• the gender of the questioner 30 is also taken into consideration, but the gender of the questioner 30 is also an attribute, and these examples can be said to generate an answer that takes into consideration both the situation, such as clothing, and the attribute.
• FIGS. 11 and 12 show examples in which the facial expression or emotion of the questioner 30 is taken into consideration as the situation in which the questioner 30 finds himself. In both of the examples shown in FIG. 11 and FIG. 12, the questioner 30 is asking where his/her home is.
• the questioner 30 asks "Where is the platform?" in a calm and normal manner.
• the automatic answering system 10 recognizes that the questioner 30 is normal, i.e., calm, from the questioner's facial expression, movements, tone of voice, etc., and generates a response while engaging in a dialogue with the questioner 30.
• the automatic answering system 10 asks the questioner 30, "Where do you want to take the train?”, and the questioner 30 replies, "I want to go to XX," and based on the response from the questioner 30, the automatic answering system 10 generates and outputs the response, "If you are going to XX, please proceed to platform 3."
• the questioner 30 appears flustered and asks, "Where is the platform?"
• the automated response system 10 recognizes that the questioner 30 is in a panic and is in a state of panic based on the questioner's facial expression, movements, tone of voice, etc., and outputs a response including multiple types of information at once without engaging in a dialogue with the questioner 30.
• the automated response system 10 simultaneously displays the information "Platform 3 for direction XX,” “Platform 4 for direction YY,” and “Platform 6 for direction ZZ.”
• the content of the response and the method of interaction with the questioner 30 may be determined depending on the facial expression or emotions of the questioner 30.
• FIG. 13 and 14 show examples in which the facial expression or emotion of the questioner 30 and the number of people are taken into consideration as the situation of the questioner 30.
• the automatic answering system 10 recognizes that the questioner 30 is in a high-energy state (in a good mood and excited state) based on the questioner's facial expression, movements, tone of voice, etc., and also recognizes that the questioner 30 is part of a group, and generates a response that provides directions to lively establishments "Bar A" and "Bar B.”
• the response also includes evidence information
• the output unit 15 also displays information such as, "For all of you who are excited, how about a lively bar that offers all-you-can-drink?"
• the questioner 30 is a man and woman holding hands and asks, "Can you recommend a bar?"
• the automatic response system 10 recognizes that the questioner 30 is a male-female couple based on the questioner's facial expression, movements, tone of voice, etc., and generates a response recommending the quiet bars "Bar C" and "Bar D.”
• the response also includes evidence information, just like the examples shown in Figures 8, 9, and 10, and the information "How about a quiet bar for a couple?" is also presented to the output unit 15.
• the automatic response system 10 may generateSEA-friendly answers that do not distinguish between the two genders, or answers that take both genders into consideration. For example, when providing restroom guidance, an answer that includes both genders may be generated by answering the locations of men's and women's restrooms regardless of the user's gender.
• rules for generating strig-friendly answers may be stored in the data storage unit 113 as facility data or separately from the facility data, and the answer generation unit 14 may refer to these rules when creating answers.
• ethical standards to be reflected in answers not limited to strig considerations, may be defined as rules, and these rules may be stored in the data storage unit 113 as facility data or separately from the facility data, and the answer generation unit 14 may refer to these rules when creating answers.
• a generation AI is used in the answer generation unit 14, these rules may be specified to the generation AI in advance.
• FIG. 15 is a diagram showing an example configuration of a computer system that realizes each of the information processing units 20 of this embodiment. As shown in Figure 15, this computer system includes a control unit 101, an input unit 102, a memory unit 103, a display unit 104, a communication unit 105, and an output unit 106, which are connected via a system bus 107.
• the control unit 101 is, for example, a processor such as a CPU (Central Processing Unit), and executes a program describing the processing in the information processing unit 20 of this embodiment.
• the input unit 102 is composed of, for example, a keyboard, buttons, a mouse, etc., and is used by the user of the computer system to input various information.
• the memory unit 103 includes various types of memory such as RAM (Random Access Memory) and ROM (Read Only Memory) and storage devices such as a hard disk, and stores programs to be executed by the control unit 101, necessary data obtained during processing, etc.
• the memory unit 103 is also used as a temporary storage area for programs.
• the control unit 101 and the memory unit 103 for example, constitute a processing circuit.
• the processing circuit may be a single circuit or multiple circuits.
• the display unit 104 is composed of a display, LCD (Liquid Crystal Display), etc., and displays various screens to the user of the computer system. It should be noted that a touch panel in which the input unit 102 and display unit 104 are integrated may also be used.
• the communication unit 105 is a receiver and transmitter that perform communication processing.
• the output unit 106 is a speaker or the like. Note that FIG. 15 is just an example, and the configuration of the computer system that realizes each of the information processing units 20 is not limited to the example shown in FIG. 15. For example, the output unit 106 may not be provided.
• the program is installed into the storage unit 103 from, for example, a CD-ROM or DVD-ROM inserted in a CD (Compact Disc)-ROM drive or DVD (Digital Versatile Disc)-ROM drive (not shown). Then, when the program is executed, the program read from the storage unit 103 is stored in the main memory area of the storage unit 103. In this state, the control unit 101 performs the processing of each of the information processing units 20 of this embodiment in accordance with the program stored in the storage unit 103.
• the programs describing the processing in each information processing unit 20 are provided on a CD-ROM or DVD-ROM as recording media.
• this is not limiting.
• programs provided via a transmission medium such as the Internet via the communications unit 105 it is also possible to use programs provided via a transmission medium such as the Internet via the communications unit 105.
• the program of this embodiment causes a computer system to execute, for example, the steps of acquiring a question about facility usage from a facility user who is a user of the facility, acquiring additional information indicating the status of the facility user who asked the question, generating an answer to the question using the additional information, the question, and facility data related to the facility, and outputting the answer.
• the preprocessing unit 112 and answer generation unit 14 shown in FIG. 1 are realized by the control unit 101 shown in FIG. 15 executing a program stored in the storage unit 103.
• the storage unit 103 is also used to realize the preprocessing unit 112 and answer generation unit 14.
• the data acquisition unit 111 shown in FIG. 1 is realized by at least one of the communication unit 105 and the input unit 102 shown in FIG. 15. Some of the functions of the data acquisition unit 111 may be realized by the control unit 101 and the storage unit 103.
• the data storage unit 113 shown in FIG. 1 is part of the storage unit 103 shown in FIG. 15.
• the information processing unit 20 may be realized by multiple computer systems. For example, the information processing unit 20 may be realized by a cloud computer system.
• the question acquisition unit 12 of this embodiment is realized by at least one of an input means such as a touch panel or keyboard, and a microphone.
• the situation acquisition unit 13 of this embodiment is realized by at least one of a sensor such as a camera, an input means such as a touch panel or keyboard, and a microphone.
• the output unit 15 is realized by at least one of a display such as a touch panel, and a speaker. As described above, if the display that realizes the output unit 15 includes a display and is a touch panel, the touch panel may also function as the question acquisition unit 12. Furthermore, this touch panel may also function as the situation acquisition unit 13.
• the entire automatic response system 10 may be considered to be the computer system illustrated in FIG. 15.
• the computer system may include a microphone as the input unit 102, and may further include a sensor as the situation acquisition unit 13.
• the question acquisition unit 12 is realized by the input unit 102
• the situation acquisition unit 13 is realized by at least one of the sensor and the input unit 102
• the output unit 15 is realized by at least one of the display unit 104 and the output unit 106.
• the automated response system 10 of this embodiment uses the situation of the questioner 30 to generate an answer to the question from the questioner 30. This allows for a more appropriate answer to be provided to the facility user compared to simply answering a specific question from the facility user.
• the automated response system 10 may also generate an answer based on attributes in addition to the situation, and in this case, it is possible to provide the facility user with a more appropriate answer based on the attributes.
• by including dynamic facility data as facility data it is possible to provide the facility user with appropriate answers and information based on the current state of the facility.
• FIG. 16 is a diagram showing an example of the configuration of an automatic response system according to the second embodiment.
• the automatic response system 10a of this embodiment includes an automatic response device 20a and a terminal device 60.
• the automatic response device 20a is similar to the information processing unit 20 of the first embodiment, except that a transmitting/receiving unit 16 is added.
• the terminal device 60 includes a question acquisition unit 12, a situation acquisition unit 13, an output unit 15, and a transmitting/receiving unit 17.
• Components having the same functions as those of the first embodiment are assigned the same reference numerals as those of the first embodiment, and redundant explanations will be omitted. Below, differences from the first embodiment will be mainly explained.
• the terminal device 60 is a device that can be operated by the facility user, such as a smartphone, tablet, or personal computer.
• the terminal device 60 may also be a mobile terminal that can be carried by the facility user.
• the facility user uses the automatic response system 10a by using the terminal device 60, for example, at home or on the way to the facility. Note that the location where the facility user uses the automatic response system 10a is not limited to this and may be any location, even within the facility.
• the question acquisition unit 12 of the terminal device 60 acquires questions about the facility from facility users by at least one of voice and text.
• the questions may include images.
• the question acquisition unit 12 outputs the acquired questions to the transmission/reception unit 17.
• the question acquisition unit 12 is, for example, at least one of an input means such as a touch panel or keyboard of a smartphone, tablet, personal computer, etc., and a microphone.
• an image is included in the question, for example, the facility user specifies image data stored in the terminal device 60, and the input means accepts the specification.
• the question acquisition unit 12 may convert the voice data into text data and output it to the transmission/reception unit 17, or may output the voice data directly to the transmission/reception unit 17.
• the situation acquisition unit 13 acquires additional information indicating the situation of the questioner 30, who is a facility user asking a question (the situation in which the questioner finds himself/herself). Similar to embodiment 1, the additional information may also be information indicating the attributes of the facility user.
• the situation acquisition unit 13 may acquire the additional information in the form of at least one of audio and text, or may acquire image data of the questioner 30 as the additional information.
• the output unit 15 may output a screen to the questioner 30 asking questions such as the number of people using the facility and whether or not assistive devices will be used, and the answers entered on that screen may be acquired as the additional information.
• the situation acquisition unit 13 may be realized, for example, by a camera built into the terminal device 60, such as a smartphone, tablet, or personal computer, and may output video data acquired by the camera to the transmission/reception unit 17 as the additional information.
• This camera may, for example, be an internal camera that captures the questioner from inside the terminal device 60.
• the output unit 15 outputs the answer by at least one of audio output and display.
• the output unit 15 is, for example, at least one of the display of the terminal device 60 and the microphone of the terminal device 60, but is not limited to these.
• the display of the terminal device 60 may be a touch panel. In this case, the touch panel may also function as at least one of the question acquisition unit 12 and the situation acquisition unit 13.
• the transmission/reception unit 17 communicates with the automatic response device 20a, thereby exchanging information with the automatic response device 20a. For example, the transmission/reception unit 17 transmits the question received from the question acquisition unit 12 and the additional information received from the situation acquisition unit 13 to the automatic response device 20a. The transmission/reception unit 17 also outputs the response received from the automatic response device 20a to the output unit 15.
• the operations of the question acquisition unit 12, situation acquisition unit 13, output unit 15, and transceiver unit 17 described above may be performed by installing application software that provides facility guidance services on the terminal device 60, or by the terminal device 60 accessing the automatic response device 20a.
• the automatic response device 20a may function as a web server, and the terminal device 60 may access the web server to realize the above operations.
• the transmitter/receiver unit 16 of the automatic response device 20a communicates with the terminal device 60 to exchange information with the terminal device 60.
• the transmitter/receiver unit 16 outputs the question and additional information received from the terminal device 60 to the answer generation unit 14.
• the transmitter/receiver unit 16 also outputs the answer received from the answer generation unit 14 to the terminal device 60.
• the answer generation unit 14, like the answer generation unit 14 in embodiment 1, generates an answer using the question, additional information, and facility data, and outputs the generated answer to the transmitter/receiver unit 16.
• the automatic response device 20a of this embodiment is realized by a computer system, like the information processing unit 20 in embodiment 1. Note that the automatic response device 20a may be installed inside or outside the facility.
• facility users can ask questions using the terminal device 60, allowing them to use the automatic response system 10a regardless of their location. Therefore, for example, before using a facility, they can obtain information about the facility in advance, either at home or on the way to the facility. This allows facility users to use the facility efficiently based on the information they obtain in advance. Furthermore, by including map information for generating a route to the facility in the facility data, when a facility user asks how to get to the facility, it is possible to present the route from the facility user's current location to the facility as an answer. Furthermore, for example, facility data may also include data on reported lost items.
• the automatic response system 10a can respond with whether or not a corresponding lost item has been reported.
• FIGS. 17 and 18 are diagrams showing specific examples of responses that take the situation into consideration in the automatic response system 10a of this embodiment. In both of the examples shown in FIG. 17 and FIG. 18, it is assumed that it is raining around the facility, and that the facility data includes data indicating the weather.
• the facility is a department store
• the questioner 30, who is visiting the department store asks a question using the terminal device 60 before arriving at the department store.
• the questioner 30 is raining, and the questioner 30 is holding an umbrella.
• the terminal device 60 accepts the question, which is voice data, "Please tell me the way to the department store," and transmits the question and accompanying information, which is video data of the questioner 30, to the automatic response device 20a.
• the automatic response device 20a determines that even though it is raining, the questioner 30 is holding an umbrella and should be guided along the normal route, and generates the normal route to the department store (for example, the shortest route) as an answer and transmits the answer to the terminal device 60.
• the terminal device 60 displays the route to the department store as an answer.
• the questioner 30 is holding an umbrella, but this is not limiting.
• the automatic response device 20a may generate a response that guides the questioner 30 along a normal route even if it is raining. In other words, the automatic response device 20a may generate a response that guides the questioner 30 along a normal route as long as the questioner 30 is holding an umbrella, regardless of the state of the umbrella.
• the facility is a conference hall where a briefing session is scheduled.
• the questioner 30 is dressed formally and does not have an umbrella.
• the terminal device 60 receives a question in the form of audio data, "Please tell me the way to the briefing session venue," and transmits the question and accompanying information, which is video data of the questioner 30, to the automatic answering device 20a.
• the automatic answering device 20a determines that it is raining, the questioner 30 is not using an umbrella, and is dressed formally, and therefore should guide the questioner 30 along a covered route that will protect them from the rain.
• the automatic answering device 20a generates an answer that includes a route that will protect them from the rain and transmits the answer to the terminal device 60.
• the terminal device 60 displays the route to the conference hall as the answer.
• the automatic answering device 20a can provide an answer that is appropriate for the questioner 30.
• facility users ask questions using terminal device 60, and terminal device 60 outputs answers. This allows facility users to use automatic response system 10a even when they are outside the facility.
• transceiver unit 16 By adding a transceiver unit 16 to the automatic response system 10 shown in embodiment 1 and having the transceiver unit 16 communicate with the terminal device 60, it may be possible to respond to both questions from facility users within the facility as described in embodiment 1 and questions using the terminal device 60 as described in this embodiment.
• Embodiment 3. 19 is a diagram showing an example of the configuration of an automatic response system according to the third embodiment.
• the automatic response system 10b of this embodiment is similar to the automatic response system 10 of the first embodiment, except that a rating acquisition unit 18 is added and the data storage unit 113 further stores rating information.
• Components having the same functions as those of the first embodiment are assigned the same reference numerals as those of the first embodiment, and redundant explanations will be omitted. Below, differences from the first embodiment will be mainly explained.
• the evaluation acquisition unit 18 acquires an evaluation result from the questioner 30 indicating an evaluation of the answer, and stores the acquired evaluation result in the data storage unit 113 along with the corresponding question and answer.
• evaluation information is stored in the data storage unit 113
• this is not limiting, and an evaluation information storage unit that stores evaluation information separate from the data storage unit 113 may be provided.
• the output unit 15 outputs a question inquiring about the evaluation of the answer from the automatic response system 10b, such as "Did you get the information you wanted?" or "Was the answer appropriate?" Furthermore, if the evaluation from the questioner 30 includes negative words such as "unsatisfactory" or "not good," the evaluation acquisition unit 18 may cause the output unit 15 to output a question further asking the questioner 30 what the problem was, and may accept input from the questioner 30.
• the evaluation acquisition unit 18 may acquire the evaluation result by voice or as text data.
• the output unit 15 may present options indicating the evaluation results, and the evaluation acquisition unit 18 may acquire the selection made by the questioner 30 as the evaluation result.
• the evaluation acquisition unit 18 may evaluate the content of the answer based on the behavior of the questioner 30. For example, after the automatic response system 10b outputs a route to a destination to a questioner 30 who has asked for directions, it may analyze the behavior of the questioner 30 using video data captured of the questioner 30, and if the questioner 30 is still lost, it may evaluate that the answer was difficult to understand.
• the evaluation information may be used, for example, together with the corresponding questions and answers, for the answer generation unit 14 to re-learn or additionally learn, or may be referenced by the answer generation unit 14 when generating answers.
• the operator or administrator of the automatic response system 10b may check the evaluation information, use the results to identify areas for improvement, and instruct the answer generation unit 14 on the identified areas for improvement.
• the operator or administrator of the automatic response system 10b may check the evaluation information, use the results to determine what type of re-learning or additional learning the answer generation unit 14 should perform, and the determined results may be used to perform the re-learning or additional learning.
• the evaluation acquisition unit 18 is, for example, at least one of an input means via operation and a microphone.
• the input means, microphone, and other hardware may be shared with the question acquisition unit 12.
• an evaluation acquisition unit 18 may be added to the automatic answering system 10a of embodiment 2 to reflect the evaluation results, or an evaluation acquisition unit 18 may be added to an automatic answering system that combines embodiments 1 and 2 to reflect the evaluation results.
• 10, 10a, 10b Automatic response system
• 11 Data management unit
• 12 Question acquisition unit
• 13 Situation acquisition unit
• 14 Answer generation unit
• 15 Output unit
• 16, 17 Transmitting/receiving unit
• 18 Evaluation acquisition unit
• 20 Information processing unit
• 20a Automatic response device
• 30 Questioner
• 40 Main unit
• 50 Camera
• 60 Terminal device
• 111 Data acquisition unit
• 112 Preprocessing unit
• 113 Data storage unit
• 142 Generation unit.

Landscapes

• Engineering & Computer Science (AREA)
• Databases & Information Systems (AREA)
• Theoretical Computer Science (AREA)
• Data Mining & Analysis (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
• Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Priority Applications (2)

Applications Claiming Priority (1)

Publications (1)

Family

ID=96656972

Family Applications (1)

Country Status (2)

Citations (2)

Family Cites Families (1)

• 2024
  • 2024-04-17 WO PCT/JP2024/015272 patent/WO2025220150A1/ja active Pending
  • 2024-04-17 JP JP2024559956A patent/JP7721018B1/ja active Active

Patent Citations (2)

Also Published As

Similar Documents

Legal Events