WO2019208327A1 - Système d'aide au service clientèle - Google Patents

Système d'aide au service clientèle Download PDF

Info

Publication number
WO2019208327A1
WO2019208327A1 PCT/JP2019/016317 JP2019016317W WO2019208327A1 WO 2019208327 A1 WO2019208327 A1 WO 2019208327A1 JP 2019016317 W JP2019016317 W JP 2019016317W WO 2019208327 A1 WO2019208327 A1 WO 2019208327A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
information
unit
analysis
order
Prior art date
Application number
PCT/JP2019/016317
Other languages
English (en)
Japanese (ja)
Inventor
和夫 金子
Original Assignee
Kaneko Kazuo
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kaneko Kazuo filed Critical Kaneko Kazuo
Publication of WO2019208327A1 publication Critical patent/WO2019208327A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/12—Hotels or restaurants
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/10—Speech classification or search using distance or distortion measures between unknown speech and reference templates

Definitions

  • the present invention relates to a technology for supporting customer service.
  • the customer calls the customer service.
  • the fact is that it is very difficult to arrange a customer service due to factors such as budget and lack of human resources. As a result, there is a concern that customer satisfaction is lowered. Or, in the first place, even if you are dissatisfied with customer service, there are many customers who hesitate to call the customer service. In this case, the customer's dissatisfaction with the customer service cannot be resolved.
  • the customer's request is satisfied as a result of the customer service being taken appropriately, the customer may already have a certain amount of dissatisfaction or stress at the time of calling the customer service. The bad impression is not completely wiped out.
  • An object of this invention is to grasp
  • the present invention is installed in a place where a product is placed or a place where a service is provided to a user, and obtains an image acquisition means for acquiring an image showing a scene around the user, and a voice related to the user's utterance
  • An information processing apparatus includes a sound acquisition unit that performs analysis, an analysis unit that analyzes the image and the sound, and an output unit that outputs a result of the analysis.
  • FIG. 1 is a schematic diagram of a customer service support system 10. The figure which shows the installation state of the terminal 100.
  • FIG. The example of the content displayed on the screen of the terminal 100 (the 1).
  • the example of the content displayed on the screen of the terminal 100 (the 2).
  • the example of the content displayed on the screen of the terminal 100 (the 3).
  • the example of the content displayed on the screen of the terminal 100 (the 4).
  • the example of the content displayed on the screen of the terminal 100 (the 5).
  • FIG. An example of the content displayed on the screen of the terminal 100 (part 7).
  • DESCRIPTION OF SYMBOLS 100 ... Terminal, 101 ... Microphone camera, 102 ... Speaker, 103 ... Touch panel, 180 ... Shooting unit, 181 ... Image processing unit, 190 ... Audio input unit, 191, ..Audio signal processing unit, 130 ... analysis unit, 140 ... operation unit, 150 ... timer unit, 110 ... information generation unit, 160 ... storage unit, 120 ... output unit, 121: Display unit, 122: Audio output unit, 123 ... Communication unit, 200 ... Server, 210 ... Communication unit, 240 ... Storage unit, 220 ... Control unit, 221 ... Staff management unit, 223 ... Feedback management unit, 222 ... Order management unit
  • FIG. 1 shows an outline of the customer service support system 10.
  • the customer service support system 10 includes a terminal 100, a server 200, and an employee terminal 300.
  • Staff STF1 and STF2 are employees who are on the floor and serve customers. Employees include waitresses who carry food to the table, boys, or floor managers who manage and supervise employees.
  • Staff STF3 is a cook who is in the kitchen and makes food. Note that the role of the employee and the terminal to be used need not be clear. Hereinafter, it is simply referred to as “employee”.
  • Terminals 100 are provided corresponding to one or more customers who receive the service. Further, the number of terminals 100 is an example.
  • Server 200 communicates with terminal 100 and employee terminal 300 (300-1, 300-2, 300-3).
  • Each employee terminal 300 (300-1, 300-2, 300-3) is carried by an employee or installed in a store, and has a function of performing wireless communication with the server 200 and a function of notifying by information image or voice.
  • the information processing terminal notifies information to employees such as customer service and cooks.
  • Each employee terminal 300 performs wireless communication with the server 200 and, when receiving a customer service or other instruction from the server 200, notifies the employee of information.
  • the number of employee terminals 300 shown in the figure is an example.
  • FIG. 2 shows the installation state of the terminal 100.
  • At least one terminal 100 is provided for a table, a counter, one user who receives service provision, or one user group.
  • it is a tablet information terminal placed on one table TBL1 on which four customers CST1 to CST4 (hereinafter referred to as users) receiving service provision are seated.
  • the terminal 100 includes a microphone camera 101, a speaker 102, and a touch panel 103.
  • the form and installation position of the terminal 100 are examples, and may be a wall hanging, for example.
  • the microphone camera 101 captures a scene including the table TBL1 and the customer CST and collects environmental sounds including the utterance of the customer CST.
  • the speaker 102 is a speaker for notifying information to the user by voice.
  • the touch panel 103 is a display device with a touch panel, which is operated by the customer CST to receive services such as ordering and calling, and information and instructions are input to the terminal 100, and the customer CST is serviced by characters and images. A device that displays content and other information.
  • Microphone camera 101 and / or speaker 102 may be provided separately from touch panel 103.
  • the speaker 102 may be omitted.
  • notification of information to the user is performed only with characters and images.
  • the terminal 100 the user is allowed to perform an input operation related to an order acceptance or other service request, or browse information related to service contents such as a menu, while at least one user's voice and the surroundings of the user It is only necessary that the terminal 100 can acquire an image showing a scene.
  • the surrounding scene of the user is, for example, the contents of the provided dishes, the number of tables, the table image reflecting the situation where the dishes are consumed, etc. and the user situation (contents of actions and actions such as eating and drinking, It includes at least one of images indicating a situation in which a service is provided (eating and drinking) such as an aspect of an action such as speed and a facial expression.
  • the acquired image may be a still image or a moving image.
  • FIG. 3 is a functional block diagram of the terminal 100.
  • the terminal 100 includes an information generation unit 110, an output unit 120, an analysis unit 130, an operation unit 140, a timing unit 150, a storage unit 160, a photographing unit 180, an image processing unit 181, an audio input unit 190, and an audio signal processing unit 191. Including.
  • the operation unit 140 is realized as the touch panel 103, for example, and is operated by the user.
  • the timer unit 150 is realized by a clock circuit or the like, and is appropriately referred to by the information generation unit 110 to provide information on the current time.
  • the photographing unit 180 and the image processing unit 181 are realized as the microphone camera 101.
  • the photographing unit 180 includes an optical system, a lens, a photographing mechanism, a light receiving element, and a control mechanism, and photographs a scene around the terminal 100 (in other words, around the user).
  • the image processing unit 181 is realized by an image processing processor, and extracts the feature amount of the shot scene from the image data shot by the shooting unit 180 using a known algorithm.
  • the feature amount is, for example, the content and timing of various operations including the presence / absence of a plate on the table, the degree to which the dish is consumed, the facial expression of the user, the movement of the user's mouth and the movement of chopsticks.
  • the feature amount extraction timing is executed, for example, every predetermined number of frames.
  • the imaging unit 180 and the image processing unit 181 function as an image acquisition unit that acquires an image showing a scene around the user.
  • the generated feature amount is supplied to the analysis unit 130 in real time or at predetermined time intervals.
  • the audio input unit 190 and the audio signal processing unit 191 are realized as the microphone camera 101.
  • the voice input unit 190 is a condenser microphone.
  • the voice signal processing unit 191 includes a voice signal processor that performs text removal processing and voice recognition processing from waveform data acquired by the voice input unit 190 to generate text data.
  • the voice input unit 190 and the voice signal processing unit 191 function as a voice acquisition unit that acquires voice related to the user's speech. That is, the user's utterances are collected and information (words, phrases, etc.) for adding the contents of the utterances is generated. Information characterizing the content of the generated utterance is supplied to the analysis unit 130.
  • the analysis unit 130 is realized by one or more general-purpose or dedicated processors, and performs image analysis and sound analysis using the image acquired by the image acquisition unit and the sound acquired by the sound acquisition unit. Therefore, it functions as an analysis means for analyzing the user's situation.
  • image analysis is not limited to identifying external features such as user actions (moving faces, limbs, etc.) and facial expressions, but also the background, causes, intentions, etc. Including deciding.
  • the speech analysis includes not only specifying the linguistic meaning of a spoken word or sentence but also determining the background and situation of the utterance, the intention of the utterance, and the like.
  • the analysis unit 130 uses the image showing the scene to specify the content of the utterance and generates deeper level information such as intention and background.
  • the analysis unit 130 first performs speech analysis on the acquired speech data to identify the linguistic meaning (superficial meaning) of the spoken word or sentence, and subsequently captures the captured image. Based on this, a deeper meaning such as the intention and the intention of the specified utterance content is estimated. For example, based on the user's actions (turn around, crawl, stand up, etc.) and facial expressions (changes) indicated by the captured image, the acquired utterance means order contents or an impression of cooking Whether it is just a tweet (exposure of emotions) or some sort of request or request for the store clerk.
  • At least one of the relationship (combination) between the operation content and the utterance content and the relationship between the timing when the operation is performed and the timing when the utterance is performed can be used.
  • Information regarding these relevances and determination algorithms are stored in the storage unit 160.
  • the analysis unit 130 determines whether the utterance means the content of the order according to the degree of progress of eating and drinking determined based on the captured image. Specifically, the degree of progress of eating and drinking is determined based on the movement of the mouth, bridge, and fork, the level of consumption of dishes on the table, the level of change of dishes, and the like indicated by the image.
  • the utterance may be analyzed in consideration of the meaning of the word in addition to the image.
  • the analysis unit 130 may reflect the result of the voice analysis in the image analysis process. For example, in the case where a user's action is confirmed from the image, when the utterance of a word that is substantially the same timing as the action is detected, the action is determined as “the user turned around”, while When a specific word is detected, it is determined that the action is “an action indicating that he / she is lost in order” or “an action looking for a store clerk”.
  • the analysis unit 130 may reflect the result of the image analysis in the sound analysis process, may reflect the result of the sound analysis in the image analysis process, or mix image analysis and sound analysis. You may perform integrally (for example, simultaneously and parallelly).
  • the order and contents of the image processing and the sound analysis processing are not particularly limited, and the user's behavior and / or the intention of the speech is estimated by correlating the image and the sound.
  • the analysis result is supplied to the information generation unit 110.
  • the analysis unit 130 when it is determined that the detected utterance means the content of the order, the analysis unit 130 generates information in which the flag information indicating the order item is added to the text information indicating the order item, and the information generation unit 110. To supply.
  • order information is generated based on the supplied information.
  • impression the user's impression, evaluation, dissatisfaction, demand, etc.
  • the analysis unit 130 cuts out the voice data of the utterance part or converts it into text information indicating the utterance content, adds flag information indicating information such as an impression, and the information generation unit 110. To supply. If it is determined that the content is neither an order content nor an impression, no information may be supplied to the information generation unit 110.
  • the information generation unit 110 is realized by one or more general-purpose or dedicated processors, and controls each unit of the terminal 100 by reading a control program from the storage unit 160 and executing it. Specifically, based on information (analysis result) supplied from the analysis unit 130 and signals supplied from the operation unit 140 and the time measuring unit 150, and using information read from the storage unit 160 as necessary, Information including a notification execution instruction and notification content is generated and output to the output unit 120. Specifically, one or more analysis results generated by the analysis unit 130, operation contents acquired from the operation unit 140, analysis results generated by the analysis unit 130, and operation signals input from the operation unit 140, The information is generated based on at least one of the timing information acquired from the time measuring unit 150 and the user attribute read from the storage unit 160.
  • the information generated by the information generation unit 110 includes the following four types.
  • the first is information related to the order received by the operation unit 140 or the voice input unit 190 (information about the order item and provision timing), which is mainly provided to the employee terminal 300-3 (order information). It is said).
  • the second is information mainly transmitted to the employee terminals 300-1 and / or 300-2, which is generated based on the signal input from the operation unit 140 or the information generation unit 110 based on the information from the analysis unit 130.
  • the information (referred to as “necessity information”) related to the necessity of the employee's reception and the specific contents of customer service (eg, asking the user's request, providing water, lowering the dish, etc.).
  • the third is information (referred to as user support information) that is supplied to the display unit 121 or the audio output unit 122 to make various notifications, suggestions, requests, and the like to the user.
  • the fourth is information (referred to as feedback information) indicating the user's impressions, evaluations, dissatisfactions, requests, etc. (hereinafter referred to as impressions) for the product and / or service.
  • the method for generating the necessity information or the user support information is executed as follows, for example. If it is presumed that the ordering operation is not successful based on the user's action taken, such as when the operation unit 140 detects that the user repeats the same operation, ? "," I can't find tuna ... ", or an empty plate is collected on the table, the user's facial expression is stiff, or the user is angry. If the user is in a state where the face orientation changes drastically, or repeats a predetermined action pattern (input to the touch panel) for a predetermined time and repeats a similar word (such as “That?”) If you are at least based on at least one of image and sound, you can't operate well, want to lower the dish, don't know the accounting method, etc.
  • the information generation unit 110 determines the necessity of generating a screen that supports the operation or the reception by the employee based on the content of the estimation. Then, based on the determination result and the determination result, reception information and user support information corresponding to the estimation result are generated.
  • the user support information includes information for identifying products or services (such as recommended products) corresponding to the photographed scene, information for supporting operations related to orders, and messages for prompting the user to provide feedback and evaluation. May be included. Further, the user support information may include an image corresponding to the information.
  • the image corresponding to the scene may be, for example, information related to a product or service determined in accordance with the content (amount or type) of the user's order, the speed or timing of the order, Information that is not directly related to a product or service such as an avatar image may be used. By displaying such an image, the entertainment property at the time of ordering operation is improved.
  • the information generation unit 110 may determine that it should be recommended to place an order having the same content as the immediately preceding order content.
  • the pace of eating and drinking for example, by storing the standard time required for one item of eating and drinking in the storage unit 160, and comparing this standard time and the time calculated from the analysis result of the analysis unit 130, Determine the pace of eating and drinking.
  • the information generation unit 110 always displays the order contents of this time.
  • the product suggests that the user should order frequently, or the product should be proposed based on the order history of other users with similar user attributes May be.
  • the information generation unit 110 gives impressions and evaluations regarding products and services such as “This is really delicious!”, “Let's go to this store again”, “How about the attitude of the employee just before ...”
  • the analysis unit 130 confirms an utterance that is estimated to mean, it may be determined that an opportunity to request provision of the evaluation has arrived, and it may be determined that the request should be executed to the user. .
  • the information generation unit 110 includes user support information including an instruction to display the ordered product history screen or an instruction to display an operation prompting message for displaying the order history screen on the screen. Generate.
  • the information generation unit 110 may adjust the timing of providing food and drink based on the order timing received from the operation unit 140 and the analysis result of the scene generated by the analysis unit 130. For example, when the analysis unit 130 obtains an analysis result that the user's utterance amount is large, the speed at which the food on the table is consumed is slow, or the time for holding the chopsticks or fork is short, the operation unit 140 In the case of an accepted order, an instruction to intentionally delay the provision timing is generated and included in the order information. On the contrary, when it is determined that the eating and drinking speed is fast, the provision timing of the goods related to this order is rushed (flag information indicating that priority is given to orders of other users is generated). The provision timing may be determined based on the type of food already applied and user attributes.
  • utterance content (voice data) or video (facial expression etc.) itself may be included in the feedback information, or the information generation unit 110 generates information obtained by processing the voice data and image data. May be.
  • the order ID, the order time, and the satisfaction index (for example, information of “2” in five stages) are generated.
  • the storage unit 160 is a storage device such as a hard disk or a semiconductor memory, and displays parameters, tables, analysis algorithms, and menus used by the analysis unit 130 in addition to an OS program for realizing the operation of the terminal 100 described later.
  • a program for realizing generation (user interface) of a screen for receiving an order is stored.
  • user attributes are stored in the storage unit 160.
  • the user attributes may include information such as the user's age and sex, as well as the user's relationship with the product to the service (order content history, store visit history, frequency of times, etc.).
  • the storage unit 160 stores image and sound data that is acquired by the photographing unit 180 and the voice input unit 190 and reflects the scene, or an analysis result generated by the analysis unit 130 (contents of the user's utterances). Is stored). Note that the data generated by the photographing unit 180 and the voice input unit 190 is likely to contain information relating to the user's privacy. It is preferable to store the information after it has been processed into unspecified information.
  • the output unit 120 includes a display unit 121, an audio output unit 122, and a communication unit 123, and outputs information reflecting the analysis result of the analysis unit 130 supplied from the information generation unit 110. Specifically, the display unit 121 and the voice output unit 122 notify the user, and the communication unit 123 provides information to the server 200 or the employee terminal 300.
  • the display unit 121 includes a liquid crystal panel and a drive circuit, and is realized as the touch panel 103.
  • the display unit 121 notifies the user of information using characters or images.
  • the audio output unit 122 includes a microphone and an A / D conversion circuit, and is realized as the speaker 102.
  • the voice output unit 122 notifies the user of information by voice.
  • the display unit 121 and the audio output unit 122 may notify the same information or different information. In the former case, one of the display unit 121 and the audio output unit 122 may be omitted.
  • the communication unit 123 is realized as a communication module for transmitting information to the server 200 in accordance with a predetermined wireless communication standard such as IEEE, and transmits order information, reception information, and feedback information to the server 200.
  • Information transmitted to server 200 is transmitted in association with information for identifying each terminal 100 that is the transmission source of the information, and stored in server 200.
  • FIG. 6 shows a screen SC1 displayed on the touch panel 103 by the output unit 120.
  • the terminal 100 is installed in a conveyor belt sushi restaurant and has a function of receiving an order by a user operation.
  • the screen SC1 includes an object OB10, an object OB11, an object OB12, an object OB13, an object OB30, an object OB31, an object OB40, an object OB41, and an object OB42.
  • a user touches each object to select a desired object.
  • the object OB10, the object OB11, the object OB12, and the object OB13 are used for selecting the type of product.
  • the object OB20 displays a list of products (in this example, sushi) and allows the user to select what he wants to order.
  • the object OB30 displays the product selected by the object OB20. When the object OB31 is selected, the order of the product displayed on the object OB30 is confirmed.
  • Objects OB40 and OB41 are objects (soft buttons) for performing screen transition.
  • Object OB42 is selected when it is desired to call an employee.
  • the user selects an object displayed in the object OB20 and drags it into the object OB30.
  • order information is generated by the information generation unit 110, and is transmitted via the communication unit 123.
  • an employee call instruction is generated by the information generation unit 110 and transmitted to the server 200 via the communication unit 123.
  • FIG. 4 is a functional block diagram of the server 200.
  • the server 200 includes a communication unit 210, a control unit 220, and a storage unit 240.
  • the communication unit 210 is a communication interface for transmitting and receiving information between each terminal 100 and each employee terminal 300.
  • the storage unit 240 is an information storage device such as a hard disk or a semiconductor memory, and stores a program for realizing the operation of the server 200 described later.
  • the control unit 220 includes a staff management unit 221, an order management unit 222, and a feedback management unit 223.
  • the notification information received from the terminal 100 includes information indicating that a call has been received from the user or other information indicating that an employee needs to respond to the staff management unit 221.
  • An instruction is transmitted to at least one or more employee terminals 300 corresponding to the notification information among 300-1, 300-2, and 300-3.
  • the staff management unit 221 Upon receiving the order information from the terminal 100, the staff management unit 221 transmits the order information to the employee terminal 300-3. Note that the staff management unit 221 may rewrite information regarding the order timing included in the order information in consideration of the order status received from another table.
  • the feedback management unit 223 transmits to the employee terminal 300 information about the content of the order received from the terminal 100, information about the timing of provision, and information regarding the employee call. Specifically, the order information is transmitted to the employee terminal 300-3, and the response necessity information is transmitted to at least one of the employee terminal 300-1 and the employee terminal 300-2.
  • FIG. 5 shows an operation example of the terminal 100.
  • the terminal 100 intermittently performs image acquisition (S502) and audio acquisition (S504) at a predetermined timing, and analyzes the acquired image data and audio data (S506). Note that the audio acquisition timing and the video acquisition timing do not need to match. It is only necessary that the acquired image and sound are correlated in time.
  • the information generation unit 110 determines whether or not order support is required, such as whether the user is troublesome in order input operation based on the analysis result in the analysis unit 130 (S508). (S508: YES), a screen for supporting the order operation is generated (S510).
  • FIG. 7 shows a screen SC2 displayed on the touch panel 103 in S510.
  • the objects OB50 to OB53 are displayed so as to overlap the normal screen.
  • the objects OB50 to 53 for example, browse the same menu page and select the same item, but have not been dragged to the object OB30 (that is, there is no predetermined hand movement) or “tuna”
  • the screen is displayed when it is estimated that the user would like to order tuna, for example, when the user looks at the screen for a predetermined period or more (ie, the user does not move for a predetermined period).
  • the tuna order is confirmed.
  • the order of the tuna is confirmed without performing an operation on the object OB30 and the object OB31.
  • the information generation unit 110 generates order information indicating that a tuna order has been ordered.
  • a screen (not shown) for explaining the operation is displayed.
  • the display of the objects OB50 to 53 is terminated, and the display content returns to the screen of FIG.
  • the information generation unit 110 determines whether or not it is appropriate to propose a product (S512). When it is determined to be appropriate (S512: YES), the information generation unit 110 generates the proposal content, and the proposal content is displayed on the touch panel 103 (S514).
  • the information generation unit 110 receives the order even if a predetermined time has elapsed since the previous order. If it is not done, or if it is estimated that the user wants a drink, such as when a thirsty product such as dried fish is ordered, it is determined that a drink should be proposed.
  • FIG. 8 shows a screen SC ⁇ b> 3 displayed on the screen of the touch panel 103.
  • an object OB54, an object OB55, an object OB56, and an object OB57 are displayed in an overlapping manner.
  • This example shows a case where it is determined that a drink should be proposed.
  • a menu (not shown) is displayed.
  • the terminal 100 receives the utterance content, determines that the item included in the utterance content is an order, and confirms the order.
  • the objects OB54 to OB57 are deleted, and the display content returns to the previous screen.
  • the information generation unit 110 may determine a product to be proposed based on the tendency of the ordered product and / or the preference information described in the user attribute.
  • FIG. 9 shows a screen SC4 of contents displayed on the screen of the terminal 100.
  • the objects OB58, OB59, and OB60 are displayed in an overlapping manner.
  • a list of recommended products (not shown) is displayed.
  • the objects OB58 to OB60 are deleted, and the display content returns to the previous screen.
  • the presence of a recommended product is not notified, but recommended content may be presented as shown in FIG. 10 (object OB61).
  • the information generation unit 110 determines that the proposal is not appropriate at present (S512: NO), whether or not the content of the utterance acquired by the voice input unit 190 describes an impression about the provided product or service, etc. Is determined (S516). Specifically, the information generation unit 110 determines whether or not a speech presumed to be a customer service attitude, an impression of an ordered dish, or the like is made from the utterance content. When it is determined that it is an impression, etc., consent to provide the voice or image reflecting the impression etc. acquired by the terminal 100 to the operator of the customer service support system 10 or the store operation side where the terminal 100 is set Is requested from the user (S518).
  • the information generation unit 110 When the user consents (S520: YES), the information generation unit 110 temporarily displays the utterance content stored in the storage unit 160 at the timing of acquiring the utterance and the user who spoke (or the same scene as the user). The information is transmitted to the server 200 in association with the attribute of the shared user. The server 200 that has received the utterance content stores it in the storage unit 240 (S522).
  • FIG. 11 shows a screen SC6 displayed on the terminal 100 in S518.
  • the objects OB62, OB63, and OB64 are displayed in an overlapping manner.
  • the information generation unit 110 displays the sound or video data itself indicating the impression or the sound or video data. Is read out from the storage unit 160 generated from, and feedback information is generated and transmitted to the server 200 via the communication unit 123.
  • the object OB64 S520: NO
  • the objects OB62 to OB64 are deleted and the screen returns to the previous screen.
  • the information generation unit 110 subsequently determines whether or not an employee needs to be handled (S524). For example, the information generation unit 110 may respond to an employee when there is an expression that the employee wants to deal with in the utterance content, or when a voice that is loud between users or a fighting action is detected. Determine that it is necessary. When it is determined that a response is required (S524: YES), the information generation unit 110 generates response necessity information and transmits it to the server 200 via the communication unit 123. If it is determined that no response is required (S524: NO), the process returns to S502.
  • S524 YES
  • FIG. 12 shows a screen SC7 displayed on the terminal 100 when it is determined that a response is necessary.
  • the object OB65, the object OB66, and the object OB67 are displayed in an overlapping manner.
  • generation of response necessity information and provision to the server 200 are executed (S526).
  • a screen SC8 as shown in FIG. 13 may be displayed.
  • the object OB66 is selected on the screen SC8
  • information for instructing to clean up the table is transmitted to the employee terminal 300-1 or the employee terminal 300-2 via the server 200.
  • the object OB68 is selected, the objects OB68 to OB70 are deleted and the screen returns to the previous screen. Thereafter, S502 to S526 are repeated.
  • the scene is analyzed from the viewpoint of audio and video. And based on the result of this analysis, a customer's demand for goods and services can be estimated, and a service in accordance with the estimated demand can be provided. If this request includes information about customer service, the necessity for reception is automatically determined, and an instruction is sent to the employee only when necessary. As a result, it is not necessary to excessively arrange the customer service in order to maintain the service quality.
  • an impression related to the customer's service is acquired. The information about the user's impressions and reactions accumulated in this way can be used to develop new products and improve the service quality of employees.
  • the request for consent is executed at a timing determined to be appropriate based on the scene, the possibility of harming the customer's mood is reduced, and the possibility that the questionnaire can be cooperated is increased.
  • the terminal 100 may determine whether the user is ready to place an order, such as when the user is seated, and may notify the user when it is determined that the user is ready.
  • the analysis unit 130 detects that a user has arrived based on the acquired video data, and in addition, determines attribute information (number of people, sex, age, etc.) of the user.
  • the information generation unit 110 controls the display unit 121 and the audio output unit based on the content of the determination. For example, either one of waitress images and waiter images stored in advance is selected and displayed according to the gender of the user. Then, depending on the user's age, a welcome message such as “Come on. Please tell me when you place an order. The touch panel is fine.” Or, please talk to this terminal when the order is decided.
  • the guidance method can be changed according to the user's attributes.
  • one attribute may be determined based on the commonality or superiority of the attribute values (for example, male 3
  • the user attribute may be male for a group of two people and two women), and statistical processing (such as calculating an average value of age) may be performed to determine the attribute of one user group.
  • the terminal 100 when the terminal 100 does not detect the user, the terminal 100 operates in the power saving mode in which a function necessary for receiving the order of the display unit 121 or the audio output unit 122 is not realized, and triggered by detecting the user, The display unit 121 and the audio output unit 122 may be operated. In this case, it is not necessary for the user to perform an operation for operating the terminal 100.
  • the server 200 accumulates the order information in the storage unit 240, and based on the accumulated information, the control unit 220 generates information on the order tendency for each season or each day of the week. Or a staffing plan may be determined. This is expected to contribute to improving the freshness of the food provided, reducing waste, preventing orders from being sold out, and improving the efficiency of staffing (increase or decrease in part-time jobs).
  • the specification of the order contents is mainly performed by a touch operation, and the order by voice is an auxiliary role.
  • the order instruction by voice is mainly accepted, and input using the touch panel 103 is performed only when necessary. You may be prompted. Alternatively, only one of reception of an order by touch operation and reception of an order by voice input may be received.
  • the user may designate either or both of the touch operation and the voice input.
  • an object for accepting designation for enabling one or both of an order by touch operation and an order by voice input is displayed on the screen, and the user is allowed to select a desired order method or switch the selection.
  • the terminal 100 performs an operation mainly for ordering by voice, if the meaning of the utterance content (order content) cannot be heard by the analysis unit 130, the user is prompted to utter again.
  • the generation unit 110 uses the display unit 121 and the audio output unit 122 to prompt the user to input order details using the touch panel 103.
  • the information generation unit 110 determines that the order reception by voice is not successful, the information generation unit 110 transmits response information to the server 200 and instructs the employee to direct the order directly to the user. Also good. As a result, it is expected that user satisfaction will be improved.
  • the service to which the present invention is applied is not limited to the provision of food and drink.
  • it may be the sale of goods.
  • the service providing place is not limited to the inside of the store, but may be, for example, a passenger plane or a train.
  • the terminal 100 may be provided on the front surface of each seat such as an aircraft or a vehicle (the back surface of the back of the front seat), and may be applied to food and drink provision services by crew members in the cabin or in the vehicle.
  • the terminal 100 may be provided with a function of paying the consideration (fee) for receiving the service.
  • a card reading mechanism for a credit card is provided in the terminal 100, and a user is allowed to select one of a plurality of payment methods, and a screen for guiding a cash payment is generated for a user who wishes to make a cash payment.
  • a screen for guiding the card insertion operation to the desired user is generated.
  • the timing for guiding the payment processing or the timing for supporting the payment processing may be determined based on the scene information. For example, if the order is not entered even after a predetermined time has passed since the last received order, if the movement of chopsticks and forks has not been confirmed for a certain period of time, the last ordered dish such as dessert has been ordered If the utterance suggesting the end of the order is confirmed, such as “full or full” or “Is it over now” or “Is it to account?”, The information generation unit 110 determines that the meal has ended. To do. Alternatively, when the analysis unit 130 detects that the user has stood up from the chair, it may be determined that the meal has ended. In this case, for example, when no order item is provided, when a dish remains on the table, or when a user's carry-on item such as a bag is detected, it is a temporary absence from the meal. May be determined.
  • the information generation unit 110 displays a waitress or waiter image on the screen and emits a synthesized voice such as “Thank you for coming today.
  • the output unit 120 is controlled to do so.
  • a message such as “How about the last tea?” May be displayed on the display unit 121.
  • the service provider side can inform the user that the order has been recognized.
  • the terminal 100 performs processing for guiding to accounting. For example, when the user agrees to provide tea or at the timing when tea is provided, the display unit 121 and / or the audio output unit 122 reads “Pay at the cash register with the card reader of this table when paying. A message for supporting the payment process is output as an image or sound.
  • the terminal 100 When payment at the cash register is selected, for example, the terminal 100 notifies “Please tell the table number at the cash register”, and the order contents and the amount charged by the terminal 100 associated with the table number are sent to the server.
  • the voucher is acquired from 200, a slip is generated based on the acquired information, and presented to the user.
  • the terminal 100 notifies “Please check out at the cashier”, extracts an image of the user's face or information for identifying the user from the image, and identifies the identification information of the terminal 100 Correspondingly, it is wirelessly transmitted to a cash register terminal (a cash register or other payment terminal provided in the store and operated by an employee; not shown).
  • the server 200 is inquired of the server 200 for the identifier of the terminal 100 received in association with the user authentication information, and the order content (content of the provided service) and the billing amount that can be stored in the server 200 in association with the terminal 100 are obtained. Obtain from the server 200. This eliminates the need for the employee to take the slip to the table and the user to take the slip to the cash register at the time of accounting.
  • the server 200 is provided with the function of the information generation unit 110.
  • the information generated by the analysis unit 130 is sequentially transmitted to the server 200, and order information, user support information, response necessity information and feedback information are generated in 200, and the terminal 100 receives user support information from the server 200. It will be received and notified to the user.
  • a step of capturing a scene around the user a step of acquiring the content spoken by the user, the captured image and the acquired sound
  • the step of performing the analysis based on the above and the step of outputting the result of the analysis may be executed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Tourism & Hospitality (AREA)
  • Acoustics & Sound (AREA)
  • Accounting & Taxation (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Finance (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Resources & Organizations (AREA)
  • Primary Health Care (AREA)
  • Development Economics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • User Interface Of Digital Computer (AREA)
  • Cash Registers Or Receiving Machines (AREA)

Abstract

L'invention concerne un dispositif de traitement d'informations (terminal 100) qui comprend : un moyen de photographie (partie de photographie 180) destiné à photographier une scène autour d'un utilisateur ; un moyen d'acquisition de voix (partie d'entrée vocale 190) destiné à acquérir ce qui a été prononcé par l'utilisateur ; des moyens d'analyse (partie d'analyse 130, partie de génération d'informations 110) destinés à analyser la voix acquise sur la base de l'image photographiée ; et un moyen de sortie (partie de sortie 120) destiné à délivrer le résultat de l'analyse.
PCT/JP2019/016317 2018-04-23 2019-04-16 Système d'aide au service clientèle WO2019208327A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018-082131 2018-04-23
JP2018082131A JP6535783B1 (ja) 2018-04-23 2018-04-23 接客支援システム

Publications (1)

Publication Number Publication Date
WO2019208327A1 true WO2019208327A1 (fr) 2019-10-31

Family

ID=67023766

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/016317 WO2019208327A1 (fr) 2018-04-23 2019-04-16 Système d'aide au service clientèle

Country Status (2)

Country Link
JP (1) JP6535783B1 (fr)
WO (1) WO2019208327A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7376272B2 (ja) 2019-08-02 2023-11-08 東京瓦斯株式会社 顧客管理システム、プログラム、装置および方法
JP6857270B1 (ja) * 2020-03-06 2021-04-14 株式会社Ｓｈｏｗｃａｓｅ Ｇｉｇ 注文管理支援サーバ、注文管理支援システム、注文管理支援方法
US20230289863A1 (en) * 2020-07-07 2023-09-14 Ntt Docomo, Inc. Recommendation system
JP6915184B1 (ja) * 2020-09-23 2021-08-04 株式会社日本システムプロジェクト 飲食物注文受付システム及び飲食物注文受付装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004252497A (ja) * 2002-01-15 2004-09-09 Masanobu Kujirada 飲食店において料理又は飲料を提供するための方法及びシステム
JP2006208751A (ja) * 2005-01-28 2006-08-10 Kyocera Corp 発声内容認識装置
JP2007004282A (ja) * 2005-06-21 2007-01-11 Oki Electric Ind Co Ltd 注文処理システム
JP2016035703A (ja) * 2014-08-04 2016-03-17 富士通株式会社 飲食ペース評価プログラム、飲食ペース評価方法、コンピュータ及び飲食ペース評価システム
JP2017162027A (ja) * 2016-03-07 2017-09-14 セイコーソリューションズ株式会社 注文入力装置および注文入力方法
JP6240291B1 (ja) * 2016-10-27 2017-11-29 ヤフー株式会社 決定装置、決定方法、及び決定プログラム

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009059286A (ja) * 2007-09-03 2009-03-19 Nec Corp 情報分析システムおよび情報分析方法
JP2011221683A (ja) * 2010-04-07 2011-11-04 Seiko Epson Corp 接客支援装置、接客支援方法およびプログラム
JP6671764B2 (ja) * 2014-07-25 2020-03-25 株式会社ビーマップ 携帯端末無線ｌａｎオーダーシステム、携帯端末無線ｌａｎオーダーシステムの制御方法、携帯端末無線ｌａｎオーダーシステムのプログラム及び記録媒体
JP6068408B2 (ja) * 2014-09-26 2017-01-25 京セラドキュメントソリューションズ株式会社 ワークフロー制御装置及びプログラム
US10373235B2 (en) * 2014-09-29 2019-08-06 Tabletop Media, LLC Table-side information device imaging capture
US20170364854A1 (en) * 2014-12-04 2017-12-21 Nec Corporation Information processing device, conduct evaluation method, and program storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004252497A (ja) * 2002-01-15 2004-09-09 Masanobu Kujirada 飲食店において料理又は飲料を提供するための方法及びシステム
JP2006208751A (ja) * 2005-01-28 2006-08-10 Kyocera Corp 発声内容認識装置
JP2007004282A (ja) * 2005-06-21 2007-01-11 Oki Electric Ind Co Ltd 注文処理システム
JP2016035703A (ja) * 2014-08-04 2016-03-17 富士通株式会社 飲食ペース評価プログラム、飲食ペース評価方法、コンピュータ及び飲食ペース評価システム
JP2017162027A (ja) * 2016-03-07 2017-09-14 セイコーソリューションズ株式会社 注文入力装置および注文入力方法
JP6240291B1 (ja) * 2016-10-27 2017-11-29 ヤフー株式会社 決定装置、決定方法、及び決定プログラム

Also Published As

Publication number Publication date
JP2019191795A (ja) 2019-10-31
JP6535783B1 (ja) 2019-06-26

Similar Documents

Publication Publication Date Title
WO2019208327A1 (fr) Système d'aide au service clientèle
US11093995B2 (en) Monitoring of customer consumption activity and management based on monitoring
US9310880B2 (en) Self-service computer with dynamic interface
US9367869B2 (en) System and method for virtual display
US20060186197A1 (en) Method and apparatus for wireless customer interaction with the attendants working in a restaurant
WO2014057704A1 (fr) Système de communication d'informations de produits, dispositif de communication d'informations de produits et dispositif de sortie d'informations de produits
KR101976366B1 (ko) 멀티미디어 기반의 고객 맞춤형 메뉴 추천이 가능한 무인 음식판매점 시스템
US8041605B1 (en) Advance order system
US10679469B2 (en) System and method of managing unavailable items in shared screen data
JP5142306B1 (ja) 商品情報提供システム、商品情報提供装置、及び商品情報出力装置
JP2019003361A (ja) 席管理装置、プログラム及びロボット
CN106952084A (zh) 自助点单支付系统及该系统的使用方法
JP6952453B2 (ja) 接客システム及び接客方法
JP2007264702A (ja) 空席待ちの客が事前注文可能なセルフオーダーｐｏｓシステム
JP5220953B1 (ja) 商品情報提供システム、商品情報提供装置、及び商品情報出力装置
KR20160128986A (ko) 전자테이블을 이용한 음식 주문 관리 시스템 및 그 방법, 서빙용 이송 장치
US11238859B2 (en) Voice-based transaction processing with location-based activation
US20210097461A1 (en) Multiple point of sale (pos) overall wait time optimization
JP2019053650A (ja) 自走式装置
US10749834B2 (en) Information processing apparatus, information processing method, program, and information processing system for sending a message with an image attached thereto
KR20190084394A (ko) 주문 관리 시스템
KR20160068266A (ko) 전자테이블을 이용한 음식 주문 관리 시스템 및 그 방법, 서빙용 이송 장치
US20200394628A1 (en) Store clerk call system and store clerk call processing method
JP7417119B2 (ja) 情報処理システム、情報処理システムの制御方法、電子機器、プログラム
KR20210070784A (ko) 단말을 이용한 무인 주문 서비스 제공 시스템

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19793030

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19793030

Country of ref document: EP

Kind code of ref document: A1