KR102216783B1 - System for providing order service for non-face-to-face and noncontact based on speech recognition - Google Patents

Abstract

A voice recognition-based non-face-to-face and non-contact order service providing system is provided. The system comprises: a kiosk that recognizes the user′s speech, identifies an order object with natural language processing, outputs the same as a voice, and performs a payment process after receiving the voice confirmation of the order object outputted by the voice from a user; a member store terminal that outputs an order sheet including the order object paid at the kiosk and records and stores the order object and a payment amount as sales data; and an order service providing server that includes a registration unit for mapping and registering the kiosk and the member store terminal, an assignment unit for assigning an order identification code corresponding to the order object when the selection and payment of the order object are made based on online to offline (O2O) from a user terminal, and a transmission unit for transmitting the stored order object data mapped to the order identification code to the member store terminal when the order identification code is scanned at the kiosk. Accordingly, it is possible to identify who can enter the store without having to employ a separate employee.

Description

Voice recognition-based non-face-to-face and non-contact order service provision system {SYSTEM FOR PROVIDING ORDER SERVICE FOR NON-FACE-TO-FACE AND NONCONTACT BASED ON SPEECH RECOGNITION}

The present invention relates to a system for providing a non-face-to-face and non-contact order service based on voice recognition, and provides a system capable of placing an order with a user's voice utterance without contact with a person or a screen.

Due to the continuous development of information technology, the installation and operation cost of equipment with TBSS (Technology-Based Self-Service), which induces customers to participate in service production by themselves, is lowered, and the efficiency is higher. In addition, as the minimum wage continues to rise, the demand for TBSS is steadily increasing in various types of service industries that require corresponding technologies and services. Initially, TBSS was able to provide more functions and utility to customers beyond simple task performance, and recently, information providing kiosks are used to guide information in large shopping malls or department stores, current location, and information in stores. The kiosk representing TBSS can be said to be a device that is installed in public facilities or streets for unmanned automation of various tasks so that customers can easily use it.In order to categorize the kiosk, information provision, promotion, information exchange, sales, and companies It provides various functions such as customer relationship.

At this time, a platform for operating a kiosk based on voice recognition was researched and developed. In this regard, Korean Patent Publication No. 2020-0000010 (published on January 2, 2020) and Korean Registered Patent No. 10-2100704 (2020 Announcement on April 14, 2014), a voice recognition activation menu to provide a voice recognition function is provided on the menu selection screen of the display unit where the transaction selection menu for various financial services is displayed when using a self-banking kiosk. In the case of selecting the voice recognition activation menu at, the customer's voice command is input through the voice input unit provided in the kiosk, and the voice command is recognized through the voice recognition function, and a transaction selection menu related to the recognized voice command is selected. The components are extracted and provided collectively through the screen of the display unit, a kiosk body equipped with a display, a plurality of input means provided on the surface of the kiosk body to receive the talker's voice signal, noise around the talker in addition to the voice signal Configurations of a control unit for removing is and an output unit for outputting information requested by a talker to a display based on a voice signal are disclosed.

However, as the pandemic is prolonged due to COVID-19, even if the touch-based kiosk is changed to voice-based, the initial screen is turned on to return to the order screen by touching it because the promotional video is running or turned off. Turn On), but the number of users using touch-based kiosks is decreasing because the fear that the user himself touches the screen again touched by various people in this process is approaching. In addition, when customers entering a kiosk-equipped store are individually heat-checked and sent in, there must be an employee who checks the temperature, and even if a thermal imaging camera is installed, an employee who has to watch the screen shot with the thermal imaging camera is required. Accordingly, research and development of a platform that induces customers to use kiosks themselves by adding fun elements while maintaining non-face-to-face and contactless orders and payments at the store is required.

In one embodiment of the present invention, the user does not have to touch the screen by setting to return to the initial order screen when the face of a person, especially eyes and nose, is recognized so as to maintain non-contact beyond non-face-to-face. By performing the check using a thermal imaging camera or infrared sensor, it is possible to identify who can enter the store without having a separate staff, and because the order is executed based on voice, you do not have to touch the touch screen of the kiosk to place an order. Even if you place an order based on O2O (Online to Offline), just scanning the QR code will directly enter the order form into the POS (Point of Sale) affiliated store terminal, so you can order while maintaining non-face-to-face and contactless, and wallpaper suitable for age and gender. We can provide a method of providing non-face-to-face and non-contact ordering services based on voice recognition that can induce users to use the kiosk themselves by providing interesting elements by providing a service that takes pictures and prints on the spot by selecting artificial intelligence. have. However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, an embodiment of the present invention recognizes a user's voice utterance, identifies an order object by natural language processing, and outputs it as a voice, and outputs it as a voice. A kiosk that performs a payment process after receiving the confirmation of the order object by voice from the user, outputs an order form including the order object paid at the kiosk, and records and stores the order object and payment amount as sales data, and a kiosk and affiliate store A registration unit that maps and registers a terminal, a grant unit that assigns an order identification code corresponding to the order object when an order object is selected and payment is made based on O2O (Online to Offline) from a user terminal, and the order identification code is scanned at a kiosk. If so, it includes an order service providing server including a transmission unit for transmitting the stored order object data mapped to the order identification code to the affiliated store terminal.

According to any one of the above-described problem solving means of the present invention, the user does not touch the screen by setting the order to return to the initial order screen when the human face, especially eyes and nose, is recognized so as to maintain non-contact beyond non-face-to-face. By performing the user's heat check using a thermal imaging camera or infrared sensor, it is possible to distinguish who can enter the store without having a separate staff, and because the order is executed based on voice, the touch of the kiosk is used to place an order. You do not need to touch the screen, and even if you place an order based on O2O (Online to Offline), just scan the QR code to directly enter the order form into the POS (Point of Sale) affiliated store terminal, so you can order while maintaining non-face-to-face and contactlessness. And by providing a service that takes pictures and prints on the spot by selecting a background screen suitable for the gender with artificial intelligence, it is possible to induce users to use the kiosk by themselves by providing interesting elements.

1 is a diagram illustrating a system for providing a non-face-to-face and non-contact order service based on voice recognition according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating an order service providing server included in the system of FIG. 1.
3 and 4 are diagrams for explaining an embodiment in which a voice recognition-based non-face-to-face and non-contact order service is implemented according to an embodiment of the present invention.
5 is a screen captured of a video for directly demonstrating a voice recognition-based non-face-to-face and non-contact order service according to an embodiment of the present invention.
6 is a flowchart illustrating a method of providing a non-face-to-face and non-contact order service based on voice recognition according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included, and one or more other features, not excluding other components, unless specifically stated to the contrary. It is to be understood that it does not preclude the presence or addition of any number, step, action, component, part, or combination thereof.

The terms "about", "substantially" and the like, as used throughout the specification, are used in or close to the numerical value when manufacturing and material tolerances specific to the stated meaning are presented, and are used in the sense of the present invention. To assist, accurate or absolute figures are used to prevent unfair use of the stated disclosure by unscrupulous infringers. As used throughout the specification of the present invention, the term "step (to)" or "step of" does not mean "step for".

In the present specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, or two or more units may be realized using one hardware. Meanwhile,'~ unit' is not meant to be limited to software or hardware, and'~ unit' may be configured to be in an addressable storage medium or configured to reproduce one or more processors. Thus, as an example,'~ unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , Subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. The components and functions provided in the'~ units' may be combined into a smaller number of elements and'~ units', or may be further divided into additional elements and'~ units'. In addition, components and'~ units' may be implemented to play one or more CPUs in a device or a security multimedia card.

In this specification, some of the operations or functions described as being performed by the terminal, device, or device may be performed instead in a server connected to the terminal, device, or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal, device, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal means mapping or matching the unique number of the terminal or the identification information of the individual, which is the identification information of the terminal. Can be interpreted as.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a system for providing a non-face-to-face and non-contact order service based on voice recognition according to an embodiment of the present invention. Referring to FIG. 1, a system for providing a non-face-to-face and contactless order service based on voice recognition 1 includes at least one user terminal 100, an order service providing server 300, at least one kiosk 400, and at least one It may include an affiliated store terminal 500. However, since the voice recognition-based non-face-to-face and non-contact order service providing system 1 of FIG. 1 is only an embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1.

In this case, each component of FIG. 1 is generally connected through a network 200. For example, as shown in FIG. 1, at least one user terminal 100 may be connected to the order service providing server 300 through the network 200. In addition, the order service providing server 300 may be connected to at least one user terminal 100, at least one kiosk 400, and at least one affiliated store terminal 500 through the network 200. In addition, at least one kiosk 400 may be connected to the order service providing server 300 through the network 200. Further, the at least one affiliated store terminal 500 may be connected to at least one user terminal 100, an order service providing server 300, and at least one kiosk 400 through the network 200.

Here, the network refers to a connection structure in which information exchange is possible between respective nodes such as a plurality of terminals and servers, and examples of such networks include a local area network (LAN) and a wide area communication network (WAN: Wide Area Network), Internet (WWW: World Wide Web), wired/wireless data communication network, telephone network, wired/wireless television communication network, etc. Examples of wireless data networks include 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), and Wi-Fi. , Internet, LAN(Local Area Network), Wireless LAN(Wireless Local Area Network), WAN(Wide Area Network), PAN(Personal Area Network), RF(Radio Frequency), Bluetooth(Bluetooth) network, NFC( Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc. are included, but are not limited thereto.

In the following, the term “at least one” is defined as a term including the singular number and the plural number, and even if the term “at least one” does not exist, each component may exist in the singular or plural, and may mean the singular or plural. It will be self-evident. In addition, it will be possible to change according to the embodiment that each component is provided in a singular or plural.

The at least one user terminal 100 may be a terminal of a user who places an O2O-based order using a web page, an app page, a program, or an application related to a voice recognition-based non-face-to-face and non-contact order service. At this time, the user terminal 100 accesses the order service providing server 300 to select an affiliate store, outputs a list of at least one orderable menu at the affiliate store, and when any one order object is selected, performs payment to place an order. It may be a terminal that receives an order identification code, for example, a QR code from the service providing server 300. At this time, when the QR code is scanned at the kiosk 400, it is configured to browse the ordered menu, and the scanned QR code may no longer be used or may be deleted or deactivated.

Here, the at least one user terminal 100 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a navigation system, a notebook equipped with a web browser, a desktop, a laptop, and the like. In this case, the at least one user terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one user terminal 100, for example, as a wireless communication device that is guaranteed portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS(Personal Handyphone System), PDA(Personal Digital Assistant), IMT(International Mobile Telecommunication)-2000, CDMA(Code Division Multiple Access)-2000, W-CDMA(W-Code Division Multiple Access), Wibro(Wireless Broadband Internet) ) It may include all kinds of handheld-based wireless communication devices such as terminals, smart phones, smart pads, and tablet PCs.

The order service providing server 300 may be a server that provides a voice recognition-based non-face-to-face and contactless order service web page, an app page, a program, or an application. In addition, the order service providing server 300 may be a server that manages at least one kiosk 400 by assigning a unique code, for example, an ID, and registers in association with at least one affiliated store terminal 500. If the affiliate store terminal 500 is completely removed from the unmanned store, it is also possible to store only the kiosk 400. In addition, the order service providing server 300 generates an order event in the user terminal 100 for O2O service, and when scanned in the kiosk 400, transmits order data to the affiliated store terminal 500 to create an ordered menu. It may be a server that outputs an order form. In addition, when an error occurs in the kiosk 400 for the management of at least one kiosk 400, the order service providing server 300 collects error data to learn a pattern and at least one kiosk ( 400) may be a server that updates the voice recognition chatbot.

Here, the order service providing server 300 may be implemented as a computer that can access a remote server or terminal through a network. Here, the computer may include, for example, a navigation system, a notebook equipped with a web browser, a desktop, a laptop, and the like.

The at least one kiosk 400 may be a device that receives an order based on voice recognition and performs payment using a web page, app page, program, or application related to a voice recognition-based non-face-to-face and contactless order service. At this time, the at least one kiosk 400 is a device that receives an order by switching to an order screen, that is, a home screen, when a feature indicating that a person's face, especially eyes or nose, is recognized while outputting a promotional image by default. I can. In addition, at least one kiosk 400 may perform a heat check, and when the user terminal 100 scans the QR code, it may be a device that transmits the result of scanning the QR code to the affiliated store terminal 500. . In addition, the at least one kiosk 400 may be a device that learns a background screen corresponding to an age or gender by artificial intelligence, then takes a picture of a user, and immediately prints it. In addition, the at least one kiosk 400 may be a device that receives an order based on voice recognition and outputs a comment according to each step as a voice message to receive an order and perform payment based on voice rather than touch. Of course, it's obvious that it's not excluding touch-based. In addition, at least one kiosk 400 may be a device that generates an error report when an error occurs in voice recognition and transmits it to the order service providing server 300, periodically or in real time by the order service providing server 300 It may be an updated device.

Here, at least one kiosk 400 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a navigation system, a notebook equipped with a web browser, a desktop, a laptop, and the like. In this case, the at least one kiosk 400 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one kiosk 400, for example, as a wireless communication device that is guaranteed portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) It may include all types of handheld-based wireless communication devices such as terminals, smartphones, smart pads, and tablet PCs.

At least one affiliated store terminal 500 is an order service providing server 300 from the kiosk 400 or the user terminal 100 using a web page, app page, program or application related to a voice recognition-based non-face-to-face and contactless order service. It may be a terminal that outputs the order received via and accumulates sales.

Here, the at least one affiliated store terminal 500 may be implemented as a computer that can access a remote server or terminal through a network. Here, the computer may include, for example, a navigation system, a notebook equipped with a web browser, a desktop, a laptop, and the like. At this time, the at least one affiliated store terminal 500 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one affiliated store terminal 500 is, for example, a wireless communication device with guaranteed portability and mobility, and includes a Point of Sale (POS), a Personal Communication System (PCS), a Global System for Mobile communications (GSM), and a PDC ( Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), All types of handheld-based wireless communication devices such as Wibro (Wireless Broadband Internet) terminals, smartphones, smart pads, and tablet PCs may be included.

FIG. 2 is a block diagram illustrating an order service providing server included in the system of FIG. 1, and FIGS. 3 and 4 are a diagram illustrating a non-face-to-face and contactless order service based on voice recognition according to an embodiment of the present invention. It is a diagram for explaining an embodiment, and FIG. 5 is a screen captured of a video for directly demonstrating a voice recognition-based non-face-to-face and non-contact order service according to an embodiment of the present invention.

Referring to FIG. 2, the order service providing server 300 may include a registration unit 310, a granting unit 320, a transmission unit 330, and a relearning unit 340.

The order service providing server 300 according to an embodiment of the present invention or another server (not shown) operating in conjunction with at least one user terminal 100, at least one kiosk 400, and at least one affiliated store terminal ( 500) when transmitting a voice recognition based non-face-to-face and contactless order service application, program, app page, web page, etc., at least one user terminal 100, at least one kiosk 400, and at least one affiliated store terminal ( 500) may install or open a voice recognition-based non-face-to-face and contactless order service application, program, app page, web page, and the like. In addition, a service program may be driven in at least one user terminal 100, at least one kiosk 400, and at least one affiliated store terminal 500 by using a script executed in a web browser. Here, the web browser is a program that allows you to use the Web (WWW: World Wide Web) service, which means a program that receives and displays hypertext described in HTML (Hyper Text Mark-up Language). For example, Netscape , Explorer, Chrome, etc. In addition, the application refers to an application program on the terminal, and includes, for example, an app running on a mobile terminal (smartphone).

Referring to FIG. 2, the registration unit 310 may map and register the kiosk 400 and the affiliated store terminal 500. The kiosk 400 recognizes the user's voice utterance, identifies the order object by natural language processing, outputs it as a voice, and performs a payment process after receiving the voice confirmation of the order object output as a voice from the user. can do. The affiliated store terminal 500 may output an order book including an order object paid by the kiosk 400, and record and store the order object and the payment amount as sales data.

Each kiosk 400 is managed by mapping and storing a unique code, for example, ID, and collecting menus such as beverages or food sold by each affiliated store from each affiliated store terminal 500 and outputting them at each kiosk 400 You can register. In this case, the order service providing server 300 according to an embodiment of the present invention may provide two main functions. One is a function of managing the kiosk 400 and the other is a function of connecting the user terminal 100 and the kiosk 400 for O2O service.

The kiosk 400 itself to perform a process corresponding to a certain degree or configuration may vary depending on the load on the network. All basic order and payment configurations are performed at the kiosk 400, and whether only error checks or updates are to be performed at the order service providing server 300, or detailed orders are all performed at the order service providing server 300, depending on the embodiment. It may vary according to the situation of the installed store and the amount of resources such as computing resources or networking resources, and is not limited to any one embodiment. However, in an embodiment of the present invention, it is assumed that the ordering process is performed by the kiosk 400, data transmission and reception for O2O service, and bug fixes or updates are performed by the order service providing server 300. Of course, it is not limited to this embodiment as described above.

When the user terminal 100 selects and pays an order object based on O2O (Online to Offline), the granting unit 320 may assign an order identification code corresponding to the order object. If orders and payments have been made at the end of the user terminal 100, now the user only needs to go to the corresponding store and retrieve the food or drink. At this time, if the user terminal 100 directly shows the order identification code to the store staff, this is also a face-to-face service. Accordingly, the kiosk 400 scans the QR code, and the kiosk 400 allows the order form to go to the affiliated store terminal 500, that is, the POS, thereby reducing the action of the user to contact the employee by one more time. Finally, if the drink or food is served and the order number is attached to the drink or food where the drink comes out, just go to it. In other words, there is no face-to-face contact with the staff or the touch screen of the kiosk 400 from payment to find a drink or food.

When the order identification code is scanned by the kiosk 400, the transmission unit 330 may transmit the stored order object data mapped to the order identification code to the affiliated store terminal 500. In this case, the scan may be an infrared scanner that scans a barcode or a separate device that scans a QR code. Since the scanner does not have to directly contact the scanner and the smartphone (user terminal 100), there is no need to contact other objects during the scanning process. Although the promotion screen (advertisement screen) is output on the kiosk 400 by default, the kiosk 400 returns to the order mode at the moment when the kiosk 400 recognizes a person's eyes or nose by the facial recognition sensor 410 to be described later. You don't even have to touch the screen. After the user shows his or her face toward the kiosk 400, it is completed by placing a screen on which the QR code is floating on his smartphone, that is, the user terminal 100 toward the scanner provided in the kiosk 400.

In this case, the kiosk 400 may include a facial recognition sensor 410 provided to output a screen for performing an order process by switching to a home screen only when eyes and noses in the face of a person are recognized. For example, the facial recognition sensor 410 may be a high frequency sensor, but is not limited thereto. For example, when a person approaches using a proximity sensor, a promotion screen may be displayed and then converted to an order mode screen. In this case, since what kind of sensor is used may vary according to each embodiment, it is not limited to the above and is not excluded for reasons not listed.

The kiosk 400 may include a body temperature measuring device 440 for measuring a person's body temperature, and may be set to output an alarm when a body temperature higher than a preset reference body temperature is measured. In this case, the body temperature measuring device 440 may be a thermal imaging camera, but is not limited thereto.

The kiosk 400 may include a camera 430 for photographing a user who is an orderer, and a printer 420 for outputting a photographed user as a picture. In this case, a background screen previously mapped to the user's age and gender and stored may be set as a background excluding the user as a subject and output. In this case, a technique for separating the subject from the foreground is not specified as any one because various techniques can be used. Briefly, first, the photo is binarized and pre-processed to delete the rest of the background except for the user, which is the subject, and after recognizing the subject as outline recognition, the remaining background is deleted, and the stored background is pre-mapped to age and gender. It is to synthesize and take a picture and print it. At this time, in order to check the age, an algorithm for estimating the age after facial recognition is required. After estimating the age, the preferred background screen is automatically inserted in the corresponding age group. If the background screen is dissatisfied, the user may request replacement of the background screen by voice, such as "another background", and the kiosk 400 You can change the background to another and print it on the screen and ask for confirmation such as "Do you like this background?"

The kiosk 400 may include an identification device (not shown) that scans or reads an order identification code. Accordingly, the user terminal 100 can complete the order by going directly to the kiosk 400 and scanning the QR code without going to the store staff with the order identification code. In this case, if the order identification code is a barcode or QR code, an infrared scanner or the like may be used, but if the order identification code is a different type of order identification code, a corresponding scanner or reader may be used.

The relearning unit 340 manages at least one kiosk 400 based on a unique code, and when voice recognition error data is reported from at least one kiosk 400, the error part and the cause of the error are determined by voice recognition. After updating the data, it is possible to retrain a previously built artificial intelligence speech recognition algorithm by grasping the pattern of errors. In this case, the kiosk 400 may be at least one kiosk 400 located in at least one place and assigned a unique code. When the device is not driven well, the relearning unit 340 may report an error to the relearning unit 340 for all driving processes until a word or sentence containing such a keyword, including onomatopoeia or abusive language, which people use a lot, appears. At this time, the relearning unit 340 receives the error part and the cause of the error from the manager terminal (not shown), learns to find the error part and the pattern of the cause of the error, and then at least one kiosk 400 so as not to repeat it. You can proceed with the process of updating.

In this case, the kiosk 400 may include an artificial intelligence chatbot to operate based on voice recognition. Chatbot is a compound word of chat and robot, and refers to an interactive system that interacts with people based on artificial intelligence, a technology that combines the artificial intelligence of a robot with an interactive interface. With the advancement of artificial intelligence technology for chatbots, machine learning or deep learning technology is applied to chatbots to understand users' questions, understand their intentions, and provide appropriate answers, making it possible to perform more complex tasks. In addition, as the spread of smartphones and mobile messenger users increase rapidly, most smartphone users use messenger apps, so if you provide chatbot services using messenger apps, you can provide chatbot services to users in a familiar environment without installing new apps. Became possible. Accordingly, existing messenger companies are developing a messenger platform with artificial intelligence technology as well as opening their own messenger platform APIs to preoccupy the chatbot market.

Chatbots can be classified into rule-based chatbots and machine learning-based chatbots according to their operation method, one-time Q&A chatbots and continuous conversational chatbots according to information exchange method, and search model chatbots and generation model chatbots according to the answer generation method. The rule-based chatbot operates based on a predefined rule, and includes an interpretation rule that interprets the user's input, a reaction rule that responds to the user's input, or a rule that generates a response. Rule-based chatbot development does not require a large amount of data and is relatively easy to implement.For rule-based chatbots, high-quality conversation services are possible if rules are well defined, but a lot of time and manpower are required to well define many rules. do.

Machine learning-based chatbots use machine learning algorithms to understand natural language and generate or select appropriate answers. The NLP (Natural Language Process) module decomposes the natural language input by the user by using various linguistic resources, and the NLU (Natural Language Understanding) module understands the meaning of the decomposed natural language and answers refined information. It is passed to the final classification model or generation model for generation. Furthermore, by using the result of the NLU module, the machine learning algorithm selects a previously defined answer or the NLG (Natural Language Generation) module generates a new answer. Machine learning-based chatbots can communicate at a high level, but require a high level of natural language processing technology, and the NLP module and NLU module require many linguistic resources and data (corporate dictionary, morpheme dictionary, semantic translation dictionary).

In addition, machine learning modules made to perform a specific task often need to be newly developed depending on the application field, and most are difficult to recycle. One-time Q&A chatbots do not require a conversation management function and operate in a similar form to a search engine that presents the most relevant answers to questions. Most of the FAQ chatbots that provide fixed answers to frequently asked questions are designed in a one-time Q&A format. Continuous conversational chatbots have a conversation start and end, collect information from users to achieve a specific purpose, and end the conversation when a final response is made based on the collected information. Since it is impossible for chatbots to respond to various topics, most of them have a structured conversation format (scenario) depending on the topic of a specific conversation. A chatbot system that operates according to a predetermined scenario is called a frame-based system. The frame-based discourse system constructs a slot, which is a set of information for answering a user's question, generates a question based on the slot, and develops the discourse according to the state in which the requested information is collected.

Search-based chatbots work by selecting and presenting the most appropriate answer among answers stored in the existing knowledge base according to the user's question. Generation-based chatbots generate and provide new answers that are not previously defined according to user questions. The generative model chatbot tends to make a grammatical error or generate an unusual answer because it has a large degree of freedom to generate an answer, and a very large amount of learning data is required to create a generative model chatbot. Currently, most commercially available chatbots are search-based chatbots or hybrid methods that combine a search model and a generation model, and recently, many researchers are conducting research to create a chatbot by applying a deep learning algorithm (Encoder-Decoder, Seq2Seq).

In one embodiment of the present invention, the above-described artificial intelligence chatbot implements an order process to operate based on voice. The ordering process is a formal conversation between employees and customers, such as what kind of beverage to order, what size to order, whether ice or hot, and whether to add toppings. have. Accordingly, it is possible to implement simply by selecting the user for each step, and real-time processing is possible. In addition, since the artificial intelligence chatbot can query by voice and the user speaks only keywords, the elderly and children can easily use it. In addition, for users who cannot understand the voice properly, the voice message is outputted on the screen at the same time, so the convenience is high.

For example, a microphone can be built into a speaker for speech recognition, and the user's speech received from the microphone is converted into text by converting it into STT (Speech To Text). Then, the converted text is transmitted to perform a function suitable for the user's intention. That is, a cloud-based API may be used as a speech to text (STT) module for converting a user's voice input from a microphone into text, but is not limited thereto. In addition, when the processed text is synthesized again into speech, the speech is output to the user through the process of TTS (Text To Speech).

In addition, due to the nature of the kiosk 400, since it is difficult to change or cancel after the order has already been performed, if you want to change or cancel, it is cumbersome because you must run to the store staff again to request cancellation and perform the change. In this case, in an embodiment of the present invention, the kiosk 400 is used as a financial information cloud using the blockchain's Tendermint algorithm, and the user can easily check his or her payment and payment details, and can change and cancel immediately. You can also use a payment platform.

Blockchain Tendermint algorithm is a consensus algorithm used by Cosmos, which is a consensus algorithm improved by mixing Delegated Proof-of-Stake (DPoS) and PBFT concepts to suit public and private networks. Existing consensus algorithms generate blocks in a'pre-block generation, post-consensus' method, which can lead to a problem of partial branching. For example, when blocks A and B occur at the same time, the node selects one of the two. In this process, the determination of the previous block is determined according to whether the subsequent block is determined. Since this requires a lot of time, it is difficult to apply it to a service that requires an immediate transaction. On the other hand, Tendermint, on the contrary, has a mechanism of ‘preliminary consensus and post-block generation’. Therefore, users can retrieve the latest record information even if they do not have all the previous data, and through this, they can safely use blockchain-based services.

The current kiosk's payment system has a structure in which payment information is transmitted to the store owner in one direction. Apart from this, a cloud service that allows users to easily manage their financial information can be used through a system in which the user and the store owner directly exchange and share payment information. First, when a user pays through the kiosk 400, payment information is immediately generated as a block in the network of nodes. The contents of the information stored in these blocks basically include the payment information specified in the receipt. Data on payment location information, amount, item, and card authorization number are stored in blocks, so nodes within the network can freely store and view data through agreement.

After the above stored financial information is primarily stored in the kiosk 400, the personal mobile, that is, the user terminal 100 allows the user to access. After paying at the kiosk 400, users can check their payment and payment information in real time on mobile, and if they want to cancel or change, they can immediately refund and repay using mobile data without the need for a separate receipt. have. The user's financial information cloud service through interworking between the kiosk 400 and the user terminal 100 allows users to easily pay and refund with only data without papers, away from the existing payment system based on receipts. This improves the existing inefficiency, which had to check receipts in order to check payment information, and increases the convenience of consumption activities by allowing users to more easily access their payment information.

Current kiosks can only operate one merchant payment system per device. However, this causes waste of resources and an economic burden because the kiosk 400 must be installed for each necessary store. In order to solve this problem, it is necessary to use a common platform to replace the payment method of many merchants with fewer devices, so a platform that shares the kiosk 400 based on the Tendermint algorithm of the blockchain may be used.

For example, each kiosk 400, the user terminal 100, and the affiliated store terminal 500 of the affiliated store owner have their own codes. In the payment details, codes in the order of [Affiliate Store Terminal 500, User Terminal 10), and Kiosk 400] are stored as blocks, and the stored blocks are shared on the network of numerous kiosks 400. The user can view the block containing his/her payment information even at a kiosk 400(N) other than the kiosk 400(1) paid by him/her. Or, if another kiosk 400(N) requests to change or cancel the payment, the request details are stored in a block, and when the nodes agree through agreement, the payment details are actually reflected. That is, since payment information is all shared in blocks, there is no need to have the kiosk 400 uniquely in one store. Several stores share one kiosk 400 payment platform, so that users can proceed with payment of various stores in the same kiosk 400.

In the above case, the process driven in the user terminal 100 is as follows. First, it is assumed that the user of the user terminal 100 having the A code has made the merchant payment of the affiliated store terminal 500 to which the C code is assigned at the kiosk 400 having the B code. At this time, the payment information is stored as a block with contents [A, B, C]. If user A wants to view the payment information [A, B, C], they can read it directly from another kiosk B’ (400(B')) without having to find the B kiosk 400(B). In addition, if there is an error in payment and a change is required, such request information arrives directly to the affiliated store terminal 400 C when a change request is made. The user terminal 100 A may check the approval result of C in the mobile cloud linked in real time. In other words, the user can directly send a change request using his/her payment information without having to go to the kiosk 400(B) where payment was made to modify his/her payment information. In addition, since a common payment platform is used, it is possible to simplify the payment process of the user by solving the payment of not only the C store but also other stores in the B kiosk 400 (B).

When the above-described shared kiosk 400 is popularized, the accessibility of the user's payment and refund procedure is increased primarily, and secondarily, the kiosk is paradoxically because a single kiosk 400 can be shared by small merchants or self-employed people. The sales rate of 400 could also increase. In addition, if users and store owners share with one payment platform, transparency about financial information is enhanced. Opportunities to create secondary services using payment data can also increase.

Hereinafter, the operation process according to the configuration of the order service providing server of FIG. 2 will be described in detail with reference to FIGS. 3 to 5 as an example. However, it will be apparent that the embodiment is only any one of various embodiments of the present invention, and is not limited thereto.

Referring to FIG. 3, (a) the order service providing server 300 is configured to output a promotion screen by default with a face recognition sensor 410, and immediately switch to an order mode when a person's face is recognized. In addition, it is possible to check whether or not a person can enter the store through the body temperature measuring device 440, and if there is a high fever, an alarm may sound and a message to receive an examination may be outputted as a voice message. In addition, the camera 430 may take a picture of the user, and a desired background may be inserted and synthesized with the subject of the user, and then the picture may be printed or printed through the printer 420.

On the other hand, (a) when the user places an order directly at the kiosk 400, the order is made based on voice, and even when the card is inserted, the kiosk 400 does not directly contact the card, so it is possible to maintain a contactless state. have. And, (c) in the case of payment by O2O from the user terminal 100, if only the QR code is scanned or read on the kiosk 400, the order form is transmitted to the affiliated store terminal 500, which is a POS, so also the store staff as well as the kiosk 400 There is no need to contact.

4A is a home screen converted to an order mode. Usually, a promotional image or an advertisement image is output, and when a person is detected through the facial recognition sensor 410 of the kiosk 400, the order mode is switched as shown in FIG. 4A. At this time, when the user who is the orderer utters "Please give me 1 cup of ice Americano", "Ice", "Americano", and "1 cup" are entered as keywords, respectively, and a price corresponding thereto is presented to the user and a confirmation is requested. If confirmed, it checks whether it is takeout or not and proceeds with payment. This process will be described in detail with reference to FIG. 5.

FIG. 4C shows a configuration in which the kiosk 400 measures body temperature, prints a photo, changes a background, and recognizes text to be inserted into a photo by voice. Body temperature can be measured using a thermal imaging camera or infrared sensor, or after determining the age and gender by facial recognition, the preferred background of this group can be automatically inserted, or it can be switched to a different background at the user's instruction (voice). Or, if the user wants to put a speech bubble in a picture and put text in it, it can be seen that all are possible based on voice. Figure 4d shows the O2O service, which allows the order to be identified by a QR code and shows two methods of placing an order based on the QR code. One is as described above and the other is a method of attaching a QR code in a coffee shop and placing an order through the user terminal 100 by O2O. This method also ensures that no contact with the employee occurs, so it can be kept non-face-to-face and non-contact.

4E is a table comparing the kiosk 400 according to an embodiment of the present invention and a kiosk of other companies. In one embodiment of the present invention, voice recognition AI is applied to prevent non-face-to-face orders and the spread of the coronavirus, and thermal imaging cameras and facial recognition cameras are used.However, the kiosks of other companies are entangled in the hands of several people with a touch order. It only discloses how to spread. The UI/UX that responds to the ordering method is also complex, so people who are not familiar with the machine well or who are not familiar with the machine run to the store staff again, so non-face-to-face and contactless work becomes a futile task.

<Example>

5 is a partial capture of a video that directly demonstrates an order service according to an embodiment of the present invention. Referring to FIG. 5A, a screen saying "I will order words" is displayed, but this is not an order mode screen. In order to place an order, it is necessary to switch to the order mode screen, but the kiosk 400 according to an embodiment of the present invention recognizes a face to check whether there is a person who wants to order by recognizing a face and then switches to the order mode. After recognizing the face in this way, it is switched to the order screen as shown in FIG. 5B. At this time, since the video was captured, the voice could not be expressed, but the kiosk 400 on the screen of FIG. 5B currently says, "This is a coffee menu. Please tell me the menu you want", and this comment is also displayed in text on the screen. And, the demonstrator in the video uttered "Please give me a cup of warm Americano", and as shown in Fig. 5c, the kiosk 400 says "You have chosen a warm Americano" and displays an audio comment on the screen so that the user can check the ordered menu. Do. Of course, the menu of "Warm Americano" is also displayed as a selected menu at the bottom, so you can check it intuitively. At this time, the kiosk 400 said, "Please tell me about payment or an additional drink," and the demonstrator in the video selected "tea drink".

When the demonstrator selects "tea drink", the kiosk 400, as shown in FIG. 5D, has passed to the tea drink menu, and outputs a voice message saying "Please tell me the menu you want". At this time, the demonstrator uttered, "Please give me a cup of ginger tea", and the kiosk 400 as shown in Fig. 5e outputs a voice comment saying, "You have selected a cup of ginger tea. Please tell us about payment or an additional drink." It is being displayed on the screen at the same time. At this time, the demonstrator voiced "payment", and the kiosk 400 uttered "Do you want to take out? Please say no to takeout" as shown in FIG. 5F, and this voice comment is also output as text on the screen at the same time. Can be confirmed.

At this time, the demonstrator uttered "Takeout", and as shown in FIG. 5G, the kiosk 400 outputs a voice message saying "Please insert the payment card into the payment machine", and it can be confirmed that it is also output as text on the screen at the same time. have. The demonstrator inserted the payment card into the card insertion slot, and the kiosk 400 immediately proceeded with the payment. After inserting into the card insertion slot, the kiosk 400 outputs a voice message saying "Payment has been completed. Please receive a receipt and a waiting number" as shown in FIG. 5H, and similarly, it is simultaneously displayed on the screen. When the order is completed and the drink is available, the orderer simply picks up the drink.

The matters not described for the method of providing a non-face-to-face and non-contact order service based on voice recognition of FIGS. 2 to 5 are the same as those described for the method of providing a non-face-to-face and non-contact order service based on voice recognition through FIG. Since it can be easily inferred from the description, the description will be omitted below.

6 is a diagram illustrating a process of transmitting and receiving data between components included in the system for providing a non-face-to-face and contactless order service based on voice recognition of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of a process in which data is transmitted/received between each component will be described with reference to FIG. 6, but the present application is not limitedly interpreted as such an embodiment, and is illustrated in FIG. 6 according to various embodiments described above. It is apparent to those skilled in the art that the process of transmitting and receiving data may be changed.

6, the order service providing server maps and registers a kiosk and an affiliated store terminal (S6100).

Then, the order service providing server, when the selection and payment of the order object is made based on O2O (Online to Offline) from the user terminal, gives an order identification code corresponding to the order object (S6200).

Finally, when the order identification code is scanned at the kiosk, the order service providing server transmits the stored order object data mapped to the order identification code to the affiliated store terminal (S6300).

The order between the above-described steps (S6100 to S6300) is only an example and is not limited thereto. That is, the order of the above-described steps (S6100 to S6300) may be mutually changed, and some of the steps may be executed or deleted at the same time.

The matters not described for the method of providing the voice recognition-based non-face-to-face and contactless order service of FIG. 6 are the same as those described for the method of providing the voice recognition-based non-face-to-face and contactless order service through FIGS. Since it can be easily inferred from the description, the description will be omitted below.

The method for providing a non-face-to-face and non-contact order service based on voice recognition according to an embodiment described with reference to FIG. 6 is also in the form of a recording medium including instructions executable by a computer such as an application or program module executed by a computer. Can be implemented. Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Further, the computer-readable medium may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The method for providing a non-face-to-face and non-contact order service based on voice recognition according to an embodiment of the present invention described above is an application basically installed in a terminal (this may include a program included in a platform or an operating system basically installed in the terminal) It may be executed by, and may be executed by an application (ie, a program) directly installed on the master terminal by a user through an application providing server such as an application store server, an application, or a web server related to the service. In this sense, the method for providing a non-face-to-face and contactless order service based on voice recognition according to an embodiment of the present invention described above is implemented as an application (i.e., a program) installed in the terminal or directly installed by the user. It can be recorded on a computer-readable recording medium.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

Claims (7)

A kiosk 400 for recognizing a user's voice utterance, identifying an order object by natural language processing, outputting the order object as a voice, and performing a payment process after receiving a voice confirmation of the order object output as a voice from the user;
An affiliate store terminal 500 for outputting an order form including an order object paid by the kiosk 400, and recording and storing the order object and the payment amount as sales data; And
Registration unit 310 for mapping and registering the kiosk 400 and the affiliated store terminal 500, when selection and payment of an order object is made based on O2O (Online to Offline) from a user terminal, order identification corresponding to the order object A grant unit 320 for assigning a code, and a transmission unit 330 that transmits the stored order object data mapped to the order identification code to the affiliated store terminal 500 when the order identification code is scanned by the kiosk 400 Including an order service providing server 300; including,
The kiosk 400 includes; a facial recognition sensor 410 provided to output a screen for performing an order process by switching to a home screen only when eyes and noses in the face of a person are recognized,
The kiosk 400 includes a body temperature measuring device 440 for measuring a person's body temperature; Including, is set to output an alarm when a body temperature above a preset reference body temperature is measured,
The kiosk 400 includes an identification device for scanning or reading the order identification code,
The kiosk 400 is at least one kiosk 400 located in at least one place and assigned a unique code, and the order service providing server 300 includes the at least one kiosk 400 with a unique code. Based on management, and when voice recognition error data is reported from the at least one kiosk 400, the error part and the cause of the error are updated in the voice recognition big data, and then the pattern of the error is identified, and the previously built artificial intelligence Further comprising a; relearning unit 340 for re-learning the speech recognition algorithm,
The kiosk 400 includes an artificial intelligence chatbot to operate based on voice recognition,
The kiosk 400 is a voice recognition-based non-face-to-face and contactless order, characterized in that, based on the Tendermint algorithm of the blockchain, when a user cancellation request of payment details is inputted through a payment process. Service delivery system.
delete delete delete delete delete delete

Images