JP2018106524A - Interactive device, interactive method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which no such interaction using an image input by a user was available in the prior art.SOLUTION: The device comprises: a receiving part for receiving image input information representing an image sent by a user terminal or string input information representing a string; a response acquiring part for acquiring the analysis result of the image input information and thereby acquiring response information using the analysis result; and a transmitting part for transmitting the response information to the user terminal. An interactive device in which the response acquiring part processes the acquisition of the response information differently between when it receives the image input information and when it receives the string input information can achieve an appropriate interaction using the image input by the user.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an interactive apparatus that interacts with a user.

  Conventionally, there has been an interactive device that can generate an utterance that matches a user's preference (see Patent Document 1).

  Conventionally, there has been a spoken dialogue apparatus using a spoken dialogue sequence state notation method in which a scenario can be easily added or modified (see Patent Document 2).

  Furthermore, there has conventionally been an interactive system having the following configuration (see Patent Document 3). In other words, a classification unit that inputs input data and classifies the inputted input data into one of a plurality of categories, and a state transition starting from the category classified by the classification unit is interactive with the user State transition memory for registering a dialogue pair that is a pair of the dialogue management means that leads to the final state and the input data that has been input to the classification means and the state transition that has been advanced by the dialogue management means And new category extraction means for extracting a new category that is a new candidate for the category from the plurality of dialogue pairs registered by the state transition storage means, and the classification means includes the new category extraction means The dialogue system is characterized in that the new category extracted by (2) is added to the category that can be classified.

Japanese Patent Laying-Open No. 2015-225414 JP 2006-053470 A JP 2011-215742 A

  However, in the prior art, a dialog using an image input by the user cannot be performed.

  The interactive apparatus according to the first aspect of the invention acquires a reception unit that receives image input information that is an image transmitted from a user terminal, acquires an analysis result of the image input information, and acquires response information using the analysis result. It is an interactive apparatus comprising a response acquisition unit and a transmission unit that transmits response information to a user terminal.

  With this configuration, a dialogue using an image input by the user can be performed.

  In the interactive apparatus according to the second aspect of the present invention, in contrast to the first aspect of the invention, the reception unit also receives character string input information that is a character string transmitted from the user terminal, and the response acquisition unit receives the image input. The interactive device is different in the process of acquiring response information when receiving information and when receiving character string input information.

  With this configuration, an appropriate dialogue using an image and a character string input by the user can be performed.

  The interactive apparatus according to the third aspect of the invention stores one or more pieces of product information having a product image that is an image of the product and one or more attribute values of the product with respect to the first or second invention. A product information storage unit is further provided, the response acquisition unit acquires one or more product images from the product information storage unit using the analysis result, and the transmission unit is one or more product images acquired by the response acquisition unit. It is an interactive device that transmits certain response information to a user terminal.

  With this configuration, product information can be proposed to the user using an image input by the user.

  In the dialog device according to the fourth aspect of the present invention, in contrast to the third aspect of the invention, the receiving unit receives a selection instruction indicating selection of one product image among the one or more product images transmitted by the transmission unit. The interactive apparatus further includes a purchase processing unit that performs processing for purchasing a product corresponding to the selection instruction.

  With this configuration, the user can purchase a product while interacting.

  Further, in the dialogue apparatus according to the fifth aspect of the present invention, in response to any one of the first to fourth aspects, the response acquisition unit includes image input information transmitting means for transmitting the image input information to the image analysis apparatus, and image analysis. An interactive device comprising analysis result receiving means for receiving an analysis result of image input information from the device.

  With this configuration, the maintainability of the system can be improved by separating the interactive apparatus and the image analysis apparatus that analyzes the image.

  According to the dialogue apparatus according to the present invention, a dialogue using an image input by a user can be performed.

The figure which shows the conceptual diagram of the dialogue system A in Embodiment 1. FIG. Block diagram of the dialogue system A Block diagram of the dialogue apparatus 1 A flowchart for explaining an operation example of the dialogue apparatus 1 Figure showing the dialogue information management table Figure showing the same output example Figure showing the same output example Figure showing the same output example Figure showing an example of a screen for purchasing the product Figure showing the same output example Overview of the computer system Block diagram of the computer system

  Hereinafter, embodiments of an interactive device and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

  (Embodiment 1)

  In this embodiment, an interactive system including an interactive apparatus capable of inputting an image will be described.

  In the present embodiment, a dialog system including a dialog device in which response information is acquired differently depending on whether a character string is input or an image is input will be described.

  In the present embodiment, a case will be described in which the response information to the image input information is product information.

  In the present embodiment, a dialogue system in which a user can purchase a product using a product image recommended by dialogue will be described.

  Furthermore, in the present embodiment, a dialog system that transmits an image from the dialog device to the image analysis device when the user input received by the dialog device is an image will be described.

  FIG. 1 is a conceptual diagram of a dialogue system A in the present embodiment. The dialogue system A includes a dialogue device 1, one or more user terminals 2, or an image analysis device 3. Note that the number of image analysis devices 3 may be two or more.

  Moreover, although the interactive apparatus 1 and the image analysis apparatus 3 are server apparatuses, such as a cloud server and an ASP server, for example, the kind is not ask | required. Furthermore, the user terminal 2 is what is called a smart phone, a mobile phone, a portable terminal, a personal computer, a notebook computer, a tablet terminal etc., for example, The kind is not ask | required. The interactive device 1 and one or more user terminals 2 can communicate with each other via a network such as the Internet or a LAN. The interactive device 1 and the image analysis device 3 can communicate with each other via a network such as the Internet or a LAN.

  FIG. 2 is a block diagram of the dialogue system A in the present embodiment. FIG. 3 is a block diagram of the dialogue apparatus 1.

  The interactive device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and a transmission unit 14.

  The storage unit 11 includes a dialogue information storage unit 111 and a product information storage unit 112.

  The processing unit 13 includes a response acquisition unit 131 and a purchase processing unit 132.

  The response acquisition unit 131 includes a response acquisition unit 1311, an image analysis unit 1312, an image input information transmission unit 1313, and an analysis result reception unit 1314.

  The user terminal 2 includes a terminal storage unit 21, a terminal reception unit 22, a terminal processing unit 23, a terminal transmission unit 24, a terminal reception unit 25, and a terminal output unit 26.

  The image analysis device 3 includes a product information storage unit 112, an analysis reception unit 31, an image analysis unit 1312, and an analysis transmission unit 32.

  The storage unit 11 constituting the interactive apparatus 1 can store various types of information. The various types of information are, for example, dialogue information described later and product information described later.

  The dialogue information storage unit 111 stores dialogue information. The dialogue information is information for acquiring response information with respect to the received input information. The dialogue information stored in the dialogue information storage unit 111 may be dialogue information used in a known dialogue device including the above-described dialogue device according to the related art.

  Interactive information, for example, a character string transmitted from the user terminal 2 (e.g., "Hello") and response information (e.g., "Hello") is information pair with. In such a case, the dialog information storage unit 111 normally stores two or more dialog information. Dialog information includes, for example, an input character string pattern (for example, “What is“ variable 1 ”?”) And a response information pattern (for example, “variable 1” is “variable 2”). In this case as well, dialogue information storage unit 111 normally stores two or more dialogue information, where “variable 1” and “variable 2” are variables, The dialogue information may be state transition information indicating the flow of dialogue, etc. That is, the dialogue information may be information for obtaining response information with respect to the received input. Dialog information is usually information for obtaining response information for received character string input information, or for obtaining response information for analysis results of received image input information. Information.

  The product information storage unit 112 stores one or more product information. The product information is information about the product. Here, the term “commodity” is intended for sale and includes a so-called service, and is widely understood. The product information usually has a product image that is an image of the product and one or more attribute values of the product. The merchandise information includes, for example, a merchandise identifier for identifying the merchandise, a merchandise name, a price (unit price), a merchandise attribute value (eg, color, size, place of origin, material, shape, etc.), a picture of the merchandise, a picture of the merchandise, etc. . The product identifier may be referred to as, for example, a product number or a product ID.

  The receiving unit 12 receives one or more input information transmitted from the user terminal 2. Note that the receiving unit 12 may receive two or more pieces of input information at the same time or at the same timing. The input information is, for example, image input information or character string input information. The image input information is an image input by the user to the user terminal 2. The image input information is, for example, a photograph or a screen shot. The character string input information is a character string input to the user terminal 2 by the user. Note that the character string input information may include information input to the user terminal 2 by voice by the user. Further, when the user inputs voice to the user terminal 2, the process of recognizing the voice and acquiring the character string input information may be performed by the user terminal 2 or the processing unit 13 described later. Furthermore, when the receiving unit 12 receives one or more image input information, it is preferable that the character string input information is not received at the same timing. The reception at the same timing may be considered as reception together.

  The receiving unit 12 receives a selection instruction indicating selection of one product image among the one or more product images transmitted by the transmission unit 14. It is preferable that the selection instruction has a product identifier for identifying the product or a product type identifier for identifying the type of the product. The selection instruction may have a product image identifier for identifying the product image. The product image identifier may be considered as a product identifier.

  The processing unit 13 performs various processes. The various processes are, for example, processes of the response acquisition unit 131 and the purchase processing unit 132.

  The response acquisition unit 131 acquires an analysis result of one or more image input information, and acquires response information using the analysis result. The analysis result is, for example, one or more objects obtained by image recognition of one or more pieces of image input information. The object is, for example, the name of an object in the image input information, the kind name of the object in the image input information, or the like. Note that processing for recognizing an object in an image is a known technique, and thus detailed description thereof is omitted. In addition, for analysis of image input information, for example, the Internet homepage (URL: https: //dev.smt.docomo.ne.jp/? P = docs.api.page & api_name = image_recognition & p_name = api_reference), (URL: http: //www.deqwas.com/service/camera.html) etc. can be used.

  Further, the response acquisition unit 131 may transmit one or more pieces of image input information to the image analysis device 3, receive an analysis result from the image analysis device 3, and acquire response information from the analysis result. The response acquisition unit 131 may analyze one or more pieces of image input information, acquire an analysis result, and acquire response information using the analysis result. The analysis result may be response information, or the response acquisition unit 131 may acquire response information using the analysis result.

  Further, the response acquisition unit 131 may change an apparatus that performs image analysis according to the received one or more pieces of image input information. That is, if the received one or more pieces of image input information satisfy a predetermined condition, the response acquisition unit 131 transmits the one or more pieces of image input information to the image analysis device 3 and sets the predetermined condition. If not satisfied, the one or more pieces of image input information may be analyzed by themselves. In addition, when one or more received image input information does not satisfy a predetermined condition, the response acquisition unit 131 transmits the image input information to the image analysis device 3 and satisfies a predetermined condition. May analyze the one or more pieces of image input information by itself. In addition, the response acquisition unit 131 determines one or more image analysis devices 3 of the two or more image analysis devices 3 according to the received one or more image input information, and the one or more image analysis devices 3. The image input information may be transmitted to. The predetermined condition may be a condition related to an analysis result, a condition related to an attribute value of one or more image input information, a condition related to the user terminal 2 that has transmitted the image input information or a user, and the like. But it ’s okay. The condition relating to the analysis result is, for example, a condition relating to the attribute value (for example, type, number, etc.) of the object in the image input information. Further, the conditions relating to the attribute value of the image input information are, for example, the data size, resolution, etc. of the image input information.

  In the response acquisition unit 131, it is preferable that the process of acquiring the response information is different depending on whether the image input information is received or the character string input information is received.

The processing is different, for example, from one of the following (1) to (7).
(1) When image input information is received, the image input information is transmitted to the image analysis apparatus 3, and when character string input information is received, the character string input is not transmitted to the image analysis apparatus 3. Get response information from information.
(2) When image input information is received, image analysis is performed. When character string input information is received, response information corresponding to the character string input information is acquired without performing image analysis. The process of acquiring response information corresponding to the character string input information can be realized by, for example, the process of a known dialog device as described above.
(3) When image input information is received, a similar image is searched using the image input information. When character string input information is received, response information corresponding to the character string input information is acquired.
(4) When character string input information is received, response information is acquired using the dialog information in the dialog information storage unit 111, and when image input information is received, response information is acquired without using the dialog information.
(5) When character string input information is received, response information is acquired using the dialog information in the dialog information storage unit 111. When image input information is received, image analysis is performed, and the analysis result is converted into dialog information. Apply and get response information.
(6) When character string input information is received, response information is acquired using the dialog information in the dialog information storage unit 111. When image input information is received, the response information is transmitted to the image analysis device 3. The result of image analysis is obtained from, and response information is obtained.
(7) When image input information is received, it is transmitted to the image analysis device 3, an analysis result is obtained from the image analysis device 3, response information is acquired, and character string input information is received, The data is transmitted to an external device different from the analysis device 3, the analysis result is obtained from the external device, and response information is acquired.

  It is preferable that the response acquisition unit 131 acquires one or more pieces of product information from the product information storage unit 112 using the analysis result of the image input information. Note that the product information includes, for example, a product image.

  The response acquisition unit 1311 acquires response information from the character string input information received by the receiving unit 12 using dialogue information. The processing of the response acquisition unit 1311 can be realized by processing of a known dialog device.

  The image analysis unit 1312 performs image analysis on the image input information received by the receiving unit 12 and acquires an analysis result. A known technique is sufficient as the technique of image analysis. For the image analysis means 1312, for example, a known image recognition technique can be used.

  The image input information transmitting unit 1313 transmits the image input information received by the receiving unit 12 to the image analysis device 3. The image input information transmission unit 1313 may transmit the image input information to the image analysis apparatus 3 only when the image input information satisfies a predetermined condition. Further, the image input information transmitting unit 1313 may transmit the image input information to one of the two or more image analysis devices 3 according to the image input information or the analysis result of the image input information. .

  The analysis result receiving unit 1314 receives the analysis result of the image input information from the image analysis device 3. When there are two or more image analysis devices 3, the analysis result receiving unit 1314 receives the analysis results from the image analysis device 3 that has analyzed the image input information.

  The purchase processing unit 132 performs a process for purchasing a product corresponding to the selection instruction. The process for purchasing a product is, for example, acquisition of a web page for purchasing a product, or URL of the web page, or acquisition and transmission. The process for purchasing a product is, for example, a payment process for purchase. In addition, it may be considered that the process for purchasing a product widely includes processes related to purchasing the product.

  The transmission unit 14 transmits the response information acquired by the response acquisition unit 131 to the user terminal 2. The transmission unit 14 transmits one or more product images acquired by the response acquisition unit 131 to the user terminal 2. The one or more product images are response information.

  Various information is stored in the terminal storage unit 21 constituting the user terminal 2. The various types of information are, for example, user identifiers that identify users. The user identifier may be information for identifying the user terminal 2. The user identifier is, for example, an ID, a telephone number, a mail address, an IP address, a MAC address, browser cookie information, or the like.

  The terminal reception unit 22 receives various information and instructions. The various information and instructions are, for example, image input information, character string input information, and selection instructions.

  The input means for various information and instructions may be anything such as a touch panel, a keyboard, a mouse, or a menu screen. The terminal reception unit 22 can be realized by a device driver for input means such as a touch panel and a keyboard, control software for a menu screen, and the like.

  The terminal processing unit 23 performs various processes. The various processes are, for example, processes for configuring information and instructions received by the terminal receiving unit 22 into information and instructions to be transmitted to the dialogue apparatus 1. The various processes are, for example, processes for configuring information to be output from information received by the terminal reception unit 25.

  The terminal transmission unit 24 transmits various information and instructions to the dialogue apparatus 1. The various information and instructions are, for example, information and instructions received by the terminal receiving unit 22, processing results of the terminal processing unit 23, and the like.

  The terminal receiving unit 25 receives various types of information from the interactive device 1. The various information is, for example, response information.

  The terminal output unit 26 outputs information received by the terminal receiving unit 25. The information received by the terminal receiving unit 25 may include information configured by the terminal processing unit 23 using the information received by the terminal receiving unit 25.

  Here, the output is usually a display on a display, but projection using a projector, printing with a printer, sound output, transmission to an external device, accumulation in a recording medium, other processing devices and others This is a concept that includes delivery of processing results to other programs.

  The product information storage unit 112 constituting the image analysis device 3 stores one or more pieces of product information having a product image that is an image of the product and one or more attribute values of the product.

  The analysis receiving unit 31 receives image input information from the interactive apparatus 1. The analysis receiving unit 31 may receive two or more pieces of image input information together.

  The analysis transmission unit 32 transmits the analysis result acquired by the image analysis unit 1312. The analysis result may be response information.

  In the above block diagram, both the product information storage unit 112 of the interactive device 1 and the product information storage unit 112 of the image analysis device 3 exist, but either one may be used. In the above block diagram, both the image analysis unit 1312 of the interactive apparatus 1 and the image analysis unit 1312 of the image analysis apparatus 3 exist, but either one may be used.

  The storage unit 11, the dialogue information storage unit 111, the product information storage unit 112, and the terminal storage unit 21 are preferably non-volatile recording media, but can also be realized by volatile recording media.

  The process in which information is stored in the storage unit 11 or the like is not limited. For example, information may be stored in the storage unit 11 or the like via a recording medium, information transmitted via a communication line or the like may be stored in the storage unit 11 or the like, Alternatively, information input via the input device may be stored in the storage unit 11 or the like.

  The receiving unit 12, the terminal receiving unit 25, the analysis receiving unit 31, and the analysis result receiving unit 1314 are usually realized by a wireless or wired communication unit, but may be realized by a unit that receives a broadcast.

  The processing unit 13, the response acquisition unit 131, the purchase processing unit 132, the response acquisition unit 1311, the image analysis unit 1312, and the terminal processing unit 23 can be usually realized by an MPU, a memory, or the like. The processing procedure of the processing unit 13 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The image input information transmission unit 1313, the terminal transmission unit 24, and the analysis transmission unit 32 are usually realized by a wireless or wired communication unit, but may be realized by a broadcasting unit.

  The terminal output unit 26 may be considered as including or not including an output device such as a display or a speaker. The terminal output unit 26 may be implemented by output device driver software, or output device driver software and an output device.

  Next, the operation of the dialogue system A will be described. First, an operation example of the interactive apparatus 1 will be described using the flowchart of FIG.

  (Step S401) The receiving unit 12 determines whether or not input information has been received. If the input information is received, the process goes to step S402. If the input information is not received, the process goes to step S412.

  (Step S402) The response acquisition unit 131 determines whether or not the received input information is character input information. If it is character input information, go to step S403, and if it is not character input information, go to step S405.

  (Step S403) The response acquisition unit 131 applies the received input information to the dialog information in the dialog information storage unit 111, and acquires response information.

  (Step S404) The transmission unit 14 transmits the response information acquired by the response acquisition unit 131 to the user terminal 2 that has transmitted the input information. The process returns to step S401.

  (Step S405) The response acquisition unit 131 determines whether or not the received input information is image input information. If it is image input information, go to step S406, and if it is not image input information, go to step S411.

  (Step S406) The response acquisition unit 131 determines whether or not to transmit the received image input information to the image analysis device 3. If it is determined to transmit to the image analysis apparatus 3, the process goes to step S407. If it is determined not to transmit to the image analysis apparatus 3, the process goes to step S409.

  (Step S407) The image input information transmitting unit 1313 transmits the received image input information to the image analyzing apparatus 3.

  (Step S408) The analysis result receiving unit 1314 determines whether or not the analysis result of the image input information has been received from the image analysis device 3. If the analysis result is received, the process goes to step S410. If the analysis result is not received, the process returns to step S408.

  (Step S409) The image analysis unit 1312 performs image analysis on the image input information and acquires an analysis result.

  (Step S410) The response acquisition unit 1311 acquires response information using the analysis result of the image input information. Go to step S404.

  (Step S411) The transmission unit 14 transmits an error message to the user terminal 2. The process returns to step S401. The error message is, for example, a message indicating that the response information cannot be changed.

  (Step S412) The receiving unit 12 determines whether a selection instruction is received from the user terminal 2. If a selection instruction has been received, the process proceeds to step S413. If a selection instruction has not been received, the process returns to step S401.

  (Step S413) The purchase processing unit 132 performs a purchase process for purchasing a product corresponding to the selection instruction. The process returns to step S401.

  In the flowchart of FIG. 4, the process ends when the power is turned off or the process ends.

  Next, the operation of the user terminal 2 will be described. The terminal reception unit 22 receives information and instructions from the user. Next, the terminal processing unit 23 has a data structure for transmitting information and instructions. Next, the terminal transmission unit 24 transmits information and instructions to the dialogue apparatus 1. Then, the terminal receiving unit 25 receives information from the interactive apparatus 1 according to the transmitted information and instructions. The terminal output unit 26 outputs information received by the terminal receiving unit 25. Note that the information output here is, for example, response information.

  Next, the operation of the image analysis device 3 will be described. First, the analysis receiving unit 31 receives image input information from the interactive apparatus 1. Then, the image analysis unit 1312 analyzes the image input information and acquires the analysis result. Next, the analysis transmission unit 32 transmits the analysis result acquired by the image analysis unit 1312 to the dialogue apparatus 1.

  Hereinafter, a specific operation of the dialogue system A in the present embodiment will be described. A conceptual diagram of the dialogue system A is shown in FIG.

  Now, the dialogue information storage unit 111 stores the dialogue information management table shown in FIG. The dialogue information management table stores one or more records having “ID”, “input information”, “analysis information”, and “response information”. “Input information” and “response information”, “analysis information” and “response information”, or “input information”, “analysis information”, and “response information” may be referred to as dialogue information.

  The record of “ID = 1” in FIG. 5 indicates that when a dialog start instruction is received, the acquired response information is a sentence “What kind of product are you looking for?”. The record of “ID = 2” is a case where image input information is received. When the analysis result of the image input information is two or more product type identifiers, the acquired response information is “which category In addition to the sentence “Looking for?”, It indicates that the menu is composed of two or more product type identifiers as analysis results. “<Menu> </ menu>” in the response information of “ID = 2” indicates that a menu is configured. “<For> </ for>” indicates that the tag <product type identifier> sandwiched between them is repeated. That is, the information sandwiched between “<menu> </ menu>” in the response information of “ID = 2” indicates that a menu having two or more product type identifiers that are analysis results as menu items is configured. Show. In the record of “ID = 3”, when the input information transmitted from the user terminal 2 is a product type identifier (for example, “A”), the acquired response information is a sentence “A is it”. In addition, it indicates that the result is a process sandwiched between “<process> </ process>”. <Execute> in “<Process> </ Process>” is a process to be executed, and “<Execution> Image Search” searches for an image that is the same as or similar to the image input information transmitted from the user terminal 2. Indicates to do. The image search here is a search for a product image (product information) in the product information storage unit 112. “<Output> Product Image” indicates that a matched product image is output as a result of the search. The record of “ID = 4” is obtained when image input information is received, and the analysis result of the image input information is one product type identifier (for example, “B”). The response information is a product image that is the same as or similar to the image input information by searching the product information storage unit 112 using the received image input information in addition to the sentence “There is a good B”. This indicates that a product image of a product whose product type identifier is “B” is acquired and output.

  In such a situation, it is assumed that the user inputs a dialog application activation instruction to the user terminal 2. In addition, this dialogue application is a dialogue application that can propose a product to the user. This dialogue application is a dialogue application that plays the role of a virtual store clerk.

  Next, the terminal reception unit 22 of the user terminal 2 receives an instruction to start a dialog application. Next, the terminal transmission unit 24 transmits a dialog start instruction corresponding to the activation instruction to the dialog device 1. Note that the activation instruction is, for example, an instruction to an icon of a dialog application on the user terminal 2.

  Next, the receiving unit 12 of the dialogue apparatus 1 receives a dialogue start instruction. Then, the response acquisition unit 1311 of the response acquisition unit 131 acquires response information corresponding to the dialog start instruction from the dialog information management table of the dialog information storage unit 111. That is, the response acquisition unit 1311 acquires response information “What kind of product is the customer looking for?”.

  Next, the transmission unit 14 transmits the response information acquired by the response acquisition unit 131 to the user terminal 2 that has transmitted the input information.

  Next, the terminal receiving unit 25 of the user terminal 2 receives the response information. Then, the terminal output unit 26 outputs the response information. Such an output is 601 in FIG.

  Next, it is assumed that the user inputs an image of clothes into the dialogue application. Such an output is 602 in FIG. Note that the input of the image 602 to the application is the user's intention that he wants clothes like this image.

  Then, the terminal transmission unit 24 transmits the image input information 602 to the dialogue apparatus 1.

  Next, the receiving unit 12 of the interactive apparatus 1 receives input information from the user terminal 2. Then, the response acquisition unit 131 determines that the received input information is image input information.

  Next, the response acquisition unit 131 performs image analysis of the received image input information, and specifies that the type of the object in the image input information is “clothes”. Then, the image input information transmission unit 1313 transmits the received image input information to the image analysis device 3. Here, the response acquisition unit 131 transmits the image input information to the image analysis device 3 or causes the image analysis unit 1312 to perform image analysis according to the type of the object in the image input information. To do. Here, when the object type in the image input information is clothes, the image input information is transmitted to the image analysis device 3, and in other cases, the image input information is analyzed by the image analysis means 1312. .

  Next, the analysis receiving unit 31 of the image analysis device 3 receives the image input information. Then, the image analysis means 1312 of the image analysis device 3 analyzes the image input information and acquires three product type identifiers “coat”, “tops”, and “skirt”. That is, the image analysis unit 1312 analyzes the image input information 602 and determines that the product type identifier is “coat”. In addition to the product type identifier “coat”, the image analysis unit 1312 also acquires product type identifiers “tops” and “skirt” that match “coat”. Then, the analysis transmission unit 32 transmits the three product type identifiers “coat”, “tops”, and “skirt” to the dialogue apparatus 1. Note that the image analysis method of the image analysis means 1312 does not matter.

  Next, the analysis result receiving unit 1314 of the dialogue apparatus 1 receives the analysis result of the image input information from the image analysis apparatus 3.

  Next, since the analysis result of the image input information is the three product type identifiers, the response acquisition unit 1311 determines that it matches the record of “ID = 2” in FIG. Then, the response acquisition unit 1311 acquires the sentence “Are you looking for the category?” From the response information of the record of “ID = 2” in FIG. 5 and uses each of the three product type identifiers as menu items. Configure the menu. In addition, the sentence “Are you looking for the category?” And the corresponding menu are the response information.

  Next, the transmission unit 14 transmits the response information acquired by the response acquisition unit 131 to the user terminal 2 that has transmitted the image input information 602.

  Next, the terminal receiving unit 25 of the user terminal 2 receives the response information. Then, the terminal output unit 26 outputs the response information. An example of such output is 701 in FIG.

  Next, it is assumed that the user selects the menu item “coat” from the menu 701 in FIG. Here, the user may instruct the menu item “coat” or may input the character string “coat”.

  Then, the terminal reception unit 22 of the user terminal 2 receives the product type identifier “coat”. Next, the terminal transmission unit 24 transmits the product type identifier “coat” to the dialogue apparatus 1. The product type identifier “coat” is an example of input information. The information to be transmitted is preferably information indicating that it is a product type identifier (for example, <product type identifier>) and “coat”, but it may be only the character string “coat”.

  Next, the receiving unit 12 receives the input information (product type identifier “coat”). Then, the response acquisition unit 131 determines that the input information (product type identifier “coat”) matches “ID = 3” in FIG. Then, the response acquisition unit 131 applies the input information to the response information of “ID = 3” to form the sentence “It is a coat”. Further, the image analysis unit 1312 transmits the product type identifier “coat” to the image analysis device 3.

  Next, the analysis receiving unit 31 of the image analysis apparatus 3 receives the product type identifier “coat”. Next, the image analysis unit 1312 stores a product image corresponding to the product type identifier “coat”, which is the same as or similar to the image 602 (similarity is greater than or equal to or greater than the threshold), in the product analysis information of the image analysis apparatus 3. Obtained from the unit 112. Here, it is assumed that the image analysis unit 1312 has acquired three product images including the same image. Next, the analysis transmission unit 32 transmits the three product images to the dialogue apparatus 1.

  Next, the analysis result receiving unit 1314 of the dialogue apparatus 1 receives three product images.

  Then, the transmission unit 14 transmits the sentence “It is a coat” and three product images to the user terminal 2. The sentence “It is a coat” and the three product images are response information.

  Next, the terminal receiving unit 25 of the user terminal 2 receives the response information. The terminal output unit 26 outputs the response information. In this output example, in FIG. 8, reference numeral 801 denotes input information (product type identifier “coat”). Reference numeral 802 denotes a sentence “Coat” in the response information, and reference numeral 803 denotes three product images in the response information. Note that the three product images in FIG. 8 are selectable. That is, each of the three product images can accept a selection instruction.

  Next, it is assumed that the user inputs a selection for an image 804 among the three product images 803 in FIG.

  Then, the terminal reception unit 22 of the user terminal 2 receives a selection for the 804 image. And the terminal process part 23 comprises the selection instruction | indication corresponding to the said selection. It is assumed that this selection instruction has an image identifier for specifying 804 images. Next, the terminal transmission unit 24 transmits the selection instruction to the interactive apparatus 1.

  Next, the receiving unit 12 of the interactive apparatus 1 receives the selection instruction.

  Next, the purchase processing unit 132 acquires, from the product information storage unit 112, the URL of the web page for product purchase that is paired with the image identifier included in the selection instruction. Note that the product information in the product information storage unit 112 includes a product image, an image identifier, a URL of a web page for product purchase, and the like.

  Then, the transmission unit 14 transmits the URL of the web page or the web page to the user terminal 2.

  Next, the terminal receiving unit 25 of the user terminal 2 receives the URL of the web page or the web page, and the terminal output unit 26 outputs, for example, a screen for product purchase shown in FIG.

  Thereafter, the user can check the details of the product from the user terminal 2 or proceed with the purchase of the product.

  As described above, according to the present embodiment, a dialog using an image input by a user can be performed.

  Further, according to the present embodiment, an appropriate dialogue using an image and a character string input by the user can be performed. That is, a flexible dialogue using both a character string (sentence) that easily conveys the user's intention and an image that is intuitive and easy to understand and generally has a large amount of information becomes possible.

  Moreover, according to this Embodiment, merchandise information can be proposed to a user using the image which the user input. In such a case, the interactive device 1 plays the role of a virtual store clerk.

  Moreover, according to this Embodiment, the user can purchase goods, interacting.

  Furthermore, according to the present embodiment, it is possible to separate the dialogue apparatus from the image analysis apparatus that analyzes the image. As a result, for example, system maintainability can be improved. Further, by selecting an image analysis device (an engine may be used) according to the image input information, an image analysis process suitable for the image input information can be performed, and the accuracy of the image analysis is improved.

  In the present embodiment, the response information 701 in FIG. 7 may be the response information 1001 in FIG. That is, the response acquisition unit 1311 may acquire the response information 1001 instead of the response information 701. The response acquisition unit 1311 may acquire product type identifiers of “coat”, “tops”, and “skirt” by image recognition of the image 602, and may also acquire position information indicating the position of each product. In such a case, the response acquisition unit 1311 arranges selectable buttons “coat”, “tops”, and “skirt” at the position indicated by the position information on the image 602 to configure the response information 1001. Then, the transmission unit 14 transmits the response information 1001 to the user terminal 2 that has transmitted the image input information 602. Next, the terminal receiving unit 25 of the user terminal 2 receives the response information 1001. And the terminal output part 26 outputs the response information 1001 (refer FIG. 10). Note that a process for recognizing an image and acquiring a product type identifier of a product in the image together with information indicating the product region is possible with a known technique, and thus detailed description is omitted.

  Further, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. In addition, the software which implement | achieves the interactive apparatus 1 in this Embodiment is the following programs, for example. In other words, this program obtains response information using the analysis result obtained from the computer, a receiving unit that receives image input information that is an image transmitted from a user terminal, and the analysis result of the image input information. A response acquisition unit that performs the function and a transmission unit that transmits the response information to the user terminal.

  FIG. 10 shows the external appearance of a computer that executes the programs described in this specification and realizes the interactive devices and the like of the various embodiments described above. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 10 is an overview diagram of the computer system 300, and FIG. 11 is a block diagram of the system 300.

  In FIG. 10, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

  In FIG. 11, in addition to the CD-ROM drive 3012, the computer 301 includes an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012, a ROM 3015 for storing programs such as a bootup program, and an MPU 3013. It includes a RAM 3016 for temporarily storing application program instructions and providing a temporary storage space, and a hard disk 3017 for storing application programs, system programs, and data. Although not shown here, the computer 301 may further include a network card that provides connection to a LAN.

  A program that causes the computer system 300 to execute the functions of the interactive device 1 and the like of the above-described embodiment may be stored in the CD-ROM 3101, inserted into the CD-ROM drive 3012, and further transferred to the hard disk 3017. Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored in the hard disk 3017. The program is loaded into the RAM 3016 at the time of execution. The program may be loaded directly from the CD-ROM 3101 or the network.

  The program does not necessarily include an operating system (OS), a third party program, or the like that causes the computer 301 to execute the functions of the interactive apparatus 1 according to the above-described embodiment. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 300 operates is well known and will not be described in detail.

  In the above program, in the step of transmitting information, the step of receiving information, etc., processing performed by hardware, for example, processing performed by a modem or an interface card in the transmission step (only performed by hardware) Processing) is not included.

  Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  Further, in each of the above embodiments, it goes without saying that two or more communication units existing in one apparatus may be physically realized by one medium.

  In each of the above embodiments, each process may be realized by centralized processing by a single device, or may be realized by distributed processing by a plurality of devices.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the dialogue apparatus according to the present invention has an effect that a dialogue using an image input by a user can be performed, and is useful as a dialogue robot or the like that proposes product information.

DESCRIPTION OF SYMBOLS 1 Dialogue device 2 User terminal 3 Image analysis device 11 Storage unit 12 Reception unit 13 Processing unit 14 Transmission unit 21 Terminal storage unit 22 Terminal reception unit 23 Terminal processing unit 24 Terminal transmission unit 25 Terminal reception unit 26 Terminal output unit 31 Analysis reception unit 32 Analysis transmission unit 111 Dialog information storage unit 112 Product information storage unit 131 Response acquisition unit 132 Purchase processing unit 1311 Response acquisition unit 1312 Image analysis unit 1313 Image input information transmission unit 1314 Analysis result reception unit

Claims (7)

A receiving unit that receives image input information that is an image transmitted from the user terminal;
A response acquisition unit that acquires an analysis result of the image input information and acquires response information using the analysis result;
A dialogue apparatus comprising: a transmission unit that transmits the response information to the user terminal. The receiver is
Receives character string input information that is a character string sent from the user terminal,
The response acquisition unit
The interactive apparatus according to claim 1, wherein the process of acquiring response information differs depending on whether image input information is received or character string input information is received. A product information storage unit that stores one or more pieces of product information having a product image that is an image of the product and one or more attribute values of the product;
The response acquisition unit
Using the analysis result, obtain one or more product images from the product information storage unit,
The transmitter is
The interactive apparatus according to claim 1, wherein response information that is one or more product images acquired by the response acquisition unit is transmitted to the user terminal. The receiver is
Receiving a selection instruction indicating selection of one of the one or more product images transmitted by the transmission unit;
The interactive apparatus according to claim 3, further comprising a purchase processing unit that performs processing for purchasing a product corresponding to the selection instruction. The response acquisition unit
Image input information transmitting means for transmitting the image input information to an image analysis device;
The interactive apparatus according to claim 1, further comprising an analysis result receiving unit configured to receive an analysis result of the image input information from the image analysis apparatus. A dialogue method realized by a reception unit, a response acquisition unit, and a transmission unit,
A receiving step in which the receiving unit receives image input information that is an image transmitted from a user terminal;
The response acquisition unit acquires an analysis result of the image input information, and uses the analysis result to acquire response information; and
And a transmission step in which the transmission unit transmits the response information to the user terminal. Computer
A receiving unit that receives image input information that is an image transmitted from the user terminal;
A response acquisition unit that acquires an analysis result of the image input information and acquires response information using the analysis result;
The program for functioning as a transmission part which transmits the said response information to the said user terminal.