EP4250213A1 - Information processing device, information processing system, information processing method, and program - Google Patents

Information processing device, information processing system, information processing method, and program Download PDF

Info

Publication number
EP4250213A1
EP4250213A1 EP21894735.6A EP21894735A EP4250213A1 EP 4250213 A1 EP4250213 A1 EP 4250213A1 EP 21894735 A EP21894735 A EP 21894735A EP 4250213 A1 EP4250213 A1 EP 4250213A1
Authority
EP
European Patent Office
Prior art keywords
information
subject
inquiry
response
server apparatus
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP21894735.6A
Other languages
German (de)
French (fr)
Other versions
EP4250213A4 (en
Inventor
Tsukasa Takenaka
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
At Robotics Inc
Original Assignee
At Robotics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by At Robotics Inc filed Critical At Robotics Inc
Publication of EP4250213A1 publication Critical patent/EP4250213A1/en
Publication of EP4250213A4 publication Critical patent/EP4250213A4/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33Querying
    • G06F16/332Query formulation
    • G06F16/3329Natural language query formulation or dialogue systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/01Customer relationship services
    • G06Q30/015Providing customer assistance, e.g. assisting a customer within a business location or via helpdesk
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33Querying
    • G06F16/338Presentation of query results
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/08Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00Payment architectures, schemes or protocols
    • G06Q20/08Payment architectures
    • G06Q20/20Point-of-sale [POS] network systems
    • G06Q20/208Input by product or record sensing, e.g. weighing or scanner processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00Payment architectures, schemes or protocols
    • G06Q20/30Payment architectures, schemes or protocols characterised by the use of specific devices or networks
    • G06Q20/308Payment architectures, schemes or protocols characterised by the use of specific devices or networks using the Internet of Things
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00Payment architectures, schemes or protocols
    • G06Q20/30Payment architectures, schemes or protocols characterised by the use of specific devices or networks
    • G06Q20/32Payment architectures, schemes or protocols characterised by the use of specific devices or networks using wireless devices
    • G06Q20/322Aspects of commerce using mobile devices [M-devices]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0623Item investigation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0631Item recommendations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0639Item locations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0641Shopping interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/60Type of objects
    • G06V20/68Food, e.g. fruit or vegetables
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P80/00Climate change mitigation technologies for sector-wide applications
    • Y02P80/10Efficient use of energy, e.g. using compressed air or pressurized fluid as energy carrier

Definitions

  • the present invention relates to an information processing apparatus, an information processing system, an information processing method, and a program.
  • Patent Document 1 discloses a plurality of lighting apparatuses connected to the Internet.
  • an information processing apparatus comprises a controller.
  • the controller is configured to read information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject and connection information on a server apparatus.
  • the controller is configured to connect to the server apparatus based on the connection information.
  • inquiry information on inquiry for the subject is input by a user
  • the controller is configured to transmit the inquiry information and the subject information to the server apparatus.
  • the controller is configured to receive from the server apparatus response information generated in the server apparatus based on the inquiry information and the subject information. Further, the controller is configured to output the response information as a response from the subject.
  • a program for realizing a software in the present embodiment may be provided as a non-transitory computer readable medium that can be read by a computer or may be provided for download from an external server or may be provided so that the program can be activated on an external computer to realize functions thereof on a client terminal (so-called cloud computing).
  • the "unit" may include, for instance, a combination of hardware resources implemented by a circuit in a broad sense and information processing of software that can be concretely realized by these hardware resources. Further, various information is performed in the present embodiment, and the information can be represented by, for instance, physical values of signal values representing voltage and current, high and low signal values as a set of binary bits consisting of 0 or 1, or quantum superposition (so-called qubits), and communication/calculation can be performed on a circuit in a broad sense.
  • qubits quantum superposition
  • the circuit in a broad sense is a circuit realized by combining at least an appropriate number of a circuit, a circuitry, a processor, a memory, or the like.
  • a circuit includes application specific integrated circuit (ASIC), programmable logic device (e.g., simple programmable logic device (SPLD), complex programmable logic device (CPLD), field programmable gate array (FPGA)), or the like.
  • ASIC application specific integrated circuit
  • SPLD simple programmable logic device
  • CPLD complex programmable logic device
  • FPGA field programmable gate array
  • FIG. 1 is a diagram showing an example of a system configuration of an information processing system 1000.
  • the information processing system 1000 comprises, as a system configuration, a server apparatus 100, a client apparatus 110, and a subject 130 to which a talk mark 120 is attached.
  • the server apparatus 100 and the client apparatus 110 are communicatively connected via a network 150.
  • the client apparatus 110 is an example of an information processing apparatus.
  • the client apparatus 110 is configured to image the talk mark 120 attached to the subject 130 based on user operation.
  • the client apparatus 110 is configured to read information from a captured image including the talk mark 120 attached to the subject 130.
  • the information read by the client apparatus 110 includes information on the subject 130 and connection information on the server apparatus 100.
  • the client apparatus 110 is configured to connect to the server apparatus 100 based on the connection information.
  • the client apparatus 110 is configured to display an icon indicating the subject 130 based on the subject information.
  • a user conduct inquiry for the subject 130 with respect to the displayed icon.
  • inquiry information on inquiry for the subject 130 is input by a user, the client apparatus 110 is configured to transmit the inquiry information on inquiry for the subject 130 and the subject information to the server apparatus 100.
  • the client apparatus 110 is configured to receive from the server apparatus 100 response information generated in the server apparatus 100 based on the inquiry information and the subject information. Further, the client apparatus 110 is configured to output the response information as a response from the subject 130.
  • FIG. 2 is a diagram showing an example of a hardware configuration of the client apparatus 110.
  • the client apparatus 110 comprises, as a hardware configuration, a controller 201, a storage unit 202, an imaging unit 203, an input/output unit 204, an voice input/output unit 205, and a communication unit 206.
  • the controller 201 is configured to control entire client apparatus 110.
  • the storage unit 202 is configured to store program and data or the like used by the controller 201 in executing processing based on the program. Function of the client apparatus 110 is implemented by the controller 201 executing processing based on the program stored in the storage unit 202.
  • the imaging unit 203 is configured to image a subject. In the present embodiment, the imaging unit 203 is configured to image the talk mark 120 attached to the subject 130.
  • the talk mark is an example of a predetermined mark.
  • the input/output unit 204 is configured to input information to the controller 201 in response to user operation and display information in response to information processing of the controller 201.
  • the voice input/output unit 205 is configured to input voice of a user or output voice according to control of the controller 201.
  • the communication unit 206 is configured to connect the client apparatus 110 to the network 150 and manage communication with other apparatus.
  • the storage unit 202 is an example of a storage medium.
  • FIG. 3 is a diagram showing an example of a hardware configuration of the server apparatus 100.
  • the server apparatus 100 comprises, as a hardware configuration, a controller 301, a storage unit 302, and a communication unit 303.
  • the controller 301 is configured to control entire server apparatus 100.
  • the storage unit 302 is configured to store program and data or the like used by the controller 301 in executing processing based on the program.
  • Function of the server apparatus 100 is implemented by the controller 301 executing processing based on the program stored in the storage unit 302.
  • the communication unit 303 is configured to connect the server apparatus 100 to the network 150 and manage communication with other apparatus.
  • FIG. 4 is a sequence diagram showing an example of information processing of an information processing system 1000.
  • the controller 201 images the talk mark 120 attached to the subject 130 based on user operation.
  • food especially a beverage in a plastic bottle will be described as an example of the subject 130.
  • the controller 201 analyzes a captured image including the talk mark 120 attached to the subject 130 and reads information from the captured image.
  • the information read by the controller 201 includes subject information on the subject 130 and connection information on the server apparatus 100.
  • the controller 201 connects with the server apparatus 100 based on the connection information.
  • the controller 201 transmits a connection request to the server apparatus 100 based on the connection information acquired in SQ401.
  • the server apparatus 100 receives the connection request from the client apparatus 110.
  • the server apparatus 100 executes processing regarding connection between the server apparatus 100 and the client apparatus 110 based on the connection request.
  • the server apparatus 100 transmits to the client apparatus 110 a connection result indicating whether the connection is successful.
  • the server apparatus 100 will be described assuming that the connection is successful and a connection result indicating that the connection is successful is transmitted to the client apparatus 110.
  • FIG. 5 is a diagram showing a concept of function of the information processing system 1000.
  • an icon 500 is an example of an object indicating a subject based on a captured image.
  • an icon is a symbol that retains shape of an original object.
  • a user inputs an inquiry for the subject 130 displayed on the input/output unit 204, more specifically, for an icon corresponding to the subject 130.
  • the inquiry is a question, a doubt, an inquiry, etc. for the subject 130 to which the talk mark 120 is attached. More specifically, when the subject 130 is food, the inquiry includes question regarding food information, inquiry regarding event information related to the food, inquiry regarding sales promotion information, etc.
  • the inquiry may include a purchase request (settlement request), etc., indicating purchase intention for the subject 130.
  • the client apparatus 110 inputs the inquiry of the user by voice.
  • the controller 201 converts the input inquiry information into text and displays it as a character string on the input/output unit 204 (510 in FIG. 5 ). Further, when the inquiry information on the inquiry for the subject 130 is input by the user, in SQ405, the controller 201 transmits to the server apparatus 100 an inquiry request including the inquiry information on the inquiry for the subject 130 (e.g., the inquiry information converted to text) and the subject information.
  • the server apparatus 100 receives the inquiry request.
  • the server apparatus 100 is a sales management server apparatus or the like that conducts sales management of food.
  • the server apparatus 100 When receiving the inquiry request, the server apparatus 100 inputs the inquiry information on inquiry for the subject to a learned model identified by the subject information included in the inquiry request, and acquires response information on response to the inquiry for the subject from the learned model.
  • the learned model is a learned model that has learned a correct answer to an inquiry for an assumed subject. If the subject is tea in a plastic bottle, the inquiry includes origin of the tea, producer of the tea, health benefit of the tea, recipe for cooking using the tea, how to drink the tea in a good way, or the like.
  • the learned model is prepared in advance for each subject and stored in the storage unit 302.
  • the server apparatus 100 generates an inquiry response including the acquired response information and transmits to the client apparatus 110.
  • the controller 201 outputs response information as a response from the subject 130. More specifically, the controller 201 outputs response information from the icon 500 as a response from the subject. In the present embodiment, the controller 201 outputs the response information by voice through the voice input/output unit 205, as is output from the icon 500 as a response from the subject. The controller 201 further displays the response information as a character string on the input/output unit 204 (520 in FIG. 5 ).
  • the information processing system 1000 repeats processing from SQ405 to SQ407.
  • the user inputs purchase intention for the subject 130, or more specifically, for the icon corresponding to the subject 130, displayed on the input/output unit 204.
  • the purchase intention is, for instance, a word indicating purchase intention, such as "buy” or "purchase”.
  • the client apparatus 110 inputs the purchase intention of the user through voice.
  • the controller 201 transmits a purchase request including information indicating purchase of the subject, subject information identifying the subject 130, and identification information identifying the client apparatus 110 to the server apparatus 100.
  • the server apparatus 100 When receiving the purchase request, the server apparatus 100 specifies the user based on the identification information identifying the client apparatus 110 included in the purchase request, and specifies settlement information or the like that the user has previously registered.
  • the settlement information includes, for instance, a credit card number or a bank name and an account number of a bank account, etc.
  • the server apparatus 100 transmits a settlement request including the settlement information, a price of the subject 130 identified by subject identification information, etc. to a settlement system 400 that performs settlement based on the specified settlement information.
  • the server apparatus 100 receives a result of settlement processing from the settlement system 400.
  • the server apparatus 100 is described as receiving information indicating that the settlement is completed.
  • the server apparatus 100 and the settlement system 400 are configured separately, but the server apparatus 100 may include function of the settlement system 400.
  • the server apparatus 100 transmits the result of settlement processing to the client apparatus 110.
  • the controller 201 receives the result of settlement processing from the server apparatus 100. That is, the controller 201 receives from the server apparatus 100 the result of settlement processing processed by the server apparatus 100 based on the information indicating purchase of the subject, the subject information, and the identification information identifying the client apparatus 110.
  • the controller 201 outputs the results of settlement processing from the server apparatus 100. For instance, the controller 201 displays the result of settlement processing on the input/output unit 204 or outputs in voice through the voice input/output unit 205.
  • the server apparatus 100 transmits a signal indicating that the settlement has already been completed for the subject 130 to a gate installed at an exit of the store, in such a manner that the door does not close at the gate even if the user leaves with the subject 130.
  • the server apparatus 100 transmits to the client apparatus 110 information indicating that the settlement for the subject 130 has already been completed.
  • the client apparatus 110 may then store the information indicating that the settlement for the subject 130 has already been completed in the storage unit 202, and make the information readable at the gate. In this way, the door may not close at the gate even if the user leaves with the subject 130.
  • the user can acquire information on a subject that is not connected to the Internet while conversing with the subject. Further, if the user likes the subject, the user can conduct settlement (or payment) and take the subject home without going through a cash register, etc.
  • the user may purchase two or more subjects 130, and thus may transmit a request to the server apparatus 100 to have two or more subjects delivered to home.
  • the server apparatus 100 Upon receiving such a request, the server apparatus 100 performs the settlement described above based on information on number of subjects included in the request, and further transmits the request for delivery of the requested subject to an external server apparatus, etc. that manages delivery, as well as transmits information such as predetermined delivery address of user, information identifying the subject, the number of subjects to be delivered, or the like.
  • processing of the server apparatus 100 or the client apparatus 110 can be accelerated due to simple configuration. Similarly, power saving for the server apparatus 100 or the client apparatus 110 can be achieved.
  • a first variation will be described with food as an example of the subject 130. More specifically, as shown in FIG. 6 , which will be described later, description will be given by taking a Welsh onion as an example.
  • the controller 201 When inquiry information on a recipe using food, which is the subject 130, is input by a user through voice via the voice input/output unit 205, the controller 201 transmits an inquiry request including the inquiry information on the recipe using the food and the subject information to the server apparatus 100. Then, the server apparatus 100 receives the inquiry request.
  • the server apparatus 100 When receiving the inquiry request, the server apparatus 100 inputs inquiry information on a recipe using food to a learned model identified by subject information included in the inquiry request, and acquires response information on response to an inquiry for the recipe using the food from the learned model.
  • the response information includes information on recipe using food.
  • the server apparatus 100 generates a response to the inquiry including the acquired response information and transmits to the client apparatus 110.
  • the controller 201 receives the response to the inquiry from the server apparatus 100. That is, the controller 201 receives from the server apparatus 100 response information on the recipe generated in the server apparatus 100 based on the inquiry information and the subject information.
  • the controller 201 outputs response information as a response from the subject 130.
  • the user may input an inquiry regarding the sales location where other food is sold.
  • inquiry information on the sales location where other food used in the recipe is sold is input by the user through voice via the voice input/output unit 205, the controller 201 transmits to the server apparatus 100 an inquiry request including the inquiry information on the location where other food used in the recipe is sold, other food information on the other food, food information on the subject 130, and location information on the client apparatus 110.
  • the server apparatus 100 When receiving the inquiry request, the server apparatus 100 inputs to the learned model identified by the subject information included in the inquiry request the inquiry information regarding the location where other food used in the recipe is sold, other food information on the other food, and the location information on the client apparatus 110, and acquires, from the learned model, information on location where other food is sold within a predetermined distance from location of the client apparatus 110.
  • the server apparatus 100 transmits an inquiry response including response information to the client apparatus 110 using the acquired information as response information. That is, the controller 201 receives, from the server apparatus 100, the response information regarding the location where other food is sold, which is generated in the server apparatus based on the inquiry information, the other food information, and the location information.
  • the controller 201 displays map information around a location where the client apparatus 110 is located on the input/output unit 204, and outputs information on the location where other food is sold on the map information based on the response information.
  • FIG. 6 is a diagram showing an example of a screen displayed on the input/output unit 204 of the client apparatus 110 according to the first variation, wherein the screen displays information on other subject included in a recipe and information on a sales location where the subject is sold.
  • the subject 130 is a Welsh onion.
  • a talk mark 120 is attached to the Welsh onion, which is the subject 130.
  • An object 600 on a screen shown in FIG. 6 indicates a current location of a user. Further, the screen shown in FIG. 6 indicates that the user is at a grocery store 622 in a shopping district.
  • beef 610 and tofu 611 are displayed as other foods included in a recipe for sukiyaki, which is one of dishes using Welsh onion.
  • the screen of FIG. 6 shows that the beef 610 is sold at a butcher shop 620 and tofu 611 is sold at a tofu shop 621.
  • the controller 201 generates the screen shown in FIG. 6 based on information from the server apparatus 100 and displays on the input/output unit 204, thereby allowing the user to immediately acquire information on location where other foods used in the recipe are sold.
  • the client apparatus 110 is described as generating a screen, but the server apparatus 100 may generate a screen and transmit to the client apparatus 110. The client apparatus 110 may then display the screen received from the server apparatus 100 on the input/output unit 204, etc.
  • FIG. 7 shows an example of a system configuration of an information processing system 1000 according to a second variation.
  • a subject 130 is a machine.
  • a user can talk to the machine as if using the client apparatus 110 to acquire information such as operation status or power consumption of the machine.
  • the user who is able to understand transition of power consumption of the machine by the information obtained from the server apparatus 100, can change settings according to his/her purpose, for example, to stop operation of the machine at night.
  • the server apparatus 100 is a production management server apparatus or the like that performed management of the production facility.
  • the subject 130 to which the talk mark 120 is attached may be a logistics product.
  • an inquiry by a user includes question regarding management information, routing information, maintenance information, etc. on the logistics product.
  • the server apparatus 100 is a logistics management server apparatus or the like that manages the logistics product.
  • voice is described as an example of input.
  • the input is not limited to voice and may be input of text information such as chat.
  • the client apparatus 110 receives, from the server apparatus 100, information regarding the subject 130 as information on the subject 130 to which the talk mark 120 is attached, such as campaign information on the subject, information on a store actually using the subject, etc., and outputs the information.
  • the server apparatus 100 can identify a user who purchased the subject 130, the server apparatus 100 may transmit, to the user who purchased the subject 130, a location or URL, etc., where users who purchased the same subject can communicate with each other.
  • the present invention may be provided in each of the following aspects.
  • the information processing apparatus wherein: the controller is configured to allow an input/output unit to display an object indicating the subject based on the captured image, and output the response information from the object as a response from the subject.
  • the information processing apparatus wherein: the controller is configured to when the inquiry information is input through voice by a user, transmit the inquiry information and the subject information to the server apparatus, and output the response information through voice from the object as a response from the subject.
  • the information processing apparatus wherein: the controller is configured to when purchase intention regarding the subject is input by a user, transmit the information indicating purchase of the subject, the subject information, and identification information identifying the information processing apparatus to the server apparatus, receive from the server apparatus a result of settlement processing processed by the server apparatus based on the information indicating purchase of the subject, the subject information, and the identification information identifying the information processing apparatus, and output the result of the settlement processing.
  • the information processing apparatus comprising: an imaging unit configured to image a predetermined mark attached to a subject, wherein the controller is configured to read information from a captured image including the predetermined mark imaged by the imaging unit.
  • the information processing apparatus wherein: the subject is food.
  • the information processing apparatus wherein: the controller is configured to when inquiry information on a recipe using the food is input by a user, transmit an inquiry request including the inquiry information on the recipe using the food and the subject information to the server apparatus, receive from the server apparatus response information on the recipe generated in the server apparatus based on the inquiry information and the subject information, and output the response information as a response from the subject.
  • the information processing apparatus wherein: the controller is configured to when inquiry information on a location where other food used in the recipe is sold is input, transmit the inquiry information on the location where other food used in the recipe is sold, other food information on the other food, and location information on the information processing apparatus to the server apparatus, receive from the server apparatus response information on the location where the other food is sold, which is generated in the server apparatus based on the inquiry information, the other food information, and the location information, and output information on the location where other food is sold on map information based on the response information.
  • the information processing apparatus wherein: the subject is a machine for a production equipment.
  • An information processing system configured to: read information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject; when inquiry information on inquiry for the subject is input by a user, input the inquiry information to a learned model identified by the subject information, and acquire response information on response to the inquiry from the learned model; and output the response information as a response from the subject.
  • the information processing system further configured to: allow an input/output unit to display an object indicating the subject based on the captured image, and output the response information from the object as a response from the subject.
  • the information processing system further configured to: when the inquiry information is input through voice by a user, input the inquiry information and the subject information to a learned model, and acquire response information on a response to the inquiry from the learned model; and output the response information from the object as a response from the subject through voice.
  • An information processing method executed by an information processing apparatus comprising: a first step of reading information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject and connection information on a server apparatus; a second step of connecting to the server apparatus based on the connection information; a third step of, when inquiry information on inquiry for the subject is input by a user, transmitting the inquiry information and the subject information to the server apparatus; a fourth step of receiving from the server apparatus response information generated in the server apparatus based on the inquiry information and the subject information; and a fifth step of outputting the response information as a response from the subject.
  • An information processing method executed by an information processing system comprising: a first step of reading information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject; a second step of, when inquiry information on inquiry for the subject is input by a user, inputting the inquiry information and the subject information to a learned model, and acquiring response information on an inquiry to the subject from the learned model; and a third step of outputting the response information as a response from the subject.
  • a program configured to allow a computer to function as the controller of the information processing apparatus.
  • the present invention may be provided as a non-transitory computer readable medium that stores the program described above.
  • the subject to which the talk mark 120 is attached may be a logistics product.
  • a user can talk to the logistics product as if using the client apparatus 110 and acquire management information, routing information, maintenance information (support information), purchase information, settlement information, etc. on the logistics product.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Accounting & Taxation (AREA)
  • General Physics & Mathematics (AREA)
  • Finance (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Development Economics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Acoustics & Sound (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Human Resources & Organizations (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Tourism & Hospitality (AREA)
  • Computing Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Telephonic Communication Services (AREA)

Abstract

An aspect of the present invention provides an information processing device. This information processing device includes a controller. The controller reads information from a captured image including a prescribed mark on an object. The information includes object information pertaining to the object and connection information of a server device. Connection with the server device is made on the basis of the connection information. When a user inputs inquiry information regarding the object, the object information and the inquiry information regarding the object are sent to the server device. Response information generated by the server device on the basis of the inquiry information and the object information are received from the server device. The response information is output as a response from the object.

Description

    BACKGROUND TECHNICAL FIELD
  • The present invention relates to an information processing apparatus, an information processing system, an information processing method, and a program.
  • RELATED ART
  • With the spread of IoT (Internet of Things) technology, various apparatuses have come to be connected to the Internet. Patent Document 1 discloses a plurality of lighting apparatuses connected to the Internet.
  • PRIOR ART DOCUMENTS Patent document
  • [Patent Document 1] JP 2015-41438 A
  • SUMMARY Problems to be Solved by Invention
  • However, not all subjects have a controller as well as a communication unit and are connected to the Internet. It is also costly to allow all subjects comprising a controller and a communication unit.
  • Means for Solving Problems
  • According to an aspect of the present invention, an information processing apparatus is provided. The information processing apparatus comprises a controller. The controller is configured to read information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject and connection information on a server apparatus. The controller is configured to connect to the server apparatus based on the connection information. When inquiry information on inquiry for the subject is input by a user, the controller is configured to transmit the inquiry information and the subject information to the server apparatus. The controller is configured to receive from the server apparatus response information generated in the server apparatus based on the inquiry information and the subject information. Further, the controller is configured to output the response information as a response from the subject.
  • BRIEF DESCRIPTION OF DRAWINGS
    • FIG. 1 is a diagram showing an example of a system configuration of an information processing system.
    • FIG. 2 is a diagram showing an example of a hardware configuration of a client apparatus.
    • FIG. 3 is a diagram showing an example of a hardware configuration of a server apparatus.
    • FIG. 4 is a sequence diagram showing an example of information processing of an information processing system.
    • FIG. 5 is a diagram showing a concept of function of an information processing system.
    • FIG. 6 is a diagram showing an example of a screen displayed on an input/output unit of a client apparatus according to a first variation, wherein the screen displays information on other subject included in a recipe and information on a sales location where the subject is sold.
    • FIG. 7 is a diagram showing an example of a system configuration of an information processing system according to a second variation.
    DETAILED DESCRIPTION
  • Hereinafter, embodiment of the present invention will be described with reference to the drawings. Various features described in the embodiment below can be combined with each other.
  • A program for realizing a software in the present embodiment may be provided as a non-transitory computer readable medium that can be read by a computer or may be provided for download from an external server or may be provided so that the program can be activated on an external computer to realize functions thereof on a client terminal (so-called cloud computing).
  • In the present embodiment, the "unit" may include, for instance, a combination of hardware resources implemented by a circuit in a broad sense and information processing of software that can be concretely realized by these hardware resources. Further, various information is performed in the present embodiment, and the information can be represented by, for instance, physical values of signal values representing voltage and current, high and low signal values as a set of binary bits consisting of 0 or 1, or quantum superposition (so-called qubits), and communication/calculation can be performed on a circuit in a broad sense.
  • Further, the circuit in a broad sense is a circuit realized by combining at least an appropriate number of a circuit, a circuitry, a processor, a memory, or the like. In other words, it is a circuit includes application specific integrated circuit (ASIC), programmable logic device (e.g., simple programmable logic device (SPLD), complex programmable logic device (CPLD), field programmable gate array (FPGA)), or the like.
  • < First embodiment> 1. System configuration
  • FIG. 1 is a diagram showing an example of a system configuration of an information processing system 1000. The information processing system 1000 comprises, as a system configuration, a server apparatus 100, a client apparatus 110, and a subject 130 to which a talk mark 120 is attached. The server apparatus 100 and the client apparatus 110 are communicatively connected via a network 150. The client apparatus 110 is an example of an information processing apparatus.
  • (Overview of processing of information processing system 1000)
  • The client apparatus 110 is configured to image the talk mark 120 attached to the subject 130 based on user operation. The client apparatus 110 is configured to read information from a captured image including the talk mark 120 attached to the subject 130. Here, the information read by the client apparatus 110 includes information on the subject 130 and connection information on the server apparatus 100. The client apparatus 110 is configured to connect to the server apparatus 100 based on the connection information. The client apparatus 110 is configured to display an icon indicating the subject 130 based on the subject information. A user conduct inquiry for the subject 130 with respect to the displayed icon. When inquiry information on inquiry for the subject 130 is input by a user, the client apparatus 110 is configured to transmit the inquiry information on inquiry for the subject 130 and the subject information to the server apparatus 100. The client apparatus 110 is configured to receive from the server apparatus 100 response information generated in the server apparatus 100 based on the inquiry information and the subject information. Further, the client apparatus 110 is configured to output the response information as a response from the subject 130.
  • 2. Hardware configuration (1) Hardware configuration of client apparatus 110
  • FIG. 2 is a diagram showing an example of a hardware configuration of the client apparatus 110. The client apparatus 110 comprises, as a hardware configuration, a controller 201, a storage unit 202, an imaging unit 203, an input/output unit 204, an voice input/output unit 205, and a communication unit 206. The controller 201 is configured to control entire client apparatus 110. The storage unit 202 is configured to store program and data or the like used by the controller 201 in executing processing based on the program. Function of the client apparatus 110 is implemented by the controller 201 executing processing based on the program stored in the storage unit 202. The imaging unit 203 is configured to image a subject. In the present embodiment, the imaging unit 203 is configured to image the talk mark 120 attached to the subject 130. The talk mark is an example of a predetermined mark. The input/output unit 204 is configured to input information to the controller 201 in response to user operation and display information in response to information processing of the controller 201. The voice input/output unit 205 is configured to input voice of a user or output voice according to control of the controller 201. The communication unit 206 is configured to connect the client apparatus 110 to the network 150 and manage communication with other apparatus. The storage unit 202 is an example of a storage medium.
  • (2) Hardware configuration of server apparatus 100
  • FIG. 3 is a diagram showing an example of a hardware configuration of the server apparatus 100. The server apparatus 100 comprises, as a hardware configuration, a controller 301, a storage unit 302, and a communication unit 303. The controller 301 is configured to control entire server apparatus 100. The storage unit 302 is configured to store program and data or the like used by the controller 301 in executing processing based on the program. Function of the server apparatus 100 is implemented by the controller 301 executing processing based on the program stored in the storage unit 302. The communication unit 303 is configured to connect the server apparatus 100 to the network 150 and manage communication with other apparatus.
  • 3. information processing
  • FIG. 4 is a sequence diagram showing an example of information processing of an information processing system 1000.
  • In SQ401, the controller 201 images the talk mark 120 attached to the subject 130 based on user operation. In the present embodiment, food, especially a beverage in a plastic bottle will be described as an example of the subject 130. The controller 201 analyzes a captured image including the talk mark 120 attached to the subject 130 and reads information from the captured image. Here, the information read by the controller 201 includes subject information on the subject 130 and connection information on the server apparatus 100. The controller 201 connects with the server apparatus 100 based on the connection information.
  • In SQ402, the controller 201 transmits a connection request to the server apparatus 100 based on the connection information acquired in SQ401. The server apparatus 100 receives the connection request from the client apparatus 110. The server apparatus 100 executes processing regarding connection between the server apparatus 100 and the client apparatus 110 based on the connection request.
  • In SQ403, the server apparatus 100 transmits to the client apparatus 110 a connection result indicating whether the connection is successful. In the present embodiment, the server apparatus 100 will be described assuming that the connection is successful and a connection result indicating that the connection is successful is transmitted to the client apparatus 110.
  • In SQ404, the controller 201 displays an icon on the input/output unit 204 indicating a subject based on the captured image. FIG. 5 is a diagram showing a concept of function of the information processing system 1000. As shown in FIG. 5, an icon 500 is an example of an object indicating a subject based on a captured image. Here, an icon is a symbol that retains shape of an original object.
  • A user inputs an inquiry for the subject 130 displayed on the input/output unit 204, more specifically, for an icon corresponding to the subject 130. The inquiry is a question, a doubt, an inquiry, etc. for the subject 130 to which the talk mark 120 is attached. More specifically, when the subject 130 is food, the inquiry includes question regarding food information, inquiry regarding event information related to the food, inquiry regarding sales promotion information, etc. In addition, as described later, the inquiry may include a purchase request (settlement request), etc., indicating purchase intention for the subject 130. In the present embodiment, the client apparatus 110 inputs the inquiry of the user by voice. When the inquiry information on inquiry for the subject 130 is input through voice by the user via the voice input/output unit 205, the controller 201 converts the input inquiry information into text and displays it as a character string on the input/output unit 204 (510 in FIG. 5). Further, when the inquiry information on the inquiry for the subject 130 is input by the user, in SQ405, the controller 201 transmits to the server apparatus 100 an inquiry request including the inquiry information on the inquiry for the subject 130 (e.g., the inquiry information converted to text) and the subject information. The server apparatus 100 receives the inquiry request. In the example according to the present embodiment, the server apparatus 100 is a sales management server apparatus or the like that conducts sales management of food.
  • When receiving the inquiry request, the server apparatus 100 inputs the inquiry information on inquiry for the subject to a learned model identified by the subject information included in the inquiry request, and acquires response information on response to the inquiry for the subject from the learned model. The learned model is a learned model that has learned a correct answer to an inquiry for an assumed subject. If the subject is tea in a plastic bottle, the inquiry includes origin of the tea, producer of the tea, health benefit of the tea, recipe for cooking using the tea, how to drink the tea in a good way, or the like. Moreover, the learned model is prepared in advance for each subject and stored in the storage unit 302.
  • In SQ406, the server apparatus 100 generates an inquiry response including the acquired response information and transmits to the client apparatus 110.
  • In SQ407, the controller 201 outputs response information as a response from the subject 130. More specifically, the controller 201 outputs response information from the icon 500 as a response from the subject. In the present embodiment, the controller 201 outputs the response information by voice through the voice input/output unit 205, as is output from the icon 500 as a response from the subject. The controller 201 further displays the response information as a character string on the input/output unit 204 (520 in FIG. 5).
  • When the inquiry information is input by the user, the information processing system 1000 repeats processing from SQ405 to SQ407.
  • If a user intends to purchase the subject 130 as a result of inquiry, the user inputs purchase intention for the subject 130, or more specifically, for the icon corresponding to the subject 130, displayed on the input/output unit 204. The purchase intention is, for instance, a word indicating purchase intention, such as "buy" or "purchase". In the present embodiment, the client apparatus 110 inputs the purchase intention of the user through voice. When the purchase intention of the user for the subject 130 is input through voice via the voice input/output unit 205, in SQ408, the controller 201 transmits a purchase request including information indicating purchase of the subject, subject information identifying the subject 130, and identification information identifying the client apparatus 110 to the server apparatus 100.
  • When receiving the purchase request, the server apparatus 100 specifies the user based on the identification information identifying the client apparatus 110 included in the purchase request, and specifies settlement information or the like that the user has previously registered. The settlement information includes, for instance, a credit card number or a bank name and an account number of a bank account, etc. In SQ409, the server apparatus 100 transmits a settlement request including the settlement information, a price of the subject 130 identified by subject identification information, etc. to a settlement system 400 that performs settlement based on the specified settlement information.
  • In SQ410, the server apparatus 100 receives a result of settlement processing from the settlement system 400. In the present embodiment, the server apparatus 100 is described as receiving information indicating that the settlement is completed. In the present embodiment, the server apparatus 100 and the settlement system 400 are configured separately, but the server apparatus 100 may include function of the settlement system 400.
  • In SQ411, the server apparatus 100 transmits the result of settlement processing to the client apparatus 110. The controller 201 receives the result of settlement processing from the server apparatus 100. That is, the controller 201 receives from the server apparatus 100 the result of settlement processing processed by the server apparatus 100 based on the information indicating purchase of the subject, the subject information, and the identification information identifying the client apparatus 110.
  • In SQ412, the controller 201 outputs the results of settlement processing from the server apparatus 100. For instance, the controller 201 displays the result of settlement processing on the input/output unit 204 or outputs in voice through the voice input/output unit 205.
  • When the subject 130 is placed in a store and the settlement is completed, the server apparatus 100 transmits a signal indicating that the settlement has already been completed for the subject 130 to a gate installed at an exit of the store, in such a manner that the door does not close at the gate even if the user leaves with the subject 130. Alternatively, the server apparatus 100 transmits to the client apparatus 110 information indicating that the settlement for the subject 130 has already been completed. The client apparatus 110 may then store the information indicating that the settlement for the subject 130 has already been completed in the storage unit 202, and make the information readable at the gate. In this way, the door may not close at the gate even if the user leaves with the subject 130.
  • According to the processing in the present embodiment, the user can acquire information on a subject that is not connected to the Internet while conversing with the subject. Further, if the user likes the subject, the user can conduct settlement (or payment) and take the subject home without going through a cash register, etc.
  • If the user likes the subject 130, the user may purchase two or more subjects 130, and thus may transmit a request to the server apparatus 100 to have two or more subjects delivered to home. Upon receiving such a request, the server apparatus 100 performs the settlement described above based on information on number of subjects included in the request, and further transmits the request for delivery of the requested subject to an external server apparatus, etc. that manages delivery, as well as transmits information such as predetermined delivery address of user, information identifying the subject, the number of subjects to be delivered, or the like.
  • Further, according to the present embodiment, it is possible to provide a low-cost technology for presenting information while conversing with a subject. In addition, according to the present embodiment, processing of the server apparatus 100 or the client apparatus 110 can be accelerated due to simple configuration. Similarly, power saving for the server apparatus 100 or the client apparatus 110 can be achieved.
  • <First variation>
  • A first variation will be described with food as an example of the subject 130. More specifically, as shown in FIG. 6, which will be described later, description will be given by taking a Welsh onion as an example.
  • When inquiry information on a recipe using food, which is the subject 130, is input by a user through voice via the voice input/output unit 205, the controller 201 transmits an inquiry request including the inquiry information on the recipe using the food and the subject information to the server apparatus 100. Then, the server apparatus 100 receives the inquiry request.
  • When receiving the inquiry request, the server apparatus 100 inputs inquiry information on a recipe using food to a learned model identified by subject information included in the inquiry request, and acquires response information on response to an inquiry for the recipe using the food from the learned model. The response information includes information on recipe using food.
  • The server apparatus 100 generates a response to the inquiry including the acquired response information and transmits to the client apparatus 110. The controller 201 receives the response to the inquiry from the server apparatus 100. That is, the controller 201 receives from the server apparatus 100 response information on the recipe generated in the server apparatus 100 based on the inquiry information and the subject information.
  • The controller 201 outputs response information as a response from the subject 130.
  • When a user wants to know a sales location where other food (other food) included in the output recipe is sold, the user may input an inquiry regarding the sales location where other food is sold. When inquiry information on the sales location where other food used in the recipe is sold is input by the user through voice via the voice input/output unit 205, the controller 201 transmits to the server apparatus 100 an inquiry request including the inquiry information on the location where other food used in the recipe is sold, other food information on the other food, food information on the subject 130, and location information on the client apparatus 110.
  • When receiving the inquiry request, the server apparatus 100 inputs to the learned model identified by the subject information included in the inquiry request the inquiry information regarding the location where other food used in the recipe is sold, other food information on the other food, and the location information on the client apparatus 110, and acquires, from the learned model, information on location where other food is sold within a predetermined distance from location of the client apparatus 110. The server apparatus 100 transmits an inquiry response including response information to the client apparatus 110 using the acquired information as response information. That is, the controller 201 receives, from the server apparatus 100, the response information regarding the location where other food is sold, which is generated in the server apparatus based on the inquiry information, the other food information, and the location information.
  • The controller 201 displays map information around a location where the client apparatus 110 is located on the input/output unit 204, and outputs information on the location where other food is sold on the map information based on the response information.
  • FIG. 6 is a diagram showing an example of a screen displayed on the input/output unit 204 of the client apparatus 110 according to the first variation, wherein the screen displays information on other subject included in a recipe and information on a sales location where the subject is sold.
  • In the example shown in FIG. 6, the subject 130 is a Welsh onion. A talk mark 120 is attached to the Welsh onion, which is the subject 130. An object 600 on a screen shown in FIG. 6 indicates a current location of a user. Further, the screen shown in FIG. 6 indicates that the user is at a grocery store 622 in a shopping district. In the screen of FIG. 6, beef 610 and tofu 611 are displayed as other foods included in a recipe for sukiyaki, which is one of dishes using Welsh onion. Moreover, the screen of FIG. 6 shows that the beef 610 is sold at a butcher shop 620 and tofu 611 is sold at a tofu shop 621.
  • The controller 201 generates the screen shown in FIG. 6 based on information from the server apparatus 100 and displays on the input/output unit 204, thereby allowing the user to immediately acquire information on location where other foods used in the recipe are sold.
  • In the first variation, the client apparatus 110 is described as generating a screen, but the server apparatus 100 may generate a screen and transmit to the client apparatus 110. The client apparatus 110 may then display the screen received from the server apparatus 100 on the input/output unit 204, etc.
  • <Second variation>
  • In the first embodiment and the first variation, food is described as an example of the subject 130. However, the subject to which the talk mark 120 is attached is not limited to food. For instance, the subject may be a car, a train, etc., or a machine in a production facility installed in a factory, etc. FIG. 7 shows an example of a system configuration of an information processing system 1000 according to a second variation. In the example shown in FIG. 7, a subject 130 is a machine. For example, if the machine is attached with a talk mark 120, a user can talk to the machine as if using the client apparatus 110 to acquire information such as operation status or power consumption of the machine. The user, who is able to understand transition of power consumption of the machine by the information obtained from the server apparatus 100, can change settings according to his/her purpose, for example, to stop operation of the machine at night.
  • When the subject 130 to which the talk mark 120 is attached is a machine of production facility, an inquiry by a user includes question regarding usage information, handling information, manipulation information, operation information, maintenance information, etc. on the facility. In such an example, the server apparatus 100 is a production management server apparatus or the like that performed management of the production facility.
  • As another example, the subject 130 to which the talk mark 120 is attached may be a logistics product. When the subject 130 is a logistics product, an inquiry by a user includes question regarding management information, routing information, maintenance information, etc. on the logistics product. In such an example, the server apparatus 100 is a logistics management server apparatus or the like that manages the logistics product.
  • <Third variation>
  • In the above-described embodiment, etc., voice is described as an example of input. However, the input is not limited to voice and may be input of text information such as chat.
  • <Other variation>
  • The client apparatus 110 receives, from the server apparatus 100, information regarding the subject 130 as information on the subject 130 to which the talk mark 120 is attached, such as campaign information on the subject, information on a store actually using the subject, etc., and outputs the information.
  • In addition, since the server apparatus 100 can identify a user who purchased the subject 130, the server apparatus 100 may transmit, to the user who purchased the subject 130, a location or URL, etc., where users who purchased the same subject can communicate with each other.
  • < Appendix>
  • The present invention may be provided in each of the following aspects.
  • The information processing apparatus, wherein: the controller is configured to allow an input/output unit to display an object indicating the subject based on the captured image, and output the response information from the object as a response from the subject.
  • The information processing apparatus, wherein: the controller is configured to when the inquiry information is input through voice by a user, transmit the inquiry information and the subject information to the server apparatus, and output the response information through voice from the object as a response from the subject.
  • The information processing apparatus, wherein: the controller is configured to when purchase intention regarding the subject is input by a user, transmit the information indicating purchase of the subject, the subject information, and identification information identifying the information processing apparatus to the server apparatus, receive from the server apparatus a result of settlement processing processed by the server apparatus based on the information indicating purchase of the subject, the subject information, and the identification information identifying the information processing apparatus, and output the result of the settlement processing.
  • The information processing apparatus, comprising: an imaging unit configured to image a predetermined mark attached to a subject, wherein the controller is configured to read information from a captured image including the predetermined mark imaged by the imaging unit.
  • The information processing apparatus, wherein: the subject is food.
  • The information processing apparatus, wherein: the controller is configured to when inquiry information on a recipe using the food is input by a user, transmit an inquiry request including the inquiry information on the recipe using the food and the subject information to the server apparatus, receive from the server apparatus response information on the recipe generated in the server apparatus based on the inquiry information and the subject information, and output the response information as a response from the subject.
  • The information processing apparatus, wherein: the controller is configured to when inquiry information on a location where other food used in the recipe is sold is input, transmit the inquiry information on the location where other food used in the recipe is sold, other food information on the other food, and location information on the information processing apparatus to the server apparatus, receive from the server apparatus response information on the location where the other food is sold, which is generated in the server apparatus based on the inquiry information, the other food information, and the location information, and output information on the location where other food is sold on map information based on the response information.
  • The information processing apparatus, wherein: the subject is a machine for a production equipment.
  • An information processing system, configured to: read information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject; when inquiry information on inquiry for the subject is input by a user, input the inquiry information to a learned model identified by the subject information, and acquire response information on response to the inquiry from the learned model; and output the response information as a response from the subject.
  • The information processing system, further configured to: allow an input/output unit to display an object indicating the subject based on the captured image, and output the response information from the object as a response from the subject.
  • The information processing system, further configured to: when the inquiry information is input through voice by a user, input the inquiry information and the subject information to a learned model, and acquire response information on a response to the inquiry from the learned model; and output the response information from the object as a response from the subject through voice.
  • An information processing method executed by an information processing apparatus, comprising: a first step of reading information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject and connection information on a server apparatus; a second step of connecting to the server apparatus based on the connection information; a third step of, when inquiry information on inquiry for the subject is input by a user, transmitting the inquiry information and the subject information to the server apparatus; a fourth step of receiving from the server apparatus response information generated in the server apparatus based on the inquiry information and the subject information; and a fifth step of outputting the response information as a response from the subject.
  • An information processing method executed by an information processing system, comprising: a first step of reading information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject; a second step of, when inquiry information on inquiry for the subject is input by a user, inputting the inquiry information and the subject information to a learned model, and acquiring response information on an inquiry to the subject from the learned model; and a third step of outputting the response information as a response from the subject.
  • A program configured to allow a computer to function as the controller of the information processing apparatus.
  • Of course, the present invention is not limited to the above aspects.
  • For instance, the present invention may be provided as a non-transitory computer readable medium that stores the program described above.
  • Moreover, the above-described embodiment and variations may be implemented in any combination.
  • The subject to which the talk mark 120 is attached may be a logistics product. For example, if the talk mark 120 is attached to a logistics product, a user can talk to the logistics product as if using the client apparatus 110 and acquire management information, routing information, maintenance information (support information), purchase information, settlement information, etc. on the logistics product.
  • Finally, various embodiments of the present invention have been described, but these are presented as examples and are not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the abstract of the invention. The embodiment and its modifications are included in the scope and abstract of the invention and are included in the scope of the invention described in the claims and the equivalent scope thereof.
  • REFERENCE SIGNS LIST
    • 100: Server apparatus
    • 110: Client apparatus
    • 120: Talk mark
    • 130: Subject
    • 150: Network
    • 201: Controller
    • 202: Storage unit
    • 203: Imaging unit
    • 204: Input/output unit
    • 205: Audio input/output unit
    • 206: Communication unit
    • 301: Controller
    • 302: Storage unit
    • 303: Communication unit
    • 400: Settlement system
    • 500: Icon
    • 600: Object
    • 610: Beef
    • 611: Tofu
    • 622: Grocery store
    • 1000: Information processing system

Claims (15)

  1. An information processing apparatus, comprising:
    a controller configured to
    read information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject and connection information on a server apparatus,
    connect to the server apparatus based on the connection information,
    when inquiry information on inquiry for the subject is input by a user, transmit the inquiry information and the subject information to the server apparatus,
    receive from the server apparatus response information generated in the server apparatus based on the inquiry information and the subject information, and
    output the response information as a response from the subject.
  2. The information processing apparatus according to claim 1, wherein:
    the controller is configured to
    allow an input/output unit to display an object indicating the subject based on the captured image, and
    output the response information from the object as a response from the subject.
  3. The information processing apparatus according to claim 2, wherein:
    the controller is configured to
    when the inquiry information is input through voice by a user, transmit the inquiry information and the subject information to the server apparatus, and
    output the response information through voice from the object as a response from the subject.
  4. The information processing apparatus according to any one of claims 1 to 3, wherein:
    the controller is configured to
    when purchase intention regarding the subject is input by a user, transmit the information indicating purchase of the subject, the subject information, and identification information identifying the information processing apparatus to the server apparatus,
    receive from the server apparatus a result of settlement processing processed by the server apparatus based on the information indicating purchase of the subject, the subject information, and the identification information identifying the information processing apparatus, and
    output the result of the settlement processing.
  5. The information processing apparatus according to any one of claims 1 to 4, comprising:
    an imaging unit configured to image a predetermined mark attached to a subject, wherein
    the controller is configured to read information from a captured image including the predetermined mark imaged by the imaging unit.
  6. The information processing apparatus according to any one of claims 1 to 5, wherein:
    the subject is food.
  7. The information processing apparatus according to claim 6, wherein:
    the controller is configured to
    when inquiry information on a recipe using the food is input by a user, transmit an inquiry request including the inquiry information on the recipe using the food and the subject information to the server apparatus,
    receive from the server apparatus response information on the recipe generated in the server apparatus based on the inquiry information and the subject information, and
    output the response information as a response from the subject.
  8. The information processing apparatus according to claim 7, wherein:
    the controller is configured to
    when inquiry information on a location where other food used in the recipe is sold is input, transmit the inquiry information on the location where other food used in the recipe is sold, other food information on the other food, and location information on the information processing apparatus to the server apparatus,
    receive from the server apparatus response information on the location where the other food is sold, which is generated in the server apparatus based on the inquiry information, the other food information, and the location information, and
    output information on the location where other food is sold on map information based on the response information.
  9. The information processing apparatus according to any one of claims 1 to 5, wherein:
    the subject is a machine for a production equipment.
  10. An information processing system, configured to:
    read information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject;
    when inquiry information on inquiry for the subject is input by a user, input the inquiry information to a learned model identified by the subject information, and acquire response information on response to the inquiry from the learned model; and
    output the response information as a response from the subject.
  11. The information processing system according to claim 10, further configured to:
    allow an input/output unit to display an object indicating the subject based on the captured image, and
    output the response information from the object as a response from the subject.
  12. The information processing system according to claim 11, further configured to:
    when the inquiry information is input through voice by a user, input the inquiry information and the subject information to a learned model, and acquire response information on a response to the inquiry from the learned model; and
    output the response information from the object as a response from the subject through voice.
  13. An information processing method executed by an information processing apparatus, comprising:
    a first step of reading information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject and connection information on a server apparatus;
    a second step of connecting to the server apparatus based on the connection information;
    a third step of, when inquiry information on inquiry for the subject is input by a user, transmitting the inquiry information and the subject information to the server apparatus;
    a fourth step of receiving from the server apparatus response information generated in the server apparatus based on the inquiry information and the subject information; and
    a fifth step of outputting the response information as a response from the subject.
  14. An information processing method executed by an information processing system, comprising:
    a first step of reading information from a captured image including a predetermined mark attached to a subject, wherein the information includes subject information on the subject;
    a second step of, when inquiry information on inquiry for the subject is input by a user, inputting the inquiry information and the subject information to a learned model, and acquiring response information on an inquiry to the subject from the learned model; and
    a third step of outputting the response information as a response from the subject.
  15. A program configured to allow a computer to function as the controller of the information processing apparatus according to any one of claims 1 to 9.
EP21894735.6A 2020-11-20 2021-11-18 Information processing device, information processing system, information processing method, and program Pending EP4250213A4 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020193545A JP7175025B2 (en) 2020-11-20 2020-11-20 Information processing device, information processing system, information processing method and program
PCT/JP2021/042500 WO2022107863A1 (en) 2020-11-20 2021-11-18 Information processing device, information processing system, information processing method, and program

Publications (2)

Publication Number Publication Date
EP4250213A1 true EP4250213A1 (en) 2023-09-27
EP4250213A4 EP4250213A4 (en) 2024-01-24

Family

ID=81709079

Family Applications (1)

Application Number Title Priority Date Filing Date
EP21894735.6A Pending EP4250213A4 (en) 2020-11-20 2021-11-18 Information processing device, information processing system, information processing method, and program

Country Status (5)

Country Link
US (1) US20240020325A1 (en)
EP (1) EP4250213A4 (en)
JP (2) JP7175025B2 (en)
CN (1) CN116615741A (en)
WO (1) WO2022107863A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7175025B2 (en) 2020-11-20 2022-11-18 株式会社アットロボティクス Information processing device, information processing system, information processing method and program

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002208072A (en) 2001-01-09 2002-07-26 Ricoh Co Ltd Customer service system
JP2003157477A (en) * 2001-11-19 2003-05-30 Hitachi Ltd Tag management server and information acquiring/using system using it
JP3609398B1 (en) 2003-11-11 2005-01-12 株式会社青山キャピタル Product information provision system
JP2005327077A (en) * 2004-05-14 2005-11-24 Sony Corp Shopping support system, shopping support method and portable terminal device
US20090289764A1 (en) * 2008-05-21 2009-11-26 Fu-Sheng Chiu Instant promotional media using an identification tag
JP2011113417A (en) * 2009-11-27 2011-06-09 Amitsukusu:Kk Rental property management device, communication terminal, rental property management method, rental property management program, and rental property management system
EP2510484A4 (en) * 2009-12-13 2015-01-07 Intuit Inc Systems and methods for purchasing products from a retail establishment using a mobile device
JP5696119B2 (en) * 2012-10-04 2015-04-08 ヤフー株式会社 Advertisement distribution apparatus, advertisement distribution method, and advertisement distribution program
CN103902598B (en) 2012-12-27 2018-11-09 北京京东尚科信息技术有限公司 Mobile terminal and commodity information query method
JP6207848B2 (en) 2013-03-07 2017-10-04 三井造船株式会社 Bioethanol recovery method and recovery system
JP2015041438A (en) 2013-08-21 2015-03-02 アプリックスIpホールディングス株式会社 Illumination control system, terminal, and illumination control method for the same
WO2015041985A1 (en) * 2013-09-20 2015-03-26 Flashback Survey, Inc. Using scanable codes to obtain a service
CN105931103A (en) 2016-04-21 2016-09-07 聂迪 Scanning-based shopping method and shopping system
JP6113896B1 (en) * 2016-07-04 2017-04-12 株式会社フューチャーアプリケーション Sales support system, sales support device, and sales support method
US9864949B1 (en) 2016-10-31 2018-01-09 International Business Machines Corporation Cognition model-based product assist
KR20180083228A (en) * 2017-01-12 2018-07-20 삼성전자주식회사 Electronic device and online buy method in the electronic device
JP6604612B2 (en) 2018-04-06 2019-11-13 Zホールディングス株式会社 Calculation device, calculation method, and calculation program
US11170161B2 (en) * 2018-05-11 2021-11-09 Citrix Systems, Inc. Generating conversation models from documents
JP6495514B1 (en) * 2018-05-29 2019-04-03 クックパッド株式会社 Information processing system, information processing apparatus, information processing method, and program
JP7175025B2 (en) 2020-11-20 2022-11-18 株式会社アットロボティクス Information processing device, information processing system, information processing method and program

Also Published As

Publication number Publication date
JP7175025B2 (en) 2022-11-18
US20240020325A1 (en) 2024-01-18
JP2022082154A (en) 2022-06-01
WO2022107863A1 (en) 2022-05-27
CN116615741A (en) 2023-08-18
JP2023011785A (en) 2023-01-24
JP7535326B2 (en) 2024-08-16
EP4250213A4 (en) 2024-01-24

Similar Documents

Publication Publication Date Title
US10719821B2 (en) Merchandise sales data processing apparatus, and program therefor
CA2900605C (en) Methods and systems for providing payment credentials
US20180018739A1 (en) Electronic receipt system, commodity sales data processing apparatus, electronic receipt management server and method
JP2015232756A (en) Sales data processor and program
US20150356835A1 (en) Point-of-sale terminal that sets a special display flag in electronic receipt data
US20180293641A1 (en) Order Processing System, Order Processing Method
CN110580641A (en) method and apparatus for exchanging articles
EP4250213A1 (en) Information processing device, information processing system, information processing method, and program
WO2016068058A1 (en) Membership management device, membership management method, and recording medium
JP2018055564A (en) Information processing equipment, point providing method, and program
JP2020113005A (en) Information processing method, information processing device, and program
JPWO2016136077A1 (en) Information processing apparatus, information processing system, program, and control method
US20220311527A1 (en) Broadcasting contextual information through modification of audio and video interfaces
KR20130033501A (en) System and method for providing mobile membership service
JP6888243B2 (en) Information processing equipment, information processing methods, and programs
CN106875238B (en) Invoice information generation method and device
CN110751455A (en) Method and device for processing joint name service
JP2020184209A (en) Information processing method, information processing device, and information processing program
JP7355458B2 (en) Information processing device, information processing method, and program
JP7521175B2 (en) Customer information processing device, customer information processing method, and program
WO2022210336A2 (en) Broadcasting contextual information through modification of audio and video interfaces
CN107195143B (en) Configuration information transmission method and system
KR20130126013A (en) Method and apparatus for managing shopping information
JP2021117505A (en) Information processing system, information processing method and computer program
JP2024015730A (en) Information distribution support device, information distribution support method, program and recording medium

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20230602

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Free format text: PREVIOUS MAIN CLASS: G06Q0030020000

Ipc: G06Q0020200000

A4 Supplementary search report drawn up and despatched

Effective date: 20240103

RIC1 Information provided on ipc code assigned before grant

Ipc: G06Q 30/015 20230101ALI20231220BHEP

Ipc: G06Q 30/0601 20230101ALI20231220BHEP

Ipc: G06Q 10/08 20230101ALI20231220BHEP

Ipc: G06Q 30/06 20230101ALI20231220BHEP

Ipc: G06Q 30/02 20230101ALI20231220BHEP

Ipc: G06Q 20/32 20120101ALI20231220BHEP

Ipc: G06Q 20/30 20120101ALI20231220BHEP

Ipc: G06Q 20/20 20120101AFI20231220BHEP

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)