JP6240291B1 - Determination apparatus, determination method, and determination program - Google Patents

Abstract

To appropriately determine a processing mode related to goods or services. A determination apparatus according to the present application includes a first reception unit, a second reception unit, and a determination unit. The first reception unit is first sensor information detected by the first sensor, and receives first sensor information related to a user designation for a product or service displayed in a predetermined area. The second reception unit is second sensor information detected by the second sensor, and receives second sensor information related to a user's intention for a product or service specified based on the first sensor information. A determination part determines the process aspect regarding the purchase object which is goods or services specified based on 1st sensor information based on 2nd sensor information. [Selection] Figure 2

Description

  The present invention relates to a determination device, a determination method, and a determination program.

  In recent years, various technologies related to electronic commerce such as online shopping have been provided. For example, a technique for performing processing related to purchase based on a procedure word recognized by a user terminal is provided.

JP 2003-44756 A JP 2014-186214 A

  However, it is difficult to appropriately determine the processing mode related to the product or service in the above-described conventional technology. For example, when a procedure word is used, it is necessary to set the procedure word in advance, and it is difficult to appropriately determine a processing mode related to goods or services.

  The present application has been made in view of the above, and an object thereof is to provide a determination device, a determination method, and a determination program for appropriately determining a processing mode related to a product or a service.

  The determination device according to the present application is first sensor information that is detected by a first sensor and that receives first sensor information related to a user's designation for a product or service displayed in a predetermined area; 2nd reception information detected by 2 sensors, The 2nd reception part which receives the 2nd sensor information about the user's intention to the goods or service specified based on the 1st sensor information, The 1st sensor information And a determination unit that determines a processing mode related to a purchase target that is a product or service specified based on the second sensor information.

  According to one aspect of the embodiment, there is an effect that it is possible to appropriately determine a processing aspect related to a product or service.

FIG. 1 is a diagram illustrating an example of a determination process according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of the determination device according to the embodiment. FIG. 3 is a diagram illustrating an example of a device information storage unit according to the embodiment. FIG. 4 is a diagram illustrating an example of a content information storage unit according to the embodiment. FIG. 5 is a diagram illustrating an example of a product information storage unit according to the embodiment. FIG. 6 is a diagram illustrating an example of a shopping list information storage unit according to the embodiment. FIG. 7 is a flowchart illustrating an example of the determination process according to the embodiment. FIG. 8 is a diagram illustrating a configuration example of a determination device according to a modification. FIG. 9 is a diagram illustrating an example of the model information storage unit according to the modification. FIG. 10 is a hardware configuration diagram illustrating an example of a computer that realizes the function of the determination apparatus.

  Hereinafter, a determination apparatus, a determination method, and a mode for executing a determination program according to the present application (hereinafter referred to as “embodiment”) will be described in detail with reference to the drawings. Note that the determination device, the determination method, and the determination program according to the present application are not limited to the embodiment. In the following embodiments, the same portions are denoted by the same reference numerals, and redundant description is omitted.

(Embodiment)
[1. Decision process)
First, the determination process according to the embodiment will be described. FIG. 1 is a diagram illustrating an example of a determination process according to the embodiment. The determination apparatus 100 illustrated in FIG. 1 provides a determination service that determines a processing mode related to a product or a service based on various detected sensor information. Note that the product or service mentioned here may include such a target as long as it can be traded. In the following, “product or service” may be simply referred to as “product”.

  As illustrated in FIG. 1, the determination system 1 according to the embodiment includes a terminal device AP11 and a determination device 100. The terminal device AP11 and the determination apparatus 100 are connected to be communicable via a predetermined network. In addition, the determination system 1 illustrated in FIG. 1 may include a plurality of terminal devices AP11 and a plurality of determination devices 100.

  In the example illustrated in FIG. 1, the terminal device AP11 has a projector function, and displays content by projecting an image of the content onto a predetermined display surface. FIG. 1 shows a case where the terminal device AP11 projects the content identified by the content ID “CT11” (hereinafter also referred to as “content CT11”) onto the table top surface PP11 of the table. That is, the terminal device AP11 displays the content CT11 on the desk surface PP11 by projecting the content CT11 on the desk surface PP11. As shown in FIG. 1, the content CT11 is content that includes, for example, a plurality of products and proposes a lifestyle to the browsing user. For example, the content displayed by the terminal device AP11 is not limited to the content CT11, and may be various content such as content including a plurality of products related to a predetermined event such as Christmas or athletic meet. In this way, the terminal device AP11, for example, displays content such as magazine pages including a plurality of products based on a predetermined concept sequentially according to the user's operation, so that the user can obtain the predetermined concept. A plurality of products can be browsed. Further, the terminal device AP11 is not limited to the desktop surface PP11, and may display the content in any target such as a space as long as the content can be displayed.

  In the example of FIG. 1, the terminal device AP11 includes an infrared sensor as the first sensor SN1. Accordingly, the terminal device AP11 has a function of detecting a user designation for the displayed content. For example, the terminal device AP11 captures infrared rays emitted from the first sensor SN1 with an image sensor or the like and detects the change thereof, or detects reflection of the infrared rays, thereby detecting information on coordinates in the displayed content (hereinafter, “ (Also referred to as “coordinate information”) may be detected as the first sensor information. In this case, for example, the first sensor SN1 of the terminal device AP11 (hereinafter sometimes simply referred to as “terminal device AP11”) may have a function of emitting infrared light and a function of detecting infrared light. For example, the first sensor SN1 of the terminal device AP11 may have a function of radiating infrared rays and a function of detecting changes in infrared rays using image recognition technology.

  In the example of FIG. 1, the terminal device AP11 emits infrared rays along the desk surface PP11 on which the content CT11 is displayed, and is reflected when the user performs an operation such as pointing the content CT11 with a finger or the like. Is detected. For example, the terminal device AP11 may detect coordinate information such as orthogonal coordinates (XY coordinates) corresponding to the area where the content CT11 is displayed on the desk surface PP11. For example, the terminal device AP11 may detect coordinate information of XY coordinates in which the horizontal direction of the content CT11 in FIG. 1 is the X-axis direction and the vertical direction is the Y-axis direction. In this case, for example, the terminal device AP11 may detect coordinate information of XY coordinates with the lower left position of the content CT11 in FIG. 1 as the origin (0, 0). Note that the above is an example, and the terminal device AP11 may detect a position where the user has designated the displayed content by appropriately using various conventional techniques.

  Thereby, terminal device AP11 and the determination apparatus 100 can specify the product designated by the user based on which product is displayed at the position designated by the user for the displayed content and the detected coordinate information. . Note that the above is an example, and the terminal device AP11 may detect a user's designation by appropriately using various conventional techniques. The first sensor SN1 is not limited to the infrared sensor, and any sensor may be used as long as it can detect the user's designation for the displayed content. For example, the terminal device AP11 transmits coordinate information related to the user designation for the content detected by the first sensor SN1 to the determination device 100 as the first sensor information.

  Further, the terminal device AP11 includes a voice sensor as the second sensor SN2. Accordingly, the terminal device AP11 has a voice recognition function, detects surrounding sounds, and performs various processes according to the detected sounds. Hereinafter, information detected by the second sensor SN2 may be referred to as second sensor information. Hereinafter, the second sensor SN2 of the terminal device AP11 may be simply referred to as “terminal device AP11”. The terminal device AP11 may have a voice recognition function. In this case, the terminal device AP11 includes information related to the intention of the user who made the speech estimated based on the information related to the user's speech detected by the second sensor SN2 (hereinafter also simply referred to as “intention” or “intention information”). ) May be estimated.

  The terminal device AP11 can transmit and receive information to and from an external device such as the determining device 100. For example, the terminal device AP11 receives information related to content to be displayed from the determination device 100, or transmits information related to detection of various sensors to the determination device 100. Note that the terminal device AP11 referred to here may be capable of transmitting and receiving information to and from the determining device 100 via another device. Further, for example, the terminal device AP11 may be a smartphone, a tablet-type terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), etc. A device in which an application is installed may be used.

  Note that processing such as content display processing by the terminal device AP11 may be realized by, for example, JavaScript (registered trademark). When the display process described above is performed by a predetermined application or when the display process is performed by a dedicated application, the terminal device AP11 may include, for example, an application control unit that controls the predetermined application or the dedicated application. .

  The determination system 1 may have any configuration as long as the determination device 100 can acquire information used for the determination process, and may include a device having each function of the terminal device AP11. For example, in the determination system 1, each function of the terminal device AP11 may be realized by a device that displays content, and each device such as the first sensor SN1 and the second sensor SN2. For example, in the determination system 1, the determination apparatus 100 may have the function of the terminal device AP11. In this case, the determination apparatus 100 and the terminal device AP11 may be integrated. Further, the determination device 100 may be in any form as long as it can realize a function related to the determination process. For example, the determination device 100 is integrated with a table having the desk surface PP11 or is distributed among a plurality of information processing devices. Or may be realized.

  The determination apparatus 100 provides a determination service for determining a processing mode related to a product specified based on the first sensor information based on the second sensor information. Further, the determination device 100 may be an information processing device that provides content displayed by the terminal device AP11 to the terminal device AP11. Further, the determination apparatus 100 may be able to acquire information from an information processing apparatus that provides a content distribution service. In this case, the determination system 1 may include an information processing apparatus that provides a content distribution service, for example.

  For example, the determination apparatus 100 may be an information processing apparatus that provides an electronic commerce service such as online shopping. Moreover, the determination apparatus 100 may be able to acquire information from an information processing apparatus that provides an electronic commerce service. In this case, the determination system 1 may include, for example, an information processing apparatus that provides an electronic commerce service such as online shopping.

  Moreover, the determination apparatus 100 may be able to acquire information from a voice recognition server that provides a voice recognition service. In this case, the determination system 1 may include a voice recognition server. In the example of FIG. 1, the determination device 100 and the voice recognition server will not be described on the assumption that the user's utterance is recognized or the uttered user is identified by appropriately using various conventional techniques. For example, the determination apparatus 100 estimates user intention information based on the second sensor information received from the terminal device AP11.

  Hereinafter, an example of the determination process will be described with reference to FIG. In the example illustrated in FIG. 1, the terminal device AP11 is arranged in a house where the user U11, the user U12, and the user U13 live. Hereinafter, when the user U11, the user U12, and the user U13 are described without being distinguished, they may be simply referred to as “users”. For example, the terminal device AP11 is arranged on a table in a house, and the content CT11 is displayed on the desk surface PP11 of the arranged table.

  In the example of FIG. 1, the terminal device AP11 has already acquired the content CT11 from an external device such as the determining device 100, and the terminal device AP11 displays the content CT11 on the table top surface PP11 of the table in accordance with a user operation or the like. Suppose you are. Specifically, it is assumed that the terminal device AP11 displays the content CT11 in the region RG1 in the desk surface PP11. In FIG. 1, the region indicated by the symbol “RG1” overlaps the region where the content CT11 is illustrated. The terminal device AP11 displays not only the content CT11 but also various contents on the desk surface PP11 of the table. For example, the terminal device AP11 displays content other than the content CT11 on the desk surface PP11 in accordance with an operation for changing the display by the user. For example, the terminal device AP11 may change the content displayed on the desk surface PP11 in accordance with an operation in which the user moves the finger FG1 in a predetermined direction on the region RG1 of the desk surface PP11.

  Further, as shown in FIG. 1, the content CT11 displayed on the desk surface PP11 includes five products IT11 to IT15. As described above, “product IT * (* is an arbitrary numerical value)” indicates that the product is a product identified by the product ID “IT *”. For example, when “product IT11” is described, the product is a product identified by the product ID “IT11”.

  For example, the product IT11 corresponds to the product “Shampoo A” included in the content CT11. Further, for example, the product IT12 corresponds to the product “refill product of shampoo A” included in the content CT11. For example, the product IT13 corresponds to the product “towel A” included in the content CT11. In addition, for example, the product IT14 corresponds to the product “car A” included in the content CT11. For example, the product IT15 corresponds to the product “pot A” included in the content CT11. As described above, FIG. 1 shows a case where the users U11 to U13 are browsing the content CT11 including various products.

  First, the user U12 browsing the content CT11 designates the product IT14 in the content CT11 with the finger FG1 (step S11). For example, when the user U12 indicates the position where the product IT14 in the content CT11 is included with the finger FG1 or touches the part on the desk surface PP11 where the product IT14 is displayed, for example, the date IT11 specify.

  Then, in response to designation by the user U12 with the finger FG1, the first sensor SN1 of the terminal device AP11 detects the coordinate information of the position designated by the user U12 with the finger FG1 as first sensor information (step S12). In the example of FIG. 1, the terminal device AP11 detects the coordinate information of the position specified by the user U12 with the finger FG1. Specifically, the terminal device AP11 detects coordinate information indicating that the position designated by the user U12 with the finger FG1 is the position (x1, y1). Note that “x1” and “y1” here are arbitrary values corresponding to positions included in the range in which the product IT14 is displayed in the content CT11.

  Then, the terminal device AP11 transmits the detected first sensor information to the determination device 100 (step S13). Thereby, the determination apparatus 100 receives the first sensor information from the terminal device AP11. In the example of FIG. 1, the terminal device AP11 displays first sensor information indicating that the position specified by the user is the position (x1, y1) and information indicating that the content specified by the user is the content CT11. Send. In addition, when the determination apparatus 100 has already acquired information regarding which content the terminal device AP11 displays when specified by the user, information for identifying the content specified by the user may not be transmitted.

  And the determination apparatus 100 which received 1st sensor information from terminal device AP11 specifies the goods which the user specified based on 1st sensor information. In the example of FIG. 1, the determination device 100 specifies a product designated by the user based on the received first sensor information and the content information stored in the content information storage unit 122.

  In the example of FIG. 1, the determination apparatus 100 stores, as shown in the content information storage unit 122, information indicating the products included in each content and the range in which the products are located in the content. For example, as shown in the content information storage unit 122, the determination apparatus 100 stores information indicating that the content CT11 includes five products IT11 to IT15. The content information storage unit 122 stores information indicating that the product IT11 is located in the range AR11 in the content CT11. The range AR11 may be any information as long as it is information indicating the range that the product IT11 occupies in the content CT11. For example, the range AR11 may be coordinate information.

  Further, the content information storage unit 122 stores information indicating that the product IT12 is located in the range AR12 in the content CT11. In addition, the content information storage unit 122 stores information indicating that the product IT13 is located in the range AR13 in the content CT11. In addition, the content information storage unit 122 stores information indicating that the product IT14 is located in the range AR14 in the content CT11. The content information storage unit 122 stores information indicating that the product IT15 is located in the range AR15 in the content CT11.

  In the example of FIG. 1, the determination apparatus 100 specifies the product designated by the user based on the first sensor information and the ranges AR11 to AR15 of the products IT11 to IT15 stored in the content information storage unit 122 ( Step S14). For example, when the coordinates (x1, y1) of the first sensor information are included in any of the ranges AR11 to AR15, the determination device 100 specifies a product designated by the user. In the example of FIG. 1, since the coordinates (x1, y1) of the first sensor information are included in the range AR14, the determination apparatus 100 specifies that the product designated by the user is the product IT14. Further, the determining apparatus 100 specifies that the product IT14 is the product name “car A” based on the correspondence between the product ID and the product name stored in the product information storage unit 123.

  Thereafter, the user U12 performs the utterance PA11 “I want this” in the vicinity of the terminal device AP11 (second sensor SN2) (step S15). For example, at the date DT12 after the date DT11, the user U12 performs the utterance PA11 "I want this" for the product IT14 specified in step S11. In the example of FIG. 1, also in step S15, the case where the user U12 continues to specify the product IT14 with the finger FG1 is shown. However, if the product specified by the user U12 has been specified, the user U12 It may be in a state where designation of the product IT14 by FG1 is cancelled.

  Then, in response to the user U12 performing the utterance PA11, the second sensor SN2 of the terminal device AP11 detects the voice information of the utterance PA11 by the user U12 as the second sensor information (step S16). In the example of FIG. 1, the terminal device AP11 detects voice information corresponding to the utterance PA11 “I want this” by the user U12. In the example of FIG. 1, only the utterance PA11 is illustrated for the sake of simplicity, but the users U11 to U13 may perform various conversations regarding the product IT14. Further, the second sensor SN2 of the terminal device AP11 may detect utterances other than the utterance PA11 as the second sensor information.

  Then, the terminal device AP11 transmits the detected second sensor information to the determination device 100 (step S17). Thereby, the determination apparatus 100 receives the second sensor information from the terminal device AP11. In the example of FIG. 1, the terminal device AP11 transmits second sensor information related to the user's utterance. Specifically, the terminal device AP11 transmits the voice information of the utterance PA11 as the second sensor information.

  The terminal device AP11 may transmit the voice information of the utterance PA11 to the voice recognition server, acquire the character information of the utterance PA11 from the voice recognition server, and transmit the acquired character information to the determination apparatus 100. Further, the terminal device AP11 may transmit information indicating that the user who performed the acquired utterance PA11 is the user U12 to the determination device 100. Further, when the terminal device AP11 estimates the utterance intention, the terminal device AP11 may transmit the estimated utterance intention to the determination apparatus 100.

  The determination apparatus 100 that has received the second sensor information from the terminal device AP11 estimates the user's intention based on the second sensor information (step S18). For example, the determination apparatus 100 estimates that the user desires to purchase the product IT14 specified in step S14 by analyzing the received second sensor information. For example, the determination apparatus 100 converts the received voice information of the utterance PA11, which is the second sensor information, into character information, and estimates the user's intention based on the converted character information. For example, the deciding device 100 specifies that the utterance PA11 includes the adjective pronoun “this” and the adjective “wanted” indicating that the user desires to purchase a product corresponding to the pronoun pronoun. For example, the determination apparatus 100 presumes that the user desires to purchase a product corresponding to the pronoun by analyzing the character information of the utterance PA11 by appropriately using various conventional techniques such as syntax analysis. . Thereby, the determination apparatus 100 estimates that the user intends to purchase the product IT14 based on the content of the utterance PA11.

  Thus, in step S18, the determination apparatus 100 estimates that the user desires to purchase a product corresponding to the indicating pronoun based on the second sensor information. Then, in step S14, the determining apparatus 100 specifies that the product designated by the user is the product IT14. Therefore, the determination apparatus 100 estimates that the user desires to purchase the car A that is the product IT14.

  And the determination apparatus 100 determines the process aspect regarding goods IT14 according to the user's estimated intention with respect to goods IT14. In the example of FIG. 1, since it is presumed that the user desires to purchase the car A that is the product IT14, the determination apparatus 100 determines to add the product IT14 to a predetermined shopping list (step S19). . Specifically, the determination device 100 determines to add the product IT14 to the shopping list associated with the terminal device AP11. Then, as shown in the shopping list information storage unit 124, the determination device 100 adds the product IT14 to the shopping list associated with the terminal device AP11. In the example of FIG. 1, the determination device 100 determines to add the product IT14 to the shopping list associated with the terminal device AP11.

  Thus, the determination apparatus 100 determines the process aspect regarding the goods IT14 specified based on 1st sensor information based on 2nd sensor information. Further, in the example of FIG. 1, the determination device 100 determines to add the product IT14 to the shopping list associated with the terminal device AP11, whereby a plurality of users U11 to U13 have a conversation, and which user purchases the product. Even if it is not possible to specify whether or not the user intends, the product can be appropriately added to the shopping list. Further, the determination apparatus 100 determines that the product IT14 is added to the shopping list associated with the terminal device AP11, so that even if the user who specified the product and the user who made the utterance about the product are different, Appropriately add items to the shopping list. As described above, the determination device 100 appropriately determines a processing mode related to a product even when a plurality of users browse the content while having a conversation and specify a product or make an utterance indicating intention to the product. can do. Moreover, the determination apparatus 100 may transmit information regarding the product added to the shopping list associated with the terminal device AP11 to the user at a predetermined timing.

  Further, in the example of FIG. 1, for the sake of simplicity, the process up to the determination of the processing mode related to the product IT14 is shown. However, the users U11 to U13 can specify various products following the product IT14, or display them. You may also have a conversation about the content that has been played. And the determination apparatus 100 may determine the process aspect regarding goods sequentially based on the designation | designated and conversation of goods by a user. For example, when the user designates the product IT11 after step S19 and continues the conversation, the determining apparatus 100 may determine a processing mode related to the product IT11. For example, the determination apparatus 100 may determine the processing mode related to the product IT15 when the user specifies the product IT15 after the determination of the processing mode related to the product IT11 and continues the conversation. In addition, when the user switches the display to content other than the content CT11 and continues the conversation, the determination apparatus 100 may determine the processing mode related to the product designated by the user based on the user's designation or conversation. .

  Further, in the example of FIG. 1, in order to simplify the explanation, the case where the product IT14 is added to the shopping list associated with the terminal device AP11 is shown. Depending on the intention, the processing mode may be varied. For example, the determination apparatus 100 may vary the processing mode according to the estimated level (strongness) of the user's purchase intention. For example, when it is estimated that the user's intention to purchase is high based on the user's utterance such as “I want it now” or “Absolutely buy”, the determination device 100 immediately purchases the designated product, that is, performs a settlement process. You may go. In the example of FIG. 1, since the utterance PA11 includes the content that the user wants to purchase “desired”, the determination apparatus 100 presumes that the user U12 has a high intention to purchase, and immediately purchases the designated product IT14, that is, settles. You may perform the process of. In this case, the determination apparatus 100 may perform the settlement process using the account of the user U12, or the settlement process using the account ACN11 (see FIG. 3) of the user U11 associated with the terminal device AP11. You may go. In addition, for example, the determination apparatus 100 is designated when it is estimated that the user has a low intention to purchase but is interested in the product based on the user's utterances such as “Ikomoka” and “I am interested”. Products may be added to a hold list or shopping list. Thus, the determination apparatus 100 may vary the processing mode for the designated product based on the level of the user's intention to purchase based on the user's utterance.

  As described above, the determination apparatus 100 specifies a product specified by the user based on the first sensor information, and processes related to the product (purchase target) specified based on the first sensor information based on the second sensor information. Determine aspects. As described above, the determination apparatus 100 identifies the product designated by the user based on the first sensor information, and estimates the user's intention for the product based on the second sensor information. As a result, the determination apparatus 100 performs processing in two stages of product identification and estimation of the user's intention, thereby identifying the accuracy of the product specified by the user and estimating the user's intention for the product. Accuracy can be increased. Therefore, the determination apparatus 100 can appropriately determine the processing mode related to the product or service.

  In the example described above, the determination device 100 uses information detected by the infrared sensor as the first sensor information, and uses information detected by the voice sensor as the second sensor information. Thus, the determination apparatus 100 can improve the accuracy of specifying a product specified by the user and the accuracy of estimating the user's intention for the product by using information detected by different sensors in each process. .

  In the example of FIG. 1, the determination apparatus 100 increases the specific accuracy of the product specified by the user by specifying the product displayed at the position designated by the user with the finger FG1 or the like as the product specified by the user. Can do. For example, when the product specified by the user is specified based on the voice information, it is difficult to specify the product specified by the user when the user speaks with a pronoun such as “this” or “that”. Further, in the example of FIG. 1, since the content CT11 includes “shampoo A” that is the product IT11 and “refilled products of shampoo A” that is the product IT12, the user utters “you must buy shampoo A”. Even so, it is difficult to specify which of the product IT11 and the product IT12 is specified. On the other hand, in the example of FIG. 1, the determination device 100 specifies the product displayed at the position designated by the user with the finger FG1 or the like as the product specified by the user, so that the product specified by the user is the product IT11 and the product. It is possible to increase the specific accuracy of the IT12.

  Note that the determination device 100 may specify the product specified by the user based on the audio information when the product specified by the user can be specified by the audio information. As described above, the determination apparatus 100 may identify the product designated by the user or estimate the user's intention for the product based on the content of the information detected by each sensor. Further, as illustrated in FIG. 1, the determination apparatus 100 may accept only the second sensor information related to the utterance performed while the user U12 continues to specify the product IT14 with the finger FG1. Thereby, the determination apparatus 100 can match the user's intention with the product designated by the user more accurately.

  In the above-described example, the infrared sensor is used as the first sensor SN1 and the product to be specified is specified based on the instruction by the user's finger FG1, but the determination device 100 may use various sensors as the first sensor. Good. For example, the determination apparatus 100 may use a motion sensor or the like that detects a user gesture or the like as the first sensor. In this case, the determination apparatus 100 may specify a product to be designated based on a user gesture or the like. For example, the determination apparatus 100 may use a sensor or the like that detects the user's line of sight as the first sensor. In this case, the determination apparatus 100 may specify a product to be specified based on the user's line of sight. Moreover, the determination apparatus 100 may specify the goods to designate based on the some 1st sensor information which a some 1st sensor detects using several sensors as mentioned above as a 1st sensor.

  Moreover, in the example mentioned above, the determination apparatus 100 may specify that the user who performed the utterance PA11 is the user U12 by analyzing the second sensor information that is the voice information of the received utterance PA11. For example, the determination apparatus 100 may specify that the user who performed the utterance PA11 is the user U12 by analyzing the audio information of the utterance PA11 based on the audio information associated with each user. . Note that the user who performed the utterance PA11 may be specified by the determination device 100 or another device. When the other device specifies the user who performed the utterance PA11, the determination device 100 acquires information identifying the user who performed the utterance PA11 from the other device. Thus, the determination apparatus 100 can improve the accuracy of estimation of the user's intention by specifying the user who has spoken. For example, the determination apparatus 100 may estimate the user's intention by using different models according to the specified user. Details of this point will be described later.

  Moreover, when the information regarding the number is included in the second sensor information, the determination device 100 may determine the processing mode regarding the designated product based on the information regarding the number. In the example of FIG. 1, when the second sensor information is voice information including information on the number of pieces such as “I want three,” the item IT14 is added to the shopping list associated with the terminal device AP11. Three may be added.

  In the above-described example, the determination apparatus 100 adds the product IT14 designated to the shopping list associated with the terminal device AP11, that is, the shopping list associated with the device. Various processing modes may be determined as appropriate. For example, the determination apparatus 100 may add the product IT14 specified in the shopping list associated with the user. For example, as illustrated in the device information storage unit 121 (see FIG. 3), the determination apparatus 100 may add the product IT14 specified in the shopping list of the user U11 who is a user associated with the terminal device AP11. .

  For example, when the user who performed the utterance PA11 can specify that the user who performed the utterance PA11 is the user U12, the determination apparatus 100 may add the product IT14 to the shopping list of the user U12. In the example of FIG. 1, the determination apparatus 100 adds the product IT14 to the shopping list of the user U12 when the user who performed the utterance PA11 can be identified as the user U12, and cannot identify the user who performed the utterance PA11. The product IT14 may be added to the shopping list of the user U11 associated with the terminal device AP11. Thus, the determination apparatus 100 may determine the processing mode regarding the designated product based on various information including the second sensor information.

  Further, for example, when the user's intention estimated based on the second sensor information indicates purchase of a product, the determination device 100 may determine that the product is to be settled. In the example of FIG. 1, the determination apparatus 100 may determine that the product IT14 is to be settled because the purchase intention for the designated product IT14 is estimated. In this case, the determination apparatus 100 determines that the payment using the account of the user U12 is performed when the user who performed the utterance PA11 can be identified as the user U12, and the terminal who has performed the utterance PA11 cannot be identified, It may be determined that the payment is made using the account ACN11 (see FIG. 3) of the user U11 associated with the device AP11.

  For example, when the user's intention estimated based on the second sensor information indicates interest in a product, the determination device 100 may determine to add the product to the hold list. For example, when a product is specified, if the user does not reach the intention of purchase such as “Is this good?” Or “I want to buy it someday” and makes an utterance indicating interest in the product, the determination device 100 May decide to add the item to the hold list. For example, the determination apparatus 100 may add a product to the hold list associated with the device. For example, when the user who has made an utterance can be identified, the determination apparatus 100 may add a product to the user's hold list. Further, for example, when the user who has spoken cannot be identified, the determination apparatus 100 may add a product to the user's hold list associated with the device.

  Moreover, the determination apparatus 100 and the terminal device AP11 may change the display mode of the content displayed by the terminal device AP11 according to the user's utterance. In the example of FIG. 1, the content CT11 displayed on the desk surface PP11 may change according to the user's utterance. For example, when the user U13 designates the white pot A that is the product IT15 and utters “the red one”, the determination device 100 and the terminal device AP11 use the white pot in the content CT11 displayed by the terminal device AP11. The display of A may be changed to a red pot A. For example, when the terminal device AP11 has a voice recognition function and has acquired an image (content) corresponding to each color of each product in advance, the user U13 designates the white pot A and “this red guy” The display of the white pot A in the content CT11 may be changed to the red pot A according to the utterance. For example, the determination apparatus 100 causes the terminal device AP11 to change the display of the white pot A in the content CT11 to the red pot A in response to the designation of the white pot A by the user U13 and the utterance of “this red guy”. You may instruct. For example, the determination apparatus 100 transmits an image (contents) corresponding to the red pot A to the terminal device AP11 in response to the designation of the white pot A by the user U13 and the utterance of “this red guy”. It may be instructed to change the display of the white pot A displayed on the PP 11 to the red pot A.

  Further, for example, when the user U11 specifies the towel A that is the product IT11 and utters “this color difference”, the determination device 100 and the terminal device AP11 are white in the content CT11 displayed by the terminal device AP11. The display of the towel A may be changed to a towel A of another color. For example, when the user U11 designates the towel A that is the product IT11 and utters “this color difference”, the terminal device AP11 uses a white towel A of another color (for example, blue) that the towel A can handle. The display may be changed to towel A. Note that the above-described display change may be performed by the terminal device AP11 using various techniques as appropriate. For example, the terminal device AP11 may acquire (image) data corresponding to each color of each product in advance and switch data to be displayed. Further, for example, the terminal device AP11 may change the display mode of the content by superimposing and displaying it with an overlay or the like.

[2. Configuration of decision device]
Next, the configuration of the determination apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration example of the determination device according to the embodiment. As illustrated in FIG. 2, the determination device 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The determination device 100 includes an input unit (for example, a keyboard and a mouse) that receives various operations from an administrator of the determination device 100 and a display unit (for example, a liquid crystal display) for displaying various types of information. May be.

(Communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card). The communication unit 110 is connected to the network by wire or wireless, and transmits / receives information to / from the terminal device AP11. The communication unit 110 may transmit and receive information to and from an information processing apparatus that provides an electronic commerce service or a voice recognition server.

(Storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. As illustrated in FIG. 2, the storage unit 120 according to the embodiment includes a device information storage unit 121, a content information storage unit 122, a product information storage unit 123, and a shopping list information storage unit 124.

(Device information storage unit 121)
The device information storage unit 121 according to the embodiment stores various information regarding various devices such as the terminal device AP11. For example, the device information storage unit 121 stores various pieces of information related to various devices used for collecting the first sensor information and the second sensor information. FIG. 3 shows an example of the device information storage unit 121 according to the embodiment. The device information storage unit 121 illustrated in FIG. 3 includes items such as “device ID”, “user ID”, and “account ID”.

  “Device ID” indicates identification information for identifying a device. “User ID” indicates identification information for identifying a user. For example, “user ID” indicates identification information for identifying a user associated with a device. For example, a plurality of user IDs may be associated with each device ID. “Account ID” indicates identification information for identifying an account. For example, “Account ID” indicates identification information for identifying an account associated with the user.

  For example, in the example illustrated in FIG. 3, the device (terminal device AP11) identified by the device ID “AP11” is associated with the user (user U11) identified by the user ID “U11”. . Further, the user (user U11) identified by the user ID “U11” indicates that the account identified by the account ID “ACN11” is associated.

  The device information storage unit 121 is not limited to the above, and may store various types of information according to the purpose. The device information storage unit 121 may store information on the limit amount. For example, the device information storage unit 121 may store information on past payment amounts.

(Content information storage unit 122)
The content information storage unit 122 according to the embodiment stores various types of information related to content. For example, the content information storage unit 122 stores various types of information related to content displayed by a device having display means such as the terminal device AP11. FIG. 4 is a diagram illustrating an example of a content information storage unit according to the embodiment. The content information storage unit 122 illustrated in FIG. 4 includes items such as “content ID”, “product ID”, and “range”.

  “Content ID” indicates identification information for identifying the content. For example, the content identified by the content ID “CT11” corresponds to the content CT11 shown in the example of FIG. “Product ID” indicates identification information for identifying a product. The “range” indicates a position in the content where the product identified by the corresponding product ID is included. In the example of FIG. 4, the “range” is indicated by an abstract code such as AR11, but the “range” may be a coordinate indicating the position in the content of the product identified by the corresponding product ID. Good.

  For example, in the example illustrated in FIG. 4, the content identified by the content ID “CT11” includes the product identified by the product ID “IT11”, the product identified by the product ID “IT12”, and the product ID “IT13”. ”, A product identified by the product ID“ IT14 ”, and a product identified by the product ID“ IT15 ”. For example, in the example illustrated in FIG. 4, the product identified by the product ID “IT11” is located in the range AR11 in the content CT11. For example, the range AR11 corresponds to a predetermined range located approximately at the center in the content CT11.

  For example, in the example illustrated in FIG. 4, the product identified by the product ID “IT12” is located in the range AR12 in the content CT11. For example, in the example illustrated in FIG. 4, the product identified by the product ID “IT13” is located in the range AR13 in the content CT11. For example, in the example illustrated in FIG. 4, the product identified by the product ID “IT14” is located in the range AR14 in the content CT11. For example, in the example illustrated in FIG. 4, the product identified by the product ID “IT15” is located in the range AR15 in the content CT11.

  The content information storage unit 122 is not limited to the above, and may store various types of information according to the purpose.

(Product information storage unit 123)
The product information storage unit 123 according to the embodiment stores various types of information related to products. FIG. 5 is a diagram illustrating an example of a product information storage unit according to the embodiment. The product information storage unit 123 illustrated in FIG. 5 includes items such as “product ID” and “product name”.

  “Product ID” indicates identification information for identifying a product. “Product name” indicates a specific name of a product or service.

  Further, in the example illustrated in FIG. 5, the product identified by the product ID “IT11” is “Shampoo A”. Further, in the example shown in FIG. 5, the product identified by the product ID “IT12” indicates “shampoo A refill product”, that is, a product for refilling shampoo A. In the example illustrated in FIG. 5, the product identified by the product ID “IT13” is “towel A”. Further, in the example illustrated in FIG. 5, the product identified by the product ID “IT14” is “car A”. Further, in the example illustrated in FIG. 5, the product identified by the product ID “IT15” is “pot A”.

  The product information storage unit 123 is not limited to the above, and may store various information according to the purpose. For example, the product information storage unit 123 may store information related to the price of the product. For example, the product information storage unit 123 may store information on stores (stores) that handle products and information on stock.

(Shopping list information storage unit 124)
The shopping list information storage unit 124 according to the embodiment stores various information related to shopping (purchase). For example, the shopping list information storage unit 124 stores various information related to a predetermined product. FIG. 6 is a diagram illustrating an example of a shopping list information storage unit according to the embodiment. The shopping list information storage unit 124 illustrated in FIG. 6 includes items such as “target ID” and “shopping list”.

  “Target ID” indicates information for identifying a target associated with a list of products. The “target ID” may be any ID that identifies a target associated with a list of products, such as an ID that identifies a terminal device such as “AP11” or an ID that identifies a user such as “U1”. An ID for identifying such an object may be included. The “shopping list” indicates information relating to the product associated with the target. For example, the “shopping list” indicates the association between the product specified based on the first sensor information and the target.

  The “shopping list” includes items such as “product” and “date and time”. For example, “product” indicates information for identifying a product for which payment has been made in the group. The “date and time” indicates the date and time when the corresponding product was settled. The “date and time” may be the date and time when the corresponding product is input.

  The “shopping list” may include an item such as “input user”. The “input user” in the “shopping list” may be a user who specifies a corresponding product, or may be a user who indicates an intention for the corresponding product. For example, the “input user” in the “shopping list” may be a user whose first sensor information is detected or a user whose second sensor information is detected.

  For example, in the example illustrated in FIG. 6, the target identified by the target ID “AP11” is the terminal device AP11 identified by “AP11”. For example, in the example illustrated in FIG. 6, as illustrated in the “shopping list”, the product identified by the product ID “IT1” is added to the shopping list corresponding to the terminal device AP11 at the date “DT1”. Further, for example, it indicates that the product identified by the product ID “IT14” is added to the shopping list corresponding to the terminal device AP11 at the date “DT12”.

  The shopping list information storage unit 124 is not limited to the above, and may store various information according to the purpose. For example, the shopping list information storage unit 124 may store information related to the amount of merchandise. For example, the shopping list information storage unit 124 indicates, in addition to the “shopping list”, a “settlement completed list” indicating information on products for which payment has been made, and information on products for which payment has been suspended by a user operation. A “hold list” or the like may be stored.

(Control unit 130)
Returning to the description of FIG. 2, the control unit 130 is a controller and is stored in a storage device inside the determination apparatus 100 by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. Various programs (corresponding to an example of a determination program) are implemented by using the RAM as a work area. The control unit 130 is a controller, and is realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  As shown in FIG. 2, the control unit 130 includes a first reception unit 131, a second reception unit 132, an estimation unit 133, a determination unit 134, and a transmission unit 135, and a determination process described below. Realize or execute the functions and operations of Note that the internal configuration of the control unit 130 is not limited to the configuration illustrated in FIG. 2, and may be another configuration as long as the determination process described later is performed.

(First reception part 131)
The 1st reception part 131 receives various information. For example, the first receiving unit 131 receives various types of information from an external information processing apparatus. For example, the 1st reception part 131 is the 1st sensor information detected by 1st sensor SN1, Comprising: The 1st sensor information regarding the designation | designated of the user with respect to the goods or service displayed on the predetermined area | region is received. For example, the 1st reception part 131 receives the 1st sensor information detected by the infrared sensor as 1st sensor SN1. For example, the 1st reception part 131 receives the 1st sensor information regarding designation | designated of the user with respect to one goods or service among the goods or services displayed in multiple in the predetermined area | region.

  In FIG. 1, the 1st reception part 131 receives 1st sensor information from terminal device AP11. In the example of FIG. 1, the first receiving unit 131 indicates that the first sensor information indicating that the position specified by the user is the position (x1, y1) and that the content specified by the user is the content CT11. Information is received from the terminal device AP11. Further, for example, the first receiving unit 131 may receive only the first sensor information indicating that the position specified by the user is the position (x1, y1) from the terminal device AP11.

(Second reception part 132)
The 2nd reception part 132 receives various information. For example, the second reception unit 132 receives various types of information from an external information processing apparatus. For example, the 2nd reception part 132 is 2nd sensor information detected by 2nd sensor SN2, Comprising: The 2nd sensor information regarding a user's intention with respect to the goods or service specified based on 1st sensor information is received. For example, the 2nd reception part 132 receives the 2nd sensor information detected by the audio | voice sensor as 2nd sensor SN2.

  In FIG. 1, the 2nd reception part 132 receives 2nd sensor information from terminal device AP11. In the example of FIG. 1, the 2nd reception part 132 receives the 2nd sensor information regarding a user's utterance from terminal device AP11. For example, the second reception unit 132 receives the voice information of the utterance PA11 from the terminal device AP11 as the second sensor information. The first reception unit 131 and the second reception unit 132 may be integrated.

(Estimation unit 133)
The estimation unit 133 estimates and specifies various information. For example, the estimation unit 133 estimates the user's intention for the purchase target based on the second sensor information. For example, the estimation unit 133 may estimate the user's intention for the purchase target based on the model that varies depending on the user and the second sensor information. For example, the estimation unit 133 uses the model learned based on the user's intention estimated in the past and the information regarding the user's behavior with respect to the product or service whose processing mode is determined based on the intention. By inputting sensor information, the user's intention for the purchase target may be estimated. Details of this point will be described later.

  In the example of FIG. 1, the estimation unit 133 identifies the product designated by the user based on the first sensor information from the terminal device AP11. For example, the estimation unit 133 identifies the product designated by the user based on the first sensor information and the content information stored in the content information storage unit 122. For example, the estimation unit 133 identifies the product designated by the user based on the first sensor information and the ranges AR11 to AR15 of the products IT11 to IT15 stored in the content information storage unit 122. For example, when the coordinates (x1, y1) of the first sensor information are included in any of the ranges AR11 to AR15, the estimation unit 133 identifies the product designated by the user. In the example of FIG. 1, since the coordinates (x1, y1) of the first sensor information are included in the range AR14, the estimation unit 133 specifies that the product specified by the user is the product IT14. Further, for example, the estimating unit 133 specifies that the product IT14 is the product name “car A” based on the correspondence between the product ID and the product name stored in the product information storage unit 123.

  In the example of FIG. 1, the estimation unit 133 estimates the user's intention based on the second sensor information from the terminal device AP11. For example, the estimating unit 133 analyzes the received second sensor information, and estimates that the user desires to purchase the product IT14 specified based on the first sensor information. For example, the estimation unit 133 converts the voice information of the utterance PA11, which is the second sensor information, into character information, and corresponds to the indicating pronoun “this” and the indicating pronoun for the utterance PA11 based on the converted character information. Specify that it contains content that indicates that you wish to purchase the item. For example, the estimation unit 133 analyzes the character information of the utterance PA11 by appropriately using various conventional techniques such as syntax analysis, and thereby estimates that the user desires to purchase a product corresponding to the indicating pronoun. .

  For example, the estimation unit 133 estimates that the user desires to purchase a product corresponding to the indicating pronoun based on the second sensor information. Further, in the example of FIG. 1, since the estimation unit 133 specifies that the product designated by the user is the product IT14, the estimation unit 133 estimates that the user desires to purchase the car A that is the product IT14. .

(Determining unit 134)
The determination unit 134 makes various determinations. For example, the determination unit 134 determines, based on the second sensor information, a processing mode related to a purchase target that is a product or service specified based on the first sensor information. For example, the determination unit 134 determines to add the purchase target to the predetermined list when the user's intention estimated based on the second sensor information satisfies a predetermined condition. For example, when the user's intention estimated based on the second sensor information indicates purchase of a purchase target, the determination unit 134 determines to perform payment for the purchase target. For example, when the user's intention estimated based on the second sensor information indicates an interest in the purchase target, the determination unit 134 determines to add the purchase target to the hold list.

  In the example of FIG. 1, the determination unit 134 determines a processing mode related to the product IT14 according to the user's intention for the product IT14 estimated by the estimation unit 133. For example, since it is presumed that the user desires to purchase the car A that is the product IT14, the determination unit 134 determines to add the product IT14 to a predetermined shopping list. For example, the determination unit 134 determines to add the product IT14 to the shopping list associated with the terminal device AP11. For example, as illustrated in the shopping list information storage unit 124, the determination unit 134 adds the product IT14 to the shopping list associated with the terminal device AP11. It is determined that the product IT14 is added to the shopping list associated with the terminal device AP11. In the example of FIG. 1, the determination unit 134 determines a processing mode related to the product IT14 specified based on the first sensor information based on the second sensor information.

(Transmitter 135)
The transmission unit 135 transmits various information. For example, the transmission unit 135 transmits various types of information to an external information processing apparatus. For example, when a product is added to the shopping list in the shopping list information storage unit 124, the transmission unit 135 may transmit information regarding addition of the product to various devices used by the user at a predetermined timing. For example, the transmission unit 135 may transmit information about the product added to the shopping list to the terminal device AP11, and the terminal device AP11 may output information about the product received at a predetermined timing. For example, the terminal device AP11 may display information about the product at a timing when the user uses the terminal device AP11 again after the user finishes the action regarding shopping.

[3. Processing flow
Next, the procedure of the determination process by the determination system 1 according to the embodiment will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of the determination process according to the embodiment.

  As illustrated in FIG. 7, the first reception unit 131 of the determination device 100 receives first sensor information related to the user's product specification (step S <b> 101). In the example of FIG. 1, the 1st reception part 131 receives the 1st sensor information which shows that the user designated the car A from terminal device AP11.

  And the 2nd reception part 132 of the determination apparatus 100 receives the 2nd sensor information regarding the purchase intention of a user's goods (step S102). In the example of FIG. 1, the second receiving unit 132 receives second sensor information indicating that the user intends to purchase the car A from the terminal device AP11.

  Then, the determination part 134 of the determination apparatus 100 determines the process aspect regarding the goods or service specified based on 1st sensor information based on 2nd sensor information (step S103). In the example of FIG. 1, the determination unit 134 adds the basket A designated by the user to the shopping list in the shopping list information storage unit 124.

[4. (Modification)
For example, the determination apparatus 100A according to the modification may generate a model and estimate the user's intention using the generated model. This point will be described with reference to FIGS. In addition, about the structure similar to embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

[4-1. Configuration of decision device]
Next, the configuration of the determination apparatus 100A according to the modification will be described with reference to FIG. FIG. 8 is a diagram illustrating a configuration example of a determination device according to a modification. As illustrated in FIG. 8, the determination apparatus 100A includes a communication unit 110, a storage unit 120A, and a control unit 130A. The determination device 100A includes an input unit (for example, a keyboard and a mouse) that acquires various operations from an administrator of the determination device 100A and a display unit (for example, a liquid crystal display) that displays various types of information. May be.

(Storage unit 120A)
The storage unit 120A is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 8, the storage unit 120A according to the modification includes a device information storage unit 121, a content information storage unit 122, a product information storage unit 123, a shopping list information storage unit 124, and a model information storage unit 125.

(Model information storage unit 125)
The model information storage unit 125 according to the modification stores information related to learning. For example, the model information storage unit 125 stores a plurality of model information. FIG. 9 is a diagram illustrating an example of a model information storage unit according to the embodiment. The model information storage unit 125 illustrated in FIG. 9 includes items such as “feature 1” to “feature 3” corresponding to each model M1, M2, and the like as model information.

  For example, in the example illustrated in FIG. 9, the model information regarding the model M1 includes feature 1 weight “0.5”, feature 2 weight “−0.4”, feature 3 weight “0.2”, and the like. Indicates that Further, for example, in the example illustrated in FIG. 9, the model information regarding the model M <b> 2 has a feature 1 weight of “−0.3”, a feature 2 weight of “0.1”, and a feature 3 weight of “−0. 2 "etc. For example, when the feature (feature value) of the model is expressed by an m-dimensional vector, the number of features is m, and the weights of the features 1 to m are stored.

  The model information storage unit 125 is not limited to the above, and may store various model information according to the purpose.

(Control unit 130A)
Returning to the description of FIG. 8, the control unit 130A is a controller, and for example, various programs (corresponding to examples of a determination program and a learning program) stored in a storage device inside the determination device 100A by a CPU, an MPU, or the like. Is implemented by using the RAM as a work area. The control unit 130A is a controller, and is realized by an integrated circuit such as an ASIC or FPGA, for example.

  As shown in FIG. 8, the control unit 130A includes a first reception unit 131, a second reception unit 132, an estimation unit 133A, a determination unit 134, a transmission unit 135, and a learning unit 136. Implement or execute the functions and operations of the information processing described.

(Learning unit 136)
The learning unit 136 learns various information. For example, the learning unit 136 generates a model based on various information. For example, the learning unit 136 learns based on the user's intention estimated in the past and information on the user's behavior with respect to the product whose processing mode is determined based on the intention.

  For example, when the user's intention is estimated to be purchase intention and the user purchases a product added to the shopping list, the learning unit 136 may perform learning using the information as a positive example. For example, the learning unit 136 may perform learning using the second sensor information when the user's intention is estimated to be a purchase intention and the user has purchased a product added to the shopping list as a positive example.

  For example, the learning unit 136 may perform learning using the information as a negative example when it is estimated that the user's intention is to purchase and the user has not purchased a product added to the shopping list. For example, the learning unit 136 may perform learning by using the second sensor information as a negative example when the user's intention is presumed to be purchase and the user does not purchase a product added to the shopping list. .

  For example, when the user's intention is not intended to purchase and the user does not purchase a product that has not been added to the shopping list, the learning unit 136 may perform learning using the information as a positive example. For example, the learning unit 136 may perform learning using the second sensor information as a positive example when the user's intention is not intended to purchase and the user has not purchased a product that has not been added to the shopping list. Good.

  For example, when the user purchases a product that is presumed that the user's intention is not the purchase intention and has not been added to the shopping list, the learning unit 136 may perform learning using the information as a negative example. For example, the learning unit 136 may perform learning by using second sensor information as a negative example when the user purchases a product that is presumed that the user's intention is not the intention to purchase and has not been added to the shopping list.

  For example, the learning unit 136 may generate a model that outputs a larger value as the user's intention is more likely to be a purchase intention. The learning unit 136 may generate a model that outputs a smaller value as the user's intention is less likely to be a purchase intention. For example, the learning unit 136 generates a model that outputs a value (score) closer to 1 as the user's intention is more likely to be a purchase intention and closer to 0 as the user's intention is less likely to be a purchase intention. May be.

  For example, the learning unit 136 may generate a model for each user. For example, the learning unit 136 may generate a model for each user specified by the estimation unit 133A. For example, the learning unit 136 may generate a model for each user classification. For example, the learning unit 136 may generate a model for each of various user classifications such as males in their 30s and females in their 20s. For example, the learning unit 136 may generate a model for each user classification based on the user attribute specified by the estimation unit 133A. The learning unit 136 is not limited to a plurality of models, and may generate one model that is commonly used by all users. Note that the above is an example, and the learning unit 136 may generate a model based on any information. For example, the learning unit 136 may generate a model that outputs a score as input of the first sensor information and the second sensor information.

(Estimation unit 133A)
The estimation unit 133A estimates the user's intention using the model generated by the learning unit 136. For example, the estimation unit 133A may estimate the user's intention based on the score output by inputting the second sensor information to the model. For example, the estimation unit 133A may perform estimation of the user's intention based on the score output by inputting the second sensor information or the first sensor information to the model. For example, the estimation unit 133A uses various conventional techniques as appropriate to identify a user whose second sensor information is detected, and estimates a user's intention using a model selected according to the identified user. May be. For example, the estimation unit 133A analyzes the voice information that is the second sensor information to identify the user who made the utterance, and uses the model selected according to the identified user to estimate the user's intention. You may go.

  For example, the estimation unit 133A may estimate the user's intention as a purchase intention when the score output by the model is equal to or greater than a predetermined threshold. For example, the estimation unit 133A may estimate that the user's intention is not a purchase intention when the score output by the model is less than a predetermined threshold. Further, for example, the estimation unit 133A may estimate that the user's intention is the intention of the purchase examination when the score output by the model is within a predetermined range that is less than a predetermined threshold. In this case, for example, when the score output by the model is further lower than a predetermined range less than a predetermined threshold, the estimation unit 133A may estimate that the user's intention is not intended to purchase.

  Note that, when a model used for estimating the user's intention is acquired from an external information processing apparatus, the determination apparatus 100A may not include the learning unit 136.

[5. effect〕
As described above, the determination device 100 according to the embodiment and the determination device 100A according to the modification include the first reception unit 131, the second reception unit 132, and the determination unit 134. The 1st reception part 131 is the 1st sensor information detected by 1st sensor SN1, Comprising: The 1st sensor information regarding the designation | designated of the user with respect to the goods or service displayed on the predetermined area | region is received. The 2nd reception part 132 is 2nd sensor information detected by 2nd sensor SN2, Comprising: The 2nd sensor information regarding a user's intention with respect to the goods or service specified based on 1st sensor information is received. The determination part 134 determines the process aspect regarding the purchase target which is a goods or service specified based on 1st sensor information based on 2nd sensor information.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification specify the product or service based on the first sensor information, and based on the second sensor information related to the user's intention for the specified product or service. Can be estimated. Thereby, the determination apparatuses 100 and 100A can appropriately determine the processing mode related to the product or service.

  In addition, in the determination device 100 according to the embodiment and the determination device 100A according to the modification, the determination unit 134 determines that the purchase target is predetermined when the user's intention estimated based on the second sensor information satisfies a predetermined condition. Decide to add to the list.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification add the purchase target to the predetermined list when the user's intention estimated based on the second sensor information satisfies the predetermined condition. By determining, it is possible to appropriately determine the processing mode related to the product or service.

  Further, in the determination device 100 according to the embodiment and the determination device 100A according to the modification, the determination unit 134 determines whether the purchase target is purchased when the user's intention estimated based on the second sensor information indicates purchase of the purchase target. Decide to make a payment.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modified example perform settlement of the purchase target when the intention of the user estimated based on the second sensor information indicates purchase of the purchase target. By determining, it is possible to appropriately determine the processing mode related to the product or service.

  In addition, in the determination device 100 according to the embodiment and the determination device 100A according to the modification, the determination unit 134 determines that the purchase target is obtained when the user's intention estimated based on the second sensor information indicates interest in the purchase target. To add to the pending list.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modified example place the purchase target in the hold list when the intention of the user estimated based on the second sensor information indicates interest in the purchase target. By determining to add, the processing mode regarding goods or a service can be determined appropriately.

  In the determination device 100 according to the embodiment and the determination device 100A according to the modification, the first reception unit 131 receives first sensor information detected by an infrared sensor as the first sensor SN1.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification appropriately receive the first sensor information detected by the infrared sensor as the first sensor SN1, thereby appropriately specifying the product or service designated by the user. Can be specified. Thereby, the determination apparatuses 100 and 100A can appropriately determine the processing mode related to the product or service.

  In the determination device 100 according to the embodiment and the determination device 100A according to the modification, the second reception unit 132 receives second sensor information detected by a voice sensor as the second sensor SN2.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification receive the second sensor information detected by the voice sensor as the second sensor SN2, and thereby the user's intention for the specified product or service. Can be estimated based on the user's voice. Thereby, the determination apparatuses 100 and 100A can appropriately determine the processing mode related to the product or service.

  Further, in the determination device 100 according to the embodiment and the determination device 100A according to the modification, the first reception unit 131 relates to a user designation for one product or service among a plurality of products or services displayed in a predetermined area. The first sensor information is received.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification receive first sensor information related to the designation of a user for one product or service among a plurality of products or services displayed in a predetermined area. Thus, even when a plurality of objects are displayed, the product or service designated by the user can be appropriately specified. Thereby, the determination apparatuses 100 and 100A can appropriately determine the processing mode related to the product or service.

  Moreover, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification include estimation units 133 and 133A. The estimation units 133 and 133A estimate the user's intention with respect to the purchase target based on the second sensor information.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modified example appropriately determine the processing mode related to the product or service by estimating the user's intention for the purchase target based on the second sensor information. be able to.

  In addition, in the determination device 100 according to the embodiment and the determination device 100A according to the modification, the estimation units 133 and 133A determine the intention of the user with respect to the purchase target based on the model that varies depending on the user and the second sensor information. presume.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modified example estimate the user's intention with respect to the purchase target based on the model that varies depending on the user and the second sensor information. Alternatively, the processing mode related to the service can be appropriately determined.

  In addition, in the determination device 100 according to the embodiment and the determination device 100A according to the modification, the estimation units 133 and 133A include a user's intention estimated in the past and a product or service whose processing mode is determined based on the intention. The second sensor information is input to a model learned based on information on the user's behavior with respect to the user's behavior, and the user's intention with respect to the purchase target is estimated.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modified example relate to the user's intention estimated in the past and the user's behavior with respect to the product or service whose processing mode is determined based on the intention. By using the model learned based on the information, it is possible to appropriately determine the processing mode related to the product or service.

[6. Hardware configuration)
The determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification described above are realized by a computer 1000 having a configuration illustrated in FIG. 10, for example. FIG. 10 is a hardware configuration diagram illustrating an example of a computer that realizes the function of the determination apparatus. The computer 1000 includes a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I / F) 1500, input / output interface (I / F) 1600, and media interface (I / F) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each unit. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

  The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. The communication interface 1500 receives data from other devices via the network N and sends the data to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the network N.

  The CPU 1100 controls an output device such as a display and a printer and an input device such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. In addition, the CPU 1100 outputs the generated data to the output device via the input / output interface 1600.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads such a program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. Etc.

  For example, when the computer 1000 functions as the determination devices 100 and 100A, the CPU 1100 of the computer 1000 implements the functions of the control units 130 and 130A by executing a program loaded on the RAM 1200. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them. However, as another example, these programs may be acquired from other devices via the network N.

  As described above, some of the embodiments and modifications of the present application have been described in detail with reference to the drawings. However, these are merely examples, and various aspects can be made based on the knowledge of those skilled in the art including the aspects described in the disclosure line of the invention. It is possible to carry out the present invention in other forms that have been modified and improved.

[7. Others]
In addition, among the processes described in the above-described embodiments and modifications, all or a part of the processes described as being automatically performed can be manually performed, or are described as being performed manually. All or part of the processing can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the document and drawings can be arbitrarily changed unless otherwise specified. For example, the various types of information illustrated in each drawing is not limited to the illustrated information.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  In addition, the above-described embodiments and modifications can be combined as appropriate within a range that does not contradict processing contents.

  In addition, the “section (module, unit)” described above can be read as “means” or “circuit”. For example, the first reception unit can be read as first reception means or a first reception circuit.

DESCRIPTION OF SYMBOLS 1 Determination system 100 Determination apparatus 121 Device information storage part 122 Content information storage part 123 Merchandise information storage part 124 Shopping list information storage part 130 Control part 131 1st reception part 132 2nd reception part 133 Estimation part 134 Determination part 135 Transmission part AP11 Terminal equipment SN1 First sensor SN2 Second sensor N Network

Claims (12)

First sensor information detected by a first sensor that detects a user's instruction on a projection surface on which a product or service is projected, the first sensor information relating to a user designation for the product or service displayed on the projection surface A first reception unit for receiving
A second receiving unit that receives second sensor information relating to a user's intention with respect to a product or service identified based on the first sensor information, the second sensor information being detected by a second sensor ;
A determination unit that determines, based on the second sensor information, a processing mode related to a purchase target that is a product or service specified based on the first sensor information ;
A determination apparatus comprising: The determination unit
The determination apparatus according to claim 1, wherein when the user's intention estimated based on the second sensor information satisfies a predetermined condition, it is determined to add the purchase target to a predetermined list. The determination unit
3. The method according to claim 1, wherein when the intention of the user estimated based on the second sensor information indicates purchase of the purchase target, the purchase target is determined to be settled. Decision device. The determination unit
The user's intention estimated based on the second sensor information indicates that the purchase target is added to the holding list when the user's intention indicates interest in the purchase target. The determination apparatus according to item 1. The first reception unit
The determination apparatus according to any one of claims 1 to 4, wherein the first sensor information detected by an infrared sensor as the first sensor is received. The second reception unit
The determination apparatus according to any one of claims 1 to 5, wherein the second sensor information detected by a voice sensor as the second sensor is received. The first reception unit
7. The determination device according to claim 1, wherein first sensor information related to a user designation for one product or service among a plurality of products or services displayed on the projection plane is received. . An estimation unit that estimates a user's intention for the purchase target based on the second sensor information;
The determination apparatus according to claim 1, further comprising: The estimation unit includes
The determination apparatus according to claim 8 , wherein the intention of the user with respect to the purchase target is estimated based on a model that varies depending on the user and the second sensor information. The estimation unit includes
Inputting the second sensor information into a model learned based on a user's intention estimated in the past and information on the user's behavior with respect to a product or service whose processing mode is determined based on the intention; The determination device according to claim 9 , wherein the intention of the user with respect to the purchase target is estimated. A decision method performed by a computer,
First sensor information detected by a first sensor that detects a user's instruction on a projection surface on which a product or service is projected, the first sensor information relating to a user designation for the product or service displayed on the projection surface A first reception step of receiving
A second receiving step of receiving second sensor information relating to a user's intention with respect to a product or service identified based on the first sensor information, the second sensor information being detected by a second sensor ;
A determination step of determining a processing mode related to a purchase target, which is a product or service specified based on the first sensor information, based on the second sensor information;
The determination method characterized by including. First sensor information detected by a first sensor that detects a user's instruction on a projection surface on which a product or service is projected, the first sensor information relating to a user designation for the product or service displayed on the projection surface A first acceptance procedure for accepting
A second reception procedure for receiving second sensor information relating to a user's intention with respect to a product or service identified based on the first sensor information, the second sensor information being detected by a second sensor ;
A determination procedure for determining, based on the second sensor information, a processing mode related to a purchase target that is a product or service specified based on the first sensor information ;
A determination program characterized by causing a computer to execute.

Images