JP6546244B2 - DECISION DEVICE, DECISION METHOD, AND DECISION PROGRAM - Google Patents

Description

  The present invention relates to a determination apparatus, a determination method, and a determination program.

  In recent years, various techniques related to electronic commerce such as online shopping have been provided. For example, there is provided a technology for performing processing related to purchase based on a procedure word that is voice-recognized by a user terminal.

Japanese Patent Application Publication No. 2003-44756 JP, 2014-186214, A

  However, in the above-mentioned prior art, it is difficult to appropriately determine the processing mode regarding the product or service. For example, in the case of using a procedure word, it is necessary to set the procedure word in advance, and it is difficult to appropriately determine the processing mode regarding the product or service.

  The present application is made in view of the above, and an object of the present application is to provide a determination device, a determination method, and a determination program for appropriately determining a processing mode regarding a product or service.

  A determination apparatus according to the present application is a first reception unit that receives first sensor information regarding specification of a user for a product or service displayed in a predetermined area, which is first sensor information detected by a first sensor; A second reception unit that receives second sensor information that is the second sensor information detected by the second sensor and that is the user's intention for the product or service specified based on the first sensor information; and the second sensor information Based on the first sensor information, an estimation unit for estimating the user's intention for the product or service identified based on the first sensor information, and a processing aspect related to a purchase target that is the product or service identified based on the first sensor information And a determination unit that determines based on the estimation result of the estimation unit.

  According to one aspect of the embodiment, it is possible to appropriately determine the processing mode related to a product or service.

FIG. 1 is a diagram illustrating an example of the determination process according to the embodiment. FIG. 2 is a diagram showing an example of the configuration of the determination apparatus according to the embodiment. FIG. 3 is a diagram illustrating an example of a device information storage unit according to the embodiment. FIG. 4 is a diagram showing an example of the content information storage unit according to the embodiment. FIG. 5 is a diagram showing an example of the product information storage unit according to the embodiment. FIG. 6 is a diagram illustrating an example of the shopping list information storage unit according to the embodiment. FIG. 7 is a flowchart illustrating an example of the determination process according to the embodiment. FIG. 8 is a diagram showing a configuration example of a determination device according to a modification. FIG. 9 is a diagram illustrating an example of a model information storage unit according to a modification. FIG. 10 is a hardware configuration diagram showing an example of a computer that implements the function of the determination device.

  Hereinafter, a determination apparatus, a determination method, and an embodiment for implementing a determination program (hereinafter, referred to as “embodiment”) according to the present application will be described in detail with reference to the drawings. In addition, the determination apparatus which concerns on this application, the determination method, and the determination program are not limited by this embodiment. Moreover, the same code | symbol is attached | subjected to the same site | part in the following each embodiment, and the overlapping description is abbreviate | omitted.

(Embodiment)
[1. Decision processing]
First, the determination process according to the embodiment will be described. FIG. 1 is a diagram illustrating an example of the determination process according to the embodiment. The determination apparatus 100 illustrated in FIG. 1 provides a determination service that determines a processing mode related to a product or service based on the detected various sensor information. In addition, if goods or a service here are the objects which can be transacted, such objects may be included, and "goods or service" may only be described as "goods" below.

  As shown in FIG. 1, the determination system 1 according to the embodiment includes a terminal device AP11 and a determination apparatus 100. The terminal device AP11 and the determination apparatus 100 are communicably connected via a predetermined network. The determination system 1 illustrated in FIG. 1 may include a plurality of terminal devices AP11 and a plurality of determination devices 100.

  In the example illustrated in FIG. 1, the terminal device AP11 has a projector function, and displays the content by projecting an image of the content on a predetermined display surface. FIG. 1 shows a case where the terminal device AP11 projects the content (hereinafter also referred to as “content CT11”) identified by the content ID “CT11” on the desk top PP11 of the table. That is, the terminal device AP11 displays the content CT11 on the desktop surface PP11 by projecting the content CT11 on the desktop surface PP11. As shown in FIG. 1, the content CT11 is, for example, content that includes a plurality of products and proposes a lifestyle to a user who browses. For example, the content displayed by the terminal device AP11 is not limited to the content CT11, and may be various content such as content including a plurality of products related to a predetermined event such as Christmas or an athletic meet. As described above, the terminal device AP11 displays the contents such as a page of a magazine including a plurality of products on the basis of a predetermined concept sequentially to the user according to the user's operation. You can browse multiple products based on Further, the terminal device AP11 may display the content on any target such as a space, as long as the content can be displayed without being limited to the desk top PP11.

  Further, in the example of FIG. 1, the terminal device AP11 includes an infrared sensor as the first sensor SN1. Thus, the terminal device AP11 has a function of detecting the designation of the user for the displayed content. For example, the terminal device AP11 picks up an infrared ray emitted by the first sensor SN1 with an image sensor or the like and detects a change thereof, or detects a reflection of the infrared ray, thereby detecting information related to coordinates in the displayed content (hereinafter Coordinate information may also be detected as first sensor information. In this case, for example, the first sensor SN1 of the terminal device AP11 (hereinafter, may be simply referred to as "terminal device AP11") may have a function of emitting infrared light and a function of detecting infrared light. For example, the first sensor SN1 of the terminal device AP11 may have a function of emitting infrared light, and a function of detecting a change in infrared light by image recognition technology.

  In the example of FIG. 1, the terminal device AP11 emits infrared radiation along the desktop surface PP11 on which the content CT11 is displayed, and the infrared radiation reflected when the user performs an operation such as pointing the content CT11 with a finger or the like To detect For example, the terminal device AP11 may detect coordinate information such as orthogonal coordinates (XY coordinates) corresponding to the area where the content CT11 is displayed on the desktop surface PP11. For example, the terminal device AP11 may detect coordinate information of XY coordinates with the horizontal direction of the content CT11 in FIG. 1 as the X-axis direction and the vertical direction as the Y-axis direction. In this case, for example, the terminal device AP11 may detect coordinate information of XY coordinates with the lower left position of the content CT11 in FIG. 1 as the origin (0, 0). Note that the above is an example, and the terminal device AP11 may detect the position at which the user has designated the displayed content, using various conventional techniques as appropriate.

  Thus, the terminal device AP11 or the determination device 100 can specify the product specified by the user based on which product is displayed at the position where the displayed content and the detected coordinate information are specified by the user. . The above is an example, and the terminal device AP11 may detect the designation of the user using various conventional techniques as appropriate. The first sensor SN1 is not limited to the infrared sensor, and any sensor may be used as long as it can detect the user's designation of the displayed content. For example, the terminal device AP11 transmits, as first sensor information, coordinate information on the designation of the user for the content detected by the first sensor SN1 to the determination device 100.

  The terminal device AP11 also has a voice sensor as a second sensor SN2. Thus, the terminal device AP11 has a voice recognition function, detects surrounding sounds, and performs various processing in accordance with the detected sounds. Below, the information detected by 2nd sensor SN2 may be described as 2nd sensor information. Moreover, below, 2nd sensor SN2 of terminal device AP11 may only be described as "terminal device AP11." The terminal device AP11 may have a voice recognition function. In this case, the terminal device AP11 also refers to information on the intention of the user who made the utterance estimated based on the information on the user's utterance detected by the second sensor SN2 (hereinafter, also simply referred to as "intention" or "intention information" ) May be estimated.

  Further, the terminal device AP11 can transmit and receive information with an external device such as the determination device 100. For example, the terminal device AP11 receives information on the content to be displayed from the determination device 100, and transmits information on detection of various sensors to the determination device 100. Here, the terminal device AP11 may be able to transmit and receive information to and from the determination device 100 via another device. For example, the terminal device AP11 may be a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), etc. It may be a device on which the application is installed.

  Note that processing such as display processing of content by the terminal device AP11 may be realized by JavaScript (registered trademark) or the like, for example. In addition, when the display processing described above is performed by a predetermined application or when the display processing is performed by a dedicated application, the terminal device AP11 may have, for example, an application control unit that controls the predetermined application or the dedicated application. .

  The determination system 1 may have any configuration as long as the determination apparatus 100 can acquire information used for the determination process, and an apparatus having each of the functions of the terminal device AP11 may be included. For example, in the determination system 1, each function of the terminal device AP11 may be realized by a device that displays content, or each device such as the first sensor SN1 or the second sensor SN2. Also, for example, in the determination system 1, the determination device 100 may have the function of the terminal device AP11. In this case, the determination device 100 and the terminal device AP11 may be integrated. In addition, the determination apparatus 100 may have any form as long as it can realize the function related to the determination process, and is, for example, integrated with a table having the desk top PP11 or distributed to a plurality of information processing apparatuses It may be realized by

  The determination apparatus 100 provides a determination service that determines the processing mode related to the item identified based on the first sensor information based on the second sensor information. Further, the determination device 100 may be an information processing device that provides the terminal device AP11 with the content displayed by the terminal device AP11. In addition, the determination apparatus 100 may be able to acquire information from an information processing apparatus that provides a content distribution service. In this case, the determination system 1 may include, for example, an information processing apparatus that provides a content distribution service.

  Also, for example, the determination device 100 may be an information processing device that provides an electronic commerce service such as online shopping. In addition, the determination device 100 may be able to obtain information from an information processing device that provides an electronic commerce service. In this case, the determination system 1 may include, for example, an information processing apparatus that provides an electronic commerce service such as online shopping.

  In addition, the determination device 100 may be capable of acquiring information from a speech recognition server that provides a speech recognition service. In this case, the decision system 1 may include a speech recognition server. In the example of FIG. 1, the description will be omitted as the determination apparatus 100 and the voice recognition server recognize various user's utterances or identify the uttered user by using various conventional techniques as appropriate. Also, for example, the determination apparatus 100 estimates the user's intention information based on the second sensor information received from the terminal device AP11.

  Hereinafter, an example of the determination process will be described with reference to FIG. In the example illustrated in FIG. 1, the terminal device AP11 is disposed in a house in which the user U11, the user U12, and the user U13 reside. In the following description, when the user U11, the user U12, and the user U13 are described without distinction, they may be simply described as "user". For example, the terminal device AP11 is arranged on a table in a house, and shows the case where the content CT11 is displayed on the desk top PP11 of the arranged table.

  In the example of FIG. 1, the terminal device AP11 has already acquired the content CT11 from an external device such as the determination device 100, and the terminal device AP11 displays the content CT11 on the table top PP11 of the table according to the user's operation or the like. It shall be done. Specifically, it is assumed that the terminal device AP11 displays the content CT11 in the area RG1 in the desk top PP11. Further, in FIG. 1, the area indicated by the code “RG1” overlaps the area in which the content CT11 is illustrated. Further, the terminal device AP11 displays not only the content CT11 but also various contents on the table top PP11. For example, the terminal device AP11 displays content other than the content CT11 on the desk top PP11 in accordance with the user's operation to change the display. For example, the terminal device AP11 may change the content displayed on the desk top PP11 in accordance with an operation of the user moving the finger FG1 in a predetermined direction on the area RG1 of the desk top PP11.

  Further, as shown in FIG. 1, the content CT11 displayed on the desk top PP11 includes five products IT11 to IT15. In addition, when it describes as "commodity IT * (* is arbitrary numerical value)" in this way, it shows that the goods are goods identified by goods ID "IT *." For example, when "commodity IT11" is described, the product is a product identified by the product ID "IT11".

  For example, the product IT11 corresponds to the product "shampoo A" included in the content CT11. Further, for example, the product IT12 corresponds to the product “refill of shampoo A” included in the content CT11. Further, for example, the product IT13 corresponds to the product "towel A" included in the content CT11. Further, for example, the product IT14 corresponds to the product "car A" included in the content CT11. Further, for example, the product IT15 corresponds to the product “pot A” included in the content CT11. Thus, FIG. 1 shows the case where the users U11 to U13 browse the content CT11 including various products.

  First, the user U12 who is browsing the content CT11 specifies the product IT14 in the content CT11 with the finger FG1 (step S11). For example, the user U12 designates a position at which the product IT14 is included in the content CT11 with the finger FG1 at the date and time DT11, or touches the portion of the desk top PP11 where the product IT14 is displayed. specify.

  Then, in response to designation by the user U12 with the finger FG1, the first sensor SN1 of the terminal device AP11 detects, as first sensor information, coordinate information of a position designated by the user U12 with the finger FG1 (step S12). In the example of FIG. 1, the terminal device AP11 detects coordinate information of the position designated by the user U12 with the finger FG1. Specifically, the terminal device AP11 detects coordinate information indicating that the position designated by the user U12 with the finger FG1 is the position (x1, y1). Here, “x1” and “y1” are arbitrary values corresponding to positions included in the range in which the product IT14 in the content CT11 is displayed.

  Then, the terminal device AP11 transmits the detected first sensor information to the determination apparatus 100 (step S13). Thereby, the determination apparatus 100 receives the first sensor information from the terminal device AP11. In the example of FIG. 1, the terminal device AP11 displays first sensor information indicating that the position designated by the user is the position (x1, y1), and information indicating that the content designated by the user is the content CT11. Send. In addition, when the information regarding which content the terminal device AP11 is displaying is already acquired at the time of designation | designated by the user by the determination apparatus 100, it is not necessary to transmit the information which identifies the content designated by the user.

  Then, the determination device 100 that has received the first sensor information from the terminal device AP11 specifies the product specified by the user based on the first sensor information. In the example of FIG. 1, the determination apparatus 100 specifies the product designated by the user based on the received first sensor information and the content information stored in the content information storage unit 122.

  In the example of FIG. 1, as shown in the content information storage unit 122, the determining apparatus 100 stores the product included in each content and information indicating in which range in the content the product is located. For example, as illustrated in the content information storage unit 122, the determining device 100 stores information indicating that the content CT11 includes five products of the products IT11 to IT15. Further, the content information storage unit 122 stores information indicating that the product IT11 is located in the range AR11 in the content CT11. The range AR11 may be any information that indicates the range occupied by the product IT11 in the content CT11, and may be, for example, coordinate information.

  Further, the content information storage unit 122 stores information indicating that the product IT12 is located in the range AR12 in the content CT11. Further, the content information storage unit 122 stores information indicating that the product IT13 is located in the range AR13 in the content CT11. Further, the content information storage unit 122 stores information indicating that the product IT14 is located in the range AR14 in the content CT11. Further, the content information storage unit 122 stores information indicating that the product IT15 is located in the range AR15 in the content CT11.

  In the example of FIG. 1, the determination apparatus 100 specifies the product designated by the user based on the first sensor information and the ranges AR11 to AR15 of the respective products IT11 to IT15 stored in the content information storage unit 122 ( Step S14). For example, when the coordinates (x1, y1) of the first sensor information are included in any of the ranges AR11 to AR15, the determination device 100 specifies the item designated by the user. In the example of FIG. 1, since the coordinates (x1, y1) of the first sensor information are included in the range AR14, the determining apparatus 100 specifies that the product designated by the user is the product IT14. Further, based on the correspondence between the product ID and the product name stored in the product information storage unit 123, the determination apparatus 100 specifies that the product IT14 is the product name "car A".

  Thereafter, the user U12 performs an utterance PA11 "I want this" in the vicinity of the terminal device AP11 (the second sensor SN2) (step S15). For example, at a date and time DT12 after the date and time DT11, the user U12 performs an utterance PA11 of "I want this" for the product IT14 specified in step S11. Although the example in FIG. 1 shows that the user U12 continues to designate the product IT14 with the finger FG1 also in step S15, if the product designated by the user U12 has already been identified, the user U12 does not It may be in the state where specification of goods IT14 with FG1 is cancelled.

  Then, in response to the user U12 having made the utterance PA11, the second sensor SN2 of the terminal device AP11 detects voice information of the utterance PA11 by the user U12 as second sensor information (step S16). In the example of FIG. 1, the terminal device AP11 detects voice information corresponding to the utterance PA11 "I want this" by the user U12. Although only the speech PA11 is illustrated in the example of FIG. 1 to simplify the description, the users U11 to U13 may perform various conversations regarding the product IT14. The second sensor SN2 of the terminal device AP11 may also detect an utterance other than the utterance PA11 as the second sensor information.

  Then, the terminal device AP11 transmits the detected second sensor information to the determination apparatus 100 (step S17). Thereby, the determination apparatus 100 receives the second sensor information from the terminal device AP11. In the example of FIG. 1, the terminal device AP11 transmits second sensor information related to the user's speech. Specifically, the terminal device AP11 transmits the voice information of the utterance PA11 as the second sensor information.

  The terminal device AP11 may transmit the voice information of the utterance PA11 to the voice recognition server, acquire the character information of the utterance PA11 from the speech recognition server, and transmit the acquired character information to the determination apparatus 100. In addition, the terminal device AP11 may transmit, to the determination apparatus 100, information indicating that the user who has performed the acquired utterance PA11 is the user U12. When the terminal device AP11 estimates the intention of the utterance, the terminal device AP11 may transmit the estimated intention of the utterance to the determination device 100.

  The determination apparatus 100 that has received the second sensor information from the terminal device AP11 estimates the user's intention based on the second sensor information (step S18). For example, the determination apparatus 100 analyzes the received second sensor information to estimate that the user desires to purchase the product IT14 identified in step S14. For example, the determination apparatus 100 converts the voice information of the utterance PA11, which is the received second sensor information, into character information, and estimates the user's intention based on the converted character information. For example, the determination apparatus 100 specifies that the utterance PA11 includes an indicative pronoun "this" and an adjective "want" indicating that it desires to purchase a product corresponding to the indicative pronoun. For example, the determination apparatus 100 estimates that the user desires to purchase a product corresponding to the designated pronoun by appropriately analyzing the character information of the utterance PA11 using various conventional techniques such as syntactic analysis. . Thus, the determination apparatus 100 estimates that the user has an intention to purchase the product IT14 based on the content of the utterance PA11.

  Thus, in step S18, the determination apparatus 100 estimates that the user desires to purchase a product corresponding to the designated pronoun based on the second sensor information. Then, in step S14, the determination apparatus 100 specifies that the product designated by the user is the product IT14. Therefore, the determination apparatus 100 estimates that the user desires to purchase the car A, which is the product IT14.

  Then, the determination apparatus 100 determines the processing mode related to the product IT14 according to the estimated user's intention for the product IT14. In the example of FIG. 1, since it is estimated that the user desires to purchase the car A which is the product IT14, the determination apparatus 100 determines to add the product IT14 to a predetermined shopping list (step S19). . Specifically, the determination apparatus 100 determines to add the product IT14 to the shopping list associated with the terminal device AP11. Then, as illustrated in the shopping list information storage unit 124, the determination apparatus 100 adds the commodity IT14 to the shopping list associated with the terminal device AP11. In the example of FIG. 1, the determination apparatus 100 determines to add the product IT14 to the shopping list associated with the terminal device AP11.

  Thus, the determination apparatus 100 determines the processing mode related to the product IT 14 identified based on the first sensor information based on the second sensor information. Further, in the example of FIG. 1, the determining apparatus 100 determines that the product IT14 is to be added to the shopping list associated with the terminal device AP11, whereby the plurality of users U11 to U13 talk, and the purchase of the product by any user Even if it is not possible to identify the intention of the item, it is possible to appropriately add the item to the shopping list. Further, the determination device 100 determines that the product IT 14 is added to the shopping list associated with the terminal device AP11, so that the user who has specified the product is different from the user who has made an utterance regarding the product, Products can be added to the shopping list appropriately. As described above, the determination apparatus 100 appropriately determines the processing mode related to the product even when the plurality of users browse the content while having a conversation and perform the specification of the product or the utterance indicating the intention for the product. can do. In addition, the determination apparatus 100 may transmit information on a product added to the shopping list associated with the terminal device AP11 to the user at a predetermined timing.

  Further, in the example of FIG. 1, for the sake of simplifying the description, the processing mode for the product IT14 is shown to be determined, but the users U11 to U13 specify various products or display following the product IT14 You may have a conversation about the content being Then, the determination apparatus 100 may sequentially determine the processing mode related to the product based on the specification of the product by the user and the conversation. For example, if the user specifies the product IT11 after step S19 and continues the conversation, the determination apparatus 100 may determine the processing mode regarding the product IT11. For example, when the user designates the product IT15 after determining the processing mode related to the product IT11 and continues the conversation, the determining apparatus 100 may determine the processing mode related to the product IT15. In addition, when the user switches the display to content other than the content CT11 and continues the conversation, the determination apparatus 100 may determine the processing mode related to the product designated by the user based on the user's designation or the conversation. .

  Further, although the example of FIG. 1 shows the case where the product IT14 is added to the shopping list associated with the terminal device AP11 in order to simplify the description, the determining apparatus 100 relates to the estimated user purchasing Depending on the intention, the processing mode may be different. For example, the determination apparatus 100 may change the processing mode depending on the estimated level (weakness) of the user's purchase intention. For example, when it is estimated that the purchase intention of the user is high based on the user's utterance such as “immediately want” or “absolutely buy”, the determination apparatus 100 immediately purchases the specified product, that is, performs the payment process. You may go. In the example of FIG. 1, the determination apparatus 100 estimates that the purchase intention of the user U12 is high because the utterance PA11 includes the content desired to be purchased, and the purchase instruction IT14 is immediately purchased, that is, the settlement is performed. Processing may be performed. In this case, the determination apparatus 100 may perform the settlement process using the account of the user U12 or the settlement process using the account ACN11 (see FIG. 3) of the user U11 associated with the terminal device AP11. You may go. Also, for example, the determination device 100 is designated when it is estimated that the user has low purchase intention but is interested in the product based on the user's utterance such as "Good" or "It is anxious". Items may be added to the pending list, shopping list, etc. Thus, the determination apparatus 100 may change the processing mode for the designated product based on the level of the purchase intention of the user estimated based on the user's utterance.

  As described above, the determining apparatus 100 identifies the item designated by the user based on the first sensor information, and processes the item (purchasing object) identified based on the first sensor information based on the second sensor information. Determine the aspect. As described above, the determination apparatus 100 identifies the product designated by the user based on the first sensor information, and estimates the user's intention for the product based on the second sensor information. Thus, the determination apparatus 100 divides the processing into two steps of product specification and estimation of the user's intention, thereby estimating the user's specified accuracy of the product specified and the user's intention for the product. Accuracy can be improved. Therefore, the determination apparatus 100 can appropriately determine the processing mode related to the product or service.

  In the example described above, the determination apparatus 100 uses the information detected by the infrared sensor as the first sensor information, and uses the information detected by the voice sensor as the second sensor information. Thus, the determination apparatus 100 can improve the accuracy of specifying the product specified by the user and the accuracy of estimating the user's intention for the product by using information detected by different sensors in each process. .

  In the example of FIG. 1, the determination apparatus 100 enhances the accuracy of specifying the product specified by the user by specifying the product displayed at the position indicated by the user with the finger FG1 or the like as the product specified by the user. Can. For example, in the case where a product designated by the user is specified based on voice information, it is difficult to specify the product designated by the user when uttered with a designated pronoun such as “this” or “that”. Further, in the example of FIG. 1, since the product IT11 "Shampoo A" and the product IT12 "Shampoo A Refill" are included in the content CT11, the user utters "Shampoo A must buy" Even in this case, it is difficult to specify which of the product IT11 and the product IT12 is specified. On the other hand, in the example of FIG. 1, the determining apparatus 100 specifies the product displayed at the position indicated by the user with the finger FG1 or the like as the product specified by the user, whereby the product specified by the user is the product IT11 and the product The specific accuracy of either IT12 can be increased.

  In addition, the determination apparatus 100 may specify the product specified by the user based on the voice information, when the product specified by the user can be specified by the voice information. As described above, the determination apparatus 100 may identify the product designated by the user or estimate the user's intention for the product based on the content of the information detected by each sensor. In addition, as illustrated in FIG. 1, the determination apparatus 100 may receive only the second sensor information regarding an utterance performed while the user U12 continues specifying the product IT14 with the finger FG1. Thus, the determination apparatus 100 can more accurately associate the user's intention with the product specified by the user.

  In the example described above, although the infrared sensor is used as the first sensor SN1 and the product to be designated is specified based on the instruction by the finger FG1 of the user, the determination apparatus 100 may use various sensors as the first sensor. Good. For example, the determination apparatus 100 may use a motion sensor or the like that detects a user's gesture or the like as the first sensor. In this case, the determination device 100 may specify a product to be specified based on a user's gesture or the like. Also, for example, the determination device 100 may use a sensor or the like that detects the line of sight of the user as the first sensor. In this case, the determination apparatus 100 may specify the product to be specified based on the line of sight of the user. In addition, the determination device 100 may specify a product to be designated based on a plurality of first sensor information detected by a plurality of first sensors using the plurality of sensors as described above as the first sensor.

  Further, in the above-described example, the determination apparatus 100 may specify that the user who has performed the utterance PA11 is the user U12 by analyzing the second sensor information that is voice information of the received utterance PA11. For example, the determination apparatus 100 may specify that the user who has performed the utterance PA11 is the user U12 by analyzing the audio information of the utterance PA11 based on the audio information or the like associated with each user. . Note that the determination device 100 may identify the user who has made the utterance PA11, or another device may identify the user. When another device specifies the user who has made the utterance PA11, the determining device 100 acquires information for specifying the user who made the utterance PA11 from the other device. Thus, the determination device 100 can improve the accuracy of estimation of the user's intention by specifying the user who made the utterance. For example, the determination apparatus 100 may estimate the user's intention by selectively using the model according to the specified user, and the details of this point will be described later.

  In addition, when the second sensor information includes information related to the number, the determination device 100 may determine the processing mode related to the designated product based on the information related to the number. In the example of FIG. 1, when the second sensor information is voice information including information on the number of “three want this” etc., the determination device 100 adds the item IT14 to the shopping list associated with the terminal device AP11. You may add three.

  Further, although the example described above shows a case where the determination apparatus 100 adds the shopping list associated with the terminal device AP11, that is, the product IT14 specified in the shopping list associated with the device, the determination apparatus 100 Various processing modes may be determined as appropriate. For example, the determination device 100 may add the designated product IT14 to the shopping list associated with the user. For example, as illustrated in the device information storage unit 121 (see FIG. 3), the determination apparatus 100 may add the product IT14 designated in the shopping list of the user U11 who is the user associated with the terminal device AP11. .

  Also, for example, when it is possible to specify that the user who has made the utterance PA11 is the user U12, the determination apparatus 100 may add the product IT14 to the shopping list of the user U12. In the example of FIG. 1, when the determining apparatus 100 can specify that the user who has made the utterance PA11 is the user U12, it adds the product IT14 to the shopping list of the user U12 and can not specify the user who has made the utterance PA11. The commodity IT14 may be added to the shopping list of the user U11 associated with the terminal device AP11. In this manner, the determination apparatus 100 may determine the processing mode related to the designated product based on various information including the second sensor information.

  Also, for example, when the intention of the user estimated based on the second sensor information indicates the purchase of a product, the determination apparatus 100 may determine to settle the product. In the example of FIG. 1, the determination apparatus 100 may determine to settle the product IT14 because the purchase intention for the specified product IT14 is estimated. In this case, the determination apparatus 100 determines that the user who has made the utterance PA11 can specify that the user U12 is to make a payment using the account of the user U12, and can not specify the user who made the utterance PA11, the terminal It may be determined to make a settlement using the account ACN 11 (see FIG. 3) of the user U11 associated with the device AP11.

  Also, for example, when the intention of the user estimated based on the second sensor information indicates an interest in a product, the determination device 100 may determine to add the product to the hold list. For example, when a product is specified, the determination apparatus 100 does not reach the intention of purchase such as “This is good” or “I will buy someday,” and the like, and the determination apparatus 100 determines that the user indicates an interest in the product. May decide to add the item to the hold list. For example, the determination apparatus 100 may add the product to the hold list associated with the device. For example, when the determination device 100 can identify the user who made the utterance, the determination device 100 may add the item to the user's hold list. Also, for example, when the determination device 100 can not specify the user who made the utterance, the determination device 100 may add the product to the hold list of the user associated with the device.

  Further, the determination apparatus 100 or the terminal device AP11 may change the display mode of the content displayed by the terminal device AP11 in accordance with the user's utterance. In the example of FIG. 1, the content CT <b> 11 displayed on the desktop surface PP <b> 11 may change in accordance with the user's utterance. For example, when the user U13 designates the white pot A which is the product IT15 and utters “this red guy”, the determination apparatus 100 or the terminal device AP11 selects the white pot in the content CT11 displayed by the terminal device AP11 A may be changed to a red pot A. For example, when the terminal device AP11 has a voice recognition function and has previously acquired images (content) corresponding to each color of each product, designation of the white pot A by the user U13, and "the red guy of this" , And may change the display of the white pot A in the content CT11 to the red pot A. For example, the determination apparatus 100 causes the terminal device AP11 to change the display of the white pot A in the content CT11 to the red pot A according to the specification of the white pot A by the user U13 and the utterance of “the red guy of this”. You may instruct to For example, the determination device 100 transmits an image (content) corresponding to the red pot A to the terminal device AP11 according to the specification of the white pot A by the user U13 and the utterance of “the red guy of this”, and the desk surface The white pot A displayed in PP11 may be instructed to change the display to the red pot A.

  Further, for example, when the user U11 designates the towel A which is the product IT11 and utters “this is a different color”, the determination apparatus 100 or the terminal device AP11 is white in the content CT11 displayed by the terminal device AP11. The display of towel A may be changed to towel A of another color. For example, when the user U11 designates the towel A which is the product IT11 and utters “this is a different color”, the terminal device AP11 is a white towel A in another color (for example, blue) to which the towel A can correspond. The display may be changed on the towel A. The terminal device AP11 may perform the above-described change in display using various techniques as appropriate. For example, the terminal device AP11 may obtain (image) data corresponding to each color of each product in advance and switch data to be displayed. Also, for example, the terminal device AP11 may change the display mode of the content by superimposing it on an overlay or the like and displaying it.

[2. Configuration of determination device]
Next, the configuration of the determination apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram showing an example of the configuration of the determination apparatus according to the embodiment. As illustrated in FIG. 2, the determination apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The determination apparatus 100 includes an input unit (for example, a keyboard or a mouse) that receives various operations from the administrator of the determination apparatus 100 or the like, and a display unit (for example, a liquid crystal display or the like) for displaying various information. May be

(Communication unit 110)
The communication unit 110 is realized by, for example, a network interface card (NIC). The communication unit 110 is connected to the network by wire or wirelessly, and transmits and receives information to and from the terminal device AP11. In addition, the communication unit 110 may transmit and receive information to and from an information processing apparatus that provides an electronic commerce service or a voice recognition server.

(Storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 according to the embodiment has a device information storage unit 121, a content information storage unit 122, a product information storage unit 123, and a shopping list information storage unit 124, as shown in FIG.

(Device information storage unit 121)
The device information storage unit 121 according to the embodiment stores various information related to various devices such as the terminal device AP11. For example, the device information storage unit 121 stores various information on various devices used to collect the first sensor information and the second sensor information. FIG. 3 shows an example of the device information storage unit 121 according to the embodiment. The device information storage unit 121 illustrated in FIG. 3 has items such as “device ID”, “user ID”, and “account ID”.

  “Device ID” indicates identification information for identifying a device. Also, “user ID” indicates identification information for identifying a user. For example, the “user ID” indicates identification information for identifying the user associated with the device. For example, each device ID may be associated with a plurality of user IDs. "Account ID" indicates identification information for identifying an account. For example, “account ID” indicates identification information for identifying an account associated with the user.

  For example, in the example illustrated in FIG. 3, it is indicated that the device (terminal device AP11) identified by the device ID “AP11” is associated with the user (user U11) identified by the user ID “U11”. . Also, the user (user U11) identified by the user ID "U11" indicates that the account identified by the account ID "ACN11" is associated.

  The device information storage unit 121 is not limited to the above, and may store various information according to the purpose. The device information storage unit 121 may store information on the limit amount. For example, the device information storage unit 121 may store information on past payment amounts.

(Content information storage unit 122)
The content information storage unit 122 according to the embodiment stores various information related to content. For example, the content information storage unit 122 stores various information related to the content displayed by a device having display means such as the terminal device AP11. FIG. 4 is a diagram showing an example of the content information storage unit according to the embodiment. The content information storage unit 122 illustrated in FIG. 4 includes items such as “content ID”, “product ID”, and “range”.

  "Content ID" indicates identification information for identifying content. For example, the content identified by the content ID “CT11” corresponds to the content CT11 shown in the example of FIG. "Product ID" indicates identification information for identifying a product. Further, “range” indicates a position in the content at which the product identified by the corresponding product ID is included. In the example shown in FIG. 4, “range” is indicated by an abstract code such as AR 11, but “range” may be coordinates etc. indicating the position in the content of the product identified by the corresponding product ID Good.

  For example, in the example illustrated in FIG. 4, the content identified by the content ID “CT11” includes the item identified by the item ID “IT11”, the item identified by the item ID “IT12”, and the item ID “IT13”. It indicates that the product identified by “”, the product identified by the product ID “IT14”, and the product identified by the product ID “IT15” are included. Also, for example, in the example illustrated in FIG. 4, the item identified by the item ID “IT11” indicates that the item is located in the range AR11 in the content CT11. For example, the range AR11 corresponds to a predetermined range located substantially at the center of the content CT11.

  Further, for example, in the example illustrated in FIG. 4, the item identified by the item ID “IT12” indicates that the item is located in the range AR12 in the content CT11. Further, for example, in the example illustrated in FIG. 4, the item identified by the item ID “IT13” indicates that the item is located in the range AR13 in the content CT11. Also, for example, in the example illustrated in FIG. 4, the item identified by the item ID “IT14” indicates that the item is located in the range AR14 in the content CT11. Also, for example, in the example illustrated in FIG. 4, the item identified by the item ID “IT15” indicates that the item is located in the range AR15 in the content CT11.

  The content information storage unit 122 is not limited to the above, and may store various information according to the purpose.

(Product information storage unit 123)
The product information storage unit 123 according to the embodiment stores various pieces of information on the product. FIG. 5 is a diagram showing an example of the product information storage unit according to the embodiment. The product information storage unit 123 illustrated in FIG. 5 has items such as “product ID” and “product name”.

  "Product ID" indicates identification information for identifying a product. "Product name" indicates a specific name or the like of a product or service.

  Further, in the example shown in FIG. 5, it indicates that the product identified by the product ID “IT11” is “shampoo A”. Further, in the example shown in FIG. 5, it is indicated that the product identified by the product ID “IT12” is “Shampoo A Refill”, that is, a product for refilling shampoo A. Moreover, in the example shown in FIG. 5, it shows that the goods identified by goods ID "IT13" are "towel A". Moreover, in the example shown in FIG. 5, it shows that the goods identified by goods ID "IT14" are "car A." Moreover, in the example shown in FIG. 5, it shows that the goods identified by goods ID "IT15" are "pot A."

  The product information storage unit 123 is not limited to the above, and may store various information according to the purpose. For example, the product information storage unit 123 may store information on the price of the product. For example, the product information storage unit 123 may store information on stores (stores) that handle products or information on inventory.

(Shopping list information storage unit 124)
The shopping list information storage unit 124 according to the embodiment stores various information related to shopping (purchase). For example, the shopping list information storage unit 124 stores various information on a predetermined product. FIG. 6 is a diagram illustrating an example of the shopping list information storage unit according to the embodiment. The shopping list information storage unit 124 illustrated in FIG. 6 has items such as “target ID” and “shopping list”.

  "Object ID" indicates information identifying an object to which the list of products is associated. The “target ID” may be an ID for identifying a terminal device such as “AP11” or an ID for identifying a user such as “U1”, as long as it is an ID for identifying a target associated with a list of products. It may include an ID identifying such an object. The “shopping list” indicates information on the product associated with the target. For example, the “shopping list” indicates the correspondence between the item identified based on the first sensor information and the object.

  The "shopping list" includes items such as "goods" and "date and time". For example, “merchandise” indicates information identifying a product for which settlement has been made in the group. Also, “date and time” indicates the date and time when the settlement of the corresponding product was performed. The “date and time” may be the date and time when the corresponding product was input.

  The “shopping list” may include an item such as “input user”. The “input user” in the “shopping list” may be a user who has specified a corresponding product or may be a user who has indicated an intention for the corresponding product. For example, the “input user” in the “shopping list” may be a user whose first sensor information has been detected or a user whose second sensor information has been detected.

  For example, in the example illustrated in FIG. 6, the target identified by the target ID "AP11" indicates that the terminal device AP11 identified by "AP11". For example, in the example illustrated in FIG. 6, as illustrated in the “shopping list”, it indicates that the item identified by the item ID “IT1” has been added to the shopping list corresponding to the terminal device AP11 at the date “DT1”. Further, for example, it indicates that the product identified by the product ID “IT14” has been added to the shopping list corresponding to the terminal device AP11 at the date and time “DT12”.

  In addition, shopping list information storage part 124 may memorize not only the above but various information according to the object. For example, the shopping list information storage unit 124 may store information on the amount of goods. For example, the shopping list information storage unit 124 displays, in addition to the "shopping list", a "settled list" indicating information on the item for which settlement has been performed, and information on the item for which settlement is suspended by the user operation. The "pending list" may be stored.

(Control unit 130)
Returning to the description of FIG. 2, the control unit 130 is a controller, and is stored in a storage device in the determination device 100 by, for example, a central processing unit (CPU) or a micro processing unit (MPU). Various programs (corresponding to an example of the determination program) are realized by executing the RAM as a work area. The control unit 130 is a controller, and is realized by, for example, an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  As shown in FIG. 2, the control unit 130 includes a first reception unit 131, a second reception unit 132, an estimation unit 133, a determination unit 134, and a transmission unit 135, and the determination process described below Realize or execute the function or action of The internal configuration of the control unit 130 is not limited to the configuration illustrated in FIG. 2, and may be another configuration as long as it performs a determination process described later.

(1st reception part 131)
The first accepting unit 131 accepts various information. For example, the first reception unit 131 receives various information from an external information processing apparatus. For example, the first reception unit 131 receives the first sensor information which is the first sensor information detected by the first sensor SN1 and which relates to the specification of the user for the product or service displayed in the predetermined area. For example, the first reception unit 131 receives the first sensor information detected by the infrared sensor as the first sensor SN1. For example, the first reception unit 131 receives first sensor information regarding specification of the user for one product or service among products or services displayed in a plurality of predetermined areas.

  In FIG. 1, the first reception unit 131 receives the first sensor information from the terminal device AP11. In the example of FIG. 1, the first reception unit 131 indicates that the first sensor information indicating that the position designated by the user is the position (x1, y1) or that the content designated by the user is the content CT11. Information is received from the terminal device AP11. Also, for example, the first reception unit 131 may receive only the first sensor information indicating that the position designated by the user is the position (x1, y1) from the terminal device AP11.

(Second acceptance unit 132)
The second accepting unit 132 accepts various information. For example, the second accepting unit 132 accepts various information from an external information processing apparatus. For example, the second accepting unit 132 is the second sensor information detected by the second sensor SN2, and accepts the second sensor information regarding the intention of the user for the product or service specified based on the first sensor information. For example, the second reception unit 132 receives the second sensor information detected by the voice sensor as the second sensor SN2.

  In FIG. 1, the second receiving unit 132 receives second sensor information from the terminal device AP11. In the example of FIG. 1, the second reception unit 132 receives, from the terminal device AP11, second sensor information related to the user's speech. For example, the second accepting unit 132 accepts voice information of the utterance PA11 from the terminal device AP11 as second sensor information. The first reception unit 131 and the second reception unit 132 may be integrated.

(Estimation unit 133)
The estimation unit 133 estimates and specifies various information. For example, the estimation unit 133 estimates the user's intention for the purchase target based on the second sensor information. For example, the estimation unit 133 may estimate the intention of the user with respect to the purchase target based on the model and the second sensor information that change according to the user. For example, the estimation unit 133 may use the second learning model based on the user's intention estimated in the past and information on the user's behavior with respect to the product or service whose processing mode has been determined based on the intention. By inputting sensor information, the user's intention for the purchase target may be estimated. The details of this point will be described later.

  In the example of FIG. 1, the estimation unit 133 specifies the product designated by the user based on the first sensor information from the terminal device AP11. For example, the estimation unit 133 specifies the product designated by the user based on the first sensor information and the content information stored in the content information storage unit 122. For example, the estimation unit 133 specifies the product designated by the user based on the first sensor information and the ranges AR11 to AR15 of the products IT11 to IT15 stored in the content information storage unit 122. For example, when the coordinates (x1, y1) of the first sensor information are included in any of the ranges AR11 to AR15, the estimation unit 133 specifies the product specified by the user. In the example of FIG. 1, since the coordinates (x1, y1) of the first sensor information are included in the range AR14, the estimation unit 133 specifies that the product designated by the user is the product IT14. Also, for example, based on the correspondence between the product ID stored in the product information storage unit 123 and the product name, the estimation unit 133 specifies that the product IT14 is the product name “car A”.

  Moreover, in the example of FIG. 1, the estimation part 133 estimates a user's intention based on the 2nd sensor information from terminal device AP11. For example, the estimation unit 133 analyzes the received second sensor information to estimate that the user desires to purchase the product IT14 specified based on the first sensor information. For example, the estimation unit 133 converts the voice information of the utterance PA11 which is the second sensor information into character information, and based on the character information after conversion, the utterance PA11 corresponds to a designated pronoun “this” and its designated pronoun Identify the content that indicates that you want to purchase the product. For example, the estimation unit 133 estimates that the user desires to purchase a product corresponding to the designated pronoun by appropriately analyzing the character information of the utterance PA11 using various conventional techniques such as syntactic analysis. .

  For example, based on the second sensor information, the estimation unit 133 estimates that the user desires to purchase a product corresponding to the designated pronoun. Further, in the example of FIG. 1, since the estimation unit 133 specifies that the product specified by the user is the product IT14, it is estimated that the user desires to purchase the car A which is the product IT14. .

(Determination unit 134)
The determination unit 134 performs various determinations. For example, the determination unit 134 determines, based on the second sensor information, a processing mode related to a purchase target that is a product or service specified based on the first sensor information. For example, when the intention of the user estimated based on the second sensor information satisfies a predetermined condition, the determination unit 134 determines to add the purchase target to the predetermined list. For example, when the intention of the user estimated based on the second sensor information indicates the purchase of the purchase target, the determination unit 134 determines to perform the settlement of the purchase target. For example, when the intention of the user estimated based on the second sensor information indicates an interest in the purchase object, the determination unit 134 determines to add the purchase object to the hold list.

  In the example of FIG. 1, the determination unit 134 determines the processing mode related to the product IT14 in accordance with the user's intention for the product IT14 estimated by the estimation unit 133. For example, the determination unit 134 determines that the product IT14 is to be added to a predetermined shopping list because it is estimated that the user desires to purchase the car A, which is the product IT14. For example, the determination unit 134 determines to add the product IT14 to the shopping list associated with the terminal device AP11. For example, as illustrated in the shopping list information storage unit 124, the determining unit 134 adds the product IT14 to the shopping list associated with the terminal device AP11. It is determined to add the product IT 14 to the shopping list associated with the terminal device AP11. In the example of FIG. 1, the determination unit 134 determines, based on the second sensor information, the processing mode regarding the product IT 14 identified based on the first sensor information.

(Transmission unit 135)
The transmission unit 135 transmits various information. For example, the transmitting unit 135 transmits various information to an external information processing apparatus. For example, when a product is added to the shopping list in the shopping list information storage unit 124, the transmitting unit 135 may transmit information on the addition of the product to various devices used by the user at a predetermined timing. For example, the transmission unit 135 may transmit, to the terminal device AP11, information on the item added to the shopping list, and the terminal device AP11 may output information on the item received at a predetermined timing. For example, the terminal device AP11 may display information on a product at a timing at which the terminal device AP11 is used again after the user finishes an action related to shopping.

[3. Processing flow]
Next, the procedure of the determination process by the determination system 1 according to the embodiment will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of the determination process according to the embodiment.

  As shown in FIG. 7, the first accepting unit 131 of the determining apparatus 100 accepts first sensor information regarding specification of a product of the user (step S101). In the example of FIG. 1, the first reception unit 131 receives, from the terminal device AP11, first sensor information indicating that the user has specified the car A.

  Then, the second reception unit 132 of the determination device 100 receives the second sensor information regarding the purchase intention of the user for the product (step S102). In the example of FIG. 1, the second accepting unit 132 accepts second sensor information indicating that the user intends to purchase the car A from the terminal device AP11.

  Thereafter, the determination unit 134 of the determination device 100 determines the processing mode related to the product or service specified based on the first sensor information based on the second sensor information (step S103). In the example of FIG. 1, the determination unit 134 adds the car A specified by the user to the shopping list in the shopping list information storage unit 124.

[4. Modified example]
For example, the determination device 100A according to the modification may generate a model, and estimate the user's intention using the generated model. This point will be described using FIG. 8 and FIG. About the same composition as an embodiment, the same numerals are attached and explanation is omitted.

[4-1. Configuration of determination device]
Next, the configuration of the determination apparatus 100A according to the modification will be described with reference to FIG. FIG. 8 is a diagram showing a configuration example of a determination device according to a modification. As illustrated in FIG. 8, the determination apparatus 100A includes a communication unit 110, a storage unit 120A, and a control unit 130A. The determination apparatus 100A has an input unit (for example, a keyboard or a mouse) for acquiring various operations from the administrator or the like of the determination apparatus 100A, and a display unit (for example, a liquid crystal display or the like) for displaying various information. You may

(Storage unit 120A)
The storage unit 120A is implemented by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As illustrated in FIG. 8, the storage unit 120A according to the modification includes a device information storage unit 121, a content information storage unit 122, a product information storage unit 123, a shopping list information storage unit 124, and a model information storage unit 125.

(Model information storage unit 125)
The model information storage unit 125 according to the modification stores information on learning. For example, the model information storage unit 125 stores a plurality of model information. FIG. 9 is a diagram illustrating an example of a model information storage unit according to the embodiment. The model information storage unit 125 illustrated in FIG. 9 has items such as “feature 1” to “feature 3” and the like corresponding to the models M1 and M2 as model information.

  For example, in the example shown in FIG. 9, the model information related to the model M1 has a weight of feature 1 of “0.5”, a weight of feature 2 of “−0.4”, a weight of feature 3 of “0.2”, etc. To indicate that Further, for example, in the example illustrated in FIG. 9, in the model information regarding the model M2, the weight of feature 1 is “−0.3”, the weight of feature 2 is “0.1”, and the weight of feature 3 is “−0. 2 ”and so on. For example, when the feature (feature amount) of the model is expressed by an m-dimensional vector, the number of feature numbers is m, and the weight of the feature 1 to the feature m is stored.

  The model information storage unit 125 is not limited to the above, and may store various model information according to the purpose.

(Control unit 130A)
Returning to the explanation of FIG. 8, the control unit 130A is a controller, and for example, various programs (corresponding to an example of a determination program or a learning program) stored in a storage device inside the determination device 100A by a CPU, an MPU, or the like. Is implemented by executing the RAM as a work area. The control unit 130A is a controller, and is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

  As shown in FIG. 8, the control unit 130A includes a first reception unit 131, a second reception unit 132, an estimation unit 133A, a determination unit 134, a transmission unit 135, and a learning unit 136. Implement or execute the functions and actions of the information processing to be described.

(Learning unit 136)
The learning unit 136 learns various information. For example, the learning unit 136 generates a model based on various information. For example, the learning unit 136 learns based on the user's intention estimated in the past and information on the user's action on the product whose processing mode is determined based on the user's intention.

  For example, the learning unit 136 may estimate that the user's intention is the intention to purchase, and when the user purchases a product added to the shopping list, learning may be performed using the information as a positive example. For example, the learning unit 136 may perform learning using the second sensor information in the case where the user purchases a product added to the shopping list, which is assumed to be the intention of the user and intends to purchase.

  For example, when the user's intention is presumed to be an intention to purchase, and the user does not purchase the item added to the shopping list, the learning unit 136 may perform learning using the information as a negative example. For example, the learning unit 136 may perform learning using, as a negative example, the second sensor information in the case where the user's intention is presumed to be the intention to purchase, and the user does not purchase the item added to the shopping list. .

  For example, when the user's intention is presumed not to be purchased and the user does not purchase a product not added to the shopping list, the learning unit 136 may perform learning using the information as a positive example. For example, even if the learning unit 136 performs learning using the second sensor information as a positive example when the user does not purchase a product that is not intended to be purchased by the user and is not added to the shopping list. Good.

  For example, the learning unit 136 may perform learning using the information as a negative example when it is estimated that the user's intention is not the intention to purchase and the product purchased by the user is not added to the shopping list. For example, the learning unit 136 may perform learning by using, as a negative example, second sensor information in the case where the user purchases a product that is not intended to be purchased by the user and is not added to the shopping list.

  For example, the learning unit 136 may generate a model that outputs a larger value as the user's intention is more likely to be a purchase intention. The learning unit 136 may also generate a model that outputs a smaller value as the user's intention is less likely to be a purchase intention. For example, the learning unit 136 generates a model that outputs a value (score) closer to 1 when the user's intention is more likely to be a purchase intention, and as the user's intention is less likely to be a purchase intention. You may

  For example, the learning unit 136 may generate a model for each user. For example, the learning unit 136 may generate a model for each user identified by the estimation unit 133A. Also, for example, the learning unit 136 may generate a model for each user classification. For example, the learning unit 136 may generate a model for each of various user classifications such as a 30's male and a 20's female. For example, the learning unit 136 may generate a model for each user classification based on the attribute of the user specified by the estimation unit 133A. In addition, the learning unit 136 may generate one model that is commonly used by all users, not limited to a plurality of models. The above is an example, and the learning unit 136 may generate a model based on any information. For example, the learning unit 136 may generate a model that outputs a score as an input of the first sensor information and the second sensor information.

(Estimation unit 133A)
The estimation unit 133A uses the model generated by the learning unit 136 to estimate the user's intention. For example, the estimation unit 133A may estimate the user's intention based on the score output by inputting the second sensor information into the model. For example, the estimation unit 133A may estimate the user's intention based on the score output by inputting the second sensor information or the first sensor information to the model. For example, the estimation unit 133A identifies the user whose second sensor information has been detected, using various conventional techniques as appropriate, and estimates the user's intention using a model selected according to the identified user. May be For example, the estimation unit 133A identifies the user who made the utterance by analyzing the voice information that is the second sensor information, and estimates the user's intention using the model selected according to the identified user. You may go.

  For example, when the score output by the model is equal to or greater than a predetermined threshold, the estimation unit 133A may estimate the user's intention as the purchase intention. Also, for example, when the score output by the model is less than a predetermined threshold, the estimation unit 133A may estimate that the user's intention is not a purchase intention. Also, for example, when the score output from the model is within a predetermined range less than a predetermined threshold, the estimation unit 133A may estimate the user's intention as the purchase consideration intention. In this case, for example, when the score output from the model is lower than a predetermined range less than a predetermined threshold, the estimation unit 133A may estimate that the user's intention is not the intention of purchase.

  When acquiring a model used to estimate the user's intention from an external information processing apparatus, the determination apparatus 100A may not have the learning unit 136.

[5. effect〕
As described above, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification include the first reception unit 131, the second reception unit 132, and the determination unit 134. The first accepting unit 131 is the first sensor information detected by the first sensor SN1, and accepts the first sensor information related to the designation of the user for the product or service displayed in the predetermined area. The second accepting unit 132 is the second sensor information detected by the second sensor SN2, and accepts the second sensor information regarding the user's intention for the product or service specified based on the first sensor information. The determination unit 134 determines, based on the second sensor information, a processing mode related to a purchase target that is a product or service specified based on the first sensor information.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification specify the product or service based on the first sensor information, and based on the second sensor information on the user's intention for the specified product or service. Can be estimated. Thereby, the determination apparatus 100, 100A can appropriately determine the processing mode related to the product or service.

  Further, in the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification, when the intention of the user estimated based on the second sensor information satisfies a predetermined condition, the determination unit 134 determines a predetermined purchase target. Decide to add to the list.

  Thereby, when the intention of the user estimated based on the second sensor information satisfies a predetermined condition, the determination device 100 according to the embodiment and the determination device 100A according to the modification add the purchase target to the predetermined list. By determining, the processing aspect regarding goods or services can be determined appropriately.

  Further, in the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification, when the intention of the user estimated based on the second sensor information indicates purchase of a purchase target, the determination unit 134 is a purchase target. Decide to settle.

  Thereby, when the intention of the user estimated based on the second sensor information indicates the purchase of the purchase object, the determination device 100 according to the embodiment and the determination device 100A according to the modification perform settlement of the purchase object. By determining, the processing aspect regarding goods or services can be determined appropriately.

  Further, in the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification, when the intention of the user estimated based on the second sensor information indicates an interest in the purchase target, the determination unit 134 is a purchase target. Decide to add to the pending list.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification set the purchase target to the hold list when the intention of the user estimated based on the second sensor information indicates an interest in the purchase target. By deciding to add, it is possible to appropriately determine the processing aspect regarding the product or service.

  In addition, in the determination device 100 according to the embodiment and the determination device 100A according to the modification, the first reception unit 131 receives the first sensor information detected by the infrared sensor as the first sensor SN1.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification appropriately receive the first sensor information detected by the infrared sensor as the first sensor SN1, thereby appropriately applying the product or service designated by the user. Can be identified. Thereby, the determination apparatus 100, 100A can appropriately determine the processing mode related to the product or service.

  In addition, in the determination device 100 according to the embodiment and the determination device 100A according to the modification, the second reception unit 132 receives the second sensor information detected by the voice sensor as the second sensor SN2.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification receive the second sensor information detected by the voice sensor as the second sensor SN2, thereby the user's intention for the identified product or service Can be estimated based on the user's voice. Thereby, the determination apparatus 100, 100A can appropriately determine the processing mode related to the product or service.

  Further, in the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification, the first reception unit 131 relates to designation of the user for one product or service among products or services displayed in a plurality of predetermined areas. Accept first sensor information.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification receive the first sensor information regarding the user's designation for one product or service among the products or services displayed in a plurality of predetermined areas. Thereby, even when a plurality of objects are displayed, it is possible to appropriately identify the product or service designated by the user. Thereby, the determination apparatus 100, 100A can appropriately determine the processing mode related to the product or service.

  Further, the determination device 100 according to the embodiment and the determination device 100A according to the modification include estimation units 133 and 133A. The estimation units 133 and 133A estimate the user's intention for the purchase target based on the second sensor information.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification appropriately determine the processing mode related to the product or service by estimating the user's intention for the purchase target based on the second sensor information. be able to.

  Further, in the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification, the estimation units 133 and 133A calculate the user's intention for the purchase target based on the model and the second sensor information that change according to the user. presume.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification estimate the intention of the user with respect to the purchase target based on the model that fluctuates according to the user and the second sensor information. Or the processing aspect regarding a service can be determined appropriately.

  Further, in the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification, the estimation units 133 and 133A are goods or services for which the processing mode is determined based on the intention of the user estimated in the past and the intention. The second sensor information is input to the model learned on the basis of the information on the user's action for the user to estimate the user's intention for the purchase target.

  Thereby, the determination apparatus 100 according to the embodiment and the determination apparatus 100A according to the modification relate to the user's intention for the user estimated in the past and the action of the user for the product or service for which the processing mode is determined based on the intention. By using the information and the model learned based on the information, it is possible to appropriately determine the processing aspect regarding the product or service.

[6. Hardware configuration]
The determination apparatus 100 according to the embodiment described above and the determination apparatus 100A according to the modification are realized by, for example, a computer 1000 configured as shown in FIG. FIG. 10 is a hardware configuration diagram showing an example of a computer that implements the function of the determination device. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 to control each part. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, a program depending on the hardware of the computer 1000, and the like.

  The HDD 1400 stores a program executed by the CPU 1100, data used by the program, and the like. The communication interface 1500 receives data from another device via the network N, sends the data to the CPU 1100, and transmits data generated by the CPU 1100 to the other device via the network N.

  The CPU 1100 controls an output device such as a display or a printer and an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 acquires data from an input device via the input / output interface 1600. The CPU 1100 also outputs the generated data to the output device via the input / output interface 1600.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides the CPU 1100 with the program via the RAM 1200. The CPU 1100 loads such a program from the recording medium 1800 onto the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a digital versatile disc (DVD) or a phase change rewritable disc (PD), a magneto-optical recording medium such as a magneto-optical disk (MO), a tape medium, a magnetic recording medium, or a semiconductor memory. Etc.

  For example, when the computer 1000 functions as the determination device 100, 100A, the CPU 1100 of the computer 1000 implements the functions of the control units 130, 130A by executing the program loaded on the RAM 1200. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them, but as another example, these programs may be acquired from another device via the network N.

  Although some of the embodiments and modifications of the present application have been described above in detail based on the drawings, these are merely examples, and various aspects can be obtained based on the knowledge of those skilled in the art, including the aspects described in the rows of the disclosure of the invention. It is possible to carry out the present invention in other forms that have been modified and improved.

[7. Other]
Further, among the processes described in the embodiment and the modification, all or part of the process described as being automatically performed can be manually performed, or it is described as manually performed. All or part of the treatment may be performed automatically by known methods. In addition, information including processing procedures, specific names, various data and parameters shown in the above-mentioned documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

  Further, each component of each device illustrated is functionally conceptual, and does not necessarily have to be physically configured as illustrated. That is, the specific form of the distribution and integration of each device is not limited to the illustrated one, and all or a part thereof may be functionally or physically dispersed in any unit depending on various loads, usage conditions, etc. It can be integrated and configured.

  Moreover, it is possible to combine suitably the embodiment and modification which were mentioned above in the range which does not make process content contradictory.

  In addition, the "section (module, unit)" described above can be read as "means" or "circuit". For example, the first reception unit can be read as a first reception means or a first reception circuit.

DESCRIPTION OF SYMBOLS 1 determination system 100 determination device 121 device information storage unit 122 content information storage unit 123 product information storage unit 124 shopping list information storage unit 130 control unit 131 first reception unit 132 second reception unit 133 estimation unit 134 determination unit 135 transmission unit AP11 Terminal equipment SN1 1st sensor SN2 2nd sensor N network

Claims (9)

A first reception unit that receives the first sensor information detected by the first sensor, the first sensor information regarding specification of the user for the product or service displayed in the predetermined area;
A second reception unit that receives second sensor information that is the second sensor information detected by the second sensor and that is about the user's intention for the product or service specified based on the first sensor information;
A learning unit that generates a model for each user classification by learning based on the user's intention estimated in the past and information on the user's behavior for the product or service whose processing mode has been determined based on the user's intention When,
An estimation unit configured to estimate an intention of the user for a product or service specified based on the first sensor information by inputting the second sensor information to the model for each of the user classifications generated by the learning unit; ,
A determination unit configured to determine, based on the estimation result of the estimation unit, a processing mode related to a purchase target that is a product or service specified based on the first sensor information ;
The learning unit learns, as a positive example, a case in which the intention of the user estimated in the past by the estimation unit matches the behavior of the user for the purchase target whose processing mode is determined based on the intention. Or, by learning as a negative example, the case where the user's intention for the purchase target whose processing mode is determined based on the user's intention estimated in the past by the estimation unit and the user's action for the purchase object does not match , Generate the model,
Determining device which is characterized a call. The determination unit is
Determining apparatus according to claim 1, characterized in that determining the intention of the user is estimated based on the second sensor information when a predetermined condition is satisfied, adding the purchase target to the predetermined list. The determination unit is
The determination device according to claim 1 or 2 , wherein if the user's intention estimated based on the second sensor information indicates the purchase of the purchase object, it is determined that the purchase object is to be settled. . The determination unit is
The method according to any one of claims 1 to 3 , wherein if the user's intention estimated based on the second sensor information indicates an interest in the purchase object, the purchase object is determined to be added to the hold list. The determination device according to item 1. The first reception unit is
The determination apparatus according to any one of claims 1 to 4 , wherein the first sensor information detected by an infrared sensor as the first sensor is received. The second reception unit is
The determination apparatus according to any one of claims 1 to 5 , wherein the second sensor information detected by an audio sensor as the second sensor is received. The first reception unit is
7. The determination according to any one of claims 1 to 6 , wherein first sensor information regarding specification of a user for one product or service among a plurality of products or services displayed in the predetermined area is received. apparatus. A decision method that the computer executes,
A first accepting step of accepting first sensor information detected by the first sensor, the first sensor information relating to the designation of the user for the product or service displayed in the predetermined area;
A second receiving step of receiving second sensor information which is the second sensor information detected by the second sensor and which relates to the user's intention for the product or service specified based on the first sensor information;
A learning step of generating a model for each user classification by learning based on the user's intention estimated in the past and information on the user's behavior for the product or service whose processing mode has been determined based on the user's intention When,
Estimating the intention of the user for the product or service specified based on the first sensor information by inputting the second sensor information to the model for each of the user classifications generated in the learning step; ,
Determining a processing mode related to a purchase target, which is a product or service specified based on the first sensor information, based on an estimation result of the estimation step ;
In the learning step, learning is performed by using as a positive example a case where the user's intention estimated in the past in the estimation step matches the user's action on the purchase target whose processing mode is determined based on the intention. Or, by learning as a negative example, the case where the user's intention for the past determined in the estimation step and the user's action for the purchase target whose processing mode is determined based on the intention does not match , Generate the model,
Decision wherein a call. A first acceptance procedure for accepting the first sensor information detected by the first sensor, the first sensor information relating to the specification of the user for the product or service displayed in the predetermined area;
A second reception procedure for receiving second sensor information which is second sensor information detected by a second sensor, and which relates to a user's intention for a product or service specified based on the first sensor information;
A learning procedure that generates a model for each user classification by learning based on the user's intention estimated in the past and information on the user's behavior for the product or service whose processing mode has been determined based on the user's intention When,
An estimation procedure for estimating an intention of the user for a product or service specified based on the first sensor information by inputting the second sensor information to the model for each of the user classifications generated in the learning procedure; ,
And causing a computer to execute a determination step of determining a processing mode related to a purchase target, which is a product or service specified based on the first sensor information, based on an estimation result of the estimation step .
In the learning procedure, learning is performed by using, as a positive example, a case where the user's intention estimated in the past in the estimation procedure and the user's action for the purchase target whose processing mode is determined based on the intention matches. Or, by learning as a negative example, the case where the user's intention for the purchase target whose processing mode is determined based on the user's intention estimated in the past in the estimation procedure and the user's action for the purchase object does not match , Generate the model,
Determining program, wherein a call.

Images