JP6504279B2 - Electronic equipment system - Google Patents

Description

The present invention relates to an electronic device system .

  Heretofore, there has been proposed a sales promotion system which acquires behavior information related to purchase of goods by customers and links the information to sales promotion (see, for example, Patent Document 1).

JP, 2010-277256, A

  However, in the conventional sales promotion system, it can not be said that necessary information can be sufficiently acquired, and it can not be said that it is convenient.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an easy-to-use electronic device.

  The electronic device according to the present invention includes a first input unit (42, 46) for inputting operation information related to a user's purchase, and a second input unit (42, 46) for inputting emotion information related to the user's purchase. ing.

  In this case, a detection unit (46) may be provided to detect the degree of satisfaction or dissatisfaction of the user based on the information input by the first input unit and the second input unit. Further, an instruction unit (46) may be provided which instructs the user to respond based on the information input by the first input unit and the second input unit. Further, the first input unit may input information related to the movement of the user's hand. Further, the second input unit may input information on the living body of the user.

  Further, in the electronic device of the present invention, at least one of the first input unit and the second input unit may input information on eyes from the user side. In this case, a communication unit (42) that can communicate with an acquisition device that acquires information on eyes from the user side may be provided. In this case, the communication unit may communicate with a communication unit (25) provided in the acquisition device. The communication unit may communicate with the acquisition device via a communication relay device (20) different from the acquisition device. In this case, the communication unit may communicate with the acquisition device via near field communication or human body communication with the communication relay device. The acquisition device may include an imaging unit (11) that performs imaging, and an analysis unit (46) that analyzes imaging data captured by the imaging unit.

  Further, in the electronic device according to the present invention, the first input unit may input the operation information in accordance with a change in the emotion information input by the second input unit.

  The electronic device according to the present invention comprises an input unit (42, 46) for inputting information on eyes from the user side, and an estimation unit for estimating emotion information on purchase of the user based on information on eyes from the user side 46) and.

  In this case, the estimation unit may estimate emotion information on the purchase of the user based on whether the information on the eye from the user matches or is similar to a predetermined pattern.

  An electronic device according to the present invention is an electronic device (10) attachable to the vicinity of a user's face, the acquisition unit (17) acquiring information on eyes from the user side, and the information acquired by the acquisition unit And a communication unit (16) for communicating with an external device (20) in contact with the user via human body communication.

  Note that although the present invention has been described above in association with the reference numerals of the drawings representing one embodiment in order to explain the present invention in an easy-to-understand manner, the present invention is not limited to this. May be improved as appropriate, and at least a part may be replaced with another component. Furthermore, the configuration requirements without particular limitation on the arrangement are not limited to the arrangements disclosed in the embodiments, and can be arranged at positions where the functions can be achieved.

  The present invention has the effect of being able to provide a user-friendly electronic device.

FIG. 1 is a block diagram showing the configuration of an electronic device system according to a first embodiment. It is a figure which shows the utilization condition of a mounting apparatus, a cage apparatus, and an articles | goods apparatus. It is a perspective view of a mounting device. Fig.4 (a) is a flowchart which shows a process of the mounting apparatus concerning 1st Embodiment, FIG.4 (b) is a flowchart which shows the process of the articles | goods apparatus concerning 1st Embodiment. It is a flow chart which shows processing of a basket device concerning a 1st embodiment. It is a flowchart which shows the process of the server concerning 1st Embodiment. FIG. 7A and FIG. 7B are diagrams (part 1) showing an example of a gaze image. It is a figure (the 2) showing an example of a look image. It is a figure which shows the order of the articles | goods picked up by a certain user from entering a store until it passes a cash register. It is a block diagram which shows the structure of the electronic device system which concerns on 2nd Embodiment. Fig.11 (a) is a flowchart which shows a process of the mounting apparatus concerning 2nd Embodiment, FIG.11 (b) is a flowchart which shows the process of the articles | goods apparatus concerning 2nd Embodiment. It is a flow chart which shows processing of a basket device concerning a 2nd embodiment. It is a flowchart which shows the process of the server concerning 2nd Embodiment. Fig.14 (a) and FIG.14 (b) are figures which show an example of shopping DB.

First Embodiment
Hereinafter, the electronic device system according to the first embodiment will be described in detail based on FIGS. 1 to 9. FIG. 1 is a block diagram showing the configuration of the electronic device system 100 according to the first embodiment.

  As shown in FIG. 1, the electronic device system 100 includes a mounting device 10, a basket device 20, an article device 30, and a server 40. As shown in FIG. 2, the mounting device 10 is a glasses-type terminal worn on the face by a user (shopper) who visits a store. The basket device 20 is a device provided to the basket 120 installed in a store. The article apparatus 30 is an apparatus provided to an article (goods) 130 displayed in a store. The server 40 is an information processing apparatus provided in a store or a data center. Note that communication between the mounting device 10 and the basket device 20, and between the article device 30 and the basket device 20, enables communication (human body communication) via the human body, and between the basket device 20 and the server 40. , Wireless communication is possible.

(Mounting device 10)
The mounting apparatus 10 includes an imaging unit 11, a display unit 12, an operation unit 13, a microphone 14, a storage unit 15, a communication unit 16, and a control unit 17, as shown in FIG. In addition, the mounting apparatus 10 is shown by the perspective view in FIG. As shown in FIG. 3, the mounting device 10 includes a spectacles-type frame 110. The configuration of the mounting apparatus 10 illustrated in FIG. 1 and not illustrated in FIG. 3 is provided inside the frame 110 or in part of the frame 110.

  The imaging unit 11 includes a lens, an imaging element, an image processing unit, and the like, and images a still image or a moving image. The imaging unit 11 is provided near the end of the frame 110 (near the user's right eye), as shown in FIG. Therefore, when the user wears the mounting apparatus 10 (the state shown in FIG. 2), it is possible to capture an image in the direction in which the user is facing (looking at). In addition, a predetermined part of the captured image (for example, a part slightly shifted to the left side from the center) can be regarded as a part (part in which the user is looking at).

  The display unit 12 includes a projector provided inside the frame 110 or in the vicinity of the frame 110, and a prism for guiding a projected image from the projector to the eyes of the user. The display unit 12 displays various information under the instruction of the control unit 17.

  The operation unit 13 is a touch pad provided in the frame 110. The operation unit 13 detects the movement of the user's finger, receives an operation from the user, and transmits the received operation information to the control unit 17.

  The details of the imaging unit 11, the display unit 12, and the operation unit 13 are also disclosed, for example, in US Patent Application Publication No. 2013/0044042.

  The microphone 14 is provided in the frame 110 and collects voices uttered by the user and voices around the user. The voice collected by the microphone 14 is voice-recognized by a voice recognition unit (not shown), and the voice recognition result is transmitted to the control unit 17. The control unit 17 executes a process (for example, an execution process of a command) based on the speech recognition result. The control unit 17 may execute speech recognition. Further, the control unit 17 stores the collected voice data in the storage unit 15.

  The storage unit 15 is, for example, a nonvolatile semiconductor memory such as a flash memory, and stores image data captured by the imaging unit 11, audio data collected by the microphone 14, display data displayed on the display unit 12, and the like. .

  The communication unit 16 performs human body communication with other devices. The communication unit 16 includes an electrode unit 16a provided on the frame 110 and capable of contacting the user, and a human body communication unit 16b performing human body communication using the electrode unit 16a. Human body communication includes a current method in which a weak current is supplied to the human body to modulate the current to transmit information, and an electric field method in which the electric field induced on the surface of the human body is modulated to transmit information. . In the present embodiment, it is possible to use any type of current system and electric field system. Note that human body communication is possible not only when the user directly contacts the electrode unit 16a of the mounting apparatus 10, but also when the electrode unit 16a and the user are not directly contacting. In the present embodiment, the communication unit 16 transmits the image data captured by the imaging unit 11 and the audio data collected by the microphone 14 to the car apparatus 20. The communication unit 16 may perform proximity communication with another device, such as Bluetooth (registered trademark), RFID (Radio Frequency Identification), TransferJet (registered trademark), or the like.

  The mounting apparatus 10 may have a wireless communication unit instead of the communication unit 16 or in combination with the communication unit 16. In this case, image data and audio data may be directly transmitted to the server 40 by wireless communication, or image data and audio data may be transmitted to the car apparatus 20 by wireless communication.

  The control unit 17 controls the entire mounting device 10 in an integrated manner. The control unit 17 includes a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), and the like. In the present embodiment, the control unit 17 performs control related to imaging of the imaging unit 11, control related to display of the display unit 12, and the like.

  The mounting device 10 may not be a glasses-type terminal. For example, it may be a wearable terminal wearable at any position on the body, and part of its functions may be separated. As one example, as a contact lens type terminal provided with a display unit, another function may be provided to a wristwatch type terminal.

(The basket device 20)
As illustrated in FIG. 1, the basket device 20 includes a biometric sensor 21, a calendar unit 22, a storage unit 23, a first communication unit 24, a second communication unit 25, a control unit 26, and the like.

  The biological sensor 21 is provided in a portion (hand portion) of the basket 120 that the user can touch when the user holds the basket 120. The biological sensor 21 detects a change in biological information of the body such as heart rate, blood pressure, sweat rate, and includes, for example, an LED (light emitting diode) and a photosensor, and a sweat sensor. For example, by using an LED and a photo sensor, a heart rate can be detected by receiving light reflected by the light emitted from the LED with a finger using the photo sensor. In addition, the perspiration sensor can detect the amount of perspiration by measuring the impedance of the hand with a plurality of electrodes. In the present embodiment, the output from the biological sensor 21 is sent to the control unit 26. Then, the control unit 26 estimates a change in the user's emotion (morning emotion) based on the output of the living body sensor 21.

  The calendar unit 22 acquires time information such as year, month, day, and time, and outputs the time information to the control unit 26. In addition, the calendar unit 22 measures time.

  The storage unit 23 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores information related to an item obtained by the user as described later. The storage unit 23 also stores an ID number (car ID) for uniquely identifying the car apparatus 20.

  The first communication unit 24 performs human body communication with another device. The first communication unit 24 may perform near-field communication with another device. In the present embodiment, the first communication unit 24 communicates with the communication unit 16 of the mounting apparatus 10 and the communication unit 32 of the object apparatus 30 using human body communication.

  The first communication unit 24 includes an electrode unit 24a provided on the handle of the basket 120 and capable of contacting the user's hand or arm, and a human body communication unit 24b performing human body communication via the electrode unit 24a. .

  The second communication unit 25 performs wireless communication with another device. In the present embodiment, wireless communication is performed with the server 40, and article information of an article touched by the user, image data captured by the imaging unit 11, voice data collected by the microphone 14, detection of the biological sensor 21 The result is sent to the server 40. The basket device 20 according to the present embodiment is a communication relay device that relays communication of item information between the item device 30 and the server 40 and communication of image data and voice data between the mounting device 10 and the server 40. It has a function as

  The control unit 26 controls the entire basket device 20, and includes a CPU, a RAM, a ROM, and the like. In the present embodiment, the control unit 26 controls information communication between the article device 30, the mounting device 10, and the server 40.

  The basket device 20 may be provided in a cart used in a store. In this case, it is preferable to provide the living body sensor 21 and the electrode unit 24 a of the first communication unit 24 in the grip portion of the cart.

(Article device 30)
The article device 30 is an article (article) sold in a store or the like, and is provided to an article that comes in contact with the body of a visitor (user), such as food, clothes, shoes, watches, accessories, golf clubs, and bags. . In the present embodiment, for convenience of explanation, the article apparatus 30 will be described as being provided in a commodity (food, etc.) displayed in a supermarket.

  The article apparatus 30 includes a storage unit 31, a communication unit 32, and a control unit 33, as shown in FIG.

  The storage unit 31 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores information on an article (article) to which the article apparatus 30 is attached. Specifically, the storage unit 31 stores the item information (item information) of the item to which the item device 30 is attached. For example, when the article device 30 is attached to a processed food (for example, canned food), the storage unit 31 uses the article information as the manufacturer (seller), the product name, the raw material name, the expiration date, the content, the origin Remember the country etc.

  The control unit 33 controls the entire article apparatus 30, and includes a CPU, a RAM, a ROM, and the like. In the present embodiment, the control unit 33 controls the transmission of the item information stored in the storage unit 31 to the basket device 20. In addition, the control part 33 and the memory | storage part 31 may be provided in any position of articles | goods.

  The communication unit 32 performs human body communication with other devices. The communication unit 32 may perform close proximity communication with another device. In the present embodiment, the communication unit 32 includes an electrode unit 32a provided at a position in contact with a user (for example, a hand), and a human body communication unit 32b performing human body communication using the electrode unit 32a. In the present embodiment, as shown in FIG. 2, the human body between the first communication unit 24 of the basket device 20 and the communication unit 32 of the article device 30 with the article 130 held by the user holding the basket 120. Communication is established. As a result, the article information stored in the storage unit 31 is transmitted to the basket device 20. Note that human body communication is possible not only when the user directly contacts the electrode unit 32a of the article apparatus 30, but also when the electrode unit 32a and the user are not in direct contact (eg, when wearing gloves).

(Server 40)
As illustrated in FIG. 1, the server 40 includes a storage unit 41, a communication unit 42, an optical character reader (OCR) unit 43, an image recognition unit 44, and a control unit 46.

  The storage unit 41 is, for example, a non-volatile semiconductor memory such as a flash memory, a hard disk drive or the like, and information acquired from the basket device 20, that is, image data captured by the imaging unit 11 of the mounting device 10 or the microphone 14 The voice data collected by use and the article information transmitted from the article apparatus 30 are stored. In addition, the storage unit 41 stores face image data and product image data of an employee (shop clerk) of a store, and various data used when analyzing the purchase behavior of the user. In the present embodiment, as described later, the control unit 46 determines whether the image data captured by the imaging unit 11 of the mounting apparatus 10 in cooperation with the image recognition unit 44 includes the employee's face and goods. To detect

  The communication unit 42 performs wireless communication with another device. In the present embodiment, the communication unit 42 performs wireless communication with the second communication unit 25 of the car apparatus 20 by wireless communication.

  The OCR unit 43 recognizes characters included in the image data captured by the imaging unit 11 of the mounting apparatus 10 and converts the characters into text data. For example, when the imaging unit 11 captures the image data of FIG. 7 (b), the OCR unit 43 recognizes characters, and the text data "Sorry. Currently waiting for arrival. The next arrival date is undecided. To get

  The image recognition unit 44 extracts feature points of the image data captured by the imaging unit 11 of the mounting apparatus 10 using a feature amount detection algorithm such as SURF (Speeded Up Robust Features) or SHIFT (Scale Invariant Feature Transform), for example. The image recognition is performed by comparison with a plurality of standard patterns to be recognized stored in 41. In the present embodiment, the image recognition unit 44 compares the employee's face image data and product image data stored in the storage unit 41 with the employee and the product in the image captured by the imaging unit 11. Detect if it is included.

  The control unit 46 controls the entire server 40, and includes a CPU, a RAM, a ROM, and the like.

  Next, processing of the electronic device system 100 will be described in detail using FIGS. 4 to 9. In the present embodiment, when the user (shopper) wearing the mounting apparatus 10 visits a store (for example, a supermarket), the basket 120 is held to perform shopping. In addition, the user determines the article to be purchased by picking up an article (article) 30 provided with the article apparatus 30 in the store or putting it in the basket 120, and performs the purchase process at the cash register Do.

(Process of mounting device 10)
The process of the mounting apparatus 10 is shown by the flowchart by Fig.4 (a).

  In the process of FIG. 4A, the control unit 17 of the mounting apparatus 10 waits in step S10 until human body communication with the car apparatus 20 is established. In this case, when the user wearing the mounting apparatus 10 enters the store and holds the basket 120, and the user touches the basket apparatus 20, the communication unit 16 of the mounting apparatus 10 and the first communication of the basket apparatus 20 Communication with the unit 24 is established. When human body communication with the basket device 20 is established, the control unit 17 shifts to step S12. Note that, instead of step S10, the process after step S12 may be executed when the power of the mounting apparatus 10 is turned on after the user enters the store. In this case, the mounting apparatus 10 is provided with a GPS (Global Positioning System) module and a position detection apparatus capable of position detection according to the communication status with a wireless LAN (Local Area Network) base station installed in the store, The control unit 17 may determine whether the user has entered a store based on the detection result of the position detection device. Also, conversely to this, the mounting apparatus 10 may always supply power to the communication unit 16 and the mounting apparatus 10 may be activated by the establishment of human body communication with the car apparatus 20.

  When the process proceeds to step S12, the control unit 17 instructs the imaging unit 11 to execute imaging from the user side. In this case, the imaging unit 11 may capture either a still image or a moving image. In the present embodiment, the imaging unit 11 will be described as one that captures a still image. The control unit 17 stores an image (hereinafter, referred to as a line-of-sight image) captured by the imaging unit 11 in the storage unit 15. In step S12, in addition to the imaging of the visual line image by the imaging unit 11, the control unit 17 collects the voice including the user's voice and surrounding sounds using the microphone 14, and stores the voice as voice data in the storage unit 15. You may memorize.

  Next, in step S14, the control unit 17 transmits the line-of-sight image to the car device 20 via the communication unit 16. The control unit 17 measures the moving speed of the user, for example, by providing the mounting apparatus 10 with an accelerometer or an angular velocity meter, and when the moving speed of the user reaches a predetermined moving speed (for example, 1 km / h or less) The line-of-sight image may be transmitted to the device 20. The user may release the basket device 20 during shopping. Therefore, when human body communication is possible, the control unit 17 transmits the line-of-sight image to the cage device 20 by human body communication, and when the human body communication is not possible, the line-of-sight image is transmitted by proximity communication or wireless communication. It may be transmitted to the basket device 20. When wireless communication is performed, the mounting apparatus 10 needs to include a wireless communication unit.

  Next, in step S16, the control unit 17 determines whether human-body communication with the basket device 20, which has been input from the cash register terminal, has been established. Here, the input from the cashier terminal is transmitted from the cashier terminal to the car 20 by communication between the car 20 and the cashier terminal when the user brings the car 120 to the cashier. , Means to input the end of shopping information. That is, when human body communication between the basket device 20 and the mounting device 10, which has been input from the cash register terminal, is established, it means that the user's shopping has ended. Therefore, if the shopping has not ended and the determination in step S16 is negative, the process returns to step S12, and the processing and determination in steps S12, S14, and S16 are repeated. That is, the imaging of the line-of-sight image by the imaging unit 11 and the transmission to the basket device 20 are repeated. On the other hand, when the user's shopping is finished and the determination in step S16 is affirmed, the whole process of FIG. 4A is finished.

  Note that, instead of step S16, the control unit 17 may determine whether or not the user (mounting device 10) has left the store. In this case, as described above, the mounting apparatus 10 needs to be provided with a position detection device such as a GPS.

(Processing of the article device 30)
In FIG. 4B, the process of the article device 30 is shown in a flowchart.

  In the process of FIG. 4B, in step S20, the control unit 33 of the article device 30 stands by until human body communication with the car device 20 is newly established. In this case, as shown in FIG. 2, human body communication with the basket device 20 is newly established at the timing when the user holding the basket 120 picks up the article 130 from the product shelf or the like, and the process moves to step S22.

  At step S22, the control unit 33 instructs the communication unit 32 to transmit the article information stored in the storage unit 31 to the car apparatus 20.

  After that, the process returns to step S20, and the control unit 33 repeats the processing and determination of steps S20 and S22.

  By executing the process of FIG. 4B, it becomes possible for the user holding the basket 120 to transmit information of the item 130 obtained during shopping to the basket device 20 one by one.

(Processing of the basket device 20)
The process of the basket apparatus 20 is shown by the flowchart in FIG.

  In the process of FIG. 5, in step S30, the control unit 26 of the car device 20 stands by until human body communication with the mounting device 10 is established. When human body communication with the mounting apparatus 10 is established, the control unit 26 proceeds to step S32. The timing for shifting to step S32 is the same as the timing for shifting to step S12 in FIG. 4A described above.

  When the process proceeds to step S32, the control unit 26 acquires the line-of-sight image transmitted from the mounting apparatus 10 and stores the line-of-sight image in the storage unit 23. Note that, for example, an accelerometer or an angular velocity meter is provided in the basket device 20 to measure the moving speed of the user, and when the moving speed of the user reaches a predetermined moving speed (for example, 1 km / h or less) May be acquired and stored in the storage unit 23.

  Next, in step S34, the control unit 26 determines whether new item information has been received. If the determination in step S34 is negative, that is, if new item information has not been received, the process proceeds to step S38. On the other hand, if the determination in step S34 is affirmed, that is, if new item information is received, the process proceeds to step S36.

  At step S36, the control unit 26 acquires article information and stores the acquired article information in the storage unit 23. Thereafter, the process proceeds to step S38.

  When the process proceeds to step S38, the control unit 26 executes detection using the biosensor 21. Note that the order of the processes of steps S32, S34, S36, and S38 may be changed as appropriate, and the processes of the respective steps may be performed simultaneously and in parallel.

  Next, in step S40, the control unit 26 determines, based on the detection result of the biological sensor 21, whether or not the user has a change in emotion. In this case, for example, based on the detection result of the biological sensor 21, it is assumed that the user has a change in emotion when the heart rate rises sharply, when the blood pressure rises sharply, when the amount of sweating increases rapidly, etc. to decide. In addition, as an example where the user's emotion changes, there is a chance that the time service starts and a chance to purchase the product cheaply is encountered, the product scheduled to be purchased is sold out, or the cash register is crowded. is there. If the determination in step S40 is negative, the process proceeds to step S44, but if the determination in step S40 is affirmed, the process proceeds to step S42.

  When the process proceeds to step S42, the control unit 26 instructs the second communication unit 25 to store, in the storage unit 23, the line-of-sight image and the item information acquired immediately before the change in emotion (hereinafter referred to as first information). ) To the server 40. Note that “immediately before the change in emotion” means, for example, several seconds before the occurrence of the change in emotion. In some cases, the item information may not be acquired immediately before the change in emotion. This is a case where the change in emotion is not due to holding the item 130. In this case, the control unit 26 may transmit the gaze image to the server 40 as the first information.

  Here, line-of-sight images as shown in FIGS. 7A, 7B, and 8 are assumed as the line-of-sight image acquired immediately before the change in emotion. The line-of-sight image in FIG. 7A is an image showing that the user has encountered a time service, and more specifically, an image showing that the user has seen the seal of the time service attached to the product. Further, the line-of-sight image in FIG. 7B is an image when the product that the user is trying to purchase is sold out, and more specifically, is an image indicating that the user has viewed a notice awaiting arrival. Further, the line-of-sight image in FIG. 8 is an image indicating that the user has seen a state in which the cash register is crowded.

  The first information is transmitted to the server 40 with the car ID added. That is, the server 40 manages the first information using the car ID.

  After the determination in step S40 is negative or after the process in step S42 is performed, in step S44, the control unit 26 determines whether there is an input from the cash register terminal. In this case, as described above, when the user finishes shopping and brings the basket 120 to the cashier, the input from the cashier terminal is from the cashier terminal by communication between the basket device 20 and the cashier terminal. It means the input of the shopping end information transmitted to the basket device 20. If the determination in step S44 is negative, that is, if the user's shopping has not ended, the process returns to step S32, and the processing and determination in steps S32 to S44 are repeated. On the other hand, when the determination in step S44 is affirmed, the process proceeds to step S46.

  When the process proceeds to step S46, the control unit 26 instructs the second communication unit 25 to send all the line-of-sight images acquired after step S30, the article information, and the biological information (hereinafter referred to as second information) to the server 40. Send. That is, the control unit 26 controls, through the second communication unit 25, the line-of-sight image obtained while the user is shopping, the item information of the item obtained by the user, and while the user is shopping The information on the change in emotion obtained is sent to the server 40. The control unit 26 may transmit, to the server 40, voice data acquired using the microphone 14 of the mounting apparatus 10 while the user is shopping. It is assumed that an acquisition time of information (obtained from the calendar unit 22) is attached as a time stamp to each information transmitted from the control unit 26. Further, the second information is transmitted to the server 40 with the car ID added. That is, the server 40 manages the second information using the car ID. After that, the control unit 26 ends the entire process of FIG.

  As described above, when the control unit 26 executes the processing of FIG. 5, the first information is transmitted from the basket device 20 to the server 40 at a timing when the user has a change in emotion while shopping. The second information is transmitted from the basket device 20 to the server 40 at the timing when the user finishes the shopping.

(Processing of server 40)
The process of the server 40 is shown by a flowchart in FIG.

  In the process of FIG. 6, the control unit 46 of the server 40 stands by until there is an input from the basket device 20 in step S50. That is, the control unit 46 proceeds to step S52 when the first information or the second information is transmitted by executing the step S42 or S46 of FIG. 5 in the car apparatus 20.

  In step S52, the control unit 46 determines whether the input information is the first information. That is, the control unit 46 determines whether line-of-sight information or article information acquired by the basket device 20 is input immediately before the user has a change in emotion. If the determination here is affirmed, the process proceeds to step S54.

  At step S54, the control unit 46 executes emotion analysis based on the first information. Specifically, the control unit 46 analyzes the line-of-sight image acquired immediately before the change in emotion using the OCR unit 43 or the image recognition unit 44, or the article information acquired immediately before the change in emotion. Based on the analyzed result, the user's emotion analysis (analysis of satisfaction or dissatisfaction) is performed.

  For example, when the line-of-sight image is the image shown in FIG. 7A, the control unit 46 can detect the character “20% discounted from the morning market time service display price” by using the OCR unit 43. . In this case, since a word positive for the shopper, such as time service and 20% off, is included, the control unit 46 analyzes that the user is pleased and is in a satisfactory state (positive state). The word positive for the shopper is stored in the storage unit 41.

  In addition, when the line-of-sight image is the image shown in FIG. 7 (b), the control unit 46 uses the OCR unit 43 to "sorry. We are currently waiting for arrival. The next arrival date is undecided." The character can be detected. In this case, the control unit 46 analyzes that the user is in a disappointing state (negative state) because the shopper contains words that are negative for the shopper, such as sorry, waiting for arrival, or undecided. It is assumed that the negative word for the shopper is stored in the storage unit 41.

  Further, when the line-of-sight image is the image shown in FIG. 8, the control unit 46 uses the image recognition unit 44 to find that the cash register is crowded or the person waiting for the cash register has a dissatisfied face It can be detected that (no smile). In this case, the control unit 46 determines that the user is unhappy and in an unsatisfactory state (negative state).

  In addition, when the line-of-sight image includes the face image of the employee, the control unit 46 determines that the user's emotion has changed due to the employee. In this case, the control unit 46 uses the image recognition unit 44 to recognize whether the face of the employee is a smile or not. In addition, the image recognition unit 44 stores the face image of the employee stored in the storage unit 41 as to whether or not the image of the face included in the gaze image is an image of the face of the employee and whether it is a smile. Judge using data. Then, the control unit 46 determines that the user is in a sad or angry dissatisfaction state (negative state) when the employee is not smiling, and is satisfied when the employee is smiling. It is determined that it is in the state (positive state).

  Furthermore, when the state in which the user holds the item in the line-of-sight image is imaged, the control unit 46 determines that a change in emotion has occurred due to seeing or touching the item. In addition, the control unit 46 may be able to determine whether the user is pleased or hesitant, for example, by using the item information acquired by the cart device 20 in combination. For example, if the item information includes information such as limited quantity, it can be determined that the user is pleased. Also, if it is determined from the item information that the expiration date has expired, it can be determined that the user is lonely.

  Returning to FIG. 6, in the next step S56, the control unit 46 determines whether or not a response is necessary based on the emotion analysis result. Specifically, the control unit 46 determines that a response is necessary when the user is in an unsatisfactory state such as being lonely or angry (in a negative state). If the determination in step S56 is negative, the process returns to step S50. On the other hand, when the determination in step S56 is affirmed, the process proceeds to step S58.

  At step S58, the control unit 46 executes a response process based on the emotion analysis result. For example, when the line-of-sight image is as shown in FIG. 4B, the control unit 46 uses the communication unit 42 to transmit a message of an apology stored in the storage unit 41 and various messages such as arrival schedule Send to 20. Then, the control unit 26 of the car device 20 transmits various messages to the mounting device 10 via the first communication unit 24. In this case, in the mounting apparatus 10, the control unit 17 uses the display unit 12 to display (project) various messages.

  Further, for example, when the line-of-sight image is as shown in FIG. 8, the control unit 46 transmits a message of an apology for crowded cash register to the control unit 17 of the mounting apparatus 10 via the car apparatus 20. In this case, the control unit 17 displays (projects) an apology message on the display unit 12. The control unit 46 may output an apology message for the cash register congestion using the in-store broadcast. In addition, the control unit 46 may output a message (message of cash register support) to the effect that the employee who is cashed in should be refilled by using the in-store broadcast. Alternatively, the control unit 46 may output a cashier support message to a terminal held by the employee. In addition, the control unit 46 indicates the progress of the cash register (processing time per customer) and the level of skill of the clerk in charge of the cash register (apprentice or veteran) to the user who saw the situation where the cash register is crowded May be notified.

  Also, for example, when it is determined from the line-of-sight image that there is a problem with the response of the employee, the control unit 46 controls the terminal held by the other employee or the store manager, etc. Output a message that a problem has occurred. In addition, if the position detection device capable of detecting the position according to the communication sensitivity with the wireless LAN base station is provided in the basket device 20, the control unit 46 causes a trouble based on the detection result of the position detection device. The location of the occurrence can be notified to other employees and store managers.

  Furthermore, for example, when it is determined from the line-of-sight image that the customer is hated or angry with the article, the control unit 46 uses the in-store broadcast or the terminal held by the employee to be an employee. May output a message to the effect that it should respond directly. Also in this case, the control unit 46 may notify the employee of the position of the customer.

  The first information may include voice data collected by the microphone 14 immediately before the change in emotion. The control unit 46 indicates that the user is angry or dissatisfied when the voice data contains anger or sigh of the user or an apology of the employee by analysis of the voice data (negative state (negative state) It can be determined that

  It is assumed that the pattern of the correspondence process described above is predetermined, and the pattern is stored in the storage unit 41.

  After the process of step S58 ends as described above, the process returns to step S50.

  On the other hand, if the determination in step S52 is negative, that is, if the information input from the basket device 20 is the second information, the process proceeds to step S60.

  At step S60, the control unit 46 performs purchase behavior analysis based on the second information. In this case, the control unit 46 analyzes what the user has seen, bought, and purchased while shopping. In other words, it can be said that analysis is made as to which product the user did not see or take.

  FIG. 9 shows the order of items picked up by a certain user from entering the store to passing the cash register. Further, in FIG. 9, the item purchased by the user is underlined. The control unit 46 can use the purchase information acquired from the cash register terminal as the information on the item purchased by the user.

  In FIG. 9, the user has bought curry roux A and curry roux B in hand but eventually purchases curry roux A. In this case, the control unit 46 purchases the curry roux A by using the line-of-sight image when the curry roux A is picked up and the line-of-sight image when the curry roux B is picked up as reference. To analyze. For example, when price information is included for a long time in an area (a predetermined position (approximately the center) of the gaze image) at which the user is gazing, it is determined that the curry roux A has been purchased based on the price. In addition, when the display of hotness, the photograph of the package, the display of ingredients, and the expiration date were included for a long time in the area the user was looking at, Curry Lou A was purchased based on the information. I will judge. Note that the control unit 46 extracts, using the OCR unit 43, whether the display of the price or hotness, the component, the expiration date, and the like are included in the area in which the user is gazing.

  In addition, what is necessary is just to judge the judgment standard at the time of purchasing also about beef A and B similarly. For example, when the display of the internal capacity, the display of the part, the price per 100 g, and the like are included in the area where the user is gazing at a long time, the control unit 46 selects beef based on the information. It is determined that B has been purchased.

  In addition, the control unit 46 refers to data acquired by a plurality of users to count what another product the user who has purchased a certain product has compared, and as a result of comparing and considering. It is possible to count what decide the product. The analysis result of the judgment criteria at the time of purchase may be provided to the manufacturer or producer of each product. Manufacturers and producers can use this analysis result for future product development and product improvement.

  In addition, the control unit 46 can count the order in which the customer takes an article by referring to the data in FIG. 9. In this case, the control unit 46 can analyze the tendency of the movement of the customer in the store by using the counting result. Thereby, the control unit 46 can obtain an appropriate arrangement of articles or shelves based on the movement of the customer.

  In addition, the control unit 46 counts the number and position of the user in a positive state and the number and position of a negative state in the period from entry to exit of the store, whereby the degree of customer satisfaction or dissatisfaction Can be analyzed by location (by department).

  Further, the control unit 46 may analyze the line-of-sight image when the customer changes from a negative state to a positive state. As a result of analyzing the line-of-sight image, when the line-of-sight image includes an employee's face during the change from the negative state to the positive state, the control unit 46 solves the problem by the employee's response. Can be analyzed. In addition, when the handling process using the mounting apparatus 10 or in-store broadcasting is performed while changing from the negative state to the positive state, it can be analyzed that the handling process is appropriate.

  As described above in detail, according to the first embodiment, the control unit 46 and the communication unit 42 of the server 40 input the line-of-sight image and the article information as the operation information related to the purchase of the user The detection result of the biological sensor 21 is input as emotion information. As a result, the control unit 46 can analyze the purchase based on the line-of-sight image, the item information, and the detection result of the biological sensor 21 to obtain and provide useful information on the purchase. Therefore, usability by the user can be improved. In this case, by using a line-of-sight image taken from the side of the user, compared with the case where an image taken from the shelf side of the store is used, the user actually looks at the situation, ie, an article ahead of the line of sight It becomes possible to analyze in consideration of display, situation etc.

  Here, conventionally, when a user wearing a device equipped with the imaging unit 11 such as the mounting device 10 has entered the store, the store's appearance and products are picked up without permission, so the store It can not be said that there is a merit for the side. On the other hand, in the first embodiment, instead of allowing the user to wear the mounting apparatus 10 and entering the shop, the shop side uses the car apparatus 20 for the image data captured by the imaging unit 11. I am going to get it. Thus, the store can analyze the behavior and emotion of the user until the product purchase by analyzing the image data.

  Also, in the past, the degree of satisfaction with shopping etc. was to be collected by a questionnaire to the user, but it is necessary for the user to access the website of the store and answer questions or write answers on paper It was embarrassing. On the other hand, in the first embodiment, the image data acquired by the mounting apparatus 10 and the biometric information of the user are associated with each other and provided to the store side, so that the user's satisfaction is unknowingly It is possible to transmit the degree of dissatisfaction to the store.

  In the first embodiment, a message to the user and a message to the employee are output based on the gaze image, the item information, and the detection result of the biological sensor 21. This makes it possible to immediately respond to the user who feels unsatisfactory.

  Further, in the first embodiment, the control unit 46 determines that the user holds the article from the gaze image and the article information. As a result, the control unit 46 can acquire and provide useful information related to purchasing by analyzing the information that the user holds the item, the order of holding, and the like.

  Further, in the first embodiment, the control unit 46 can easily determine whether a change in emotion has occurred in the user, because the detection result of the biological sensor 21 is used. Further, by analyzing the line-of-sight image of the user along with the detection result of the biological sensor 21, it is possible to accurately determine whether the user's emotion has become positive or negative.

  In the first embodiment, the mounting device 10 captures a gaze image, and transmits the captured gaze image to the car device 20 in contact with the user via human body communication. Thereby, the basket device 20 can acquire only the line-of-sight image of the user holding the basket 120. That is, the line-of-sight image is surely acquired from the mounting apparatus 10 of the user who entered the store, and the line-of-sight image is acquired from the mounting apparatus 10 of a person who has not entered the store (for example, a person who passed by the store). It is possible to reliably prevent it.

  In the first embodiment, the first information is transmitted to the server 40 when there is a change in the detection result of the biometric sensor 21, that is, when there is a change in the user's emotion (S42) ). In this way, information that requires analysis can be transmitted to the server 40 at an appropriate timing.

  In addition, although the case where the cage apparatus 20 includes the biosensor 21 has been described in the first embodiment, the present invention is not limited to this, and the mounting apparatus 10 may include the biosensor. In this case, a living body sensor that contacts a part of the face may be provided in a part of the frame of the mounting apparatus 10 or in the vicinity of the frame. Further, a wearable bio-sensor may be connected to the earlobe of the wearing apparatus 10.

  In the first embodiment, the case where the article device 30 is provided on the article 130 has been described. However, the present invention is not limited to this, and the article apparatus 30 may not necessarily be provided in the article 130 when article information is not necessary in emotion analysis in step S54 and purchase behavior analysis in step S60. For example, when a state in which an item is held in a hand is imaged in the line-of-sight image, the image recognition may be performed to specify the item held in a hand.

  In the first embodiment, although the case where the article device 30 performs human body communication with the basket device 20 has been described, the present invention is not limited thereto, and the article device 30 may perform human body communication with the mounting device 10 It is good to do. In this case, the article information acquired by the mounting apparatus 10 may be transmitted from the mounting apparatus 10 to the car apparatus 20 in a state where human body communication is established between the mounting apparatus 10 and the car apparatus 20.

  In the first embodiment, the detection result of the biosensor 21 is used. However, the present invention is not limited to this, and the detection result of the biosensor 21 may not be used. In this case, the control unit 46 estimates that the user is in a positive state, for example, when it is determined that the user gazes at the mark of FIG. 7A for a predetermined time based on the gaze image. You may do it. Further, for example, when the control unit 46 determines that the user gazes at the sticker sheet of FIG. 7B or the registration waiting state of FIG. 7C for a predetermined time based on the gaze image, the user is negative. It is possible to assume that it is in the In this case, the control unit 46 stores in advance a mark, a sticker sheet, and an image pattern of a situation predicted to be in a positive or negative state when viewed by the user, and compares the image pattern with the gaze image. And it is sufficient.

  In the first embodiment, the case where the predetermined position of the line-of-sight image is regarded as the center position of the line of sight has been described, but the present invention is not limited to this. For example, it has an infrared irradiating unit and an infrared light receiving unit, and the infrared irradiating unit irradiates infrared rays to the eye, and the infrared light reflected by the eye is received by the infrared receiving unit to detect the gaze direction. The detection unit may be separately provided in the mounting apparatus 10.

  Note that the server 40 may add up customer satisfaction levels and dissatisfaction levels in a plurality of stores and publish the results as user evaluations of the respective stores on one web site. In this case, for example, the evaluation of each store can be announced by the number of stars, the score of satisfaction, or the like.

Second Embodiment
Next, an electronic device system 100 ′ according to the second embodiment will be described in detail based on FIGS.

  The configuration of the electronic device system 100 'is shown in a block diagram in FIG. In FIG. 10, the configuration different from the electronic device system 100 of the first embodiment among the electronic device system 100 'is shown by a bold line frame.

  As shown in FIG. 10, in the electronic device system 100 ', the mounting device 10' includes the speaker 18, the car device 20 'includes the weight sensor 27, and the article device 30' includes the light emitting unit 34. The configuration of the server 40 'is the same as that of the server 40 of the first embodiment.

  The speaker 18 is used to provide audio information to the user wearing the mounting device 10 '. As the speaker 18, in addition to earphones and headphones, a speaker having directivity and capable of providing audio information only to a user wearing the mounting device 10 'can be adopted. It is assumed that the storage unit 15 of the mounting apparatus 10 'stores article information corresponding to the light emission pattern of the light emission unit 34 of the article apparatus 30' described later.

  The weight sensor 27 is provided at a part of the car device 20 ′ and is a sensor for detecting a change in the internal volume of the car 120.

  The light emitting unit 34 includes a light emitting element such as a light emitting diode (LED), and emits light with a light emitting pattern predetermined for each article. The light emission of the light emitting unit 34 is controlled by the control unit 33. The control unit 17 of the mounting apparatus 10 ′ estimates that the user looks at an article corresponding to the light emission pattern when the light emission pattern is imaged within a predetermined range of the angle of view in moving image shooting by the imaging unit 11. The on / off switching timing of the light emission by the light emitting unit 34 (the blinking interval of the light emission pattern) is set in accordance with the frame rate of the imaging unit 11. A visible light LED may be used as the light emitting unit 34 or an infrared LED may be used. When an infrared LED is used, an infrared imaging device may be used for the imaging unit 11.

  Next, processing of the electronic device system 100 'in the second embodiment will be described in detail along the flowcharts of FIGS. 11 to 13 and with reference to other drawings as appropriate. Also in the second embodiment, a case where the electronic device system 100 'is used in a supermarket will be described.

(Processing of mounting device 10 ')
The process of mounting device 10 'is shown by the flowchart by Fig.11 (a). In FIG. 11A, the steps different from the process (FIG. 4A) of the mounting apparatus 10 of the first embodiment are indicated by thick frames.

  In the process of FIG. 11A, the control unit 17 of the mounting device 10 'stands by until human body communication with the car device 20' is established in step S10. When human body communication with the basket device 20 'is established, the control unit 17 proceeds to step S110.

  When the process proceeds to step S110, the control unit 17 instructs the imaging unit 11 to perform imaging (moving image) from the user side.

  Next, in step S112, the control unit 17 determines whether the light emission of the light emitting unit 34 has been captured within a predetermined range of the angle of view. That is, it is determined whether the user has seen the article provided with the article apparatus 30 '. If the determination in step S112 is negative, the process proceeds to step S116, but if the determination in step S112 is affirmed, the process proceeds to step S114.

  When the process proceeds to step S114, the control unit 17 transmits the article information seen by the user to the car apparatus 20 '. Specifically, the control unit 17 acquires article information corresponding to the light emission pattern detected within the predetermined range of the angle of view of the imaging unit 11 from the storage unit 15, and the communication device 16 It transmits to the first communication unit 24.

  After the determination in step S112 is negative or after the process in step S114 is performed, when the process proceeds to step S116, the control unit 17 determines whether the information is received from the car apparatus 20 '. The information received from the basket device 20 'is information related to the article or the service of the store transmitted from the server 40'. If the determination here is denied, the processing proceeds to step S16, but if the determination is affirmative, the processing proceeds to step S118.

  At step S118, the control unit 17 outputs the information received from the car apparatus 20 '. The details of the process of step S118 will be described later.

  After the determination in step S116 is denied or after the process of step S118 is performed, when the process proceeds to step S16, the control unit 17 establishes human-body communication with the basket device 20 'which has received an input from the cash register terminal. Determine if it is or not. Here, with the input from the cashier terminal, when the user finishes shopping and the cashier 120 is brought to the cashier, communication between the cart device 20 'and the cashier terminal allows the cashier terminal 20 to communicate with the cashier terminal. It means input of the end of shopping information sent to '. That is, when human body communication between the basket device 20 'and the mounting device 10' which has been input from the cash register terminal is established, it means that the user's shopping has ended. Therefore, if the shopping has not ended and the determination in step S16 is negative, the process returns to step S112, and the processing and determination in steps S112 to S16 are repeated. On the other hand, when the user's shopping is finished and the determination in step S16 is affirmed, the whole process of FIG. 11A is finished.

(Processing of the article device 30 ')
In FIG. 11 (b), the process of the article device 30 'is shown in the form of a flowchart. In addition, in FIG.11 (b), the step different from the process (FIG.4 (b)) of the articles | goods apparatus 30 of 1st Embodiment is shown by the bold line frame.

  As shown in FIG. 11B, in the article device 30 ', in step S120, the control unit 33 performs light emission using the light emitting unit 34. In this case, the control unit 33 causes the light emitting unit 34 to emit light with a light emission pattern corresponding to the article provided with the article device 30 '. The other processing is the same as that of FIG.

(Process of basket device 20 ')
In FIG. 12, the process of the car apparatus 20 'is shown in a flowchart. In the process of FIG. 12, in step S130, the control unit 26 stands by until human body communication with the mounting apparatus 10 'is established. When human body communication with the mounting apparatus 10 'is established and the process proceeds to step S132, the control unit 26 determines whether or not article information is received from the mounting apparatus 10'.

  If the determination in step S132 is affirmed, the process proceeds to step S134, and the control unit 26 transmits the article information received from the mounting device 10 'to the server 40' as the article information of the viewed article, and step S136. Migrate to On the other hand, if the determination in step S132 is negative, the process proceeds to step S136 without passing through step S134.

  In step S136, the control unit 26 determines whether item information has been received from the item device 30 '. If the determination in step S136 is affirmed, the process proceeds to step S138, in which the control unit 26 transmits the article information received from the article device 30 'to the server 40' as the article information of the touched article, and step S140. Migrate to On the other hand, if the determination in step S136 is negative, the process proceeds to step S140 without passing through step S138.

  In step S140, the control unit 26 determines whether the detected value of the weight sensor 27 has changed. Here, the case where the detection value of the weight sensor 27 has changed means that the article has been newly inserted into the car 120. Although the detection value of the weight sensor 27 also changes when the articles in the basket 120 are taken out, it will be described here that the articles are not taken out from the basket 120 for convenience of explanation. If the determination in step S140 is affirmed, the process proceeds to step S142, and the control unit 26 changes the item information of the item touched immediately before into the item information of the item put in the basket, that is, the item the user intends to purchase To the server 40 'as the On the other hand, if the determination in step S140 is negative, the process proceeds to step S144 without passing through step S142.

  In step S144, the control unit 26 determines whether information has been received from the server 40 '. The information received from the server 40 'is information related to the service of the article or the store. If the determination in step S144 is affirmed, the process proceeds to step S146, the control unit 26 transmits the information received from the server 40 'to the mounting apparatus 10', and proceeds to step S148. On the other hand, if the determination in step S144 is negative, the process proceeds to step S148 without passing through step S146.

  In step S148, the control unit 26 determines whether there is an input from the cashier terminal. Here, the input from the cash register terminal means the input of the shopping end information as described above.

  If the determination in step S148 is negative, the process proceeds to step S132, and the processing and determination in steps S132 to S148 are repeated. However, if the determination in step S148 is affirmed, the control unit 26 performs the process shown in FIG. End all processing.

(Processing of server 40 ')
In FIG. 13, the process of the server 40 'is shown in a flowchart.

  In the process of FIG. 13, in step S <b> 150, the control unit 46 stands by until the article information is received from the car device 20 ′. That is, when any one of the article information of the article viewed by the user, the article information of the article touched by the user, and the article information of the article placed in the basket 120 is input, the control unit 46 proceeds to step S152.

  In step S152, the control unit 46 updates the shopping DB stored in the storage unit 41. Here, the shopping DB has a data structure as shown in FIG. 14 (a). As shown in FIG. 14 (a), it has the article name, seen, touched, and put in the basket. When the control unit 46 receives the item information of the item viewed by the user, stores “o” in the “Viewed item” column of the item, and when the item information of the item touched by the user is received, the control unit 46 touches the item A circle is stored in the field, and when the item information of an article put in the basket is received by the user, the circle is stored in the box put in the basket of the article.

  Next, in step S154, the control unit 46 analyzes the shopping DB. For example, the control unit 46 refers to the shopping DB of FIG. 14A, and when a predetermined time has passed after holding the beef A in hand (when the new article information is not received for a predetermined time) Can analyze that the user may need information of an item (for example, beef B) that can be a comparison control with beef A. In addition, the control unit 46 can estimate that the user makes curry from, for example, an article put in a basket, so the amount of beef necessary for making curry (Xg for four dishes, Yg for eight dishes, etc.) Can be analyzed as the user may need it. In addition, the control unit 46 may be required by the user for information on bargains for curry dinners (such as pickles, loin cutlets, desserts, etc.) and information on store campaign information (such as buying an extra Z yen, which can pull the bully). Can be analyzed.

  Next, in step S156, the control unit 46 determines whether there is information to be notified to the user as a result of the analysis in step S154. If the determination here is denied, the process returns to step S150, but if the determination is affirmed, the process proceeds to step S158. When the user touches the same article multiple times, or when the user touches an article similar to the previously touched article, the same information as the information notified in the past notifies the user of the same information. It may be judged. Therefore, the control unit 46 may restrict the notification on the same information as the information notified in the past, not on the information to be notified to the user. That is, the control unit 46 limits the information to be notified when the same article is touched a plurality of times or viewed a plurality of times, or when an article similar to the article touched or seen before is touched or viewed. You may do it. By doing this, it is possible to prevent the same user from being repeatedly notified of the same information.

  The control unit 46 may change the information to be notified in accordance with the number of times the same article is touched or the number of times the same article is viewed. For example, the control unit 46 sets the information of the item itself at the first time, the information of the item to be compared with the item at the second time, and the discount information related to the item at the third time. You may do it. By doing this, it becomes possible to improve the possibility that a customer who is lost in purchasing an item can purchase the item. Instead of this, according to the time when the user touched the same article, the user may be notified from information not notified to the user from the second time onwards.

  If the determination in step S156 is affirmed, the control unit 46 proceeds to step S158 and transmits information to notify the user to the car apparatus 20 '. When the process of step S158 is performed, in the process of FIG. 12, the determination of step S144 is affirmed, and the process of step S146 (the process of transmitting the information received from the server to the mounting apparatus 10 ′) is performed. . Further, when the process of step S146 is executed in FIG. 12, the determination of step S116 of FIG. 11A is affirmed, and the process of step S118 (process of outputting the received information) is executed. In step S118, the control unit 17 uses the display unit 12 to display information to be notified to the user, and uses the speaker 18 to output information to be notified to the user. After step S158, the process returns to step S150, and the processes of steps S150 to S158 are repeated.

  The control unit 46 may perform various analyzes based on the shopping DB (for example, FIG. 14B) of the user who has finished shopping. In this case, for example, based on FIG. 14 (b), the control unit 46 compares curry leu A with curry leu B and beef A with beef B to analyze that curry leu B and beef B are purchased. it can. The control unit 46 can also analyze the order of walking in the store, the order of purchase, and the like. In addition, the control unit 46 can identify and analyze an item that has not been purchased just by looking at the item, and an item that has been picked up but not purchased. The control unit 46 may provide the analysis result to a manufacturer, a seller or the like of the product.

  As described above in detail, according to the second embodiment, at least one of the fact that the user has obtained the article and the fact that the article has entered a predetermined range within the field of view of the user according to the second embodiment Accordingly, the item information of the item is input (S150: Yes), and notification regarding the input item information is performed (S158). Therefore, in the second embodiment, it is possible to notify the user of useful information related to an item touched by the user or an item viewed by the user. Thereby, the usability of the user can be improved.

  Further, in the second embodiment, the control unit 46 inputs the information of the product scheduled to be purchased by the user put in the basket 120 as the item information, and performs notification according to the input product. Thereby, it is possible to notify the user of useful information related to the product which the user intends to purchase. Further, since the control unit 46 inputs the information of the product scheduled to be purchased from the basket device 20 'used at the time of shopping, it is possible to input accurate information as the information of the product scheduled to be purchased.

  Further, in the second embodiment, since the notification of information to the user is performed from the display unit 12 of the mounting apparatus 10 ′ and the directional speaker 18, the user's Notifications can be made according to purchasing behavior.

  In the second embodiment, the control unit 46 may determine that the two items are compared and examined when the user is looking at another item while holding the item.

  In the second embodiment, although an example of using the electronic device system 100 'in a supermarket has been described, the present invention is not limited to this, and the electronic device system 100' may be used in a clothing store or the like. For example, when the user picks up the clothes, the control unit 46 buys one more piece of information on presence / absence of inventory such as different colors, different sizes, information as to whether or not there are items on sale with similar items. If you buy more than ¥ 2000 and discount information etc. is notified using the display unit 12 or the speaker 18, you can know a specific user (a member such as a member of the store without knowing other users) ) Can only be notified. In addition, the server 40 'may transmit discount information, present information, and a message of celebration to the mounting apparatus 10' when a user of a member of a store visits a birthday.

  In the second embodiment, the control unit 17 transmits the article information acquired by the mounting apparatus 10 ′ to the server 40 ′ via the car apparatus 20 ′. However, the present invention is limited thereto. It is not a thing. For example, when the mounting apparatus 10 'has a communication unit capable of wireless communication, the information may be transmitted from the mounting apparatus 10' to the server 40 'by wireless communication.

  In the second embodiment, although the case where the article apparatus 30 'includes the light emitting unit 34 has been described, the present invention is not limited thereto. The article apparatus 30' may not include the light emitting unit 34. In this case, the control unit 17 may identify the article viewed by the user by analyzing the image captured by the imaging unit 11. For example, the control unit 17 may recognize characters included in the display (for example, characters of a product tag or a POP advertisement) and specify an item viewed by the user.

  In the first and second embodiments, the control unit 46 may consider the type of user when analyzing information. If the type of the user is a member of the store, it can be grasped from the registered information (name, date of birth, address, shopping history). In this case, the purchasing behavior of the user may be analyzed for each gender and age of the user.

  Each embodiment mentioned above is an example of a suitable operation of the present invention, and can be combined suitably. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 10 mounting apparatus 11 imaging part 12 display part 16 communication part 17 control part 18 speaker 20 basket apparatus 42 communication part 46 control part 110 flame | frame 120 basket

Claims (19)

Is installed in a store, when the user proximity communication with possession device possessed a visiting detector for detecting the visiting of the user into the shop,
A first input unit for inputting operation information related to a user's purchase when the user's entry is detected by the entry detection unit;
A second input unit capable of communicating with the possessing device and inputting biological information on the living body of the user;
An electronic device system comprising: The second input unit repeatedly inputs the biometric information of the user when the entry detection unit detects the entry of the user.
The electronic device system according to claim 1, further comprising an emotion detection unit configured to detect a change in emotion of the user based on the biological information input by the second input unit. The first input unit, when the entry detection unit detects the entry of the user and the emotion detection unit detects a change in the user's emotion, inputs operation information related to the purchase of the user,
The first input unit does not input operation information related to the purchase of the user if the emotion detection unit does not detect a change in emotion of the user even if the entrance detection unit detects the entrance of the user. The electronic device system according to Item 2. The electronic device system according to any one of claims 1 to 3, wherein the first input unit can communicate with a communication unit of an item held by the user.   The electronic device system according to any one of claims 1 to 4, wherein the second input unit inputs biometric information of the user that can be used to estimate a change in emotion. The electronic device system according to any one of claims 1 to 5 , further comprising: a third input unit installed in the store and configured to input shopping end information of the user. An emotion detection unit that detects a change in emotion of the user based on the biological information input by the second input unit;
And a third input unit installed in the store to input shopping end information of the user.
The first input unit detects a change in emotion of the user by the emotion detection unit after the entrance detection unit detects the entrance of the user, and the entrance detection unit inputs the user 2. The electronic device system according to claim 1, wherein when shopping end information of the user is input by the third input unit after the detection of, operation information regarding purchasing of the user is input. An emotion detection unit that detects a change in emotion of the user based on the biological information input by the second input unit;
And a third input unit installed in the store to input shopping end information of the user.
The first input unit is
When the change in emotion of the user is detected by the emotion detection unit after the entrance detection unit detects the entrance of the user, operation information related to the purchase of the user before the emotion change is input as the first information. ,
When shopping end information of the user is input by the third input unit after the entry detection unit detects the entry of the user by the third input unit, operation information related to purchasing of the user after the entry is second information The electronic device system according to claim 1, wherein The electronic according to any one of claims 1 to 8 , further comprising: a detection unit that detects the degree of satisfaction or unsatisfaction of the user based on information input by the first input unit and the second input unit. Equipment system . The electronic device system according to any one of claims 1 to 9 , further comprising: an instruction unit configured to instruct a response to the user based on information input by the first input unit and the second input unit. 11. The electronic device system according to claim 10 , wherein the instruction unit instructs the user to respond via the possessed device possessed by the user. The electronic device system according to any one of claims 1 to 11 , wherein the first input unit inputs information on the movement of the user's hand. The electronic device system according to any one of claims 1 to 12 , wherein at least one of the first input unit and the second input unit inputs information on an eye from the user side. The electronic device system according to claim 13 , further comprising: a communication unit capable of communicating with an acquisition device that acquires information on eyes from the user side. The electronic device system according to claim 14 , wherein the communication unit communicates with a communication unit provided in the acquisition device. The electronic device system according to claim 15 , wherein the communication unit communicates with the acquisition device via a communication relay device different from the acquisition device. The electronic device system according to claim 16 , wherein the communication unit communicates with the acquisition device via near field communication or human body communication with the communication relay device. The acquisition device includes an imaging unit that performs imaging.
Electronic device system according to any one of claims 1 4 to 1 7 comprising an analysis unit for analyzing the imaging data to which the imaging unit has captured. The electronic device system according to any one of claims 1 to 18 , wherein the first input unit inputs the operation information according to a change in the biological information input by the second input unit.

Images