JP2015111357A - Electronic apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a convenient electronic apparatus.SOLUTION: An electronic apparatus includes first input units (42, 46) that receive the input of operation information on the purchase by a user, and second input units (42, 46) that receive the input of emotion information on the purchase by the user.

Description

  The present invention relates to an electronic device.

  2. Description of the Related Art Conventionally, a sales promotion system that acquires behavior information related to purchase of merchandise by a store visitor and links the information to sales promotion has been proposed (for example, see Patent Document 1).

JP 2010-277256 A

  However, in the conventional sales promotion system, it cannot be said that necessary information is sufficiently acquired, and it is not easy to use.

  The present invention has been made in view of the above problems, and an object thereof is to provide an electronic device that is easy to use.

  The electronic device of the present invention includes a first input unit (42, 46) for inputting operation information related to user purchase, and a second input unit (42, 46) for inputting emotion information related to the user purchase. ing.

  In this case, a detection unit (46) may be provided that detects satisfaction or dissatisfaction of the user based on information input by the first input unit and the second input unit. Moreover, you may provide the instruction | indication part (46) which instruct | indicates the response | compatibility to the said user based on the information which the said 1st input part and the said 2nd input part input. In addition, the first input unit may input information regarding the movement of the user's hand. The second input unit may input information related to the user's living body.

  In the electronic device of the present invention, at least one of the first input unit and the second input unit may input information about the eyes from the user side. In this case, you may provide the communication part (42) which can communicate with the acquisition apparatus which acquires the information regarding the eye from the said user side. In this case, the communication unit may communicate with a communication unit (25) provided in the acquisition device. Moreover, the said communication part is good also as communicating with the said acquisition apparatus via the communication relay apparatus (20) different from the said acquisition apparatus. In this case, the communication unit may communicate with the acquisition device via proximity communication or human body communication with the communication relay device. The acquisition apparatus may include an imaging unit (11) that performs imaging, and an analysis unit (46) that analyzes imaging data captured by the imaging unit.

  In the electronic device of the present invention, the first input unit may input the operation information in accordance with a change in the emotion information input by the second input unit.

  An electronic device according to the present invention includes an input unit (42, 46) for inputting information related to eyes from the user side, and an estimation unit for estimating emotion information related to purchase of the user based on information related to eyes from the user side. 46).

  In this case, the estimation unit may estimate the emotion information related to the user's purchase based on whether or not the information related to the eyes from the user side matches or is similar to a predetermined pattern.

  The electronic device according to the present invention is an electronic device (10) that can be worn near a user's face, and includes an acquisition unit (17) that acquires information about the eyes from the user side, and information acquired by the acquisition unit. A communication unit (16) that communicates with an external device (20) in contact with the user via human body communication.

  In addition, in order to explain the present invention in an easy-to-understand manner, the above description has been made in association with the reference numerals of the drawings representing one embodiment. However, the present invention is not limited to this, and the configuration of an embodiment described later May be modified as appropriate, or at least a part thereof may be replaced with another component. Further, the configuration requirements that are not particularly limited with respect to the arrangement are not limited to the arrangement disclosed in the embodiment, and can be arranged at a position where the function can be achieved.

  The present invention has an effect of providing an electronic device that is easy to use.

It is a block diagram which shows the structure of the electronic device system which concerns on 1st Embodiment. It is a figure which shows the utilization condition of a mounting apparatus, a basket apparatus, and an article | item apparatus. It is a perspective view of a mounting apparatus. FIG. 4A is a flowchart showing processing of the mounting device according to the first embodiment, and FIG. 4B is a flowchart showing processing of the article device according to the first embodiment. It is a flowchart which shows the process of the basket apparatus concerning 1st Embodiment. It is a flowchart which shows the process of the server concerning 1st Embodiment. FIG. 7A and FIG. 7B are diagrams (part 1) illustrating an example of a line-of-sight image. It is a figure (the 2) which shows the example of a look image. It is a figure which shows the order of the articles | goods picked up in a hand after a certain user enters a shop until passing a cash register. It is a block diagram which shows the structure of the electronic device system which concerns on 2nd Embodiment. FIG. 11A is a flowchart showing processing of the mounting device according to the second embodiment, and FIG. 11B is a flowchart showing processing of the article device according to the second embodiment. It is a flowchart which shows the process of the basket apparatus concerning 2nd Embodiment. It is a flowchart which shows the process of the server concerning 2nd Embodiment. FIG. 14A and FIG. 14B are diagrams illustrating an example of the shopping DB.

<< First Embodiment >>
Hereinafter, the electronic device system according to the first embodiment will be described in detail with reference to FIGS. FIG. 1 is a block diagram showing the configuration of an electronic device system 100 according to the first embodiment.

  As shown in FIG. 1, the electronic device system 100 includes a mounting device 10, a basket device 20, an article device 30, and a server 40. As shown in FIG. 2, the wearing device 10 is a glasses-type terminal that is worn on the face by a user (shopper) visiting the store. The basket device 20 is a device provided in the basket 120 installed in the store. The article device 30 is an apparatus provided in an article (product) 130 displayed in the store. The server 40 is an information processing apparatus provided in a store or a data center. Communication between the mounting device 10 and the basket device 20 and between the article device 30 and the basket device 20 is possible via a human body (human body communication), and between the basket device 20 and the server 40. Wireless communication is possible.

(Mounting device 10)
As shown in FIG. 1, the mounting device 10 includes an imaging unit 11, a display unit 12, an operation unit 13, a microphone 14, a storage unit 15, a communication unit 16, and a control unit 17. In FIG. 3, the mounting device 10 is shown in a perspective view. As shown in FIG. 3, the mounting apparatus 10 includes a glasses-type frame 110. The configuration of the mounting device 10 illustrated in FIG. 1 and not illustrated in FIG. 3 is provided inside the frame 110 or a part of the frame 110.

  The imaging unit 11 includes a lens, an imaging device, an image processing unit, and the like, and captures still images and moving images. As shown in FIG. 3, the imaging unit 11 is provided near the end of the frame 110 (near the user's right eye). For this reason, in a state where the user wears the mounting apparatus 10 (the state shown in FIG. 2), an image in a direction in which the user is facing (viewing) can be taken. In addition, a predetermined portion (for example, a portion slightly deviated to the left from the center) of the captured image can be regarded as a portion where the user is turning his / her line of sight (a portion being watched).

  The display unit 12 includes a projector provided in or near the frame 110 and a prism for guiding a projection image from the projector to the eyes of the user. The display unit 12 displays various information under the instruction of the control unit 17.

  The operation unit 13 is a touch pad provided on the frame 110, detects the movement of the user's finger, receives an operation from the user, and transmits the received operation information to the control unit 17.

  Details of the imaging unit 11, the display unit 12, and the operation unit 13 are also disclosed in, for example, US Published Patent No. 2013/0044042.

  The microphone 14 is provided in the frame 110 and collects voices uttered by the user and voices around the user. The voice collected by the microphone 14 is voice-recognized by a voice recognition unit (not shown), and the voice recognition result is transmitted to the control unit 17. The control unit 17 executes processing based on the voice recognition result (for example, command execution processing). The control unit 17 may perform voice recognition. Further, the control unit 17 stores the collected audio data in the storage unit 15.

  The storage unit 15 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores image data captured by the imaging unit 11, audio data collected by the microphone 14, display data displayed on the display unit 12, and the like. .

  The communication unit 16 performs human body communication with other devices. The communication unit 16 includes an electrode unit 16a that is provided on the frame 110 and can contact a user, and a human body communication unit 16b that performs human body communication using the electrode unit 16a. Human body communication includes a current system in which a weak current is passed through the human body and the current is modulated to transmit information, and an electric field system in which information is transmitted by modulating an electric field induced on the surface of the human body. . In this embodiment, it is possible to use either the current method or the electric field method. Note that human body communication is possible not only when the user directly contacts the electrode portion 16a of the mounting apparatus 10, but also when the user does not directly contact the electrode portion 16a and the user. In the present embodiment, the communication unit 16 transmits image data captured by the imaging unit 11 and audio data collected by the microphone 14 to the basket device 20. Note that the communication unit 16 may perform near field communication with other devices such as Bluetooth (registered trademark), RFID (Radio Frequency Identification), TransferJet (registered trademark), or the like.

  Note that the mounting device 10 may have a wireless communication unit instead of the communication unit 16 or in combination with the communication unit 16. In this case, image data and sound data may be directly transmitted to the server 40 by wireless communication, or image data and sound data may be transmitted to the basket device 20 by wireless communication.

  The control unit 17 comprehensively controls the entire mounting apparatus 10. The control unit 17 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. In the present embodiment, the control unit 17 performs control related to imaging of the imaging unit 11, control related to display of the display unit 12, and the like.

  Note that the wearing device 10 may not be a glasses-type terminal. For example, it may be a wearable terminal that can be worn at any position on the body, and some of its functions may be separated. As an example, a wristwatch type terminal may have other functions as a contact lens type terminal provided with a display unit.

(Cage device 20)
As shown in FIG. 1, the basket device 20 includes a biological sensor 21, a calendar unit 22, a storage unit 23, a first communication unit 24, a second communication unit 25, a control unit 26, and the like.

  The biosensor 21 is provided in a portion (a handle portion) that the hand touches when the user holds the cage 120 in the cage 120. The biological sensor 21 detects changes in biological information of the body such as heart rate, blood pressure, and amount of sweat, and includes, for example, an LED (light emitting diode), a photo sensor, and a sweat sensor. For example, by using an LED and a photosensor, a heart rate can be detected by receiving light reflected by the photosensor when light emitted from the LED is applied to a finger. In addition, the sweat sensor can detect the amount of sweat by measuring the impedance of the hand with a plurality of electrodes. In the present embodiment, the output from the biosensor 21 is sent to the control unit 26. Then, the control unit 26 estimates a change in the user's emotion (feeling emotional) based on the output of the biological sensor 21.

  The calendar unit 22 acquires time information such as year, month, date, and time and outputs the time information to the control unit 26. The calendar unit 22 measures time.

  The storage unit 23 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores information related to an article obtained by the user as will be described later. Further, the storage unit 23 stores an ID number (cage ID) for uniquely identifying the basket device 20.

  The first communication unit 24 performs human body communication with other devices. Note that the first communication unit 24 may perform near field communication with other devices. In the present embodiment, the first communication unit 24 communicates with the communication unit 16 of the mounting device 10 and the communication unit 32 of the object device 30 using human body communication.

  The first communication unit 24 includes an electrode unit 24a that is provided on the handle portion of the cage 120 and that can contact the user's hand or arm, and a human body communication unit 24b that performs human body communication via the electrode unit 24a. .

  The second communication unit 25 performs wireless communication with other devices. In the present embodiment, wireless communication with the server 40 is performed, and article information of an article touched by the user, image data captured by the imaging unit 11, audio data collected by the microphone 14, and detection of the biosensor 21 The result is transmitted to the server 40. Note that the basket device 20 of the present embodiment is a communication relay device that relays communication of article information between the article device 30 and the server 40 and communication of image data and audio data between the mounting device 10 and the server 40. As a function.

  The control unit 26 controls the entire basket device 20 and includes a CPU, a RAM, a ROM, and the like. In the present embodiment, the control unit 26 controls information communication with the article device 30, the mounting device 10, and the server 40.

  The basket device 20 may be provided in a cart used in a store. In this case, it is preferable that the biosensor 21 and the electrode portion 24a of the first communication unit 24 are provided in the grip portion of the cart.

(Article device 30)
The article device 30 is an article (product) sold in a store or the like, and is provided on an article that comes into contact with the body of a visitor (user), such as food, clothes, shoes, watches, accessories, golf clubs, and bags. . In the present embodiment, for the sake of convenience of explanation, the article device 30 will be described as being provided on a product (foodstuff or the like) displayed in a supermarket.

  As illustrated in FIG. 1, the article device 30 includes a storage unit 31, a communication unit 32, and a control unit 33.

  The storage unit 31 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores information related to an article (product) to which the article device 30 is attached. Specifically, the storage unit 31 stores article information (product information) of an article to which the article device 30 is attached. For example, when the article device 30 is attached to a processed food (for example, canned food), the storage unit 31 includes, as article information, a manufacturer (seller), a product name, a raw material name, an expiration date, an internal capacity, Remember the country.

  The control unit 33 controls the entire article device 30 and includes a CPU, a RAM, a ROM, and the like. In the present embodiment, the control unit 33 controls transmission of the article information stored in the storage unit 31 to the basket device 20. The control unit 33 and the storage unit 31 may be provided at any position of the article.

  The communication unit 32 performs human body communication with other devices. Note that the communication unit 32 may perform near field communication with other devices. In this embodiment, the communication part 32 is provided with the electrode part 32a provided in the position which contacts a user (for example, hand), and the human body communication part 32b which performs human body communication using the electrode part 32a. In the present embodiment, as shown in FIG. 2, the human body between the first communication unit 24 of the basket device 20 and the communication unit 32 of the product device 30 with the user holding the basket 120 in the hand. Communication is established. Thereby, the article information stored in the storage unit 31 is transmitted to the basket device 20. Note that human body communication is possible not only when the user directly contacts the electrode part 32a of the article device 30, but also when the electrode part 32a and the user are not in direct contact (such as when wearing gloves).

(Server 40)
As shown in FIG. 1, the server 40 includes a storage unit 41, a communication unit 42, an OCR (Optical Character Reader) unit 43, an image recognition unit 44, and a control unit 46.

  The storage unit 41 is, for example, a nonvolatile semiconductor memory such as a flash memory, a hard disk drive, or the like, and stores information acquired from the basket device 20, that is, image data captured by the imaging unit 11 of the mounting device 10 and the microphone 14. The voice data collected by use and the article information transmitted from the article device 30 are stored. In addition, the storage unit 41 stores face image data and product image data of store employees (store clerk) and various data used when analyzing user purchase behavior. In the present embodiment, as will be described later, the control unit 46 cooperates with the image recognition unit 44 to determine whether the image data captured by the imaging unit 11 of the mounting apparatus 10 includes the employee's face or product. Is detected.

  The communication unit 42 performs wireless communication with other devices. In the present embodiment, the communication unit 42 performs wireless communication with the second communication unit 25 of the basket device 20 by wireless communication.

  The OCR unit 43 recognizes characters included in the image data captured by the imaging unit 11 of the mounting apparatus 10 and converts them into text data. For example, when the imaging unit 11 captures the image data of FIG. 7B, the OCR unit 43 recognizes the character and the text data “Sorry. Currently waiting for arrival. The next arrival time is undecided. Is obtained.

  The image recognition unit 44 extracts feature points of the image data captured by the imaging unit 11 of the mounting apparatus 10 by using a feature amount detection algorithm such as SURF (Speeded Up Robust Features) or SHIFT (Scale Invariant Feature Transform), for example. Image recognition is performed by comparing with a plurality of recognition target standard patterns stored in 41. In the present embodiment, the image recognition unit 44 compares the employee's face image data and product image data stored in the storage unit 41, and the employee and the product are included in the image captured by the imaging unit 11. Detect whether it is included.

  The control unit 46 controls the entire server 40 and includes a CPU, a RAM, a ROM, and the like.

  Next, processing of the electronic device system 100 will be described in detail with reference to FIGS. In the present embodiment, when a user (shopper) wearing the mounting apparatus 10 visits a store (for example, a supermarket), the basket 120 is held and shopping is performed. In addition, the user determines an article to be purchased by picking up an article (product) 30 provided with the article device 30 in the store or putting it in the basket 120, and performs purchase processing at a cash register. To do.

(Processing of mounting device 10)
FIG. 4A shows the processing of the mounting apparatus 10 in a flowchart.

  In the process of FIG. 4A, the control unit 17 of the mounting device 10 stands by until human communication with the basket device 20 is established in step S10. In this case, when the user wearing the mounting device 10 enters the store and holds the basket 120 and the user's hand touches the basket device 20, the communication unit 16 of the mounting device 10 and the first communication of the basket device 20. Communication with the unit 24 is established. When human body communication with the basket device 20 is established, the control unit 17 proceeds to step S12. Instead of step S10, the process from step S12 may be executed when the user turns on the power of the mounting apparatus 10 after entering the store. In this case, the mounting device 10 includes a GPS (Global Positioning System) module or a position detection device capable of detecting a position according to a communication state with a wireless LAN (Local Area Network) base station installed in the store, The control unit 17 may determine whether the user has entered the store based on the detection result of the position detection device. On the contrary, the mounting apparatus 10 may always supply power to the communication unit 16 so that the mounting apparatus 10 is activated when human body communication with the basket apparatus 20 is established.

  If transfering it to step S12, the control part 17 will instruct | indicate the imaging part 11, and will perform the imaging from a user side. In this case, the imaging unit 11 may capture either a still image or a moving image. In the present embodiment, the image capturing unit 11 is described as capturing a still image. The control unit 17 stores an image captured by the imaging unit 11 (hereinafter referred to as a line-of-sight image) in the storage unit 15. In step S12, in addition to capturing the line-of-sight image by the imaging unit 11, the control unit 17 uses the microphone 14 to collect voice including the user's voice and surrounding sounds, and stores the voice data in the storage unit 15 as voice data. You may remember.

  Next, in step S <b> 14, the control unit 17 transmits a line-of-sight image to the basket device 20 via the communication unit 16. For example, the control unit 17 includes an accelerometer or an angular velocity meter in the mounting device 10 to measure the moving speed of the user. When the moving speed of the user reaches a predetermined moving speed (for example, 1 km / h or less), the controller 17 A line-of-sight image may be transmitted to the device 20. The user may release his / her hand from the basket device 20 during shopping. Therefore, when the human body communication is possible, the control unit 17 transmits the visual line image to the basket device 20 through the human body communication. When the human body communication is impossible, the control unit 17 transmits the visual line image through the proximity communication or the wireless communication. You may make it transmit to the basket apparatus 20. FIG. In addition, when performing wireless communication, it is necessary for the mounting apparatus 10 to include a wireless communication unit.

  Next, in step S <b> 16, the control unit 17 determines whether or not human body communication with the basket device 20 input from the cashier terminal is established. Here, the input from the cash register terminal is transmitted from the cash register terminal to the car device 20 by communication between the car device 20 and the cash register terminal when the user has the car 120 at the cash register. , Meaning the input of shopping end information. That is, when human body communication is established between the basket device 20 that has received an input from the cash register terminal and the mounting device 10, it means that the user's shopping has ended. Therefore, if the shopping has not ended and the determination in step S16 is negative, the process returns to step S12, and the processing and determination in steps S12, S14, and S16 are repeated. That is, the imaging of the line-of-sight image by the imaging unit 11 and the transmission to the basket device 20 are repeated. On the other hand, when the user's shopping is completed and the determination in step S16 is affirmed, all the processes in FIG.

  In addition, it replaces with step S16 and the control part 17 may judge whether the user (mounting apparatus 10) left the store. In this case, as described above, the mounting device 10 needs to be provided with a position detection device such as a GPS.

(Processing of article device 30)
FIG. 4B shows a process of the article device 30 in a flowchart.

  In the process of FIG. 4B, the control unit 33 of the article device 30 stands by until human body communication with the basket device 20 is newly established in step S20. In this case, as shown in FIG. 2, human body communication with the basket device 20 is newly established at the timing when the user holding the basket 120 picks up the article 130 from the commodity shelf or the like, and the process proceeds to step S22.

  When the process proceeds to step S <b> 22, the control unit 33 issues an instruction to the communication unit 32 and transmits the article information stored in the storage unit 31 to the basket device 20.

  Thereafter, the process returns to step S20, and the control unit 33 repeats the processes and determinations of steps S20 and S22.

  By executing the process of FIG. 4B, the user holding the basket 120 can transmit the information of the article 130 obtained during the shopping to the basket device 20 one by one.

(Processing of the basket device 20)
FIG. 5 shows a process of the basket device 20 in a flowchart.

  In the process of FIG. 5, in step S <b> 30, the control unit 26 of the basket device 20 waits until human body communication with the mounting device 10 is established. When the human body communication with the mounting device 10 is established, the control unit 26 proceeds to step S32. In addition, the timing which transfers to step S32 is the same as the timing which transfers to step S12 of Fig.4 (a) mentioned above.

  When the process proceeds to step S <b> 32, the control unit 26 acquires the line-of-sight image transmitted from the mounting apparatus 10 and stores it in the storage unit 23. For example, an accelerometer or an angular velocity meter is provided in the basket device 20 to measure the moving speed of the user. When the moving speed of the user reaches a predetermined moving speed (for example, 1 km / h or less), the line-of-sight image is displayed from the mounting device 10. May be acquired and stored in the storage unit 23.

  Next, in step S34, the control unit 26 determines whether new article information has been received. If the determination in step S34 is negative, that is, if new article information has not been received, the process proceeds to step S38. On the other hand, if the determination in step S34 is affirmative, that is, if new article information is received, the process proceeds to step S36.

  When the process proceeds to step S <b> 36, the control unit 26 acquires article information and stores it in the storage unit 23. Thereafter, the process proceeds to step S38.

  If transfering it to step S38, the control part 26 will perform the detection using the biosensor 21. FIG. Note that the processing order of steps S32, S34, S36, and S38 may be changed as appropriate, and the processing of each step may be executed concurrently.

  Next, in step S <b> 40, the control unit 26 determines whether or not the user has changed in emotion based on the detection result of the biometric sensor 21. In this case, for example, from the detection result of the biosensor 21, when the heart rate suddenly increases, when the blood pressure rapidly increases, or when the amount of sweating increases rapidly, the user has changed emotions. to decide. Examples of changes in the user's emotions are when the time service starts and you encounter a chance to purchase a product cheaply, the product you were planning to purchase is sold out, or the cash register is crowded. is there. If the determination in step S40 is negative, the process proceeds to step S44. If the determination in step S40 is positive, the process proceeds to step S42.

  In step S42, the control unit 26 issues an instruction to the second communication unit 25, and the line-of-sight image and article information (hereinafter referred to as first information) acquired immediately before the emotion change stored in the storage unit 23. ) To the server 40. Note that “immediately before emotional change” means, for example, several seconds before the emotional change occurs. In some cases, the article information may not be acquired immediately before the emotion change. This is a case where the emotional change is not caused by holding the article 130. In this case, the control unit 26 may transmit the line-of-sight image to the server 40 as the first information.

  Here, as the line-of-sight image acquired immediately before the emotion change, a line-of-sight image as shown in FIGS. 7A, 7B, and 8 is assumed. The line-of-sight image in FIG. 7A is an image indicating that the user has encountered the time service, and specifically, an image indicating that the user has seen the seal of the time service attached to the product. Further, the line-of-sight image in FIG. 7B is an image in the case where the product that the user is trying to purchase is sold out. Specifically, the line-of-sight image is an image showing that the user has seen a notice of waiting for arrival. Further, the line-of-sight image in FIG. 8 is an image showing that the user has seen the cash register being congested.

  The first information is transmitted to the server 40 with the cage ID added. That is, the server 40 manages the first information using the basket ID.

  After step S40 is denied or after the process of step S42 is executed, the control unit 26 determines whether or not there is an input from the cashier terminal in step S44. In this case, the input of the cashier terminal means that, as described above, when the user finishes shopping and brings the basket 120 to the cashier, the cashier terminal 20 communicates with the cashier terminal by communication between the cashier device 20 and the cashier terminal. It means the input of shopping end information transmitted to the basket device 20. If the determination in step S44 is negative, that is, if the user's shopping has not ended, the process returns to step S32, and the processes and determinations in steps S32 to S44 are repeated. On the other hand, if the determination in step S44 is affirmative, the process proceeds to step S46.

  In step S46, the control unit 26 issues an instruction to the second communication unit 25, and all line-of-sight images, article information, and biological information (hereinafter referred to as second information) acquired after step S30 are transmitted to the server 40. Send. That is, the control unit 26, via the second communication unit 25, the line-of-sight image obtained while the user is shopping, the item information of the article obtained by the user, and the user is shopping The sent emotion change information is transmitted to the server 40. Note that the control unit 26 may transmit the voice data acquired using the microphone 14 of the mounting apparatus 10 to the server 40 while the user is shopping. It is assumed that the information acquisition time (obtained from the calendar unit 22) is given to each piece of information transmitted from the control unit 26 as a time stamp. The second information is transmitted to the server 40 with the cage ID added. That is, the server 40 manages the second information using the basket ID. Thereafter, the control unit 26 ends all the processes in FIG.

  As described above, when the control unit 26 executes the process of FIG. 5, the first information is sent from the basket device 20 to the server 40 at the timing when the emotion is changed while the user is shopping. The second information is transmitted from the basket device 20 to the server 40 at the timing when the user finishes shopping.

(Processing of server 40)
FIG. 6 shows the processing of the server 40 in a flowchart.

  In the process of FIG. 6, the control unit 46 of the server 40 stands by until there is an input from the basket device 20 in step S50. That is, the control unit 46 proceeds to step S52 when the first information or the second information is transmitted by executing step S42 or S46 of FIG.

  If transfering it to step S52, the control part 46 will judge whether the input information is 1st information. In other words, the control unit 46 determines whether or not the line-of-sight information or the article information acquired by the basket device 20 is input immediately before the user has changed emotions. When judgment here is affirmed, it transfers to step S54.

  If transfering it to step S54, the control part 46 will perform the emotion analysis based on 1st information. Specifically, the control unit 46 analyzes the line-of-sight image acquired immediately before the emotion changes using the OCR unit 43 and the image recognition unit 44, and the article information acquired immediately before the emotion changes. Based on the analysis result, the user's emotion analysis (analysis of satisfaction or dissatisfaction) is performed.

  For example, when the line-of-sight image is the image shown in FIG. 7A, the control unit 46 can detect the character “20% discount from the morning market time service display price” by using the OCR unit 43. . In this case, since a positive word for the shopper such as time service and 20% discount is included, the control unit 46 analyzes that the user is happy and satisfied (positive state). Note that words that are positive for the shopper are stored in the storage unit 41.

  Further, when the line-of-sight image is the image of FIG. 7B, the control unit 46 uses the OCR unit 43 to “sorry. Currently waiting for arrival. The next arrival time is undecided.” Can be detected. In this case, since there are negative words for the shopper such as sorry, waiting for arrival and undecided, the control unit 46 analyzes that the user is sad and unsatisfied (negative state). It is assumed that words that are negative for the shopper are stored in the storage unit 41.

  In addition, when the line-of-sight image is the image of FIG. 8, the control unit 46 uses the image recognition unit 44 so that the cash register is crowded or the person waiting for the cash register is unhappy. It is possible to detect (not smiling). In this case, the control unit 46 determines that the user is sad and unsatisfied (a negative state).

  In addition, when the line-of-sight image includes an image of the employee's face, the control unit 46 determines that the user's emotion has changed due to the employee. In this case, the control unit 46 uses the image recognition unit 44 to recognize whether or not the employee's face is smiling. Note that the image recognition unit 44 determines whether the face image included in the line-of-sight image is an employee face image or a smile, and stores the employee face image stored in the storage unit 41. Judge using data. Then, when the employee is not smiling, the control unit 46 determines that the user is sad or angry and is in an unsatisfied state (negative state). If the employee is smiling, the control unit 46 is satisfied. Judged to be in a state (positive state).

  Furthermore, when the state where the article is held in the hand is captured in the line-of-sight image, the control unit 46 determines that the emotion has changed due to seeing or touching the article. Note that the control unit 46 may be able to determine whether the user is pleased or sad by using the article information acquired by the basket device 20 together. For example, if the item information includes information such as quantity limitation, it can be determined that the user is delighted. Further, when it is found from the article information that the expiration date has expired, it can be determined that the user is sad.

  Returning to FIG. 6, in the next step S <b> 56, the control unit 46 determines whether or not a response is necessary based on the emotion analysis result. Specifically, the control unit 46 determines that a response is necessary when the user is in an unsatisfactory state (eg, in a negative state) such as sad or angry. If the determination in step S56 is negative, the process returns to step S50. On the other hand, if the determination in step S56 is affirmative, the process proceeds to step S58.

  If transfering it to step S58, the control part 46 will perform the corresponding | compatible process based on an emotion analysis result. For example, when the line-of-sight image is FIG. 4B, the control unit 46 sends various messages such as an apology message and a next arrival schedule stored in the storage unit 41 via the communication unit 42. 20 to send. Then, the control unit 26 of the basket device 20 transmits various messages to the mounting device 10 via the first communication unit 24. In this case, in the mounting apparatus 10, the control unit 17 displays (projects) various messages using the display unit 12.

  For example, when the line-of-sight image is shown in FIG. 8, the control unit 46 transmits an apology message that the cash register is congested to the control unit 17 of the mounting device 10 via the basket device 20. In this case, the control unit 17 displays (projects) an apology message on the display unit 12. The control unit 46 may output an apology message for the cash register congestion using in-store broadcasting. In addition, the control unit 46 may output a message (a message for cashier support) indicating that the employees who make cashiers should be replenished using in-store broadcasting. Alternatively, the control unit 46 may output a cashier support message to a terminal held by the employee. In addition, the control unit 46, for a user who sees a situation where the cash register is congested, the progress of the cash register (processing time per customer) and the skill level of the clerk in charge of the cash register (apprentice or veteran) May be notified.

  Further, for example, when it is determined from the line-of-sight image that there is a problem in the correspondence of the employee, the control unit 46 sends the customer and the employee to the terminal held by another employee or store manager. A message indicating that a problem has occurred is output. If the car apparatus 20 is provided with a position detection device capable of detecting the position according to the communication sensitivity with the wireless LAN base station, the control unit 46 can troubleshoot based on the detection result of the position detection apparatus. You can inform other employees, store managers, etc. of the occurrence location.

  Further, for example, when it is determined from the line-of-sight image that the customer is sad or angry with the article, the control unit 46 uses the in-store broadcast or a terminal held by the employee to May output a message to the effect that Also in this case, the control unit 46 may notify the employee of the customer's position.

  The first information may include voice data collected by the microphone 14 immediately before the emotion change. When the voice data includes a user's screaming or sighing or an employee's apology by analyzing the voice data, the control unit 46 is angry or unsatisfied (negative state). It can be determined that

  It is assumed that the above-described corresponding processing pattern is determined in advance, and the pattern is stored in the storage unit 41.

  After the process of step S58 is completed as described above, the process returns to step S50.

  On the other hand, if the determination in step S52 is negative, that is, if the information input from the basket device 20 is the second information, the process proceeds to step S60.

  If transfering it to step S60, the control part 46 will perform the purchase behavior analysis based on 2nd information. In this case, the control unit 46 analyzes what the user sees during the shopping, what he has picked up, and what he has purchased. In other words, it can be said that it is analyzing which product the user did not see or picked up.

  FIG. 9 shows the order of articles picked up after a user enters the store and passes through the cash register. Moreover, in FIG. 9, the article purchased by the user is underlined. In addition, the control part 46 can utilize the purchase information acquired from the cash register terminal as information of the goods purchased by the user.

  In FIG. 9, the user has picked up Carreru A and Carreau B, but has finally purchased Carrero A. In this case, the control unit 46 refers to the line-of-sight image when picking up Carreru A and the line-of-sight image when picking up Carreru B, and purchases Carrere A as a criterion. Analyze. For example, if price information is included for a long time in a region (predetermined position (substantially center) of the line-of-sight image) that the user is gazing at, it is determined that Carrero A has been purchased based on the price. In addition, when the area that the user is gazing in includes spiciness indications, package pictures, ingredient indications, and expiration dates for a long period of time, Carrero A was purchased based on such information. Judge. It should be noted that the control unit 46 uses the OCR unit 43 to extract whether or not the price or pain display, the component, the expiration date, or the like is included in the region being watched by the user.

  It should be noted that for beef A and B as well, the determination criteria at the time of purchase may be determined. For example, if the area being watched by the user includes a display of content, a display of a part, a price per 100 g, etc. for a long time, the control unit 46 determines the beef based on the information. It is determined that B has been purchased.

  Further, the control unit 46 refers to data acquired by a plurality of users, so that a user who has purchased a product sums up what another product has been compared and has been purchased as a result of the comparative review. You can count what was the deciding factor of the product. It should be noted that the analysis result of the determination criteria at the time of purchase may be provided to the manufacturer or producer of each product. Manufacturers and producers can use the analysis results for future product development and product improvement.

  Moreover, the control part 46 can total the order that the customer picked up the article by referring to the data of FIG. In this case, the control unit 46 can analyze the tendency of the movement in the customer's store by using the counting result. Thereby, the control part 46 can obtain | require suitable arrangement | positioning of goods and a shelf based on a customer's movement.

  Further, the control unit 46 counts the number of times and positions where the user has become positive and the number of times and positions where the user has become negative during the period from entering the store to leaving the store. Can be analyzed for each position (for each sales floor).

  Moreover, the control part 46 is good also as analyzing a visual line image when a customer changes from a negative state to a positive state. As a result of analyzing the line-of-sight image, if the face of the employee is included in the line-of-sight image during the change from the negative state to the positive state, the control unit 46 has solved the trouble by the correspondence of the employee. Can be analyzed. Further, when the corresponding process using the wearing device 10 or the in-store broadcasting is performed during the change from the negative state to the positive state, it can be analyzed that the corresponding process is appropriate.

  As described above in detail, according to the first embodiment, the control unit 46 and the communication unit 42 of the server 40 input line-of-sight images and article information as operation information related to user purchase, and The detection result of the biological sensor 21 is input as emotion information. As a result, the control unit 46 can analyze the purchase based on the line-of-sight image, the article information, and the detection result of the biological sensor 21, and can acquire and provide useful information related to the purchase. Therefore, usability for the user can be improved. In this case, by using the line-of-sight image captured from the user's side, compared to the case of using the image captured from the store shelf side, the situation that the user is actually looking at, i.e., the article ahead of the line of sight Analysis in consideration of display, situation, etc. becomes possible.

  Here, conventionally, when a user wearing a device equipped with the imaging unit 11 such as the wearing device 10 enters the store, the state and products in the store are captured without permission. It could not be said that there was a merit for the side. On the other hand, in the first embodiment, instead of allowing the user to wear the mounting device 10 and enter the store, the store side sends the image data captured by the imaging unit 11 via the basket device 20. I am trying to get it. Thereby, the store can analyze the behavior and emotions until the user purchases the product by analyzing the image data.

  Conventionally, shopping satisfaction and the like have been collected by a questionnaire for the user, but the user needs to access the store website and answer the question or write the answer on paper. It was annoying. On the other hand, in the first embodiment, the image data acquired by the mounting device 10 and the user's biometric information are associated with each other and provided to the store side. It is possible to convey the degree of dissatisfaction to the store side.

  In the first embodiment, a message to the user and a message to the employee are output based on the line-of-sight image, the article information, and the detection result of the biological sensor 21. This makes it possible to immediately respond to a user who feels unsatisfied.

  In the first embodiment, the control unit 46 determines that the user holds the article from the line-of-sight image and the article information. Thus, the control unit 46 can acquire and provide useful information related to purchase by analyzing information that the user has held the article, the order in which the article was held, and the like.

  Further, in the first embodiment, since the control unit 46 uses the detection result of the biosensor 21, it is possible to easily determine whether or not an emotional change has occurred in the user. Further, by analyzing the user's line-of-sight image together with the detection result of the biosensor 21, it is possible to accurately determine whether the user's emotion has become positive or negative.

  In the first embodiment, the mounting device 10 captures a line-of-sight image and transmits the captured line-of-sight image to the basket device 20 that is in contact with the user via human body communication. Thereby, the basket device 20 can acquire only the line-of-sight image of the user holding the basket 120. That is, the line-of-sight image is reliably acquired from the wearing device 10 of the user who has entered the store, and the line-of-sight image is acquired from the mounting device 10 of a person who has not entered the store (for example, a person who has passed near the store). Can be reliably prevented.

  In the first embodiment, the first information is transmitted to the server 40 when the detection result of the biosensor 21 changes, that is, when the user's emotion changes (S42). ). Thereby, the information which needs to be analyzed can be transmitted to the server 40 at an appropriate timing.

  In addition, although the case where the cage apparatus 20 includes the biosensor 21 has been described in the first embodiment, the present invention is not limited thereto, and the mounting apparatus 10 may include a biosensor. In this case, a biological sensor that contacts a part of the face of the mounting apparatus 10 or a part of the face may be provided in the vicinity of the frame. Further, a biosensor that can be mounted on the earlobe may be connected to the mounting apparatus 10.

  In the first embodiment, the case where the article device 30 is provided on the article 130 has been described. However, the present invention is not limited to this, and the article device 30 may not necessarily be provided in the article 130 when the article information is unnecessary in the emotion analysis in step S54 and the purchase behavior analysis in step S60. For example, in a line-of-sight image, when an image of a state where an article is held in a hand is captured, the article held in the hand can be identified by performing image recognition.

  In the first embodiment, the case where the article device 30 performs human body communication with the basket device 20 has been described. However, the present invention is not limited thereto, and the article device 30 communicates with the mounting device 10 through human body communication. It is good also as performing. In this case, the article information acquired by the mounting apparatus 10 may be transmitted from the mounting apparatus 10 to the basket apparatus 20 in a state in which human body communication is established between the mounting apparatus 10 and the basket apparatus 20.

  In the first embodiment, the case where the detection result of the biosensor 21 is used has been described. However, the present invention is not limited to this, and the detection result of the biosensor 21 may not be used. In this case, for example, when the control unit 46 determines that the user is staring at the mark in FIG. 7A for a predetermined time based on the line-of-sight image, the control unit 46 estimates that the user is in a positive state. It is good to do. Further, for example, when the control unit 46 determines that the user is staring at the sticker in FIG. 7B or the registration waiting state in FIG. It is good also as estimating that it is in a state. In this case, the control unit 46 stores in advance image patterns of marks, stickers, and situations that are predicted to be positive or negative when viewed by the user, and compares the image pattern with the line-of-sight image. And it is sufficient.

  In the first embodiment, the case where the predetermined position of the line-of-sight image is regarded as the center position of the line-of-sight has been described, but the present invention is not limited to this. For example, a line of sight that has an infrared irradiation unit and an infrared light receiving unit, irradiates the eyeball with infrared rays from the infrared irradiation unit, and detects the direction of the line of sight by receiving infrared rays reflected by the eyeball at the infrared light receiving unit. It is good also as providing a detection part in the mounting apparatus 10 separately.

  Note that the server 40 may aggregate customer satisfaction and dissatisfaction levels at a plurality of stores and publish them as user evaluations for each store on one Web site. In this case, for example, the evaluation of each store can be announced by the number of stars or the score of satisfaction.

<< Second Embodiment >>
Next, an electronic device system 100 ′ according to the second embodiment will be described in detail based on FIGS.

  FIG. 10 is a block diagram showing the configuration of the electronic device system 100 ′. In FIG. 10, a configuration different from the electronic device system 100 of the first embodiment in the electronic device system 100 ′ is indicated by a thick line frame.

  As shown in FIG. 10, in the electronic device system 100 ′, the mounting device 10 ′ includes the speaker 18, the basket device 20 ′ includes the weight sensor 27, and the article device 30 ′ includes the light emitting unit 34. The configuration of the server 40 'is the same as that of the server 40 of the first embodiment.

  The speaker 18 is used to provide audio information to a user wearing the mounting apparatus 10 '. As the speaker 18, in addition to the earphone and the headphone, a speaker having directivity and capable of providing audio information only to a user wearing the mounting apparatus 10 'can be employed. It is assumed that the storage unit 15 of the mounting device 10 'stores article information corresponding to a light emission pattern of a light emitting unit 34 of an article device 30' described later.

  The weight sensor 27 is a sensor that is provided in a part of the basket device 20 ′ and detects a change in the internal capacity of the basket 120.

  The light emitting unit 34 includes a light emitting element such as an LED (Light Emitting Diode), and emits light with a light emission pattern predetermined for each article. Note that the light emission of the light emitting unit 34 is controlled by the control unit 33. In the moving image shooting by the imaging unit 11, the control unit 17 of the mounting apparatus 10 ′ estimates that the user has seen an article corresponding to the light emission pattern when the light emission pattern is captured within a predetermined range of the angle of view. Note that the on / off switching timing of the light emission by the light emitting unit 34 (light emission pattern blinking interval) is set in accordance with the frame rate of the imaging unit 11. Note that a visible light LED or an infrared LED may be used as the light emitting unit 34, and when an infrared LED is used, an infrared imaging element may be used for the imaging unit 11.

  Next, processing of the electronic device system 100 ′ according to the second embodiment will be described in detail along the flowcharts of FIGS. 11 to 13 with reference to other drawings as appropriate. Also in the second embodiment, a case where the electronic device system 100 ′ is used in a supermarket will be described.

(Processing of mounting device 10 ')
FIG. 11A shows the processing of the mounting apparatus 10 ′ in a flowchart. In FIG. 11 (a), steps different from the processing of the mounting apparatus 10 of the first embodiment (FIG. 4 (a)) are indicated by bold lines.

  In the process of FIG. 11A, the control unit 17 of the mounting device 10 'stands by in step S10 until human body communication with the basket device 20' is established. When the human body communication with the basket device 20 'is established, the control unit 17 proceeds to step S110.

  In step S110, the control unit 17 instructs the imaging unit 11 to execute imaging (moving image) from the user side.

  Next, in step S112, the control unit 17 determines whether or not the light emission of the light emitting unit 34 is imaged within a predetermined range of the angle of view. That is, it is determined whether or not the user has viewed an article provided with the article device 30 '. If the determination in step S112 is negative, the process proceeds to step S116. If the determination in step S112 is positive, the process proceeds to step S114.

  In step S114, the control unit 17 transmits the article information viewed by the user to the basket device 20 '. Specifically, the control unit 17 acquires article information corresponding to the light emission pattern detected within a predetermined range of the angle of view of the imaging unit 11 from the storage unit 15, and transmits the article information of the basket device 20 ′ via the communication unit 16. Transmit to the first communication unit 24.

  After the determination of step S112 is denied or after the processing of step S114 is performed, when the process proceeds to step S116, the control unit 17 determines whether information is received from the basket device 20 '. Note that the information received from the basket device 20 'is information related to goods and store services transmitted from the server 40'. If the determination is negative, the process proceeds to step S16. If the determination is positive, the process proceeds to step S118.

  In step S118, the control unit 17 outputs the information received from the basket device 20 '. Details of the processing in step S118 will be described later.

  After the determination in step S116 is denied, or after the processing in step S118 is performed, when the process proceeds to step S16, the control unit 17 establishes human body communication with the basket device 20 ′ input from the cashier terminal. Determine whether or not. Here, the input from the cashier terminal means that when the user finishes shopping and has the basket 120 at the cashier, the cashier apparatus 20 from the cashier terminal 20 communicates with the cashier apparatus 20 ′ and the cashier terminal. It means the input of shopping end information sent to '. That is, when human body communication is established between the basket device 20 ′ and the mounting device 10 ′ input from the cash register terminal, it means that the user's shopping is finished. Therefore, if shopping is not completed and the determination in step S16 is negative, the process returns to step S112, and the processing and determination in steps S112 to S16 are repeated. On the other hand, when the user's shopping is finished and the determination in step S16 is affirmed, all the processes in FIG. 11A are finished.

(Processing of article device 30 ')
FIG. 11B shows a process of the article device 30 ′ in a flowchart. In FIG. 11B, steps different from the processing of the article device 30 of the first embodiment (FIG. 4B) are indicated by bold lines.

  As shown in FIG. 11B, in the article device 30 ′, the control unit 33 emits light using the light emitting unit 34 in step S <b> 120. In this case, the control unit 33 causes the light emitting unit 34 to emit light with a light emission pattern corresponding to the article on which the article device 30 ′ is provided. Other processes are the same as those in FIG.

(Processing of basket device 20 ')
FIG. 12 is a flowchart showing the process of the basket device 20 ′. In the process of FIG. 12, in step S130, the control unit 26 waits until human body communication with the mounting apparatus 10 ′ is established. If human body communication with mounting apparatus 10 'is materialized and it transfers to step S132, control part 26 will judge whether article information was received from mounting apparatus 10'.

  When the determination in step S132 is affirmed, the process proceeds to step S134, and the control unit 26 transmits the article information received from the mounting apparatus 10 ′ to the server 40 ′ as the article information of the viewed article, and step S136. Migrate to On the other hand, if the determination in step S132 is negative, the process proceeds to step S136 without passing through step S134.

  In step S136, the control unit 26 determines whether or not article information has been received from the article device 30 '. When the determination in step S136 is affirmed, the process proceeds to step S138, and the control unit 26 transmits the article information received from the article device 30 ′ to the server 40 ′ as the article information of the article touched, and step S140. Migrate to On the other hand, if the determination in step S136 is negative, the process proceeds to step S140 without passing through step S138.

  If transfering it to step S140, the control part 26 will judge whether the detected value of the weight sensor 27 changed. Here, the case where the detection value of the weight sensor 27 changes means that an article has been newly placed in the basket 120. Note that the detection value of the weight sensor 27 also changes when the article in the basket 120 is taken out, but here, for convenience of explanation, it is assumed that the article is not taken out from the basket 120. If the determination in step S140 is affirmative, the process proceeds to step S142, and the control unit 26 uses the article information of the article that was touched immediately before, as the article information of the article that has been put in the basket, that is, the item that the user plans to purchase Information to the server 40 ′. On the other hand, if the determination in step S140 is negative, the process proceeds to step S144 without passing through step S142.

  In step S144, the control unit 26 determines whether information is received from the server 40 '. The information received from the server 40 ′ is information related to goods and store services. If the determination in step S144 is affirmative, the process proceeds to step S146, and the control unit 26 transmits the information received from the server 40 'to the mounting apparatus 10', and the process proceeds to step S148. On the other hand, if the determination in step S144 is negative, the process proceeds to step S148 without passing through step S146.

  In step S148, the control unit 26 determines whether there is an input from the cashier terminal. Here, the input from the cash register terminal means the input of the shopping end information as described above.

  If the determination in step S148 is negative, the process proceeds to step S132, and the processing and determination in steps S132 to S148 are repeated. If the determination in step S148 is affirmative, the control unit 26 End all processing.

(Processing of server 40 ')
FIG. 13 is a flowchart showing the process of the server 40 ′.

  In the process of FIG. 13, in step S150, the control unit 46 stands by until article information is received from the basket device 20 '. That is, the control unit 46 proceeds to step S152 when any of the article information of the article viewed by the user, the article information of the article touched by the user, and the article information of the article placed in the basket 120 is input.

  If transfering it to step S152, the control part 46 will update shopping DB stored in the memory | storage part 41. FIG. Here, the shopping DB has a data structure as shown in FIG. As shown to Fig.14 (a), it has each column of an article name, seen, touched the hand, and put in the basket. When the article information of the article viewed by the user is received, the control unit 46 stores ○ in the column of the article viewed, and when the article information of the article touched by the user is received, the control section 46 touched the article. ○ is stored in the column, and when the user receives the article information of the article placed in the basket, ○ is stored in the column of the article placed in the basket.

  Next, in step S154, the control unit 46 analyzes the shopping DB. For example, the control unit 46 refers to the shopping DB in FIG. 14A, and when the predetermined time has elapsed since the beef A was held in the hand (when new article information is not received for the predetermined time). Can analyze that there is a possibility that the user needs information on an article (for example, beef B) that can be compared with beef A. In addition, since the control unit 46 can estimate that the user makes curry from, for example, an article placed in the basket, the amount of beef required for making the curry (Xg for 4 dishes, Yg for 8 dishes, etc.) ) Information may be analyzed by the user. In addition, the control unit 46 may require the user to have information on curry garnish (Fukujinzuke, roasted cutlet, dessert, etc.) and bargain information on the store (if you buy more Z yen, you can get a lottery). Can be analyzed.

  Next, in step S156, the control unit 46 determines whether there is information to notify the user as a result of the analysis in step S154. If the determination is negative, the process returns to step S150. If the determination is positive, the process proceeds to step S158. In addition, when the user touches the same article several times, or when the user touches an article similar to the article touched before, the same information as the information notified in the past is the information notified to the user. Sometimes it is judged. Therefore, the control unit 46 may limit the notification, assuming that the same information as the information notified in the past is not information to be notified to the user. That is, the control unit 46 restricts information to be notified when the same article is touched or viewed multiple times, or when an article similar to the article touched or seen before is touched or viewed. It is good to do. In this way, it is possible to prevent the same information from being repeatedly notified to the same user.

  Note that the control unit 46 may change the information to be notified in accordance with the number of times the same article is touched or the number of times the same article is viewed. For example, the control unit 46 should notify the information of the article itself for the first time, the information of the article as a comparison reference of the article for the second time, and the profit information related to the article for the third time. It is good as well. By doing in this way, it becomes possible to improve the possibility that a customer who is confused about purchasing an article will purchase the article. Instead of this, the user may be notified from information not notified to the user for the second and subsequent times according to the time when the user touches the same article.

  When the determination in step S156 is affirmed, the control unit 46 proceeds to step S158, and transmits information notified to the user to the basket device 20 '. When the process of step S158 is performed, in the process of FIG. 12, the determination of step S144 is affirmed, and the process of step S146 (process for transmitting information received from the server to the mounting apparatus 10 ′) is executed. . In addition, when the process in step S146 is executed in FIG. 12, the determination in step S116 in FIG. 11A is affirmed, and the process in step S118 (process for outputting received information) is executed. In step S <b> 118, the control unit 17 displays information notified to the user using the display unit 12, or outputs information notified to the user using the speaker 18. In addition, after step S158, it returns to step S150 and repeats the process of step S150-S158.

  In addition, the control part 46 is good also as performing various analysis based on shopping DB (for example, FIG.14 (b)) of the user who complete | finished shopping. In this case, for example, based on FIG. 14 (b), the control unit 46 compares Carreru A and Carrero B, and beef A and beef B, and analyzes that Carrero B and beef B have been purchased. it can. The control unit 46 can also analyze the order of walking in the store, the order of purchase, and the like. In addition, the control unit 46 can identify and analyze an article that was not purchased just by looking at the product, or an article that was picked up but not purchased. The control part 46 is good also as providing an analysis result to the manufacturer, sales agency, etc. of goods.

  As described above in detail, according to the second embodiment, the control unit 46 determines that the user has picked up the article and / or that the article has entered a predetermined range within the user's field of view. In response, the article information of the article is input (S150: affirmative), and notification regarding the input article information is made (S158). Therefore, in the second embodiment, useful information related to an article touched by the user or an article viewed by the user can be notified to the user. Thereby, a user's usability can be improved.

  Further, in the second embodiment, the control unit 46 inputs information on a product scheduled to be purchased by the user put in the basket 120 as the item information, and performs notification according to the input product. Thereby, the user can be notified of useful information related to the product to be purchased. Moreover, since the control part 46 inputs the information of the goods scheduled to purchase from the basket apparatus 20 'used at the time of shopping, it can input accurate information as the information of the goods scheduled to purchase.

  In the second embodiment, since the user is notified of information from the display unit 12 of the mounting apparatus 10 ′ or the directional speaker 18, the user's information is provided for each user without being known to other users. Notification according to purchase behavior can be performed.

  In the second embodiment, the control unit 46 may determine that the two products are being compared and examined when the user is looking at another item while holding the item.

  In the second embodiment, the example in which the electronic device system 100 ′ is used in the supermarket has been described. However, the present invention is not limited thereto, and the electronic device system 100 ′ may be used in a clothing store or the like. For example, when the user picks up clothes, the control unit 46 purchases one more piece of information about whether there is a stock, such as a different color or a different size, whether there is a similar product on sale, and so on. By discounting information such as discounts when using a display unit 12 or the speaker 18 when buying more than 2000 yen, a specific user (a member of a store, etc.) Can only be notified. Further, the server 40 ′ may transmit discount information, present information, and a congratulatory message to the mounting apparatus 10 ′ when a store user visits on the birthday.

  In the second embodiment, the control unit 17 has described the case where the article information acquired in the mounting device 10 ′ is transmitted to the server 40 ′ via the basket device 20 ′. However, the present invention is not limited to this. It is not a thing. For example, when the mounting apparatus 10 ′ has a communication unit capable of wireless communication, information may be transmitted from the mounting apparatus 10 ′ to the server 40 ′ by wireless communication.

  In the second embodiment, the article device 30 ′ has the light emitting unit 34. However, the present invention is not limited thereto, and the article device 30 ′ may not have the light emitting unit 34. In this case, the control part 17 should just identify the article | item which the user saw by analyzing the image which the imaging part 11 imaged. For example, the control unit 17 may identify characters (for example, a product tag or a POP advertisement character) included in the display and specify an article viewed by the user.

  In the first and second embodiments, the control unit 46 may consider the type of user when analyzing information. If the user type is a store member, it can be grasped from registered information (name, date of birth, address, shopping history). In this case, it is good also as analyzing a user's purchasing behavior for every sex and age of a user.

  Each embodiment mentioned above is an example of suitable implementation of the present invention, and can be combined suitably. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Wearing device 11 Imaging part 12 Display part 16 Communication part 17 Control part 18 Speaker 20 Basket apparatus 42 Communication part 46 Control part 110 Frame 120 Basket

Claims (15)

A first input unit for inputting operation information related to the user's purchase;
An electronic apparatus comprising: a second input unit that inputs emotion information related to the user's purchase.   The electronic apparatus according to claim 1, further comprising: a detection unit that detects satisfaction or dissatisfaction of the user based on information input by the first input unit and the second input unit.   The electronic apparatus according to claim 1, further comprising: an instruction unit that instructs a response to the user based on information input by the first input unit and the second input unit.   4. The electronic device according to claim 1, wherein the first input unit inputs information related to a movement of the user's hand. 5.   The electronic device according to claim 1, wherein the second input unit inputs information related to the user's living body.   6. The electronic apparatus according to claim 1, wherein at least one of the first input unit and the second input unit inputs information about an eye from the user side.   The electronic apparatus according to claim 6, further comprising a communication unit capable of communicating with an acquisition device that acquires information about eyes from the user side.   The electronic device according to claim 7, wherein the communication unit communicates with a communication unit provided in the acquisition device.   The electronic device according to claim 7, wherein the communication unit communicates with the acquisition device via a communication relay device different from the acquisition device.   The electronic device according to claim 9, wherein the communication unit communicates with the acquisition device via proximity communication or human body communication with the communication relay device. The acquisition device includes an imaging unit that performs imaging,
The electronic apparatus according to claim 7, further comprising an analysis unit that analyzes image data captured by the imaging unit.   The electronic device according to any one of claims 1 to 11, wherein the first input unit inputs the motion information in accordance with a change in the emotion information input by the second input unit. An input unit for inputting information about the eyes from the user side;
An electronic device comprising: an estimation unit configured to estimate emotion information related to purchase of the user based on information related to eyes from the user side.   The said estimation part estimates the emotion information regarding the said user's purchase based on whether the information regarding the eye from the said user corresponds or resembles a predetermined pattern, It is characterized by the above-mentioned. The electronic device described. An electronic device that can be worn near the user's face,
An acquisition unit for acquiring information about the eyes from the user side;
An electronic apparatus comprising: a communication unit that communicates information acquired by the acquisition unit with an external device that is in contact with the user via human body communication.

Images