JP2022097524A - Electronic apparatus system and transmission method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a user-friendly electronic apparatus system.

SOLUTION: An electronic apparatus system includes: an entrance detection unit that is arranged in a store and detects user's entrance to the store; a recognition unit that recognizes an article held by the user on the basis of image information captured in the store; an acquisition unit that acquires information on the article recognized by the recognition unit; and a transmission unit that transmits the information on the article acquired by the acquisition unit to a device in the store.

SELECTED DRAWING: Figure 10

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to an electronic device system and a transmission method.

Conventionally, a sales promotion system has been proposed in which behavior information regarding a visitor's purchase of a product is acquired and the information is linked to sales promotion (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2010-277256

However, it cannot be said that the conventional sales promotion system can sufficiently acquire the necessary information, and it cannot be said that it is easy to use.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an easy-to-use electronic device system.

The electronic device of the present invention is an input unit (46, 42) for inputting information on the article according to at least one of the fact that the article is picked up by the user and the article is within a predetermined range in the user's field of view. ), And a notification unit (46) for notifying information about the article input by the input unit.

In this case, the first control that limits the information notified by the notification unit when the user picks up the article a plurality of times or when the article enters a predetermined range within the user's field of view a plurality of times. The unit (46) may be provided. In this case, the first control unit makes the information notified by the notification unit different depending on the number of times the article is picked up or the number of times the article is within a predetermined range in the user's field of view. May be. In addition, when the user picks up a similar article similar to the article, or when the similar article enters a predetermined range in the user's field of view, the second control limits the information notified by the notification unit. The unit (46) may be provided.

In the electronic device of the present invention, the input unit inputs the information of the product to be purchased by the user as the information of the article, and the notification unit notifies the information according to the product to be purchased. May be good. In this case, the input unit may input the information of the product to be purchased from the external device (20'). Further, the input unit may input information according to the product to be purchased from the external device via the storage device (120) for storing the product to be purchased.

Further, in the electronic device of the present invention, the notification unit may include at least one of a display unit (12) for displaying and an audio output unit (18) for outputting audio. In this case, the audio output unit may be a directional audio output unit. Further, the electronic device of the present invention may be provided with a mounting portion (110) that can be mounted near the user's face.

In addition, in order to explain the present invention in an easy-to-understand manner, the present invention has been described with reference to the reference numerals of the drawings representing one embodiment, but the present invention is not limited to this, and the configuration of the embodiments described later is not limited thereto. May be appropriately improved, or at least a part thereof may be replaced with another component. Further, the configuration requirements without particular limitation on the arrangement are not limited to the arrangement disclosed in the embodiment, and can be arranged at a position where the function can be achieved.

The present invention has the effect of being able to provide an easy-to-use electronic device system.

It is a block diagram which shows the structure of the electronic device system which concerns on 1st Embodiment. It is a figure which shows the usage situation of a mounting device, a basket device, and an article device. It is a perspective view of the mounting device. FIG. 4A is a flowchart showing the processing of the mounting device according to the first embodiment, and FIG. 4B is a flowchart showing the processing of the article device according to the first embodiment. It is a flowchart which shows the processing of the basket device which concerns on 1st Embodiment. It is a flowchart which shows the processing of the server which concerns on 1st Embodiment. 7 (a) and 7 (b) are views (No. 1) showing an example of a line-of-sight image. It is a figure (the 2) which shows the example of the line-of-sight image. It is a figure which shows the order of the goods which a certain user picked up from entering a store to passing through a cash register. It is a block diagram which shows the structure of the electronic device system which concerns on 2nd Embodiment. FIG. 11A is a flowchart showing the processing of the mounting device according to the second embodiment, and FIG. 11B is a flowchart showing the processing of the article device according to the second embodiment. It is a flowchart which shows the processing of the basket device which concerns on 2nd Embodiment. It is a flowchart which shows the processing of the server which concerns on 2nd Embodiment. 14 (a) and 14 (b) are diagrams showing an example of a shopping DB.

<< First Embodiment >>
Hereinafter, the electronic device system according to the first embodiment will be described in detail with reference to FIGS. 1 to 9. FIG. 1 shows a block diagram of the configuration of the electronic device system 100 according to the first embodiment.

As shown in FIG. 1, the electronic device system 100 includes a mounting device 10, a basket device 20, an article device 30, and a server 40. As shown in FIG. 2, the wearing device 10 is a glasses-type terminal worn on the face by a user (shopper) who visits the store. The basket device 20 is a device provided in the basket 120 installed in the store. The article device 30 is a device provided in the article (commodity) 130 displayed in the store. The server 40 is an information processing device provided in a store, a data center, or the like. Communication via the human body (human body communication) is possible between the mounting device 10 and the basket device 20, and between the article device 30 and the basket device 20, and between the basket device 20 and the server 40. , Wireless communication is possible.

(Mounting device 10)
As shown in FIG. 1, the mounting device 10 includes an image pickup unit 11, a display unit 12, an operation unit 13, a microphone 14, a storage unit 15, a communication unit 16, and a control unit 17. Note that FIG. 3 shows the mounting device 10 in a perspective view. As shown in FIG. 3, the wearing device 10 includes a spectacle-shaped frame 110. It should be noted that the configuration of the mounting device 10 shown in FIG. 1 and not shown in FIG. 3 is assumed to be provided inside the frame 110 or a part of the frame 110.

The image pickup unit 11 includes a lens, an image pickup element, an image processing unit, and the like, and captures a still image or a moving image. As shown in FIG. 3, the image pickup unit 11 is provided in the vicinity of the end portion of the frame 110 (near the user's right eye). Therefore, in the state where the user wears the wearing device 10 (the state shown in FIG. 2), it is possible to capture an image in the direction in which the user is facing (viewing). Further, a predetermined portion of the captured image (for example, a portion slightly shifted to the left from the center) can be regarded as a portion to which the user is looking (a portion to be gazed at).

The display unit 12 has a projector provided inside the frame 110 or in the vicinity of the frame 110, and a prism for guiding the projected image from the projector to the user's eyes. The display unit 12 displays various information under the instruction of the control unit 17.

The operation unit 13 is a touch pad provided on the frame 110, detects the movement of the user's finger, accepts an operation from the user, and transmits the accepted operation information to the control unit 17.

The details of the image pickup unit 11, the display unit 12, and the operation unit 13 are also disclosed in, for example, US Patent No. 2013/0044042.

The microphone 14 is provided in the frame 110 and collects the voice emitted by the user and the voice around the user. The voice collected by the microphone 14 is recognized by a voice recognition unit (not shown), and the voice recognition result is transmitted to the control unit 17. The control unit 17 executes a process based on the voice recognition result (for example, a command execution process). The control unit 17 may execute voice recognition. Further, the control unit 17 stores the collected voice data in the storage unit 15.

The storage unit 15 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores image data captured by the image pickup unit 11, audio data collected by the microphone 14, display data displayed on the display unit 12, and the like. ..

The communication unit 16 performs human body communication with other devices. The communication unit 16 includes an electrode unit 16a provided on the frame 110 and capable of contacting the user, and a human body communication unit 16b for performing human body communication using the electrode unit 16a. In human body communication, there are a current method in which a weak current is passed through the human body and the current is modulated to transmit information, and an electric field method in which an electric field induced on the surface of the human body is modulated to transmit information. .. In this embodiment, either a current method or an electric field method can be used. It should be noted that human body communication is possible not only when the user is in direct contact with the electrode portion 16a of the mounting device 10, but also when the electrode portion 16a and the user are not in direct contact with each other. In the present embodiment, the communication unit 16 transmits the image data captured by the image pickup unit 11 and the voice data collected by the microphone 14 to the basket device 20. The communication unit 16 may perform proximity communication with other devices such as Bluetooth (registered trademark), RFID (Radio Frequency Identification), and TransferJet (registered trademark).

The wireless communication unit may be provided in the mounting device 10 in place of the communication unit 16 or in combination with the communication unit 16. In this case, the image data or the voice data may be transmitted directly to the server 40 by wireless communication, or the image data or the voice data may be transmitted to the basket device 20 by wireless communication.

The control unit 17 comprehensively controls the entire mounting device 10. The control unit 17 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. In the present embodiment, the control unit 17 controls the image pickup of the image pickup unit 11 and the display of the display unit 12.

The wearing device 10 does not have to be a spectacle-type terminal. For example, it may be a wearable terminal that can be worn at any position on the body, or a part of its functions may be separated. As an example, as a contact lens type terminal provided with a display unit, a wristwatch type terminal may have other functions.

(Basket device 20)
As shown in FIG. 1, the basket device 20 includes a biological sensor 21, a calendar unit 22, a storage unit 23, a first communication unit 24, a second communication unit 25, a control unit 26, and the like.

The biosensor 21 is provided in a portion of the basket 120 that the user touches when holding the basket 120 (handle portion). The biological sensor 21 detects changes in biological information of the body such as heart rate, blood pressure, and sweating amount, and includes, for example, an LED (light emitting diode) and a photo sensor, and a sweating sensor. For example, by using an LED and a photo sensor, the heart rate can be detected by receiving the light reflected by the photo sensor when the light emitted by the LED is applied to a finger. In addition, the sweating sensor can detect the amount of sweating by measuring the impedance of the hand with a plurality of electrodes. In the present embodiment, the output from the biosensor 21 is sent to the control unit 26. Then, the control unit 26 estimates changes in the user's emotions (emotions and sorrows) based on the output of the biological sensor 21.

The calendar unit 22 acquires time information such as year, month, day, and time, and outputs it to the control unit 26. In addition, the calendar unit 22 measures the time.

The storage unit 23 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores information about an article obtained by the user as described later. Further, the storage unit 23 stores an ID number (basket ID) for uniquely identifying the basket device 20.

The first communication unit 24 is for performing human body communication with other devices. The first communication unit 24 may perform proximity communication with other devices. In the present embodiment, the first communication unit 24 communicates with the communication unit 16 of the mounting device 10 and the communication unit 32 of the article device 30 by using human body communication.

The first communication unit 24 has an electrode unit 24a provided on the handle portion of the basket 120 and capable of contacting the user's hand or arm, and a human body communication unit 24b that performs human body communication via the electrode unit 24a. ..

The second communication unit 25 performs wireless communication with other devices. In the present embodiment, wireless communication is performed with the server 40, and the article information of the article touched by the user, the image data captured by the imaging unit 11, the voice data collected by the microphone 14, and the detection of the biosensor 21 are performed. The result is transmitted to the server 40. The basket device 20 of the present embodiment is a communication relay device that relays the communication of article information between the article device 30 and the server 40, and the communication of image data and voice data between the mounting device 10 and the server 40. It has the function as.

The control unit 26 controls the entire basket device 20, and includes a CPU, RAM, ROM, and the like. In the present embodiment, the control unit 26 controls information communication between the article device 30, the mounting device 10, and the server 40.

The basket device 20 may be provided in a cart used in a store. In this case, it is preferable that the biosensor 21 and the electrode portion 24a of the first communication unit 24 are provided on the grip portion of the cart.

(Article device 30)
The article device 30 is an article (commodity) sold at a store or the like, and is provided for an article that comes into contact with the body of a visitor (user) such as groceries, clothes, shoes, watches, accessories, golf clubs, bags, and the like. .. In the present embodiment, for convenience of explanation, the article device 30 will be described as being provided in a product (food product or the like) displayed in a supermarket.

As shown in FIG. 1, the article device 30 includes a storage unit 31, a communication unit 32, and a control unit 33.

The storage unit 31 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores information about an article (commodity) to which the article device 30 is attached. Specifically, the storage unit 31 stores the article information (product information) of the article to which the article device 30 is attached. For example, when the article device 30 is attached to a processed food (for example, canned food), the storage unit 31 uses the manufacturer (seller), the product name, the raw material name, the expiration date, the content, and the origin as the article information. Remember the country etc.

The control unit 33 controls the entire article device 30, and includes a CPU, RAM, ROM, and the like. In the present embodiment, the control unit 33 controls the transmission of the article information stored in the storage unit 31 to the basket device 20. The control unit 33 and the storage unit 31 may be provided at any position of the article.

The communication unit 32 performs human body communication with other devices. The communication unit 32 may perform proximity communication with other devices. In the present embodiment, the communication unit 32 includes an electrode unit 32a provided at a position in contact with a user (for example, a hand), and a human body communication unit 32b that performs human body communication using the electrode unit 32a. In the present embodiment, as shown in FIG. 2, a human body between the first communication unit 24 of the basket device 20 and the communication unit 32 of the article device 30 in a state where the user holding the basket 120 holds the article 130. Communication is established. As a result, the article information stored in the storage unit 31 is transmitted to the basket device 20. It should be noted that human body communication is possible not only when the user is in direct contact with the electrode portion 32a of the article device 30, but also when the electrode portion 32a and the user are not in direct contact (such as when wearing gloves).

(Server 40)
As shown in FIG. 1, the server 40 includes a storage unit 41, a communication unit 42, an OCR (Optical Character Reader) unit 43, an image recognition unit 44, and a control unit 46.

The storage unit 41 is, for example, a non-volatile semiconductor memory such as a flash memory, a hard disk drive, or the like, and stores information acquired from the basket device 20, that is, image data captured by the image pickup unit 11 of the mounting device 10 and a microphone 14. The voice data collected by using and the article information transmitted from the article device 30 are stored. Further, the storage unit 41 stores face image data and product image data of a store employee (clerk), and various data used when analyzing the purchase behavior of the user. In the present embodiment, as will be described later, the control unit 46 cooperates with the image recognition unit 44 to determine whether the image data captured by the image pickup unit 11 of the mounting device 10 includes an employee's face or a product. Is detected.

The communication unit 42 performs wireless communication with other devices. In the present embodiment, the communication unit 42 wirelessly communicates with the second communication unit 25 of the basket device 20 by wireless communication.

The OCR unit 43 recognizes characters included in the image data captured by the image pickup unit 11 of the mounting device 10 and converts them into text data. For example, when the image pickup unit 11 captures the image data of FIG. 7 (b), the OCR unit 43 recognizes characters and text data "Sorry. Currently waiting for arrival. The next arrival time is undecided. To get.

The image recognition unit 44 extracts the feature points of the image data captured by the image pickup unit 11 of the mounting device 10 by a feature amount detection algorithm such as SURF (Speeded Up Robust Features) or SHIFT (Scale Invariant Feature Transform), and stores the feature points. Image recognition is performed by comparing with a plurality of standard patterns of recognition targets stored in 41. In the present embodiment, the image recognition unit 44 compares with the employee's face image data and product image data stored in the storage unit 41, and the employee or product is included in the image captured by the image pickup unit 11. Detect if it is included.

The control unit 46 controls the entire server 40, and includes a CPU, RAM, ROM, and the like.

Next, the processing of the electronic device system 100 will be described in detail with reference to FIGS. 4 to 9. In the present embodiment, when a user (shopper) wearing the mounting device 10 visits a store (for example, a supermarket), the basket 120 is held and shopping is performed. In addition, the user decides the goods to be purchased by picking up the goods (goods) 130 provided with the goods device 30 in the store or putting them in the basket 120, and performs the purchase process at the cash register. do.

(Processing of mounting device 10)
FIG. 4A shows a flowchart of the processing of the mounting device 10.

In the process of FIG. 4A, the control unit 17 of the mounting device 10 waits in step S10 until human body communication with the basket device 20 is established. In this case, when the user wearing the mounting device 10 enters the store and the user touches the basket device 20 by holding the basket 120, the communication unit 16 of the mounting device 10 and the first communication of the basket device 20 are performed. Communication with the unit 24 is established. When the human body communication with the basket device 20 is established, the control unit 17 proceeds to step S12. Instead of step S10, the processes after step S12 may be executed when the power of the mounting device 10 is turned on after the user enters the store. In this case, the mounting device 10 is provided with a GPS (Global Positioning System) module and a position detection device capable of position detection according to the communication status with a wireless LAN (Local Area Network) base station installed in the store. The control unit 17 may determine whether or not the user has entered the store based on the detection result of the position detection device. On the contrary, the mounting device 10 may always supply power to the communication unit 16 so that the mounting device 10 is activated by the establishment of human body communication with the basket device 20.

When the process proceeds to step S12, the control unit 17 issues an instruction to the image pickup unit 11 and executes imaging from the user side. In this case, the image pickup unit 11 may capture either a still image or a moving image. In this embodiment, the image pickup unit 11 will be described as taking a still image. The control unit 17 stores an image captured by the image pickup unit 11 (hereinafter referred to as a line-of-sight image) in the storage unit 15. In step S12, in addition to the image pickup of the line-of-sight image by the image pickup unit 11, the control unit 17 collects the voice including the user's voice and surrounding sounds by using the microphone 14, and stores the voice data in the storage unit 15. You may remember it.

Next, in step S14, the control unit 17 transmits the line-of-sight image to the basket device 20 via the communication unit 16. The control unit 17 is provided with, for example, an accelerometer or an angular velocity meter in the mounting device 10 to measure the moving speed of the user, and when the moving speed of the user reaches a predetermined moving speed (for example, 1 km / h or less), the basket The line-of-sight image may be transmitted to the device 20. The user may take his / her hand off the basket device 20 during shopping. Therefore, the control unit 17 transmits the line-of-sight image to the basket device 20 by human body communication when human body communication is possible, and transmits the line-of-sight image by proximity communication or wireless communication when human body communication is not possible. It may be transmitted to the basket device 20. When performing wireless communication, the mounting device 10 needs to be provided with a wireless communication unit.

Next, in step S16, the control unit 17 determines whether or not human body communication with the basket device 20 input from the cash register terminal has been established. Here, the input from the cash register terminal is transmitted from the cash register terminal to the basket device 20 by communication between the basket device 20 and the cash register terminal when the user has the basket 120 at the cash register. , Means to enter shopping end information. That is, when the human body communication between the basket device 20 and the mounting device 10 input from the cash register terminal is established, it means that the user's shopping is completed. Therefore, if the shopping is not completed and the determination in step S16 is denied, the process returns to step S12, and the processing / determination in steps S12, S14, and S16 is repeated. That is, the image pickup of the line-of-sight image by the image pickup unit 11 and the transmission to the basket device 20 are repeated. On the other hand, when the user's shopping is completed and the determination in step S16 is affirmed, the entire process of FIG. 4A is completed.

In addition, instead of step S16, the control unit 17 may determine whether or not the user (wearing device 10) has left the store. In this case, as described above, the mounting device 10 needs to be provided with a position detecting device such as GPS.

(Processing of article device 30)
FIG. 4B shows a flowchart of the processing of the article device 30.

In the process of FIG. 4B, the control unit 33 of the article device 30 waits in step S20 until human body communication with the basket device 20 is newly established. In this case, as shown in FIG. 2, at the timing when the user holding the basket 120 picks up the article 130 from the product shelf or the like, human body communication with the basket device 20 is newly established, and the process proceeds to step S22.

When the process proceeds to step S22, the control unit 33 issues an instruction to the communication unit 32 and transmits the article information stored in the storage unit 31 to the basket device 20.

After that, the process returns to step S20, and the control unit 33 repeats the processing / determination of steps S20 and S22.

By executing the process of FIG. 4B, the user holding the basket 120 can transmit the information of the article 130 obtained during shopping to the basket device 20 one by one.

(Processing of basket device 20)
FIG. 5 shows a flowchart of the processing of the basket device 20.

In the process of FIG. 5, in step S30, the control unit 26 of the basket device 20 waits until human body communication with the mounting device 10 is established. When the human body communication with the mounting device 10 is established, the control unit 26 shifts to step S32. The timing of shifting to step S32 is the same as the timing of shifting to step S12 of FIG. 4A described above.

When the process proceeds to step S32, the control unit 26 acquires the line-of-sight image transmitted from the mounting device 10 and stores it in the storage unit 23. For example, an accelerometer or an angular velocity meter is provided in the basket device 20 to measure the movement speed of the user, and when the movement speed of the user reaches a predetermined movement speed (for example, 1 km / h or less), a line-of-sight image from the mounting device 10. May acquire the transmitted image and store it in the storage unit 23.

Next, in step S34, the control unit 26 determines whether or not new article information has been received. If the determination in step S34 is denied, that is, if new article information has not been received, the process proceeds to step S38. On the other hand, if the determination in step S34 is affirmed, that is, when new article information is received, the process proceeds to step S36.

When the process proceeds to step S36, the control unit 26 acquires the article information and stores it in the storage unit 23. After that, the process proceeds to step S38.

When the process proceeds to step S38, the control unit 26 executes detection using the biosensor 21. The order of the processes of steps S32, S34, S36, and S38 may be changed as appropriate, and the processes of each step may be executed in parallel.

Next, in step S40, the control unit 26 determines whether or not the user has an emotional change based on the detection result of the biological sensor 21. In this case, for example, from the detection result of the biological sensor 21, it is said that the user has a change in emotion when the heart rate suddenly rises, the blood pressure suddenly rises, the sweating amount suddenly increases, and the like. to decide. Examples of changes in user emotions include when the time service starts and you encounter a chance to purchase a product at a low price, the product you were planning to purchase is sold out, or the cash register is crowded. be. If the determination in step S40 is denied, the process proceeds to step S44, but if the determination in step S40 is affirmed, the process proceeds to step S42.

When the process proceeds to step S42, the control unit 26 issues an instruction to the second communication unit 25, and the line-of-sight image and article information (hereinafter referred to as first information) stored in the storage unit 23 and acquired immediately before the emotional change. ) Is transmitted to the server 40. The term immediately before the emotional change means, for example, a few seconds before the emotional change occurs. In some cases, the article information is not acquired immediately before the emotional change. This is the case where the emotional change is not due to holding the article 130. In this case, the control unit 26 may transmit the line-of-sight image as the first information to the server 40.

Here, as the line-of-sight image acquired immediately before the emotional change, the line-of-sight images as shown in FIGS. 7 (a), 7 (b), and 8 are assumed. The line-of-sight image of FIG. 7A is an image showing that the user has encountered the time service, and specifically, is an image showing that the user has seen the time service sticker attached to the product. Further, the line-of-sight image of FIG. 7B is an image when the product that the user is trying to purchase is sold out, and specifically, is an image showing that the user has seen a notice waiting for arrival. Further, the line-of-sight image of FIG. 8 is an image showing that the user has seen a state in which the cash register is crowded.

The first information is transmitted to the server 40 with the basket ID added. That is, the server 40 manages the first information by using the basket ID.

After the step S40 is denied or the process of step S42 is executed, the control unit 26 determines in step S44 whether or not there is an input from the cash register terminal. In this case, the input of the cash register terminal is, as described above, from the cash register terminal by communication between the basket device 20 and the cash register terminal when the user finishes shopping and brings the basket 120 to the cash register. It means inputting shopping end information transmitted to the basket device 20. If the determination in step S44 is denied, that is, if the user's shopping has not been completed, the process returns to step S32, and the processing / determination in steps S32 to S44 is repeated. On the other hand, if the determination in step S44 is affirmed, the process proceeds to step S46.

When the process proceeds to step S46, the control unit 26 issues an instruction to the second communication unit 25, and all the line-of-sight images, article information, and biometric information (hereinafter referred to as second information) acquired after step S30 are sent to the server 40. Send. That is, the control unit 26 uses the second communication unit 25 to obtain a line-of-sight image while the user is shopping, information on the goods that the user has obtained, and while the user is shopping. The emotional change information obtained in the above is transmitted to the server 40. The control unit 26 may transmit the voice data acquired by using the microphone 14 of the wearing device 10 to the server 40 while the user is shopping. It is assumed that each information transmitted from the control unit 26 is given an information acquisition time (obtained from the calendar unit 22) as a time stamp. Further, the second information is transmitted to the server 40 with the basket ID added. That is, the server 40 manages the second information by using the basket ID. After that, the control unit 26 ends all the processing of FIG.

As described above, by executing the process of FIG. 5 by the control unit 26, the first information is transmitted from the basket device 20 to the server 40 at the timing when the user's emotions change while shopping. The second information is transmitted from the basket device 20 to the server 40 at the timing when the user finishes shopping.

(Processing of server 40)
FIG. 6 shows the processing of the server 40 in a flowchart.

In the process of FIG. 6, the control unit 46 of the server 40 waits until there is an input from the basket device 20 in step S50. That is, the control unit 46 shifts to step S52 at the stage where the first information or the second information is transmitted by executing step S42 or S46 of FIG. 5 in the basket device 20.

When the process proceeds to step S52, the control unit 46 determines whether or not the input information is the first information. That is, the control unit 46 determines whether or not the line-of-sight information or the article information acquired by the basket device 20 has been input immediately before the user has a change in emotion. If the determination here is affirmed, the process proceeds to step S54.

When the process proceeds to step S54, the control unit 46 executes the sentiment analysis based on the first information. Specifically, the control unit 46 analyzes the result of analyzing the line-of-sight image acquired immediately before the emotion changes using the OCR unit 43 and the image recognition unit 44, and the article information acquired immediately before the emotion changes. Based on the analysis result, the user's emotion analysis (analysis of satisfaction or dissatisfaction) is performed.

For example, when the line-of-sight image is the image of FIG. 7A, the control unit 46 can detect the character "20% discount from the morning market time service display price" by using the OCR unit 43. .. In this case, since the words positive for the shopper such as time service and 20% discount are included, the control unit 46 analyzes that the user is happy and satisfied (positive state). It is assumed that a word positive for the shopper is stored in the storage unit 41.

In addition, when the line-of-sight image is the image shown in FIG. 7B, the control unit 46 uses the OCR unit 43 to "sorry. It is currently backordered. The next arrival time is undecided." Can be detected. In this case, since negative words for the shopper such as sorry, backordered, and undecided are included, the control unit 46 analyzes that the user is in a sad and unsatisfied state (negative state). It is assumed that a word negative for the shopper is stored in the storage unit 41.

Further, when the line-of-sight image is the image of FIG. 8, the control unit 46 uses the image recognition unit 44 to make the cash register crowded and the person waiting for the cash register is dissatisfied. It is possible to detect that (not a smile). In this case, the control unit 46 determines that the user is in a sad and unsatisfactory state (negative state).

Further, when the line-of-sight image includes an image of an employee's face, the control unit 46 determines that the user's emotions have changed due to the employee. In this case, the control unit 46 recognizes whether or not the employee's face is smiling by using the image recognition unit 44. The image recognition unit 44 stores the employee's face image stored in the storage unit 41 as to whether or not the face image included in the line-of-sight image is an employee's face image and whether or not the image is a smile. Judgment using data. Then, the control unit 46 determines that the user is in a sad or angry unsatisfied state (negative state) when the employee is not smiling, and is satisfied when the employee is smiling. Judge that it is a state (positive state).

Further, when the state of holding the article in the hand is imaged in the line-of-sight image, the control unit 46 determines that the emotion has changed due to seeing or touching the article. In some cases, the control unit 46 can determine whether the user is happy or sad by using the article information acquired by the basket device 20 together. For example, when the article information includes information such as a limited quantity, it can be determined that the user is pleased. Further, when it is found from the article information that the expiration date has expired, it can be determined that the user is sad.

Returning to FIG. 6, in the next step S56, the control unit 46 determines whether or not a response is necessary based on the emotion analysis result. Specifically, the control unit 46 determines that it is necessary to take measures when the user is in an unsatisfied state (when in a negative state) such as being sad or angry. If the determination in step S56 is denied, the process returns to step S50. On the other hand, if the determination in step S56 is affirmed, the process proceeds to step S58.

When the process proceeds to step S58, the control unit 46 executes a response process based on the emotion analysis result. For example, when the line-of-sight image is shown in FIG. 4B, the control unit 46 sends various messages such as an apology message and the next arrival schedule stored in the storage unit 41 to the basket device via the communication unit 42. Send to 20. Then, the control unit 26 of the basket device 20 transmits various messages to the mounting device 10 via the first communication unit 24. In this case, in the mounting device 10, the control unit 17 displays (projects) various messages using the display unit 12.

Further, for example, when the line-of-sight image is shown in FIG. 8, the control unit 46 transmits a message of apology for the congestion of the cash register to the control unit 17 of the mounting device 10 via the basket device 20. In this case, the control unit 17 displays (projects) an apology message on the display unit 12. The control unit 46 may output an apology message for the congestion of the cash register by using the in-store broadcast. Further, the control unit 46 may output a message (message of cash register support) to the effect that the employee who performs the cash register should be replenished by using the in-store broadcast. Alternatively, the control unit 46 may output a cash register support message to the terminal held by the employee. In addition, the control unit 46 determines the progress of the cash register (processing time per customer) and the skill level of the clerk in charge of the cash register (apprenticeship or veteran) for the user who sees the situation where the cash register is crowded. May be notified.

Further, for example, when it is determined from the line-of-sight image that there is a problem in the response of the employee, the control unit 46 receives the customer and the employee with respect to the terminal held by another employee, the store manager, or the like. Output a message to the effect that a problem has occurred. If the basket device 20 is provided with a position detection device capable of detecting the position according to the communication sensitivity with the wireless LAN base station, the control unit 46 will have trouble based on the detection result of the position detection device. It is possible to inform other employees, store managers, etc. of the location of occurrence.

Further, for example, when it is determined from the line-of-sight image that the customer is sad or angry at the article, the control unit 46 uses the in-store broadcast or the terminal held by the employee to use the employee. May output a message to the effect that it should respond directly. In this case as well, the control unit 46 may inform the employee of the position of the customer.

The first information may include voice data collected by the microphone 14 immediately before the emotional change. According to the analysis of the voice data, the control unit 46 is in an angry or unsatisfied state (negative state) when the voice data contains a user's angry voice, a sigh, or an employee's apology. It can be judged that it is in).

It is assumed that the pattern of the corresponding processing described above is predetermined, and that the pattern is stored in the storage unit 41.

After the processing of step S58 is completed as described above, the process returns to step S50.

On the other hand, if the determination in step S52 is denied, that is, if the information input from the basket device 20 is the second information, the process proceeds to step S60.

When the process proceeds to step S60, the control unit 46 executes the purchase behavior analysis based on the second information. In this case, the control unit 46 analyzes what the user sees, picks up, and purchases in the middle of shopping. In other words, it can be said that it analyzes which products the user did not see or pick up.

FIG. 9 shows the order of the items picked up by a user from entering the store to passing through the cash register. Further, in FIG. 9, the articles purchased by the user are underlined. The control unit 46 can use the purchase information acquired from the cash register terminal as the information of the goods purchased by the user.

In FIG. 9, the user picks up the curry-A and the curry-B, but finally purchases the curry-A. In this case, the control unit 46 purchased the curry-A as a criterion by referring to the line-of-sight image when the curry-A was picked up and the line-of-sight image when the curry-B was picked up. To analyze. For example, if the price information is included for a long time in the area (predetermined position (almost center) of the line-of-sight image) that the user is gazing at, it is determined that the curry roux A has been purchased based on the price. In addition, if the area that the user is watching contains a spiciness display, a package photo, an ingredient display, or a long-term expiration date, the curry roux A was purchased based on such information. Judge. The control unit 46 uses the OCR unit 43 to extract whether or not the area that the user is watching includes the display of price and spiciness, the ingredients, and the expiration date.

Similarly, for beef A and B, the judgment criteria at the time of purchase may be judged. For example, when the area that the user is gazing at includes the display of the content, the display of the part, the price per 100 g, and the like for a long time, the control unit 46 is based on the information of the beef. It is determined that B has been purchased.

Further, the control unit 46 refers to the data acquired by a plurality of users, aggregates what the other product that the user who purchased one product has compared and examined, and purchased as a result of the comparison and examination. It is possible to aggregate what was the deciding factor of the product. The analysis result of the judgment criteria at the time of purchase may be provided to the manufacturer or producer of each product. Manufacturers and producers can use the results of this analysis for future product development and product improvement.

Further, the control unit 46 can total the order in which the customer picks up the article by referring to the data in FIG. In this case, the control unit 46 can analyze the tendency of the customer's movement in the store by using the aggregation result. As a result, the control unit 46 can request an appropriate arrangement of articles and shelves based on the movement of the customer.

Further, the control unit 46 aggregates the number and position of the user's positive state and the number and position of the user's negative state between entering and leaving the store, thereby satisfying and dissatisfiing the customer. Can be analyzed by position (by sales floor).

Further, the control unit 46 may analyze the line-of-sight image when the customer changes from a negative state to a positive state. As a result of analyzing the line-of-sight image, the control unit 46 solved the problem by the employee's response when the line-of-sight image included the employee's face while changing from the negative state to the positive state. Can be analyzed. Further, when the response process using the mounting device 10 or the in-store broadcast is performed while the negative state changes to the positive state, it can be analyzed that the response process was appropriate.

As described in detail above, according to the first embodiment, the control unit 46 and the communication unit 42 of the server 40 input line-of-sight images and article information as operation information related to the user's purchase, and relate to the user's purchase. The detection result of the biological sensor 21 is input as emotion information. As a result, the control unit 46 can analyze the purchase based on the line-of-sight image, the article information, and the detection result of the biological sensor 21, and can acquire and provide useful information on the purchase. Therefore, the usability by the user can be improved. In this case, by using the line-of-sight image taken from the user's side, the situation actually seen by the user, that is, the article or the article in front of the line of sight, is compared with the case of using the image taken from the shelf side of the store. Analysis that takes into account the display, situation, etc. is possible.

Here, conventionally, when a user wearing a device equipped with an image pickup unit 11 such as a mounting device 10 enters a store, the inside of the store and products are imaged without permission. It could not be said that there was a merit for the side. On the other hand, in the first embodiment, instead of allowing the user to wear the mounting device 10 and enter the store, the store side transfers the image data captured by the imaging unit 11 via the basket device 20. I'm going to get it. As a result, the store can analyze the behavior and emotions of the user up to the purchase of the product by analyzing the image data.

Also, in the past, shopping satisfaction was collected by questionnaires to users, but users need to access the store's website to answer questions or write answers on paper. It was annoying. On the other hand, in the first embodiment, the image data acquired by the mounting device 10 and the biometric information of the user are linked and provided to the store side, so that the user unknowingly satisfies the user. It is possible to convey the degree of dissatisfaction to the store side.

Further, in the first embodiment, a message to the user and a message to the employee are output based on the line-of-sight image, the article information, and the detection result of the biosensor 21. This makes it possible to immediately respond to a user who feels unsatisfied.

Further, in the first embodiment, the control unit 46 determines that the user holds the article from the line-of-sight image and the article information. As a result, the control unit 46 can acquire and provide useful information regarding purchasing by analyzing the information that the user holds the goods, the order in which the goods are held, and the like.

Further, in the first embodiment, since the control unit 46 uses the detection result of the biological sensor 21, it is possible to easily determine whether or not the user has an emotional change. Further, by analyzing the user's line-of-sight image together with the detection result of the biosensor 21, it is possible to accurately determine whether the user's emotion is positive or negative.

Further, in the first embodiment, the wearing device 10 captures a line-of-sight image and transmits the captured line-of-sight image to the basket device 20 in contact with the user via human body communication. As a result, the basket device 20 can acquire only the line-of-sight image of the user holding the basket 120. That is, the line-of-sight image is surely acquired from the wearing device 10 of the user who has entered the store, and the line-of-sight image is acquired from the wearing device 10 of a person who has not entered the store (for example, a person who has passed near the store). It can be surely prevented from being stored.

Further, in the first embodiment, the first information is transmitted to the server 40 when there is a change in the detection result of the biosensor 21, that is, when there is a change in the user's emotion (S42). ). As a result, the information that needs to be analyzed can be transmitted to the server 40 at an appropriate timing.

In the first embodiment, the case where the basket device 20 includes the biosensor 21 has been described, but the present invention is not limited to this, and the mounting device 10 may include the biosensor. In this case, a biosensor that comes into contact with a part of the frame of the mounting device 10 or a part of the face near the frame may be provided. Further, a biosensor that can be attached to the earlobe may be connected to the attachment device 10.

Further, in the first embodiment, the case where the article device 30 is provided in the article 130 has been described. However, the present invention is not limited to this, and when the article information is not required in the sentiment analysis in step S54 or the purchase behavior analysis in step S60, the article device 30 does not necessarily have to be provided in the article 130. For example, in the line-of-sight image, when the state of holding an article in a hand is captured, the image recognition may be performed to identify the article held in the hand.

In the first embodiment, the case where the article device 30 performs human body communication with the basket device 20 has been described, but the present invention is not limited to this, and the article device 30 has a human body communication with the mounting device 10. May be done. In this case, the article information acquired by the mounting device 10 may be transmitted from the mounting device 10 to the basket device 20 in a state where human body communication is established between the mounting device 10 and the basket device 20.

In the first embodiment, the case where the detection result of the biological sensor 21 is used has been described, but the present invention is not limited to this, and the detection result of the biological sensor 21 may not be used. In this case, the control unit 46 estimates that the user is in a positive state when it is determined that the user is staring at the mark of FIG. 7A for a predetermined time, for example, based on the line-of-sight image. You may do it. Further, when the control unit 46 determines, for example, based on the line-of-sight image that the user is staring at the poster in FIG. 7 (b) or the waiting state for registration in FIG. 7 (c) for a predetermined time, the user is negative. It may be presumed that it is in such a state. In this case, the control unit 46 stores in advance an image pattern of a mark, a sticker, or a situation that is predicted to be in a positive or negative state when viewed by the user, and compares the image pattern with the line-of-sight image. And it is sufficient.

In the first embodiment, the case where a predetermined position of the line-of-sight image is regarded as the center position of the line-of-sight has been described, but the present invention is not limited to this. For example, it has an infrared irradiation unit and an infrared light receiving unit, and the infrared irradiation unit irradiates the eyeball with infrared rays, and the infrared rays reflected by the eyeball are received by the infrared light receiving unit to detect the line-of-sight direction. The detection unit may be separately provided in the mounting device 10.

The server 40 may aggregate the satisfaction and dissatisfaction of customers in a plurality of stores and publish it as a user evaluation of each store on one website. In this case, for example, the evaluation of each store can be announced by the number of stars, the score of satisfaction, and the like.

<< Second Embodiment >>
Next, the electronic device system 100'according to the second embodiment will be described in detail with reference to FIGS. 10 to 14.

FIG. 10 shows the configuration of the electronic device system 100'in a block diagram. In FIG. 10, among the electronic device systems 100', a configuration different from that of the electronic device system 100 of the first embodiment is shown by a thick line frame.

As shown in FIG. 10, in the electronic device system 100', the mounting device 10'is provided with a speaker 18, the basket device 20'is provided with a weight sensor 27, and the article device 30'is provided with a light emitting unit 34. The configuration of the server 40'is the same as that of the server 40 of the first embodiment.

The speaker 18 is used to provide voice information to the user who wears the wearing device 10'. As the speaker 18, in addition to earphones and headphones, a speaker having directivity and capable of providing voice information only to a user who wears the wearing device 10'can be adopted. It is assumed that the storage unit 15 of the mounting device 10'stores the article information corresponding to the light emitting pattern of the light emitting unit 34 of the article device 30'described later.

The weight sensor 27 is provided in a part of the basket device 20'and is a sensor for detecting a change in the internal capacity of the basket 120.

The light emitting unit 34 includes a light emitting element such as an LED (Light Emitting Diode), and emits light in a light emitting pattern predetermined for each article. The light emission of the light emitting unit 34 is controlled by the control unit 33. It is presumed that the control unit 17 of the mounting device 10'sees an article corresponding to the light emission pattern when the light emission pattern is imaged within a predetermined range of the angle of view in the moving image shooting by the image pickup unit 11. It is assumed that the on / off switching timing (blinking interval of the light emission pattern) of the light emission by the light emitting unit 34 is set according to the frame rate of the image pickup unit 11. A visible light LED or an infrared LED may be used as the light emitting unit 34, and when an infrared LED is used, an infrared image pickup element may be used for the image pickup unit 11.

Next, the processing of the electronic device system 100'in the second embodiment will be described in detail with reference to other drawings according to the flowcharts of FIGS. 11 to 13. In addition, also in this 2nd Embodiment, the case where the electronic device system 100'is used in a supermarket will be described.

(Processing of mounting device 10')
FIG. 11A shows a flowchart of the process of the mounting device 10'. In FIG. 11A, a step different from the processing of the mounting device 10 of the first embodiment (FIG. 4A) is shown by a thick line frame.

In the process of FIG. 11A, the control unit 17 of the mounting device 10'stands by in step S10 until human body communication with the basket device 20'is established. When the human body communication with the basket device 20'is established, the control unit 17 proceeds to step S110.

When the process proceeds to step S110, the control unit 17 issues an instruction to the image pickup unit 11 and executes imaging (moving image) from the user side.

Next, in step S112, the control unit 17 determines whether or not the light emitted from the light emitting unit 34 has been imaged within a predetermined range of the angle of view. That is, it is determined whether or not the user has seen the article provided with the article device 30'. If the determination in step S112 is denied, the process proceeds to step S116, but if the determination in step S112 is affirmed, the process proceeds to step S114.

When the process proceeds to step S114, the control unit 17 transmits the article information seen by the user to the basket device 20'. Specifically, the control unit 17 acquires article information corresponding to the light emission pattern detected within a predetermined range of the angle of view of the image pickup unit 11 from the storage unit 15, and the basket device 20'via the communication unit 16. It transmits to the first communication unit 24.

When the process proceeds to step S116 after the determination in step S112 is denied or the process in step S114 is performed, the control unit 17 determines whether or not information has been received from the basket device 20'. The information received from the basket device 20'is information related to goods and store services transmitted from the server 40'. If the determination here is denied, the process proceeds to step S16, but if the determination is affirmed, the process proceeds to step S118.

When the process proceeds to step S118, the control unit 17 outputs the information received from the basket device 20'. The details of the process of this step S118 will be described later.

When the process proceeds to step S16 after the determination in step S116 is denied or the process in step S118 is performed, the control unit 17 establishes human body communication with the basket device 20'input from the cash register terminal. Judge whether or not. Here, the input from the cash register terminal means that when the user finishes shopping and brings the basket 120 to the cash register, the basket device 20'is communicated between the basket device 20'and the cash register terminal. 'It means to enter the shopping end information sent to. That is, when the human body communication between the basket device 20'and the wearing device 10'that has been input from the cash register terminal is established, it means that the user's shopping is completed. Therefore, if the shopping is not completed and the determination in step S16 is denied, the process returns to step S112 and the processing / determination in steps S112 to S16 is repeated. On the other hand, when the user's shopping is completed and the determination in step S16 is affirmed, the entire process of FIG. 11A is completed.

(Processing of article device 30')
FIG. 11B shows a flowchart of the processing of the article device 30'. In addition, in FIG. 11B, a step different from the processing of the article apparatus 30 of the first embodiment (FIG. 4B) is shown by a thick line frame.

As shown in FIG. 11B, in the article device 30', in step S120, the control unit 33 emits light using the light emitting unit 34. In this case, the control unit 33 causes the light emitting unit 34 to emit light in a light emitting pattern corresponding to the article provided with the article device 30'. Other processing is the same as in FIG. 4 (b).

(Processing of basket device 20')
In FIG. 12, the processing of the basket device 20'is shown in a flowchart. In the process of FIG. 12, in step S130, the control unit 26 waits until the human body communication with the wearing device 10'is established. When the human body communication with the mounting device 10'is established and the process proceeds to step S132, the control unit 26 determines whether or not the article information has been received from the mounting device 10'.

If the determination in step S132 is affirmed, the process proceeds to step S134, and the control unit 26 transmits the article information received from the mounting device 10'to the server 40'as the article information of the seen article, and steps S136. Move to. On the other hand, if the determination in step S132 is denied, the process proceeds to step S136 without going through step S134.

When the process proceeds to step S136, the control unit 26 determines whether or not the article information has been received from the article device 30'. If the determination in step S136 is affirmed, the process proceeds to step S138, and the control unit 26 transmits the article information received from the article device 30'to the server 40'as the article information of the touched article, and steps S140. Move to. On the other hand, if the determination in step S136 is denied, the process proceeds to step S140 without going through step S138.

When the process proceeds to step S140, the control unit 26 determines whether or not the detected value of the weight sensor 27 has changed. Here, when the detected value of the weight sensor 27 changes, it means that the article is newly put in the basket 120. Although the detected value of the weight sensor 27 changes even when the article in the basket 120 is taken out, here, for convenience of explanation, it is assumed that the article is not taken out from the basket 120. If the determination in step S140 is affirmed, the process proceeds to step S142, and the control unit 26 converts the article information of the article touched immediately before into the article information of the article in the basket, that is, the article to be purchased by the user. It is transmitted to the server 40'as the information of. On the other hand, if the determination in step S140 is denied, the process proceeds to step S144 without going through step S142.

When the process proceeds to step S144, the control unit 26 determines whether or not information has been received from the server 40'. The information received from the server 40'is information related to goods and store services. If the determination in step S144 is affirmed, the process proceeds to step S146, and the control unit 26 transmits the information received from the server 40'to the mounting device 10'and proceeds to step S148. On the other hand, if the determination in step S144 is denied, the process proceeds to step S148 without going through step S146.

When the process proceeds to step S148, the control unit 26 determines whether or not there has been an input from the cash register terminal. Here, the input from the cash register terminal means the input of the shopping end information as described above.

If the determination in step S148 is denied, the process proceeds to step S132 and the processing / determination in steps S132 to S148 is repeated. If the determination in step S148 is affirmed, the control unit 26 is shown in FIG. End all processing.

(Processing of server 40')
FIG. 13 shows the process of the server 40'in a flowchart.

In the process of FIG. 13, in step S150, the control unit 46 waits until the article information is received from the basket device 20'. That is, when any one of the article information of the article seen by the user, the article information of the article touched by the user, and the article information of the article put in the basket 120 is input, the control unit 46 shifts to step S152.

When the process proceeds to step S152, the control unit 46 updates the shopping DB stored in the storage unit 41. Here, the shopping DB has a data structure as shown in FIG. 14A. As shown in FIG. 14 (a), it has columns for the name of the article, seen, touched, and put in a basket. When the control unit 46 receives the article information of the article seen by the user, it stores a circle in the column of the article seen by the user, and when it receives the article information of the article touched by the user, the control unit 46 touches the article. ○ is stored in the column, and when the user receives the article information of the article put in the basket, ○ is stored in the column of the article put in the basket of the article.

Next, in step S154, the control unit 46 analyzes the shopping DB. For example, when the control unit 46 refers to the shopping DB of FIG. 14A and a predetermined time has passed since the beef A was held in the hand (when new article information is not received for a predetermined time). Can analyze that the user may need information on an article (eg, beef B) that can be compared with beef A. Further, since the control unit 46 can estimate that the user will make curry from the articles put in the basket, for example, the amount of beef required for making curry (Xg for 4 dishes, Yg for 8 dishes, etc.) ) Information may be needed by the user. In addition, the control unit 46 may require information on bargain information on curry garnish (Fukujinzuke, roast cutlet, dessert, etc.) and store campaign information (such as if you buy another Z yen, you can draw a lottery). It can be analyzed that there is.

Next, in step S156, the control unit 46 determines whether or not there is information to be notified to the user as a result of the analysis in step S154. If the determination here is denied, the process returns to step S150, but if the determination is affirmed, the process proceeds to step S158. When the user touches the same article multiple times, or when the user touches an article similar to the previously touched article, the same information as the information previously notified is the information notified to the user. It may be judged. Therefore, the control unit 46 may limit the notification of the same information as the information previously notified, assuming that the information should not be notified to the user. That is, the control unit 46 limits the information to be notified when the same article is touched or viewed multiple times, or when an article similar to the previously touched or viewed article is touched or viewed. You may do it. By doing so, it is possible to prevent the same user from being repeatedly notified of the same information.

The control unit 46 may change the information to be notified according to the number of times the same article is touched or the number of times the same article is seen. For example, the control unit 46 makes the information to be notified the first time the information of the article itself, the second time the information of the article to be compared and controlled by the article, and the third time the information related to the article. It may be that. By doing so, it is possible to improve the possibility that a customer who is uncertain about purchasing an article will purchase the article. Instead of this, depending on the time when the user touches the same article, the user may be notified from the information which has not been notified to the user from the second time onward.

If the determination in step S156 is affirmed, the control unit 46 proceeds to step S158 and transmits the information to be notified to the user to the basket device 20'. When the process of step S158 is performed, the determination of step S144 is affirmed in the process of FIG. 12, and the process of step S146 (the process of transmitting the information received from the server to the mounting device 10') is executed. .. Further, when the process of step S146 is executed in FIG. 12, the determination of step S116 of FIG. 11A is affirmed, and the process of step S118 (process of outputting the received information) is executed. In step S118, the control unit 17 uses the display unit 12 to display the information to be notified to the user, and the speaker 18 to output the information to be notified to the user. After step S158, the process returns to step S150 and the processes of steps S150 to S158 are repeated.

The control unit 46 may perform various analyzes based on the shopping DB (for example, FIG. 14B) of the user who has finished shopping. In this case, the control unit 46 may compare curry-A and curry-B, and beef A and beef B, and analyze that curry-B and beef B have been purchased, for example, based on FIG. 14 (b). can. The control unit 46 can also analyze the order of walking in the store, the order of purchase, and the like. In addition, the control unit 46 can identify and analyze an item that was not purchased just by looking at the item and an item that was picked up but not purchased. The control unit 46 may provide the analysis result to the manufacturer, seller, or the like of the product.

As described in detail above, according to the second embodiment, the control unit 46 determines that the user has obtained the article and that the article is within a predetermined range in the user's field of view. Accordingly, the article information of the article is input (S150: affirmative), and the notification regarding the input article information is given (S158). Therefore, in the second embodiment, it is possible to notify the user of useful information related to the article touched by the user or the article seen by the user. As a result, the usability of the user can be improved.

Further, in the second embodiment, the control unit 46 inputs the information of the product to be purchased by the user put in the basket 120 as the product information, and gives a notification according to the input product. As a result, useful information related to the product that the user plans to purchase can be notified to the user. Further, since the control unit 46 inputs the information of the product to be purchased from the basket device 20'used at the time of shopping, accurate information can be input as the information of the product to be purchased.

Further, in the second embodiment, since the information is notified to the user from the display unit 12 of the mounting device 10'or the directional speaker 18, the user's information is not known to other users. Notifications can be made according to purchasing behavior.

In the second embodiment, the control unit 46 may determine that the two products are being compared and examined when the user is holding an article and looking at another article.

In the second embodiment, the example of using the electronic device system 100'in a supermarket has been described, but the present invention is not limited to this, and the electronic device system 100'may be used in a clothing store or the like. For example, when the user picks up clothes, the control unit 46 provides information on the presence or absence of inventory such as different colors and sizes, information on whether or not there are similar products on sale, and buys one more item. By using the display unit 12 and the speaker 18 to notify the discount information such as the discount when the purchase is 2000 yen or more, the specific user (specialty of store members, etc.) without being known to other users. Can only be notified. Further, the server 40'may send discount information, present information, and a congratulatory message to the wearing device 10'when the user of the store member visits the store on his / her birthday.

In the second embodiment, the control unit 17 has described the case where the article information acquired by the mounting device 10'is transmitted to the server 40' via the basket device 20', but the present invention is limited to this. It's not a thing. For example, when the mounting device 10'has a communication unit capable of wireless communication, information may be transmitted from the mounting device 10'to the server 40'by wireless communication.

In the second embodiment, the case where the article device 30'has a light emitting unit 34 has been described, but the present invention is not limited to this, and the article device 30'may not have the light emitting unit 34. In this case, the control unit 17 may identify the article seen by the user by analyzing the image captured by the image pickup unit 11. For example, the control unit 17 may recognize characters included in the display (for example, characters of a product tag or a POP advertisement) to identify an article viewed by the user.

In the first and second embodiments, the control unit 46 may consider the type of user when analyzing the information. If the user is a member of the store, the type of user can be grasped from the registered information (name, date of birth, address, shopping history). In this case, the user's purchasing behavior may be analyzed for each user's gender and age.

Each of the above-described embodiments is an example of a preferred embodiment of the present invention, and can be appropriately combined. However, the present invention is not limited to this, and various modifications can be made without departing from the gist of the present invention.

10 Mounting device 11 Imaging unit 12 Display unit 16 Communication unit 17 Control unit 18 Speaker 20 Basket device 42 Communication unit 46 Control unit 110 Frame 120 Basket

Claims (15)

An entry detection unit installed in a store that detects the user's entry into the store,
A recognition unit that recognizes an article held by the user based on image information captured at the store.
An acquisition unit that acquires information about the article recognized by the recognition unit, and
An electronic device system including a transmission unit that transmits information about the article acquired by the acquisition unit to the device of the store. The electronic device system according to claim 1, wherein the image information is imaged based on proximity communication from a communication unit provided in the article. The first or second aspect of claim 1 or 2, wherein the store entry detection unit is a device installed in the store and detects the user's entry by communicating with the user's possession device. Electronic device system. The electronic device system according to any one of claims 1 to 3, wherein the transmission unit transmits the information to the device of the store and the terminal possessed by the clerk of the store. The transmitting unit is characterized in that when the user picks up the article a plurality of times, or when the article enters a predetermined range within a predetermined range of the user a plurality of times, the transmission of the information is restricted. The electronic device system according to any one of claims 1 to 4. The claim is characterized in that the transmitting unit makes the content of the information different depending on the number of times the user picks up the article or the number of times the article enters a predetermined range in the user's field of view. The electronic device system according to 5. The transmitting unit limits the transmission of the information when the user picks up a similar article similar to the article or when the similar article falls within a predetermined range in the user's field of view. The electronic device system according to any one of claims 1 to 6, wherein the electronic device system is characterized. The acquisition unit acquires information on the product that the user plans to purchase as information on the article.
The electronic device system according to any one of claims 1 to 7, wherein the transmission unit transmits information according to the product to be purchased. The electronic device system according to claim 8, wherein the acquisition unit acquires information on the product to be purchased from an external device. The electronic device system according to claim 9, wherein the acquisition unit acquires information according to the product to be purchased from the external device via a storage device for storing the product to be purchased. The electronic device system according to any one of claims 1 to 10, wherein the recognition unit recognizes the article by using the weight sensor provided in the store and the image information captured in the store. .. The electronic device system according to any one of claims 1 to 11, wherein the recognition unit recognizes the order of walking in the store until the user picks up the article and / or the order of purchase. The electronic device system according to any one of claims 1 to 12, wherein the recognition unit recognizes whether or not the user has purchased the article. The electronic device system according to any one of claims 1 to 13, wherein the recognition unit recognizes the article that the user did not purchase. Detects the user's entry into the store and detects
Based on the image information captured at the store, it recognizes the article held by the user and recognizes it.
Get information about the article
A transmission method comprising a computer performing a process of transmitting information about the article to the device of the store.

Images