JP2024009011A - Electronic device system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic device system that is easy to use.

SOLUTION: An electronic device system comprises: a detection unit for detecting the entry of a user into a store based on information from a device owned by the user; an image recognition unit for recognizing an item in the store by inputting information captured within the store; an acquisition unit for acquiring information related to the item recognized by the image recognition unit; and a transmission unit for transmitting information to be displayed on the device owned by the user, the information related to the item acquired by the acquisition unit.

SELECTED DRAWING: Figure 10

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to an electronic device system.

2. Description of the Related Art Conventionally, a sales promotion system has been proposed that acquires behavioral information regarding store visitors' purchases of products and links the information to sales promotions (for example, see Patent Document 1).

Japanese Patent Application Publication No. 2010-277256

However, conventional sales promotion systems cannot be said to be able to obtain sufficient information and are not easy to use.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an easy-to-use electronic device system.

The electronic device of the present invention has an input unit (46, 42 ), and a notification section (46) that provides notification regarding the information on the article input by the input section.

In this case, a first control that limits the information that the notification unit reports when the user picks up the item multiple times or when the item enters a predetermined range within the user's visual field multiple times. (46). In this case, the first control unit may vary the information reported by the notification unit depending on the number of times the item is held in hand or the number of times the item enters a predetermined range within the visual field of the user. You can also use it as Further, a second control that limits information to be notified by the notification unit when the user picks up a similar item similar to the item, or when the similar item enters a predetermined range within the user's visual field. (46).

In the electronic device of the present invention, the input unit inputs information on a product that the user plans to purchase as information on the product, and the notification unit reports information corresponding to the product that the user plans to purchase. Good too. In this case, the input unit may input information about the product to be purchased from an external device (20'). Further, the input unit may input information corresponding to the product to be purchased from the external device via a storage device (120) that stores the product to be purchased.

Furthermore, in the electronic device of the present invention, the notification section may include at least one of a display section (12) that performs display and an audio output section (18) that outputs sound. In this case, the audio output section may be a directional audio output section. Furthermore, the electronic device of the present invention may include a mounting section (110) that can be mounted near the user's face.

In order to explain the present invention in an easy-to-understand manner, the above explanation has been made in correspondence with the reference numerals in the drawings representing one embodiment, but the present invention is not limited to this, and the structure of the embodiment described later is not limited to this. may be modified as appropriate, or at least a portion thereof may be replaced with other components. Further, the configuration elements whose arrangement is not particularly limited are not limited to the arrangement disclosed in the embodiments, but can be arranged at a position where the function can be achieved.

The present invention has the advantage of being able to provide an easy-to-use electronic equipment system.

FIG. 1 is a block diagram showing the configuration of an electronic device system according to a first embodiment. It is a figure showing the usage situation of a mounting device, a basket device, and an article device. It is a perspective view of a mounting device. FIG. 4(a) is a flowchart showing the processing of the mounting device according to the first embodiment, and FIG. 4(b) is a flowchart showing the processing of the article device according to the first embodiment. It is a flowchart which shows the processing of the cage device concerning a 1st embodiment. 3 is a flowchart showing processing by the server according to the first embodiment. FIGS. 7A and 7B are diagrams (part 1) showing examples of line-of-sight images. FIG. 2 is a diagram (part 2) showing an example of a line-of-sight image. FIG. 3 is a diagram showing the order of items picked up by a certain user from when he/she enters the store until he/she passes through the cash register. FIG. 2 is a block diagram showing the configuration of an electronic device system according to a second embodiment. FIG. 11(a) is a flowchart showing the processing of the mounting device according to the second embodiment, and FIG. 11(b) is a flowchart showing the processing of the article device according to the second embodiment. It is a flowchart which shows the processing of the cage device concerning a 2nd embodiment. 7 is a flowchart showing processing by a server according to a second embodiment. FIGS. 14(a) and 14(b) are diagrams showing an example of a shopping DB.

《First embodiment》
Hereinafter, the electronic device system according to the first embodiment will be described in detail based on FIGS. 1 to 9. FIG. 1 shows a block diagram of the configuration of an electronic device system 100 according to the first embodiment.

As shown in FIG. 1, the electronic device system 100 includes a mounting device 10, a basket device 20, an article device 30, and a server 40. As shown in FIG. 2, the wearing device 10 is a glasses-shaped terminal that is worn on the face of a user (shopper) visiting a store. The basket device 20 is a device provided in a basket 120 installed in a store. The article device 30 is a device provided for articles (commodities) 130 displayed in a store. The server 40 is an information processing device installed in a store, a data center, or the like. Note that communication via the human body (human body communication) is possible between the mounting device 10 and the basket device 20 and between the article device 30 and the basket device 20, and between the basket device 20 and the server 40. , wireless communication is possible.

(Mounting device 10)
As shown in FIG. 1, the mounting device 10 includes an imaging section 11, a display section 12, an operation section 13, a microphone 14, a storage section 15, a communication section 16, and a control section 17. Note that FIG. 3 shows the mounting device 10 in a perspective view. As shown in FIG. 3, the mounting device 10 includes an eyeglass-shaped frame 110. Note that the configuration of the mounting device 10 shown in FIG. 1 and not shown in FIG. 3 is provided inside the frame 110 or in a part of the frame 110.

The imaging unit 11 includes a lens, an image sensor, an image processing unit, and the like, and captures still images and moving images. The imaging unit 11 is provided near the end of the frame 110 (near the user's right eye) as shown in FIG. 3 . Therefore, when the user is wearing the mounting device 10 (the state shown in FIG. 2), an image in the direction in which the user is facing (viewing) can be captured. Further, a predetermined portion of the captured image (for example, a portion slightly shifted to the left from the center) can be regarded as a portion toward which the user is directing his/her line of sight (a portion at which the user is gazing).

The display unit 12 includes a projector provided inside the frame 110 or near the frame 110, and a prism for guiding the projected image from the projector to the user's eyes. The display unit 12 displays various information under instructions from the control unit 17.

The operation unit 13 is a touch pad provided on the frame 110 and detects the movement of a user's finger, receives an operation from the user, and transmits the received operation information to the control unit 17.

Note that the details of the imaging unit 11, display unit 12, and operation unit 13 are also disclosed in, for example, US Published Patent Application No. 2013/0044042.

The microphone 14 is provided in the frame 110 and collects the voice emitted by the user and the voices around the user. The voice collected by the microphone 14 is recognized by a voice recognition unit (not shown), and the voice recognition result is transmitted to the control unit 17 . The control unit 17 executes processing (for example, command execution processing) based on the voice recognition result. Note that the control unit 17 may perform voice recognition. Further, the control unit 17 stores the collected audio data in the storage unit 15.

The storage unit 15 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores image data captured by the imaging unit 11, audio data collected by the microphone 14, display data displayed on the display unit 12, etc. .

The communication unit 16 performs human body communication with other devices. The communication section 16 includes an electrode section 16a that is provided on the frame 110 and can be contacted by a user, and a human body communication section 16b that performs human body communication using the electrode section 16a. Human body communication includes current methods, in which a weak current is passed through the human body and modulated to transmit information, and electric field methods, in which information is transmitted by modulating the electric field induced on the surface of the human body. . In this embodiment, it is possible to use either the current method or the electric field method. Note that human body communication is possible not only when the user directly contacts the electrode section 16a of the mounting device 10, but also when the electrode section 16a and the user are not in direct contact. In this embodiment, the communication unit 16 transmits image data captured by the imaging unit 11 and audio data collected by the microphone 14 to the car device 20. Note that the communication unit 16 may perform close proximity communication with other devices using, for example, Bluetooth (registered trademark), RFID (Radio Frequency Identification), TransferJet (registered trademark), or the like.

Note that the mounting device 10 may include a wireless communication section in place of the communication section 16 or in combination with the communication section 16. In this case, the image data and audio data may be directly transmitted to the server 40 by wireless communication, or the image data and audio data may be transmitted to the car device 20 by wireless communication.

The control unit 17 controls the entire mounting device 10 in an integrated manner. The control unit 17 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. In this embodiment, the control unit 17 performs control related to imaging by the imaging unit 11, control related to displaying by the display unit 12, and the like.

Note that the mounting device 10 does not need to be a glasses-type terminal. For example, it may be a wearable terminal that can be worn at any position on the body, and some of its functions may be separate. For example, a wristwatch-type terminal may be provided with other functions as a contact lens-type terminal equipped with a display section.

(Cage device 20)
As shown in FIG. 1, the basket device 20 includes a biosensor 21, a calendar section 22, a storage section 23, a first communication section 24, a second communication section 25, a control section 26, and the like.

The biosensor 21 is provided in a portion of the basket 120 that the user touches when holding the basket 120 (handle portion). The biosensor 21 detects changes in biometric information of the body, such as heart rate, blood pressure, and amount of perspiration, and includes, for example, an LED (light emitting diode), a photo sensor, and a perspiration sensor. For example, by using an LED and a photosensor, the heart rate can be detected by receiving the light that is reflected when a finger is exposed to light emitted by the LED. Further, the sweat sensor can detect the amount of sweat by measuring the impedance of the hand using a plurality of electrodes. In this embodiment, the output from the biosensor 21 is sent to the control unit 26. Then, the control unit 26 estimates changes in the user's emotions (emotions, anger, sorrow, and happiness) based on the output of the biosensor 21.

The calendar section 22 acquires time information such as year, month, day, and time, and outputs it to the control section 26 . Further, the calendar section 22 measures time.

The storage unit 23 is, for example, a non-volatile semiconductor memory such as a flash memory, and stores information regarding the item held by the user, as will be described later. Furthermore, the storage unit 23 stores an ID number (cart ID) for uniquely identifying the car device 20.

The first communication unit 24 performs human body communication with other devices. Note that the first communication unit 24 may perform close proximity communication with other devices. In this embodiment, the first communication unit 24 communicates with the communication unit 16 of the mounting device 10 and the communication unit 32 of the article device 30 using human body communication.

The first communication section 24 includes an electrode section 24a that is provided on the handle portion of the basket 120 and can be contacted with the user's hand or arm, and a human body communication section 24b that performs human body communication via the electrode section 24a. .

The second communication unit 25 performs wireless communication with other devices. In this embodiment, wireless communication is performed with the server 40, and information on items touched by the user, image data captured by the imaging unit 11, audio data collected by the microphone 14, and detection of the biosensor 21 The results are sent to the server 40. The basket device 20 of this embodiment is a communication relay device that relays communication of article information between the article device 30 and the server 40 and communication of image data and audio data between the mounting device 10 and the server 40. It has the function of

The control unit 26 controls the entire car device 20, and includes a CPU, RAM, ROM, and the like. In this embodiment, the control unit 26 controls information communication between the article device 30, the mounting device 10, and the server 40.

Note that the basket device 20 may be provided in a cart used in a store. In this case, it is preferable that the biosensor 21 and the electrode part 24a of the first communication part 24 be provided in the grip part of the cart.

(Article device 30)
The article device 30 is an article (product) sold at a store or the like, and is installed on an article that comes into contact with the body of a store visitor (user), such as food, clothing, shoes, watches, accessories, golf clubs, bags, etc. . In the present embodiment, for convenience of explanation, the article device 30 will be described as being provided in products (food products, etc.) displayed in a supermarket.

The article device 30 includes a storage section 31, a communication section 32, and a control section 33, as shown in FIG.

The storage unit 31 is, for example, a nonvolatile semiconductor memory such as a flash memory, and stores information regarding the article (commodity) to which the article device 30 is attached. Specifically, the storage unit 31 stores article information (product information) of the article to which the article device 30 is attached. For example, when the article device 30 is attached to processed food (for example, canned food), the storage unit 31 stores article information such as manufacturer (seller), product name, raw material name, expiration date, content amount, and origin. Memorize countries etc.

The control unit 33 controls the entire article device 30, and includes a CPU, RAM, ROM, and the like. In this embodiment, the control unit 33 controls the transmission of article information stored in the storage unit 31 to the basket device 20. Note that the control section 33 and the storage section 31 may be provided at any position of the article.

The communication unit 32 performs human body communication with other devices. Note that the communication unit 32 may perform close proximity communication with other devices. In this embodiment, the communication section 32 includes an electrode section 32a provided at a position where it comes into contact with a user (for example, a hand), and a human body communication section 32b that performs human body communication using the electrode section 32a. In the present embodiment, as shown in FIG. Communication is established. Thereby, the article information stored in the storage section 31 is transmitted to the basket device 20. Note that human body communication is possible not only when the user directly contacts the electrode section 32a of the article device 30, but also when the electrode section 32a and the user are not in direct contact (such as when wearing gloves).

(Server 40)
As shown in FIG. 1, the server 40 includes a storage section 41, a communication section 42, an OCR (Optical Character Reader) section 43, an image recognition section 44, and a control section 46.

The storage unit 41 is, for example, a non-volatile semiconductor memory such as a flash memory, a hard disk drive, etc., and stores information acquired from the basket device 20, that is, image data captured by the imaging unit 11 of the mounting device 10 and the microphone 14. The voice data collected using the device 30 and the article information transmitted from the article device 30 are stored. Furthermore, the storage unit 41 stores face image data and product image data of store employees (clerks), and various data used when analyzing user purchasing behavior. In the present embodiment, as will be described later, the control unit 46 cooperates with the image recognition unit 44 to determine whether the image data captured by the imaging unit 11 of the mounting device 10 includes the employee's face or the product. Detect.

The communication unit 42 performs wireless communication with other devices. In this embodiment, the communication unit 42 performs wireless communication with the second communication unit 25 of the car device 20 by wireless communication.

The OCR unit 43 recognizes characters included in the image data captured by the imaging unit 11 of the mounting device 10 and converts the characters into text data. For example, when the image capturing unit 11 captures the image data shown in FIG. 7(b), the OCR unit 43 performs character recognition and prints the text "Sorry. We are currently backordered. The next arrival date is undecided. ”.

The image recognition unit 44 extracts feature points of the image data captured by the imaging unit 11 of the mounting device 10 using a feature detection algorithm such as SURF (Speeded Up Robust Features) or SHIFT (Scale Invariant Feature Transform), and stores the extracted feature points in the storage unit. Image recognition is performed by comparing with a plurality of standard patterns of recognition targets stored in 41. In the present embodiment, the image recognition unit 44 compares the employee's face image data and the product image data stored in the storage unit 41 to identify whether the employee or the product is in the image captured by the imaging unit 11. Detect whether it is included.

The control unit 46 controls the entire server 40, and includes a CPU, RAM, ROM, and the like.

Next, the processing of the electronic device system 100 will be described in detail using FIGS. 4 to 9. In this embodiment, it is assumed that when a user (shopper) wearing the wearing device 10 comes to a store (for example, a supermarket), he or she holds the basket 120 and goes shopping. In addition, the user decides the item to purchase by picking up the item (product) 130 provided with the item device 30 in the store or placing it in the basket 120, and processes the purchase at the cash register. do.

(Processing of mounting device 10)
FIG. 4A shows a flowchart of the processing of the mounting device 10.

In the process of FIG. 4A, the control unit 17 of the mounting device 10 waits until human body communication with the basket device 20 is established in step S10. In this case, when the user wearing the mounting device 10 enters the store and touches the basket device 20 by holding the basket 120, the first communication between the communication unit 16 of the mounting device 10 and the basket device 20 begins. Communication with the unit 24 is established. When human body communication with the car device 20 is established, the control unit 17 moves to step S12. Note that instead of step S10, the processes from step S12 onward may be executed when the user turns on the power of the mounting device 10 after entering the store. In this case, the mounting device 10 is provided with a position detection device that can detect the position according to the communication status with a GPS (Global Positioning System) module or a wireless LAN (Local Area Network) base station installed in the store, The control unit 17 may determine whether the user has entered the store based on the detection result of the position detection device. Moreover, on the contrary, the mounting device 10 may always supply power to the communication unit 16, and the mounting device 10 may be activated upon establishment of human body communication with the basket device 20.

In step S12, the control unit 17 issues an instruction to the imaging unit 11 to perform imaging from the user side. In this case, the imaging unit 11 may capture either a still image or a moving image. Note that in this embodiment, the imaging unit 11 will be described as one that captures still images. The control unit 17 stores an image captured by the imaging unit 11 (hereinafter referred to as a line-of-sight image) in the storage unit 15. Note that in step S12, in addition to capturing the line-of-sight image by the imaging unit 11, the control unit 17 uses the microphone 14 to collect audio including the user's voice and surrounding sounds, and stores it in the storage unit 15 as audio data. You may memorize it.

Next, in step S14, the control unit 17 transmits the line-of-sight image to the car device 20 via the communication unit 16. Note that the control unit 17 measures the user's movement speed by, for example, providing the mounting device 10 with an accelerometer or an angular velocity meter, and when the user's movement speed reaches a predetermined movement speed (for example, 1 km per hour or less), the control unit 17 measures the movement speed of the user. The line-of-sight image may also be transmitted to the device 20. Note that the user may take his hand off the basket device 20 during shopping. Therefore, if human body communication is possible, the control unit 17 transmits the line of sight image to the car device 20 by human body communication, and if human body communication is not possible, the control unit 17 transmits the line of sight image by proximity communication, wireless communication, etc. It may also be transmitted to the basket device 20. In addition, when performing wireless communication, the mounting device 10 needs to be equipped with a wireless communication section.

Next, in step S16, the control unit 17 determines whether or not human body communication has been established with the basket device 20 that received the input from the cash register terminal. Here, input from the cash register terminal is transmitted from the cash register terminal to the cart device 20 through communication between the cart device 20 and the cash register terminal when the user takes the cart 120 to the cash register. , means input of shopping completion information. That is, when human body communication is established between the basket device 20 and the mounting device 10 that received input from the cash register terminal, it means that the user's shopping has ended. Therefore, if the shopping has not been completed and the determination at step S16 is negative, the process returns to step S12 and the processing and determination at steps S12, S14, and S16 are repeated. That is, the imaging of the line-of-sight image by the imaging unit 11 and the transmission to the car device 20 are repeated. On the other hand, if the user has finished shopping and the determination in step S16 is affirmative, the entire process in FIG. 4(a) ends.

Note that instead of step S16, the control unit 17 may determine whether the user (wearing device 10) has left the store. In this case, as described above, the mounting device 10 needs to be provided with a position detection device such as a GPS.

(Processing of article device 30)
FIG. 4(b) shows the processing of the article device 30 in a flowchart.

In the process of FIG. 4B, the control unit 33 of the article device 30 waits until human body communication with the basket device 20 is newly established in step S20. In this case, as shown in FIG. 2, human body communication with the basket device 20 is newly established at the timing when the user holding the basket 120 picks up the article 130 from the product shelf, etc., and the process moves to step S22.

In step S22, the control section 33 issues an instruction to the communication section 32 to transmit the article information stored in the storage section 31 to the basket device 20.

After that, the process returns to step S20, and the control unit 33 repeats the processing and determination in steps S20 and S22.

By executing the process shown in FIG. 4B, the user holding the basket 120 can transmit information about the articles 130 that he or she has picked up while shopping to the basket device 20 one by one.

(Processing of the basket device 20)
FIG. 5 shows the processing of the car device 20 in a flowchart.

In the process of FIG. 5, in step S30, the control unit 26 of the basket device 20 waits until human body communication with the mounting device 10 is established. When the human body communication with the mounting device 10 is established, the control unit 26 moves to step S32. Note that the timing of moving to step S32 is the same as the timing of moving to step S12 in FIG. 4(a) described above.

In step S<b>32 , the control unit 26 acquires the line-of-sight image transmitted from the mounting device 10 and stores it in the storage unit 23 . Note that, for example, the car device 20 is provided with an accelerometer or an angular velocity meter to measure the user's movement speed, and when the user's movement speed reaches a predetermined movement speed (for example, 1 km per hour or less), the line-of-sight image is transmitted from the mounting device 10. The transmitted image may be acquired and stored in the storage unit 23.

Next, in step S34, the control unit 26 determines whether new article information has been received. If the determination in step S34 is negative, that is, if new article information has not been received, the process moves to step S38. On the other hand, if the determination in step S34 is affirmative, that is, if new article information is received, the process moves to step S36.

When proceeding to step S36, the control unit 26 acquires article information and stores it in the storage unit 23. After that, the process moves to step S38.

When proceeding to step S38, the control unit 26 executes detection using the biosensor 21. Note that the order of the processing in steps S32, S34, S36, and S38 may be changed as appropriate, and the processing in each step may be executed simultaneously.

Next, in step S40, the control unit 26 determines whether or not the user's emotions have changed based on the detection result of the biosensor 21. In this case, for example, the detection result of the biosensor 21 indicates that the user has experienced a change in emotion, such as when the heart rate suddenly increases, when the blood pressure suddenly increases, or when the amount of sweat increases suddenly. to decide. Examples of how a user's emotions may change include when a time service starts and they encounter an opportunity to purchase a product at a lower price, when the product they were planning to purchase is sold out, or when the cash register is crowded. be. If the determination in step S40 is negative, the process moves to step S44, but if the determination in step S40 is affirmative, the process moves to step S42.

In step S42, the control unit 26 issues an instruction to the second communication unit 25, and the line-of-sight image and article information (hereinafter referred to as first information) stored in the storage unit 23 and acquired immediately before the emotional change. ) is sent to the server 40. Note that "immediately before the emotional change" means, for example, several seconds before the emotional change occurs. Note that article information may not be acquired immediately before an emotion change. This is a case where the emotional change is not caused by holding the article 130. In this case, the control unit 26 may transmit the line-of-sight image to the server 40 as the first information.

Here, as the line-of-sight image acquired immediately before the emotional change, line-of-sight images as shown in FIGS. 7(a), 7(b), and 8 are assumed. The line-of-sight image in FIG. 7A is an image indicating that the user has encountered a time service, and specifically, is an image indicating that the user has seen a time service sticker attached to a product. Further, the line-of-sight image in FIG. 7B is an image when the product that the user was trying to purchase is sold out, and specifically, it is an image showing that the user has seen a notice indicating that the product is backordered. Furthermore, the line-of-sight image in FIG. 8 is an image showing that the user sees the cash register being crowded.

Note that the first information is transmitted to the server 40 with the basket ID added thereto. That is, the server 40 manages the first information using the basket ID.

After step S40 is negative or after the process of step S42 is executed, the control unit 26 determines in step S44 whether or not there is an input from the cashier terminal. Input from the cash register terminal in this case means, as described above, when the user finishes shopping and takes the cart 120 to the cash register, input from the cash register terminal is performed by communication between the cart device 20 and the cash register terminal. This means the input of shopping completion information transmitted to the cart device 20. If the determination in step S44 is negative, that is, if the user has not finished shopping, the process returns to step S32 and the processes and determinations in steps S32 to S44 are repeated. On the other hand, if the determination in step S44 is affirmative, the process moves to step S46.

When proceeding to step S46, the control unit 26 issues an instruction to the second communication unit 25, and sends all line-of-sight images, article information, and biological information (hereinafter referred to as second information) acquired after step S30 to the server 40. Send. That is, the control unit 26 transmits, via the second communication unit 25, the line-of-sight image obtained while the user is shopping, the article information of the article the user has in hand, and the information while the user is shopping. The emotion change information obtained is transmitted to the server 40. Note that the control unit 26 may transmit audio data acquired using the microphone 14 of the wearing device 10 to the server 40 while the user is shopping. Note that each piece of information transmitted from the control unit 26 is given the acquisition time of the information (obtained from the calendar unit 22) as a timestamp. Further, the second information is transmitted to the server 40 with the basket ID added thereto. That is, the server 40 manages the second information using the basket ID. After that, the control unit 26 ends all the processing in FIG.

As described above, when the control unit 26 executes the process shown in FIG. 5, the first information is sent from the cart device 20 to the server 40 at the timing when the user's emotions change while shopping. The second information is transmitted from the cart device 20 to the server 40 at the timing when the user finishes shopping.

(Processing of server 40)
FIG. 6 shows a flowchart of the processing of the server 40.

In the process of FIG. 6, the control unit 46 of the server 40 waits until there is an input from the basket device 20 in step S50. That is, the control unit 46 moves to step S52 when the first information or the second information is transmitted by executing step S42 or S46 of FIG. 5 in the car device 20.

When proceeding to step S52, the control unit 46 determines whether the input information is the first information. That is, the control unit 46 determines whether the line of sight information or article information acquired by the basket device 20 was input immediately before the user's emotion changed. If the determination here is affirmative, the process moves to step S54.

When proceeding to step S54, the control unit 46 executes emotional analysis based on the first information. Specifically, the control unit 46 uses the OCR unit 43 and the image recognition unit 44 to analyze the line-of-sight image acquired immediately before the emotion changed, or the article information acquired immediately before the emotion changed. Based on the analyzed results, the user's sentiment analysis (analysis of whether they are satisfied or dissatisfied) is performed.

For example, if the line-of-sight image is the image shown in FIG. 7(a), the control unit 46 can use the OCR unit 43 to detect the text “Morning market time service 20% off the displayed price.” . In this case, since words such as time service and 20% discount are included that are positive for shoppers, the control unit 46 analyzes that the user is happy and satisfied (positive state). Note that it is assumed that words that are positive for shoppers are stored in the storage unit 41.

In addition, when the line-of-sight image is the image shown in FIG. 7(b), the control unit 46 uses the OCR unit 43 to display a message saying, ``We are sorry. We are currently on backorder.The next arrival date is undecided.'' can be detected. In this case, since words that are negative for shoppers, such as sorry, backordered, and undecided, are included, the control unit 46 analyzes that the user is sad and unsatisfied (negative state). It is assumed that words that are negative for shoppers are stored in the storage unit 41.

In addition, when the line-of-sight image is the image shown in FIG. 8, the control unit 46 uses the image recognition unit 44 to detect that the cash register is crowded or that the person waiting at the cash register has a dissatisfied face. (not smiling) can be detected. In this case, the control unit 46 determines that the user is sad and unsatisfied (negative state).

Further, if the line-of-sight image includes an image of the employee's face, the control unit 46 determines that the user's emotion has changed due to the employee. In this case, the control unit 46 uses the image recognition unit 44 to recognize whether or not the employee's face is smiling. Note that the image recognition unit 44 determines whether the face image included in the line-of-sight image is an image of an employee's face or not, and whether or not it is a smiling face image, based on the employee's face image stored in the storage unit 41. Use data to make decisions. When the employee is not smiling, the control unit 46 determines that the user is sad or angry, which is an unsatisfactory state (negative state), and when the employee is smiling, the control unit 46 determines that the user is in an unsatisfied state (negative state). state (positive state).

Further, if the line-of-sight image captures the state in which the object is being held in the hand, the control unit 46 determines that a change in emotion has occurred due to seeing or touching the object. Note that the control unit 46 may be able to determine whether the user is happy or sad, for example, by also using the article information acquired by the basket device 20. For example, if the product information includes information such as limited quantity, it can be determined that the user is happy with the product. Furthermore, if it is determined from the product information that the product has expired, it can be determined that the user is sad.

Returning to FIG. 6, in the next step S56, the control unit 46 determines whether or not a response is necessary based on the emotion analysis result. Specifically, the control unit 46 determines that a response is necessary when the user is in an unsatisfactory state (in a negative state), such as when the user is sad or angry. If the determination at step S56 is negative, the process returns to step S50. On the other hand, if the determination in step S56 is affirmative, the process moves to step S58.

When proceeding to step S58, the control unit 46 executes response processing based on the emotion analysis result. For example, when the line-of-sight image is as shown in FIG. Send to 20. Then, the control unit 26 of the basket device 20 transmits various messages to the mounting device 10 via the first communication unit 24. In this case, in the mounting device 10, the control unit 17 uses the display unit 12 to display (project) various messages.

Further, for example, when the line-of-sight image is shown in FIG. 8, the control unit 46 transmits a message of apology for the crowded cash register to the control unit 17 of the mounting device 10 via the basket device 20. In this case, the control unit 17 displays (projects) an apology message on the display unit 12. Note that the control unit 46 may output a message apologizing for the congestion at the cash register using in-store broadcasting. Further, the control unit 46 may output a message (message for supporting the cash register) to the effect that the number of employees who work at the cash register should be supplemented using in-store broadcasting. Alternatively, the control unit 46 may output a cashier support message to a terminal held by an employee. The control unit 46 also informs the user of the situation where the cash register is crowded, such as the progress of the cash register (processing time per customer) and the proficiency level of the clerk in charge of the cash register (whether he is an apprentice or a veteran). It may also be possible to notify.

For example, if it is determined from the line of sight image that there is a problem with the employee's response, the control unit 46 sends a terminal held by another employee or store manager to communicate between the customer and the employee. Outputs a message that a problem has occurred. Note that if the cage device 20 is provided with a position detection device that can detect the position according to the communication sensitivity with the wireless LAN base station, the control unit 46 can troubleshoot based on the detection result of the position detection device. The location of the occurrence can be notified to other employees, store managers, etc.

Further, for example, if it is determined from the line of sight image that the customer is sad or angry about the item, the control unit 46 uses in-store broadcasts or a terminal held by the employee to notify the employee. You may also output a message to the effect that the person should respond directly. In this case as well, the control unit 46 may notify the employee of the customer's location.

Note that the first information may include audio data collected by the microphone 14 immediately before the emotional change. When the voice data is analyzed and the voice data includes the user's angry voice or sigh, or the employee's words of apology, the control unit 46 determines that the user is angry or in an unsatisfied state (a negative state). ).

Note that it is assumed that the pattern of the above-mentioned corresponding processing is determined in advance, and that the pattern is stored in the storage unit 41.

After the process of step S58 is completed as described above, the process returns to step S50.

On the other hand, if the determination in step S52 is negative, that is, if the information input from the car device 20 is the second information, the process moves to step S60.

When proceeding to step S60, the control unit 46 executes purchase behavior analysis based on the second information. In this case, the control unit 46 analyzes what the user looks at, what he picks up, and what he purchases while shopping. In other words, it can be said that it analyzes which products users do not view or pick up.

FIG. 9 shows the order of items picked up by a certain user from when he/she enters the store until he/she passes through the cash register. Furthermore, in FIG. 9, the items purchased by the user are underlined. Note that the control unit 46 can use purchase information obtained from the cash register terminal as information on the items purchased by the user.

In FIG. 9, the user picks up curry roux A and curry roux B, but ultimately purchases curry roux A. In this case, the control unit 46 refers to the line-of-sight image when picking up curry roux A and the line-of-sight image when picking up curry roux B. Analyze. For example, if price information is included for a long time in the area that the user is gazing at (a predetermined position (approximately the center) of the line-of-sight image), it is determined that Curry Roux A has been purchased based on the price. In addition, if the area the user was looking at included the spiciness display, package photo, ingredient display, and expiration date for a long time, it was possible to purchase Curry Roux A based on that information. I judge that. It is assumed that the control unit 46 uses the OCR unit 43 to extract whether or not the area that the user is looking at includes information such as price, spiciness, ingredients, expiration date, etc.

Note that the criteria for purchasing beef A and B may be determined in the same manner. For example, if the area that the user is looking at includes a display of content amount, a display of parts, a price per 100g, etc. for a long time, the control unit 46 may display beef based on such information. It is determined that B has been purchased.

In addition, by referring to data acquired by multiple users, the control unit 46 can aggregate information about other products that a user who purchased a certain product has compared and considered, and whether the user who purchased a certain product has purchased the product as a result of the comparison. It is possible to tally up what was the deciding factor for the product. Note that the analysis results of the judgment criteria at the time of purchase may be provided to the manufacturer or producer of each product. Manufacturers and producers can use the results of this analysis for future product development and product improvement.

Furthermore, by referring to the data in FIG. 9, the control unit 46 can tally the order in which the customers picked up the items. In this case, the control unit 46 can analyze trends in customer movement within the store by using the tally results. Thereby, the control unit 46 can determine appropriate placement of articles and shelves based on the movement of customers.

Furthermore, the control unit 46 calculates customer satisfaction and dissatisfaction levels by totaling the number of times and positions where the user is in a positive state and the number and positions where the user is in a negative state from entering the store to leaving the store. can be analyzed by location (by sales area).

Further, the control unit 46 may analyze a gaze image when the customer changes from a negative state to a positive state. As a result of analyzing the line-of-sight image, the control unit 46 determines that if the employee's face is included in the line-of-sight image during the change from a negative state to a positive state, the problem is resolved by the employee's response. It can be analyzed as follows. Furthermore, if a response process using the mounting device 10 or in-store broadcasting was performed during the change from a negative state to a positive state, it can be analyzed that the response process was appropriate.

As described above in detail, according to the first embodiment, the control unit 46 and the communication unit 42 of the server 40 input line-of-sight images and article information as operation information related to the user's purchase, and The detection results of the biosensor 21 are input as emotional information. This makes it possible for the control unit 46 to perform purchase-related analysis based on the line-of-sight image, article information, and detection results of the biosensor 21, and to acquire and provide useful information regarding the purchase. Therefore, usability for the user can be improved. In this case, by using a line-of-sight image taken from the user's side, it is possible to see the situation that the user is actually looking at, i.e., the items in front of the user's line of sight. Analysis that takes into account display, situation, etc. becomes possible.

Here, conventionally, when a user wearing a device equipped with an imaging unit 11 such as the wearing device 10 enters a store, images of the inside of the store and products are taken without permission. It could not be said that there was any benefit to either side. In contrast, in the first embodiment, instead of allowing the user to wear the mounting device 10 and enter the store, the store sends the image data captured by the imaging unit 11 via the basket device 20. I am planning to receive it. This allows the store to analyze the user's behavior and emotions leading up to the purchase of the product by analyzing the image data.

In addition, conventionally, shopping satisfaction was collected through surveys of users, but users now have to access the store's website, answer questions, and write down their answers on paper. It was quite annoying. In contrast, in the first embodiment, by linking the image data acquired by the wearing device 10 with the user's biometric information and providing it to the store, the user can unconsciously assess the user's satisfaction level. It is possible to communicate the level of dissatisfaction and dissatisfaction to the store side.

Further, in the first embodiment, a message to the user or a message to the employee is output based on the line-of-sight image, article information, and the detection result of the biosensor 21. This makes it possible to immediately respond to users who feel unsatisfied.

Further, in the first embodiment, the control unit 46 determines whether the user has held the article from the line-of-sight image and article information. This makes it possible for the control unit 46 to acquire and provide useful information regarding purchasing by analyzing the information that the user has held the item, the order in which the item was held, and the like.

Further, in the first embodiment, since the control unit 46 uses the detection results of the biosensor 21, it is possible to easily determine whether or not the user's emotions have changed. Furthermore, by analyzing the user's gaze image together with the detection results of the biosensor 21, it is possible to accurately determine whether the user's emotions have become positive or negative.

Furthermore, in the first embodiment, the mounting device 10 captures a line-of-sight image and transmits the captured line-of-sight image to the basket device 20 that is in contact with the user via human body communication. Thereby, the basket device 20 can acquire only the line-of-sight image of the user holding the basket 120. That is, the line-of-sight image is reliably acquired from the wearing device 10 of a user who has entered the store, and the line-of-sight image is acquired from the wearing device 10 of a person who has not entered the store (for example, a person passing near the store). It is possible to reliably prevent it from being put away.

Further, in the first embodiment, when there is a change in the detection result of the biosensor 21, that is, when there is a change in the user's emotion, the first information is transmitted to the server 40 (S42 ). Thereby, information that requires analysis can be transmitted to the server 40 at an appropriate timing.

In addition, although the case where the cage|basket apparatus 20 was equipped with the biosensor 21 was demonstrated in the said 1st Embodiment, it is not limited to this, and the mounting apparatus 10 may be equipped with a biosensor. In this case, a biosensor that comes into contact with a part of the face may be provided in a part of the frame of the mounting device 10 or in the vicinity of the frame. Further, a biosensor that can be worn on an earlobe may be connected to the mounting device 10.

Further, in the first embodiment, the case where the article device 30 is provided on the article 130 has been described. However, the present invention is not limited to this, and the article device 30 does not necessarily need to be provided on the article 130 if the article information is not required in the sentiment analysis in step S54 or the purchase behavior analysis in step S60. For example, if a line-of-sight image captures an object being held in the hand, the object held in the hand can be identified by performing image recognition.

Note that in the first embodiment, a case has been described in which the article device 30 performs human body communication with the basket device 20; however, the article device 30 is not limited to this, and the article device 30 performs human body communication with the mounting device 10. You may also do this. In this case, article information acquired by the mounting device 10 may be transmitted from the mounting device 10 to the basket device 20 while human body communication is established between the mounting device 10 and the basket device 20.

In the first embodiment, a case has been described in which the detection results of the biosensor 21 are used, but the present invention is not limited to this, and the detection results of the biosensor 21 may not be used. In this case, the control unit 46 estimates that the user is in a positive state, for example, when it determines that the user is staring at the mark in FIG. 7(a) for a predetermined period of time based on the gaze image. You can also do it. For example, if the control unit 46 determines that the user is staring at the notice board in FIG. 7(b) or the checkout waiting state in FIG. 7(c) based on the line-of-sight image, It may also be assumed that the state is in a certain state. In this case, the control unit 46 stores in advance an image pattern of a mark, a poster, or a situation that is predicted to result in a positive or negative state when viewed by the user, and compares the image pattern with the line-of-sight image. And it is sufficient.

Note that in the first embodiment, a case has been described in which the predetermined position of the line-of-sight image is regarded as the center position of the line-of-sight, but the present invention is not limited to this. For example, the line of sight includes an infrared irradiation section and an infrared reception section, and the line of sight is detected by emitting infrared rays from the infrared irradiation section to the eyeball, and receiving the infrared rays reflected by the eyeball at the infrared reception section. The detection section may be provided separately in the mounting device 10.

Note that the server 40 may aggregate customer satisfaction and dissatisfaction levels at a plurality of stores and publish the results as user evaluations for each store on one website. In this case, for example, the evaluation of each store can be made public using the number of stars, satisfaction score, etc.

《Second embodiment》
Next, the electronic device system 100' according to the second embodiment will be described in detail based on FIGS. 10 to 14.

FIG. 10 shows a block diagram of the configuration of an electronic device system 100'. In FIG. 10, the configurations of the electronic device system 100' that are different from the electronic device system 100 of the first embodiment are indicated by bold lines.

As shown in FIG. 10, in the electronic device system 100', the mounting device 10' includes the speaker 18, the basket device 20' includes the weight sensor 27, and the article device 30' includes the light emitting section 34. Note that the configuration of the server 40' is similar to the server 40 of the first embodiment.

The speaker 18 is used to provide audio information to the user wearing the wearing device 10'. As the speaker 18, in addition to earphones and headphones, it is possible to employ a speaker that has directivity and can provide audio information only to the user wearing the wearing device 10'. It is assumed that the storage section 15 of the mounting device 10' stores article information corresponding to the light emission pattern of the light emitting section 34 of the article device 30', which will be described later.

The weight sensor 27 is provided in a part of the car device 20' and is a sensor for detecting a change in the content of the car 120.

The light emitting unit 34 includes a light emitting element such as an LED (Light Emitting Diode), and emits light in a predetermined light emitting pattern for each article. Note that the light emission of the light emitting section 34 is controlled by the control section 33. The control unit 17 of the mounting device 10' estimates that the user has viewed an article corresponding to the light emission pattern when the image capture unit 11 captures an image of the light emission pattern within a predetermined range of the angle of view during video shooting. It is assumed that the timing of switching on/off the light emission by the light emitting unit 34 (flashing interval of the light emission pattern) is set in accordance with the frame rate of the imaging unit 11. Note that a visible light LED or an infrared LED may be used as the light emitting unit 34, and when an infrared LED is used, an infrared imaging element may be used in the imaging unit 11.

Next, the processing of the electronic device system 100' in the second embodiment will be described in detail along the flowcharts of FIGS. 11 to 13 and with appropriate reference to other drawings. In the second embodiment as well, a case will be described in which the electronic device system 100' is used in a supermarket.

(Processing of mounting device 10')
FIG. 11A shows a flowchart of the processing of the mounting device 10'. Note that in FIG. 11(a), steps that are different from the processing of the mounting device 10 of the first embodiment (FIG. 4(a)) are indicated by a thick line frame.

In the process of FIG. 11A, the control unit 17 of the mounting device 10' waits until human body communication with the basket device 20' is established in step S10. When the human body communication with the car device 20' is established, the control section 17 moves to step S110.

In step S110, the control unit 17 issues an instruction to the imaging unit 11 to perform imaging (video) from the user side.

Next, in step S112, the control unit 17 determines whether the light emission from the light emitting unit 34 has been imaged within a predetermined range of the angle of view. That is, it is determined whether the user has seen the article on which the article device 30' is installed. If the judgment in step S112 is negative, the process moves to step S116, but if the judgment in step S112 is affirmative, the process moves to step S114.

In step S114, the control unit 17 transmits information on the articles viewed by the user to the basket device 20'. Specifically, the control unit 17 acquires article information corresponding to the light emitting pattern detected within a predetermined range of the angle of view of the imaging unit 11 from the storage unit 15, and transmits it to the basket device 20' via the communication unit 16. It is transmitted to the first communication unit 24.

After the determination in step S112 is negative or after the process in step S114 is performed, the process proceeds to step S116, and the control unit 17 determines whether or not information has been received from the car device 20'. Note that the information received from the cart device 20' is information regarding goods and store services transmitted from the server 40'. If the judgment here is negative, the process moves to step S16, but if the judgment is affirmative, the process moves to step S118.

In step S118, the control unit 17 outputs the information received from the car device 20'. Note that details of the process in step S118 will be described later.

After the determination in step S116 is negative or after the process in step S118 is performed, the control unit 17 proceeds to step S16, and the control unit 17 determines that human body communication has been established with the cart device 20' that received the input from the cash register terminal. Determine whether or not. Here, input from the cash register terminal means that when the user finishes shopping and takes the cart 120 to the cash register, the input from the cash register terminal is transmitted to the cart device 20 by communication between the cart device 20' and the cash register terminal. ' indicates the input of shopping completion information. In other words, when human body communication is established between the basket device 20' and the mounting device 10' that received input from the cash register terminal, it means that the user's shopping has ended. Therefore, if the shopping has not been completed and the determination in step S16 is negative, the process returns to step S112 and the processes and determinations in steps S112 to S16 are repeated. On the other hand, if the user has finished shopping and the determination in step S16 is affirmative, the entire process in FIG. 11(a) ends.

(Processing of article device 30')
FIG. 11(b) shows a flowchart of the processing of the article device 30'. Note that in FIG. 11(b), steps that are different from the processing of the article device 30 of the first embodiment (FIG. 4(b)) are indicated by bold lines.

As shown in FIG. 11(b), in the article device 30', in step S120, the control section 33 causes the light emitting section 34 to emit light. In this case, the control section 33 causes the light emitting section 34 to emit light in a light emitting pattern corresponding to the article to which the article device 30' is installed. Other processing is the same as in FIG. 4(b).

(Processing of the cage device 20')
FIG. 12 shows a flowchart of the processing of the car device 20'. In the process of FIG. 12, in step S130, the control unit 26 waits until human body communication with the mounting device 10' is established. When human body communication with the mounting device 10' is established and the process moves to step S132, the control unit 26 determines whether or not article information has been received from the mounting device 10'.

If the determination in step S132 is affirmative, the process moves to step S134, where the control unit 26 transmits the article information received from the mounting device 10' to the server 40' as article information of the viewed article, and in step S136 to move to. On the other hand, if the determination in step S132 is negative, the process proceeds to step S136 without passing through step S134.

In step S136, the control unit 26 determines whether article information has been received from the article device 30'. If the determination in step S136 is affirmative, the process moves to step S138, where the control unit 26 transmits the article information received from the article device 30' to the server 40' as article information of the touched article, and in step S140 to move to. On the other hand, if the determination in step S136 is negative, the process proceeds to step S140 without passing through step S138.

In step S140, the control unit 26 determines whether the detected value of the weight sensor 27 has changed. Here, a case where the detected value of the weight sensor 27 changes means that an article is newly placed in the basket 120. Although the detection value of the weight sensor 27 also changes when an article in the basket 120 is taken out, for convenience of explanation, the explanation will be given assuming that no article is taken out from the basket 120. If the determination in step S140 is affirmative, the process moves to step S142, and the control unit 26 converts the item information of the item that was touched just before into the item information of the item that was placed in the cart, that is, the item that the user plans to purchase. The information is sent to the server 40'. On the other hand, if the determination in step S140 is negative, the process proceeds to step S144 without passing through step S142.

Proceeding to step S144, the control unit 26 determines whether information has been received from the server 40'. Note that the information received from the server 40' is information regarding goods and store services. If the determination in step S144 is affirmative, the process proceeds to step S146, where the control unit 26 transmits the information received from the server 40' to the mounting device 10', and the process proceeds to step S148. On the other hand, if the determination in step S144 is negative, the process proceeds to step S148 without passing through step S146.

When proceeding to step S148, the control unit 26 determines whether there is an input from the cashier terminal. Here, input from the cash register terminal means input of shopping completion information, as described above.

If the determination in step S148 is negative, the process moves to step S132, and the processes and determinations in steps S132 to S148 are repeated. However, if the determination in step S148 is affirmative, the control unit 26 performs the process shown in FIG. Finish all processing.

(Processing of server 40')
FIG. 13 shows a flowchart of the processing of the server 40'.

In the process of FIG. 13, in step S150, the control unit 46 waits until it receives article information from the basket device 20'. That is, at the stage where any one of the article information of the article viewed by the user, the article information of the article touched by the user, and the article information of the article placed in the basket 120 is input, the control unit 46 moves to step S152.

When proceeding to step S152, the control unit 46 updates the shopping DB stored in the storage unit 41. Here, the shopping DB has a data structure as shown in FIG. 14(a). As shown in FIG. 14(a), it has columns for item name, seen, touched, and put in cart. When the control unit 46 receives article information about an article that the user has viewed, it stores a circle in the "Seen" column of the article, and when it receives article information about an article that the user has touched, it stores information about the item that the user has touched. ○ is stored in the Added to Cart column, and when the item information of the item that the user has put in the cart is received, an ○ is stored in the Added to Cart column for the item.

Next, in step S154, the control unit 46 analyzes the shopping DB. For example, the control unit 46 refers to the shopping DB in FIG. 14(a), and if a predetermined time has elapsed since the beef A was held in hand (in a case where new product information is not received for a predetermined time), It can be analyzed that there is a possibility that the user needs information about an item (for example, beef B) that can be compared with beef A. In addition, since the control unit 46 can estimate that the user will make curry based on the items placed in the basket, the control unit 46 can also estimate the amount of beef needed to make curry (such as Xg for 4 dishes, Yg for 8 dishes, etc.). ) information that users may need. In addition, the control unit 46 may indicate that the user needs information about bargains on curry side dishes (Fukujinzuke, loin cutlet, desserts, etc.) and store campaign information (such as if you buy Z yen more, you can win a lottery ticket). It can be analyzed if there is.

Next, in step S156, the control unit 46 determines whether or not there is information to be notified to the user as a result of the analysis in step S154. If the judgment here is negative, the process returns to step S150, but if the judgment is affirmative, the process moves to step S158. In addition, in cases where the user touches the same item multiple times, or when the user touches an item similar to the item previously touched, the same information notified in the past may be used as the information notified to the user. Sometimes it is judged. Therefore, the control unit 46 may limit notification of information that is the same as information notified in the past, considering it not to be information that should be notified to the user. That is, the control unit 46 limits the information to be notified when the same item is touched or seen multiple times, or when an item similar to an item previously touched or seen is touched or seen. You can also do it. By doing so, it is possible to prevent the same user from being repeatedly notified of the same information.

Note that the control unit 46 may change the information to be notified depending on the number of times the same item is touched or the number of times the same item is viewed. For example, the control unit 46 sets, as the information to be notified, the information about the article itself the first time, the information about the article to be compared with the article at the second time, and the advantageous information related to the article at the third time. It may also be a thing. By doing so, it is possible to improve the possibility that a customer who is undecided about purchasing an item will purchase the item. Alternatively, the user may be notified of information that has not been notified to the user since the second time, depending on the time when the user touched the same item.

If the determination in step S156 is affirmative, the control unit 46 moves to step S158 and transmits information to be notified to the user to the basket device 20'. In addition, when the process of step S158 is performed, in the process of FIG. 12, the determination of step S144 is affirmed, and the process of step S146 (process of transmitting the information received from the server to the mounting device 10') is executed. . Moreover, when the process of step S146 in FIG. 12 is executed, the determination in step S116 of FIG. 11(a) is affirmed, and the process of step S118 (the process of outputting the received information) is executed. In step S118, the control unit 17 uses the display unit 12 to display information to be notified to the user, and uses the speaker 18 to output information to be notified to the user. Note that after step S158, the process returns to step S150, and the processes of steps S150 to S158 are repeated.

Note that the control unit 46 may perform various analyzes based on the shopping DB (for example, FIG. 14(b)) of the user who has completed shopping. In this case, the control unit 46 may analyze that curry roux A and curry roux B and beef A and beef B were compared and that curry roux B and beef B were purchased based on, for example, FIG. 14(b). can. The control unit 46 can also analyze the order of walking in the store, the order of purchases, and the like. Further, the control unit 46 can identify and analyze articles that the user merely looked at but did not purchase, and articles that the user picked up but did not purchase. The control unit 46 may provide the analysis results to the manufacturer, seller, etc. of the product.

As described above in detail, according to the second embodiment, the control unit 46 receives at least one of the following: when the user picks up the article, and when the article enters a predetermined range within the user's visual field. Accordingly, the article information of the article is input (S150: affirmative), and a notification regarding the input article information is issued (S158). Therefore, in the second embodiment, it is possible to notify the user of useful information related to the article that the user has touched or the article that the user has seen. Thereby, user-friendliness can be improved.

Furthermore, in the second embodiment, the control unit 46 inputs information on a product that the user plans to purchase, which is placed in the cart 120, as product information, and provides notification in accordance with the input product. Thereby, useful information related to the product that the user plans to purchase can be notified to the user. Furthermore, since the control unit 46 inputs information on the products to be purchased from the cart device 20' used during shopping, accurate information can be input as the information on the products to be purchased.

In addition, in the second embodiment, since information is notified to the user from the display unit 12 or the directional speaker 18 of the wearing device 10', each user can be notified of the information without other users knowing. Notifications can be made according to purchasing behavior.

Note that in the second embodiment, the control unit 46 may determine that the user is comparing and considering both products when the user is holding one product in his hand and looking at another product.

Although the second embodiment described above describes an example in which the electronic device system 100' is used in a supermarket, the present invention is not limited to this, and the electronic device system 100' may be used in a clothing store or the like. For example, when the user picks up a piece of clothing, the control unit 46 may provide information on whether the item is in stock, such as a different color or size, information on whether there is a similar item on sale, or information on whether or not to buy another item. By using the display unit 12 and speaker 18 to notify discount information such as a discount when purchasing more than 2,000 yen, and a discount when purchasing more than 2,000 yen, it is possible to notify a specific user (such as a member of a store) of discount information without the knowledge of other users. may be notified only to Mr. Further, the server 40' may transmit discount information, present information, and congratulatory messages to the wearing device 10' when a user who is a member of the store comes to the store on his or her birthday.

In the second embodiment, the control unit 17 has explained the case where the article information acquired by the mounting device 10' is transmitted to the server 40' via the basket device 20', but the present invention is not limited to this. It's not a thing. For example, if the mounting device 10' has a communication unit capable of wireless communication, information may be transmitted from the mounting device 10' to the server 40' by wireless communication.

Note that, in the second embodiment, a case has been described in which the article device 30' includes the light emitting section 34, but the present invention is not limited to this, and the article device 30' may not include the light emitting section 34. In this case, the control unit 17 may identify the article viewed by the user by analyzing the image captured by the imaging unit 11. For example, the control unit 17 may identify the article viewed by the user by recognizing characters included in the display (for example, characters on a product tag or POP advertisement).

Note that in the first and second embodiments described above, the control unit 46 may consider the type of user when analyzing the information. If the user is a member of the store, the type of user can be determined from registered information (name, date of birth, address, shopping history). In this case, the user's purchasing behavior may be analyzed by user's gender or age.

The embodiments described above are preferred embodiments of the present invention, and can be combined as appropriate. However, the present invention is not limited to this, and various modifications can be made without departing from the gist of the present invention.

10 Mounting device 11 Imaging section 12 Display section 16 Communication section 17 Control section 18 Speaker 20 Cart device 42 Communication section 46 Control section 110 Frame 120 Cart

Claims (1)

a detection unit that detects the user's entry into the store based on information from a device owned by the user;
an image recognition unit into which information captured in the store is input and which recognizes articles in the store;
an acquisition unit that acquires information regarding the article recognized by the image recognition unit;
An electronic equipment system comprising: a transmitting unit that transmits information for displaying information regarding the article acquired by the acquiring unit on the device owned by the user.

Images