JP2019032778A - Purchase assisting system - Google Patents

Abstract

To comprehensively analyze a consumer's willingness to purchase by grasping in-front-of-racks psychology and in-front-of-racks behavior of a user, and implement humanoid-robot based assistance on purchasing.SOLUTION: The purchase assisting system includes: a commodity display rack system including a sensing unit including a first sensor device for acquiring a face image of a consumer located in front of a commodity display rack, and a second sensor device for acquiring a movement of the consumer for commodities on the commodity display rack; an attendance robot that performs an utterance and/or an action regarding attendance to a consumer; and a management device that receives each sensor detection information acquired by the sensing unit and determines willingness of the consumer to purchase. The management device identifies an in-front-of-racks psychological pattern on the basis of an expression and line of sight of the consumer which are detected from the face image, identifies an in-front-of-racks behavioral pattern on the basis of the movement of the consumer for a commodity, and determines a purchase willingness pattern of the consumer corresponding to each of these patterns.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a purchasing support technology for comprehensively analyzing consumers' willingness to purchase merchandise display shelves and providing customer service by a humanoid robot.

  As described in Patent Documents 1 and 2, techniques have been proposed in which advertising information and the like are displayed to consumers to promote purchase motivation.

Japanese Patent Laying-Open No. 2015-88166 JP2015-156211A

  However, conventional techniques such as those described in Patent Documents 1 and 2 display the product according to the consumer's behavior by detecting the movement and line of sight of the product. Since only a part is detected, it is not possible to comprehensively grasp the consumer's psychology and willingness to purchase, and cannot provide information suitable for the consumer's psychology and behavior.

  Furthermore, in the technologies as described in Patent Documents 1 and 2, consumers can receive product display and effects that increase purchase motivation through the product display shelf, but in order to receive customer service, they actually ask the store clerk. There was a need. In this case, the consumer must search for a clerk when there is no clerk nearby. In addition, depending on the store clerk, the contents of the customer service may change, and there is a case in which the customer service according to the consumer's psychology and purchase intention cannot be received.

  Therefore, an object of the present invention is to provide a purchasing support system for providing a customer service by a humanoid robot by comprehensively analyzing consumers' willingness to purchase by grasping the user's pre-shelf psychology and pre-shelf behavior with respect to the merchandise display shelf. It is to provide.

(1) The purchase support system of the present invention acquires a commodity display shelf, a first sensor device that acquires a consumer's face image located in front of the shelf, and a consumer's movement with respect to the commodity displayed on the commodity display shelf. A product display shelf system including a sensing unit provided on a product display shelf, a customer service robot that performs speech and / or operations related to customer service, and the product via a network. A management device that is connected to the display shelf system and receives each sensor detection information acquired by the sensing unit to determine a consumer's willingness to purchase.

  The management device identifies a pre-shelf psychological pattern based on a consumer's facial expression and line of sight detected from the face image, and identifies a pre-shelf behavior pattern based on a consumer's movement with respect to the product An analysis control unit for discriminating a consumer's purchase willing pattern corresponding to the pre-shelf psychological pattern and the pre-shelf behavior pattern, and utterances and / or actions related to the customer service to be performed by the customer service robot for each of the plurality of purchase willing patterns. And a content control unit that selects the customer service control information according to the determined purchase willingness pattern and provides the selected customer service control information.

(2) In the above (1), the storage unit stores the customer service control information for each consumer's pre-shelf psychological pattern and / or the pre-shelf behavior pattern, and the content control unit includes: The customer service control information corresponding to the identified pre-shelf psychological pattern and / or the customer service control information corresponding to the identified pre-shelf behavior pattern is selected, and the pre-shelf psychological pattern and / or the pre-shelf behavior pattern It is possible to provide the customer service control information according to the situation.

(3) In the above (1) or (2), the customer service control information may include utterance information in a plurality of languages, and the management device collects the service via the customer service robot or the product display shelf system. The language used by the consumer is determined based on the uttered voice of the consumer, and the content control unit selects and provides the customer service control information including the utterance information according to the determined language used. Can be configured.

(4) In the above (1) to (3), the analysis control unit extracts a consumer attribute including a sex and an age group of a consumer from the face image based on a predetermined face recognition process, and the face image And the user profile information associated with the consumer attribute is generated and stored in the storage unit, and the user profile information stored in the storage unit is specified by performing pattern matching based on the face image, The user profile information in which the store visit history to the store where the display shelf is installed and the purchase history of the product displayed on the product display shelf acquired from a predetermined sales management system for managing the store sales history are specified Can be stored in association with Then, the content control unit determines the customer service control information based on any one or a combination of the gender, the age group, the store visit history, and the purchase history of the consumer based on the user profile information. It is possible to select and provide the selected customer service control information.

(5) In the above (1) to (3), the analysis control unit generates user profile information associated with the face image, stores the user profile information in the storage unit, and performs pattern matching based on the face image. The user profile information stored in the storage unit is specified, and a purchase history of a product displayed on a product display shelf acquired from a predetermined sales management system that manages a sales history of a store is associated with the user profile information Can be remembered. And, based on the result of the process of transmitting the customer service control information for controlling the humanoid robot and the purchase history, the result of the transmission process that led to the purchase of goods is the consumer's pre-shelf psychological pattern, It can comprise so that the learning control part evaluated for every said before-shelf action pattern and the said purchase willingness may be provided.

(6) In the above (5), the learning control unit may transmit the product according to the language used based on the utterance voice of the consumer collected via the customer service robot or the product display shelf system. Can be configured to evaluate each of the consumer's pre-shelf psychological pattern, the pre-shelf behavior pattern, and the purchase intention.

(7) In the above (1) to (5), the sensing unit may further include a third sensor device that detects the movement of a product displayed on a product display shelf, and the analysis control unit includes the first sensor unit. The sensor detection information of the two-sensor device and the third sensor device can be combined to specify whether the consumer has picked up a specific product as the pre-shelf behavior pattern.

(8) In the above (1) to (7), the customer service robot further includes a fourth sensor device corresponding to the first sensor device and a fifth sensor device corresponding to the second sensor device. The management device can replace each sensor detection information acquired by the sensing unit directly with each sensor detection information acquired by the fourth sensor device and the fifth sensor device from the service robot, or The product can be received via the merchandise display shelf system connected to the customer service robot, and the consumer's willingness to purchase can be determined.

  According to the present invention, it is possible to comprehensively analyze the consumer's willingness to purchase by grasping the user's pre-shelf psychology and pre-shelf behavior with respect to the merchandise display shelf, and to perform purchase support with customer service by a humanoid robot. .

It is a block diagram of the purchase assistance system which provides the customer service by the humanoid robot of 1st Embodiment. 1 is a configuration block diagram of a merchandise display shelf system, a humanoid robot, and a management device that constitute a purchase support system of a first embodiment. It is a figure for demonstrating the sensing unit function, the analytic unit function, the output unit function, and the customer service function of a humanoid robot which are contained in the purchase support method of 1st Embodiment. It is a figure which shows an example of the user profile of 1st Embodiment. It is a figure which shows the delivery example based on the user action (consumer action) of 1st Embodiment, and the example of control of speech / motion of a humanoid robot. It is a figure which shows an example of the psychological analysis before shelf and the behavior analysis before shelf of 1st Embodiment. It is a figure which shows an example of the delivery content pattern with respect to the purchase willingness analysis result based on the shelf front psychology and the shelf front action of 1st Embodiment, and a purchase willingness analysis result. It is a figure which shows an example of the speech / action pattern with respect to the purchase willingness analysis result based on the shelf front psychology and the shelf front action of 1st Embodiment, and a purchase willingness analysis result. It is a figure which shows an example of control of the humanoid robot by the matching in the 1st phase of 1st Embodiment. It is a figure which shows an example of control of the humanoid robot by the matching in the 2nd phase of 1st Embodiment. It is a figure which shows an example of control of the humanoid robot by the matching in the 3rd phase of 1st Embodiment. It is a figure which shows the processing flow of the goods display shelf system and management apparatus of 1st Embodiment. It is a figure which shows the processing flow of the whole purchase assistance system in the 1st, 2nd phase of 1st Embodiment. It is a figure which shows the processing flow of the whole purchase assistance system in the 3rd phase of 1st Embodiment.

  Hereinafter, embodiments will be described with reference to the drawings.

(First embodiment)
1 to 13 are diagrams showing a first embodiment. FIG. 1 is a configuration diagram of a purchase support system (hereinafter referred to as a purchase support system) using a humanoid robot via a product display shelf according to the present embodiment. The purchase support system includes a product display shelf system 100, a humanoid robot (corresponding to a customer service robot) 200, and a management device (corresponding to a purchase support management device) 300. The commodity display shelf system 100 and the humanoid robot 200 are connected by wire or wirelessly. The merchandise display shelf system 100 and the management device 300 are connected by wire or wireless via a network such as an IP network.

  As shown in FIG. 1, the product display shelf system 100 includes a shelf main body 1 that extends in the height (Z direction) and product display portions 2A, 2B, and 2C that extend forward from the shelf main body 1. Based product display shelves. The plurality of merchandise display portions 2A, 2B, 2C are spaced apart from each other in the height direction at a predetermined interval. The product display parts 2A, 2B, 2C have a predetermined lateral width (X direction) and depth (Y direction), and protrude from the shelf main body part 1 in the Y direction. The product display parts 2A, 2B, 2C can also be provided such that the upper surfaces of the product display parts 2A, 2B, 2C are inclined downward with respect to the height direction. The product display shelf shown in FIG. 1 is an example, and may be a shelf in which a product display area having at least a predetermined width and depth is formed.

  As illustrated in FIG. 1, the commodity display shelf system 100 includes a camera 110A (corresponding to a first sensor device), a motion sensor 110B (corresponding to a second sensor device), and a capacitance sensor 110C (third sensor device). A display unit 120A, a monitor unit 120B, a speaker 120C, an incense generator 120D, and an output unit 120, and a control unit 130.

  The camera 110 </ b> A is a photographing device that photographs a user (consumer) in front of the shelf, and photographs a user's still image or moving image. The motion sensor 110B is a non-contact type motion sensor including a distance image sensor using infrared rays or a laser, and can detect a distance to an object in the image. As the distance image sensor, there are a “Light Coding (pattern irradiation method)” method and a “TOF (Time of Flight)” method, and the distance is measured by irradiating infrared rays or a laser and reading the reflection. In combination with a camera that detects the position of an object separately from the distance image sensor, for example, the movement of a person is detected based on the position and distance.

  The camera 110 </ b> A and the motion sensor 110 </ b> B are provided on the upper part of the product display shelf in order to photograph at least the upper body of the user located in front of the shelf. On the other hand, the capacitance sensor 110C is provided on the upper surface on which the product is placed in the product display units 2A, 2B, 2C. The capacitance sensor 110C is a capacitance type proximity sensor, and a product is placed on the capacitance sensor 110C. When the user picks up the product placed on the hand, the capacitance changes, so that the presence or absence of the product placed on the product display shelf can be detected.

  In the present embodiment, the combination of the motion sensor 110B and the capacitance sensor 110C makes it possible to accurately grasp that the user has picked up the product as a pre-shelf action. It is also possible to grasp that the user has picked up the product using any one of the camera 110A, the motion sensor 110B, and the capacitance sensor 110C. For example, the product picked up in the user's image taken by the camera 110A is detected, the product is detected as an object by the motion sensor 110B, and the action of the product moving with the user is detected. By detecting the presence or absence of a product at 110C, it is possible to grasp that the user has picked up the product in front of the shelf. It is also possible to grasp that the user has picked up the product by arbitrarily combining the sensor detection information of the sensing unit 110.

  Next, the display device 120 </ b> A is provided on the upper part of the merchandise display shelf, and displays content distributed (transmitted) from the management device 300. On the other hand, the monitor unit 120B is, for example, a unit in which a plurality of small display devices are arranged side by side. As shown in FIG. 1, the monitor unit 120 </ b> B is provided on the end surface of the product display portions 2 </ b> A, 2 </ b> B, and 2 </ b> C and is disposed facing the user located in front of the shelf.

  The speaker 120C outputs reproduced audio data (music, BGM, sound effects, etc.). The audio data includes audio associated with content (video, image, text information, advertisement information) displayed on the display device 120A and the monitor unit 120B and audio unrelated to the content. The incense generating device 120D outputs a scent of cosmetics and perfume, a scent of food, and the like. For example, a fragrance (raw material) that is a source of scent can be installed and the scent can be sent out by a blower such as a fan. A plurality of fragrances (raw materials) may be installed in advance, and for example, a plurality of scents may be switched and sent in response to the analysis results of the user's pre-shelf psychology, pre-shelf behavior, and purchase intention.

  The control unit 130 is connected to the management apparatus 300 via a network, transmits the sensor detection information of the sensing unit 110 to the management apparatus 300, and receives content (including output control information) provided from the management apparatus 300.

  The humanoid robot 200 has a head 201, an upper body 202 including a right arm 202A and a left arm 202B, and a lower body 203, each corresponding to each part of the human body. For example, the right arm portion 202A and the left arm portion 202B each include a right hand portion or a left hand portion, and can point to or grasp an object. Further, the lower body 203 includes, for example, a traveling mechanism and a walking mechanism, and can move the main body of the humanoid robot 200.

  The display input device 274 is, for example, a touch panel display, and can display various types of information to the user and accept information input by the user. The display input device 274 can be installed on the upper body 202. Further, the humanoid robot 200 can be provided with the display input device 274 with the right arm portion 202A and the left arm portion 202B.

  The humanoid robot 200 can speak. For example, it is possible to introduce products for increasing the user's willingness to purchase by speaking. The humanoid robot 200 can perform gestures like a human by operating each unit. The humanoid robot 200 can perform sales promotion activities (customer interaction behavior) by speaking and / or movement at the store. The sales promotion activities include, for example, product explanations and guidance, customer calls, and the like.

  Hereinafter, performing customer service to a consumer by utterance and / or action may be described as “customer service behavior”. In addition, customer service behavior by speech and / or action is defined in the customer service control information. The customer service control information is obtained by associating the utterance information defining the utterance voice and the operation information defining the operation content. The humanoid robot 200 performs customer service based on the customer service control information. Although the customer service behavior may be described as “speech / motion”, the case where the humanoid robot 200 performs both speech and motion and the case where at least one of them is performed is included.

  The humanoid robot 200 is connected to the control unit 130 of the commodity display shelf system 100 by wire or wirelessly. As a result, the humanoid robot 200 is connected to the management apparatus 300 via the merchandise display shelf 100. The humanoid robot 200 transmits user language information, which will be described later, to the management apparatus 300 via the product display shelf 100. The humanoid robot 200 receives customer service control information via the merchandise display shelf 100.

  In other words, the control unit 130 receives user language information from the humanoid robot 200 and transmits it to the management apparatus 300. In addition, the control unit 130 receives the customer service control information provided from the management device 300 and transmits the customer service control information to the humanoid robot 200.

  FIG. 2 is a configuration block diagram of the commodity display shelf system 100, the humanoid robot 200, and the management device 300 that constitute the purchase support system. Note that the management device 300 of this embodiment can be connected to one or a plurality of product display shelf systems 100 via a network.

  The commodity display shelf system 100 includes the sensing unit 110, the output unit 120, and the control unit 130 as described above, and the control unit 130 includes the control device 131, the communication device 132, and the storage device 133. The communication device 132 performs communication control with the management device 300 and also performs communication control with the humanoid robot 200. The storage device 133 stores information necessary for control by the control unit 130, various information transmitted from the humanoid robot 200, and various information transmitted from the management device 300.

  The sensor control unit 131A of the control device 131 controls each sensor device of the sensing unit 110 and collects sensor detection information output from each sensor device. The sensor detection information is transmitted to the management device 300 via the communication device 132.

  The output control unit 131B receives content and output control information transmitted from the management device 300, and performs output control on each output device of the output unit 120. Further, the output control unit 131B includes a robot cooperation unit 131C. The robot cooperation unit 131 </ b> C receives the customer service control information provided from the management device 300 and performs output control for outputting to the humanoid robot 200.

  The humanoid robot 200 includes a robot system 220. The robot system 220 includes a control device 230, a communication device 240, a storage device 250, a sensor device (corresponding to a fifth sensor device) 260, and an input / output unit 270. The communication device 240 performs communication control with the merchandise display shelf system 100 by wired or wireless communication means.

  The storage device 250 stores information necessary for control by the control device 230, various information transmitted from the product display shelf system 100, and various information provided from the management device 300 via the product display shelf system 100. Further, the storage device 250 stores, for example, a language database (described later) in which speech (utterance information) output in the speech of the humanoid robot 200 is stored for each language used.

  The sensor device 260 has the same function as the motion sensor 110B, for example, and detects the position and movement of the user with respect to the humanoid robot 200. Thereby, the humanoid robot 200 can grasp the position and movement of the user and serve customers. Further, the position and movement of the user with respect to the merchandise display shelf can be detected using the sensor device 260 in the same manner as the motion sensor 110B.

  The input / output unit 270 includes a speaker 271, a microphone 272, a camera 273 (corresponding to a fourth sensor device), and a display input device 274. The speaker 271 outputs a voice when the humanoid robot 200 speaks. The microphone 272 collects sound generated by the user's utterance. The camera 273 is a photographing device that photographs the user in front of the humanoid robot 200 and has the same function as the camera 110A.

  The sensor control unit 231 of the control device 230 controls the camera 273 and the sensor device 260 and collects sensor detection information output from each. The sensor detection information is transmitted from the communication device 240 to the management device 300 via the product display shelf system 100.

  The robot controller 232 of the control device 230 controls the customer service behavior of the humanoid robot 200 based on the customer service control information provided from the management device 300 via the merchandise display shelf system 100. In addition, the robot control unit 232 executes voice recognition processing for specifying the user's language based on the user's voice collected by the microphone 272. When the user language is specified, user language information, which is information indicating the user language, is transmitted from the communication device 240 to the management device 300 via the product display shelf system 100. Thereby, as will be described later, the customer service control information including the voice corresponding to the specified language used by the user is provided from the management device 300. The robot control unit 232 controls the utterance of the humanoid robot 200 by outputting a voice corresponding to the user's language based on the customer service control information, and controls the operation of the humanoid robot 200 corresponding to the output voice. .

  The management device 300 includes a communication device 310, a control device 320, and a storage device 330. The communication device 310 performs communication control with the merchandise display shelf system 100. The storage device 330 stores information necessary for control by the control device 320, various information transmitted from the product display shelf system 100, and various information transmitted from the humanoid robot 200 via the product display shelf system 100.

  The control device 320 includes an analysis control unit 321, a content control unit 322, and a learning control unit 323. FIG. 3 is a diagram for explaining a sensing unit function, an analytic unit function, an output unit function, and a customer service function (speech / operation function) included in the purchase support method of the present embodiment. The sensing unit function and the output unit function are realized by the merchandise display shelf system 100, and the analytic unit function is realized by the management device 300. The customer service function is realized by the humanoid robot 200.

  As illustrated in FIG. 3, the analysis control unit 321 performs each process of a user identification process, a pre-shelf psychological analysis process, a pre-shelf behavior analysis process, and a purchase motivation analysis process.

  The user identification process is a user profile registration / update process, which performs a face recognition process using the user image data captured by the camera 110A to identify (register) the user. For example, the user's gender and age group can be analyzed by face recognition technology and registered as a user profile. At this time, it is possible to determine whether the user is a new user or a repeater by performing pattern matching with the face image of the already registered user profile. In the case of a repeater, store visit history such as the number of visits is registered and updated as a user profile.

  In addition, in the user specifying process, in addition to the face image data, for example, image data obtained by photographing the whole body of the user or a plurality of users is used, and clothes, body shape, height, presence / absence of wearing glasses, group Appearance features such as composition can be analyzed and registered and updated as a user profile. Furthermore, in the user specifying process, a voice recognition process can be performed based on the user's voice collected by the microphone 272, and the user's language can be specified.

  For example, the voice recognition processing is performed based on a language database stored in advance in the customer service control information stored in the storage device 250 of the humanoid robot 200 and stored in the language database. The user's language can be specified (discriminated) by matching the user's voice. For example, the language database "Hello", "Hello (Hello)", and "(" 尓 "in Hitohen) good (Nihao)", a plurality of speech using different languages in the same sense (or, based on a speech Text information) is prepared, and the user's voice is matched from among them, and the language used is specified by selecting the voice closer to the user's voice. The specified user language (user language information) can be registered as a user profile.

  Further, by specifying the language used by the voice recognition process, as described later, customer service control information including output voice (utterance information) corresponding to the specified language used is selected, and customer service control corresponding to the language used is performed. It can be carried out.

  FIG. 4 is a diagram illustrating an example of a user profile. The user profile includes user face image data, gender, age group, language (user language information), appearance characteristics, facial expressions, line of sight, product information, stay time, number of visits, purchase history, and the like. The analysis control unit 321 grasps the gender, age group, facial expression, and line of sight of the user using face recognition technology, and registers them in the user profile. Moreover, the analysis control part 321 can grasp | ascertain the use language of the identified user as mentioned above, and can register it to a user profile. Furthermore, it is possible to register product picking information based on behavior analysis in front of the shelves described later, staying time in front of the shelves detected using the sensing unit 110 as a human sensor, and the number of visits described above.

  A known method can be applied to the face recognition technique executed by the analysis control unit 321. For example, gender, age group, facial expression, etc. can be detected by analyzing a face image using feature points and patterns. Further, the position of the eyes (iris, cornea, pupil) of the user's face image taken with a visible camera or infrared camera is detected, and displayed on the product display units 2A, 2B, 2C based on the detected eye position. It is possible to grasp a specific product viewed by the selected user. In addition, each product displayed on the product display units 2A, 2B, and 2C and position information of the product can be stored in the storage device 330 in advance.

  The purchase history can be acquired from a sales management system (POS system). For example, the face image of the user is acquired at the time of checkout settlement, and the sales management system sets the checkout settlement result (information for identifying the purchased product) and the face image of the user as a set via the product display shelf system 100 or directly. To the management device 300. The user's face image may be transmitted to the management apparatus 300 separately from the sales management system. For example, the face image of the user can be taken at the cash register and transmitted to the management device 300 in synchronization with the timing at which the POS information is transmitted from the cash register to the sales management system at the checkout. The management apparatus 300 can associate the two based on the time stamp of the POS information acquired from the sales management system and the acquired time stamp of the face image of the user.

  The analysis control unit 321 can search the user profile using the user's face image associated with the cashier settlement result (POS information), and store the purchased product as a purchase history in the corresponding user profile.

  The pre-shelf psychological analysis process analyzes the user's pre-shelf psychology for the product of the product display shelf system 100 based on the user's facial expression and line-of-sight movement identified by the face recognition process described above.

  The pre-shelf behavior analysis process combines the user behavior detected by the motion sensor 110B and the movement of the product detected by the capacitance sensor 110C, and the user picks up the product (information on the product hand). ). The information collected by the merchandise hand is grasped as a pre-shelf action. Further, as described above, the position information of the product placed on the product display shelf system 100 can be stored in the storage device 330 in advance, and the position information of the product detected by the capacitance sensor 110C is also stored in advance. It is remembered. For this reason, it is possible to specify which product is the product actually picked up. As the information on the product hand registered in the user profile, for example, it is possible to grasp the action taken on the hand for a specific product such as “I picked up the product A”.

  The purchase motivation analysis process is a process of analyzing the user's purchase motivation using the pre-shelf psychological analysis result by the pre-shelf psychological analysis process and the pre-shelf behavior analysis result by the pre-shelf behavior analysis process. The details of each analysis process of the pre-shelf psychology, the pre-shelf behavior, and the purchase motivation will be described later.

  FIG. 5 is a diagram illustrating an example of content distribution control based on user behavior (consumer behavior) and customer service control of the humanoid robot 200. Control of content distribution control and customer service control of the humanoid robot 200 proceeds in stages of the first to third phases. The first phase is a stage in which the user who came in front of the shelf is specified and control is performed based on the user specifying result (user characteristic). The second phase is a stage in which the user's pre-shelf psychology and pre-shelf behavior are analyzed, and control is performed based on the pre-shelf psychological analysis result and the pre-shelf behavior analysis result. The third phase is a stage in which control is performed based on the purchase intention analysis result.

  The content control unit 322 uses the user characteristics, the pre-shelf psychological analysis result, the pre-shelf behavior analysis result, and the purchase motivation analysis result in combination to provide the content and output control information to the product display shelf system 100, and a humanoid robot. The customer service control information provided to 200 is extracted and transmitted.

  As shown in FIG. 5, when the user comes in front of the merchandise display shelf system 100 (in front of the shelf), it is grasped that the user exists in front of the shelf based on the sensor detection information of the motion sensor 110B. At this time, a user's face image is image | photographed with camera 110A, a user's facial expression and eyes | visual_axis are grasped | ascertained, specifying a user using face recognition information. In addition, as described above, the appearance characteristics of the user are grasped by photographing with the camera 110A. On the other hand, the humanoid robot 200 promotes the user's utterance by an utterance promoting action (described later), and grasps the language used by the user who uttered the speech using speech recognition processing.

  In the first phase, the user profile is specified by matching based on the face image of the user. When a user profile exists, as a known user, a product candidate is extracted using the purchase history of the user profile, and content (product information) and customer service control information related to the extracted product candidate are selected. The content is transmitted to the product display shelf system 100, and the customer service control information is transmitted to the humanoid robot 200 via the product display shelf system 100.

  On the other hand, if the user profile does not exist, it is assumed that the user is a new user, and the gender, age group, appearance feature, and user who is grasped by the humanoid robot 200 are grasped from the captured image (including the user's face image and whole body image). Product candidates suitable for the language used are extracted, and contents and customer service control information related to the extracted product candidates are selected. Similarly to the above, the content is transmitted to the product display shelf system 100, and the customer service control information is transmitted to the humanoid robot 200 via the product display shelf system 100.

  As described above, the content based on the specified user profile and the customer service control information are transmitted by any one of the gender, age group, appearance feature, language, store visit history, and purchase history of the consumer (user) based on the user profile. Based on one or a combination of two or more, the content and the customer service control information are selected, and the content is displayed on the output unit, or the transmission process for controlling the customer service behavior of the humanoid robot 200 is performed. be able to. For example, for female repeaters who have visited the store more than 5 times, offer preferential reservation information for limited items or offer discounted deals over female repeaters and male users who have visited less than 5 times You can do it.

  The storage unit 330 of the management apparatus 300 stores information provided to the product display shelf system 100 and information provided to the humanoid robot 200, and is provided as content and customer service control information, respectively. For example, by selecting candidate products for the purchase history in advance, selecting candidate products for each gender, age group, appearance feature, and language, the content and customer service control information Can be accumulated.

  In the second phase, based on the user's line-of-sight information, for example, it can be grasped that the product B is being viewed. Therefore, the product viewed by the user can be specified based on the line-of-sight information, and the product description information and advertisement information of the specified product B can be extracted and transmitted to the product display shelf system 100. At this time, output (display) control information is sent to the product display shelf system 100 so as to display an effect (emphasis or decoration) in the display of product description information and advertisement information related to the product B actually viewed by the user. Can be sent.

  In the second phase, it is possible to grasp whether the product is picked up as the user action before the shelf. Based on the information picked up by the product hand, the content related to the product A picked up is extracted and transmitted to the product display shelf system 100, or the profit information related to the product A picked up is extracted and the product is extracted. Or transmitted to the display shelf system 100. The profit information is, for example, information that recommends a product C that can be purchased at the same time as the product A (a discount can be applied or a combination effect is high).

  In the third phase, the user's willingness to purchase is analyzed based on the pre-shelf psychology by facial expression and line of sight, and the pre-shelf behavior by the product information, and the content related to the analyzed purchasing motivation is extracted and the product display shelf To system 100. Content distribution and display control based on the user's willingness to purchase, for example, switching the displayed content to content that encourages the willingness to purchase, processing the displayed content with effects, and playing music and audio Or additional display of deals.

  As described above, in this embodiment, based on the user action in front of the shelf of the product display shelf system 100, not only the advertisement and the profit information are displayed, but the user's mentality in front of the shelf and the action in front of the shelf are grasped, and the user Detailed content distribution and display control suitable for customers' willingness to purchase.

  One sensor device is insufficient to comprehensively grasp the user's pre-shelf psychology and willingness to purchase, and the product display shelf system 100 of this embodiment includes a sensing unit 110 including a plurality of sensor devices. And comprehensively grasp the user's pre-shelf psychology and willingness to purchase.

  The purchase support system according to the present embodiment is not dependent on a single sensor device, but is a sensing unit 110 including a plurality of sensor devices. The consumer attribute (profile), pre-shelf psychology (psychological situation), and pre-shelf behavior are determined. It is grasped and transmitted to the management apparatus 300, and the analytic unit of the management apparatus 300 analyzes the psychological state and behavior of the user in front of the shelves of the product display shelf system 100 to comprehensively extract the user's willingness to purchase. Then, it is possible to provide the content that prompts the purchase to the purchase motivation comprehensively grasped from the user's psychological state and behavior through the output unit 120 of the merchandise display shelf system 100 to promote the purchase behavior of the user. it can.

  On the other hand, the robot control unit 232 controls the humanoid robot 200 based on the customer service control information provided from the management device 300 via the product display shelf system 100. The customer service control information is information for controlling the speech / motion of the humanoid robot 200 corresponding to each of the user characteristics, the pre-shelf psychological analysis result, the pre-shelf behavior analysis result, and the purchase intention analysis result. It is extracted from the customer service control information database stored in the storage device 330.

  In the first phase, when it is determined that the user exists in front of the shelf, the humanoid robot 200 performs an utterance promotion action for causing the user to utter. In the utterance promotion action, for example, the humanoid robot 200 utters “Please speak” to the user and waits for the user's utterance.

  When the user speaks, the humanoid robot 200 identifies the language used by the user by voice recognition processing, and speaks with a voice corresponding to the language used by the user. The language used by the user may be specified by a method other than the voice recognition process. For example, the user's language may be specified by displaying a list of languages used on the display input device 274 and allowing the user to select.

  The humanoid robot 200 provides user language information to the management apparatus 300 via the merchandise display shelf system 100. Thereby, the management apparatus 300 grasps the language in addition to the user's sex, age group, and appearance characteristics, and provides the humanoid robot 200 with the customer service control information extracted as described above. Thereby, the humanoid robot 200 can perform an utterance / motion corresponding to the user characteristic.

  The content of the operation included in the customer service control information can be set for each language used. For example, when the specified language is Japanese, the action when the humanoid robot 200 guides the user is set to the action of moving the wrist with the back of the hand facing up, while the specified language is English. In some cases, it can be set to move the wrist with the palm up. In this way, depending on the specified language, different actions can be set even for actions that represent the same meaning, and actions according to the user can be performed.

  In the second phase, the humanoid robot 200 performs an utterance / motion corresponding to the psychoanalysis result in front of the shelf based on the user's facial expression and line-of-sight information. For example, the customer service control information for explaining the product B by the humanoid robot 200 is extracted together with the product description information and advertisement information of the product B specified based on the line-of-sight information as described above, and the extracted customer service control information Can be provided to the humanoid robot 200. In this way, the humanoid robot 200 responds to the pre-shelf psychological analysis result in conjunction with the product description information of the product B being highlighted on the product display shelf 100 based on the pre-shelf psychological analysis result. Product B is introduced by utterance / motion.

  Further, in the second phase, the humanoid robot 200 performs an utterance / action corresponding to the behavior analysis result in front of the shelf based on the information taken by the user's product hand. For example, as described above, content related to the product A taken by the user and profit information related to the product A taken by the user are extracted, and the product A is explained by the humanoid robot 200. It is possible to extract customer service control information for transmitting profit information related to the customer information and provide the extracted customer service control information to the humanoid robot 200. In this way, based on the result of the pre-shelf behavior analysis, the content of the product A is displayed on the product display shelf 100, or the discount information related to the product A taken in hand is displayed. The humanoid robot 200 introduces the product A and advertises the profit information with the utterance / action corresponding to the behavior analysis result in front of the shelf.

  In the third phase, the humanoid robot 200 performs an utterance / motion corresponding to the purchase will analysis result based on the pre-shelf psychology based on the facial expression and line-of-sight information and the pre-shelf behavior based on the product information. For example, together with the extraction of content related to the purchase motivation analyzed as described above, the humanoid robot 200 communicates discount information that encourages purchases and utterances / motions that sellouts continue at other stores. Customer service control information can be extracted, and the extracted customer service control information can be provided to the humanoid robot 200. In this manner, in conjunction with the display of the content that boosts the user's purchase intent on the product display shelf based on the purchase will analysis result, the humanoid robot 200 utters the speech / corresponding to the purchase will analysis result. Drive user purchases with actions.

  As described above, in the present embodiment, the customer service is provided based on the customer's pre-shelf psychology, the pre-shelf behavior, and the customer service control information according to the purchase motivation, thereby providing the customer service by the humanoid robot. Can promote action.

  FIG. 6 is a diagram illustrating an example of the pre-shelf psychological analysis and the pre-shelf behavior analysis of the present embodiment. The pre-shelf psychological analysis performs user facial expression analysis and line-of-sight analysis using a photographed image, and analyzes the user's pre-shelf psychology from both the user's facial expression and line of sight. In facial expression analysis, for example, feature points of facial expressions such as bright, confused, and no expression are patterned in advance to determine whether the user's facial expression is “bright”, “confused”, or “no expression”. In addition, various facial expression analysis such as “joy”, “surprise”, “anger”, “sadness” can also be performed.

  In the line-of-sight analysis, if the position of the eyes (iris, cornea, pupil) of the user's face image detected as described above is within a predetermined range for a predetermined time, the line of sight is concentrated on one point. Or if the position fluctuates up, down, left, and right for a predetermined time, it can be determined that it is dispersed.

  The pre-shelf psychology is pre-patterned by a combination of the facial expression analysis result and the line-of-sight analysis result, and the pre-shelf psychological patterns A to E are stored in the storage device 330 in advance as shown in FIG. Yes. The analysis control unit 321 uses the facial expression analysis result and the line-of-sight analysis result to extract a pre-shelf psychological pattern that is set in advance and identifies the user's pre-shelf psychology.

  As described above, the pre-shelf behavior analysis is performed by combining the user behavior detected by the motion sensor 110B and the movement of the merchandise detected by the capacitance sensor 110C. Pattern). For example, it is possible to specify that a product is not picked up, a specific product is picked up, a specific product picked up is returned to the shelf, and the like.

  FIG. 7A is a diagram illustrating an example of a purchase intention analysis result based on the pre-shelf psychology and the pre-shelf behavior of the present embodiment, and a distribution content pattern corresponding to the purchase intention analysis result. As shown in FIG. 7A, the willingness to purchase is pre-patterned by a combination of the specified pre-shelf psychological pattern and pre-shelf behavior pattern, and each analysis result thus patterned is stored in the storage device 330 in advance. Yes. The analysis control unit 321 extracts a purchase intention analysis result pattern set in advance using the pre-shelf psychological pattern and the pre-shelf behavior pattern, and specifies the user's purchase intention.

  The content control unit 322 extracts content corresponding to the user's willingness to purchase specified by the analysis control unit 321 from the content group, and transmits the content to the product display shelf system 100. The storage device 330 stores in advance a distribution content pattern (purchase support information) in which a purchase intention analysis result and a content to be distributed are linked, and the content control unit 322 uses the purchase intention analysis result to distribute content. The pattern is specified, and the corresponding content is extracted from the content group stored in the storage device 330 and transmitted to the merchandise display shelf system 100.

  Each content included in the content group can prepare, for example, a plurality of contents 1 for each product, and a distribution content pattern corresponding to the analysis result 1 can be individually set for each product. Therefore, for example, the content control unit 322 extracts the content 1 corresponding to the analysis result 1, but extracts the content 1 corresponding to the specific product picked up by the user specified by the analysis control unit 321 from the content group. Then, it can be transmitted to the merchandise display shelf system 100.

  FIG. 7B is a diagram illustrating an example of a purchase willingness analysis result based on the pre-shelf psychology and the pre-shelf behavior and an utterance / action pattern corresponding to the purchase willingness analysis result, and corresponds to FIG. 7A. Therefore, the content delivered by the merchandise display shelf system 100 and the speech / operation of the humanoid robot 200 executed based on the customer service control information can be performed in conjunction with each other as described later.

  The content control unit 322 extracts customer service control information (utterance / motion) corresponding to the user's willingness to purchase specified by the analysis control unit 321 from the customer service control information database, and the humanoid robot via the product display shelf system 100 200. The storage device 330 stores in advance an utterance / motion pattern in which the purchase will analysis result and the utterance / motion of the humanoid robot 200 are linked, and the content control unit 322 uses the purchase will analysis result to speak. / The action pattern is specified, and the corresponding customer service control information is extracted from the customer service control information database stored in the storage device 330 and transmitted to the humanoid robot 200.

  As for each customer service control information (utterance / action) included in the customer service control information database, for example, a plurality of utterances / actions 1 (customer service control information 1) are prepared for each product in the same manner as each content included in the content group. The utterance / motion pattern corresponding to the analysis result 1 can be set individually for each product. That is, for example, the content control unit 322 extracts the utterance / action 1 corresponding to the analysis result 1, but the utterance / action 1 corresponding to the specific product picked up by the user specified by the analysis control unit 321. It can be extracted from the content group and transmitted to the humanoid robot 200 via the product display shelf system.

  8 to 10 are diagrams showing examples of customer service control and distribution content control of the humanoid robot 200 in the first to third phases.

  FIG. 8 is a diagram illustrating an example of customer service control and distribution content control of the humanoid robot 200 based on user characteristics in the first phase. The analysis control unit 321 specifies user characteristics from preset user characteristic patterns using information on gender, age group, language, and appearance characteristics.

  The content control unit 322 extracts customer service control information that matches the user characteristics specified by the analysis control unit 321. The humanoid robot 200 speaks / operates based on the extracted customer service control information, and urges the user to be interested in the product.

  For example, when the gender of the specified user characteristic is male, the age group is 25 to 29 years old, the language is Japanese, and the appearance characteristic is suit, the humanoid robot 200 sets the user characteristic pattern to “male after work”. The corresponding utterance / action is performed as a pattern. At this time, the speech / motion by the humanoid robot 200 is based on the customer service control information corresponding to the user's language used, extracted and transmitted from the customer service control information prepared for each language used in advance in the storage device 330 of the management device 300. Done. For example, when the language used by the identified user is Japanese, the utterance / action is performed based on the customer service control information corresponding to Japanese.

  In addition, as described above, the analysis control unit 321 refers to an existing user profile or creates a new user profile for the user in front of the shelf. The content control unit 322 extracts distribution content that matches information included in the user profile. The merchandise display shelf 100 promotes purchases based on the extracted distribution content.

  FIG. 9 is a diagram illustrating an example of customer service control and distribution content control of the humanoid robot 200 based on the pre-shelf psychology and the pre-shelf behavior in the second phase. As shown in FIG. 6, the analysis control unit 321 specifies the user's pre-shelf psychology from preset pre-shelf psychological patterns. Similarly, the analysis control unit 321 identifies the user's pre-shelf behavior from preset pre-shelf behavior patterns.

  The content control unit 322 extracts customer service control information that matches the pre-shelf psychology specified by the analysis control unit 321. Similarly, the content control unit 322 extracts customer service control information that matches the specified pre-shelf behavior. The humanoid robot 200 speaks / operates based on the extracted customer service control information, and urges the user to be interested in the product.

  For example, when the user's gaze is concentrated on a specific product and the user's facial expression is a lost expression, the gaze analysis result is “concentration”, and the facial expression analysis result is “stuck”, the pre-shelf psychological pattern B is The customer service control information that is specified (see FIG. 6A) and corresponds to the pre-shelf psychological pattern B is extracted. Thereby, the humanoid robot 200 performs an utterance / motion corresponding to the pre-shelf psychological pattern B.

  For example, if it is determined that the user's action in front of the shelf (taken by the product) is “taken the product”, the action pattern Y in front of the shelf is specified (see FIG. 6B), and the action in front of the shelf is determined. The customer service control information corresponding to the pattern Y is extracted. Thereby, the humanoid robot 200 performs an utterance / motion corresponding to the behavior pattern Y before the shelf.

  In addition, the content control unit 322 extracts distribution content that matches the specified pre-shelf psychology, and the merchandise display shelf 100 promotes purchases based on the extracted distribution content. Similarly, the content control unit 322 extracts distribution content that matches the specified pre-shelf behavior, and the merchandise display shelf 100 promotes purchases based on the extracted distribution content.

  FIG. 10 is a diagram illustrating an example of customer service control and distribution content control of the humanoid robot 200 based on the willingness to purchase in the third phase. As illustrated in FIGS. 7A and 7B, the analysis control unit 321 specifies the user's purchase intention from among preset purchase intention patterns.

  The content control unit 322 extracts customer service control information that matches the willingness to purchase specified by the analysis control unit 321. The humanoid robot 200 speaks / operates based on the extracted customer service control information, and urges the user to be interested in the product.

  For example, when a purchase motivation analysis based on the pre-shelf psychological pattern A and the pre-shelf behavior pattern Z is performed, the user's purchase motivation is specified as the analysis result 3 (see FIG. 7B), and customer service control information (see FIG. 7B) Speech / motion 4) is extracted. Thereby, the humanoid robot 200 can perform an utterance / motion corresponding to the analysis result 3.

  Similarly, the content control unit 322 extracts distribution content that matches the specified purchase intention. The merchandise display shelf 100 promotes purchases based on the extracted distribution content.

  As shown in FIG. 8, in the first phase, utterances / actions are performed according to the specified user language, but in the second and third phases of FIGS. Customer service control information prepared for each language used Customer service according to the language used by the user can be performed. In other words, customer service can be performed for each of the specified user's pre-shelf psychology, pre-shelf behavior, and purchase intent according to the specified language of use.

  By preparing the customer service control information for each language used, it is possible to perform customer service according to the language used by the user as described above. Further, in the customer service control information prepared for each language used, it is possible to set the content of the operation corresponding to the output voice. By making such a setting possible, as described above, when the specified user's language is Japanese or English, the humanoid robot 200 can be used in accordance with the specified language. Various actions can be set.

  FIG. 11 is a diagram illustrating a processing flow of the merchandise display shelf system 100 and the management apparatus 300 according to the present embodiment. As shown in FIG. 11, the merchandise display shelf system 100 controls the operation of the sensing unit 110, collects (acquires) the sensor detection results in real time (S101), and stores them in the management apparatus 300 as sensing information (sensor detection information). Transmit (S102).

  The product display shelf system 100 (control unit 130) detects that a user approaching the product display shelf system 100 is within a predetermined range, and triggers that a user is detected in front of the shelf. In addition, it may be configured to perform operation control of the sensing unit 110 and transmission control of sensing information to the humanoid robot 200 and / or the management apparatus 300.

  The management apparatus 300 matches the face image of the user profile already stored (registered) using the user's captured image received from the merchandise display shelf system 100 (S301), and determines whether or not the user profile is registered. (S302). If the user profile does not exist (NO in S302), the management apparatus 300 determines that the user is a new user, calculates gender and age from the user's face image (S303), and newly registers the profile (S304). It is also possible to calculate appearance features using the user's whole body image in addition to the face image in step S303, and register the appearance features as a profile in step S304. Further, as described later, user language information provided from the humanoid robot 200 can be registered as a profile.

  When a user profile already exists (YES in S302) and when a user profile is newly registered, the management apparatus 300 extracts content and customer service control information based on the user profile (user characteristics) (S311).

  Next, in step S305, the user's pre-shelf psychology described above is specified, and content and customer service control information corresponding to the specified pre-shelf psychology are extracted (S306). After the pre-shelf psychology is specified or when the pre-shelf behavior is detected (YES in S307), the user's pre-shelf behavior is specified (S308). Thereafter, the user's willingness to purchase based on the specified pre-shelf psychology and pre-shelf behavior is determined (S309), and the content and customer service control information based on the determined purchasing motivation are extracted (S310). At this time, as described above, content and customer service control information related only to the pre-shelf behavior specified in step S308 can also be extracted.

  The management apparatus 300 performs transmission processing (distribution processing) of each content and each customer service control information extracted in relation to the user profile, the pre-shelf psychology, the pre-shelf behavior, and the purchase intention (S312). For example, a transmission process for displaying the extracted content on a single or a plurality of displays is performed. In addition, a transmission process for causing the humanoid robot 200 to execute speech / motion based on the extracted customer service control information is performed. At this time, the customer service control information is transmitted to the humanoid robot 200 via the merchandise display shelf system 100.

  Specifically, each content is individually displayed on each small display device of the monitor unit 120B, or as described in FIG. 5, the user profile, the pre-shelf psychology, the pre-shelf behavior, and the purchase intention In order, each content can be switched and displayed. Further, different contents can be displayed on each of the display device 120A and the monitor unit 120B. Furthermore, different contents can be displayed on each of a plurality of small display devices constituting the monitor unit 120B, or a plurality of small displays can be displayed. It is also possible to display the content divided into each small display device using the device as one display unit.

  In addition, since the monitor unit 120B has a plurality of small display devices arranged side by side, it is also possible to display a plurality of narrative contents on each small display device.

  The merchandise display shelf system 100 displays the content received from the management device 300 on the output unit 120, outputs sounds such as music, voice, and sound effects from the speaker 120C, and outputs scent from the incense generator 120D. Control (user facing processing) is performed (S103). These display and output control information can be received together with the content transmitted from the management apparatus 300 or alone.

  After the transmission processing in step S312, the management apparatus 300 determines whether or not the user in front of the product display shelf has left the predetermined front shelf area based on the sensing information received from the product display shelf system 100 (S313). ), If it has not left (NO in S313), the process returns to step S305, and based on the sensing information changing in real time, the content and customer service control information extraction and transmission processing based on the pre-shelf psychology, the pre-shelf behavior, and the purchase intention Repeat. Although the motion sensor 110B is used as a human sensor, the present invention is not limited to this, and a human sensor such as a separate infrared sensor may be provided.

  On the other hand, when it is determined that the user in front of the merchandise display shelf has left the predetermined shelf front area (YES in S313), the user profile of the user specified in step S301 is updated. For example, the pre-shelf psychology specified in step S305 and the pre-shelf behavior specified in step S308 are registered in the user profile (S314). Further, based on the product purchase information acquired from the sales management system, the purchased product is registered in the purchase history of the user profile (S315). Then, after registration / update of the user profile, the process waits until a user (consumer) is detected in the front shelf area based on the sensing information (S316). In addition, as above-mentioned, a shelf front area is a range which can detect that a user exists in front of a shelf.

  In the user profile update process in steps S314 and S315, the content and output control information (content distribution result) actually transmitted to the product display shelf system 100 and the customer service control information (human type) transmitted to the humanoid robot 200 are displayed. The customer service result) can be stored in the storage device 330 in association with the user profile. This processing can be performed by the content control unit 322.

  With reference to FIG.12 and FIG.13, the processing flow of the whole purchasing assistance system via the goods display shelf and humanoid robot of this embodiment is demonstrated. Hereinafter, the commodity display shelf system 100 will be described as performing sensor detection and sensing information transmission (S101, S102) in real time as described above. The same step numbers as those in FIG. 11 are replaced with the above description, but the description will be supplemented as necessary. Steps S103A to S103D correspond to S103 in FIG. Steps S312A to S3D correspond to S312 in FIG.

  FIG. 12 is a diagram showing a processing flow in the first phase and the second phase of the purchase support system of the present embodiment. In the first phase, the merchandise display rack system 100 detects that a user approaching the merchandise display rack system 100 has entered a predetermined range, and transmits it to the management device 300 and the humanoid robot 200 as sensing information.

  When the humanoid robot 200 receives sensing information indicating that the user has entered the predetermined range, the humanoid robot 200 executes an utterance promoting action (S201). When the user's voice is input to the microphone 272 due to the user's utterance, the humanoid robot 200 performs voice recognition processing and identifies the user's language (S202). The humanoid robot 200 transmits the specified result of the language used as user language information to the management apparatus 300 via the product display shelf system 100. As described above, the display input device 274 may allow the user to specify the language used.

  The management apparatus 300 can register / update the language information provided from the humanoid robot 200 in the user profile (S304). The language information can be registered / updated by a new user profile or an existing user profile.

  In the subsequent steps, the customer service control information corresponding to the specified language is extracted (S311, S306, S309, S310) and transmitted to the humanoid robot 200 (S312A to D). As a result, the humanoid robot 200 can output a voice corresponding to the specified language used and speak, and can perform an operation corresponding to the output voice.

  By the transmission process (S312A), the merchandise display shelf system 100 receives the content. Thereby, the merchandise display shelf system 100 performs the user facing process based on the user profile (user characteristic) (S103A). On the other hand, the humanoid robot 200 receives customer service control information. Thereby, the humanoid robot 200 performs customer service with the utterance / motion based on the user characteristics (S203A). Thereafter, the purchase support system of the present embodiment proceeds to the second phase.

  If there is no utterance by the user, step S202 may be omitted and language information may not be registered as a user profile. Further, step S201 may be omitted, and step S202 may be executed to wait for the user's speech, or step S202 may be omitted together with step S201, and the transmission process (S312A) may be waited.

  In the second phase, the merchandise display shelf system 100 detects the facial expression and line of sight of the user who is viewing the merchandise and transmits it to the management apparatus 300 as sensing information. Thereafter, by the transmission process (S312B), the product display shelf system 100 receives the content and the humanoid robot 200 receives the customer service control information in the same manner as described above. Thereby, the merchandise display shelf system 100 performs the user facing process based on the user's pre-shelf psychology (S103B). Further, the humanoid robot 200 performs customer service by speaking / motion based on the user's pre-shelf psychology (S203B).

  Further, in the second phase, the merchandise display shelf system 100 detects the action (pre-shelf action) in the user's merchandise hand (YES in S307), and transmits the sensing information to the management apparatus 300. Thereafter, by the transmission process (S312C), the product display shelf system 100 receives the content and the humanoid robot 200 receives the customer service control information in the same manner as described above. Thereby, the merchandise display shelf system 100 performs the user facing process based on the user's pre-shelf behavior (S103C). In addition, the humanoid robot 200 performs customer service by utterance / motion based on the user's pre-shelf behavior (S203C).

  When the user's pre-shelf psychology and pre-shelf behavior are specified, the purchase support system of the present embodiment proceeds to the third phase.

  FIG. 13 is a diagram showing a processing flow in the third phase of the purchase support system of the present embodiment. In the third phase, following the second phase, the user's willingness to purchase is determined based on the user's pre-shelf psychology and the pre-shelf behavior identified by the management device 300 (S310). Based on the extraction processing (S311).

  Thereafter, by the transmission process (S312D), the product display shelf system 100 receives the content and the humanoid robot 200 receives the customer service control information in the same manner as described above. Thereby, the merchandise display shelf system 100 performs a user facing process based on the user's willingness to purchase (S103D). Further, the humanoid robot 200 performs customer service by speaking / motion based on the user's willingness to purchase (S203D).

  The merchandise display shelf system 100 detects that the user has left the predetermined range or has left the predetermined range and a certain time has passed. As a result, the merchandise display shelf system 100 determines that the user has left the shelf (YES in S313), and transmits sensing information indicating that the user has left the predetermined range to the management device 300 and the humanoid robot 200. . Thereafter, the merchandise display shelf system 100 stands by until a user is detected in the shelf front area (within a predetermined range) (S316).

  When receiving the sensing information indicating that the user has left the predetermined range, the humanoid robot 200 waits until the user is detected in the area in front of the shelf (within the predetermined range) (S316). When receiving the sensing information indicating that the user has left the predetermined range, the management apparatus 300 performs the processes of steps S314 and S315 and similarly waits (S316). When the merchandise display shelf system 100 detects that the user has entered the predetermined range, the purchase support system of the present embodiment proceeds to the first phase.

  If the user does not leave the predetermined range in the third phase (S313-NO), the process returns to the second phase and the same processing as described above is continued.

  In addition, in FIG. 11-13, customer service control information is extracted together with the contents and transmitted to the product display shelf system 100 and the humanoid robot 200, respectively, but only one of them is extracted and transmitted as necessary. May be. In addition, each utterance / motion process (S203A to D) of the humanoid robot 200 and each user facing process (S103A to D) of the commodity display shelf system 100 can be performed in conjunction with each other, but are not necessarily performed simultaneously. Absent. For example, only the utterance / motion process of the humanoid robot 200 may be performed as necessary, or only the user facing process of the merchandise display shelf system 100 may be performed.

  Next, returning to FIG. 4, the machine learning function of this embodiment will be described. The learning control unit 323 uses the purchase history of the user profile stored in the storage device 330, the content distribution result, and the customer service result of the humanoid robot to use the content pattern that leads to the purchase of the product and the user's purchase that leads to the purchase of the product. It is possible to extract a behavior pattern, a customer service pattern of a humanoid robot that leads to purchase of a product, a combination of a content pattern that leads to purchase of a product and a customer service pattern of the humanoid robot 200, and the like.

  For example, the content A provided to the user who was viewing the product A is compared with the number of purchases of the product A, and if the number of purchases is equal to or greater than a predetermined number, the content A is determined to be effective in promoting purchases. be able to. On the other hand, when the number of purchases is less than the predetermined number, it can be determined (evaluated) that the content A has no effect on the promotion of purchase.

  Similarly, the customer service control information A transmitted for controlling the customer service behavior of the humanoid robot 200 for the user who was viewing the product A is compared with the number of purchases of the product A, and the purchase number is equal to or greater than a predetermined number. The customer service behavior of the humanoid robot 200 based on the customer service control information A can be determined to have an effect on purchase promotion. On the other hand, when the number of purchases is less than the predetermined number, it is possible to determine (evaluate) that the customer service behavior of the humanoid robot 200 based on the customer service control information A is not effective in promoting the purchase.

  Further, when the content provision and the customer service behavior of the humanoid robot 200 are linked, the combination of the provided content A and the customer service control information A is compared with the number of purchases of the product A, and the purchase is performed in the same manner as described above. By analyzing whether the number is equal to or greater than the predetermined number or less than the predetermined number, it is possible to determine (evaluate) whether or not there is an effect on purchase promotion.

  Moreover, the learning control part 323 can extract each pattern and the combination of patterns which led to the purchase of goods similarly to the above for every use language of the consumer specified by the speech recognition process. Further, in the same manner as described above, the learning control unit 323 may determine whether each content, each customer service control information, or each combination of the content and the customer service control information is effective for purchasing promotion for each language used. it can.

  In addition, in combination with facial expression, line of sight, product information, gender, age group, appearance characteristics, language, and staying time, detailed analysis and learning control of each content effect and customer service behavior (customer service control information) The unit 323 changes the combination of each content extracted in relation to the user profile, the pre-shelf psychology, the pre-shelf behavior, and the purchase motivation, the combination of each customer service control information, and each combination of the content and the customer service control information. Machine learning can be performed. Further, the purchase intention analysis result can be updated / deleted, or a new purchase analysis result can be generated and added.

  Although the embodiment has been described above, the control unit 130 of the product display shelf system 100 may be physically or externally connected to the product display shelf by being connected to the sensing unit 110 and the output unit 120 by wire or wirelessly. Is possible.

  In addition, the management apparatus 300 transmits content to the product display shelf system 100 via the network. However, the present invention is not limited to this, and for example, content data is collectively collected from the management apparatus 300 to the product display shelf system 100 at a predetermined timing. The content can be stored in advance on the control unit 130 side. In this case, the management apparatus 300 may generate and extract output control information for each content and transmit it to the product display shelf system 100 without transmitting content data (actual data such as images, videos, and character information). it can. The control unit 130 can extract the content stored in advance in the storage device 133 based on the received output control information and perform output control of the output unit 120.

  In addition to the above-described hardware configuration, the management device 300 includes a display device such as a liquid crystal display, a CPU that controls the entire device (each unit), a memory (main storage device), a mouse, a keyboard, a touch panel, a scanner, and the like. Operation input means, output means such as a printer and a speaker, an auxiliary storage device (hard disk or the like), and the like can be provided.

  Further, in the above embodiment, the customer service by the humanoid robot 200 has been described as an example. However, for example, customer service by a robot imitating an animal such as a dog may be performed. In addition, any robot can be used as long as it can perform customer service by gestures.

  In the above-described embodiment, the humanoid robot 200 is connected to the management apparatus 300 via the merchandise display shelf system 100. However, the humanoid robot 200 may be directly connected to the management apparatus 300 by wire or wireless to transmit and receive various types of information. it can.

  The control device 230 can be physically connected to the humanoid robot 200 by connecting to the communication device 240, the storage device 250, the sensor device 260, and the input / output unit 270 in a wired or wireless manner. For example, the control device 230 may be provided on a product display shelf. Similarly, the microphone 272 may be provided on the commodity display shelf. In addition, the robot cooperation unit 131C may be omitted by including the function of the robot cooperation unit 131C of the commodity display shelf system 100 in the function of the robot control unit 232. That is, the customer service control information can be received only by the robot controller 232 and the customer service behavior of the humanoid robot 200 can be controlled.

  Further, the language database stored in the storage device 250 of the humanoid robot 200 may be stored in the storage device 133 of the merchandise display shelf system 100 or the storage device 330 of the management device 300. Accordingly, for example, a microphone is provided in the product display shelf system 100 and voice recognition processing is performed on the product display shelf system 100 side, or voice collected by the product display shelf system 100 or the humanoid robot 200 is stored in the management apparatus 300 as voice data. It is possible to receive the voice recognition process on the management apparatus 300 side.

  Further, the camera 273 of the humanoid robot 200 may be used instead of the camera 110A of the merchandise display shelf system 100, or the sensor device 260 of the type robot 200 may be used instead of the motion sensor 110B. In other words, information related to the user can be detected by the camera 273 and the sensor device 260 (sensor detection information). Sensor detection information in the humanoid robot 200 can be transmitted to the control unit 130 or provided directly to the management apparatus 300. At this time, the management device 300 can determine the user's willingness to purchase based on the sensor detection information from the camera 273 and the sensor device 260 instead of the sensor detection information from the sensing unit 110.

  In addition, the customer service control information can be collectively transmitted from the management device 300 to the humanoid robot 200 at a predetermined timing, and the customer service control information can be stored in the storage device 250 in advance. In this case, the management apparatus 300 instructs which customer service action is to be executed based on which customer service control information is transmitted without transmitting customer service control information (actual data defining what kind of utterance and action is actually performed). Can be generated and extracted and sent to the humanoid robot 200. The robot control unit 232 can perform customer service control of the humanoid robot 200 by extracting content stored in advance in the storage device 250 based on the received customer service instruction information.

  It is also possible to install a plurality of merchandise display shelf systems 100 and humanoid robots 200 in a store and connect them to one management device 300 via a network to constitute a purchase support system. Thereby, the cooperation between the merchandise display shelves installed at positions separated in the store can be performed. For example, when there is no product that the user is looking for in the product display shelf A, the product display shelf B in which the product that the user is looking for is arranged can be specified via the network. At this time, the user can be guided or guided to the product display shelf B by the utterance / motion by the humanoid robot 200 and the display 120A and the display 120B of the product display shelf system 100. It is also possible to install a plurality of management devices and connect them to each other via a network.

  Moreover, the management apparatus 300 can also connect with the merchandise display shelf system 100 and the humanoid robot 200 of another store via a network, and can also cooperate between stores. Thereby, information required for purchase support can be shared between stores. For example, by sharing a user profile between stores, purchase promotion according to the user can be performed in other stores as well.

  In addition, each function of the present invention can be realized by a program, a computer program prepared in advance for realizing each function is stored in the auxiliary storage device, and a control unit such as a CPU is stored in the auxiliary storage device. By reading the program into the main storage device and executing the program read into the main storage device, the control unit can cause the computer device to operate the functions of the respective units of the present invention. On the other hand, each function of the present invention can be configured by individual control devices, and a plurality of control devices can be connected directly or via a network.

  Further, the program can be provided to a computer in a state where the program is recorded on a computer-readable recording medium. Computer-readable recording media include optical disks such as CD-ROM, phase change optical disks such as DVD-ROM, magneto-optical disks such as MO (Magnet Optical) and MD (Mini Disk), floppy (registered trademark) disks, Examples include magnetic disks such as removable hard disks, memory cards such as compact flash (registered trademark), smart media, SD memory cards, and memory sticks. A hardware device such as an integrated circuit (IC chip or the like) specially designed and configured for the purpose of the present invention is also included as a recording medium.

1: Shelf body 2A, 2B, 2C: Product display unit 100: Product display shelf system 110: Sensing unit 110A: Camera 110B: Motion sensor 110C: Capacitance sensor 120: Output unit 120A: Display device 120B: Monitor unit 120C : Speaker 120D: Incense generating device 130: Control unit 131: Control device 131A: Sensor control unit 131B: Output control unit 132: Communication device 133: Storage device 200: Humanoid robot 220: Robot system 230: Control device 231: Sensor control Unit 232: Robot control unit 240: Communication device 250: Storage device 260: Sensor device 270: Input / output unit 271: Speaker 272: Microphone 273: Camera 274: Display input device 300: Management device 310: Communication device 320: Control Location 321: the analysis control unit 322: the content control unit 323: a learning control unit 330: storage device

Claims (10)

Product display shelves,
Sensing provided on a merchandise display shelf, including a first sensor device that acquires a facial image of a consumer located in front of the shelves and a second sensor device that acquires consumer movements for the merchandise displayed on the merchandise display shelves A product display shelf system comprising: a unit;
A customer service robot that performs utterances and / or actions related to customer service;
A management device that is connected to the commodity display shelf system via a network, receives each sensor detection information acquired by the sensing unit, and discriminates a consumer's purchase intention,
The management device
The pre-shelf psychological pattern is specified based on the consumer's facial expression and line of sight detected from the face image, and the pre-shelf behavior pattern is specified based on the consumer's movement with respect to the product. And an analysis control unit for discriminating a consumer's purchase willing pattern corresponding to the pre-shelf behavior pattern;
Storage units each storing customer service control information that defines speech and / or actions related to the customer service to be performed by the customer service robot for each of the plurality of purchase willing patterns;
A content control unit that selects the customer service control information according to the determined purchase willingness pattern and provides the selected customer service control information;
A purchase support system characterized by comprising: The storage unit stores the customer service control information for each consumer's pre-shelf psychological pattern and / or the pre-shelf behavior pattern, respectively.
The content control unit selects the customer service control information according to the identified pre-shelf psychological pattern and / or the customer service control information according to the identified pre-shelf behavior pattern, and the pre-shelf psychological pattern and / or Alternatively, the customer support control information according to the pre-shelf behavior pattern is provided. The customer service control information includes utterance information for a plurality of languages,
The management device determines the language used by the consumer based on the utterance voice of the consumer collected via the customer service robot or the product display shelf system,
The purchase support system according to claim 1 or 2, wherein the content control unit selects and provides the customer service control information including the utterance information corresponding to the determined language used. The analysis control unit extracts consumer attributes including a sex and an age group of the consumer from the face image based on a predetermined face recognition process, and user profile information associated with the face image and the consumer attribute is extracted. Generating and storing in the storage unit, specifying the user profile information stored in the storage unit by performing pattern matching based on the face image, and a store visit history to a store where a product display shelf is installed; Storing the purchase history of the products displayed on the product display shelf acquired from a predetermined sales management system for managing the sales history of the store in association with the specified user profile information;
The content control unit selects the customer service control information based on any one or a combination of the gender, the age group, the store visit history, and the purchase history of the consumer based on the user profile information. The purchase support system according to any one of claims 1 to 3, wherein the selected customer service control information is provided. The analysis control unit generates user profile information associated with the face image and stores the user profile information in the storage unit, and performs pattern matching based on the face image and stores the user profile information stored in the storage unit. Specifying and storing the purchase history of the product displayed on the product display shelf acquired from a predetermined sales management system for managing the sales history of the store in association with the user profile information,
Based on the result of the process of transmitting the customer service control information for controlling the customer service robot and the purchase history, the result of the process of transmission that led to the purchase of the product is expressed as a psychological pattern before the shelf of the consumer, The purchase support system according to any one of claims 1 to 3, further comprising a learning control unit that evaluates each behavior pattern and each purchase motivation.   The learning control unit displays the result of the transmission process that led to the purchase of the product for each language used based on the utterance voice of the consumer collected via the customer service robot or the product display shelf system. The purchase support system according to claim 5, wherein the evaluation is performed for each psychological pattern, the pre-shelf behavior pattern, and the purchase intention. The sensing unit further includes a third sensor device that detects the movement of the product displayed on the product display shelf,
The analysis control unit identifies whether the consumer has picked up a specific product as the action pattern before the shelf by combining sensor detection information of the second sensor device and the third sensor device. The purchasing support system according to any one of claims 1 to 5. The customer service robot further includes a fourth sensor device corresponding to the first sensor device, and a fifth sensor device corresponding to the second sensor device,
The management device receives each sensor detection information acquired by the fourth sensor device and the fifth sensor device directly from the customer service robot instead of each sensor detection information acquired by the sensing unit, or the customer service robot. The purchase support system according to claim 1, wherein the purchase intention of a consumer is determined by receiving the product via the product display shelf system connected to the product. A product display shelf including a first sensor device for acquiring a facial image of a consumer positioned in front of the product display shelf, and a second sensor device for acquiring a consumer movement with respect to the product displayed on the product display shelf A product display shelf system including a sensing unit provided in
A program executed by a computer for providing customer service control information for causing the customer service robot to perform a predetermined customer service utterance and / or operation in cooperation with a customer service robot that performs customer service utterances and / or operations. Because
A first process for identifying a pre-shelf psychological pattern based on a consumer's facial expression and line of sight detected from the face image;
A second process for identifying a pre-shelf behavior pattern based on consumer movement with respect to the product;
A third process for determining a consumer's purchase willing pattern corresponding to the identified pre-shelf psychological pattern and the pre-shelf behavior pattern;
A fourth process of selecting the customer service control information associated with the determined purchase motivation pattern based on each of the customer service control information associated with each of the plurality of purchase motivation patterns;
A fifth process for providing the selected customer service control information;
Purchasing support program characterized by A product display shelf including a first sensor device for acquiring a facial image of a consumer positioned in front of the product display shelf, and a second sensor device for acquiring a consumer movement with respect to the product displayed on the product display shelf A product display shelf system including a sensing unit provided in
A purchase support management apparatus that provides customer service control information for causing a customer service robot to perform a predetermined customer service speech and / or action in cooperation with a customer service robot that performs customer service speeches and / or actions. And
The purchase support management device comprises:
The pre-shelf psychological pattern is specified based on the consumer's facial expression and line of sight detected from the face image, and the pre-shelf behavior pattern is specified based on the consumer's movement with respect to the product. And an analysis control unit for discriminating a consumer's purchase willing pattern corresponding to the pre-shelf behavior pattern;
Storage units each storing customer service control information that defines speech and / or actions related to the customer service to be performed by the customer service robot for each of the plurality of purchase willing patterns;
A content control unit that selects the customer service control information according to the determined purchase willingness pattern and provides the selected customer service control information;
A purchase support management apparatus comprising: