JP2017174222A - Guide output device, guide output method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To directly present guide information required by a user.SOLUTION: A signage device comprises an operation unit 35 accepting selection operation by a user for a plurality of guide objects, an imaging unit IM detecting action of the user as action information, a wireless LAN interface 38, a wireless LAN antenna 39, a content management memory 20 storing the accepted guide objects and the detected action information in association with each other, projection systems 21-25 for outputting guide information related to the accepted guide objects on the basis of the stored contents, and a CPU 32 for updating the contents fo the guide information output by the projection systems 21-25 on the basis of the contents stored in the content management memory 20.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a guidance output device, a guidance output method, and a program.

  2. Description of the Related Art Conventionally, a technique has been proposed for improving the service level by tracking the behavior of a visitor who has visited the store, extracting the potential request or purpose of the visitor. (For example, Patent Document 1)

JP 2010-108166 A

  The technique described in the above-mentioned patent document specifies the purpose and potential demand of the visitor from the posture and facial expression of the visitor wearing an IC tag that can identify an individual. A complicated system configuration is required, such as a wide variety of cameras including a telephoto camera and a wide-angle camera, a dedicated IC tag, and a plurality of tag readers installed in the store to read information of the IC tag.

  In addition, the device in this patent document instructs a service person possessing a receiving terminal on the store side to provide services for various visitors with different store use frequencies, and the actual service is a human service person Is running. For this reason, the system configuration itself is complicated, such as having to arrange a number of people corresponding to the visitors 1: 1, and it is considered extremely difficult to realize.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a guidance output device, a guidance output method, and a program capable of presenting guidance content required by a user. There is.

  One aspect of the present invention includes a reaction receiving unit that receives a user's response to a guidance target, a behavior detection unit that detects the user's behavior as behavior information, and the guidance target and the behavior that are reacted by the reaction receiving unit. Output means for outputting guidance information based on the behavior information detected by the detection means.

  According to the present invention, it is possible to present guidance content that a user needs.

The perspective view which shows the external appearance structure of the signage apparatus which concerns on one Embodiment of this invention. The block diagram which shows the function structure of the electronic circuit of the signage apparatus which concerns on the same embodiment. The flowchart which shows the processing content performed with the signage apparatus which concerns on the same embodiment. The figure which illustrates the path | route from the person of the tracking object which concerns on the embodiment to a signage apparatus after entering a shop with the flow line of the said person. The figure which illustrates the path | route until the person of the tracking object concerning the embodiment reaches | attains the goods selected from the signage apparatus with the flow line of the said person. The figure which illustrates the image of the guidance content projected with the signage apparatus which concerns on the embodiment.

  Hereinafter, an embodiment in which the present invention is applied to a signage device that advertises a product in a store will be described in detail with reference to the drawings.

  FIG. 1 is a perspective view showing an external configuration of the signage device 10. The signage device 10 is an electronic mannequin using projector technology, and a replaceable rectangular signage board SB is erected on the upper front end side of the device housing 11.

  The signage board SB displays an image by projecting a light image emitted from a rear projection projection lens (not shown) provided on the upper surface of the apparatus housing 11 from the back side.

  Near the center of the lower end of the signage board SB, a plurality of, in this case, four operation buttons B1 to B4 are projected together, and when the viewer touches any of these buttons, the board mounting base The operation positions can be detected by the arrayed linear infrared sensor arrays 12 each having directivity.

  Furthermore, an imaging unit IM of a wide-angle optical system for photographing the front side of the signage device 10 is provided on the front surface of the device housing 11.

  Next, the functional configuration of the electronic circuit of the signage apparatus 10 will be described with reference to FIG. The content management memory 20 is selected from data indicating the arrangement configuration of various products in the store where the signage device 10 is installed and the position of the signage device 10 in addition to the content data of a plurality of products projected on the signage board SB. Management data such as guidance content obtained by imaging the route to the display position of the product and data indicating the degree of interest of the visitor for each product to be described later are also stored.

  The content data stored in the content management memory 20 is composed of moving image data and sound data. The moving image data in the content data is read by a CPU 32 described later and sent to the projection image driving unit 21 via the system bus BS.

  The projection image drive unit 21 multiplies a frame rate according to a predetermined format, for example, 120 [frames / second], the number of color component divisions, and the number of display gradations, according to the transmitted image data. The micromirror element 22 as a display element is driven to display by faster time-division driving.

  The micromirror element 22 performs display operation by individually turning on / off each inclination angle of a plurality of micromirrors arranged in an array, for example, WXGA (horizontal 1280 pixels × vertical 800 pixels) at high speed. Thus, an optical image is formed by the reflected light.

  On the other hand, primary color lights of R, G, and B are emitted from the light source unit 23 in a time-sharing manner. The light source unit 23 includes an LED which is a semiconductor light emitting element, and repeatedly emits R, G, and B primary color lights in a time-sharing manner.

  The LED included in the light source unit 23 may include an LD (semiconductor laser) or an organic EL element as an LED in a broad sense. The primary color light from the light source unit 23 is reflected by the mirror 24 and applied to the micromirror element 22.

  Then, an optical image is formed by the reflected light from the micromirror element 22, and the formed optical image is projected from the back side onto the signage board SB via the projection lens unit 25.

  The imaging unit IM includes a wide-angle imaging lens unit 27 facing the front direction of the signage device 10 and a CMOS image sensor 28 that is a solid-state imaging device disposed at the focal position of the imaging lens unit 27.

  An image signal obtained by the CMOS image sensor 28 is digitized by the A / D converter 29 and then sent to the captured image processing unit 30.

  The captured image processing unit 30 scans and drives the CMOS image sensor 28 to execute an image capturing operation. The captured image data is converted into a data file and sent to a CPU 32 described later.

  The captured image processing unit 30 executes face recognition processing on the image signal obtained by the imaging unit IM, performs face recognition of a person from the image signal, and uses the feature data obtained by the face recognition processing as an image. The information indicating the internal position and the distance information from the photographing lens unit 27 to the face position (analogue from the focus lens position in the photographing lens unit 27) are collected and sent to the CPU 32.

  The CPU 32 controls all the operations of the above circuits. The CPU 32 is directly connected to the main memory 33 and the program memory 34. The main memory 33 is composed of, for example, an SRAM and functions as a work memory for the CPU 32. The program memory 34 is composed of an electrically rewritable nonvolatile memory such as a flash ROM, and stores an operation program executed by the CPU 32, various fixed data, and the like.

  The CPU 32 controls the signage device 10 in an integrated manner by reading out the operation program, fixed data, and the like stored in the program memory 34, developing and storing them in the main memory 33, and executing the program. To do.

  The CPU 32 executes various projection operations in accordance with operation signals from the operation unit 35. This operation unit 35 is provided on the main body of the signage device 10 to key operation signals of several operation keys including a power key, or to the operation buttons B1 to B4 that are virtually projected onto a part of the signage board SB. A detection signal from the infrared sensor array 12 that detects the operation is received, and a signal corresponding to the received operation is sent to the CPU 32.

The CPU 32 is further connected to a sound processing unit 36 and a wireless LAN interface (I / F) 38 via the system bus BS.
The sound processing unit 36 includes a sound source circuit such as a PCM sound source, converts the sound data in the content data read from the content management memory 20 during the projection operation into analog, and drives the speaker unit 37 to emit sound, or is necessary Generates a beep sound and the like.

  The wireless LAN interface 38 receives image information from a plurality of external cameras (not shown) installed in various places in the store where the signage device 10 is installed via the wireless LAN antenna 39 and a person from the image information. Information (behavior information) obtained by collecting the feature data of the face image, the information indicating the position in the image, and the distance information from the photographing lens to the face position.

Next, the operation of the above embodiment will be described.
In the signage apparatus 10, the CPU 32 reads out from the program memory 34, develops it in the main memory 33, and executes the operation program, fixed data, and the like stored in the program memory 34 in advance when the signage apparatus 10 is shipped from the factory as a product. Processes such as version upgrades as needed via the network outside the signage device 10 via the wireless LAN interface 38 and the wireless LAN antenna 39 even when the signage device 10 is installed in a store or the like. The content updated and stored by is included.

  FIG. 3 shows an external view of the signage apparatus 10 that is executed in parallel with an operation in which the CPU 32 waits for an image of a plurality of products to be projected on the signage board SB and a selection operation for any of the products to be performed. A series of processing until a specified person is identified from a plurality of pieces of image information sent from a plurality of cameras using facial image feature data and the identified person leaves the store where the signage device 10 is installed. Is extracted and shown.

  That is, the CPU 32 actually executes the processing shown in FIG. 3 in parallel with each customer at the store. In the present embodiment, it is executed in parallel for each store visitor, but the present invention is not limited to this, and processing may be performed for some of the store visitors. , Processing may be performed for all customers who visit the store.

  Initially, the CPU 32 sends image information of a new visitor's face image sent from the camera closest to the store entrance and exit, and feature data obtained from the image information by human face recognition processing. Are collectively received as information indicating a position in the image, distance information from the photographing lens unit 27 to the face position, and information collected, and the visitor is determined as a tracking target (step S101).

  Thereafter, the CPU 32 instructs all cameras including the imaging unit IM in the signage device 10 to capture images including the attendee (step S102), and thereafter obtained by each camera and the imaging unit IM. The person whose feature data matches the tracking target is extracted from the image information, and the person to be projected is determined based on the arrangement of the corresponding camera, the position in the image, the distance information to the tracking target, etc. The position on the plane in the store is specified and stored in the content management memory 20 together with time information as an action history (step S103).

  Thereafter, the degree of interest in the product is measured and updated based on the distance between the position on the plane in the store where the person to be tracked is located and the product displayed at the closest position (step S104). .

  In this case, for example, the degree of interest in the product is calculated as an integrated value obtained by integrating the degree of proximity from a predetermined distance threshold and the time of proximity to the accumulated and stored content. It shall be.

  Thereafter, the CPU 32 determines whether or not the person to be tracked has been picked up by the person to be tracked by the signage device 10 depending on whether or not a selection operation has been performed on a plurality of products projected on the signage board SB. It is determined whether or not a product has been selected (reacted) (step S105). The determination may be made by determining that the person to be tracked stops in front of the signage device 10 and is interested in a certain content. In addition, the means for determining that the product advertised by the signage device 10 is interested can be variously modified.

  Here, the person to be tracked is not photographed by the imaging unit IM, and there is no selection operation for a plurality of products projected on the signage board SB, and the merchandise selection by the signage device 10 by the person to be tracked is performed. If it is determined that there is not (No in step S105), the CPU 32 returns to the processing from step S103 again.

  In this way, the processes in steps S103 to S105 are repeatedly executed, and various products displayed on the route in the store through which the tracking target person passes until the tracking target person reaches the signage device 10 and selects an arbitrary product. Accumulate your interest in.

  If it is determined in step S105 that the person to be tracked has reached the signage device 10 and has performed an operation for selecting an arbitrary product (Yes in step S105), the CPU 32 selects the selected product corresponding to the operation. Is read out from the content management memory 20 and projected onto the signage board SB (step S106).

  FIG. 4 is a diagram illustrating a path from the tracking target person entering the store SP to the signage device 10 as the flow line ML1 of the person. In the figure, it can be understood that the person to be tracked was particularly interested in the products D, C, and B.

  Here, when the degree of interest in each product is quantified based on the degree of interest, as described above, the degree of proximity to the product and the time of proximity are integrated from a predetermined distance threshold. Assuming that it is indicated by the integrated value, in FIG. 4, it is considered that the degree of interest is high in the order of “product D> product C> product B”.

  The CPU 32 that presented the guidance content corresponding to the product selected for the person to be tracked in step S106 by projection shifts again to the step of integrating the interest level of the person. That is, the CPU 32 subsequently extracts from the image information obtained by each camera and the imaging unit IM what the person's feature data matches the tracking target, and arranges the corresponding camera, the position in the image, Then, the position on the plane in the store where the person to be projected is located is specified by the distance information to the tracking target, etc., and stored in the content management memory 20 together with time information as an action history (step S107).

  Thereafter, the degree of interest in the product is measured and updated based on the distance between the position on the plane in the store where the person to be tracked is located and the product displayed at the nearest position (step S108). .

  After that, the CPU 32 captures the image of the person to be tracked before it is captured by the camera closest to the selected product or before being captured by the camera closest to the selected product. Based on whether the person has left the store, it is determined whether or not it is time to end the tracking of the person (step S109).

  If it is determined that the image of the person to be tracked has not yet been acquired by the camera closest to the selected product but has been acquired by another camera and has not yet left the store (No in step S109), the CPU 32 Returns to the processing from step S107 again.

  In this way, the processes in steps S107 to S109 are repeatedly executed, and the person to be tracked is displayed on the route in the store through which the person to be tracked passes until it reaches the selected product or is determined to have left the store. Accumulate interest in various products.

  In step S109, the person to be tracked is photographed with the camera closest to the selected product, or the image of the person to be tracked cannot be acquired before being photographed with the camera nearest to the selected product. When it is determined that the person has left the store (Yes in Step S109), the CPU 32 collects a series of information on the degree of interest that has been accumulated up to that point and forms a file, and information on the product selected by the person to be tracked The contents are stored in the content management memory 20 (step S110).

  FIG. 5 illustrates, as the flow line ML2 of the person, the route from the person to be tracked selecting “product Z” with the signage device 10 to the position where the “product Z” is actually displayed. It is a figure to do. In the figure, it can be understood that the person to be tracked was still interested in the products C and D.

  Here, when the degree of interest is quantified, it is assumed that the degree of proximity to the product and the time of proximity are represented by an integrated value from a predetermined distance threshold value. It is considered that the degree of interest in the product D is clearly high in the order of “product D> product C”.

  In FIG. 3, based on the content stored in the content management memory 20 by the CPU 32, the content of the guidance content stored in the content management memory 20 is updated and stored as necessary (step S <b> 111). Is returned to the process from step S101.

  In the above step S111, the CPU 32 finds a product that can be judged to have a high degree of interest in common with other products in the process in which many people who have selected a product reach the selected product. If it exists, the content of the guidance content is updated so that other products are also advertised.

  FIG. 6 is a diagram illustrating an example of an image of guidance content projected on the signage board SB with respect to a tracking target person who has selected the product Z with the signage device 10. Here, when many people who have selected the product Z are highly interested in other products in the order of “product A> product C> product E”, the route of the route GL1 indicated by a broken line in the figure. Is projected on the signage board SB as an image of guidance content.

  In addition, when many people who have selected the product Z are highly interested in other products in the order of “product C> product B> product D”, the route GL2 indicated by a solid line in the figure. Is projected on the signage board SB as an image of guidance content.

  Furthermore, when many of the persons who have selected the product Z are highly interested in other products in the order of “product E> product D> product A”, the route GL3 indicated by a solid line in FIG. Is projected on the signage board SB as an image of guidance content.

  In this way, it is possible to extract a hobby or preference tendency from the interest level of the subject person or another person and reflect it in the content of the guidance content.

  As described above in detail, according to the present embodiment, it is possible to present the guidance content required by the user.

  Moreover, in the said embodiment, although the guidance content with respect to the goods selected with the signage apparatus 10 shall be shown, you may show to other areas, such as each said route | root, together with another different guidance object in this case By impressing other guidance objects with unexpectedness, a high advertising effect can be obtained.

  Furthermore, in the said embodiment, since the user's flow line was acquired as action information, it is easy to acquire a user's unconscious natural action as action information.

  In addition, since the route to the position where the selected target product is displayed is presented as the guidance content, a route that is more suitable for the selected guidance target is presented to the user. it can.

  Further, as shown in FIG. 6 above, the degree of congestion in the facility at that time, the presence of other products in the campaign, the degree of interest of the user, such as a guidance method particularly when a plurality of routes can be set It is possible to present to the user a route that seems to be optimal from a larger number of options.

  Also, in the above embodiment, when a guidance target other than the guidance target selected by the user is presented together, the content is updated according to the tendency of many users, so that more users can be persuaded. It is possible to continue presenting powerful guidance objects.

  Furthermore, in the above embodiment, since the degree of interest in not only the flow of the user but also the exhibited product is measured, more demand is extracted from the unconscious behavior of the user and presented. It can be used for guidance content.

  In that case, the specific degree of interest is measured for each different guidance object based on the time that the user has been located within a certain distance range. It can be easily constructed by operating the camera device in conjunction.

  In the above-described embodiment, the content of the guidance content stored in the content management memory 20 is appropriately updated and changed according to the tendency of the user's interest level, thereby presenting a guidance target that reflects the interest level of more users. it can.

In the above embodiment, the user's reaction is to perform an operation of reaching the signage device 10 and selecting an arbitrary product. However, the present invention is not limited to this, and the content output by the signage device 10 is watched. It is good also as a user's reaction acquired by a look extraction process etc.
In the above embodiment, the behavior history of the determined visitor is determined and stored, but the present invention is not limited to this. Recognizing a user who has reacted, and before and after the recognized user's reaction You may save action history.

  In the above embodiment, a case has been described in which the signage device 10 using the projector system of the DLP (registered trademark) (Digital Light Processing) method is used as the device for presenting the guidance content. It does not limit the device itself or the like.

  In the above embodiment, the image information from the plurality of cameras to the signage device 10 via the wireless LAN antenna 39, the feature data of the face image obtained by the face recognition of the person from the image information, and the position in the image are displayed. The information indicating the information shown and the distance information from the photographing lens to the face position is acquired. However, the present invention is not limited to this, and the signage device 10 acquires only image information from a plurality of cameras. The information may be acquired by analyzing the image information.

  In addition, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention in the implementation stage. Further, the functions executed in the above-described embodiments may be combined as appropriate as possible. The above-described embodiment includes various stages, and various inventions can be extracted by an appropriate combination of a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, if the effect is obtained, a configuration from which the constituent requirements are deleted can be extracted as an invention.

Hereinafter, the invention described in the scope of claims of the present application will be appended.
[Claim 1]
A reaction accepting means for accepting a user's reaction to the guidance target;
Behavior detecting means for detecting the behavior of the user as behavior information;
An output means for outputting guidance information based on the guidance object reacted by the reaction accepting means and the behavior information detected by the behavior detecting means;
A guidance output device comprising:
[Claim 2]
The guidance output device according to claim 1, wherein the user's reaction to the guidance target is a user's selection operation for the plurality of guidance targets.
[Claim 3]
The guidance output device according to claim 1, further comprising a storage unit that stores the guidance target received by the reaction reception unit and the behavior information detected by the behavior detection unit in association with each other.
[Claim 4]
4. The guidance output device according to claim 1, wherein the guidance information output by the output means includes guidance information related to a guidance object different from the guidance object reacted by the reaction reception means.
[Claim 5]
The guidance output device according to any one of claims 1 to 4, wherein the behavior information is information indicating a flow line of the user.
[Claim 6]
The guidance output device according to any one of claims 1 to 5, wherein the guidance information output by the output means is information indicating a route to the guidance target reacted by the reaction reception means.
[Claim 7]
The guidance output device according to claim 6, wherein the output means changes guidance information indicating a route to the guidance target based on behavior information detected by the behavior detection means.
[Claim 8]
The output means outputs, based on the content stored in the storage means, a guidance object that is included in the guidance information output by the output means and that is different from the guidance object reacted by the reaction reception means. The guidance output device according to claim 3.
[Claim 9]
9. The degree of interest measuring means for measuring the degree of interest for each guidance object different from the guidance object reacted by the reaction accepting means based on the behavior information stored in the storage means. Guidance output device.
[Claim 10]
10. The guidance output device according to claim 9, wherein the interest level measuring unit measures the interest level based on time during which a user is located within a certain distance range for each different guidance target from the behavior information. .
[Claim 11]
The output means outputs a guide object different from the guide object reacted by the reaction accepting means included in the guide information output by the output means based on the degree of interest. 10. The guidance output device according to 10.
[Claim 12]
A reaction acceptance process for accepting a user's reaction to the guidance target;
A behavior detection step of detecting the behavior of the user as behavior information;
An output step for outputting guidance information based on the guidance object reacted in the reaction acceptance step and the behavior information detected in the behavior detection step;
A guide output method characterized by comprising:
[Claim 13]
A program executed by a computer, wherein the computer is
A reaction accepting means for accepting a user's reaction to the guidance object;
Behavior detecting means for detecting the user's behavior as behavior information, and
An output means for outputting guidance information based on the guidance object reacted by the reaction acceptance means and the behavior information detected by the behavior detection means;
A program characterized by functioning as

10 ... signage device,
11 ... device housing,
12 ... Infrared sensor array,
20 ... Content management memory,
21 ... Projected image drive unit,
22 ... Micromirror element,
23 ... light source part,
24 ... Mirror,
25 ... projection lens part,
27. Photography lens part,
28 ... CMOS image sensor,
29 ... A / D converter,
30. Photographed image processing unit,
32 ... CPU,
33 ... Main memory,
34 ... Program memory,
35 ... operation unit,
36 ... sound processing unit,
37 ... Speaker unit,
38 ... Wireless LAN interface (I / F),
39: Wireless LAN antenna,
B1 to B4 ... operation buttons,
GL1 to GL3 ... (guidance) route,
IM: Imaging unit,
ML1, ML2 ... flow line,
SB ... Signage board SP ... Store.

Claims (13)

A reaction accepting means for accepting a user's reaction to the guidance target;
Behavior detecting means for detecting the behavior of the user as behavior information;
An output means for outputting guidance information based on the guidance object reacted by the reaction accepting means and the behavior information detected by the behavior detecting means;
A guidance output device comprising:   The guidance output device according to claim 1, wherein the user's reaction to the guidance target is a user's selection operation for the plurality of guidance targets.   The guidance output device according to claim 1, further comprising a storage unit that stores the guidance target received by the reaction reception unit and the behavior information detected by the behavior detection unit in association with each other.   4. The guidance output device according to claim 1, wherein the guidance information output by the output means includes guidance information related to a guidance object different from the guidance object reacted by the reaction reception means.   The guidance output device according to any one of claims 1 to 4, wherein the behavior information is information indicating a flow line of the user.   The guidance output device according to any one of claims 1 to 5, wherein the guidance information output by the output means is information indicating a route to the guidance target reacted by the reaction reception means.   The guidance output device according to claim 6, wherein the output means changes guidance information indicating a route to the guidance target based on behavior information detected by the behavior detection means.   The output means outputs, based on the content stored in the storage means, a guidance object that is included in the guidance information output by the output means and that is different from the guidance object reacted by the reaction reception means. The guidance output device according to claim 3.   9. The degree of interest measuring means for measuring the degree of interest for each guidance object different from the guidance object reacted by the reaction accepting means based on the behavior information stored in the storage means. Guidance output device.   10. The guidance output device according to claim 9, wherein the interest level measuring unit measures the interest level based on time during which a user is located within a certain distance range for each different guidance target from the behavior information. .   The output means outputs a guide object different from the guide object reacted by the reaction accepting means included in the guide information output by the output means based on the degree of interest. 10. The guidance output device according to 10. A reaction acceptance process for accepting a user's reaction to the guidance target;
A behavior detection step of detecting the behavior of the user as behavior information;
An output step for outputting guidance information based on the guidance object reacted in the reaction acceptance step and the behavior information detected in the behavior detection step;
A guide output method characterized by comprising: A program executed by a computer, wherein the computer is
A reaction accepting means for accepting a user's reaction to the guidance object;
Action detecting means for detecting the user's action as action information; and output means for outputting guidance information based on the guidance object reacted by the reaction accepting means and the action information detected by the action detecting means;
A program characterized by functioning as

Images