JP2016058074A - Behavior analysis device and behavior analysis program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a behavior analysis device and a behavior analysis program for specifying an action that a person has made to an object just before walking away from the object as a factor why the person has not purchased the object.SOLUTION: An information processor 100 for performing behavior analysis includes: a person position detection module 110; an action sequence specification module 120; and an action specification module 130. The person position detection module 110 detects the position of a person with respect to the object. The action sequence specification module 120 specifies the sequence of an action that the person has made to the object. The action specification module 130 specifies the action that the person has made to the object before walking away from the object on the basis of the action sequence specified by the action sequence specification module 120 and the position of the person with respect to the object detected by the person position detection module 110.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a behavior analysis apparatus and a behavior analysis program.

  Patent Document 1 discloses information that associates flow line data and commercial transaction data of the same person with an object to facilitate detailed analysis of the relationship between a person and a product in a specific area in a store area. And a sub-area information storage unit for storing information specifying each sub-area subdivided in the store area, and sub-area information storage when receiving the designation of the sub-area as an analysis condition From the information specifying the designated subarea in the department and each flow line data in the flow line database, extract the flow line data of the person who has passed the designated subarea, referring to the data in the association information storage section, Identify the transaction data associated with the extracted flow line data, and create information that indicates the relationship between the extracted flow line data and the transaction data It discloses to analyze the characteristics of the behavior of a person in the shop area This created information.

  Patent Document 2 has a problem of automatically recognizing a scene of an image, and further a behavior intention of a person in the image, the image processing unit extracts an object in the image acquired by the imaging unit, The attribute of the extracted object (whether or not it is a person, its action if it is a person, its name if it is an object) is determined, and the scene recognition unit selects the most probable scene from the scene candidates based on the attribute of the object. It is disclosed that the selection and estimation unit specifies the action intention that best matches the plurality of possible action intentions based on the action attribute of the person object in the recognized scene.

JP 2009-048229 A Japanese Patent Laid-Open No. 2000-293865

  The present invention provides a behavior analysis apparatus and a behavior analysis program that specify an action performed on a target object immediately before the person leaves the target object as a cause of non-purchase of the target object in the person. The purpose is that.

  The gist of the present invention for achieving the object lies in the inventions of the following items.

  According to the first aspect of the present invention, an action sequence specifying means for specifying a sequence of actions performed by a person on an object and an action included in the action sequence immediately before the person leaves the object. It is an action analysis device having action specifying means for specifying an action performed on an object as a cause leading to a determination that the person does not purchase the object.

  The invention of claim 2 further includes person position detecting means for detecting the position of the person with respect to the object, and the action specifying means is the position of the person with respect to the object detected by the person position detecting means. The behavior analysis apparatus according to claim 1, wherein an action performed on the object immediately before the person leaves the object is specified based on the above.

  The invention according to claim 3 is an action sequence specifying means for specifying a sequence of actions performed by a person on an object, and among the actions included in the action sequence, the person leaves the object. It is an action analysis program for functioning as an action specifying means for specifying an action performed on the target object immediately before as a cause leading to a determination that the person does not purchase the target object.

  The invention of claim 4 is an action sequence specifying means for specifying a sequence of actions performed by a person on an object, and an action of returning the object to an original position among actions included in the action sequence. It is an action analysis device having action specifying means for specifying the immediately preceding action as a cause leading to the decision that the person does not purchase the object.

  According to a fifth aspect of the present invention, an action sequence specifying means for specifying a sequence of actions performed by a person on an object and an action included in the action sequence, the object is placed at an original position. It is an action analysis program for functioning as an action specifying means for specifying an action immediately before returning an action as a cause leading to a decision that the person does not purchase the object.

  According to the behavior analysis apparatus of the first aspect, it is possible to specify an action performed on a target object immediately before the person leaves the target object as a cause of non-purchase of the target object for the person.

  According to the behavior analysis apparatus of the second aspect, an action can be specified based on a position of a person with respect to an object.

  According to the behavior analysis program of the third aspect, it is possible to specify an action performed on a target object immediately before the person leaves the target object as a cause of non-purchase of the target object for the person.

  According to the behavior analysis apparatus of the fourth aspect, the act immediately before the act of returning the target object to the original position can be specified as the cause of non-purchase of the target object in the target person.

  According to the behavior analysis program of the fifth aspect, the act immediately before the act of returning the target object to the original position can be specified as the cause of non-purchase of the target object in the target person.

It is a conceptual module block diagram about the structural example of 1st Embodiment. It is explanatory drawing which shows the system configuration example at the time of implement | achieving 1st Embodiment. It is a flowchart which shows the process example by 1st Embodiment. It is explanatory drawing which shows the process example by 1st Embodiment. It is a conceptual module block diagram about the structural example of 2nd Embodiment. It is explanatory drawing which shows the system configuration example at the time of implement | achieving 2nd Embodiment. It is explanatory drawing which shows the example of a data structure of a behaviour / POS correlation table. It is explanatory drawing which shows the process example by 2nd Embodiment. It is explanatory drawing which shows the process example by 2nd Embodiment. It is a conceptual module block diagram about the structural example of 3rd Embodiment. It is explanatory drawing which shows the system configuration example at the time of implement | achieving 3rd Embodiment. It is a flowchart which shows the process example by 3rd Embodiment. It is explanatory drawing which shows the process example by 3rd Embodiment. It is explanatory drawing which shows the process example by 3rd Embodiment. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

Hereinafter, examples of various preferred embodiments for realizing the present invention will be described with reference to the drawings.
<First Embodiment>
FIG. 1 is a conceptual module configuration diagram of a configuration example according to the first embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. This means that the device is controlled so as to be stored. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point. When there are a plurality of “predetermined values”, they may be different values, or two or more values (of course, including all values) may be the same. In addition, the description having the meaning of “do B when it is A” is used in the meaning of “determine whether or not it is A and do B when it is judged as A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is also included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

The information processing apparatus 100 that performs behavior analysis according to the first embodiment identifies a person's action. As shown in the example of FIG. 1, a person position detection module 110, an action sequence identification module 120, an action A specific module 130 is included.
The information processing apparatus 100 is used, for example, to specify customer behavior in a store. More specifically, it is possible to grasp important behaviors that have led to the purchase or non-purchase of products. For example, in a series of actions at the customer's sales floor, it may be used to raise the importance and notify that the action taken just before leaving is more important as the cause of the decision to purchase or not purchase .
In addition to the customer's behavior at the store, the customer's behavior at the exhibition, the behavior of the pedestrian in digital signage, etc. in the city, the behavior of the participant in the poster session, etc. may be targeted. In the following description, a customer's behavior in a store and a product on a shelf in a sales floor will be described as an example.

The person position detection module 110 is connected to the action sequence identification module 120. The person position detection module 110 detects the position of the person with respect to the object. For example, a person is extracted from a photographed image by a camera provided at a predetermined position. As a method for extracting a person from an image, an existing technique such as a face image recognition technique or a person detection technique for specifying the position of a person may be used. Accordingly, the position of the person can be specified from the position of the provided camera, the position of the captured image (position of the shooting location specified from the camera direction, angle, etc.), and the position of the person in the image. . Further, by reading information in an IC card, a portable terminal, etc. possessed by a person (for example, ID (IDentification) etc. specifying the owner) with a reader provided at a predetermined position, the person's The position may be specified. These cameras and reading devices are installed in a store, for example. In this case, the specific target person is generally a customer. It is not necessary to specify the person's name or the like, as long as the person is specified and the position where the person is located can be specified. Further, the direction of the person's face, the line of sight of the person, the position of the person's hand, and the like may be detected.
Further, when the object is at a predetermined position (such as a product on a shelf in the sales floor described above), it is not necessary to specify the position of the object. That is, the relationship between the predetermined position of the object and the position of the person can be detected.
Moreover, you may make it recognize a target object by an object recognition technique. The object may be provided with a tag, and information specifying the object may be embedded, and the information in the tag may be read by the reading device. As a result, the relationship between the position of the object and the position of the person can be detected.

The action sequence identification module 120 is connected to the person position detection module 110 and the action identification module 130. The action sequence identification module 120 identifies a sequence of actions performed by a person on an object. Here, the “sequence of actions” means “the order of actions”. As an “act”, for example, stopping or leaving in front of an object, seeing the object, touching the object, picking up the object, returning the object to its original position, other person (eg, accompanying person) , Salesclerks, etc.) and seeing tags attached to the object (for example, tags displaying prices, ingredients, origins, etc.). These actions may use existing techniques such as hand recognition using human detection techniques, face or line-of-sight direction, and conversation recognition using a microphone or the like.
Then, the date and time (year, month, day, hour, minute, second, second or less, or a combination thereof) when these actions are performed is recorded. Thereby, the sequence of actions can be specified.

The action specifying module 130 is connected to the action sequence specifying module 120. Based on the action sequence specified by the action sequence specifying module 120 and the position of the person relative to the object detected by the person position detection module 110, the action specifying module 130 Identify actions taken on the object. The specification here specifies an action that affects the object, and specifies an action that meets a predetermined condition. For example, in the case of a store, when purchasing (or not purchasing) a target product, an important action (determining whether to purchase or not purchase) Act that causes or for the main cause). In this case, as “an action that matches a predetermined condition”, for example, an action immediately before leaving may be specified, or an action that is performed first after stopping may be specified. Further, the action may be specified by weighting according to the number of times of the action, the time when the action was performed (duration), and the like. Further, the act of returning the target object to the original position as the “act immediately before leaving” may be excluded. In that case, the “act immediately before leaving” is the “act immediately before the act of returning the object to the original position”.
And, for example, in a series of actions at a person's sales floor, the action that was just taken away is more important as the cause of the decision to purchase or not purchase. You may make it do.

FIG. 2 is an explanatory diagram illustrating an example of a system configuration when the first embodiment is realized.
The information processing apparatus 100, the camera 210a, the camera 210b, and the camera 210c are connected via a communication line 290, respectively. The communication line 290 may be wireless, wired, or a combination thereof, for example, an intranet as a communication infrastructure, the Internet, or the like. For example, the camera 210 is installed in a store and photographs a person. The information processing apparatus 100 receives an image from the camera 210, analyzes a person in the image, and identifies an action that affects the object.

FIG. 3 is a flowchart illustrating a processing example according to the first exemplary embodiment.
In step S302, it is detected that a person has stopped by.
In step S304, the date and time when the user visited was saved.
In step S306, it is determined whether or not an action has been performed. If so, the process proceeds to step S308. Otherwise, the process proceeds to step S310.
In step S308, the action type and time are stored, and the process returns to step S306. The types of action here are as follows.
・ Behavior of one person Look at the product Look around around ・ Behavior with the product in hand View the product at hand Swipe the product upside down Turn the product upside down with both hands Spread the product with both hands Match clothes and other products to your body Multiple Compare products / Action with others Talk with friends

In step S310, it is determined whether or not the person has left. If the person has left, the process proceeds to step S312; otherwise, the process returns to step S306.
In step S312, the date and time of leaving is stored.
In step S314, an action that matches a predetermined condition is detected and stored from the action sequence.

FIG. 4 is an explanatory diagram illustrating a processing example according to the first exemplary embodiment. As actions of a person, in order, “stop” 410 (entering a predetermined area in the store), “action to product (removal)” 420, “view product” 430, “consultation with accompanying person” 440, “Consultation with store clerk” 450, “Consultation with accompanying person” 460, “Action (return) to product” 470, “Leaving” 480 (to leave the area) Is shown. These are specified as action sequences by the action sequence specifying module 120. In particular, in “view merchandise” 430, the act was performed once, and the time was 7 seconds. In “Consultation with the store clerk” 450, the act was performed once, and the time was 45 seconds. In “Consultation with a companion” 460, the act was performed twice, for a total of 156 seconds. Further, the stay time from “stop” 410 to “leave” 480 is 305 seconds, and it can be seen that the action for the product was performed twice.
The action identification module 130 estimates an important action from within this action sequence. For example, the following actions are specified. In the case of the predetermined condition “weight the action performed immediately before leaving”, “consultation with accompanying person” 460 is specified. In the case of the predetermined condition “weighting the first action performed by a stop-by”, “view product” 430 is specified. In the case of a predetermined condition “weighted by number of times and time”, “consultation with accompanying person” 460 is specified.
Further, “stop” 410 and “leave” 480 may be excluded as acts to be specified. This is because “stop” 410 and “leave” 480 are not suitable for the determination of purchase or non-purchase. Note that it is not necessary to distinguish between two types of actions for a product, such as “action for product (retrieve)” 420 and “action for product (return)” 470. Since it is difficult to separate and return the product by image processing, it is processed as one. In this case, you may exclude the action to goods from the action used as specific object.

<Second Embodiment>
FIG. 5 is a conceptual module configuration diagram of a configuration example according to the second embodiment.
The information processing apparatus 500 includes a person position detection module 110, an action sequence identification module 120, an action identification module 130, and a sales relationship analysis module 540. The person position detection module 110 includes a face identification module 510. In addition, the same code | symbol is attached | subjected to the site | part of the same kind as the above-mentioned embodiment, and the overlapping description is abbreviate | omitted (hereinafter the same).
The face identification module 510 recognizes the face of the person detected by the person position detection module 110 and determines whether or not the person is a predetermined person (for example, a store clerk). A person other than the target person is the second person, for example, a store clerk or a companion. In addition, the face image of the store clerk is taken in advance, and it is determined whether or not the person in the image is a store clerk.
The person position detection module 110 also extracts an action performed between the target person and another person (for example, a store clerk or a companion).
The action specifying module 130 is connected to the action sequence specifying module 120 and the sales relationship analysis module 540. Based on the action sequence, the position of the person with respect to the object, and the relationship between the person and the predetermined second person, the action identifying module 130 may determine whether the person has left the object before leaving the object. Identify actions taken against. Here, for example, when the target person is a “customer” and the second person is a “clerk”, the “relationship between the person and the predetermined second person” is “the clerk and the customer”. From this relationship and the actions performed between these persons, for example, “the action of the target person” such as “the customer consulted the store clerk” can be specified.
The sales relationship analysis module 540 is connected to the action specifying module 130. The sales relationship analysis module 540 identifies an action of the person based on the purchase information of the person who is the object. For example, an action may be specified according to the price of a product that is the object.

FIG. 6 is an explanatory diagram illustrating an example of a system configuration when the second embodiment is realized.
The information processing apparatus 500, the camera 210a, the camera 210b, the camera 210c, and the sales management apparatus 600 are connected to each other via a communication line 290. The sales management device 600 is, for example, a POS (Point Of Sale system), and passes information related to a product, purchase information of a target person, and the like to the information processing device 500. The information processing apparatus 500 receives an image from the camera 210, analyzes a person in the image, and uses the information from the sales management apparatus 600 to identify an action that affects the object.

FIG. 7 is an explanatory diagram showing an example of the data structure of the behaviour / POS correlation table 700.
The behavior / POS correlation table 700 includes a target product column 710, a column 720 for picking up a product at the beginning, a column 730 for picking up a product at the end, a consultation time column 740 with a store clerk, a consultation time column 750 with a companion, Access count column 760, and stores the degree of correlation with sales. The sales relationship analysis module 540 uses the behavior / POS correlation table 700 to identify an action having a high degree of correlation with sales.
The target product column 710 stores products. The column 720 for picking up a product at the beginning stores the degree of correlation between the act of “picking the product for the first time” and sales. The column 730 for taking a product at the end stores the degree of correlation between the act of “taking a product at the end” and sales. The consultation time column 740 with a store clerk stores the degree of correlation between “consultation time with a store clerk” and sales. The counselor's consultation time column 750 stores the degree of correlation between “consultation time with accompanying person” and sales. The item access count column 760 stores the degree of correlation between “product access count” and sales. Note that “beginning” in the column 720 for picking up the product first means within a predetermined period from the “stop” 410, and “end” in the column 730 for picking up the product in the end is “leave” 480. Within the previous predetermined period. Further, the consultation time column 740 with the store clerk and the consultation time column 750 with the companion may be divided into the first half (the first half in the period from “stop” 410 to “leave” 480) and the second half. And you may make it provide the column of a some action (the order may be included).
In each column, “◎” (high correlation degree), “◯”, “Δ”, “×” (no correlation) is described as a symbol.
For example, when the product A with a low price is the target, the act of “first picking up the product” is weighted higher than other acts, and the “consultation time with the companion” is given. , Weighting higher than other actions (“Δ”, “×”) and specifying more important actions. It also identifies whether the action performed in the first half or the second half of the action sequence is more important for the same action.
Conversely, when the analysis result by the sales relationship analysis module 540 is accumulated, the actual sales number for each product may be extracted and the behavior / POS correlation table 700 may be corrected. Then, if it is found that the correlation between the low-priced product A and the act of “getting the product first” is high, the product A is “Product POP (Point Of Purchase advertising, advertising medium in store)”. The message may be presented so as to make improvements such as “emphasizing” and “making it easy to pick up”. In addition, if it is found that the correlation between the high-priced and high-quality product B and the “consultation time with the store clerk” is high, a message is given to the product B to make improvements such as “increase the number of store clerk”. You may make it show.

8 and 9 are explanatory diagrams illustrating an example of processing according to the second embodiment. Further, the action sequence identification module 120 may use an action feature amount. The action sequence identification module 120 does not need to extract all the actions (for example, viewing a product), and may be divided on the time axis from the motion on the video using a video indexing technique. . Then, a moving image feature amount (for example, up and down movement of the hand) in an action considered to be important may be acquired based on the moving image feature amount on the divided time axis.
FIG. 8 shows, as actions of a person, “stop” 810, “action to product (removal)” 820, “up and down movement of hand” 830, “consultation with a store clerk” 840, “spread hands left and right”. 850 shows an example in which the action “Leave” 860 is performed.
FIG. 9 shows the actions of a person in order of “stop” 910, “action on product (removal)” 920, “move his / her neck left / right” 930, “spread his hand left / right” 940, “action on product” In this example, the actions of (return) ”950 and“ leave ”960 are performed.
“Spreading hands to the left and right” (actually spreading the product and viewing) is specified as an important action (“stretching hands to the left and right” 850, “spreading hands to the left and right” 940).

<Third Embodiment>
FIG. 10 is a conceptual module configuration diagram of a configuration example according to the third embodiment.
The information processing apparatus 1000 includes a person position detection module 110, an action sequence identification module 120, an action identification module 130, and a mobile terminal operation detection module 1010.
For example, the information processing apparatus 1000 detects information browsed on the mobile terminal in cooperation with the operation log of the mobile terminal at the sales floor. Examples of the portable terminal include a mobile phone (including a smartphone), a wearable computer, and the like.
The mobile terminal operation detection module 1010 is connected to the action specifying module 130. The mobile terminal operation detection module 1010 extracts the operation of the mobile terminal by the person that is associated with the action of the person. If an act of holding a mobile device over an object (photographing the tag of the object (product) with a camera) is detected, or if a visit is made to a homepage that contains information about the object, The mobile terminal operation detection module 1010 is activated when a notification that there is an operation from the mobile terminal is received, or when transmission of an operation log is received from the mobile terminal. And the operation performed in the period of the action sequence specified by the action sequence specification module 120 in the portable terminal is extracted.
The action specifying module 130 is connected to the action sequence specifying module 120 and the mobile terminal operation detecting module 1010. The action specifying module 130 specifies an action of the person based on the operation of the person's mobile terminal extracted by the mobile terminal operation detection module 1010. For example, when the operation is an operation for displaying the price of the object, the action (operation) is specified.

FIG. 11 is an explanatory diagram showing an example of a system configuration when the third embodiment is realized.
The information processing apparatus 1000, the mobile terminal 1110a, the mobile terminal 1110b, the mobile terminal 1110c, the product information presentation device 1120a, and the product information presentation device 1120b are connected via a communication line 1190, respectively. The communication line 1190 may be wireless, wired, or a combination thereof, and may be, for example, the Internet or an intranet as a communication infrastructure. In the store, the information processing apparatus 1000, the camera 210a, the camera 210b, and the camera 210c are connected via a communication line 290, respectively. A person in the store uses the mobile terminal 1110 to access the product information presentation device 1120. The information processing apparatus 1000 detects the access and identifies an action based on the access.

FIG. 12 is a flowchart illustrating a processing example according to the third exemplary embodiment.
In step S1202, it is determined whether or not there has been an operation on the mobile terminal in the action. If there is, the process proceeds to step S1204, and otherwise the process ends (step S1299).
In step S1204, an operation on the portable terminal is detected.
In step S1206, the correlation between the action, the action sequence, and the operation on the mobile terminal is detected. The correlation detection process will be described later using the examples of FIGS.
In step S1208, it is determined whether or not there is a mobile terminal that matches the target person. If there is, the process proceeds to step S1210. Otherwise, the process ends (step S1299).
In step S1210, an operation on the mobile terminal is extracted.

FIG. 13 is an explanatory diagram illustrating a processing example according to the third exemplary embodiment. The behavior of the person at the sales floor is photographed by the camera 210 and analyzed. From this analysis result (action sequence) and the operation log of the mobile terminal (may be received from the mobile terminal 1110 or from the product information presentation device 1120), it is detected who is operating the mobile terminal. And an action is specified also using operation in a portable terminal.
FIG. 13A shows, as actions of the person A in the image taken by the camera 210, “stop” 1310A, “sales tag scanning operation” 1320A, “mobile terminal operation start” 1330A, “mobile terminal operation” In this example, “end” 1340A, “consideration of product in hand” 1350A, and “leave” 1360A are performed.
FIG. 13B shows the actions of the person B in the image taken by the camera 210 in order of “stop” 1310B, “sale tag scanning operation” 1320B, “mobile terminal operation start” 1330B, “mobile terminal operation”. In this example, “end” 1340B and “leave” 1350B are performed.
FIG. 13C shows, as operations in the mobile terminal 1110a, in order “Sale Tag Scan” 1310C, “Mobile Terminal Operation (Price Display)” 1320C, “Mobile Terminal Operation (Product Information Display)” 1330C, “Mobile Terminal Screen” An example in which “OFF” 1340C is performed is shown.
FIG. 13D shows, as operations in the mobile terminal 1110b, in order, “Sales floor tag scan” 1310D, “Mobile terminal operation (product information display)” 1320D, “Mobile terminal operation (price display)” 1330D, “Mobile terminal screen” An example in which “OFF” 1340D is performed is shown.
In these examples, since the person A and the mobile terminal 1110a have a high correlation, matching (association between the person A and the mobile terminal 1110a) is performed. And since person B and portable terminal 1110b have high correlation, matching (association of person B and portable terminal 1110b) is performed. Here, a high correlation means that the date and time when the action was performed are the same. In addition, as a match here, the difference of both generation date and time should just be less than or below a predetermined value.

There are the following as techniques for matching actions on video and operations on mobile terminals.
[Action 1] A scanning action of a tag of a product at a sales floor by a portable terminal 1110 possessed by a person When a customer has a product of interest at a sales floor, the customer scans with a camera built in the mobile terminal 1110 Get product information. For example, a two-dimensional barcode is scanned.
Product information (price, detailed product description, etc.) is displayed on the mobile terminal 1110, and the customer browses the information while operating the smartphone.
As an action detection here, an action on the camera is detected. For example, an operation of reaching for the tag position on the sales floor is detected. And as detection in the portable terminal 1110, tag ID and time scanned with the portable terminal 1110 are detected.
[Action 2] Behavior of operation of portable terminal 1110 by person Scans the displayed product information by scanning with a camera built in portable terminal 1110. Related information such as product name, price, producer, and word of mouth is displayed.
As an action detection here, an action on the camera is detected. For example, the head and hand positions are detected, and the act of looking at the hand is detected. And as a detection in the portable terminal 1110, the touch to the portable terminal 1110, the screen off of the portable terminal 1110, and the movement (specifically, acceleration etc.) of the portable terminal are detected.
Using these detected information, the action of the person and the operation on the portable terminal 1110 are matched.

  FIG. 14 is an explanatory diagram illustrating a processing example according to the third exemplary embodiment. The action in FIG. 13A and the operation in the mobile terminal 1110 in FIG. 13C are matched, and “mobile terminal operation (product information display)” 1330C is specified as an action that affects the purchase of the object. That is, an operation on the mobile terminal 1110 is also a target of an action that can be specified. In “mobile terminal operation (product information display)” 1330C, product information was browsed once, and the viewing time was 15 seconds. As a result of the time weighting, “mobile terminal operation (product information display)” 1330C is specified.

  Note that the hardware configuration of the computer on which the program according to the present embodiment is executed is a general computer as illustrated in FIG. 15, specifically, a personal computer, a computer that can be a server, or the like. That is, as a specific example, the CPU 1501 is used as a processing unit (calculation unit), and the RAM 1502, the ROM 1503, and the HD 1504 are used as storage devices. As the HD 1504, for example, a hard disk or an SSD (Solid State Drive) may be used. CPU 1501 for executing programs such as the person position detection module 110, the action sequence identification module 120, the action identification module 130, the face identification module 510, the sales relationship analysis module 540, the portable terminal operation detection module 1010, etc., and the programs and data are stored. For user operations on a RAM 1502, a ROM 1503 storing a program for starting the computer, an HD 1504 which is an auxiliary storage device (may be a flash memory, etc.), a keyboard, a mouse, a touch panel, etc. A receiving device 1506 for receiving data based on the data, an output device 1505 such as a CRT or a liquid crystal display, and a communication line interface 1507 for connecting to a communication network such as a network interface card. Then, and a bus 1508 for exchanging data by connecting them. A plurality of these computers may be connected to each other via a network.

Among the above-described embodiments, the computer program is a computer program that reads the computer program, which is software, in the hardware configuration system, and the software and hardware resources cooperate with each other. Is realized.
Note that the hardware configuration shown in FIG. 15 shows one configuration example, and the present embodiment is not limited to the configuration shown in FIG. 15, and is a configuration capable of executing the modules described in the present embodiment. I just need it. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line In addition, a plurality of systems shown in FIG. 15 may be connected to each other via communication lines so as to cooperate with each other. In particular, in addition to personal computers, information appliances, copiers, fax machines, scanners, printers, and multifunction machines (image processing apparatuses having two or more functions of scanners, printers, copiers, fax machines, etc.) Etc. may be incorporated.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, or a wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

The above-described embodiment may be grasped as follows.
[A] a person position detecting means for detecting the position of the person relative to the object;
An action sequence specifying means for specifying a sequence of actions performed by the person on the object;
An action identifying means for identifying an action performed by the person on the object before the person leaves the object based on the action sequence and the position of the person relative to the object;
A behavior analysis device.
[B] The action specifying unit further specifies an action of the person based on a relationship between the person and the second person.
The behavior analysis apparatus according to [A].
[C] The action specifying means further specifies the action of the person based on the purchase information of the person.
The behavior analysis apparatus according to [A] or [B].
[D] further comprising extraction means for extracting an operation of the portable terminal by the person associated with the person's action,
The action specifying means further specifies an action of the person based on an operation of the portable terminal of the person.
The behavior analysis apparatus according to any one of [A] to [C].
[E]
A person position detecting means for detecting the position of the person relative to the object;
An action sequence specifying means for specifying a sequence of actions performed by the person on the object;
An action identifying means for identifying an action performed by the person on the object before the person leaves the object based on the action sequence and the position of the person relative to the object;
Behavior analysis program to function as
[F] Action sequence specifying means for specifying a sequence of actions performed by a person on an object;
Action specifying means for specifying an action that matches a predetermined condition among actions included in the action sequence as an action that affects the object;
A behavior analysis device.
[G] Furthermore, it has person position detecting means for detecting the position of the person with respect to the object,
As the predetermined condition, an action of the person who first visited the object, an action performed immediately before the person left the object, an action performed by the person on the object Weighting is performed by either the number of times or time.
The behavior analysis apparatus according to [F].
[H]
An action sequence specifying means for specifying a sequence of actions performed by a person on an object;
Action specifying means for specifying an action that matches a predetermined condition among actions included in the action sequence as an action that affects the object;
Behavior analysis program to function as

According to the behavior analysis apparatus of [A], it is possible to specify an action that the person has performed on the object before the person leaves the object.
According to the behavior analysis apparatus of [B], an action of the person can be specified based on the relationship between the person and the second person.
According to the behavior analysis apparatus of [C], a person's action can be specified based on the person's purchase information.
According to the behavior analysis apparatus of [D], a person's action can be specified based on operation of a person's portable terminal.
According to the behavior analysis program [E], it is possible to specify an action that the person has performed on the object before the person leaves the object.
According to the behavior analysis apparatus of [F], it is possible to specify an action that affects a target object.
According to the behavior analysis apparatus of [G], the behavior of the person who first visited the object, the behavior performed immediately before the person left the object, the number of times or the time that the person performed on the object It is possible to specify an action that affects an object on the condition that weighting is performed with any of the above.
According to the behavior analysis program [H], it is possible to identify an action that affects a target object.

DESCRIPTION OF SYMBOLS 100 ... Information processing apparatus 110 ... Person position detection module 120 ... Action sequence specification module 130 ... Action specification module 210 ... Camera 290 ... Communication line 500 ... Information processing apparatus 510 ... Face identification module 540 ... Sales relationship analysis module 600 ... Sales management apparatus DESCRIPTION OF SYMBOLS 1000 ... Information processing apparatus 1010 ... Mobile terminal operation detection module 1110 ... Mobile terminal 1120 ... Merchandise information presentation apparatus 1190 ... Communication line

Claims (5)

An action sequence identification means for identifying a sequence of actions performed by a person on an object;
Among the actions included in the action sequence, identify an action performed on the object immediately before the person leaves the object as a cause leading to a determination that the person does not purchase the object, An action analysis device having action specifying means. Furthermore, it has a person position detecting means for detecting the position of the person with respect to the object,
The action specifying means specifies an action performed on the object immediately before the person leaves the object based on the position of the person relative to the object detected by the person position detecting means. The behavior analysis apparatus according to claim 1. Computer
An action sequence identification means for identifying a sequence of actions performed by a person on an object;
Among the actions included in the action sequence, identify an action performed on the object immediately before the person leaves the object as a cause leading to a determination that the person does not purchase the object, Action identification means,
Behavior analysis program to function as An action sequence identification means for identifying a sequence of actions performed by a person on an object;
An action specifying means for specifying an action immediately before an action of returning the object to an original position among actions included in the action sequence as a cause leading to a decision that the person does not purchase the object; Having behavior analysis device. Computer
An action sequence identification means for identifying a sequence of actions performed by a person on an object;
An action specifying means for specifying an action immediately before an action of returning the object to an original position among actions included in the action sequence as a cause leading to a decision that the person does not purchase the object;
Behavior analysis program to function as

Images