JP2020166523A - Interest level information acquisition system, information processing device, and information processing program - Google Patents

Abstract

To grasp the level of interest of a person in an object of interest based on an image even if a high-definition image cannot be acquired.SOLUTION: An interest level information acquisition system comprises: an imaging unit 202 that is installed to photograph people passing near an object of interest and generates imaging data by photographing; a detection tracking unit 104 that detects and tracks people included in the imaging data; a speed determination unit 106 that determines the moving speed of the tracked people; an orientation determination unit 108 that determines the orientation of the tracked people; and an interest level information acquisition unit 110 that acquires interest level information indicating the level of interest for the object of interest of the tracked people based on the movement speed and direction.SELECTED DRAWING: Figure 2

Description

The present invention relates to an interest level information acquisition system that acquires interest level information indicating a person's interest level in an object of interest, and an information processing device and an information processing program used therein.

Conventionally, there are known devices and the like for grasping a customer's interest in a product by analyzing an image of a customer in the store.

Patent Document 1 proposes an information processing device that can more accurately grasp a product of interest to a customer who travels around the store by detecting the direction of the line of sight of a person. Further, in Patent Document 2, a learning model is used as a teacher signal for information indicating whether or not a customer has purchased a customer's face or body feature in a still image of a place where a product is advertised. Information processing devices and information processing programs to be created have been proposed.

JP-A-2017-117384 Japanese Patent No. 6428062

Conventional information processing devices have grasped a customer's interest in a product based on the direction of the customer's line of sight and the characteristics of the customer's face or body in a still image. However, for relatively large objects of interest such as event booths, event spaces, store layouts, food stalls, shops, mobile stores, etc., in order to grasp from the image how much people passing by are interested in the objects of interest, a camera It is necessary to install the camera at a relatively long distance or a wide angle of view and take a bird's-eye view of a wide area including a relatively large object of interest and its surroundings.

In such a case, it may not be possible to obtain a high-definition image that can be recognized up to the direction of the human line of sight. Even if the image quality of the image is good, if the image does not show a person's face, the direction of the person's line of sight cannot be recognized in the first place.

In addition, the fact that a person stops moving or slows down the movement speed in front of an object of interest can be analyzed from a continuous image overlooking a wide area, and is interested based on the analysis of the movement speed of such a person. It is also possible to measure the degree of interest in the object. However, it is not possible to determine whether the movement speed is reduced due to interest in the object of interest or the movement speed is reduced due to other circumstances only by the analysis of the movement speed.

In addition, some people pass by in front of the object of interest without slowing down, but may show interest by facing the object of interest. It is not possible to grasp the degree of interest of such a person only by analyzing the movement speed.

The present invention has been made in view of the above problems, and even when a high-definition image cannot be obtained, an interest level information acquisition system and information capable of grasping a person's degree of interest in an object of interest based on the image. An object of the present invention is to provide a processing device and an information processing program.

Another object of the present invention is to grasp a person's interest in an object of interest in stages as well as the presence or absence thereof.

In order to achieve the above object, the present invention is an interest level information acquisition system, which is installed so as to photograph a person passing near an object of interest, and includes an imaging unit that generates imaging data by imaging, and imaging data. Based on a detection tracking unit that detects and tracks a person, a speed determination unit that determines the movement speed of the tracked person, an orientation determination unit that determines the direction of the tracked person, and a movement speed and direction. It is provided with an interest level information acquisition unit that acquires interest level information indicating the degree of interest of the tracked person's interest target.

With this configuration, interest degree information can be acquired based on the movement speed and direction that reflect the interest degree of the person's interest. Also, with this configuration, interest level information is acquired based not only on the moving speed of the person being tracked, but also on the moving speed and direction, so whether the person was interested in the object of interest and reduced the moving speed. , It is possible to determine whether the movement speed has been reduced due to other circumstances. In addition, with this configuration, although the moving speed does not decrease in front of the object of interest, the degree of interest of the person who has shown interest can be grasped by facing the object of interest.

The interest level information acquisition unit may acquire interest level information based on the time during which the tracked person continues in a predetermined direction.

With this configuration, for example, it is possible to obtain interest level information by distinguishing between a person who is interested for a moment and a person who has taken a close look.

The interest level information acquisition unit may acquire interest level information based on the change in the moving speed of the tracked person.

With this configuration, for example, when the speed decreases in front of the object of interest, it can be determined that the degree of interest is high.

The interest level information acquisition unit may further acquire interest level information based on the position of the tracked person.

With this configuration, by combining the position and orientation of the person being tracked, the direction in which the person is facing can be specified in the real space, and it can be determined whether or not the person is interested in the object of interest.

The interest level information acquisition unit may acquire interest level information indicating the degree of interest in three or more stages.

With this configuration, a person's interest in an object of interest can be grasped step by step as well as the presence or absence thereof.

The interest level information acquisition department may acquire the number of people for each stage.

With this configuration, for example, it is possible to analyze that the degree of interest is low but many people are interested, or that there are few people who are interested but the degree of interest is high.

The orientation determination unit may determine the orientation of the person by identifying three or more points of the person.

With this configuration, even when a high-definition image that can recognize the direction of the person's line of sight cannot be obtained, the direction of the person that reflects the degree of interest in the object of interest of the person can be determined. Further, with this configuration, it is possible to determine the orientation of the person that reflects the degree of interest in the object of interest of the person even when the face of the person does not appear in the image.

The orientation determination unit may determine the orientation of a person included in the imaging data by using a learning model obtained by learning the orientation of a person using a plurality of sample images.

With this configuration, since the orientation of the person included in the image is determined by using the learning model obtained by learning the orientation of a plurality of people, the accuracy of determining the orientation of the person can be improved.

In order to achieve the above object, the present invention includes an information processing device, an input unit that receives input of imaging data from a camera installed to photograph a person passing near an object of interest, and imaging data. Based on a detection tracking unit that detects and tracks a person, a speed determination unit that determines the movement speed of the tracked person, an orientation determination unit that determines the orientation of the tracked person, and the movement speed and orientation. It is provided with an interest level information acquisition unit that acquires interest level information indicating the degree of interest of the tracked person's interest target.

With this configuration, interest degree information can be acquired based on the movement speed and direction that reflect the interest degree of the person's interest. In addition, due to this configuration, the movement speed could not be determined only from the movement speed of the person who can analyze from the continuous image from a bird's-eye view of a wide area. Whether the person was interested in the object of interest and reduced the movement speed, or the movement speed was reduced due to other circumstances. Can be determined whether or not the In addition, with this configuration, although the moving speed does not decrease in front of the object of interest, the degree of interest of the person who has shown interest can be grasped by facing the object of interest.

In order to achieve the above object, the present invention is an information processing program in which a computer is used as an input unit, a detection tracking unit, a speed determination unit, an orientation determination unit, and an interest level information acquisition that constitute the information processing device described above. Make it function as a department.

It is a figure which shows the outline of the interest degree information acquisition system which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the information processing apparatus which concerns on one Embodiment of this invention, and a camera. It is a figure which shows the human body model used for determining the face direction of a person which concerns on one Embodiment of this invention. It is a flow chart which shows the operation of the information processing apparatus which concerns on 1st Embodiment of this invention. It is a flow chart which shows the operation which concerns on the evaluation process of the information processing apparatus which concerns on 1st Embodiment of this invention. (A) It is a top view which shows the outline of the positional relationship between a person and a camera which concerns on 1st Embodiment of this invention, and (b) is for determination of interest degree information of a person which concerns on 1st Embodiment of this invention. It is a figure which shows an example of the image used. It is a flow chart which shows the operation which concerns on the evaluation process of the information processing apparatus which concerns on 2nd Embodiment of this invention. (A) It is a top view which shows the outline of the positional relationship between a person and a camera which concerns on 2nd Embodiment of this invention, and (b) is for determination of interest degree information of the person which concerns on 2nd Embodiment of this invention. It is a figure which shows an example of the image used.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the embodiments described below show an example of the case where the present invention is carried out, and the present invention is not limited to the specific configuration described below. In carrying out the present invention, a specific configuration according to the embodiment may be appropriately adopted.

[Overview of interest level information acquisition system]
FIG. 1 is a diagram showing an outline of an interest level information acquisition system according to an embodiment of the present invention. The information processing device 10 is a computer that processes information on a moving image including a person 30 photographed by the camera 20. The camera 20 transmits information on a moving image including a person 30 to the information processing device 10, and the information processing device 10 receives this information.

The camera 20 may be installed anywhere as long as it can take a bird's-eye view of the person 30 passing near the object of interest. The fact that the person 30 can be photographed from a bird's-eye view means that the camera 20 and the person 30 are separated to the extent that the information processing device 10 can determine the direction and the moving speed of the person 30. By taking a bird's-eye view of the person 30, it becomes easier to grasp the change in the behavior of the person 30. In the present embodiment, the camera 20 is installed in an event booth, an event space, a store layout, a stall, a store, a mobile store, etc. (hereinafter, referred to as "event") in which a plurality of people 30 can be photographed from a bird's-eye view. .. Further, in the present embodiment, the object of interest of the person 30 is an event.

[Information processing device configuration]
FIG. 2 is a block diagram showing a configuration of an information processing device 10 and a camera 20 according to an embodiment of the present invention. The information processing device 10 includes a control calculation unit 102, an input unit 112, a storage unit 114, and an output unit 116. The control calculation unit 102 includes a detection and tracking unit 104, a speed determination unit 106, a direction determination unit 108, and an interest level information acquisition unit 110. The camera 20 has an imaging unit 202.

The control calculation unit 102 is a control calculation device that realizes the functions of the detection and tracking unit 104, the speed determination unit 106, the orientation determination unit 108, the interest level information acquisition unit 110, the input unit 112, the storage unit 114, and the output unit 116. The control calculation unit 102 has a CPU (Central Processing Unit). The control calculation unit 102 may further have a GPU (Graphics Processing Unit). The control calculation unit 102 is telecommunications connected to each unit. The control calculation unit 102 realizes the functions of each unit by executing the program stored in the storage unit 114.

The detection and tracking unit 104 detects the person 30 included in the plurality of continuous images by analyzing the plurality of continuous images using the captured data as a plurality of continuous images (hereinafter, each image is also referred to as a “frame”). It is a detection and tracking means for tracking. The image may be a two-dimensional image or a three-dimensional image.

The detection and tracking unit 104 detects the person 30 when a new person 30 frames in the moving image. The detection and tracking unit 108 assigns an ID to the person 30 who has framed in, and stores the position of the person 30 in the storage unit 114 in association with the ID. After that, the detection and tracking unit 104 repeats the process of detecting the person 30 at a similar position in the moving image and associating it with the ID. This tracks the person 30 until it is framed out of the video.

The person 30 may be, for example, one person or a plurality of people. When the number of persons 30 is a plurality of persons, the detection and tracking unit 104 may track all of the plurality of persons or a part of the plurality of persons. In this embodiment, the detection and tracking unit 104 tracks all 30 people who are framed in the moving image.

A known algorithm may be used as the algorithm for detecting the person 30 of the detection tracking unit 104. Known algorithms may be, for example, those using Har-like features, HOG (Histograms of Oriented Gradients) features, support vector machines (SVM), and Faster R-CNNs (Faster Regions with Convolutional Neural Networks). ), YOLO (You Only Look Once), MobileNet-SSD (Mobil Net Single Shot MultiBox Detector) and the like may be used. In this embodiment, the MobileNet-SSD is used to detect the person 30.

As a technique for tracking the person 30 of the detection tracking unit 104, a known technique may be used. The known technique may be, for example, a tracking technique using a Kalman filter, a tracking technique using Deep SORT, or the like. In this embodiment, the person 30 is tracked by a tracking technique using a Kalman filter.

The speed determination unit 106 is a speed determination means for determining the moving speed of the tracked person 30 by analyzing the information of the moving image. The speed determination unit 106 calculates the moving speed of the person 30 by, for example, dividing the distance between the position of the person 30 in one frame and the position of the person 30 in the next frame by the time per frame. You can. At this time, in order to obtain accurate position information of the person 30, the speed determination unit 106 may acquire the three-dimensional position information of the person 30 by making the subject three-dimensional. The speed determination unit 106 may determine the change in the movement speed of the person 30 by repeating the calculation of the movement speed of the person 30, for example.

The orientation determination unit 108 is an orientation determination means for determining the orientation of the tracked person by analyzing the information of the moving image.

In a moving image taken from a bird's-eye view, a high-definition image that can be recognized up to the line-of-sight direction of the person 30 may not be obtained. Even if the image quality of the image is good, if the face of the person 30 is not reflected in the image, the line-of-sight direction of the person 30 cannot be recognized in the first place. Even at this time, in order to determine whether or not the person 30 is facing the event direction, it is necessary to determine the direction of the person 30 using not only the face of the person 30 but also a part other than the face. Therefore, the orientation determination unit 108 determines, for example, the orientation of the body or the orientation of the face (hereinafter, referred to as “face direction”) as the orientation of the person 30 detected and tracked by the detection and tracking unit 104.

The orientation determination unit 108 may use a known algorithm. The orientation determination unit 108 includes, for example, a support vector machine (SVM), Faster R-CNNs (Faster Regions with Convolutional Neural Networks), YOLO (You Only Look Once), MobileNet-SSD (Mobil Net Single Shot MultiBox Detector), and the like. A deep learning algorithm may be used to learn the orientation of the person 30 using a plurality of sample images, and the obtained learning model may be used to determine the orientation of the person 30 included in the image.

The orientation determination unit 108 is based on, for example, an interest degree determination angle formed by the direction toward the camera 20 starting from the position of the person 30 and the face direction 32 of the person 30 obtained by using the learning model, as will be described later. Therefore, it may be determined whether or not the face direction 32 of the person 30 is the event direction.

Further, the orientation determination unit 108 estimates the positions of body parts such as the eyes, nose, and shoulders of the person 30 shown in the moving image, and determines the face direction 32 of the person 30 from some of these points. The orientation of the person 30 may be determined by using a possible posture estimation technique. In the present embodiment, the face direction 32 of the person 30 is determined by using the posture estimation technique.

A method for determining the face direction 32 of the person 30 according to the present embodiment will be specifically described with reference to FIG. FIG. 3 is a diagram showing a human body model used for determining a person's face direction 32 according to an embodiment of the present invention. The orientation determination unit 108 estimates, for example, the positions of the facial organs and joints of the person 30. In the present embodiment, the orientation determination unit 108 estimates the positions of the nose, neck, left and right eyes, ears, shoulders, elbows, wrists, hips, knees, and ankles of the person 30 shown in the moving image.

When the orientation determination unit 108 can estimate the position of the facial organ of the person 30, for example, the face direction 32 of the person 30 may be determined using the estimated position including the estimated position of the facial organ of the person 30. When the position of the facial organ of the person 30 cannot be estimated, for example, the orientation of the body of the person 30 may be determined using the estimated position of the joint, and this body orientation may be regarded as the face direction 32 of the person 30. Further, the estimated position to be used may be any number as long as it is 3 points or more that can determine the face direction 32 of the person 30. For example, when the eyes, nose, and ears can be grasped, the face direction 32 of the person 30 can be determined based on their estimated positions. If the positions of facial organs such as eyes, nose, and ears cannot be estimated, for example, the body of a person 30 may be determined by the estimated positions of three points on the neck and shoulders, or the estimated positions on four points on both shoulders and waist. It is also possible to determine the direction of the face and set this as the face direction 32.

Next, whether or not the person 30 is facing the event direction is determined by, for example, the angle formed by the direction toward the camera 20 starting from the position of the person 30 and the face direction 32 of the person 30 (hereinafter, "interest determination angle"). The judgment may be made based on. The interest level information acquisition unit 110 determines that the face direction 32 of the person 30 is the event direction, for example, when the interest level determination angle is 0 ° or a value close to 0 ° (hereinafter, referred to as “θ ₀ ”). You can do it. On the other hand, the interest level information acquisition unit 110 may determine that the face direction 32 of the person 30 is not the event direction if the interest level determination angle is other than θ ₀ (hereinafter, referred to as “θ _x ”).

The explanation will be continued by returning to FIG. The interest level information acquisition unit 110 is an interest level information acquisition means that acquires interest level information indicating the interest level of the person 30's interest target based on the moving speed and direction of the person 30 by analyzing the information of the moving image. is there.

When considering the change in the behavior of the person 30 passing near the event, when the moving speed of the person 30 is slow and the person 30 is facing the event direction, the person 30 is considered to be interested in the event. On the contrary, when the moving speed of the person 30 is not slowed down and the person 30 is not facing the event direction, it is considered that the person 30 is not interested in the event. Therefore, the interest level information acquisition unit 110 acquires interest level information indicating the degree of interest in the event of the person 30 based on the movement speed and direction of the person 30.

In addition, the interest level information acquisition unit 110 may acquire interest level information by using another index that can reflect the interest level of the person 30 for the event. The interest level information acquisition unit 110 may acquire interest level information based on, for example, at least one of the time during which a predetermined direction is continued, the change in the moving speed of the person 30, and the position of the person 30.

Similar to the above, when considering the change in the behavior of the person 30 passing near the event, when the person 30 is facing the event for a long time and stops at a position close to the event, the person 30 is relatively interested in the event. It is considered expensive. On the contrary, when the person 30 is facing the event for a short time and moves to a position far from the event without decelerating, it is considered that the person 30 has relatively low or no interest in the event. Therefore, the interest level information acquisition unit 110 may acquire interest level information based on, for example, at least one of the time during which the predetermined direction is continued, the change in the moving speed of the person 30, and the position of the person 30.

The time during which the predetermined orientation is continued, the change in the moving speed of the person 30, and the position of the person 30 may be appropriately set to values that can be evaluated as the person 30 being interested in the event. The value set to a value that can be evaluated that the person 30 is interested in the event may be set based on, for example, the closeness between the position of the person 30 and the central position of the event, or the position of the person 30. And may be set based on the position and proximity of the person representing the event (for example, the organizer, the store owner, etc.).

The interest level information acquired by the interest level information acquisition unit 110 is information that reflects the subjectivity such as the degree of interest, the degree of interest, and the degree of attention of the person 30 with respect to the event. The interest level information acquisition unit 110 acquires interest level information indicating the degree of interest in three or more stages. In the present embodiment, the interest level information is information shown in four stages of "interest / strong", "interest / medium", "interest / weak", and "interest / no" described below.

Hereinafter, the degree of interest information according to the present embodiment will be specifically described. In the present embodiment, the degree of interest in the event of the person 30 is determined by using whether or not the person 30 has continued θ ₀ for 2 seconds or more and whether or not the person 30 has decelerated and stopped. Further, the position of the person 30 is used as an index to determine the degree of interest of the person 30 in the event.

When the person 30 continues and stops θ ₀ for 2 seconds or more, the interest level information acquisition unit 110 determines that the person 30 is interested in the event. At this time, the interest level information acquisition unit 110 acquires "interest / strong" as the interest level information.

On the other hand, when the person 30 continues θ ₀ for 2 seconds or more but only slows down the moving speed, the interest level information acquisition unit 110 determines that the person 30 is slightly interested in the event. At this time, the interest level information acquisition unit 110 acquires "interest / medium" as the interest level information.

Further, using the position of the person 30 as an index, when the person 30 continues θ ₀ for 2 seconds or more and passes the position near the event, but the movement speed does not slow down, the interest level information acquisition unit 110 may perform the operation. It is determined that the person 30 is slightly interested in the event. At this time, the interest level information acquisition unit 110 acquires "interest / weak" as the interest level information.

When none of the above conditions is satisfied, the interest level information acquisition unit 110 determines that the person 30 is not interested in the event, and acquires "interested / not interested" as the interest level information.

Further, the interest level information acquisition unit 110 may output, for example, the number of people related to the interest level information. In the present embodiment, the interest level information acquisition unit 110 outputs the number of 30 people determined to be "interest / strong", "interest / medium", "interest / weak", and "interest / none" described above. ..

The input unit 112 is an input means for receiving input of imaging data from a camera 20 installed so as to photograph a person passing near an object of interest. The method in which the input unit 112 accepts the input of image information may be, for example, a method using a storage medium, a method using wireless communication, or a method using wired communication.

The method using a recording medium may be, for example, a method using a flash memory, a CD-ROM, or a DVD-ROM. The wireless communication method may be, for example, the Internet, a wireless LAN such as Wi-Fi (registered trademark), or a short-range wireless communication method such as Bluetooth (registered trademark) or NFC (Near Field Communication). In the present embodiment, the method of receiving the input of the moving image information by the input unit 112 is a method by wireless communication.

The storage unit 114 is a memory capable of storing data. The memory may be, for example, a volatile memory (for example, RAM) or a non-volatile memory (for example, ROM). The storage unit 104 may be built in the information processing device 10 as shown in FIG. 2, or may be externally attached to the information processing device 10.

The storage unit 114 stores a program for the control calculation unit 102 to realize the functions of each unit. Further, the storage unit 114 stores the ID of the person 30 for the detection and tracking unit 104 to detect and track, and the position associated with the ID. Further, the storage unit 114 stores the moving speed of the person 30, the change in the moving speed, the degree of interest information, the number of people 30 having the degree of interest, and the like.

The output unit 116 is an output means for outputting the interest level information acquired by the interest level information acquisition unit 110. The output unit 116 is, for example, a display or a printer.

The imaging unit 202 of the camera 20 is an imaging device that is installed so as to photograph a person passing near an object of interest and generates imaging data by photographing. The technique used by the imaging unit 202 may be any known technique as long as it is a technique for generating data capable of determining the direction of a person. This technology uses, for example, 3D data such as CCD (Charge-Coupled Device) image sensors and CMOS (Complementary Metal Oxide Semiconductor) image sensors that generate 2D data such as continuous still images and moving images, and continuous 3D images. A generated 3D sensor (Depth Sensor), an event-driven sensor that detects and outputs a change in brightness included in the event camera, or the like may be used. The image capturing unit 202 inputs image information to the input unit 112. In the present embodiment, the imaging unit 202 inputs image information to the input unit 112 by a method by wireless communication.

[Operation of the information processing device according to the first embodiment]
Hereinafter, a flow for realizing the operation of the information processing device 10 according to the first embodiment will be described with the configuration of the information processing device 10 described above.

FIG. 4 is a flow chart showing the operation of the information processing apparatus 10 according to the first embodiment of the present invention. The flow according to the first embodiment is whether or not the interest rate determination angle θ ₀ of the person 30 continues for 2 seconds or more, that is, whether or not the face direction 32 of the person 30 faces the event direction for 2 seconds or more. This is a flow for determining the degree of interest of the person 30 in the event by using whether or not the moving speed of the 30 has slowed down or stopped as an index. When the person 30 is framed into the moving image and detected, the flow starts.

First, the information processing device 10 tracks the person 30 who has framed in the moving image (step S102), and determines whether or not the person 30 has framed out (step S104).

When the information processing device 10 determines that the person 30 has framed out (step S104: Yes), the flow ends.

When the information processing device 10 determines that the person 30 is not out of frame (step S104: No), it determines whether or not the face direction 32 of the person 30 faces the event direction for 2 seconds or more (step S106). When the information processing device 10 determines that the person 30 has turned to the event direction for 2 seconds or more (step S106: Yes), the information processing device 10 performs the evaluation process and then continues the tracking (step S108, step S102). On the other hand, when the information processing device 10 determines that the person 30 is not facing the event direction for 2 seconds or more, the tracking is continued without performing the evaluation process (step S106: No, step S102). The information processing device 10 repeats this flow until the person 30 frames out.

FIG. 5 is a flow chart showing an operation related to the evaluation process of the information processing apparatus 10 according to the first embodiment of the present invention. FIG. 6A is a top view showing an outline of the positional relationship between the person 30 and the camera 20 according to the first embodiment of the present invention, and FIG. 6B is the first embodiment of the present invention. It is a figure which shows an example of the image used for determining the interest degree information of the person 30 concerned. Hereinafter, the flow of the evaluation process will be described with reference to FIGS. 5 and 6 (a) and 6 (b). When the information processing device 10 determines that the person 30 faces the event direction for 2 seconds or more, the flow starts.

First, the information processing device 10 determines whether or not the person 30 has stopped (step S202).

When the information processing device 10 determines that the person 30 has stopped (step S202: Yes), the information processing device 10 determines that the person 30 is interested in the event, and acquires "interest / strong" as interest level information (step S204). The flow ends.

For example, in FIGS. 6A and 6B, the information processing apparatus 10 determines that the interest level determination angle 32A of the person 30A continues θ ₀ for 2 seconds or more, and the person 30A is stopped. Then, the information processing device 10 determines that the person 30A is interested in the event, and acquires "interest / strong" as the interest level information. Then, the information processing device 10 counts up the number of people 30 who are "interested / strong" by one person.

The explanation will be continued by returning to FIG. When the information processing device 10 determines that the person 30 has not stopped (step S202: No), it determines whether or not the person 30 has reduced the moving speed (step S206). When the information processing device 10 determines that the person 30 has slowed down (step S206: Yes), the information processing device 10 determines that the person 30 is slightly interested in the event, acquires "interest / medium" as interest degree information, and performs a flow. finish. On the other hand, when the information processing device 10 determines that the person 30 is not decelerating (step S206: No), the information processing device 10 determines that the person 30 is not interested in the event, and outputs the interest level information as "interested / not interested". , End the flow.

For example, in FIGS. 6A and 6B, the information processing apparatus 10 determines that the interest level determination angle 32B of the person 30B continues θ ₀ for 2 seconds or more, but the person 30B only slows down the moving speed. To do. Then, the information processing device 10 determines that the person 30B is slightly interested in the event, and acquires "interest / medium" as the interest level information. Then, the information processing device 10 counts up the number of people 30 who are "interested / medium" by one person.

On the other hand, for example, in FIG. 6 (a), (b) , the information processing apparatus 10, the degree of interest determination angle 32C of human 30C is despite continued theta ₀ or 2 seconds, the human 30C did not even slow down the moving speed judge. Then, the information processing device 10 determines that the person 30C is not interested in the event, and acquires "interested / not interested" as the interest level information. Then, the information processing device 10 counts up the number of people 30 who are "interested / not" by one person.

From the above, it is said that the information processing device 10 has one person 30A who is "interested / strong", one person 30B who is "interested / medium", and one person 30C who is "interested / not". Get interest level information.

As described above, in the first embodiment of the present invention, it is possible to acquire information on the degree of interest in the event of the person 30 based on the change and the direction of the movement speed of the person 30. Further, in the first embodiment of the present invention, it is possible to acquire information on the degree of interest in the event of the person 30 by using the time during which the direction of the person 30 is continued as an index. As a result, it was not possible to distinguish from only the movement speed of the person who can analyze from the continuous image overlooking a wide area, whether the person was interested in the object of interest and reduced the movement speed, or the movement speed was reduced due to other circumstances. You can tell if it was. In addition, with this configuration, although the moving speed does not decrease in front of the object of interest, the degree of interest of the person who has shown interest can be grasped by facing the object of interest.

Further, in the first embodiment according to the present invention, information on the degree of interest in the event of the person 30 can be acquired in three stages. As a result, the interest level information can be given a gradation, and more detailed interest level information can be obtained. Further, in the first embodiment of the present invention, the number of people 30 who are interested in the event can be grasped.

[Operation of the information processing device according to the second embodiment]
The flow of the information processing device 10 according to the second embodiment of the present invention is different from the flow of the information processing device 10 according to the first embodiment of the present invention, and further, by indexing the position of the person 30, the person It differs in determining the degree of interest in the 30 events. In the following, only the flow related to the evaluation process, which is a difference, will be described, and the description of the flow other than the configuration of the information processing device 10 and the evaluation process according to the second embodiment will be omitted.

FIG. 7 is a flow chart showing an operation related to the evaluation process of the information processing apparatus 10 according to the second embodiment of the present invention. FIG. 8A is a top view showing an outline of the positional relationship between the person 30 and the camera 20 according to the second embodiment of the present invention, and FIG. 8B is a second embodiment of the present invention. It is a figure which shows an example of the image used for determining the interest of the person 30 concerned. In the following, the flow of the evaluation process will be described with reference to FIGS. 7 and 8 (a) and 8 (b). When the information processing device 10 determines that the moving speed of the person 30 has slowed down, the flow starts.

First, the information processing device 10 determines whether or not the person 30 has stopped (step S302).

When the information processing device 10 determines that the person 30 has stopped (step S302: Yes), the information processing device 10 determines that the person 30 is interested in the event, and acquires "interest / strong" as interest level information (step S304). The flow ends.

For example, in FIGS. 8A and 8B, the information processing apparatus 10 determines that the interest level determination angle 32D of the person 30D continues θ ₀ for 2 seconds or more, and the person 30D is stopped. Then, the information processing device 10 determines that the person 30D is interested in the event, and acquires "interest / strong" as interest level information. Then, the information processing device 10 counts up the number of people 30 who are "interested / strong" by one person.

The explanation will be continued by returning to FIG. When the information processing device 10 determines that the person 30 has not stopped (step S302: No), it determines whether or not the person 30 has reduced the moving speed (step S306). When the information processing device 10 determines that the person 30 has slowed down (step S306: Yes), the information processing device 10 determines that the person 30 is slightly interested in the event, and acquires "interest / medium" as interest degree information (step S308). ), End the flow.

For example, in FIGS. 8A and 8B, the information processing apparatus 10 determines that the interest degree determination angle 32E of the person 30E continues θ ₀ for 2 seconds or more, but the person 30E only slows down the moving speed. To do. Then, the information processing device 10 determines that the person 30E is slightly interested in the event, and acquires "interest / medium" as the interest level information. Then, the information processing device 10 counts up the number of people 30 who are "interested / medium" by one person.

The explanation will be continued by returning to FIG. When the information processing device 10 determines that the person 30 has not decelerated (step S306: No), it determines whether or not the person 30 has passed near the event (step S310). When the information processing device 10 determines that the person 30 has passed near the event (step S310: Yes), the information processing device 10 determines that the person 30 is slightly interested in the event, and acquires "interest / weak" as interest level information. (Step S312), the flow is terminated.

On the other hand, when the information processing device 10 determines that the person 30 has not passed near the event (step S310: No), the information processing device 10 determines that the person 30 is not interested in the event, and sets the interest level information as "interested / not". Output (step S314) and end the flow.

For example, in FIGS. 8A and 8B, it is said that the information processing device 10 has passed the event near the event without decelerating, although the interest determination angle 32F of the person 30F has continued to be θ ₀ for 2 seconds or more. judge. Then, the information processing device 10 determines that the person 30F is slightly interested in the event, and acquires "interest / weak" as interest level information. Then, the information processing device 10 counts up the number of people 30 who are "interested / weak" by one.

On the other hand, for example, in FIGS. 8A and 8B, although the information processing device 10 continued the interest determination angle 32G of the person 30G for 2 seconds or more θ ₀ , the person 30G did not decelerate and passed near the event. Judge that it did not. Then, the information processing device 10 determines that the person 30G is not interested in the event, and acquires "interested / not interested" as the interest level information. Then, the information processing device 10 counts up the number of people 30 who are "interested / not" by one person.

From the above, the information processing device 10 includes one person 30D who is "interested / strong", one person 30E who is "interested / medium", one person 30F who is "interested / weak", and "interested".・ Acquire interest level information that 30G is one person who is "none".

As described above, unlike the first embodiment of the present invention, the second embodiment of the present invention can determine the degree of interest of the person 30 in the event by using the position of the person 30 as an index. As a result, in the second embodiment of the present invention, it is possible to obtain more detailed information on the degree of interest in the event of the person 30 by using the position as an index in addition to the moving speed and direction of the person 30 and the duration of the direction. ..

[Modification example]
In any of the embodiments, as long as the person 30 does not frame out, the evaluation process for determining the degree of interest of the person 30 in the event may be repeated. At this time, for the same person 30, the interest level information acquisition unit 110 may determine the degree of interest during the first evaluation process and may not determine the degree of interest during the second and subsequent evaluation processes. The degree of interest may also be determined during the evaluation process after the second time. When determining the degree of interest in the second and subsequent evaluation processes, the interest is determined multiple times for the same person 30, so the determination result of the degree of interest may be output multiple times, and the interest level may be output multiple times. The determination result of the degree of interest may be output only once by increasing the evaluation of the degree of degree such as "interest / strong x 3", "interest / strong x 2", and the like.

10 ... Information processing device 20 ... Cameras 30, 30A to G ... People 32A to G ... Interest level determination angle 102 ... Control calculation unit 104 ... Detection tracking unit 106 ... Speed Judgment unit 108 ... Direction determination unit 110 ... Interest level information acquisition unit 112 ... Input unit 114 ... Storage unit 116 ... Output unit 202 ... Imaging unit

Claims (10)

An imaging unit that is installed to photograph people passing near the object of interest and generates imaging data by imaging,
A detection tracking unit that detects and tracks a person included in the imaged data,
A speed determination unit that determines the movement speed of the tracked person,
An orientation determination unit that determines the orientation of the tracked person,
An interest level information acquisition unit that acquires interest level information indicating the degree of interest of the person to be interested in the tracked person based on the movement speed and the direction.
Interest level information acquisition system equipped with. The interest level information acquisition system according to claim 1, wherein the interest level information acquisition unit acquires the interest level information based on the time during which the tracked person continues in a predetermined direction. The interest level information acquisition system according to claim 1 or 2, wherein the interest level information acquisition unit acquires the interest level information based on a change in the movement speed of the person who has been tracked. The interest level information acquisition system according to any one of claims 1 to 3, wherein the interest level information acquisition unit further acquires the interest level information based on the tracked position of the person. The interest level information acquisition system according to any one of claims 1 to 4, wherein the interest level information acquisition unit acquires the interest level information indicating the degree of interest in three or more stages. The interest level information acquisition system according to claim 5, wherein the interest level information acquisition unit acquires the number of people for each stage. The interest level information acquisition system according to any one of claims 1 to 6, wherein the orientation determination unit determines the orientation of the person by identifying three or more points of the person. Any one of claims 1 to 7, wherein the orientation determination unit determines the orientation of a person included in the imaging data by using a learning model obtained by learning the orientation of a person using a plurality of sample images. Interest level information acquisition system described in. An input unit that accepts input of imaging data from a camera installed to photograph people passing near the object of interest,
A detection tracking unit that detects and tracks a person included in the imaged data,
A speed determination unit that determines the movement speed of the tracked person,
An orientation determination unit that determines the orientation of the tracked person,
An interest level information acquisition unit that acquires interest level information indicating the degree of interest of the person to be interested in the tracked person based on the movement speed and the direction.
Information processing device equipped with. An information processing program for causing a computer to function as the input unit, the detection and tracking unit, the speed determination unit, the orientation determination unit, and the interest level information acquisition unit that constitute the information processing device according to claim 9.

Images