WO2019188079A1

WO2019188079A1 - Person transit time measurement system and person transit time measurement method

Info

Publication number: WO2019188079A1
Application number: PCT/JP2019/008995
Authority: WO
Inventors: 小倉　慎矢
Original assignee: 株式会社日立国際電気
Priority date: 2018-03-26
Filing date: 2019-03-07
Publication date: 2019-10-03
Also published as: JPWO2019188079A1; JP7108022B2

Abstract

The present invention provides a technique whereby a transit time of a person from an entrance to an exit can be measured more precisely. An entrance side image processing unit 121 performs a process of calculating a feature amount of each person on the basis of an entrance side image. An exit side image processing unit 122 performs a process of calculating a feature amount of each person on the basis of an exit side image; a process of determining whether the same person as the person included in the exit side image is included in the entrance side image by comparing the feature amount calculated from the entrance side image with the feature amount calculated from the exit side image; and a process of recording, if the same person as the person included in the exit side image is determined to be included in the entrance side image, a difference between a captured time of the entrance side image and a captured time of the exit side image as the transit time of the person.

Description

Person transit time measuring system and person transit time measuring method

The present invention relates to a person passage time measurement system that measures the passage time of a person from one point to another point.

Traditionally, in facilities such as airport security checkpoints, commercial facilities, art museums, amusement parks, and event venues, it is possible to measure the passing time of people from one point to another and use it as analytical data to improve customer satisfaction. In addition, there is a human transit time measurement system that presents a guide to the current waiting time to a customer.

For example, in Patent Document 1, the estimated processing time of each person obtained by multiplying the reference time by a coefficient corresponding to the characteristics of each person lined up at the airport security checkpoint is lined up at the security checkpoint. Calculating the estimated total processing time of a person.
For example, in Patent Document 2, when acquiring the dwell time required for movement in the measurement area for each moving body based on the flow line of each moving body in the measurement area, a missing portion of the flow line in the measurement area is disclosed. It is disclosed that the missing part is replaced with the residence time of another person when there is a problem.
For example, in Patent Document 3, the total extension of each matrix is calculated based on a matrix line generated for each matrix for a monitoring area where one or more naturally occurring matrixes or one or more guideway matrixes exist. It is disclosed to calculate the waiting time of a matrix based on the total extension.

Japanese Patent No. 5960864 Japanese Patent No. 5789976 JP 2007-317052 A

The conventional technology cannot accurately measure the passing time of a person from one point to another, and a technique for measuring the passing time of a person with higher accuracy has been demanded. In addition, there is a problem that accurate data of visitors cannot be obtained if measurement is performed including a specific person such as a security guard or a related person. Furthermore, in the mechanism for measuring the transit time including the matrix, when only the representatives of the group are arranged in the matrix and the remaining persons of the group join from the middle, the transit time of the person who joined the group from the middle, There was also a problem that the measurement time was shorter than the actual transit time. In addition, there is a problem that the passing time of persons who have been arranged in the queue but have been rearranged after leaving the queue is measured longer than the passing time in the actual congestion situation.

The present invention has been made in view of the above-described conventional circumstances, and provides a technique capable of measuring the passage time of a person from one point to another with higher accuracy. Objective.

In the present invention, in order to achieve the above object, the person passing time measuring system is configured as follows.
That is, in a person passage time measurement system that measures a passage time required for a person to pass from a first point to a second point, a feature amount of the person included in the first image obtained by imaging the first point is calculated. 1 calculating means, second calculating means for calculating a feature amount of a person included in the second image obtained by imaging the second point, calculated from the feature amount calculated from the first image and the second image A determination means for comparing the feature quantity and determining whether the same person as the person included in the second image is included in the first image; and the same person as the person included in the second image is And a recording unit that records a difference between the shooting time of the first image and the shooting time of the second image as the passing time of the person when it is determined that the image is included in the first image. And

Such a configuration makes it possible to accurately measure the passage time required for the person to pass from the first point to the second point. Further, even when there are a plurality of persons, each person can be identified and the passing time can be individually measured. Furthermore, it is not necessary to track the movement of the person between the first point and the second point.

Note that the shooting time of the first image and the shooting time of the second image are not required to be the time when each image is shot by the imaging device, but may be a time at which they can be viewed. . That is, the time when the image was transmitted from the imaging device, the time when the image arrived at the first calculation unit or the second calculation unit, the time when the feature amount was extracted by the first calculation unit or the second calculation unit, or the like may be used. . In short, it is only necessary to be able to specify the difference between the shooting times of the first image and the second image, and it is only necessary that the time is the same for the first image and the second image.

Here, when the difference between the shooting time of the first image and the shooting time of the second image exceeds a first threshold, or when the difference is less than a second threshold smaller than the first threshold, It is preferable to exclude the recording as the passage time.
This makes it possible to exclude records of passage times that deviate significantly from the standard passage time. In other words, it is possible to exclude people who have taken actions that cause noise in the transit time survey, such as people who have been interrupted in the queue or those who have been rearranged in the queue, and the reliability of the survey results Can be increased.

The image processing apparatus further includes a storage unit that stores a feature amount of a person who is not subject to measurement of the passage time, and compares the feature amount calculated from the first image with the feature amount stored in the storage unit. It is preferable that the person included in the first image is excluded from the measurement target of the transit time when the similarity is equal to or greater than a predetermined value.
As a result, it is possible to exclude persons who do not need to investigate the transit time, such as security guards and related persons, from the survey target, and it is possible to increase the reliability of the survey results.

According to the present invention, it is possible to measure the passage time of a person from one point to another with higher accuracy.

It is a figure which shows the structural example of the person passage time measuring system which concerns on 1st Example of this invention. It is a schematic diagram of the measurement environment of person passage time It is a figure which shows the example of the process sequence of the entrance side image process part in 1st Example. It is a figure which shows the example of the process sequence of the exit side image process part in 1st Example. It is a figure which shows the structural example of the person passage time measuring system which concerns on 2nd Example of this invention. It is a figure which shows the example of the process sequence of the entrance side image process part in 2nd Example. It is a figure which shows the example of the process sequence of the exit side image process part in 2nd Example.

A person passing time measurement system according to an embodiment of the present invention will be described with reference to the drawings. In the following, a description will be given of a case where a person measures the transit time required for movement from the entrance to the exit in an environment where there is an entrance and an exit such as an airport security checkpoint or an event venue ticket office.

First Embodiment FIG. 1 shows a configuration example of a person passing time measuring system according to a first embodiment.
As shown in FIG. 1, the person passage time measurement system of this example is configured such that an entrance side imaging device 101A, an exit side imaging device 101B, and a person passage time measurement device 102 are connected to a network 100 and can communicate with each other. Is done.

The network 100 is a communication means for performing communication by connecting devices such as a dedicated network for data communication, an intranet, the Internet, and a wireless LAN.

The entrance-side imaging device 101A and the exit-side imaging device 101B perform digital conversion processing on an image captured by a CCD (Charge-Coupled Device) or CMOS (Complementary-Metal-Oxide-Semiconductor) element, and the converted image data is transferred to the network 100. Network cameras and surveillance cameras that output to a recording device (not shown).

The person passage time measuring device 102 is a device that processes the image data output from the imaging devices 100A and 100B via the network, and measures and records the person passage time. The apparatus is also equipped with a function for extracting and recording a person feature. The person passage time measuring apparatus 102 includes an entrance side image processing unit 121 and an exit side image processing unit 122.

The entrance-side image processing unit 121 performs image processing on the image data output from the entrance-side imaging device 101A. Detailed processing contents will be described later with reference to FIG.
The exit side image processing unit 122 performs image processing on the image data output from the exit side imaging device 101B. Detailed processing contents will be described later with reference to FIG.

FIG. 2 shows a schematic diagram of the measurement environment of the person passing time. FIG. 2 assumes a security checkpoint at an airport or a ticket office at an event venue. The gate 251 is a gate where people form a line, and is actually a security check gate or a ticket window. A plurality of gates 251 may be provided, and the present invention can be applied to a case where the gate 251 is not provided.

Suppose that there are an entrance and an exit in the area where the person passing time is measured. On the entrance side, an entrance-side imaging device 101A is installed so that a person passing through the entrance can be photographed from the front. On the exit side, an exit-side imaging device 101B is installed so that a person passing through the exit can be photographed from the front.

Person 252A is a person whose passage time is to be measured. In the measurement environment of this example, a person 252A photographed by the entrance-side imaging device 101A on the entrance side of the area passes through the gate 251 where many people are lined up, and after a while, the exit-side imaging device 101B on the exit side of the area. The measurement environment is reflected in In FIG. 2, a state where the person 252A has moved to the exit side is indicated by reference numeral 252B.

FIG. 3 shows an example of the processing procedure of the entrance-side image processing unit 121 in the person passage time measuring apparatus 102 of the first embodiment.
The entrance-side image processing unit 121 first performs initialization processing (S301) for securing memory and initializing parameters.
Next, the image data is received from the entrance-side imaging device 101A, and entrance-side image reception processing (S302) that is held in the memory of the person passage time measuring device 102 is performed. When the entrance-side imaging apparatus 101A compresses (encodes) an image and transmits it, a process of converting (decoding) into an uncompressed image after reception is also performed.

Next, person area detection processing (S303) for detecting a person area from the entrance side image is performed. For example, a search is made as to whether an estimated face image is included in the image.
Next, as a result of the person area detection process (S303), a person detection determination process (S304) is performed to determine whether or not a person area has been detected from the entrance-side image. If a person area is detected, the process proceeds to person feature extraction processing (S305). On the other hand, when the person area is not detected, the process proceeds to the entrance-side image reception process (S302) in order to receive a new image.

In the person feature quantity extraction process (S305), the person feature quantity is calculated as a numerical value for each person detected in the person area detection process (S303). In this example, a face feature amount is extracted as a person feature amount.
Next, a person feature / time recording process (S306) is performed in which both the person feature and the current time (entrance passage time) are stored in a memory area that can be used in common with the exit-side image processing unit 122.

Next, an end determination process (S307) for determining whether there is a condition to be ended is performed. If it is a condition to be terminated, a termination process such as releasing the secured memory is performed. If it is not the condition to end, the process proceeds to the entrance side image reception process (S302) in order to receive a new entrance side image. As a condition to be terminated, for example, there is a case where a switch of the person passage time measuring apparatus 102 is turned off by a human hand.

FIG. 4 shows an example of the processing procedure of the exit-side image processing unit 122 in the person passage time measuring apparatus 102 of the first embodiment.
The exit-side image processing unit 122 first performs initialization processing (S401) for securing memory and initializing parameters.
Next, the image data is received from the exit-side imaging device 101B, and the exit-side image reception process (S402) that is held in the memory of the person passage time measuring device 102 is performed. When the exit side imaging apparatus 101B compresses (encodes) an image and transmits it, a process of converting (decoding) it into an uncompressed image after reception is also performed.

Next, a person area detection process (S403) for detecting a person area from the exit side image is performed. For example, a search is made as to whether an estimated face image is included in the image.
Next, as a result of the person area detection process (S403), a person detection determination process (S404) for determining whether a person area is detected from the exit side image is performed. If a person area is detected, the process proceeds to person feature extraction processing (S405). On the other hand, when the person area is not detected, the process proceeds to the exit side image reception process (S402) in order to receive a new image.

In the person feature quantity extraction process (S405), the person feature quantity is calculated as a numerical value for each person detected in the person area detection process (S403). In this example, a face feature amount is extracted as a person feature amount.
Next, the same person determination that compares the person feature amount calculated in the person feature amount extraction process (S405) with the plurality of person feature amounts stored in the person feature amount / time recording process (S306) in FIG. Processing (S406) is performed. That is, the process of searching for the same person as the person arriving at the exit from among a plurality of persons who have passed through the entrance is performed by comparing the person feature obtained at the entrance with the person feature obtained at the exit. Here, for each of the plurality of person feature quantities acquired at the entrance, the similarity with the person feature quantity obtained at the exit is calculated, and the person with the highest similarity (the person with the smallest difference in person feature quantities) is calculated. ) Is the same person as the person arriving at the exit, and the process proceeds to the time difference recording process (S407). Even if the similarity is the highest, if the similarity is less than a predetermined threshold, it is determined that the same person as the person arriving at the exit was not found among the persons who passed through the entrance, and the end determination was made. The process proceeds to processing (S409).

In the time difference recording process (S407), the entrance passage time stored in the person feature amount / time recording process (S306) of FIG. 3 for the person determined to be the same person in the same person determination process (S406) and the current time The difference from the time (exit passage time) is calculated and recorded in the database or the like as the transit time required for the person to pass from the entrance to the exit.
After the calculation of the passage time, the person feature quantity and the entrance passage time relating to the person are no longer necessary, and the person feature quantity deletion process (S408) for deleting the corresponding person feature quantity and time is performed. That is, only the person feature amount and the entrance passage time relating to the person in the area between the entrance and the exit remain in the memory.

Next, an end determination process (S409) for determining whether there is a condition to be ended is performed. If it is a condition to be terminated, a termination process such as releasing the secured memory is performed. If it is not the condition to end, the process proceeds to the exit side image reception process (S402) in order to receive a new exit side image. As a condition to be terminated, for example, there is a case where a switch of the person passage time measuring apparatus 102 is turned off by a human hand.

As described above, in the person passing time measurement system according to the first example, the entrance-side image processing unit 121 performs the process (S302 to S305) of calculating the feature amount of the person from the entrance-side image, and exit-side image processing. In the unit 122, the process of calculating the feature amount of the person from the exit side image (S402 to S405) is compared with the feature amount calculated from the entrance side image and the feature amount calculated from the exit side image. Processing for determining whether the same person as the person included in the image is included in the entrance image (S406), and when it is determined that the same person as the person included in the exit image is included in the entrance image The process of recording the difference between the shooting time of the entrance-side image and the shooting time of the exit-side image as the passing time of the person (S407) is performed.

According to such a configuration, it is possible to accurately measure the passing time required for the person to pass from the entrance to the exit. Further, even when there are a plurality of persons, each person can be identified and the passing time can be individually measured. Furthermore, it is not necessary to track the movement of the person from the entrance to the exit.

In addition, after the passage time is calculated, a process is performed to delete the person feature and the shooting time related to the person.
According to such a configuration, only the person feature amount and the entrance passage time relating to the person in the area between the entrance and the exit remain in the memory, so that the same person determination process is accelerated.

Second Embodiment FIG. 5 shows a configuration example of a person passage time measuring system according to a second embodiment.
The person passage time measurement system of the second embodiment is different from the system configuration (FIG. 1) of the first embodiment in that the person passage time measurement device 102 includes a non-target person feature amount storage unit 123. Yes.
The non-target person feature amount storage unit 123 stores in advance the human feature amount of a person who is not a target in order for the entrance-side image processing unit 121 to determine a person who is not a target for the passage time measurement. .

FIG. 6 shows an example of the processing procedure of the entrance-side image processing unit 121 in the person passage time measuring apparatus 102 of the second embodiment. Processes S301 to S307 in FIG. 6 are the same as the processes (FIG. 3) of the entrance-side image processing unit 121 in the first embodiment, and thus description thereof is omitted.

In the second embodiment, the non-measurement person determination process (S501) is performed after the person feature amount extraction process (S305). In the non-measurement person determination process (S601), a difference between the person feature quantity stored in the non-target person feature quantity storage unit 123 and the person feature quantity calculated in the person feature quantity extraction process (S305) is calculated in advance. Compare with the set reference value (threshold). If the calculated difference between the person feature amounts is smaller than the reference value, it is determined that a person who is not the subject is reflected on the entrance-side imaging apparatus 101A, and the process ends without performing the person feature amount / time recording process (S306). The process proceeds to a determination process (S307).

In this way, it is possible to exclude predetermined specific persons such as security guards and related persons from being measured for passing time. This process may be determined on the exit side. However, if the process is performed on the entrance side, the process in the person feature / time recording process (S306) becomes unnecessary, and the process of the person passing time measuring apparatus 102 cannot be performed. The amount of memory is reduced.

FIG. 7 shows an example of the processing procedure of the exit-side image processing unit 122 in the person passage time measuring apparatus 102 of the second embodiment. Processes S401 to S409 in FIG. 7 are the same as the process of the exit side image processing unit 122 (FIG. 4) in the first embodiment, and thus description thereof is omitted.

In the second embodiment, standard time difference calculation processing (S701) is performed after initialization processing (S401). The standard time difference calculation process (S701) is a process for calculating the standard time difference from the average value of the passage times of a plurality of persons in the past. A plurality of persons serving as samples for calculating the standard time difference can be selected by an arbitrary method. For example, the standard time difference may be calculated from the average value of the passing times of the most recent dozens of people, or the standard time difference may be calculated from the average value of the passing times of dozens of the same past day of the week and the same time zone. May be.

Next, a value obtained by adding a predetermined numerical value to the standard time difference calculated in the standard time difference calculation process (S701) is compared with the elapsed time from the time stored in the person feature amount / time recording process (S306) as a specified upper limit time. A feature amount existence determination process exceeding the specified time (S702) is performed. Since there may be a plurality of times stored in the person feature amount / time recording process (S306), this process is repeated a plurality of times. The predetermined numerical value added to the standard time difference may be a fixed value set in advance, or the standard deviation is calculated when calculating the average value in the standard time difference calculation, and is dynamically changed such as a value obtained by adding 3σ. It may be a value. If there is a person feature quantity whose elapsed time from the storage exceeds the specified upper limit time, the process proceeds to a person feature quantity deletion process (S703). If no such person feature amount exists, the process proceeds to the exit-side image reception process (S402).

In the person feature deletion process (S703), the person feature and time are deleted in the same manner as the person feature deletion process (S408). This is different from the person feature amount deletion processing (S408) in that the person feature amount whose elapsed time from the storage exceeds the specified upper limit time is deleted.

Also, in the second embodiment, after the same person determination process (S406), a less than prescribed time determination process (S704) is performed. In the less than prescribed time determination process (S704), the time (entrance passage time) stored in the person feature / time recording process (S306) of FIG. 3 for the person determined to be the same person in the same person determination process (S406) ) And the current time (exit passage time), and the difference is compared with a specified lower limit time obtained by subtracting a predetermined numerical value from the standard time difference calculated in the standard time difference calculation process (S701). If it is determined that the calculated time difference is less than the specified time, the process shifts to the person feature amount deletion process (S408) without performing the time difference recording process (S407). The predetermined numerical value to be subtracted from the standard time difference may be a fixed value set in advance, or the standard deviation is calculated when calculating the average value in the standard time difference calculation, and is dynamically changed such as a value obtained by adding 3σ. It may be a value.

As described above, the person passage time measurement system according to the second embodiment further includes the non-target person feature amount storage unit 123 that stores the feature amount of the person who is not the target of the passage time measurement, and from the entrance-side image. The calculated feature quantity is compared with the feature quantity stored in the non-target person feature quantity storage unit 123, and when the similarity between the two is greater than or equal to a predetermined value, the person included in the entrance side image is measured for passing time. The configuration is not applicable.
In addition, when the difference between the shooting time of the entrance side image and the shooting time of the exit side image exceeds the specified upper limit time, or when it is less than the specified lower limit time, it is excluded from recording as a passing time. It has become.

According to such a configuration, a person who does not need to investigate the passage time can be excluded from the investigation target, and a record of the passage time extremely deviating from the standard passage time can be excluded. This makes it possible to further improve the reliability of the survey results.

Here, in each of the above-described embodiments, the face feature amount is used as the feature amount used in the determination of the identity of the person. However, the feature amount is also applied to other elements related to the person such as the hairstyle, clothes, and belongings. It may be calculated and used in the determination of person identity.

Further, in each of the above embodiments, the function according to the present invention is realized in one apparatus called the person passage time measuring apparatus 102, but the function according to the present invention may be realized by a plurality of apparatuses. Further, the entrance-side imaging device 101A and the exit-side imaging device 101B may be configured to cooperate with each other to realize the function according to the present invention.

Also, there may be a plurality of entrances and exits. In that case, an imaging device may be arranged for each entrance and each exit. The present invention can also be applied to measurement of transit time in an area where the entrance and exit are not clearly defined, such as measurement of transit time in a specific section of a passage.

Although the present invention has been described in detail above, it is needless to say that the present invention is not limited to the system described here and can be widely applied to systems other than those described above.
The present invention can also be provided as, for example, a method and method for executing the processing according to the present invention, a program for realizing such a method and method, and a storage medium for storing the program.

The present invention can be used in a person passage time measurement system that measures the passage time of a person from one point to another point.

100: Network 101A: Entrance side imaging device 101B: Exit side imaging device
102: Person transit time measuring device 121: Entrance side image processing unit 122: Exit side image processing unit 123: Non-target person feature amount storage unit 251:

Gate

252A, 252B: Person

Claims

In the person passing time measurement system that measures the passing time required for the person to pass from the first point to the second point,
First calculation means for calculating a feature amount of a person included in a first image obtained by imaging the first point;
Second calculating means for calculating a feature amount of a person included in a second image obtained by imaging the second point;
The feature amount calculated from the first image is compared with the feature amount calculated from the second image to determine whether the same person as the person included in the second image is included in the first image. Determination means to perform,
If it is determined that the same person as the person included in the second image is included in the first image, the difference between the shooting time of the first image and the shooting time of the second image is calculated as A person passage time measuring system comprising recording means for recording the passage time.
In the person passage time measuring system according to claim 1,
When the difference between the shooting time of the first image and the shooting time of the second image exceeds a first threshold value, or is less than a second threshold value that is smaller than the first threshold value, the passing time is Human transit time measurement system, characterized in that it is excluded from the recording of the above.
In the person passage time measuring system according to claim 1,
A storage means for storing a feature amount of a person who is not subject to measurement of the passage time;
The feature amount calculated from the first image is compared with the feature amount stored in the storage unit, and when the similarity between the two is equal to or greater than a predetermined value, the person included in the first image is measured for passing time. Person transit time measuring system, characterized in that it is excluded from the scope of
In the person passing time measuring method for measuring the passing time required for the person to pass from the first point to the second point,
Calculating a feature amount of a person included in a first image obtained by imaging the first point;
Calculating a feature amount of a person included in a second image obtained by imaging the second point;
The feature amount calculated from the first image is compared with the feature amount calculated from the second image to determine whether the same person as the person included in the second image is included in the first image. And steps to
If it is determined that the same person as the person included in the second image is included in the first image, the difference between the shooting time of the first image and the shooting time of the second image is calculated as And a step of recording the passage time as a person passage time measurement method.
In the person passage time measuring method according to claim 4,
When the difference between the shooting time of the first image and the shooting time of the second image exceeds a first threshold value, or is less than a second threshold value that is smaller than the first threshold value, the passing time is A person passing time measurement method, characterized in that it is excluded from the recording target.
In the person passage time measuring method according to claim 4,
The feature amount of the person who is not subject to the measurement of the passage time is stored in advance, the feature amount calculated from the first image is compared with the stored feature amount, and the degree of similarity between them is predetermined. A person passing time measurement method, wherein if it is equal to or greater than the value, the person included in the first image is excluded from the passing time measurement target.