JP2020113300A - Queue detecting system, method, and program - Google Patents

Abstract

To provide a queue detecting system, a queue detecting method, and a queue detecting program in which a person who does not intend to be in a queue but simply stay, etc. is not treated as a person in a queue.SOLUTION: A queue detecting system includes: a queue information input part 2 which accepts input of a queue position; a detecting part 31 which detects a subject from an input image; and a determining part 33 which determines whether the subject gets in a queue on the basis of distance between the subject and the queue position.SELECTED DRAWING: Figure 1

Description

The present invention relates to a system, method and program for detecting a matrix from a video.

Various methods have been proposed for detecting a matrix from a camera image and for analyzing the number of people forming the matrix, the length of the matrix, the waiting time, and the like.

The technique described in Patent Document 1 obtains the area of a person region from an input image by using the background difference, and divides the area by a predetermined unit area per person to estimate the number of people. The technique described in Patent Document 1 estimates the waiting time by dividing the estimated number of people by the number of visitors per unit time that is separately collected by a predetermined method.

The technique described in Patent Document 2 captures a matrix from directly above with a camera installed on the ceiling, and uses the background difference and optical flow to extract individual person regions that are staying and moving. The technique described in Patent Document 2 connects the extracted individual person regions in order to generate a matrix line, and calculates the waiting time based on the total extension and length of each matrix.

JP, 2002-329196, A JP, 2007-317052, A

However, in the techniques described in Patent Documents 1 and 2 described above, there is a case where a person who does not intend to be in a queue but merely stays is treated as a person who is in a queue. ..

An object of the present invention is to provide a technique capable of solving the above problems.

The matrix detection system according to the present invention includes a matrix information input unit that receives an input of a matrix position, a detection unit that detects an object from an input image, and a matrix of the objects based on a distance between the object and the matrix position. And a determination unit that determines whether or not they are lined up.

A matrix detection method according to the present invention receives an input of a matrix position, detects an object from an input image, and determines whether or not the object is lined up in a matrix based on a distance between the object and the matrix position. ..

The program according to the present invention is a matrix information input process that receives an input of a matrix position, a detection process that detects an object from an input image, and the objects are arranged in a matrix based on the distance between the object and the matrix position. The computer is caused to execute a determination process of determining whether or not there is.

According to the present invention, the problem that a person who does not intend to be in a line but simply stays in a line may be mistakenly processed as a person in a line may be solved. ..

It is a functional block diagram which shows the structural example of Embodiment 1 of this invention. It is explanatory drawing which shows the example of the input method of matrix information. It is explanatory drawing which shows the example of the input method of matrix information. It is an explanatory view showing an example of a generation process of queue waiting information. It is an explanatory view showing an example of a generation process of queue waiting information. It is a flow chart which shows an example of processing progress of Embodiment 1 of the present invention. It is a flow chart which shows an example of processing progress of Embodiment 1 of the present invention. It is a functional block diagram which shows the structural example of Embodiment 2 of this invention. It is a flow chart which shows an example of processing progress of Embodiment 2 of the present invention. It is a figure explaining the example of hardware constitutions of a computer which can realize each embodiment of the present invention.

First, the background of the present invention will be described to facilitate understanding of the embodiments of the present invention.

In the techniques described in the above-mentioned Patent Documents 1 and 2, even a person who does not intend to be in a queue but simply stays in a queue may process the person as a person in a queue.

For example, the technique described in Patent Document 1 obtains the area of the person region using the background difference with respect to the input image. Therefore, when a person who does not intend to be lined up in the matrix stops near the matrix, the person simply stays there. The area of the person area will be calculated including those who are present. Further, when there are a plurality of stayers at a position away from the original queue, the technique described in Patent Document 1 can recognize that the plurality of stayers are people who form the queue. The technique described in Patent Document 1 has other problems such as non-detection due to occurrence of occlusion. In the method based on the background difference as in Patent Document 1, a person area obtained on a difference basis with an actual person by the difference in the physique of the person, the presence or absence of incidental items such as luggage, and the overlapping of the persons or the hiding by a shield. An error easily occurs in the relationship with. Further, in this method, since the individual person and the arrangement order thereof are not recognized, the waiting time for each person cannot be calculated.

Further, also in the technique described in Patent Document 2, erroneous processing may occur in which a mere resident is a person who is lined up in a matrix, as in Patent Document 1. In addition, as another problem, the method described in Patent Document 2 is based on the premise that people can be detected and tracked satisfactorily. Occurrence of occlusion due to overlapping of trees and obstacles is assumed. This may result in omission of detection of persons in the occlusion area, resulting in failure to correctly connect persons arranged in a matrix, resulting in estimation of incorrect number of persons and waiting time. In order to solve such an occlusion problem, Patent Document 2 proposes to install a camera at a high position and shoot a person from directly above. However, the environment in which the camera can be installed at a high position is not necessarily established in general indoor facilities or outdoors. Further, since the installation position and the angle of view are different from those of a general surveillance camera, it is necessary to newly install the camera as a dedicated matrix detection camera, which leads to an increase in installation cost. Therefore, it is desirable to solve the occlusion problem by using an existing surveillance camera image that captures a person from diagonally above, instead of using a camera with special installation conditions.

According to the embodiments of the present invention described below, the above-described problems are solved, erroneous detection of unintentional persons in a line is prevented, and occlusion is achieved even if the camera is not a special camera such as a ceiling-mounted type. The matrix can be detected robustly.

Embodiments of the present invention will be described below with reference to the drawings.

<Embodiment 1>
FIG. 1 is a block diagram showing a configuration example of the first embodiment of the present invention.

As shown in FIG. 1, the matrix detection system according to the present embodiment includes an image input unit 1, a matrix information input unit 2, a matrix detection unit 3, and an output unit 4.

The image input unit 1 sequentially inputs, to the matrix detection unit 3, a set of a time-series image of a predetermined monitoring area and a shooting time. The image input by the image input unit 1 to the matrix detection unit 3 may be acquired by using a photographing device such as a surveillance camera. Alternatively, the image input unit 1 may input a time-series image obtained by reading the video data stored in the storage device (not shown) to the matrix detection unit 3.

The type of the object to be detected in this embodiment is not particularly limited, and may be a person, an animal, a car, a robot, or an object. In the following, detection of a person will be described as an example.

The matrix information input unit 2 inputs the generation position of a matrix to be analyzed (hereinafter referred to as “matrix position”). The matrix information input unit 2 receives the input of the matrix generation position by the user via, for example, a mouse or a touch panel of a tablet terminal. The matrix information input unit 2 may input the generation position of the matrix on the screen on which the surveillance camera image is displayed, based on a mouse operation by a user or a touch panel operation by a touch pen or a finger. Specifically, the matrix information input unit 2 may draw the starting point, the relay point, and the ending point of the foot position of the person when the matrix occurs on the screen, and input the route and the traveling direction of the matrix. At this time, the user needs to specify at least one start point and at least one end point, but the relay point may be specified as 0 point. In addition, the user may input two or more for at least one of the start point and the end point in order to express a matrix including a branch and a merge. Further, the matrix information input unit 2 may receive, from the user, the designation of the number of people who are lined up side by side for each matrix line connecting the start point, the relay point, and the end point.

Further, the input start point, the relay point, the end point, and the number of people arranged side by side may be superimposed and drawn on the screen on which the surveillance camera image is displayed so that the user can easily confirm the input content. .. The matrix information may be input in advance before the matrix detection process is started, or the changed matrix information may be reflected at any timing during the execution of the matrix detection process. ..

The input method of the matrix information is not limited to the above. For example, instead of inputting the start point, the relay point, and the end point of the foot position of the person when the matrix is generated, the user may directly input the matrix position by tracing a straight line or a curved line on the touch panel. .. Alternatively, the matrix information input unit 2 automatically inputs the matrix position by a method such as reading a file in which the matrix position is described in a predetermined format stored inside or outside the device, not based on the input by the user. You may do it.

Here, the input of the matrix information will be specifically described. FIG. 2 is a diagram showing an image of people lined up in a matrix taken by a surveillance camera. FIG. 2 shows a state in which people are lined up from the back to the front. In addition, FIG. 2 shows a state in which the last part of the procession is divided into two, but merges in the middle to form a two-column procession that continues to the beginning. FIG. 3 shows, as the matrix information, two end points (end point 1 and end point 2) at the tail end of the matrix, two relay points (relay point 1 and relay point 2), and one start point (start point of the matrix ( The starting point 1) and the input are shown. Further, FIG. 3 shows a state in which input is made as a one-column matrix from the end points 1 and 2 to the relay point 1 and as a two-column matrix from the relay point 1 to the start point 1.

The matrix information input unit 2 reflects the start point, the relay point, the end point, and the information between the points input as described above as a matrix position at an arbitrary timing. For example, the “reflect” button for reflecting the input information may be displayed on the display screen, and when the pressing of the button by the user is detected, the matrix information input unit 2 may reflect the input information. The matrix information reflected by this is used for the determination of the determination unit 33 and the like.

The matrix detection unit 3 analyzes images sequentially input from the image input unit 1 and detects a matrix in the image. The matrix detection unit 3 includes a detection unit 31, a tracking unit 32, a determination unit 33, and a generation unit 34.

The detection unit 31 detects all or some of the persons present in the image acquired from the image input unit 1. For example, the detection unit 31 may perform a process of detecting a person from an input image by using a discriminator constructed in advance to recognize a pattern of a human body region of a specific part such as a person's head or upper body. The detection unit 31 is not limited to the method using the discriminator, and any known method in pattern recognition may be used as the person detection method.

In addition, the detection unit 31 performs detection processing by limiting the detection target region based on the matrix position input from the matrix information input unit 2 in order to detect only the persons existing near the matrix position. You may do it. That is, the detection unit 31 may perform the detection processing only on the region near the input matrix position.

The detection unit 31 outputs the time of the image acquired from the image input unit 1 and the position coordinates of the detected person as the detection result. When the calibration information of the camera that captured the input image is known by being calculated in advance, the detection unit 31 uses the calibration information to determine the position coordinates on the screen, which are the detection results of the person, as world coordinates. May be converted to and output.

The tracking unit 32 tracks the person over a plurality of frames, targeting the person detected from each frame image by the detection unit 31. The tracking unit 32 may perform a person tracking process using a known algorithm such as a particle filter, for example.

The tracking unit 32 may use different movement models with respect to the movement model used to update the position of the person to be tracked, depending on whether or not the target is a person arranged in a matrix. The tracking unit 32 may refer to, for example, a queue person ID (IDentifier) list managed by the determination unit 33 to determine whether or not the persons are in line. In that case, the tracking unit 32 determines that the person ID to be tracked is registered in the queue person ID list, and is determined to be a person who is in the queue, and is not registered, is determined to be a person who is not in the queue. May be.

For people in a line, most of the time they are in a line is stationary. Therefore, it is appropriate to apply a moving model that represents a stationary state for a large amount of time (that is, hardly moves). In addition, the traveling direction for each place is known from the matrix position information obtained from the matrix information input unit 2. Therefore, it is possible to apply a movement model in which the traveling direction is limited. In other words, the tracking unit 32 can perform more robust tracking by performing a tracking process using a two-state model that represents a moving state and a stationary state with respect to persons in a matrix. The tracking unit 32 may change the weight of each state model depending on whether the person in the line is in a stationary state or a moving state.

On the other hand, there is no limit to the direction of movement for persons who are not in line. Therefore, the tracking unit 32 may apply a movement model assuming that the person is not lined up in a line in all directions at a movement speed within a predetermined range.

If the input positional information of the person is the plane coordinates on the image, and if the calibration information of the camera that has captured the input image is known due to such reasons as being calculated in advance, the tracking unit 32 determines the world coordinate. The tracking process may be performed in the system. That is, the tracking unit 32 may perform the tracking process after converting the coordinate system on the screen into the world coordinate system using the calibration information. When the position information of the person is input in the world coordinate system, the tracking unit 32 may directly perform the tracking process in the world coordinate system.

The tracking unit 32 outputs the tracking result. The tracking unit 32 outputs, for example, the information in which the person ID of each person to be tracked, the time, and the position coordinate are associated with each other as the tracking result.

The determination unit 33 calculates the moving speed of each person based on the position coordinates of the person for each time output from the tracking unit 32. The determination unit 33 compares the position coordinates of the person with the coordinates of the matrix position input from the matrix information input unit 2, and calculates the distance between each person and the matrix position.

The determination unit 33 determines a person whose moving speed and distance from the matrix position are smaller than a predetermined threshold as a person arranged in a matrix (hereinafter referred to as a “matrix person”). The determination unit 33 determines that the person whose moving speed or the distance to the matrix position is equal to or more than a predetermined threshold is a person who is not lined up in the matrix (hereinafter, referred to as “moving person”).

When the detection unit 31 detects the head as the specific part, the determination unit 33 uses the camera parameters to estimate the lower end position corresponding to the foot of the person from the position of the head, and the estimated lower end position and the matrix position. Based on the distance of, it may be determined whether or not the people are lined up in a matrix. With this, even when it is difficult to directly detect the feet, the determination unit 33 detects the head that is relatively easy to detect and estimates the feet from the head. Can be judged. The determination unit 33 may consider the possibility that the person is an adult or the child. Specifically, for example, the determination unit 33 assumes a plurality of variations of the height of the head in the world coordinate system, and uses camera parameters to estimate a plurality of lower end positions from the plurality of assumed heights. The determination unit 33 may determine whether or not the objects are lined up in a matrix based on the estimated shortest distances between the lower end positions and the matrix position.

The determination unit 33 holds a queue person ID list indicating whether each person ID being tracked is a queue person or a moving person. The queuing person ID list is a list that manages a set of person IDs of the person being tracked and information indicating whether the person is a queuing person or a moving person.

The determination unit 33 updates the matrix person ID list by using the determination result based on the distance between the moving speed and the matrix position for each person ID being tracked.

The determination unit 33 outputs information on the person who is determined to be the line-up person. The determination unit 33 outputs, for example, information in which the person ID of a person who is determined to be a matrix person, time, and position coordinates are associated with each other.

The generation unit 34 generates the entire matrix information having no defects, based on the matrix person information input from the determination unit 33 and the matrix position information input from the matrix information input unit 2. That is, the generation unit 34 estimates the undetected area, complements the information of the person lined up in the undetected area from the information of the matrix person, and generates the matrix information.

Specifically, the generation unit 34 estimates the occurrence status of occlusion (undetected area of a person due to the influence of occlusion) based on the position coordinates of the matrix person and the matrix position. The generation unit 34 estimates the presence of a person who is not detected due to the influence of occlusion from the estimated occlusion occurrence status, the detection status of the matrix people before and after the undetected area, and the like. Further, the generation unit 34 estimates an area in which a person cannot be detected because the appearance is too small (a non-detected area of the person due to the influence of the appearance). The generation unit 34 estimates the presence of a person who has not been detected from the undetected possibility due to the effect of appearance and the detection status of the line-shaped person. The generation unit 34 generates the entire matrix information having no defect by complementing the information of the person who is not detected but is estimated to exist. The generation unit 34 uses, as the entire matrix information having no deficiency, for example, the number of people in a line, the length of the line, the cumulative waiting time of each person in the line, the progress speed of the line, from the time the line starts to leave the line. Information such as the estimated waiting time may be generated.

A method of estimating the occurrence state of occlusion will be described with reference to FIGS. 4 and 5.

FIG. 4 is a diagram showing an image obtained by photographing a lined-up matrix. FIG. 4 shows that the upper left corner of the image is the starting point (top) of the matrix, the curve is folded twice in the middle, and the lower right corner is the ending point (tail) of the matrix.

Images shot at an angle of view looking down diagonally have a shallower depression angle as the distance from the camera increases. Therefore, even in a matrix in which people are arranged at equal intervals, a person farther from the camera is more likely to be hidden by a person in front. Also, the further the distance from the camera, the less visible the person. Therefore, a person present far from the camera may not be detected because the number of pixels is less than the minimum number of pixels that can detect the person. As a specific example, FIG. 4 shows that unoccluded areas are undetected in the shaded area because the occlusion and the appearance of the person are too small.

In such a situation, for example, when calculating the number of people waiting in the queue, the generating unit 34 calculates, for example, by the following method. The generation unit 34 calculates the discontinuous matrix length based on the position coordinates of the person determined to be the matrix person by the determination unit 33 and the matrix position input from the matrix information input unit 2. In addition, the generation unit 34 estimates the population density (the number of people lined up within a predetermined distance) from the area where the person position is obtained. The generation unit 34 estimates the number of people present in the interrupted matrix length based on the estimated population density, and adds up the number of people who have been detected to calculate the waiting number of people. The generation unit 34 calculates the number of waiting persons, for example, by the method described above.

When calculating the cumulative waiting time of each person, the generation unit 34 calculates, for example, by the following method. The generation unit 34 estimates the number of people existing in the occlusion area by the method described above. The generation unit 34 calculates the cumulative waiting time after the start of being arranged in a matrix for the persons detected immediately before and immediately after the occlusion area. The generation unit 34 divides the calculated time difference between the two cumulative waiting times by the estimated number of people present in the occlusion area. The generation unit 34 adds the time difference calculated by the division to the cumulative waiting time of the persons in the front in the occlusion area in order, thereby performing the interpolation processing of the cumulative waiting time. To do.

The output unit 4 outputs the matrix information input from the matrix detection unit 3. The output mode of the output unit 4 is, for example, display output. In this case, the output unit 4 may display on a display device provided outside. However, the output mode of the output unit 4 is not limited to the display, and may be another mode.

Next, the operation of the matrix detection system according to this embodiment will be described with reference to the flowcharts of FIGS. 6 and 7. FIG. 6 shows an example of a processing flow when the user inputs the matrix position. FIG. 7 shows an example of a processing flow when the matrix is detected and the matrix information of the entire matrix having no defect is generated after the matrix position is input.

The processing steps described below may be arbitrarily changed in order or executed in parallel as long as the processing content does not conflict, and other steps may be added between the processing steps. good. Further, the step described as one step for convenience may be divided into a plurality of steps and executed, and the step described for a plurality of steps may be executed as one step for convenience.

First, the processing when the user inputs the matrix position will be described with reference to the flowchart in FIG.

The matrix information input unit 2 acquires an image of a predetermined monitoring area and displays it on the screen for setting the matrix position (step SA1).

Next, the user confirms whether or not there is an unset matrix on the displayed screen. When the matrix setting is not completed and there is an unset matrix, the matrix information input unit 2 receives the start of the input of the matrix position by the user via a predetermined input device (no in step SA2). As the input device, for example, a mouse or a touch panel is applicable, and in the case of a touch panel, the user may use a touch pen or a finger to input.

The matrix information input unit 2 receives the input of the starting point of the matrix by the user (step SA3). When the matrix to be set by the user has a plurality of starting points, the matrix information input unit 2 accepts input of all the starting points in step SA3.

Next, the matrix information input unit 2 receives input of a relay point representing a branch of the matrix or a waypoint by the user (step SA4). If the matrix can be represented by a straight line connecting the start point and the end point, the input of the relay point is not necessary and the process of step SA4 is omitted. In addition, when there are a plurality of relay points in the matrix to be set by the user, the matrix information input unit 2 accepts the input of all the relay points in step SA4.

Next, the matrix information input unit 2 receives the input of the end point of the matrix (step SA5). When the matrix to be set by the user has a plurality of end points, the matrix information input unit 2 accepts input of all end points in step SA5.

The start point, the relay point, and the end point input in steps SA3, SA4, and SA5 may be displayed on the image acquired in step SA1 every time they are input, and the user may be able to check each input point.

Next, the user confirms, for each line segment connecting the input start point, relay point, and end point, whether or not there is a line segment for which the number of rows (the number of people who are lined up horizontally) is not set. .. When the setting of the number of columns is not completed and there is an unset line segment, the matrix information input unit 2 inputs the number of columns by the user via the above-mentioned input device for the unset line segment. The start is accepted (no in step SA6). The matrix information input unit 2 receives an input of the number of columns by the user (step SA7).

It should be noted that the number of each line segment and column connecting the points input in step SA7 is displayed on the image acquired in step SA1 each time it is input, and the user may confirm the number of each input line segment. Good.

When the input of the number of columns for all the line segments by the user is completed (yes in step SA6), the matrix information input unit 2 returns to step SA2 and starts accepting the input for the next matrix. The matrix information input unit 2 confirms that the user has completed the input of the number of columns for all the line segments, for example, by receiving an input by the user to that effect, or for a predetermined time without the user starting input. You may judge by progressing.

When the matrix information input unit 2 determines that the input of all the matrices by the user is completed (Yes in step SA2), the input of the matrix information in the matrix information input unit 2 is completed. The matrix information input unit 2 determines that the input of all the matrix by the user is completed, for example, by accepting the input by the user to that effect, or by the lapse of a predetermined time without the input of the user being started. You may.

Next, a process of detecting a queue and generating queue waiting information will be described with reference to the flowchart of FIG.

The detection unit 31 acquires a captured image and its capturing time from the input unit 1 (step SB1). The detection unit 31 detects all the persons existing in the image, and outputs the time and the position coordinates of the detected persons (step SB2).

Next, receiving the tracking result of the tracking unit 32, the determination unit 33 refers to the matrix person ID list regarding the tracking person obtained as the processing result for the previous frame image (step SB3), and determines whether the person is a matrix person. The determination is made (step SB4). The queue person ID list is a list that manages a pair of a person ID of a person who is being tracked and information indicating whether the person is a queue person or a moving person.

When it is a line-up person (yes in step SB4), the determination unit 33 predicts the position coordinates in the current frame image using the movement model for line-up person. The determination unit 33 calculates the position coordinates of the tracking person in the current frame image using the predicted position coordinates and the person position coordinates obtained in step SB2 (step SB5a). The determination unit 33 may use, as the movement model for the matrix person, for example, a two-state model of a movement model that represents a state in which the matrix is stationary and a movement model that represents a state in which the matrix advances. Specifically, in the moving model representing the stationary state, the determination unit 33 predicts that there is almost no change in position between the previous frame image and the current frame image. On the other hand, in the movement model representing the state of progress, the determination unit 33 performs prediction in which the movement direction is limited between the previous frame image and the current frame image based on the matrix position information obtained from the matrix information input unit 2.

On the other hand, when the person is a moving person (no in step SB4), the determination unit 33 uses the movement model for the moving person to predict the position coordinates in the current frame image. The determination unit 33 calculates the position coordinate of the tracking person in the current frame image using the predicted position coordinate and the person position coordinate obtained in step SB2 (step SB5b). As the movement model for the moving person, the determination unit 33 may use, for example, a model that is assumed to move in all directions at a movement speed within a predetermined range because the movement direction is not limited.

Next, the determination unit 33 calculates the moving speed of the person by using the person position coordinates and each time information for the past several frames from the present calculated in step SB5a and step SB5b (step SB6).

The determination unit 33 calculates an appropriate distance such as the shortest distance between the person position coordinates in the current frame and the matrix position obtained from the matrix information input unit 2 (step SB7).

The determination unit 33 uses the moving speed calculated in step SB6 and the distance calculated in step SB7 to determine whether the moving speed is smaller than a predetermined threshold and the distance is smaller than a predetermined threshold. (Step SB8). If the condition is satisfied (yes in step SB8), it is determined that the person is a matrix person (step SB9a). On the other hand, if the above conditions are not satisfied (No in step SB8), it is determined that the person is a moving person (step SB9b).

The determination unit 33 updates the queue person ID list managed by the determination unit 33, using the determination result of whether or not the person is in line in the matrix in steps SB9a and SB9b (step SB10).

Next, the generation unit 34 estimates the occurrence state of occlusion based on the person position of the person who is determined to be the matrix person in step SB9a and the matrix position acquired from the matrix information input unit 2. The generation unit 34 performs interpolation processing of matrix information existing in the occlusion area (step SB11). Specifically, when calculating the waiting number as the queue information, the generation unit 34 uses the position information of the person determined to be the queue person in step SB9a and the queue position information obtained from the queue information input unit 2. , Calculate the matrix length of the undetected area that is interrupted by occlusion. In addition, the generation unit 34 calculates the population density of the line-up persons (the number of people lined up within a predetermined distance). The generation unit 34 estimates the number of people present in the matrix length of the undetected region that has been interrupted based on the calculated population density, and adds up with the number of people who can be detected to calculate the waiting number of people.

The output unit 4 outputs the matrix information output from the generation unit 34 to an external module such as an application or a storage medium (step SB12).

The matrix information input process shown in FIG. 6 may be performed in advance before the matrix detection process shown in FIG. 7 is started. Alternatively, the matrix detection process shown in FIG. 7 is restarted using the updated matrix information by performing the process shown in FIG. 6 at an arbitrary timing during execution of the matrix detection process shown in FIG. It may be done.

The program according to the first embodiment may be any program that causes a computer to execute the series of processes described above.

As described above, according to the matrix detection system according to the first embodiment, even when a person who does not intend to be in a queue but simply stays, is processed as a person who is in a queue. The problem that there is is solved.

In addition, even in an environment where occlusion occurs, by estimating the length of the matrix of the hidden area, it becomes possible to perform the interpolation processing of the number of people. In addition, even when the matrix shape is complicated, the traveling direction for each location is known. Therefore, by utilizing the traveling direction in the moving model at the time of tracking a person in a matrix, tracking robustness is improved. Therefore, according to the matrix detection system according to the first embodiment, it is possible to detect persons arranged in a matrix and appropriately estimate matrix information such as the number of people and waiting time.

In addition, even when a person who is not intended to be in a queue stops near the queue, it is possible to determine that the person is not in a queue by setting the threshold value of the distance between the person and the queue position appropriately. Will be possible. Furthermore, in determining whether or not a person is in a line, not only the distance between the line and the line position is considered, but also the moving speed of the line is taken into consideration. In the determination in the vicinity, it is possible to accurately determine that the person is not a person in a line.

<Embodiment 2>
Next, a functional configuration of the matrix detection system according to the second exemplary embodiment of the present invention will be described with reference to FIG.

FIG. 8 is a block diagram showing a configuration example of the second embodiment of the present invention.

As shown in FIG. 8, the matrix detection system according to this embodiment includes a matrix information input unit 51, a detection unit 52, and a determination unit 53.

The matrix information input unit 51 receives input of matrix positions.

The detection unit 52 detects an object from the input image.

The determination unit 53 determines whether or not the objects are arranged in a matrix based on the distance between the objects and the matrix position.

FIG. 9 is a flowchart showing the operation of the matrix detection system according to the second embodiment.

The matrix information input unit 51 receives the input of the matrix position (step SC1).

The detection unit 52 detects an object from the input image (step SC2).

The determination unit 53 determines whether or not the objects are arranged in a matrix based on the distance between the objects and the matrix position (step SC3).

As described above, according to the matrix detection system of the second embodiment, even a person who does not intend to be in a queue but simply stays in the queue will process the person as a person in a queue. In some cases, the problem is solved.

<Hardware configuration example>
Each unit constituting the matrix detection system of each of the above-described embodiments includes a control unit, a memory, a program loaded in the memory, a storage unit such as a hard disk for storing the program, a network connection interface, and the like. It is realized by any combination.

FIG. 10 is a block diagram showing an example of the hardware configuration of the matrix detection system according to the first or second embodiment.

As shown in FIG. 10, the matrix detection system includes a CPU 11 (Central Processing Unit), a communication IF 12 (communication interface 12) for network connection, a memory 13, and a storage device 14 such as a hard disk for storing a program. , Realized by a computer device. However, the configuration of the matrix detection system is not limited to the configuration shown in FIG.

The CPU 11 operates the operating system to control the entire matrix detection system. Further, the CPU 11 reads programs and data from the recording medium mounted in, for example, a drive device into the memory 13. The CPU 11 reads out programs and data to the memory 13 so that the image input unit 1, the matrix information input unit 2, the matrix detection unit 3 (the detection unit 31, the tracking unit 32, the determination unit 33, and the generation unit 34) and the output unit 4 are provided. Each functional unit is realized.

The matrix detection system may be connected to an image input device, which is another computer, via the communication IF 12, for example.

The recording device 14 is, for example, an optical disk, a flexible disk, a magnetic optical disk, an external hard disk, a semiconductor memory, or the like, and records a computer program in a computer-readable manner. The storage device 14 may be, for example, a data storage unit that stores integrated data. Further, the computer program may be downloaded from an external computer (not shown) connected to the communication network.

The matrix detection system may be connected to the input device 15 or the output device 16. The input device 15 is, for example, a keyboard, a mouse, or the like. The output device 16 may be, for example, a display or the like, and may display the matrix information output by the output unit 4 on the display.

In addition, in the functional block diagrams of FIGS. 1 and 8, the components of each embodiment are described as being realized by one physically connected device, but the realizing means is not particularly limited. That is, two or more physically separated devices may be connected by wire or wirelessly, and the system of each embodiment may be realized by the plurality of devices.

Although the present invention has been described with reference to the exemplary embodiments and examples, the present invention is not limited to the exemplary embodiments and examples described above. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

<Other expressions of the embodiment>
The whole or part of the exemplary embodiments disclosed above can be described as, but not limited to, the following supplementary notes.

(Appendix 1)
Matrix information input means for receiving the input of the matrix position,
Detection means for detecting the object from the input image,
Based on the distance between the object and the matrix position, determination means for determining whether the object is lined up in a matrix,
Matrix detection system including.

(Appendix 2)
Based on the matrix position and the position of the target object determined to be lined up in the matrix by the determination means, the region in which the target object is undetected is estimated, and from the target object determined to be lined up in the matrix A generation unit that complements the object in the undetected area and generates matrix information,
The matrix detection system according to appendix 1, further comprising:

(Appendix 3)
The matrix detection system according to appendix 2, wherein the generation unit complements the target object in the undetected region based on the density of the target object determined to be lined up in the matrix by the determination unit.

(Appendix 4)
The generating means is
The waiting time of the objects determined to be lined up in a queue by the determination means is calculated,
The matrix detection system according to appendix 3, wherein the waiting time of the object in the undetected area is calculated based on the waiting times of the object detected in the areas before and after the undetected area.

(Appendix 5)
The determination means determines whether or not the target objects are lined up in a matrix based on the moving speed of the target object and the distance between the target object and the matrix position. The matrix detection system described in 1.

(Appendix 6)
Tracking means for tracking the position of the detected object using different movement models depending on whether the object is lined up or not,
Further including,
The matrix detection system according to appendix 5, wherein the determination unit calculates the moving speed from the tracking result of the tracking unit.

(Appendix 7)
The tracking means is
For the object determined to be waiting in line by the determination unit, based on the two movement models representing the waiting state and the movement state of the line, and the line position and traveling direction acquired from the line information input unit, the previous frame From the current frame to the current frame,
Predicting the movement state from the previous frame to the current frame based on a movement model assuming that the object that has not been determined to wait in line by the determination means moves in all directions at a speed within a predetermined range. 6. The matrix detection system according to 6.

(Appendix 8)
The matrix information input means receives inputs of at least one or more matrix start points, at least one or more matrix end points, and zero or more relay points for representing matrix waypoints or branch points, 8. The matrix detection system according to any one of appendices 1 to 7, which receives an input of the number of columns in which the objects are arranged side by side with respect to the line segment connecting the adjacent start point, relay point, and end point.

(Appendix 9)
The matrix information input means,
The matrix detection system according to appendix 8, wherein the starting point, the relay point, the ending point, and the number of columns between the points are drawn as input information and reflected as the matrix position at an arbitrary timing.

(Appendix 10)
The detection means detects a specific portion of the object,
The determination means estimates the lower end position of the target object from the position of the specific part by using a camera parameter, and the target objects are arranged in a matrix based on the distance between the estimated lower end position and the matrix position. The matrix detection system according to any one of appendices 1 to 9.

(Appendix 11)
The determining unit assumes a plurality of variations of the height of the specific portion in the world coordinate system, estimates a plurality of lower end positions from the plurality of assumed heights by using the camera parameters, and estimates a plurality of lower ends. The matrix detection system according to supplementary note 10, wherein it is determined whether or not the objects are arranged in a matrix based on the shortest distance between the position and the matrix position.

(Appendix 12)
Accepts input of matrix position,
Detect the object from the input image,
Based on the distance between the object and the matrix position, it is determined whether the object is lined up in a matrix,
Matrix detection method.

(Appendix 13)
Matrix information input processing that accepts input of matrix position,
Detection processing to detect the object from the input image,
Based on the distance between the object and the matrix position, determination processing for determining whether the object is lined up in a matrix,
A program that causes a computer to execute.

This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2015-180260 for which it applied on September 14, 2015, and takes in those the indications of all here.

According to the present invention, it is possible to provide a highly accurate queue monitoring system, which can be suitably applied to a system that detects a queue generated in a monitoring area and presents the number of people waiting and the waiting time.

Further, in the present invention, by utilizing the occurrence position information of the matrix input in advance, even in an environment where occlusion is likely to occur due to overlapping of shelves, pillars, or people, without requiring special camera installation conditions, Since it is possible to estimate the number of people waiting and the waiting time without loss, it becomes easy to introduce a queue detection system.

1 Image Input Section 2, 51 Matrix Information Input Section 3 Matrix Detection Section 4 Output Section 31, 52 Detection Section 32 Tracking Section 33, 53 Judgment Section 34 Generation Section 11 CPU
12 Communication IF
13 memory 14 storage device 15 input device 16 output device

Claims (10)

Matrix information input means for receiving the input of the matrix position,
Detection means for detecting the object from the input image,
Based on the distance between the object and the matrix position, determination means for determining whether the object is lined up in a matrix,
Matrix detection system including. Based on the matrix position and the position of the target object determined to be lined up in the matrix by the determination means, the region in which the target object is undetected is estimated, and from the target object determined to be lined up in the matrix A generation unit that complements the object in the undetected area and generates matrix information,
The matrix detection system of claim 1, further comprising: The matrix detection system according to claim 2, wherein the generation unit complements the target objects in the undetected region based on the density of the target objects determined to be lined up in the matrix by the determination unit. The generating means is
The waiting time of the objects determined to be lined up in a queue by the determination means is calculated,
The matrix detection system according to claim 3, wherein the waiting time of the object in the undetected region is calculated based on the waiting time of the object detected in the regions before and after the undetected region. The determination unit determines whether or not the target objects are lined up in a matrix based on a moving speed of the target object and a distance between the target object and the matrix position. The matrix detection system according to Crab. Tracking means for tracking the position of the detected object using different movement models depending on whether the object is lined up or not,
Further including,
The matrix detection system according to claim 5, wherein the determination unit calculates the moving speed from a tracking result by the tracking unit. The tracking means is
For the object determined to be waiting in line by the determination unit, based on the two movement models representing the waiting state and the movement state of the line, and the line position and traveling direction acquired from the line information input unit, the previous frame From the current frame to the current frame,
A movement state from the previous frame to the current frame is predicted based on a movement model assuming that the object that has not been determined to be in the queue by the determination means moves in all directions at a speed within a predetermined range. Item 6. The matrix detection system according to Item 6. The matrix information input unit receives inputs of at least one or more matrix start points, at least one or more matrix end points, and zero or more relay points for representing waypoints or branch points of the matrix, The matrix detection system according to any one of claims 1 to 7, wherein an input of the number of columns in which objects are arranged side by side is received for a line segment that connects between each adjacent start point, relay point, and end point. Accepts input of matrix position,
Detect the object from the input image,
Based on the distance between the object and the matrix position, it is determined whether the object is lined up in a matrix,
Matrix detection method. Matrix information input processing that accepts input of matrix position,
Detection processing to detect the object from the input image,
Based on the distance between the object and the matrix position, determination processing for determining whether the object is lined up in a matrix,
A program that causes a computer to execute.

Images