JP6852675B2 - Matrix detection system, method and program - Google Patents

Description

The present invention relates to a system, method and program for detecting a matrix from an image.

Various methods have been proposed for detecting a matrix from a camera image and for analyzing the number of people forming a matrix, the length, the waiting time, and the like.

The technique described in Patent Document 1 obtains the area of a person area using background subtraction from an input image, and divides the area by a predetermined unit area per person to estimate the number of people. The technique described in Patent Document 1 estimates the waiting time by dividing the estimated number of people by the number of visitors per unit time separately calculated by a predetermined method.

In the technique described in Patent Document 2, a procession is photographed from directly above with a camera installed on the ceiling, and individual human areas that are staying and moving are extracted by using background subtraction and optical flow. The technique described in Patent Document 2 sequentially connects the extracted individual person regions to generate a matrix line, and calculates the waiting time based on the total length and length of each matrix.

Japanese Unexamined Patent Publication No. 2002-329196 Japanese Unexamined Patent Publication No. 2007-317502

However, in the techniques described in Patent Documents 1 and 2 described above, even a person who does not intend to line up in a line but merely stays in the line may be treated as a person lined up in a line. ..

An object of the present invention is to provide a technique capable of solving the above-mentioned problems.

In the matrix detection system according to the present invention, the objects are arranged in a matrix based on the distance between the matrix information input unit that accepts the input of the matrix position, the detection unit that detects the object from the input image, and the object and the matrix position. Includes a determination unit for determining whether or not they are lined up.

The matrix detection method according to the present invention accepts an input of a matrix position, detects an object from the input image, and determines whether or not the objects are lined up in a matrix based on the distance between the object and the matrix position. ..

In the program according to the present invention, the objects are arranged in a matrix based on the matrix information input process that accepts the input of the matrix position, the detection process that detects the object from the input image, and the distance between the object and the matrix position. and determination processing of determining dolphin not, Ru cause the computer to execute.

According to the present invention, there is a problem that even a person who does not intend to line up in a line but simply stays in the line may be mistakenly treated as a person in line. ..

It is a functional block diagram which shows the structural example of Embodiment 1 of this invention. It is explanatory drawing which shows the example of the input method of the matrix information. It is explanatory drawing which shows the example of the input method of the matrix information. It is explanatory drawing which shows the example of the generation process of the queue waiting information. It is explanatory drawing which shows the example of the generation process of the queue waiting information. It is a flowchart which shows the example of the processing progress of Embodiment 1 of this invention. It is a flowchart which shows the example of the processing progress of Embodiment 1 of this invention. It is a functional block diagram which shows the structural example of Embodiment 2 of this invention. It is a flowchart which shows the example of the processing progress of Embodiment 2 of this invention. It is a figure explaining the hardware configuration example of the computer which can realize each embodiment of this invention.

First, the background of the present invention will be described in order to facilitate understanding of the embodiments of the present invention.

In the techniques described in Patent Documents 1 and 2 described above, even a person who does not intend to line up in a line but merely stays in the line may be treated as a person lined up in a line.

For example, the technique described in Patent Document 1 obtains the area of a person area using background subtraction for an input image. Therefore, when a person who does not intend to line up in a line stops near the line, the person simply stays there. The area of the person area including the person who is there will be calculated. Further, when there are a plurality of resident persons at positions away from the original procession, the technique described in Patent Document 1 can recognize that the plurality of resident persons are people who form the procession. Another problem with the technique described in Patent Document 1 is that it has not been detected due to the occurrence of occlusion. In the method based on background subtraction as in Patent Document 1, the person area acquired based on the difference between the actual person and the actual person due to the difference in the physique of the person, the presence or absence of incidental items such as luggage, and the overlap between the persons and the hiding by the shield. An error easily occurs in the relationship with. In addition, since this method does not recognize individual persons and their order, it is not possible to calculate the waiting time for each person.

Further, also in the technique described in Patent Document 2, an erroneous process may occur in which a mere resident is regarded as a person in a line as in Patent Document 1. Further, as another problem, the method described in Patent Document 2 is based on the premise that the detection and tracking of a person can be performed well, but in an actual environment, the persons may be different from each other depending on the installation position of the camera, the structure of the building, and the like. Occurrence of occlusion due to overlapping or shielding is expected. As a result, the detection of the person is omitted in the occlusion area, and as a result, it fails to correctly connect the people in the queue, and the wrong number of people and the waiting time are estimated. Patent Document 2 proposes to install a camera at a high position and take a picture of a person from directly above in order to solve such an occlusion problem. However, it is not always possible to install the camera at a high position in a general indoor facility or outdoors. In addition, since the installation position and angle of view are different from those of general surveillance cameras, it is necessary to newly install it as a camera dedicated to matrix detection, which leads to an increase in introduction cost. Therefore, it is desirable to solve the occlusion problem by using an existing surveillance camera image that captures a person from diagonally above, instead of using a camera with special installation conditions.

According to the embodiment of the present invention described below, the above-mentioned problems are solved, false detection of unintended persons in a line is prevented, and even if the camera is not a special camera such as a ceiling-mounted type, it can be used for occlusion. The matrix can be detected robustly.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Embodiment 1>
FIG. 1 is a block diagram showing a configuration example of the first embodiment of the present invention.

As shown in FIG. 1, the matrix detection system according to the present embodiment includes an image input unit 1, a matrix information input unit 2, a matrix detection unit 3, and an output unit 4.

The image input unit 1 sequentially inputs a set of a time-series image of a predetermined monitoring area and a shooting time to the matrix detection unit 3. The image input by the image input unit 1 to the matrix detection unit 3 may be acquired by using a photographing device such as a surveillance camera. Alternatively, the image input unit 1 may input a time-series image obtained by reading the video data stored in the storage device (not shown) into the matrix detection unit 3.

The type of object to be detected in this embodiment is not particularly limited, and may be a person, an animal, a car, a robot, or an object. In the following, the detection of a person will be described as an example.

The matrix information input unit 2 inputs the generation position of the matrix to be analyzed (hereinafter referred to as "matrix position"). The matrix information input unit 2 receives input of a matrix generation position by a user via, for example, a mouse or a touch panel of a tablet terminal. The matrix information input unit 2 may input the position where the matrix is generated on the screen on which the surveillance camera image is displayed, based on the mouse operation by the user or the touch panel operation by a touch pen, a finger, or the like. Specifically, the matrix information input unit 2 may draw the start point, the relay point, and the end point of the foot position of the person when the matrix is generated on the screen, and input the path and the traveling direction of the matrix. At this time, the user needs to specify at least one start point and one end point, respectively, but the relay point may be specified as 0 point. Further, in order to represent a matrix including branching and merging, the user may input two or more for at least one of the start point and the end point. Further, the matrix information input unit 2 may accept from the user the designation of the number of people arranged side by side for each matrix line connecting the start point, the relay point, and the end point.

Further, the input contents may be easily confirmed by the user by superimposing and drawing the input start point, relay point, end point and the number of people lined up side by side on the screen on which the surveillance camera image is displayed. .. Further, the matrix information may be input in advance before starting the matrix detection process, or the changed matrix information may be reflected at an arbitrary timing during the execution of the matrix detection process. ..

The method of inputting matrix information is not limited to the above. For example, instead of inputting the start point, relay point, and end point of the foot position of the person when the matrix is generated, the user may input the matrix position by tracing a straight line or a curve directly on the touch panel. .. Alternatively, the matrix information input unit 2 automatically inputs the matrix position by a method such as reading a file in which the matrix position is described in a predetermined format stored inside or outside the device, instead of being based on the input by the user. You may do so.

Here, the input of matrix information will be specifically described. FIG. 2 is a diagram showing an image of people lined up in a line taken by a surveillance camera. FIG. 2 shows a state in which people are lined up from the back to the front. Further, FIG. 2 shows a state in which the matrix is bifurcated at the end, but merges in the middle to form a two-row matrix and continues to the beginning. FIG. 3 shows, as matrix information, two end points (end point 1 and end point 2) at the end of the matrix, two relay points (relay point 1 and relay point 2), and one start point (relay point 1 and relay point 2) at the beginning of the matrix. It shows how the start point 1) and is input. Further, FIG. 3 shows a state in which the ending points 1 and 2 to the relay point 1 are input as a one-row matrix, and the relay points 1 to the start point 1 are input as a two-row matrix.

The matrix information input unit 2 reflects the information input as described above at the start point, the relay point, the end point, and between the points as the matrix position at an arbitrary timing. For example, when a "reflect" button for reflecting the input information is displayed on the display screen and the user's pressing of the button is detected, the matrix information input unit 2 may reflect the input information. The matrix information reflected by this will be used for the determination of the determination unit 33 and the like.

The matrix detection unit 3 analyzes the images sequentially input from the image input unit 1 and detects the matrix in the image. The matrix detection unit 3 includes a detection unit 31, a tracking unit 32, a determination unit 33, and a generation unit 34.

The detection unit 31 detects all or a part of the persons present in the image acquired from the image input unit 1. For example, the detection unit 31 may perform a process of detecting a person from an input image by using a classifier constructed in advance for recognizing a pattern of a human body region of a specific part such as the head and upper body of the person. The detection unit 31 is not limited to the method using the classifier, and the detection unit 31 may use a known method in pattern recognition as a method for detecting a person.

Further, the detection unit 31 performs detection processing by limiting the detection target area based on the matrix position input from the matrix information input unit 2 in order to detect only the person existing in the vicinity of the matrix position. You may do so. That is, the detection unit 31 may perform the detection process only on the region near the input matrix position.

The detection unit 31 outputs the time of the image acquired from the image input unit 1 and the position coordinates of the detected person as the detection result. When the calibration information of the camera that captured the input image is known by being calculated in advance, the detection unit 31 uses the calibration information to set the position coordinates on the screen, which is the detection result of the person, to the world coordinates. It may be converted to and output.

The tracking unit 32 tracks the person detected from each frame image by the detection unit 31 over a plurality of frames. The tracking unit 32 may perform the tracking process of the person by using a known algorithm such as a particle filter.

The tracking unit 32 may use a different movement model depending on whether or not the target is a person in a line with respect to the movement model used to update the position of the person to be tracked. The tracking unit 32 may refer to, for example, a matrix person ID (Identifier) list managed by the determination unit 33 regarding whether or not the person is in a line. In that case, the tracking unit 32 determines that the person ID to be tracked is a person in the queue if it is registered in the queue person ID list, and determines that the person is not in the queue if it is not registered. You may.

For people in line, most of the time they are in line is stationary. Therefore, it is appropriate to apply a movement model that represents a resting state for many hours (ie, hardly moves). Further, the traveling direction for each place is known from the matrix position information obtained from the matrix information input unit 2. Therefore, it is possible to apply a movement model in which the traveling direction is limited. That is, the tracking unit 32 can track the persons in the line more robustly by performing the tracking process using the two-state model representing the moving state and the stationary state. The tracking unit 32 may change the weight of each state model according to whether the persons in the line are in a stationary state or a moving state.

On the other hand, people who are not in line have no restrictions on the direction of movement. Therefore, the tracking unit 32 may apply a movement model that assumes movement in all directions at a movement speed within a predetermined range to people who are not in line.

When the input position information of the person is the plane coordinates on the image and the calibration information of the camera that captured the input image is known due to circumstances such as being calculated in advance, the tracking unit 32 uses the world coordinates. The tracking process may be performed in the system. That is, the tracking unit 32 may perform the tracking process after converting the coordinate system on the screen into the world coordinate system by using the calibration information. When the position information of the person is input in the world coordinate system, the tracking unit 32 may perform the tracking process in the world coordinate system as it is.

The tracking unit 32 outputs the tracking result. For example, the tracking unit 32 outputs information as a tracking result in which a person ID for each person to be tracked, a time, and position coordinates are associated with each other.

The determination unit 33 calculates the moving speed of each person based on the position coordinates of the person for each time output from the tracking unit 32. The determination unit 33 compares the position coordinates of the person with the coordinates of the matrix position input from the matrix information input unit 2, and calculates the distance between each person and the matrix position.

The determination unit 33 determines a person whose movement speed and distance from the matrix position are smaller than a predetermined threshold value as a person who is lined up in a matrix (hereinafter, referred to as a "matrix person"). The determination unit 33 determines a person whose movement speed or distance from the matrix position is equal to or greater than a predetermined threshold value as a person who does not line up in the matrix (hereinafter, referred to as "moving person").

When the detection unit 31 detects the head as a specific part, the determination unit 33 estimates the lower end position corresponding to the foot of the person from the position of the head by using the camera parameters, and determines the estimated lower end position and the matrix position. It may be determined whether or not the persons are in a line based on the distance of. As a result, even when it is difficult to directly detect the feet, the head, which is relatively easy to detect, is detected and the feet are estimated from the head, so that the determination unit 33 accurately determines whether or not the person is in line. Can be determined. The determination unit 33 may consider the possibility that the person is an adult or a child. Specifically, for example, the determination unit 33 assumes the height of the head in the world coordinate system with a plurality of variations, and estimates a plurality of lower end positions from the assumed heights by using the camera parameters. The determination unit 33 may determine whether or not the objects are lined up in a matrix based on the shortest distance between the estimated plurality of lower end positions and the matrix positions.

The determination unit 33 holds a matrix person ID list indicating whether each person ID being tracked is a procession person or a moving person. The procession person ID list is a list that manages a set of a person ID of a person being tracked and information indicating whether the person is a procession person or a moving person.

The determination unit 33 updates the matrix person ID list for each person ID being tracked by using the determination result based on the distance between the moving speed and the matrix position.

The determination unit 33 outputs information on a person who has been determined to be a queue person. The determination unit 33 outputs, for example, information in which the person ID of the person determined to be a queue person, the time, and the position coordinates are associated with each other.

The generation unit 34 generates the entire matrix information without any loss based on the information of the matrix person input from the determination unit 33 and the information of the matrix position input from the matrix information input unit 2. That is, the generation unit 34 estimates the undetected area, complements the information of the persons lined up in the undetected area from the information of the person in the matrix, and generates the matrix information.

Specifically, the generation unit 34 estimates the occurrence status of occlusion (the undetected area of the person due to the influence of occlusion) based on the position coordinates of the person in the procession and the position of the person in the procession. The generation unit 34 estimates the existence of a person who has not been detected due to the influence of the occlusion from the estimated occurrence state of occlusion and the detection state of the matrix person before and after the undetected area. In addition, the generation unit 34 estimates a region where a person cannot be detected because the visibility is too small (a region where the person is not detected due to the influence of the visibility). The generation unit 34 estimates the existence of an undetected person from the possibility of undetection due to the influence of appearance and the detection status of the queue person. The generation unit 34 generates the entire matrix information without any loss by supplementing the information of the person who has not been detected but is presumed to exist. The generation unit 34 provides the entire matrix information without any loss, for example, the number of people in the matrix, the length of the matrix, the cumulative waiting time of each person in the matrix, the progress speed of the matrix, and the time from the start of the matrix to the exit of the matrix. Information such as the estimated waiting time of may be generated.

A method for estimating the occurrence of occlusion will be described with reference to FIGS. 4 and 5.

FIG. 4 is a diagram showing an image of a matrix with folds. FIG. 4 shows a state in which the upper left of the image is the start point (beginning) of the matrix, is folded back twice in the middle, and the lower right is the end point (end) of the matrix.

An image shot with an angle of view looking down diagonally has a shallower depression angle as the distance from the camera increases. Therefore, even in a procession in which people are lined up at equal intervals, the farther the person is from the camera, the easier it is to hide in the person in front. Also, the farther away from the camera, the smaller the visibility of the person. Therefore, a person who is far from the camera may not be detected if the number of pixels is less than the minimum number of pixels that can detect the person. As a specific example, FIG. 4 shows that in the shaded area, occlusion and the visibility of a person are too small to be detected.

In such a situation, for example, when calculating the number of people waiting for the entire queue, the generation unit 34 calculates by, for example, the following method. The generation unit 34 calculates the interrupted matrix length based on the position coordinates of the person determined to be the matrix person by the determination unit 33 and the matrix position input from the matrix information input unit 2. In addition, the generation unit 34 estimates the population density (the number of people lined up within a predetermined distance) from the area where the person position is obtained. The generation unit 34 estimates the number of people existing in the interrupted queue length based on the estimated population density, and adds up the number of people that can be detected to calculate the number of people waiting. The generation unit 34 calculates the number of people waiting, for example, by the method described above.

Further, when calculating the cumulative waiting time of each person, the generation unit 34 calculates by, for example, the following method. The generation unit 34 estimates the number of people existing in the occlusion region by the method described above. The generation unit 34 calculates the cumulative waiting time from the start of lining up for the persons detected immediately before and after the occlusion area. The generation unit 34 divides the calculated time difference between the two cumulative waiting times by the estimated number of people existing in the occlusion area. The generation unit 34 performs the interpolation processing of the cumulative waiting time by adding the time difference calculated by division to the cumulative waiting time of the people lined up in front in order from the first person existing in the occlusion area. Do.

The output unit 4 outputs the matrix information input from the matrix detection unit 3. The output mode of the output unit 4 is, for example, a display output. In this case, the output unit 4 may display on an external display device. However, the output mode of the output unit 4 is not limited to the display, and may be another mode.

Next, the operation of the matrix detection system according to the present embodiment will be described with reference to the flowcharts of FIGS. 6 and 7. FIG. 6 shows an example of a processing flow when the user inputs a matrix position. FIG. 7 shows an example of a processing flow when the matrix is detected after the matrix position is input and the entire matrix information without defects is generated.

Each processing step described later may be arbitrarily changed in order or executed in parallel as long as the processing contents do not conflict with each other, or another step may be added between each processing step. good. Further, the step described as one step for convenience may be executed in a plurality of steps, or the step described in a plurality of steps may be executed as one step for convenience.

First, the process when the user inputs the matrix position will be described with reference to the flowchart of FIG.

The matrix information input unit 2 acquires an image of a predetermined monitoring area and displays it on a screen for setting a matrix position (step SA1).

Next, the user checks whether there is an unconfigured matrix on the displayed screen. When the matrix setting is not completed and there is an unset matrix, the matrix information input unit 2 accepts the start of input of the matrix position by the user via a predetermined input device (no in step SA2). The input device corresponds to, for example, a mouse or a touch panel, and if it is a touch panel, the user may input using a touch pen, a finger, or the like.

The matrix information input unit 2 accepts the input of the start point of the matrix by the user (step SA3). When there are a plurality of start points in the matrix to be set by the user, the matrix information input unit 2 accepts the input of all the start points in step SA3.

Next, the matrix information input unit 2 accepts the input of the relay point representing the branch of the matrix and the waypoint by the user (step SA4). If the matrix can be represented by a straight line connecting the start point and the end point, the process of step SA4 is omitted because the input of the relay point is not necessary. If there are a plurality of relay points in the matrix to be set by the user, the matrix information input unit 2 accepts the inputs of all the relay points in step SA4.

Next, the matrix information input unit 2 receives the input of the end point of the matrix (step SA5). When there are a plurality of end points in the matrix to be set by the user, the matrix information input unit 2 accepts the inputs of all the end points in step SA5.

The start point, relay point, and end point input in steps SA3, SA4, and SA5 may be displayed on the image acquired in step SA1 each time they are input, and the user may be able to confirm each input point.

Next, the user confirms whether or not there is a line segment for which the number of columns (the number of people lined up side by side) has not been set for each line segment connecting the input start point, relay point, and end point. .. When the setting of the number of columns is not completed and there is an unset line segment, the matrix information input unit 2 inputs the number of columns by the user to the unset line segment via the above-mentioned input device. Accept the start (no in step SA6). The matrix information input unit 2 accepts the input of the number of columns by the user (step SA7).

It should be noted that the number of lines and columns connecting the points input in step SA7 is displayed on the image acquired in step SA1 each time the input is made, and the user can confirm the number of people in each input line. Good.

When the user completes the input of the number of columns for all the line segments (yes in step SA6), the matrix information input unit 2 returns to step SA2 and starts accepting input regarding the next matrix. The matrix information input unit 2 indicates that the user has completed the input of the number of columns for all the line segments, for example, by accepting the input by the user to that effect, or by accepting the input by the user, or for a predetermined time without the start of the input by the user. It may be determined by the passage of time.

When the matrix information input unit 2 determines that the input of all the matrices by the user is completed (yes in step SA2), the input of the matrix information in the matrix information input unit 2 is completed. The matrix information input unit 2 determines that all the matrix inputs by the user have been completed, for example, by accepting the input by the user to that effect, or by the elapse of a predetermined time without the start of the user's input. You may.

Next, the process for detecting the matrix and generating the queue waiting information will be described with reference to the flowchart of FIG. 7.

The detection unit 31 acquires a captured image and a captured time thereof from the input unit 1 (step SB1). The detection unit 31 detects all the persons existing in the image and outputs the time and the position coordinates of the detected persons (step SB2).

Next, upon receiving the tracking result of the tracking unit 32, the determination unit 33 refers to the matrix person ID list with respect to the tracking person obtained as the processing result for the previous frame image (step SB3), and determines whether or not the tracking person is a matrix person. Determine (step SB4). The procession person ID list is a list that manages a set of a person ID of a person being tracked and information indicating whether the person is a procession person or a moving person.

In the case of a matrix person (yes in step SB4), the determination unit 33 predicts the position coordinates in the current frame image using the movement model for the matrix person. The determination unit 33 calculates the position coordinates of the tracked person in the current frame image by using the predicted position coordinates and the person position coordinates obtained in step SB2 (step SB5a). As the movement model for the matrix person, the determination unit 33 may use, for example, a two-state model of a movement model representing a state in which the matrix is stationary and a movement model representing a state in which the matrix advances. Specifically, in the moving model representing the stationary state, the determination unit 33 predicts that there is almost no change in position between the previous frame image and the current frame image. On the other hand, in the movement model representing the advancing state, the determination unit 33 makes a prediction in which the movement direction is limited based on the matrix position information obtained from the matrix information input unit 2 in the previous frame image and the current frame image.

On the other hand, in the case of a moving person (no in step SB4), the determination unit 33 predicts the position coordinates in the current frame image by using the moving model for the moving person. The determination unit 33 calculates the position coordinates of the tracked person in the current frame image by using the predicted position coordinates and the person position coordinates obtained in step SB2 (step SB5b). As a movement model for a moving person, the determination unit 33 may use, for example, a model that assumes movement in all directions at a movement speed within a predetermined range because there is no limitation on the movement direction.

Next, the determination unit 33 calculates the moving speed of the person using the person position coordinates for the past several frames from the present and each time information calculated in steps SB5a and SB5b (step SB6).

The determination unit 33 calculates an appropriate distance such as the shortest distance between the person position coordinates in the current frame and the matrix position obtained from the matrix information input unit 2 (step SB7).

The determination unit 33 uses the movement speed calculated in step SB6 and the distance calculated in step SB7 to determine whether the movement speed is smaller than the predetermined threshold value and the distance is smaller than the predetermined threshold value. (Step SB8). When the above conditions are satisfied (yes in step SB8), it is determined that the person is a procession person (step SB9a). On the other hand, when the above condition is not satisfied (no in step SB8), it is determined that the person is a moving person (step SB9b).

The determination unit 33 updates the matrix person ID list managed by the determination unit 33 by using the determination result of whether or not the persons are in line in the procession in steps SB9a and SB9b (step SB10).

Next, the generation unit 34 estimates the occurrence status of occlusion based on the person position of the person determined to be the matrix person in step SB9a and the matrix position acquired from the matrix information input unit 2. The generation unit 34 performs interpolation processing of the matrix information existing in the occlusion region (step SB11). Specifically, when calculating the number of people waiting as matrix information, the generation unit 34 uses the position information of the person determined to be the matrix person in step SB9a and the matrix position information obtained from the matrix information input unit 2. , Calculate the matrix length of the undetected region interrupted by occlusion. In addition, the generation unit 34 calculates the population density (the number of people lined up within a predetermined distance) of the people in line. The generation unit 34 estimates the number of people existing in the queue length of the interrupted undetected area based on the calculated population density, and calculates the number of people waiting by adding up with the detected queue people.

The output unit 4 outputs the matrix information output from the generation unit 34 to an external module such as an application or a storage medium (step SB12).

Further, the matrix information input process shown in FIG. 6 may be performed in advance before the matrix detection process shown in FIG. 7 is started. Alternatively, at an arbitrary timing during the execution of the matrix detection process shown in FIG. 7, the process shown in FIG. 6 is performed to update the matrix information, and the matrix detection process shown in FIG. 7 is restarted using the updated matrix information. It may be done.

The program according to the first embodiment may be any program that causes a computer to execute the above series of processes.

As described above, according to the matrix detection system according to the first embodiment, even if a person does not intend to line up in a line but simply stays in the line, the person is treated as a person in line. The problem that there is is solved.

Further, even in an environment where occlusion occurs, the number of people can be interpolated by estimating the length of the matrix in the hidden area. Further, even when the matrix shape is complicated, the traveling direction for each place is known. Therefore, by utilizing the traveling direction for the movement model at the time of tracking the person in the matrix, the robustness of the tracking is improved. Therefore, according to the matrix detection system according to the first embodiment, it is possible to detect people in a queue and preferably estimate matrix information such as the number of people and the waiting time.

In addition, even if a person who is not intended to line up in a line stops near the line, it is determined that the person is not lined up in a line by appropriately setting the threshold value of the distance between the person and the line position. Becomes possible. Furthermore, in determining whether or not a person is in a line, not only the distance between the person and the line position is taken into consideration, but also the movement speed of the person is taken into consideration, so that the line position of the person passing across the line is taken into consideration. In the determination in the vicinity, it is possible to accurately determine that the person is not a person in a line.

<Embodiment 2>
Next, with reference to FIG. 8, the functional configuration of the matrix detection system according to the second embodiment of the present invention will be described.

FIG. 8 is a block diagram showing a configuration example of the second embodiment of the present invention.

As shown in FIG. 8, the matrix detection system according to the present embodiment includes a matrix information input unit 51, a detection unit 52, and a determination unit 53.

The matrix information input unit 51 accepts the input of the matrix position.

The detection unit 52 detects an object from the input image.

The determination unit 53 determines whether or not the objects are lined up in a matrix based on the distance between the object and the matrix position.

FIG. 9 is a flowchart showing the operation of the matrix detection system according to the second embodiment.

The matrix information input unit 51 accepts the input of the matrix position (step SC1).

The detection unit 52 detects an object from the input image (step SC2).

The determination unit 53 determines whether or not the objects are lined up in a matrix based on the distance between the object and the matrix position (step SC3).

As described above, according to the matrix detection system according to the second embodiment, even a person who does not intend to line up in a line but simply stays in the line is treated as a person lined up in the line. The problem that there are cases is solved.

<Hardware configuration example>
Each part constituting the matrix detection system of each of the above-described embodiments includes a control unit, a memory, a program loaded in the memory, a storage unit such as a hard disk for storing the program, an interface for network connection, and the like, and consists of hardware and software. It is realized by any combination.

FIG. 10 is a block diagram showing an example of the hardware configuration of the matrix detection system according to the first embodiment or the second embodiment.

As shown in FIG. 10, the matrix detection system includes a CPU 11 (Central Processing Unit), a communication IF 12 (communication interface 12) for network connection, a memory 13, and a storage device 14 such as a hard disk for storing a program. , Realized by computer equipment. However, the configuration of the matrix detection system is not limited to the configuration shown in FIG.

The CPU 11 operates the operating system to control the entire matrix detection system. Further, the CPU 11 reads a program or data into the memory 13 from a recording medium mounted on, for example, a drive device. When the CPU 11 reads a program or data into the memory 13, the image input unit 1, the matrix information input unit 2, the matrix detection unit 3 (detection unit 31, tracking unit 32, determination unit 33 and generation unit 34) and output unit 4 Each functional part is realized.

The matrix detection system may be connected to an image input device which is another computer, for example, via a communication IF12.

The recording device 14 is, for example, an optical disk, a flexible disk, a magnetic optical disk, an external hard disk, a semiconductor memory, or the like, and records a computer program in a computer-readable manner. The storage device 14 may be, for example, a data storage unit that stores integrated data. Further, the computer program may be downloaded from an external computer (not shown) connected to the communication network.

The matrix detection system may be connected to the input device 15 or the output device 16. The input device 15 is, for example, a keyboard, a mouse, or the like. The output device 16 may be, for example, a display or the like, and the matrix information output by the output unit 4 may be displayed on the display.

In the functional block diagrams of FIGS. 1 and 8, the constituent parts of each embodiment are described to be realized by one physically connected device, but the means for realizing the components is not particularly limited. That is, two or more physically separated devices may be connected by wire or wirelessly, and the system of each embodiment may be realized by these plurality of devices.

Although the present invention has been described above with reference to the respective embodiments and examples, the present invention is not limited to the above embodiments and examples. The configuration and details of the present invention may be modified in various ways within the scope of the present invention as may be understood by those skilled in the art.

<Other expressions of the embodiment>
Some or all of the above embodiments may also be described, but not limited to:

(Appendix 1)
Matrix information input means that accepts matrix position input,
A detection means that detects an object from an input image,
A determination means for determining whether or not the objects are lined up in a matrix based on the distance between the object and the matrix position.
Matrix detection system including.

(Appendix 2)
Based on the matrix position and the position of the object determined to be lined up by the determination means, the region where the object is not detected is estimated, and from the object determined to be lined up in the matrix. A generation means that complements an object in an undetected region and generates matrix information,
The matrix detection system according to Appendix 1, further comprising.

(Appendix 3)
The matrix detection system according to Appendix 2, wherein the generation means complements the objects in the undetected region based on the density of the objects determined to be lined up in the matrix by the determination means.

(Appendix 4)
The generation means
The waiting time of the objects determined to be in line by the determination means is calculated, and the waiting time is calculated.
The matrix detection system according to Appendix 3, which calculates the waiting time of an object in the undetected region based on the waiting time of the object detected in the regions before and after the undetected region.

(Appendix 5)
The determination means determines whether or not the objects are lined up in a matrix based on the moving speed of the object and the distance between the object and the matrix position. The matrix detection system described in.

(Appendix 6)
A tracking means that tracks the location of detected objects using different movement models depending on whether the objects are in a matrix or not.
Including
The matrix detection system according to Appendix 5, wherein the determination means calculates the movement speed from the tracking result by the tracking means.

(Appendix 7)
The tracking means
For an object determined to be waiting in a queue by the determination means, the previous frame is based on two movement models representing the waiting state and the moving state of the matrix, and the matrix position and the traveling direction acquired from the matrix information input means. Predicts the movement state from to the current frame, and also
Addendum that predicts the movement state from the previous frame to the current frame based on a movement model that assumes that the object that is not determined to be waiting in line by the determination means moves in all directions at a speed within a predetermined range. 6. The matrix detection system according to 6.

(Appendix 8)
The matrix information input means accepts inputs of at least one or more matrix start points, at least one or more matrix end points, and zero or more relay points for representing matrix waypoints or branch points. The matrix detection system according to any one of Supplementary note 1 to 7, which accepts an input of the number of columns in which objects are arranged side by side with respect to a line segment connecting adjacent start points, relay points, and end points.

(Appendix 9)
The matrix information input means
The matrix detection system according to Appendix 8, wherein the start point, the relay point, the end point, and the number of columns between the points are drawn on the input image as input information and reflected as the matrix position at an arbitrary timing.

(Appendix 10)
The detection means detects a specific part of the object and
The determination means estimates the lower end position of the object from the position of the specific portion using camera parameters, and whether the objects are lined up in a matrix based on the distance between the estimated lower end position and the matrix position. The matrix detection system according to any one of Appendix 1 to 9.

(Appendix 11)
The determination means assumes the height of the specific part in the world coordinate system with a plurality of variations, estimates a plurality of lower end positions from the assumed plurality of heights using the camera parameters, and estimates the plurality of lower ends. The matrix detection system according to Appendix 10, which determines whether or not objects are lined up in a matrix based on the shortest distance between the position and the matrix position.

(Appendix 12)
Accepts matrix position input,
Detects an object from the input image and
It is determined whether or not the objects are lined up in a matrix based on the distance between the object and the matrix position.
Matrix detection method.

(Appendix 13)
Matrix information input processing that accepts matrix position input and
Detection processing to detect an object from the input image and
Judgment processing for determining whether or not the objects are lined up in a matrix based on the distance between the object and the matrix position, and
A program that causes a computer to run.

This application claims priority on the basis of Japanese application Japanese Patent Application No. 2015-180260 filed on September 14, 2015, and the entire disclosure thereof is incorporated herein by reference.

According to the present invention, it is possible to provide a highly accurate matrix monitoring system, and it can be suitably applied to a system that detects a matrix generated in a monitoring area and presents the number of people waiting and the waiting time.

Further, in the present invention, by utilizing the generation position information of the matrix input in advance, even in an environment where occlusion is likely to occur due to overlapping of shelves, pillars, or people, no special camera installation conditions are required. Since the number of people waiting and the waiting time can be estimated without any loss, the introduction of a matrix detection system becomes easy.

1 Image input unit 2, 51 Matrix information input unit 3 Matrix detection unit 4 Output unit 31, 52 Detection unit 32 Tracking unit 33, 53 Judgment unit 34 Generation unit 11 CPU
12 Communication IF
13 Memory 14 Storage device 15 Input device 16 Output device

Claims (9)

Matrix information input means that accepts user input of matrix position,
A detection means that detects an object from an input image,
A determination means for determining whether or not the objects are arranged in a matrix based on the distance between the object and the matrix position input by the user.
Only including,
The matrix information input means accepts inputs of at least one or more matrix start points, at least one or more matrix end points, and zero or more relay points for representing matrix waypoints or branch points. A matrix detection system that accepts inputs for the number of columns in which objects are arranged side by side for a line connecting between adjacent start points, relay points, and end points. Based on the matrix position and the position of the object determined to be lined up by the determination means, the region where the object is not detected is estimated, and from the object determined to be lined up in the matrix. A generation means that complements an object in an undetected region and generates matrix information,
The matrix detection system according to claim 1, further comprising. The matrix detection system according to claim 2, wherein the generation means complements the objects in the undetected region based on the density of the objects determined to be lined up in the matrix by the determination means. The generation means
The waiting time of the objects determined to be in line by the determination means is calculated, and the waiting time is calculated.
The matrix detection system according to claim 3, wherein the waiting time of the object in the undetected region is calculated based on the waiting time of the object detected in the regions before and after the undetected region. Any of claims 1 to 4, wherein the determination means determines whether or not the objects are lined up in a matrix based on the moving speed of the object and the distance between the object and the matrix position. The matrix detection system according to one. A tracking means that tracks the location of detected objects using different movement models depending on whether the objects are in a matrix or not.
Including
The matrix detection system according to claim 5, wherein the determination means calculates the movement speed from the tracking result by the tracking means. The tracking means
For an object determined to be waiting in a queue by the determination means, the previous frame is based on two movement models representing the waiting state and the moving state of the matrix, and the matrix position and the traveling direction acquired from the matrix information input means. Predicts the movement state from to the current frame, and also
A claim that predicts the movement state from the previous frame to the current frame based on a movement model that assumes that an object that is not determined to be waiting in line by the determination means moves in all directions at a speed within a predetermined range. Item 6. The matrix detection system according to item 6. Accepts user input of matrix position
Detects an object from the input image and
Based on the distance between the object and the matrix position input by the user, it is determined whether or not the object is lined up in the matrix .
In accepting the matrix position, input of at least one or more matrix start points, at least one or more matrix end points, and zero or more relay points for representing a matrix waypoint or a branch point is accepted. Accepts the input of the number of columns in which the objects are arranged side by side for the line segment connecting each adjacent start point, relay point, and end point.
Matrix detection method. Matrix information input processing that accepts input of matrix position by the user,
Detection processing to detect an object from the input image and
A determination process for determining whether or not the objects are lined up in a matrix based on the distance between the object and the matrix position input by the user.
Let the computer run
The matrix information input process accepts inputs of at least one or more matrix start points, at least one or more matrix end points, and zero or more relay points for representing matrix waypoints or branch points. A program that accepts input for the number of columns in which objects are arranged side by side for a line segment connecting adjacent start points, relay points, and end points.

Images