JPWO2017047060A1 - Matrix detection system, method and recording medium - Google Patents

Abstract

  Even a person who is not intending to line up but is merely staying in the line may be treated as a person lined up in the line. The matrix detection system according to the present invention includes a matrix information input unit that receives an input of a matrix position, a detection unit that detects an object from an input image, and a distance between the object and the matrix position. And a determination unit that determines whether or not they are arranged.

Description

  The present invention relates to a system, a method, and a recording medium for detecting a matrix from an image.

  Various methods for detecting a matrix from a camera image and methods for analyzing the number of people forming the matrix, their length, waiting time, and the like have been proposed.

  In the technique described in Patent Document 1, the area of a person region is obtained from an input image using a background difference, and the number of persons is estimated by dividing the area by a predetermined unit area per person. The technique described in Patent Literature 1 estimates the waiting time by dividing the estimated number of persons by the number of visitors per unit time separately counted by a predetermined method.

  The technique described in Patent Document 2 captures a matrix from directly above with a camera installed on a ceiling, and extracts individual person areas that are staying and moving using background difference and optical flow. The technique described in Patent Document 2 generates a matrix line by sequentially connecting the extracted individual person regions, and calculates a waiting time based on the total extension and length of each matrix.

JP 2002-329196 A JP 2007-317052 A

  However, in the techniques described in Patent Documents 1 and 2 described above, even a person who is not intending to line up in a queue but is simply staying may be processed as a person in line. .

  The objective of this invention is providing the technique which can solve the above-mentioned subject.

  The matrix detection system according to the present invention includes a matrix information input unit that receives an input of a matrix position, a detection unit that detects an object from an input image, and a distance between the object and the matrix position. And a determination unit that determines whether or not they are arranged.

  The matrix detection method according to the present invention receives an input of a matrix position, detects an object from an input image, and determines whether or not the object is arranged in a matrix based on a distance between the object and the matrix position. .

  The computer-readable recording medium according to the present invention includes a matrix information input process that receives an input of a matrix position, a detection process that detects an object from an input image, and a distance between the object and the matrix position. A program for causing a computer to execute determination processing for determining whether or not are arranged in a matrix is stored.

  According to the present invention, there is a problem that even if a person does not intend to line up in a queue but is simply staying, the person may be mistakenly processed as a person in line. .

It is a functional block diagram which shows the structural example of Embodiment 1 of this invention. It is explanatory drawing which shows the example of the input method of matrix information. It is explanatory drawing which shows the example of the input method of matrix information. It is explanatory drawing which shows the example of the production | generation process of queue waiting information. It is explanatory drawing which shows the example of the production | generation process of queue waiting information. It is a flowchart which shows the example of the process progress of Embodiment 1 of this invention. It is a flowchart which shows the example of the process progress of Embodiment 1 of this invention. It is a functional block diagram which shows the structural example of Embodiment 2 of this invention. It is a flowchart which shows the example of process progress of Embodiment 2 of this invention. It is a figure explaining the hardware structural example of the computer which can implement | achieve each embodiment of this invention.

  First, in order to facilitate understanding of the embodiments of the present invention, the background of the present invention will be described.

  In the techniques described in Patent Documents 1 and 2 described above, even a person who is not intending to line up in a queue but is simply staying may be processed as a person in line.

  For example, the technique described in Patent Document 1 uses the background difference to determine the area of the person area for the input image. Therefore, if a person who is not going to line up in the matrix stops near the matrix, the technique simply stays there. The area of the person area including those who are present is obtained. Further, when there are a plurality of staying persons at positions away from the original procession, the technique described in Patent Document 1 can recognize that the plurality of staying persons are people who form the procession. The technique described in Patent Document 1 includes undetected due to occurrence of occlusion as other problems. In a method based on background difference as in Patent Document 1, a person area acquired based on a difference from an actual person due to a difference in the physique of the person, the presence / absence of an accessory such as luggage, and the overlap between persons or hiding by a shield An error is easily generated in the relationship. In addition, this method does not recognize individual persons and their arrangement order, and therefore cannot calculate the waiting time for each person.

  Further, in the technique described in Patent Document 2, similarly to Patent Document 1, an erroneous process may be performed in which a mere staying person is a person lining up in a queue. In addition, as another problem, the method described in Patent Document 2 is based on the premise that human detection and tracking can be performed well. However, in an actual environment, depending on the installation position of the camera, the structure of the building, and the like, Occlusion due to overlapping and shielding is assumed. As a result, a person detection omission occurs in the occlusion area, and as a result, it fails to correctly connect the persons lined up in the matrix, and the wrong number of persons and waiting time are estimated. Patent Document 2 proposes that a camera is installed at a high position and a person is photographed from directly above in order to solve such an occlusion problem. However, an environment where a camera can be installed at a high position in a general indoor facility or outdoors is not always prepared. Further, since the installation position and the angle of view are different from those of a general monitoring camera, it is necessary to newly install as a camera dedicated to matrix detection, leading to an increase in introduction cost. Therefore, it is desirable to solve the problem of occlusion by using an existing surveillance camera image that captures a person from diagonally above rather than using a camera with special installation conditions.

  According to the embodiments of the present invention described below, the above-described problems are solved, erroneous detection of unintended persons in a queue is prevented, and even if the camera is not a special camera such as a ceiling-mounted camera, occlusion can be achieved. A matrix can be detected robustly.

  Embodiments of the present invention will be described below with reference to the drawings.

<Embodiment 1>
FIG. 1 is a block diagram showing a configuration example of Embodiment 1 of the present invention.

  As shown in FIG. 1, the matrix detection system according to the present embodiment includes an image input unit 1, a matrix information input unit 2, a matrix detection unit 3, and an output unit 4.

  The image input unit 1 sequentially inputs a set of a time-series image obtained by photographing a predetermined monitoring area and a photographing time to the matrix detection unit 3. An image input by the image input unit 1 to the matrix detection unit 3 may be acquired using a photographing device such as a surveillance camera. Alternatively, the image input unit 1 may input a time-series image obtained by reading video data stored in a storage device (not shown) to the matrix detection unit 3.

  Note that the type of object to be detected in the present embodiment is not particularly limited, and may be a person, an animal, a car, a robot, or an object. In the following, description will be given by taking the detection of a person as an example.

  The matrix information input unit 2 inputs a generation position of a matrix to be analyzed (hereinafter referred to as “matrix position”). The matrix information input unit 2 accepts input of a matrix generation position by a user via, for example, a mouse or a touch panel of a tablet terminal. The matrix information input unit 2 may input the generation position of the matrix on the screen on which the monitoring camera video is displayed based on the mouse operation by the user or the touch panel operation with a touch pen or a finger. Specifically, the matrix information input unit 2 may draw the start point, the relay point, and the end point of the person's foot position when the matrix is generated on the screen, and input the matrix path and the traveling direction. At this time, the user needs to designate at least one start point and one end point, but the relay point may be designated as 0 points. Further, in order to express a matrix including branching and merging, the user may input two or more of at least one of the start point and the end point. Further, the matrix information input unit 2 may accept designation of the number of people lined up horizontally for each matrix line connecting the start point, the relay point, and the end point.

  Also, the input start point, relay point, end point, and number of people lined up side by side may be drawn on the screen on which the surveillance camera video is displayed so that the user can easily check the input content. . Further, the input of the matrix information may be performed in advance before starting the matrix detection process, or the changed matrix information may be reflected at an arbitrary timing during the execution of the matrix detection process. .

  Note that the matrix information input method is not limited to the above-described one. For example, instead of inputting the start point, relay point, and end point of the person's foot position when the matrix is generated, the user may input the matrix position by tracing a straight line or curve directly on the touch panel. . Alternatively, the matrix information input unit 2 automatically inputs the matrix position by a method such as reading a file in which the matrix position is described in a predetermined format stored inside or outside the apparatus, not based on an input by the user. You may do it.

  Here, the input of matrix information will be specifically described. FIG. 2 is a diagram showing an image obtained by photographing the people lined up in the matrix with the monitoring camera. FIG. 2 shows a state in which the people are lined up from the back toward the front. In addition, FIG. 2 shows a state in which the last part of the matrix is divided into two branches, but merges in the middle to form a two-column matrix that continues to the top. FIG. 3 shows, as matrix information, two end points (end point 1 and end point 2) that are the end of the matrix, two relay points (relay point 1 and relay point 2), and one start point that is the head of the matrix ( A state in which the start point 1) is input is shown. FIG. 3 shows a state where the end points 1 and 2 to the relay point 1 are input as a one-column matrix, and the relay point 1 to the start point 1 are input as a two-column matrix.

  The matrix information input unit 2 reflects the start point, the relay point, the end point, and information between each point input as described above as a matrix position at an arbitrary timing. For example, a “reflect” button for reflecting the input information may be displayed on the display screen, and when the user presses the button, the matrix information input unit 2 may reflect the input information. Thus, the reflected matrix information is used for determination by the determination unit 33 and the like.

  The matrix detection unit 3 analyzes images sequentially input from the image input unit 1 and detects a matrix in the image. The matrix detection unit 3 includes a detection unit 31, a tracking unit 32, a determination unit 33, and a generation unit 34.

  The detection unit 31 detects all or some of the persons present in the image acquired from the image input unit 1. For example, the detection unit 31 may perform processing for detecting a person from an input image using a discriminator constructed in advance to recognize a pattern of a human body region of a specific part such as a person's head or upper body. It is not limited to the method using a discriminator, The detection part 31 should just use the well-known method in pattern recognition as a person detection method.

  In addition, the detection unit 31 performs detection processing by limiting the detection target region based on the matrix position input from the matrix information input unit 2 in order to target only persons existing in the vicinity of the matrix position. You may do it. That is, the detection unit 31 may perform the detection process only on a region near the input matrix position.

  The detection unit 31 outputs the time of the image acquired from the image input unit 1 and the position coordinates of the detected person as detection results. If the calibration information of the camera that captured the input image is known in advance, the detection unit 31 uses the calibration information to determine the position coordinates on the screen as a person detection result using the world coordinates. May be output after being converted to.

  The tracking unit 32 tracks a person over a plurality of frames for a person detected from each frame image by the detection unit 31. The tracking unit 32 may perform a person tracking process using a known algorithm such as a particle filter.

  The tracking unit 32 may use a different movement model regarding the movement model used for updating the position of the person to be tracked depending on whether the object is a person arranged in a matrix. The tracking unit 32 may refer to, for example, a procession person ID (IDentifier) list managed by the determination unit 33 as to whether or not the person is in line. In that case, the tracking unit 32 determines that the person ID to be tracked is registered in the matrix person ID list, and determines that the person is not aligned in the matrix if not registered. May be.

  Most of the people who are in line are in a stationary state. Therefore, it is appropriate to apply a movement model that represents a stationary state (that is, hardly moves) in many times. Further, the traveling direction for each place is known from the matrix position information obtained from the matrix information input unit 2. Therefore, a movement model with a limited traveling direction can be applied. In other words, the tracking unit 32 can perform tracking more robustly by performing tracking processing using the two-state model representing the moving state and the stationary state for the persons arranged in the matrix. The tracking unit 32 may change the weight of each state model depending on whether the people lined up in the matrix are stationary or moving.

  On the other hand, there are no restrictions on the direction of movement of persons who are not in the queue. For this reason, the tracking unit 32 may apply a movement model assuming that the person moves in all directions at a moving speed within a predetermined range for persons who are not arranged in a matrix.

  When the position information of the input person is a plane coordinate on the image, and the calibration information of the camera that has captured the input image is known due to circumstances such as being calculated in advance, the tracking unit 32 uses the world coordinates. The tracking process may be performed in the system. That is, the tracking unit 32 may perform the tracking process after converting the coordinate system on the screen to the world coordinate system using the calibration information. When the position information of the person is input in the world coordinate system, the tracking unit 32 may perform the tracking process in the world coordinate system as it is.

  The tracking unit 32 outputs a tracking result. For example, the tracking unit 32 outputs information in which a person ID for each person to be tracked, time, and position coordinates are linked as a tracking result.

  The determination unit 33 calculates the movement speed of each person based on the position coordinates of the person for each time output from the tracking unit 32. The determination unit 33 compares the position coordinates of the person with the coordinates of the matrix position input from the matrix information input unit 2, and calculates the distance between each person and the matrix position.

  The determination unit 33 determines a person whose distance from the moving speed and the matrix position is smaller than a predetermined threshold as a person arranged in a matrix (hereinafter referred to as “matrix person”). The determination unit 33 determines a person whose distance from the moving speed or the matrix position is equal to or greater than a predetermined threshold as a person who is not in the matrix (hereinafter referred to as “moving person”).

  When the detection unit 31 detects the head as the specific part, the determination unit 33 uses the camera parameters to estimate the lower end position corresponding to the person's feet from the position of the head, and the estimated lower end position and the matrix position Based on the distance, it may be determined whether or not people are arranged in a matrix. Thereby, even when it is difficult to directly detect the feet, the head 33 that is relatively easy to detect is detected, and the feet are estimated from the detected heads. Can be determined. The determination unit 33 may consider the possibility that the person is an adult or a child. Specifically, for example, the determination unit 33 assumes the height of the head in the world coordinate system with a plurality of variations, and estimates a plurality of lower end positions from the assumed plurality of heights using camera parameters. The determination unit 33 may determine whether or not the objects are arranged in a matrix based on the estimated shortest distances between the plurality of lower end positions and the matrix positions.

  The determination unit 33 holds a matrix person ID list indicating whether each person ID being tracked is a queue person or a moving person. The procession person ID list is a list that manages a set of a person ID of a person being tracked and information indicating whether the person is a procession person or a moving person.

  The determination unit 33 updates the matrix person ID list for each person ID being tracked using a determination result based on the distance between the moving speed and the matrix position.

  The determination unit 33 outputs information on the person who is determined to be a procession person. For example, the determination unit 33 outputs information in which a person ID of a person determined to be a procession person, time, and position coordinates are associated with each other.

  The generation unit 34 generates the entire matrix information having no deficiency based on the matrix person information input from the determination unit 33 and the matrix position information input from the matrix information input unit 2. In other words, the generation unit 34 estimates the undetected area, complements the information of the persons lined up in the undetected area from the information of the matrix person, and generates the matrix information.

  Specifically, the generation unit 34 estimates the occurrence state of an occlusion (an undetected region of a person due to the influence of occlusion) based on the position coordinates of the matrix person and the matrix position. The generation unit 34 estimates the presence of a person that has not been detected due to the influence of the occlusion, based on the estimated occurrence state of occlusion and the detection situation of matrix people before and after the undetected area. In addition, the generation unit 34 estimates a region in which a person cannot be detected because the appearance is too small (an undetected region of a person due to the influence of appearance). The generation unit 34 estimates the presence of a person who has not been detected from the possibility of undetection due to the influence of appearance and the detection state of the matrix person. The generation unit 34 generates the entire matrix information having no defect by complementing the information of the person that has not been detected but is estimated to exist. The generation unit 34 includes, for example, the number of people lined up in the matrix, the length of the matrix, the accumulated waiting time of each person lined up in the matrix, the speed of progression of the matrix, the time from starting to line up in the matrix until the exit of the matrix Information such as the estimated waiting time may be generated.

  A method for estimating the occurrence state of occlusion will be described with reference to FIGS.

  FIG. 4 is a diagram illustrating an image obtained by photographing a matrix with wrapping. FIG. 4 shows a state in which the upper left corner of the image is the starting point (top) of the matrix, it is folded twice in the middle, and the lower right corner is the end point (last tail) of the matrix.

  Images taken with an angle of view looking down diagonally have a smaller depression angle as the distance from the camera increases. For this reason, even in a matrix in which people are arranged at equal intervals, the person farther from the camera is more likely to be hidden by the person in front. Also, the further away from the camera, the smaller the appearance of the person. For this reason, a person who is far away from the camera may not be detected if the person is below the minimum number of pixels that can detect the person. As a specific example, FIG. 4 shows that in the shaded area, the occlusion and the appearance of the person are too small to be detected.

  In such a situation, for example, when calculating the waiting number of the entire queue, the generation unit 34 calculates, for example, by the following method. The generation unit 34 calculates the interrupted matrix length based on the position coordinates of the person determined as the matrix person by the determination unit 33 and the matrix position input from the matrix information input unit 2. Further, the generation unit 34 estimates the population density (number of people lined up within a predetermined distance) from the area where the person position is obtained. Based on the estimated population density, the generation unit 34 estimates the number of people present in the interrupted matrix length, and adds the number of detected people to calculate the waiting number. The generation unit 34 calculates the waiting number by the above-described method, for example.

  Moreover, when calculating the accumulated waiting time of each person, the generation unit 34 calculates, for example, by the following method. The generation unit 34 estimates the number of people existing in the occlusion area by the method described above. The generation unit 34 calculates the accumulated waiting time after the person detected immediately before and after the occlusion area starts to be arranged in the matrix. The generation unit 34 divides the calculated time difference between the two accumulated waiting times by the estimated number of persons existing in the occlusion area. The generation unit 34 adds the time difference calculated by the division to the accumulated waiting time of the persons arranged in order from the first person existing in the occlusion region, thereby performing interpolation processing of the accumulated waiting time. Do.

  The output unit 4 outputs the matrix information input from the matrix detection unit 3. The output mode of the output unit 4 is, for example, display output. In this case, the output unit 4 may perform display on a display device provided outside. However, the output mode of the output unit 4 is not limited to display, and may be other modes.

  Next, the operation of the matrix detection system according to the present embodiment will be described with reference to the flowcharts of FIGS. FIG. 6 shows an example of a processing flow when the user inputs a matrix position. FIG. 7 shows an example of a processing flow for detecting a matrix after generating a matrix position and generating the entire matrix information without a defect.

  Each processing step to be described later may be executed in any order or in parallel as long as there is no contradiction in processing contents, and other steps may be added between the processing steps. good. Further, a step described as one step for convenience may be executed by being divided into a plurality of steps, and a step described as being divided into a plurality of steps for convenience may be executed as one step.

  First, processing when a user inputs a matrix position will be described with reference to the flowchart of FIG.

  The matrix information input unit 2 acquires an image obtained by photographing a predetermined monitoring area and displays it on a screen for setting the matrix position (step SA1).

  Next, the user confirms whether or not an unset matrix exists on the displayed screen. When the matrix setting is not completed and there is an unset matrix, the matrix information input unit 2 accepts the start of input of the matrix position by the user via a predetermined input device (no in step SA2). As the input device, for example, a mouse or a touch panel is applicable, and if it is a touch panel, the user may input using a touch pen or a finger.

  The matrix information input unit 2 accepts an input of a matrix starting point by the user (step SA3). If there are a plurality of start points in the matrix to be set by the user, the matrix information input unit 2 accepts input of all start points in step SA3.

  Next, the matrix information input unit 2 accepts an input of a relay point representing a branch of a matrix or a waypoint by a user (step SA4). If the matrix can be represented by a straight line connecting the start point and the end point, it is not necessary to input a relay point, so the process of step SA4 is omitted. If there are a plurality of relay points in the matrix to be set by the user, the matrix information input unit 2 accepts input of all the relay points in step SA4.

  Next, the matrix information input unit 2 accepts input of the end point of the matrix (step SA5). If there are a plurality of end points in the matrix to be set by the user, the matrix information input unit 2 accepts input of all end points in step SA5.

  Note that the start point, the relay point, and the end point input in steps SA3, SA4, and SA5 may be displayed on the image acquired in step SA1 each time they are input, and the user may be able to confirm each input point.

  Next, the user confirms whether or not there is a line segment for which the input of the number of columns (number of people lined up) is not set for each line segment connecting the input start point, relay point, and end point. . When the setting of the number of columns is not completed and there is an unset line segment, the matrix information input unit 2 inputs the number of columns by the user via the input device described above for the unset line segment. The start is accepted (step SA6: no). The matrix information input unit 2 accepts input of the number of columns by the user (step SA7).

  Each line segment and the number of columns connecting the points input in step SA7 are displayed on the image acquired in step SA1 each time they are input, and the user can check the number of input line segments. Good.

  When the input of the number of columns for all the line segments by the user is completed by the user (Yes in step SA6), the matrix information input unit 2 returns to step SA2 and starts accepting input regarding the next matrix. The matrix information input unit 2 confirms that the input of the number of columns for all the line segments by the user has been completed, for example, by accepting an input by the user to that effect, or for a predetermined time without starting the user's input. You may determine by passing.

  When the matrix information input unit 2 determines that the input of all matrices by the user is completed (yes in step SA2), the matrix information input in the matrix information input unit 2 is completed. The matrix information input unit 2 determines that the input of all the matrices by the user has been completed, for example, by accepting an input by the user to that effect, or by elapse of a predetermined time without starting the user's input May be.

  Next, a process for detecting a queue and generating queue wait information will be described with reference to the flowchart of FIG.

  The detection unit 31 acquires a captured image and its shooting time from the input unit 1 (step SB1). The detection unit 31 detects all persons existing in the image, and outputs the time and the position coordinates of the detected person (step SB2).

  Next, upon receiving the tracking result of the tracking unit 32, the determination unit 33 refers to the matrix person ID list for the traced person obtained as the processing result for the previous frame image (step SB3), and determines whether or not the person is a matrix person. Determine (step SB4). The matrix person ID list is a list that manages a pair of a person ID of a person being tracked and information indicating whether the person is a queue person or a moving person.

  When it is a procession person (yes in step SB4), the determination unit 33 predicts the position coordinates in the current frame image using the process model for procession person. The determination unit 33 calculates the position coordinates of the tracking person in the current frame image using the predicted position coordinates and the person position coordinates obtained in Step SB2 (Step SB5a). The determination unit 33 may use, for example, a two-state model, which is a movement model representing a state in which the matrix is stationary and a movement model representing a state in which the matrix advances, as the movement model for the matrix person. Specifically, in the movement model representing the stationary state, the determination unit 33 predicts that there is almost no change in position between the previous frame image and the current frame image. On the other hand, in the movement model representing the advancing state, the determination unit 33 performs prediction in which the movement direction is limited based on the matrix position information obtained from the matrix information input unit 2 between the previous frame image and the current frame image.

  On the other hand, when it is a moving person (no in step SB4), the determination unit 33 predicts position coordinates in the current frame image using a moving model for the moving person. The determination unit 33 calculates the position coordinates of the tracking person in the current frame image using the predicted position coordinates and the person position coordinates obtained in Step SB2 (Step SB5b). The determination unit 33 may use, for example, a model that is assumed to move in all directions at a moving speed within a predetermined range because the moving direction is not limited as a moving model for a moving person.

  Next, the determination unit 33 calculates the movement speed of the person by using the person position coordinates and time information for the past several frames from the present time calculated in Step SB5a and Step SB5b (Step SB6).

  The determination unit 33 calculates an appropriate distance such as the shortest distance between the human position coordinates in the current frame and the matrix position obtained from the matrix information input unit 2 (step SB7).

  The determination unit 33 uses the movement speed calculated in step SB6 and the distance calculated in step SB7 to determine whether the movement speed is smaller than a predetermined threshold and whether the distance is smaller than a predetermined threshold. (Step SB8). When the condition is satisfied (yes in step SB8), it is determined that the person is a procession person (step SB9a). On the other hand, if the condition is not satisfied (no in step SB8), the person is determined to be a moving person (step SB9b).

  The determination unit 33 updates the matrix person ID list managed by the determination unit 33 using the determination result of whether or not the person is in the matrix in step SB9a and step SB9b (step SB10).

  Next, the generation unit 34 estimates the occurrence state of occlusion based on the person position of the person determined to be a matrix person in step SB9a and the matrix position acquired from the matrix information input unit 2. The generation unit 34 performs interpolation processing of matrix information existing in the occlusion area (step SB11). Specifically, when calculating the waiting number as the queue information, the generation unit 34 uses the position information of the person determined to be a queue person in step SB9a and the matrix position information obtained from the matrix information input unit 2. The matrix length of the undetected area interrupted by occlusion is calculated. Further, the generation unit 34 calculates the population density (number of people lined up within a predetermined distance) of the procession figures. The generation unit 34 estimates the number of persons existing in the matrix length of the undetected region that has been interrupted based on the calculated population density, and calculates the number of waiting persons by adding the detected matrix persons.

  The output unit 4 outputs the matrix information output from the generation unit 34 to an external module such as an application or a storage medium (Step SB12).

  Further, the matrix information input process shown in FIG. 6 may be performed in advance before the matrix detection process shown in FIG. 7 is started. Alternatively, at any timing when the matrix detection process shown in FIG. 7 is being executed, the process shown in FIG. 6 is performed to update the matrix information, and the matrix detection process shown in FIG. 7 is resumed using the updated matrix information. You may be made to do.

  The program according to the first embodiment may be a program that causes a computer to execute the series of processes described above.

  As described above, according to the matrix detection system according to the first embodiment, even if a person is not intended to line up in a line but is merely staying, the person is processed as a person lined up in the line The problem that there is is solved.

  Further, even in an environment where occlusion occurs, the number of persons can be interpolated by estimating the length of a hidden region matrix. Further, even when the matrix shape is complicated, the traveling direction for each place is known, so that the robustness of tracking is improved by utilizing the traveling direction for the movement model at the time of matrix person tracking. Therefore, according to the matrix detection system according to the first embodiment, it is possible to detect the persons lined up in the matrix and to appropriately estimate the matrix information such as the number of people and the waiting time.

  Also, even when a person who is not intended to line up in the queue stops near the line, it is determined that the person is not lined up in the line by appropriately setting the threshold of the distance between the person and the line position Is possible. Furthermore, in determining whether or not a person is in a matrix, not only the distance between the person and the matrix position, but also the movement speed of the person, the matrix position for the person passing across the matrix In the determination in the vicinity, it is possible to accurately determine that the person is not a person in a queue.

<Embodiment 2>
Next, a functional configuration of the matrix detection system according to the second embodiment of the present invention will be described with reference to FIG.

  FIG. 8 is a block diagram showing a configuration example of the second embodiment of the present invention.

  As shown in FIG. 8, the matrix detection system according to the present embodiment includes a matrix information input unit 51, a detection unit 52, and a determination unit 53.

  The matrix information input unit 51 receives an input of a matrix position.

  The detection unit 52 detects an object from the input image.

  The determination unit 53 determines whether or not the objects are arranged in a matrix based on the distance between the object and the matrix position.

  FIG. 9 is a flowchart showing the operation of the matrix detection system according to the second embodiment.

  The matrix information input unit 51 receives an input of a matrix position (step SC1).

  The detection unit 52 detects an object from the input image (step SC2).

  The determination unit 53 determines whether or not the objects are arranged in a matrix based on the distance between the object and the matrix position (step SC3).

  As described above, according to the matrix detection system according to the second embodiment, even a person who is not intending to line up in a queue but merely stays in the line is processed as a person lined up in the line. The problem that there is a case is solved.

<Hardware configuration example>
Each unit constituting the matrix detection system of each of the above embodiments includes a control unit, a memory, a program loaded in the memory, a storage unit such as a hard disk for storing the program, a network connection interface, and the like. Realized by any combination.

  FIG. 10 is a block diagram illustrating an example of a hardware configuration of the matrix detection system according to the first embodiment or the second embodiment.

  As shown in FIG. 10, the matrix detection system includes a CPU 11 (Central Processing Unit), a network connection communication IF 12 (communication interface 12), a memory 13, and a storage device 14 such as a hard disk for storing a program. It is realized by a computer device. However, the configuration of the matrix detection system is not limited to the configuration shown in FIG.

  The CPU 11 operates the operating system to control the entire matrix detection system. Further, the CPU 11 reads a program and data into the memory 13 from a recording medium mounted on, for example, a drive device. The CPU 11 reads the program and data into the memory 13, so that the image input unit 1, the matrix information input unit 2, the matrix detection unit 3 (the detection unit 31, the tracking unit 32, the determination unit 33, and the generation unit 34) and the output unit 4 Each functional unit is realized.

  The matrix detection system may be connected to an image input device which is another computer via the communication IF 12, for example.

  The recording device 14 is, for example, an optical disk, a flexible disk, a magnetic optical disk, an external hard disk, a semiconductor memory, or the like, and records a computer program so that it can be read by a computer. The storage device 14 may be, for example, a data storage unit that stores integrated data. The computer program may be downloaded from an external computer (not shown) connected to the communication network.

  The matrix detection system may be connected to the input device 15 or the output device 16. The input device 15 is, for example, a keyboard or a mouse. The output device 16 may be a display, for example, and may display the matrix information output by the output unit 4 on the display.

  In the functional block diagrams of FIG. 1 and FIG. 8, the components of each embodiment are described as being realized by one physically coupled device, but the means for realizing it is not particularly limited. That is, two or more physically separated devices may be connected by wire or wirelessly, and the system of each embodiment may be realized by the plurality of devices.

  As mentioned above, although this invention was demonstrated with reference to each embodiment and an Example, this invention is not limited to the above embodiment and Example. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

<Other expressions of the embodiment>
A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1)
Matrix information input means for receiving an input of a matrix position;
Detection means for detecting an object from an input image;
Determination means for determining whether or not the objects are arranged in a matrix based on a distance between the objects and the matrix position;
Matrix detection system including

(Appendix 2)
Based on the matrix position and the position of the target object determined to be aligned in the matrix by the determination means, the region where the target object is not detected is estimated, and from the target object determined to be aligned in the matrix Generating means for complementing the object in the undetected area and generating matrix information;
The matrix detection system according to claim 1, further comprising:

(Appendix 3)
The matrix detection system according to claim 2, wherein the generation unit supplements the object in the undetected region based on the density of the object determined to be arranged in the matrix by the determination unit.

(Appendix 4)
The generating means includes
Calculating the waiting time of the objects determined to be arranged in a matrix by the determining means;
The matrix detection system according to claim 3, wherein the waiting time of the object in the undetected area is calculated based on the waiting time of the object detected in the areas before and after the undetected area.

(Appendix 5)
The determination means determines whether or not the objects are arranged in a matrix based on a moving speed of the object and a distance between the object and the matrix position. The matrix detection system described in 1.

(Appendix 6)
A tracking means for tracking the position of the detected object using different movement models depending on whether or not the object is in a matrix;
Further including
The matrix detection system according to claim 5, wherein the determination unit calculates the moving speed from a tracking result obtained by the tracking unit.

(Appendix 7)
The tracking means includes
Based on the two movement models representing the waiting state and the moving state of the queue and the queue position and the traveling direction obtained from the queue information input means for the object determined to be queued by the judging means, the previous frame Predict the movement state from to the current frame,
Predict the movement state from the previous frame to the current frame based on a movement model assuming that the object that has not been determined to be queued by the determination means moves in all directions at a speed within a predetermined range. 7. The matrix detection system according to 6.

(Appendix 8)
The matrix information input means accepts input of at least one or more matrix start points, at least one or more matrix end points, and zero or more relay points for representing the waypoints or branch points of the matrix, The matrix detection system according to any one of appendices 1 to 7, which receives an input of the number of columns in which the object is arranged side by side with respect to a line segment connecting each adjacent start point, relay point, and end point.

(Appendix 9)
The matrix information input means includes
The matrix detection system according to appendix 8, wherein the start point, the relay point, the end point, and the number of columns between the points on the input image are used as input information and reflected as the matrix position at an arbitrary timing.

(Appendix 10)
The detecting means detects a specific part of the object;
The determination means estimates the lower end position of the object from the position of the specific part using camera parameters, and based on the distance between the estimated lower end position and the matrix position, whether the object is arranged in a matrix The matrix detection system according to any one of appendices 1 to 9, which determines whether or not.

(Appendix 11)
The determination means assumes the height of the specific part in the world coordinate system in a plurality of variations, uses the camera parameters to estimate a plurality of lower end positions from the assumed plurality of heights, and estimates the plurality of estimated lower ends The matrix detection system according to claim 10, wherein it is determined whether or not the objects are arranged in a matrix based on a shortest distance between the position and the matrix position.

(Appendix 12)
Accepts matrix position input,
Detect the object from the input image,
Determining whether the objects are arranged in a matrix based on the distance between the objects and the matrix position;
Matrix detection method.

(Appendix 13)
Matrix information input processing for receiving input of matrix positions;
A detection process for detecting an object from an input image;
A determination process for determining whether or not the objects are arranged in a matrix based on a distance between the object and the matrix position;
A program that causes a computer to execute.

  This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2015-180260 for which it applied on September 14, 2015, and takes in those the indications of all here.

  ADVANTAGE OF THE INVENTION According to this invention, an accurate queue monitoring system can be provided, and it can apply suitably for the system which detects the queue which generate | occur | produces in a monitoring area | region, and shows waiting number of persons and waiting time.

  Further, in the present invention, by utilizing the occurrence position information of the matrix input in advance, even in an environment where occlusion due to overlapping of shelves, pillars, and people is likely to occur, without requiring special camera installation conditions, Since the waiting number and waiting time can be estimated without loss, it is easy to introduce a matrix detection system.

DESCRIPTION OF SYMBOLS 1 Image input part 2, 51 Matrix information input part 3 Matrix detection part 4 Output part 31, 52 Detection part 32 Tracking part 33, 53 Determination part 34 Generation part 11 CPU
12 Communication IF
13 Memory 14 Storage Device 15 Input Device 16 Output Device

Claims (13)

Matrix information input means for receiving an input of a matrix position;
Detection means for detecting an object from an input image;
Determination means for determining whether or not the objects are arranged in a matrix based on a distance between the objects and the matrix position;
Matrix detection system including Based on the matrix position and the position of the target object determined to be aligned in the matrix by the determination means, the region where the target object is not detected is estimated, and from the target object determined to be aligned in the matrix Generating means for complementing the object in the undetected area and generating matrix information;
The matrix detection system according to claim 1, further comprising: The matrix detection system according to claim 2, wherein the generation unit supplements the object in the undetected region based on the density of the objects determined to be arranged in the matrix by the determination unit. The generating means includes
Calculating the waiting time of the objects determined to be arranged in a matrix by the determining means;
The matrix detection system according to claim 3, wherein the waiting time of the object in the undetected area is calculated based on the waiting time of the object detected in the areas before and after the undetected area. The said determination means determines whether the said target object is located in a matrix based on the moving speed of the said target object, and the distance of the said target object and the said matrix position. The matrix detection system described in Crab. A tracking means for tracking the position of the detected object using different movement models depending on whether or not the object is in a matrix;
Further including
The matrix detection system according to claim 5, wherein the determination unit calculates the moving speed from a tracking result obtained by the tracking unit. The tracking means includes
Based on the two movement models representing the waiting state and the moving state of the queue and the queue position and the traveling direction obtained from the queue information input means for the object determined to be queued by the judging means, the previous frame Predict the movement state from to the current frame,
The movement state from the previous frame to the current frame is predicted based on a movement model that is assumed to move in all directions at a speed within a predetermined range for an object that has not been determined to be queued by the determination unit. Item 7. The matrix detection system according to Item 6. The matrix information input means accepts input of at least one or more matrix start points, at least one or more matrix end points, and zero or more relay points for representing the waypoints or branch points of the matrix, The matrix detection system according to any one of claims 1 to 7, wherein an input of the number of columns in which the object is arranged side by side is received for a line segment connecting each adjacent start point, relay point, and end point. The matrix information input means includes
The matrix detection system according to claim 8, wherein the start point, the relay point, the end point, and the number of columns between each point are drawn on the input image as input information and reflected as the matrix position at an arbitrary timing. The detecting means detects a specific part of the object;
The determination means estimates the lower end position of the object from the position of the specific part using camera parameters, and based on the distance between the estimated lower end position and the matrix position, whether the object is arranged in a matrix The matrix detection system according to any one of claims 1 to 9. The determination means assumes the height of the specific part in the world coordinate system in a plurality of variations, uses the camera parameters to estimate a plurality of lower end positions from the assumed plurality of heights, and estimates the plurality of estimated lower ends The matrix detection system according to claim 10, wherein it is determined whether or not the objects are arranged in a matrix based on a shortest distance between the position and the matrix position. Accepts matrix position input,
Detect the object from the input image,
Determining whether the objects are arranged in a matrix based on the distance between the objects and the matrix position;
Matrix detection method. Matrix information input processing for receiving input of matrix positions;
A detection process for detecting an object from an input image;
A determination process for determining whether or not the objects are arranged in a matrix based on a distance between the object and the matrix position;
A computer-readable recording medium that stores a program for causing a computer to execute the program.

Images