JP2018156354A - Procession length estimation apparatus, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately estimate the length of a procession even in a camera that is almost in landscape orientation, even in a situation where the procession and passing persons are mixed, and even when the background has a complex texture.SOLUTION: A staying degree evaluation unit 30 evaluates the staying degree for each small block, and a congestion degree evaluation unit 40 evaluates the congestion degree for each small block. A small-block dedicated procession likelihood calculation unit 50 calculates an evaluation value of the procession likelihood. A procession state determination unit 60 determines whether the state is in a procession state by integrating evaluation values of the procession likelihood for each of a plurality of determination regions. A procession length estimation unit 70 determines how much the determination regions determined as being in the procession state are arranged in succession.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a matrix length estimation apparatus, method, and program, and more particularly, to a matrix length estimation apparatus, method, and program for estimating a queue length of a person.

  Techniques for measuring and estimating the state of a matrix at a location using a camera can be roughly classified into two from the viewpoint of basic image processing techniques.

  The first is a technique for measuring the number of people and waiting time by detecting or tracking individuals in an image. In Patent Document 1, a foreground image (background difference) and a velocity image (optical flow) are calculated using an image series input from a camera, and an individual is detected based on these images, and the matrix length and the person in the matrix are detected. The moving speed is estimated.

  The second method is a method for calculating the image feature in a predetermined region in the image without detecting the individual and estimating the degree of congestion based on the calculated image feature. In Non-Patent Document 1, focusing on the fact that the texture of an image becomes more complicated as the density of people increases (the image of a hustle becomes more complex in light and shade compared to the background such as walls and floors). As an index indicating complexity, the fractal dimension of the image texture is calculated, and the congestion degree is estimated based on the calculated value. This method or the method of estimating the degree of congestion based on the texture characteristics of such an image is robust against the problem of occlusion that occurs at the time of congestion, and has an advantage that it can be applied to a camera image close to the horizontal direction.

Japanese Unexamined Patent Publication No. 2007-317052

A.N.Marana, L.F.Costa, R.A.Lotufo, and S.A.Velastin. “On the efficacy of texture analysis for crowd monitoring”, SIBGRAPI '98, pp.354-361, Oct. 1998.

  However, the method of detecting an individual person as in the first technique described above cannot obtain a correct result in a situation in which so-called occlusion occurs in which the shooting angle of the camera is close to the sideways and people are superimposed on the image. There was a problem.

  The second technique described above has a problem that it cannot be applied to a place where there are many passers other than the queue because it cannot distinguish between congestion due to queue waiting and congestion due to passers. Also, depending on the state of the background, whether it is congested or not, depending on the state of the background, such as when there is a fine structure or pattern on the background (image without people) and there is a complicated texture, There was also a problem that it could not be judged correctly.

  The present invention has been made to solve the above-described problems, and can be applied to a camera that is close to the horizontal direction, has a situation where a matrix and passers are mixed, and has a complicated background texture. Even in such a case, it is an object to provide a matrix length estimation device, method, and program capable of accurately estimating the matrix length.

  In order to achieve the above object, a matrix length estimation apparatus according to the present invention is a matrix length estimation apparatus that estimates a queue length of a person using video captured by a fixed camera, and includes frames in chronological order. An input unit that accepts input of an image, and a staying degree evaluation unit that evaluates a person's staying degree by analyzing temporal changes of pixel values for each small block in the image based on frame images input in time series order And a degree-of-congestion evaluation unit that evaluates the degree of congestion of a person by analyzing the spatial distribution of pixel values for each small block in the image based on the frame images input in chronological order, and for each small block A matrix-likeness calculation unit for each small block that calculates an evaluation value of matrixiness based on the staying degree and the degree of congestion evaluated for the small block, and each of a plurality of determination areas set in the image A matrix state determination unit that determines whether or not the determination region is in a matrix state by integrating evaluation values of the matrix-likeness calculated for each of the small blocks included in the determination region; and the matrix state And a matrix length estimation unit that estimates the length of the queue of the person by determining how long the determination areas determined to be in line are arranged.

  A matrix length estimation method according to the present invention is a matrix length estimation method in a matrix length estimation apparatus that estimates a queue length of a person using video captured by a fixed camera, and an input unit is arranged in chronological order. Accepting input of frame images, the staying degree evaluation unit evaluates the staying degree of a person by analyzing temporal changes in pixel values for each small block in the image based on the frame images input in chronological order. Based on the frame images input in chronological order, the congestion degree evaluation unit evaluates the person's congestion degree by analyzing the spatial distribution of pixel values for each small block in the image, and it is like a matrix for each small block. For each of the small blocks, the calculation unit calculates a matrix-like evaluation value based on the staying degree and the congestion degree evaluated for the small block, and the matrix state determination unit includes a plurality of matrix state determination units set in the image. Judgment area for For each, by integrating the matrix-likeness evaluation values calculated for each of the small blocks included in the determination area, it is determined whether the determination area is in a matrix state, the matrix length estimation unit is The length of the queue for the person is estimated by determining how many consecutive determination areas determined to be in the queue state are arranged.

  The program which concerns on this invention is a program for functioning as each part of the matrix length estimation apparatus which concerns on the said invention.

  According to the matrix length estimation apparatus, method, and program of the present invention, for each small block, the staying degree and the congestion degree are evaluated, the evaluation value of the matrix likelihood is calculated, and the matrix is determined for each of the plurality of determination areas. By integrating the evaluation values of the likelihood, it is determined whether or not it is in a matrix state, and by determining how many consecutive determination areas determined to be in the matrix state are arranged, By estimating the length of the matrix, it can be applied to cameras that are close to the side, and even in situations where the matrix and passers are mixed, and even when the background has a complex texture, the matrix is accurate. The effect that the length can be estimated is obtained.

It is a block diagram which shows the structure of the matrix length estimation apparatus which concerns on embodiment of this invention. It is a figure for demonstrating the method to calculate a residence degree. It is a figure for demonstrating the method of calculating a congestion degree. It is a figure for demonstrating the method of determining a matrix state and estimating a matrix length. It is a flowchart which shows the matrix length estimation process routine in the matrix length estimation apparatus which concerns on embodiment of this invention. It is a flowchart which shows the flow of the process which evaluates the residence degree in the matrix length estimation apparatus which concerns on embodiment of this invention. It is a flowchart which shows the flow of the process which evaluates the congestion degree in the matrix length estimation apparatus which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Configuration of Matrix Length Estimation Device According to Embodiment of the Present Invention>
Next, the configuration of the matrix length estimation apparatus according to the embodiment of the present invention will be described. As shown in FIG. 1, a matrix length estimation apparatus 100 according to an embodiment of the present invention includes a CPU, a RAM, a ROM for storing a program and various data for executing a matrix length estimation processing routine described later, Can be configured with a computer including Functionally, the matrix length estimation apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 90, as shown in FIG.

  The input unit 10 acquires frame images one by one from a video file captured by a fixed camera or an image sequence such as a still image sequence. Here, input of frame images is received in time series order.

  In this embodiment, it is assumed that image data (video data) captured by a fixed camera is input to a computer and image processing is performed on the computer, but the detailed form is not limited. Also, for either real-time processing that performs processing while acquiring images (video) from a fixed camera one by one, or offline processing that performs processing while reading images (video) once recorded on a recording medium such as a hard disk. Is applicable.

  The computing unit 20 includes a staying degree evaluation unit 30, a congestion degree evaluation unit 40, a small block-wise matrix likelihood calculation unit 50, a matrix state determination unit 60, and a matrix length estimation unit 70.

  The staying degree evaluation unit 30 temporarily stays for each small block (pixel or small region) in the image by analyzing temporal changes in the pixel values of the sequentially input frame image series. Evaluate the possibility of a person (referred to here as the degree of stay). An example of a specific method for evaluating the staying degree will be described below with reference to FIG.

  The staying degree evaluation unit 30 includes a differential image generation unit 31, a differential image short time average unit 32, a differential image long time average unit 33, a small block correlation value calculation unit 34, and a staying degree calculation unit 35. ing.

  The differential image generation unit 31 calculates a spatial gradient (primary differentiation (or secondary differentiation)) of pixel values for each of the frame images using a Sobel filter or the like to generate a differential image.

  The differential image short-time average unit 32 and the differential image long-time average unit 33 average the generated differential images for a certain period of time (or a certain number of frames) from the present to the past. At this time, with regard to how far back to the past (time width when taking an average), two types of short-time and long-time ones are considered, and averaging processing is performed respectively. The time width (or the number of sheets) at the time of averaging may be set empirically, but as an example, the average for a short time is averaged at intervals of 0.5 seconds for a total of 2 seconds (about 4 sheets) For long-term average, average every 3 seconds (about 180 sheets) at 1-second intervals. As a qualitative measure, as shown in the middle figure of FIG. 2, the background edge and the staying person's edge are mainly observed by taking a short time average (the passing person's edge becomes weaker by averaging). So that the edge of the person is not observed much (including the person who is lining up in the matrix by taking the average for a long time) If the process proceeds, the average image edge will be blurred, so even if it is a matrix, it should be long enough to be expected to move. While determining by trial and error).

The correlation value calculation unit 34 for each small block, for the short-time average image of the differential image and the long-time average image of the differential image obtained by the above-described processing, individual small blocks (for example, the entire image) at the same position on the image. Correlation values are calculated for each tile divided into rectangular small blocks of 20 pixels × 20 pixels. There are various correlation value calculation algorithms such as normalized correlation and phase-only correlation. Here, the correlation calculation algorithm is not limited, and high correlation is obtained when the luminance patterns of small block images are similar. Any algorithm that outputs a value may be used.
Note that the phase-only correlation method, which extracts the phase components of the Fourier transform of two images and calculates the correlation, does not depend much on the absolute value of the pixel value, and does not depend on the absolute value of the pixel value. If the point is a sharply changing point) at the same position, a high correlation value is output, so that the short-time average image and the long-time average image described above are subject to some noise and brightness fluctuations. Even in the case where it is, a relatively stable result can be obtained.

  The staying degree calculation unit 35 calculates the staying degree for each small block based on the calculated correlation value. When the correlation value is calculated as described above, the correlation value is expected to be low in a small block in which a person staying for a certain period of time exists. That is, the smaller the correlation value, the higher the staying degree. Therefore, if the correlation value is output in the range of −1 to +1, the staying degree may be calculated by the following equation.

Residence degree = 1-(correlation value + 1) / 2

  The congestion degree evaluation unit 40 analyzes the spatial change of the pixel value, that is, the texture on the image, for each input frame image, so that each small block (pixel or small region) in the image is analyzed. Evaluate the degree of congestion (referred to here as the degree of congestion). This is because when people are crowded and people's heads and upper body are crowded on the image, the texture of the image is characteristic, so even if the number of individuals is not counted, it is roughly congested from the texture state. This is based on the knowledge that the state can be estimated. An example of a specific method for evaluating the degree of congestion will be described below with reference to FIG.

  The congestion level evaluation unit 40 includes a short-time average unit 41, a secondary differential calculation unit 42, a small block statistical value calculation unit 43, and a congestion level calculation unit 44.

  The short-time averaging unit 41 obtains a short-time average image generated by averaging the frame images with respect to a certain short time (or a certain number of frames) from the present to the past.

  The secondary differential calculation unit 42 calculates the secondary differential (which can be calculated using a Laplacian filter or the like) of the short-time average image.

  The statistical value calculation unit 43 for each small block calculates a statistical value that is an average value or a variance value of the calculated secondary differential values for each small block. For example, the average value of the secondary differential values is a simple indicator of the degree of congestion.

  The congestion degree calculation unit 44 calculates a texture evaluation value (for example, an average value of secondary differential values) that is an index of the congestion degree for each small block that is the same as the small block set by the staying degree evaluation unit 30. Similar results can be expected by using a primary differential value such as a Sobel filter instead of the secondary differential value. However, since the size of the secondary differential value corresponds well to the texture specific to garbage as garbage, You can expect better results with the second derivative. Further, not only the average value of the differential value but also its variance is considered useful as an index indicating the degree of congestion. In addition, in the present invention, there are various methods for estimating the degree of congestion from the texture feature representing the texture state in addition to the above example, including Non-Patent Document 1. Is not limited. Also, rather than simply calculating the derivative of the frame image, the method of calculating the derivative value for the image obtained by taking the time average of the image (with a relatively short time of about several seconds) can provide a more accurate matrix congestion degree. Expected to be evaluated. This is because people who are lined up have the property that they basically move while standing still, so they are often almost stationary when viewed in a short time, and by taking an average for a short time, This is because the contour is blurred and the absolute value of the differential value (or the secondary differential value) becomes small, but a high differential value can be expected from a person in the matrix. In addition, the calculation of the degree of congestion from the texture features is combined with a general foreground detection technology (motion region detection technology), and texture features are not calculated for pixels and small blocks in which no motion is detected (secondary differentiation described above). If the value is an example, the second derivative value is not calculated), and evaluation is not performed. For example, by adding ingenuity that lowers the degree of congestion in areas where there is no movement, a matrix with a complex texture is mistakenly displayed. The possibility of determining the state can be suppressed.

  Matrix-likeness calculation unit 50 for each small block integrates the staying degree and the congestion degree calculated for each small block as described above, and calculates an evaluation value for the matrixiness. When the staying degree is high and the degree of congestion is high, the small block is considered to be a matrix, and for example, an evaluation value of matrixiness can be obtained by calculating the sum or product of the staying degree and the degree of congestion.

  The matrix state determination unit 60 integrates, for each of the plurality of determination areas set in the image, the evaluation value of the matrix likelihood calculated for each of the small blocks included in the determination area. It is determined whether the work area is in a matrix state.

  Specifically, in order to estimate the matrix length, a plurality of determination areas are set in the image as shown in FIG. 4, and it is determined whether each determination area is in a matrix state. Estimate how far is extended (matrix length). This determination area is set at a position where a matrix is supposed to be formed on the image by visual observation after installing the fixed camera (typically, as shown in the example of FIG. 4, the starting point of the assumed matrix Set a plurality of judgment areas while shifting little by little in the direction in which the matrix extends. As described above, whether each determination area is in a matrix state is determined based on the matrix-likeness evaluation value calculated for each small block as described above. To do. For convenience, the average value of the matrix-likeness evaluation values of the small blocks included in each determination area is obtained, and if the average value is equal to or greater than a threshold set in advance, it is determined as a matrix state. If it is less than the threshold, it may be determined that the matrix state is not established.

  It should be noted that the matrix-like evaluation value for each small block described above may be accompanied by constant fluctuation (instability) from moment to moment depending on how the person in the place is reflected and the movement. On the other hand, when a matrix is formed, it is considered that the matrix exists with a time width of several tens of seconds to several minutes. For example, the time of the matrix evaluation value calculated for each small block By taking the average (moving average in the time direction), the final determination result can be stabilized.

  The matrix length estimation unit 70 estimates the length of a person's queue by determining how many consecutive determination areas determined to be in a matrix state are arranged.

  In addition, when the relationship between the queue length of the person and the waiting time is empirically known (for example, the queue up to the determination area 2 is 15 minutes), the estimation result of the queue length of the person is used. It is also possible to estimate the waiting time.

<Operation of Matrix Length Estimation Device According to Embodiment of the Present Invention>
Next, the operation of the matrix length estimation apparatus 100 according to the embodiment of the present invention will be described. The matrix length estimation apparatus 100 repeatedly executes the matrix length estimation processing routine shown in FIG. 5 every time the input unit 10 receives frame images in a predetermined time range in chronological order.

  First, in step S100, based on frame images input in chronological order, a person's staying degree is evaluated by analyzing temporal changes of pixel values for each small block in the image.

  In step S102, the degree of congestion of the person is evaluated by analyzing the spatial distribution of pixel values for each small block in the image based on the frame images input in chronological order.

  In step S104, for each small block, a matrix-like evaluation value is calculated based on the staying degree and the congestion degree evaluated in steps S100 and S102 for the small block.

  In step S106, for each of a plurality of determination areas set in the image, the matrix-likeness evaluation values calculated in step S104 are integrated for each of the small blocks included in the determination area. It is determined whether the determination area is in a matrix state.

  In step S108, the length of a person's queue is estimated by determining how long the determination areas determined to be in a matrix state are arranged, and output by the output unit 90. The estimation processing routine is terminated.

  Step S100 is realized by the processing routine shown in FIG.

  In step S110, for each frame image, a first derivative or a second derivative of the frame image is calculated to generate a differential image.

  In step S112, the differential image is averaged over a short time range to generate a short time average image of the differential image.

  In step S114, the differential image is averaged over a long time range to obtain a long time average image of the differential image.

  In step S116, the correlation value in the small block of the long time average image and the short time average image is calculated for each small block.

  In step S118, the staying degree is calculated for each small block based on the calculated correlation value, and the processing routine is terminated.

  Step S102 is realized by the processing routine shown in FIG.

  In step S120, the frame images are averaged over a predetermined short time range to generate a short time average image of the frame images.

  In step S122, the second-order derivative in the spatial direction of the image is calculated for the short-time average image.

  In step S124, the calculated statistical value of the second derivative is calculated for each small block.

  In step S126, the degree of congestion is calculated for each small block based on the calculated statistical value of the second derivative, and the processing routine is terminated.

  As described above, according to the matrix length estimation apparatus according to the embodiment of the present invention, for each small block, the staying degree and the congestion degree are evaluated, the evaluation value of the matrix likelihood is calculated, and a plurality of determination uses For each of the regions, by integrating the evaluation values of matrix-likeness, it is determined whether or not it is in the matrix state, and how long the determination regions determined to be in the matrix state are arranged side by side Therefore, by estimating the queue length of people, it can be applied to cameras that are close to landscape, and there are situations where queues and passers are mixed, and the background has a complex texture. However, the matrix length can be estimated with high accuracy.

  Further, in the embodiment of the present invention, the congestion degree is calculated from the texture feature without performing any individual detection, and the feature of the movement that the person in the matrix basically moves occasionally while staying is retained. By actively evaluating as a degree, and finally determining them comprehensively, it can also be applied to cameras that are close to the side where occlusion occurs, and there are situations where queues and passers are mixed, Even when the background has a complex texture, the matrix length can be accurately estimated.

  In addition, for safety management and optimizing operations in places where queues are likely to occur, such as various windows and event venues, the queue status of those locations is measured from camera images, especially the length of queues for people. Can be estimated.

  In the embodiment of the present invention, the degree of congestion is calculated from the texture features without performing any individual detection, and the feature of the movement that people in the matrix basically move from time to time is defined as the staying degree. By positively evaluating and finally judging them comprehensively, it can also be applied to cameras that are close to the side where occlusion occurs, and there are situations where there are mixed queues and passers-by, as well as background. Even in the case of having a complex texture, the matrix length can be estimated with high accuracy.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

  For example, as a method for evaluating the staying degree for each small block, a method different from the method described above may be used. Further, as a method for evaluating the degree of congestion for each small block, a method different from the method described above may be used.

DESCRIPTION OF SYMBOLS 10 Input part 20 Operation part 30 Residence degree evaluation part 31 Differential image generation part 32 Differential image short time average part 33 Differential image long time average part 34 Small block correlation value calculation part 35 Residence degree calculation part 40 Congestion degree evaluation part 41 Short Time average unit 42 Secondary differential calculation unit 43 Small block statistical value calculation unit 44 Congestion degree calculation unit 50 Small block matrix likelihood calculation unit 60 Matrix state determination unit 70 Matrix length estimation unit 90 Output unit 100 Matrix length estimation device

Claims (7)

A queue length estimation device that estimates the queue length of a person using video captured by a fixed camera,
An input unit that accepts input of frame images in chronological order;
Based on the frame images input in time series order, for each small block in the image, a staying degree evaluation unit that evaluates the staying degree of the person by analyzing the temporal change of the pixel value,
Based on the frame images input in chronological order, a congestion degree evaluation unit that evaluates the degree of congestion of a person by analyzing the spatial distribution of pixel values for each small block in the image,
For each small block, based on the staying degree and the congestion degree evaluated for the small block, a matrix-likeness calculating unit for each small block that calculates an evaluation value of matrixiness,
For each of the plurality of determination areas set in the image, the evaluation area is calculated into the matrix state by integrating the matrix-likeness evaluation values calculated for each of the small blocks included in the determination area. A matrix state determination unit for determining whether there is,
A matrix length estimation unit that estimates the length of the queue of the person by determining how continuously the determination areas determined to be in the matrix state are arranged;
A matrix length estimation apparatus including: The dwell degree evaluation unit
For each of the frame images input in chronological order, a differential image generation unit that calculates a first derivative or a second derivative of the frame image and generates a differential image;
A differential image long-time average part for obtaining a long-time average image of the differential image, which is generated by averaging the differential image over a long-time range;
The differential image is generated by averaging the differential image over a short time range, and a differential image short-time average unit for obtaining a short-time average image of the differential image;
A correlation calculation unit for each small block that calculates a correlation value in the small block of the short-time average image of the differential image and the short-time average image of the differential image for each small block;
The matrix length estimation apparatus according to claim 1, further comprising: a staying degree calculating unit that calculates the staying degree based on the calculated correlation value for each of the small blocks. The dwell degree evaluation unit
A short-time averaging unit for obtaining a short-time average image of the frame image, which is generated by averaging the frame images input in chronological order over a predetermined time range;
For the short-time average image of the frame image, a second derivative calculation unit that calculates a second derivative in the spatial direction of the image;
For each small block, a statistical value calculation unit for each small block that calculates a statistical value of the calculated second derivative,
The matrix length estimation apparatus according to claim 1, further comprising: a congestion degree calculation unit that calculates the congestion degree based on the calculated statistical value of the second derivative for each small block. A matrix length estimation method in a matrix length estimation device that estimates the queue length of a person using video captured by a fixed camera,
The input unit accepts input of frame images in chronological order,
Based on the frame images input in chronological order, the staying degree evaluation unit evaluates the staying degree of the person by analyzing the temporal change of the pixel value for each small block in the image,
Based on the frame images input in chronological order, the congestion degree evaluation unit evaluates the person's congestion degree by analyzing the spatial distribution of pixel values for each small block in the image,
A matrix-likeness calculation unit for each small block calculates an evaluation value of matrixiness based on the staying degree and the congestion degree evaluated for the small block for each of the small blocks,
The matrix state determination unit integrates the matrix-likeness evaluation values calculated for each of the small blocks included in the determination area for each of a plurality of determination areas set in the image, thereby determining the determination. Determine whether the region is in a matrix state,
A matrix length estimation method, wherein a matrix length estimation unit estimates the length of the queue of the person by determining how many consecutive determination areas determined to be in the matrix state are arranged. By evaluating the staying degree evaluation unit,
For each of the frame images input in chronological order, the differential image generation unit generates a differential image by calculating a primary differential or a secondary differential of the frame image,
A differential image long-time average part is obtained by averaging the differential image over a long-time range, and obtaining a long-time average image of the differential image,
The differential image short-time average part is obtained by averaging the differential image over a short-time range to obtain a short-time average image of the differential image,
The small block correlation calculation unit calculates, for each small block, a correlation value in the small block of the long-time average image of the differential image and the short-time average image of the differential image,
The matrix length estimation method according to claim 4, wherein the staying degree calculating unit includes calculating the staying degree based on the calculated correlation value for each of the small blocks. By evaluating the staying degree evaluation unit,
The short-time average unit obtains a short-time average image of the frame image generated by averaging the frame images input in chronological order over a predetermined time range,
The second derivative calculation unit calculates a second derivative in the spatial direction of the image for the short-time average image of the frame image,
The small block statistical value calculation unit calculates a statistical value of the calculated second derivative for each small block,
6. The matrix length estimation apparatus according to claim 4, wherein the congestion degree calculation unit includes calculating the degree of congestion on the basis of the calculated statistical value of the second derivative for each small block.
Matrix length estimation method.   The program for functioning a computer as each part of the matrix length estimation apparatus of any one of Claims 1-3.