JP2016224651A - Image processing apparatus for queue tail recognition - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recognize the tail of a queue formed in front of, for example, an entrance gate, a ticket dispenser, etc., with relatively high precision and then to estimate a waiting time of the queue by utilizing that.SOLUTION: An image processing apparatus section non-stationary body pixels extracted from a series of frame images constituting a moving image into pixel clusters formed by putting together mutually adjoining pixels and also stores a series of pixel clusters adjoining each other in a predetermined queue extending direction among the sectioned pixel clusters as pixel clusters belong to one group, searches all pixel clusters belonging to one pixel cluster group for the most distal point in the predetermined queue extending direction and stores the most distal point as a queue tail candidate, and determines the most distal point stored as the queue tail candidate as a real most distal point upon the basis of a change direction of the stored most distant point and stability after the change when the stored most distant point changes between mutually successive frame images exceeding a predetermined allowable value.SELECTED DRAWING: Figure 4

Description

  BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for recognizing the end of a matrix, and relates to an image processing apparatus for recognizing an end of a matrix suitable for, for example, estimating a waiting time in a queue that appears before an entrance gate, a ticketing machine, or the like.

  Various conventional examples are known as image processing apparatuses used for waiting time estimation in a queue that appears before an entrance gate, a ticketing machine, or the like.

  As one conventional example, an image is extracted by paying attention to individual persons forming a matrix, and the total extension of matrix lines is calculated by connecting adjacent images with a straight line, and a matrix separately estimated A device that calculates the waiting time of a matrix by dividing this by the moving speed (hereinafter referred to as “first conventional example”) is known (see Patent Document 1).

  As another conventional example, the residual image obtained by the difference between the background difference image and the inter-frame difference image is regarded as a human aggregate, and the largest aggregate among these aggregates is used as a matrix. On the other hand, by dividing the area of the matrix thus recognized by a predetermined unit area, the number of people in the matrix is obtained, and this is divided by the predetermined number of former matrix visitors per unit time. A device for obtaining time (hereinafter referred to as “second conventional example”) is known (see Patent Document 2).

  Further, as another conventional example, tracking of edge information of only a moving object is performed based on an image obtained by performing a labeling process on an image obtained by superimposing an edge background difference image and an inter-frame difference image. And generating a contour image of the moving body, and detecting the convex shape of the contour image to extract the head of the person in the image and counting the number of people based on the head (Referred to as “third conventional example”) (see Patent Document 3).

JP2007-317052A JP 2002-329196 A JP-A-2015-070359

  In the first conventional example, although there is an advantage that can be applied not only to a queue that extends in a substantially straight line but also to a queue that bends or meanders, complicated processing is required for individual extraction of person images and their connection. In addition, it takes time to process, and the computer load is large. In addition, it is difficult to individually extract people in an image in which a large number of people overlap, and the accuracy of waiting time calculation cannot be expected for complicated processing.

  In the second conventional example, it is possible to avoid complicated processing in terms of focusing on a set of people rather than focusing on individual persons constituting the matrix. However, the difference between the background difference image and the inter-frame difference image can be avoided. The algorithm that considers the residual image obtained by the difference as a collection of people and immediately recognizes the largest collection among the collections as a matrix is easily affected by passersby around the matrix.

  In the third conventional example, the algorithm of extracting the human head in the image by detecting the convex shape of the contour image can be applied to a relatively dispersed crowd such as a station square. However, when it is applied to an image in which a large number of people are likely to overlap, such as an image in which the queue is viewed from behind, it seems that it is quite difficult to extract all the heads of individual people.

  The present invention has been made in view of the above-mentioned technical background, and its purpose is to recognize the end of a queue appearing in front of an entrance gate, a ticket machine, etc. with relatively high accuracy, for example. By using this, for example, an object of the present invention is to provide an image processing apparatus for recognizing the end of a matrix that can determine the waiting time of the matrix.

  Other objects and operational effects of the present invention will be easily understood by those skilled in the art by referring to the following description of the specification.

In order to achieve the above object, an image processing apparatus for matrix tail recognition according to the present invention is configured as follows.
That is, the image processing apparatus for recognizing the end of a queue according to the present invention recognizes the end of the queue from a moving image obtained by taking a bird's-eye view of an area where a queue composed of one or more lanes is expected to appear. And a non-stationary object extracting unit, a pixel block grouping unit, a farthest point searching unit, and a farthest point determining unit.

  The non-stationary object extracting means extracts pixels constituting the non-stationary object from each of a series of frame images constituting the moving image. The pixel block grouping unit divides the extracted non-stationary body pixels into pixel blocks formed by collecting adjacent pixels into a single block, and a predetermined matrix extension of the divided pixel blocks. A series of pixel blocks that are close to each other in the direction are stored as pixel blocks belonging to one group. The farthest point searching means searches for the farthest point in a predetermined matrix extending direction from all the pixel blocks belonging to one pixel block group and stores them as matrix end candidates. The farthest point determination means, when the farthest point stored as the matrix end candidate changes beyond a predetermined allowable value between consecutive frame images, based on the change direction and the stability after the change Thus, the farthest point stored as the matrix end candidate is determined as the farthest point to be recognized as the true end of the matrix. In addition, as a method for certifying “stability after change”, not only a method for determining whether the value after change is maintained substantially the same over a constant number of continuous frames, but also a constant per constant number of continuous frames. Various methods can be employed in consideration of the influence of noise, such as determining whether or not the same is maintained at the above ratio.

  According to such a configuration, among one or two or more pixel blocks constituting a non-stationary body pixel, a series of pixel blocks adjacent to each other in a predetermined matrix extending direction is stored as a pixel block belonging to one group. Therefore, without being affected by passersby around the matrix, it is possible to reliably search only from the beginning to the end for the pixel block that is truly likely to form the matrix, and the farthest point. The search of all the pixel blocks included in the group instead of the pixel block that was last included in the group is also performed, and the possibility of the end of the matrix is looked at again within the group. The highest pixel can be the farthest point, and in addition, the farthest point obtained in this way is not directly recognized as the true farthest point, but the row is extended and the row is shortened. And according to different standards By confirming, while maintaining the recognized response can avoiding false matrix tail by passersby, etc. near matrices tail. Moreover, since the person included in the matrix is not individually recognized, the entire image processing is a combination of relatively simple elemental technologies, so the processing is simple and the processing time is relatively short. It can be easily implemented.

  In a preferred embodiment of the present invention, when the farthest point determination means is such that when the direction of change of the farthest point is away from the top of the matrix, the farthest point after the change is approximately the number of frames. Only when it is kept constant, it may include a process of determining the farthest point stored as the matrix end candidate as the farthest point to be recognized as the true end of the matrix.

  According to such a configuration, when the change direction of the farthest point is a direction away from the head of the matrix, it is only necessary to determine how many frames the farthest point after the change continues, and the farthest point after the change. It is possible to reliably determine whether or not the farthest point is stable, and misidentification of the farthest point based on a false farthest point by a passerby or the like can be avoided by a simple process.

  In a preferred embodiment of the present invention, when the farthest point determination means is a direction in which the direction of change of the farthest point approaches the head of the matrix, the matrix immediately (without waiting for a predetermined number of frames). It may include processing for determining the farthest point stored as the tail candidate as the farthest point to be recognized as the true matrix tail.

  According to such a configuration, when the end of the matrix approaches the beginning due to the advance of the matrix or the like, this is determined to be a normal end movement, and the farthest point after the change is immediately set to the true maximum. Since it is immediately recognized as a far point, it can be recognized immediately when the end of the matrix has advanced normally.

  According to one aspect of a preferred embodiment of the present invention, the overhead view shooting is performed from the rear of an assumed queue, and the predetermined queue extending direction is a downward direction of each frame image. It may be.

  According to such a configuration, since the end of the queue on the frame image moves in the vertical direction of the frame image, the assumed matrix extending direction coincides with the vertical direction of the frame image. Whether each pixel is close to or far from the end of the matrix can be easily determined using the vertical coordinate value of each pixel as it is.

  According to one aspect of a preferred embodiment of the present invention, the non-stationary body extracting unit converts the frame image into a line image obtained by extracting the contours of various images included therein, and the line image. Including a background deleting unit that deletes a line image portion corresponding to a background using background difference processing, and a sharpening unit that sharpens lines constituting the line image after the background difference processing. It may be.

  According to such a configuration, the contour extraction process is performed prior to the background deletion process, and the line image is further sharpened after the contour extraction. The outline of the non-stationary object included in the original frame image is outlined and the outline is sharpened, and it is possible to perform various subsequent arithmetic processes only on the pixels included in the outline image. By reducing the number of pixels, it is possible to perform high-speed operation by reducing the processing load.

  At this time, the contour extraction means includes a gray scale conversion means for converting the frame image into a monochrome image, a spatial primary differentiation means for performing a spatial primary differentiation process on the monochrome image, and an image after the spatial primary differentiation process. If a binarization unit that binarizes and generates a line image is included, it is possible to easily obtain a line image in which the luminance difference between all the images included in the original frame image is emphasized.

  In addition, the background deletion unit compares the value of a pixel included in a line constituting the line image with a value of a predetermined number of frames before the pixel, and deletes a pixel having a matching value as a background pixel. In the image included in the above line image, a line image of an object that has appeared within a predetermined number of frames in the past (that is, not only a true background image such as a building or a table, but also a fixed image) It is possible to reliably extract only the line images that have been removed even for packages that have been left for more than an hour.

  Further, the sharpening means removes fine stains caused by noise or the like included in the line image after the background deletion, and a broken wavy line portion of the line included in the line image after the background processing If there is an expansion means that connects and thickens, the blurring and discontinuity of the line included in the line image is repaired, and the noise stains existing in the space surrounded by the outline are deleted, By enlarging the pixel area to which the same label is attached in the labeling process, the entire non-stationary line image is divided into relatively small and large pixel clusters, which contributes to the efficiency of the grouping process described later.

  According to one aspect of a preferred embodiment of the present invention, the pixel block grouping unit attaches the same label to pixels adjacent to each other among the pixels constituting the extracted non-stationary body. The labeling means for dividing the pixels constituting the body into pixel blocks and the pixel located at the farthest point in the predetermined matrix extending direction among the pixels included in one pixel block corresponding to the head of the matrix In the predetermined search area set toward the far side in the extending direction, it is searched whether there is a pixel with another label, and when it exists, a pixel block including the pixel Is repeated for all head equivalent pixel blocks until there is no pixel with a different label for the pixel block newly stored as the same group. And a pixel cluster search unit may be.

  According to such a configuration, a search region set in the expected extending direction of the queue is assumed for a plurality of pixel clusters (contour image) each having a unique label. By performing the proximity pixel block search process, pixel blocks that make up surrounding passersby are excluded, and only a series of pixel blocks from the beginning to the end that make up the queue of each lane are extracted in order reliably. Can be related to one group.

  According to a preferred embodiment of the present invention, the farthest point searching means is a pixel located at a farthest point in a predetermined matrix extending direction among pixels included in one pixel block belonging to the same group. It is also possible to include a process of specifying the farthest point in the same group by repeating the process of searching for all pixel blocks belonging to the group.

  According to such a configuration, not only the pixel block last related to one group but also all the pixel blocks in the group are searched for a pixel located at the farthest point in the predetermined matrix extending direction. Since the process is executed, the farthest point having a high possibility of the end of the matrix can be found more reliably.

  According to an aspect of a preferred embodiment of the present invention, the farthest point determination means has a difference between the value of the current frame at the farthest point in the same group and the value of the previous frame within a predetermined allowable value. Sometimes the predetermined frame counter is incremented, while the counter control means for clearing the frame counter when the allowable value is exceeded, and the difference between the value of the current frame at the farthest point in the same group and the value of the previous frame is When the change exceeds a predetermined allowable value, when the change direction is near, the value of the true farthest point is updated with the value of the current frame, while when the change direction is far, the frame counter And the farthest point true value update control means for updating the true farthest point value with the value of the current frame after waiting for the value of a predetermined number of frames to be reached.

  According to such a configuration, when the change direction of the farthest point is a direction approaching the top of the matrix, the farthest point stored as the matrix end candidate is immediately recognized as the true end of the matrix. While it is fixed immediately without delay as a far point, when the change direction of the farthest point is a direction away from the top of the matrix, it is only necessary to determine how many frames the farthest point after the change will continue and Whether or not the farthest point is stable can be determined with certainty, and misidentification of the farthest point based on a false farthest point by a passerby or the like can be avoided by simple processing.

  The present invention viewed from another aspect can be grasped as an image processing method for matrix end recognition. That is, this image processing method for recognizing the end of a queue recognizes the end of the queue from a moving image obtained by taking a bird's-eye shot of an area in which a queue composed of one or more lanes is expected to appear. A non-stationary object extraction step, a pixel block grouping step, a farthest point search step, and a farthest point determination step.

  In the non-stationary object extraction step, pixels constituting the non-stationary object are extracted from each of a series of frame images constituting the moving image. The pixel block grouping step divides the extracted non-stationary body pixels into pixel blocks formed by collecting adjacent pixels into a single block, and extends a predetermined matrix among the divided pixel blocks. A series of pixel blocks that are close to each other in the direction are stored as pixel blocks belonging to one group. In the farthest point searching step, the farthest point in a predetermined matrix extending direction is searched from all the pixel blocks belonging to one pixel block group and stored as a matrix end candidate. The farthest point determination step is based on the change direction and the stability after the change when the farthest point stored as the matrix end candidate changes beyond a predetermined allowable value between consecutive frame images. Thus, the farthest point stored as the matrix end candidate is determined as the farthest point to be recognized as the true end of the matrix.

  According to the method invention, in addition to obtaining the same operational effects as the device invention, for example, when an image processing program necessary for a personal computer is incorporated and executed, the program is always stored in an external storage of another company. The program can be executed without being resident in the personal computer as in the case where it is placed in a device or a so-called cloud center and transferred to the personal computer as necessary.

  From another aspect, the present invention can also be understood as a computer program for causing a computer to function as an image processing apparatus for matrix end recognition. In other words, this computer program recognizes the end of the queue from a moving image obtained by taking a bird's-eye view of an area in which a queue composed of one or more lanes is expected to appear. Non-stationary object extracting means, pixel block grouping means, farthest point searching means, and farthest point determining means are included.

  The non-stationary object extracting means extracts pixels constituting the non-stationary object from each of a series of frame images constituting the moving image. The pixel block grouping unit divides the extracted non-stationary body pixels into pixel blocks formed by collecting adjacent pixels into a single block, and a predetermined matrix extension of the divided pixel blocks. A series of pixel blocks that are close to each other in the direction are stored as pixel blocks belonging to one group. The farthest point searching means searches for the farthest point in a predetermined matrix extending direction from all the pixel blocks belonging to one pixel block group and stores them as matrix end candidates. The farthest point determination means, when the farthest point stored as the matrix end candidate changes beyond a predetermined allowable value between consecutive frame images, based on the change direction and the stability after the change Thus, the farthest point stored as the matrix end candidate is determined as the farthest point to be recognized as the true end of the matrix.

  According to this program invention, in addition to obtaining the same effects as the apparatus invention, it is obvious that the program itself can exercise the right to an implementation act such as manufacturing and sales by an unauthorized third party. Yes.

  From another aspect, the present invention recognizes the end of the queue from a camera that takes a bird's-eye view of an area where a queue composed of one or more lanes is expected to appear, and a moving image obtained from the camera. It can also be grasped as a recognition system at the end of the matrix that includes an image processing device for this purpose.

  In this case, the image processing apparatus includes a non-stationary object extracting unit, a pixel block grouping unit, a farthest point searching unit, and a farthest point determining unit.

  Here, the non-stationary object extracting means extracts pixels constituting the non-stationary object from each of a series of frame images constituting the moving image. The pixel block grouping unit divides the extracted non-stationary body pixels into pixel blocks formed by collecting adjacent pixels into a single block, and a predetermined matrix extension of the divided pixel blocks. A series of pixel blocks that are close to each other in the direction are stored as pixel blocks belonging to one group. The farthest point searching means searches for the farthest point in a predetermined matrix extending direction from all the pixel blocks belonging to one pixel block group and stores them as matrix end candidates. The farthest point determination means, when the farthest point stored as the matrix end candidate changes beyond a predetermined allowable value between consecutive frame images, based on the change direction and the stability after the change Thus, the farthest point stored as the matrix end candidate is determined as the farthest point to be recognized as the true end of the matrix.

  According to the present system invention, in addition to obtaining the same operational effects as the apparatus invention, not only the image processing apparatus but also the overhead view camera itself is the equipment that constitutes the invention, and thus there is no authority. It goes without saying that the right to exercise not only a computer constituting the image processing apparatus but also a camera for overhead view photography can be exercised for an implementation act such as manufacturing and sales by a third party.

  The image processing apparatus for recognizing the end of the matrix according to the present invention further includes a correlation between the end position of the matrix and the waiting time of the matrix, which is statistically obtained in advance as the end position of the matrix obtained from the image processing apparatus. By adding a waiting time estimating means for estimating the current matrix waiting time, it is possible to realize a matrix waiting time estimating device.

  According to such a matrix waiting time estimation apparatus, for example, the statistical waiting time obtained by actually repeating the measurement corresponding to each of the matrix end reference position and each end reference position in a plurality of stages is obtained. By creating a table as a pair and executing table reference processing and appropriate interpolation processing based on the matrix end position obtained from the device of the present invention, the corresponding waiting time is determined based on the automatically generated matrix end position. Can be estimated.

  At this time, for each of the queues of the plurality of lanes, the current waiting time is estimated, and based on those waiting times, one overall waiting time corresponding to the plurality of lanes is determined. Also good.

  According to such a configuration, for example, when there are a plurality of gates such as an airport departure gate and a queue of multiple lanes is generated at each gate, the waiting time for each gate is estimated. By notifying this to the boarding passenger through an appropriate notification means, the boarding passenger can select a relatively free departure gate.

  According to the image processing apparatus for recognizing the end of a matrix according to the present invention, a series of pixel blocks that are close to each other in a predetermined matrix extending direction among one or two or more pixel blocks constituting a non-stationary body pixel. Since it is stored as a pixel block belonging to the group, it is surely searched from the head to the end only for the pixel block that is truly likely to form a matrix without being affected by passersby around the matrix. You can search for the farthest point from all the pixel blocks included in the group instead of the pixel block that was last included in the group. The most likely pixel at the end of the matrix can be the farthest point, and in addition, the farthest point obtained in this way is not directly recognized as the true farthest point, but the column is extended. Case and column shortened In the case, by re-confirmed by different criteria, while maintaining the recognized response can avoiding false matrix tail by passersby, etc. near matrices tail. Moreover, since the person included in the matrix is not individually recognized, the entire image processing is a combination of relatively simple elemental technologies, so the processing is simple and the processing time is relatively short. It can be easily implemented.

FIG. 1 is a configuration diagram of a waiting time estimation system at an airport departure gate according to an embodiment of the present invention. FIG. 2 is an explanatory diagram showing the relationship between the queue at the entrance of the security checkpoint and the shooting direction of the camera. FIG. 3 is an explanatory diagram showing the relationship between the end of the queue and the statistical waiting time. FIG. 4 is a general flowchart showing the entire program executed by the waiting time generation PC. FIG. 5 is a detailed flowchart of matrix end recognition processing. FIG. 6 is a detailed flowchart of the non-stationary body constituent pixel extraction process. FIG. 7 is a detailed flowchart of the contour extraction and sharpening process. FIG. 8 is a detailed flowchart of the background deletion process. FIG. 9 is a detailed flowchart of the grouping process between adjacent pixel blocks. FIG. 10 is a detailed flowchart of the search process for the lowest point in each group. FIG. 11 is a detailed flowchart of the lowest point determination process in each group. FIG. 12 is a detailed flowchart of the waiting time estimation process. FIG. 13 is an explanatory diagram (part 1) schematically showing a process of image processing. FIG. 14 is an explanatory diagram (part 2) schematically showing a process of image processing. FIG. 15 is an explanatory diagram (part 3) schematically illustrating a process of image processing. FIG. 16 is an explanatory diagram (part 4) schematically showing a process of image processing. FIG. 17 is an explanatory diagram illustrating an image example after the contour extraction processing. FIG. 18 is an explanatory diagram illustrating an example of an image after background deletion and sharpening processing. FIG. 19 is an explanatory diagram showing a relationship between an example of an image after grouping processing and a gate passing time.

  Hereinafter, a waiting time estimation apparatus according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings, taking a queue appearing in front of an airport departure gate as an example. Needless to say, the present invention is a queue that appears in front of a ticket office such as a railway station or a public race, or a queue that appears in front of a settlement (registry) such as a supermarket or a convenience store. Of course, it can be widely applied.

<Introduction>
In general, each departure gate of an airport is provided with a security checkpoint having a plurality of lanes for passengers parallel to each other. Each passenger passage is provided with various inspection devices such as an X-ray detector for physical inspection as well as a metal detector for physical inspection.

  In addition to the high accuracy of these inspection devices, in recent years, inspections have been carried out more carefully due to terrorism countermeasures, etc., and the minimum time required for inspection per passenger tends to increase. It is customary that a queue for inspection appears before the entrance of the passenger passage in each lane of the security inspection area. For this reason, the time required for passing through the security inspection area of the departure gate from the general passenger floor area to the passenger floor area (that is, the waiting time of the queue) depends on the length of the inspection queue. It fluctuates greatly.

  The image processing apparatus for recognizing the end of a queue according to the present invention automatically recognizes the end of the queue for each lane of the departure gate in real time, and estimates the waiting time for each lane or each departure gate, that is, the time required for passing. Can be used to

<About airport facilities>
As shown in FIG. 1, the airport to which the present embodiment system is applied is provided with four departure gates including the first gate to the fourth gate, separated from each other by an appropriate distance. As shown in FIG. 3, a security checkpoint having a four-lane passenger passage composed of the first lane (L1) to the fourth lane (L4) is provided. It is customary that a passenger queue appears in front of the passenger passage in each lane. Therefore, four queues extending substantially in parallel with each other along the extending direction of the passenger passages in the lanes (L1) to (L4) appear at each departure gate.

<Camera for overhead view photography and its arrangement>
Each of the four departure gates including the first gate to the fourth gate is provided with a camera for capturing a moving image in order to take a bird's-eye view of the passenger queue. The number of cameras for each gate and the shooting direction is whether the four queues are spaced apart from each other within the field of view of each camera, in other words, the four queues are It is determined so as not to overlap. From such a viewpoint, there are various options for the number of cameras and the shooting direction for each gate. As an example, if the number of cameras is one, it is preferable to take a bird's-eye view of four queues at a sufficient distance from the rear. However, if it is difficult to keep a sufficient distance due to the surrounding building conditions, etc., the number of cameras is set to two or more. For example, when the number of cameras is two, the first camera is adjacent. A method may be employed in which a queue for two rows is photographed from behind, and a queue for two rows adjacent by the second camera is photographed from above.

  FIGS. 1 to 3 show an example in which four queues in front of each departure gate are photographed from above with a single camera at a sufficient distance. That is, the queue of the four rows of the first gate is the first camera 2-1, the queue of the four rows of the second gate is the second camera 2-2, and the queue of the four rows of the third gate is the third. The queues of the four rows of the fourth gate and the fourth gate are photographed from a position separated from the rear by a fourth camera 2-4 from above. In FIG. 2, reference numeral 3 denotes a person who forms a queue. Further, in FIG. 3, the person 3 who constitutes the queue shows only a part of the queue in order to simplify the drawing, and the majority of other queue parts are omitted. Furthermore, in FIG. 3, it should be noted that the distance from the entrance of the departure gate to the mounting position of the camera 2 has been greatly reduced due to space limitations.

  Although various types of cameras 2 can be arbitrarily employed as the camera 2, in this example, a ceiling-mounted dome camera having a photographing rate of 30 frames per second and having a communication function is employed. As shown in Fig. 2, it is attached to the front surface of the entrance of the security inspection site, on the ceiling surface at a sufficiently spaced position.

<About the waiting time generation PC constituting the image processing apparatus of the present invention>
As shown in FIG. 1, the first camera 2-1 for taking a bird's-eye view of the queue of the four rows of the first gate, the second camera 2-2 for taking a bird's-eye view of the queue of the four rows of the second gate, The third camera 2-3 for taking a bird's-eye view of the queue of the four rows of gates and the fourth camera 2-4 for taking a bird's-eye view of the queue of the four rows of the fourth gate are each finally connected via a LAN cable. Are connected to one personal computer (hereinafter referred to as “latency generation PC”) 1 for generating a waiting time. As a personal computer, a commercially available computer having a normal performance with a main unit (including a CPU, a hard disk, a memory, a wireless LAN board, etc.), a display, and an operation unit (including a mouse, a keyboard, etc.) is arbitrarily selected. The hardware configuration and the system configuration such as the OS are well known to those skilled in the art based on various documents, and thus the description thereof is omitted. Although not shown, the waiting time generation PC 1 can be connected to an airport management server via a wireless LAN.

<Overall computer program incorporated in PC for waiting time generation>
FIG. 4 shows a general flowchart schematically showing the entire process (computer program) executed by the waiting time generation PC1. As shown in the figure, this process basically includes a predetermined camera pointer value from the initial value (step 100) to the first camera → second camera → third camera → fourth camera one after another. While updating (step 106), each time the update is performed, processing for acquiring an image from the camera designated by the pointer (step 101), and FIFO processing for storing the latest 120 frames (within the past 4 seconds) (step 102) A series of processes consisting of the process of recognizing the end of the matrix (step 103) and the process of estimating the waiting time (step 104) are repeated until the processes for all the cameras are completed (NO in step 105). Wait (YES in step 105), save the estimated waiting time value in a predetermined file (step 107), and save the file to the airport management server. Process (step 108) is repeated, and the above series of processes (steps 100 to 109) are repeated until the operation is stopped by a predetermined operation (NO in step 109). It waits (step 109 YES), and it is comprised so that a process may be complete | finished.

<Details of recognition processing at the end of the matrix>
The details of the matrix end recognition process (step 103) are shown in FIG. As shown in the figure, this processing basically includes non-stationary pixel extraction processing (step 1030), grouping of adjacent pixel clusters (step 1031), and search processing for the lowest point in each group. (Step 032) and the lowest point determination process (Step 1033) in each group.

  Here, the non-stationary body pixel extraction process (step 1030) is a process of extracting non-stationary body constituent pixels corresponding to a person or the like from a frame image acquired every 1/30 seconds. As will be described in detail with reference to the drawings, it includes a contour extraction process (step 10300), a background deletion process (step 10301), and a sharpening process (step 10302).

  In addition, the grouping process (step 1031) between adjacent pixel clusters is a pixel in which the extracted non-stationary pixels are grouped into adjacent pixels as will be described in detail later with reference to FIG. This is a process of dividing into a block and storing a series of pixel blocks close to each other in a predetermined matrix extending direction among the divided pixel blocks as a pixel block belonging to one group.

  Further, the search processing for the lowest point in each group (step 1032) is performed by extending a predetermined matrix from all the pixel blocks belonging to one pixel block group, as will be described in detail later with reference to FIG. This is a process of searching for the farthest point (in this example, the lowest point in this example) in the direction (in this example, the downward direction on the screen) and storing it as a matrix end candidate.

Further, the determination process of the lowest point in each group (step 1033) is performed between frame images in which the farthest points stored as matrix end candidates are consecutive, as will be described in detail later with reference to FIG. The farthest point stored as the matrix end candidate is the farthest point to be recognized as the true end of the matrix based on the direction of change and the stability after the change. This is a process to confirm.
<Details of extraction processing of non-stationary constituent pixels>
Details of the process of extracting the non-stationary constituent pixels are shown in FIG. As shown in the figure, this process includes a contour extraction process (step 10300) for converting a frame image into a line image obtained by extracting the contours of various images included therein, and a background image in the line image. A background deletion process (step 10301) for deleting the line image portion corresponding to the background difference process and a sharpening process (step 10302) for sharpening the lines constituting the line image after the background difference process. It consists of Therefore, in this non-stationary body constituent pixel extraction process, not only the entire individual pixel block composed of non-stationary body pixels, but only the outline of the individual pixel block is selectively extracted. The processing target pixels in the subsequent search processing and collation processing are limited to only the contour portion, and the number of target pixels is greatly reduced, which contributes to an improvement in processing speed.

<About contour extraction processing>
In this example, the contour extraction process (step 10300) includes, as shown in FIG. 7A, a grayscale process (step 10300-1) for converting a frame image into a monochrome image, and a monochrome image. On the other hand, using a known sobel filter, a spatial primary differentiation process (step 10300-2) for performing spatial primary differentiation, and a binarization process for generating a line image by binarizing the image after the spatial primary differentiation process (Step 10300-3).
As is well known, the gray scale processing (step 10300-1) is processing for equalizing RGB values based on various algorithms. In the present invention, a known method is arbitrarily adopted. Can do. The spatial first differentiation process (step 10300-2) is, in short, a process for generating a pixel value corresponding to a luminance difference between adjacent pixels, and thereby, the contour portion of the pixel block included in the grayscaled image is aspirated. While being able to stand up. The binarization process (step 10300-3) is a line image corresponding to the contour portion of the pixel block by binarizing each pixel value constituting the image after spatial first differentiation with an appropriate luminance threshold value. Can be raised.

  An image example after gray scale processing is schematically shown in FIG. 13A, and an image example after spatial primary differentiation processing and binarization processing is schematically shown in FIG. 13B. In these figures, PCG11, PCG12, and PCG13 are pixel blocks corresponding to persons and the like constituting the queue, PCG2 is a pixel block corresponding to persons and the like that do not form the queue, and PCG31 and PCG32 are background objects. 4 is a contour line of the pixel block, and 5 is a dark spot corresponding to noise. As is apparent from a comparison between the image of FIG. 13A and the image of FIG. 13B, when the spatial first-order differentiation process and the binarization process are performed on the image after the gray scale process, an outline is obtained. A line drawing including the line 4 can be obtained. However, in the line drawing in this state, the thickness of the outline 4 is not uniform, there are partly broken wavy parts, and there are not a few stigma 5 corresponding to many noises.

<About background deletion processing>
Details of the background deletion processing (step 10301) are shown in FIG. In short, the background deletion process is a process in which the pixels included in the line constituting the line image are compared with the value of the pixel before the predetermined number of frames, and the matching pixels are deleted as the background pixels. As shown in the figure, this process shifts the pixel designation (pointer) value from the initial value (for example, the upper left of the screen) one pixel at a time in the horizontal direction by shifting one pixel at a time in the vertical downward direction. (Steps 10301-1 and 10301-6), while searching for each pixel value, only if it is a contour part and matches the pixel value 120 frames before, the pixel is regarded as the background and deleted. This process (steps 10301-2, 10301-3, 10301-4) is repeatedly executed until the process is completed for all pixels (NO in step 10301-5). In this process, the image serving as the reference for the background difference is not fixed to the initial image including the original background object such as a column or table, but is always updated every 120 frame times. Even if a suitcase or other baggage placed on the airport floor is left on for more than 120 frame times (4 seconds), it will be deleted as a background item. For this reason, these packages will not be mistaken for part of the queue.

<About sharpening processing>
Details of the sharpening process (step 10302) are shown in FIG. As shown in the figure, this processing is performed by a contraction process for removing fine stains caused by noise or the like included in the line image after the background deletion, and a line included in the line image after the background process is interrupted. And an expansion process for connecting the wavy line portions to thicken them.
An image example after the background deletion process is schematically shown in FIG. 14A, and an image example after the sharpening process is schematically shown in FIG. 14B. As is apparent from the comparison between the image in FIG. 14A and the image in FIG. 14B, the image after the background deletion process is present by performing the sharpening process on the image after the background deletion process. The contour line 4 becomes a contour line 4A in which the wavy line portion is repaired and thickened, and the noise spot 5 is erased. According to the thickened outline 4A, when the labeling process is executed later, the range to which the same label is attached becomes wider, which contributes to the increase in size and the number of pixel blocks.

<Regarding the grouping process between adjacent pixel clusters>
Details of the grouping process between adjacent pixel blocks are shown in FIG. As shown in the figure, this processing is basically performed by assigning the same label to the pixels adjacent to each other among the extracted non-stationary pixels, so that the pixels constituting the non-stationary body are pixelated. A labeling process (step 10310) for dividing each block and the farthest point in the predetermined matrix extending direction among the pixels included in one pixel block corresponding to the top of the matrix (in this example, the lowest point of the screen) Predetermined search areas set in the far direction of the extending direction (in this example, the lower direction of the screen) with reference to the pixel located at (in this example, X pixels in the horizontal direction and Y in the lower direction, respectively) In the rectangular search area A) of pixels, a search is made as to whether or not another labeled pixel exists, and when it exists, the pixel block including the pixel is stored as a pixel block belonging to the same group. New processing for the same group Configured for the stored pixel block, to the pixel, labeled with a different label is no longer present, all of the top corresponding pixel block is repeated close pixel block search process (Step 10311 to 10315) and contains as.

  As is well known to those skilled in the art, the labeling process (step 10310) focuses on one pixel that forms an image and is adjacent to it (that is, four pixels on the top, bottom, left, and right) (In addition to the top, bottom, left, and right, the upper right, lower right, upper left, and lower left 8 pixels) When there are other pixels that form an image, repeat the process of attaching the same label to the pixels Say that. Here, the pixels forming the image indicate the pixels included in the bold outline 4A described above with reference to FIG.

  As a result, when the above-described labeling process ends, for example, the first label (label 1) is added to the pixel block PCG12 for all the pixels forming the outline 4A of the pixel block PCG11 in the image of FIG. The second label (label 2) is assigned to all the pixels forming the contour line 4A, and the third label (label 3) is assigned to all the pixels forming the outline 4A of the pixel block PCG13. A fourth label (label 4) is attached to all the pixels forming the outline 4A.

  The proximity pixel block search process (steps 10311 to 10315) will be described in more detail with reference to FIG. 15. First, after setting a target label (pointer) to an initial value (for example, label 1) (step 10311), As seen from the pixel (Pz) located at the lowest point among the pixels included in the pixel block (for example, the outline 4A portion of the pixel block PCG11) with the label (label 1), FIG. Pixels with different labels (for example, label 2, label 3,...) In a predetermined search area A (for example, a rectangular search area of X pixels in the horizontal direction and Y pixels in the downward direction) A Whether or not exists is determined (step 10312).

  Here, when there is a pixel with another label (for example, label 2) (YES in step 10312), the pixel block (for example, the outline 4A portion of the pixel block PCG12) including the pixel is included in the same group (for example, , Group G1) which is the first group (step 10313).

  For the pixel block newly incorporated in the group G1, the above processing is performed until no pixel with another label is found in the predetermined search area A when viewed from the lowest point (Pz) of the pixel block. Repeat (steps 10312 and 10313).

In this way, if a pixel with another label is no longer found (NO in step 10312), the target label is updated to an unprocessed value (step 10314), and the above processing (steps 10312 and 10313) is performed for all labels. Is repeated until the process is completed (NO in step 10315).
The search area A used in the above proximity pixel search processing (steps 10311 to 10315) is within the search area A that mainly extends downward on the screen, assuming that the queue extends downward in the screen. One after another, the constituent pixels of the pixel block are searched, and the found pixel blocks PCG12 and PCG13 are incorporated into the same group G1. As a result, the passersby around the group G1 In other words, only a series of pixel blocks PCG11, PCG12, and PCG13 having a high possibility of forming a queue are included.

<About the lowest point search process in each group>
Details of the lowest point search process in each group are shown in FIG. This in-group lowest point search process is, in short, the farthest point (lowest point) in a predetermined matrix extending direction (downward in the screen in this example) among the pixels included in one pixel block belonging to the same group. The process of searching for the pixel located at is repeated for all the pixel clusters belonging to the group to identify the farthest point (the lowest point) in the same groove.

  That is, as shown in the figure, this process sets the group designation (pointer) value to an initial value (for example, group G1 corresponding to the first lane) (step 10320), and then all of the groups belonging to the group. The pixel that is the lowest point is searched from the pixel block (for example, PCG1, PCG12, PCG13) (step 10321), and the process of storing the searched lowest point (step 10322) is performed one after another. (Step 10324), and repeatedly (steps 10321 and 10322), the search of all groups is waited (step 10323), and the process is terminated.

  In this way, instead of searching for the lowest point from the pixel block last incorporated in the group, it is possible to end the matrix by searching for the lowest point from all the pixel blocks belonging to the group. It is possible to reliably search for the lowest lowest point.

<About the final point determination process in each group>
Details of the lowest point determination process in each group are shown in FIG. The determination process of the lowest point in each groove is basically that the difference between the value of the current frame at the farthest point in the same group (in this example, the lowest point) and the value of the previous frame is a predetermined allowable value. The counter control process increments a predetermined frame counter when it is within, while clearing the frame counter when the allowable value is exceeded, and the value of the current frame at the farthest point in the same group (the farthest point in this example) When the difference from the value of the previous frame changes beyond a predetermined allowable value, when the direction of change is near (in this example, the upper part of the screen), the true farthest point (depending on the value of the current frame) In this example, the value of the lowest point) is updated, while when the direction of change is the far direction (in this example, the lower part of the screen), it waits for the value of the frame counter to reach a predetermined number of frames and waits for the current frame. The value of the It is intended to include the update control process of the truer farthest point farthest point true value to update the value of (the lowest point) (nadir true value).

  That is, in the figure, when processing is started (step 1330), first, the group designation (pointer) value is initialized to a value corresponding to the queue of the first lane (step 10330), and then the current frame It is determined whether the difference between the lowest point and the lowest point of the previous frame is within a predetermined allowable value (step 10331).

  Here, if it is determined that it is within the allowable value, that is, it is in a stable state with almost no change (step 10331), a predetermined frame counter for determining the stable time is incremented by +1 (step 10332), and then It is determined whether or not the value of the frame counter has exceeded a specified value (for example, 90 frames corresponding to 3 seconds), that is, whether or not a predetermined stable time has been reached (step 10333).

  If it is determined that the specified value has been exceeded (YES in step 10333), it is determined that the queue has been extended, and the current lowest point is determined as the true lowest point (step 10334). If it is determined that the specified value has not yet been exceeded (NO at step 10333), it is determined whether or not the queue has been extended, and the lowest point determination process (step 10334) is skipped.

  On the other hand, if it is determined that the difference between the lowest point of the current frame and the lowest point of the previous frame is not within the allowable value (step 10331 NO), there is a possibility that the queue has been extended or shrunk. After the frame counter is cleared (step 10337), it is further determined whether or not the lowest point of the current frame has fallen below the lowest point of the previous frame (step 10338).

  If it is determined that the lowest point of the current frame is lower than the lowest point of the previous frame (step 10338), it is determined that there is a possibility that the queue has been extended. While the point determination process (step 10334) is skipped, if it is determined that the lowest point of the current frame is not lower than the lowest point of the previous frame (step 10338 NO), the queue is shortened. Immediately determining that it has been performed, the lowest point determination process (step 10334) is executed. Thereafter, until the processing for all groups (all lanes) is completed, the group designation values are updated one after another (step 10336), and the above-described series of processing is executed and processing for all groups is completed. (Step 10335 YES), and the process ends.

  As a result of the above series of processing (steps 10330 to 10338) being repeated for each frame, when the queue is shortened and the end of the matrix moves forward, the end of the matrix is immediately updated to the value after the movement. On the other hand, when the queue is expanded and the end of the queue moves backward, the end of the queue is updated to the value after the movement after a predetermined confirmation time. Further, for example, as shown in FIG. 16, when a passer passes the end of a normal queue (see FIG. 16A) as indicated by an arrow 6, a pixel block PCG22 appears temporarily at the end of the matrix. In the case where the queue is extended by ΔY (see (b) in the figure), as described above, before the value of the frame counter reaches the specified value (step 10333 NO), the value of the frame counter is Since it is cleared (step 10337), even if there is a temporary rearward change of the end of the queue due to such a crossing of a passerby, the true end of the queue will not be mistaken.

<About waiting time estimation processing>
Next, the details of the waiting time estimation process (step 104) are shown in FIG. As shown in the figure, this processing first sets the groove designation (pointer) to an initial value (a value corresponding to the first lane) (step 1041), and then the lowest point of the group designated by the pointer. Is read (step 1042), the waiting time is obtained with reference to the correlation table with reference to the read lowest point (step 1043), and the obtained waiting time is recorded as the waiting time of the corresponding lane (step 1043).

This correlation table is prepared and prepared in advance by a statistical method. That is, in the predetermined memory, as shown in FIG. 3, when the reference lines Pref 1 to Pref 5 at the end of each matrix and the end of the matrix are at the positions of the reference lines Pref 1 to Pref 5, how long is the waiting time, respectively. The following correlation table showing the relationship with each waiting time obtained by statistically examining whether or not is stored is stored.

Matrix end position Wait time
Pref1 5 minutes or less
Pref2 about 10 minutes
Pref3 about 15 minutes
Pref4 about 20 minutes
Pref5 About 30 minutes or more
Correlation table

  In the above-described waiting time estimation process (step 1043), the end position of the matrix derived from the read bottom point is collated with each of the reference lines Pref1 to Pref5 of the above correlation table, and if necessary, by a linear approximation process or the like. The waiting time is generated by interpolating the gap between the reference lines. The waiting time obtained in this way is recorded as the waiting time of the corresponding lane (step 1044).

  The above series of processing (steps 1042 to 1044) is repeatedly executed after updating the group designation (step 1046), and waits for completion of processing of all groups (step 1045 YES), and waits for each lane. A process for obtaining the waiting time for the entire gate from the time (step 1047) and a process for recording the obtained waiting time as the waiting time for the corresponding gate (step 108) are sequentially executed, and the process ends.

  In addition, as processing (step 1047) for obtaining the waiting time of the entire gate from the waiting time for each lane, for example, when the maximum waiting time is set as the waiting time of the gate among the waiting times of each lane, the waiting time of each lane If the average is taken as the waiting time for the gate, various methods may be selected and employed as appropriate.

<Example of image showing the process of image processing>
An example of an image when the image processing described with reference to the flowcharts of FIGS. 5 to 11 and the schematic diagrams of FIGS. 13 to 16 is applied to a queue before the departure gate in an actual airport is shown in FIG. To FIG. 19 collectively. In the figure, FIG. 17 shows an image example after contour extraction processing, FIG. 18 shows an example image after background deletion and sharpening processing, FIG. 19 shows an example image after grouping processing, and each queue end position is indicated by a one-dot chain line. The corresponding waiting time is given to the drawer end. Further, among the pixel blocks, the pixel blocks grouped as a part of the queue are hatched, and the pixel blocks corresponding to other staff members are displayed with a satin appearance. Note that a white rectangular mark corresponding to the search area A is drawn in the lower left corner of the screen.

<Use of queue waiting time information>
Returning to FIG. 4, if the series of processes described above (steps 101 to 104) is completed for all the cameras of the first gate to the fourth gate (step 105), the waiting time obtained in this way. The estimated value is stored in a predetermined file (step 107), and this file is transferred to the airport-side management server for various uses. As an example of the form of use, for example, by listing the waiting time for each departure gate on a Web site operated by the airport, it can be used for browsing passengers.

<Operational effects of the system according to this embodiment>
According to the airport departure gate waiting time estimation system described above, a configuration is adopted in which waiting time is estimated by a statistical method based on the matrix end recognition result. In addition to the basic advantages of finding the total length of the queue and dividing it by the average progression speed of the queue, the processing is simpler and the load on the computer is lighter than that for calculating the queue waiting time. In order to identify the end of the queue, which is the basis for waiting time estimation, extraction processing of non-stationary constituent pixels, grouping of adjacent pixel clusters, search processing of the lowest point in each group, and lowest point in each group Since the four processes consisting of the confirmation process are adopted, the pixel block is simply extracted, and those close to each other in the lower part of the screen are grouped, and the lowest point in the matrix extending direction is located in the group. It is possible to recognize the end of the matrix with a high degree of accuracy with a simple process of finding the pixel to be processed, and the content of the bottom point determination process differs depending on whether the matrix is expanded or contracted. While maintaining the reliability, there is an advantage that the overall responsiveness of the certification does not deteriorate so much.

  In addition, in the non-stationary body pixel extraction process, only the contour of the non-stationary body pixel block is extracted, so that the number of pixels subject to various arithmetic processes in the subsequent stage can be greatly reduced. Therefore, it will contribute to speeding up the processing and reducing the load on the personal computer.

  In addition, as a method to improve the end recognition reliability when expanding the end of the matrix, the processing of comparing the lowest points searched in the group between frames is adopted, so the number of pixel blocks varies between frames. In this case, the end recognition reliability can be maintained without being affected by the influence.

  Furthermore, when extracting the contour of the non-stationary body pixel block, the contraction process and the expansion process are introduced to remove the noise smudges and to make the contour line continuous with a thick line, so that the pixels in the labeling process provided in the subsequent stage It is possible to achieve good connection between each other, increase the size and decrease the number of pixel blocks, and thereby improve the efficiency of the proximity search process between the pixel blocks in the subsequent grouping process.

<Others>
In the above embodiments, the image processing apparatus for recognizing the end of the matrix according to the present invention is applied for the purpose of estimating the waiting time, but in addition to that, it can pass according to the maximum value of the end of the recognized matrix. Automatically increase or decrease the number of lanes, or in the case of a supermarket cashier, automatically call cashier staff to increase or decrease the number of available cashiers, or based on the time series record at the end of the recognized matrix Of course, it can be applied to various usages such as analyzing the usage trend of gates to find an optimal gate operation method.

  Further, the above flowcharts (FIGS. 4 to 12) are for conceptually explaining the present invention. In actual programming, various noise filtering processes are performed in consideration of assumed noise components and the like. It goes without saying that it is applied.

  According to the image processing apparatus for recognizing the end of a queue according to the present invention, for example, it is possible to recognize the end of a queue appearing in front of an entrance gate, a ticketing machine or the like with relatively high accuracy, and use this. Thus, for example, the waiting time of the matrix can be obtained.

1 Waiting time generation PC
2-1 to 2-4 Camera 3 Person who constitutes queue 4 Contour line 4A Thickened contour line 5 Blemish corresponding to noise 6 Arrow indicating direction of passerby crossing A Search area L1 to L4 First lane to 4th lane PCG11, PCG12, PCG13 Pixel blocks constituting the queue PCG2 Pixel blocks other than the queue (passers, etc.)
PCG31, PCG32 Pixel block constituting background object

Claims (16)

An image processing apparatus for recognizing the end of a queue from a moving image obtained by bird's-eye shooting of an area where a queue composed of one or more lanes is expected to appear,
Non-stationary object extraction means for extracting pixels constituting a non-stationary object from each of a series of frame images constituting the moving image;
The extracted non-stationary body pixels are divided into pixel blocks formed by collecting adjacent pixels together, and a series of pixels adjacent to each other in the predetermined matrix extending direction among the divided pixel blocks. Pixel block grouping means for storing the block as a pixel block belonging to one group;
A farthest point search means for searching for the farthest point in a predetermined matrix extending direction from all the pixel blocks belonging to one pixel block group and storing it as a matrix end candidate;
When the farthest point stored as the matrix tail candidate changes beyond a predetermined allowable value between consecutive frame images, it is stored as the matrix tail candidate based on the change direction and the stability after the change. An image processing apparatus for recognizing the end of a matrix, including farthest point determination means for determining the farthest point to be recognized as the farthest point to be recognized as the true end of the matrix.   When the farthest point determination means is such that the direction of change of the farthest point is away from the top of the matrix, the matrix only when the farthest point after the change is maintained substantially constant over a predetermined number of frames. The image processing device for recognizing a matrix end according to claim 1, further comprising a process of determining a farthest point stored as a tail candidate as a farthest point to be recognized as a true matrix end.   When the farthest point determination means is such that the direction of change of the farthest point approaches the top of the matrix, the farthest point stored as the matrix end candidate should be immediately recognized as the true end of the matrix. The image processing apparatus for recognizing the end of a matrix according to claim 2, comprising a process for determining the farthest point.   The image processing for recognizing the end of a matrix according to claim 1, wherein the overhead view shooting is performed from the rear of an assumed queue, and the predetermined matrix extending direction is a downward direction of each frame image. apparatus. The non-stationary body extraction means is
Contour extracting means for converting the frame image into a line image obtained by extracting contours of various images included therein;
In the line image, a background image deletion unit that deletes a line image portion corresponding to a background using background difference processing;
The image processing apparatus for recognizing the end of a matrix according to claim 1, further comprising: a sharpening unit that sharpens lines constituting the line image after the background difference processing. The contour extracting means is
Gray scale converting means for converting the frame image into a monochrome image;
Spatial primary differentiation means for performing spatial primary differentiation on the monochrome image;
The image processing apparatus for recognizing the end of a matrix according to claim 5, further comprising: binarization means for binarizing the image after the spatial first differentiation process to generate a line image. The background deletion means
6. The method according to claim 5, further comprising: comparing a value of a pixel included in a line constituting the line image with a value of a predetermined number of frames before the pixel, and deleting a pixel having the same value as a background pixel. Image processing device for matrix end recognition. The sharpening means is
Contraction means for removing fine stains caused by noise or the like included in the line image after the background deletion;
6. The image processing apparatus for recognizing a matrix end according to claim 5, further comprising expansion means for connecting the broken wavy lines included in the line image after the background processing to thicken the line. The pixel block grouping means includes:
Among the pixels constituting the extracted non-stationary body, labeling means for classifying the pixels constituting the non-stationary body for each pixel block by attaching the same label to pixels adjacent to each other;
Among the pixels included in one pixel block corresponding to the top of the matrix, the pixel located at the farthest point in the predetermined matrix extending direction is used as a reference, and a predetermined value set toward the far side in the extending direction The search area is searched for whether or not another labeled pixel exists, and when it exists, the process of storing the pixel block including the pixel as a pixel block belonging to the same group is newly the same. 2. The matrix end recognition according to claim 1, further comprising: adjacent pixel block search means that repeats all of the head equivalent pixel blocks until no other labeled pixels exist for the pixel blocks stored as a group. Image processing device.   The farthest point searching means searches for a pixel located at the farthest point in a predetermined matrix extending direction among all the pixels included in one pixel block belonging to the same group. The image processing apparatus for recognizing the end of a matrix according to claim 1, comprising a process of specifying the farthest point in the same groove by repeating the pixel block. The farthest point determination means,
When the difference between the value of the current frame at the farthest point in the same group and the value of the previous frame is within a predetermined allowable value, the predetermined frame counter is incremented. Counter control means to clear,
When the difference between the value of the current frame at the farthest point in the same group and the value of the previous frame changes beyond a predetermined tolerance, when the direction of change is near, the value of the current frame While the farthest point value is updated, if the change direction is far direction, the value of the frame counter waits for the predetermined number of frames, and the true farthest point value is updated with the current frame value. The farthest point true value update control means for the matrix end recognition image processing apparatus according to claim 1. An image processing method for recognizing the end of a queue from a moving image obtained by taking a bird's-eye view of an area where a queue composed of one or more lanes is expected to appear,
A non-stationary object extraction step of extracting pixels constituting a non-stationary object from each of a series of frame images constituting the moving image;
The extracted non-stationary body pixels are divided into pixel blocks formed by collecting adjacent pixels together, and a series of pixels adjacent to each other in the predetermined matrix extending direction among the divided pixel blocks. A pixel block grouping step for storing the block as a pixel block belonging to one group;
A farthest point search step of searching for the farthest point in a predetermined matrix extending direction from all the pixel blocks belonging to one pixel block group and storing it as a matrix end candidate;
When the farthest point stored as the matrix tail candidate changes beyond a predetermined allowable value between consecutive frame images, it is stored as the matrix tail candidate based on the change direction and the stability after the change. And a farthest point determining step for determining the farthest point to be recognized as the farthest point to be recognized as the true end of the matrix. Computer
In order to recognize the end of the queue from a moving image obtained by bird's-eye shooting of an area where a queue composed of one or more lanes is expected to appear,
Non-stationary object extraction means for extracting pixels constituting a non-stationary object from each of a series of frame images constituting the moving image;
The extracted non-stationary body pixels are divided into pixel blocks formed by collecting adjacent pixels together, and a series of pixels adjacent to each other in the predetermined matrix extending direction among the divided pixel blocks. Pixel block grouping means for storing the block as a pixel block belonging to one group;
A farthest point search means for searching for the farthest point in a predetermined matrix extending direction from all the pixel blocks belonging to one pixel block group and storing it as a matrix end candidate;
When the farthest point stored as the matrix tail candidate changes beyond a predetermined allowable value between consecutive frame images, it is stored as the matrix tail candidate based on the change direction and the stability after the change. A computer program for functioning as an image processing apparatus for recognizing the end of a matrix, comprising farthest point determination means for determining the farthest point to be recognized as the farthest point to be recognized as the true end of the matrix A camera for taking a bird's-eye view of an area where a queue of one or more lanes is expected to appear;
Including an image processing device for recognizing the end of the queue from a moving image obtained from the camera,
The image processing apparatus includes:
Non-stationary object extraction means for extracting pixels constituting a non-stationary object from each of a series of frame images constituting the moving image;
The extracted non-stationary body pixels are divided into pixel blocks formed by collecting adjacent pixels together, and a series of pixels adjacent to each other in the predetermined matrix extending direction among the divided pixel blocks. Pixel block grouping means for storing the block as a pixel block belonging to one group;
A farthest point search means for searching for the farthest point in a predetermined matrix extending direction from all the pixel blocks belonging to one pixel block group and storing it as a matrix end candidate;
When the farthest point stored as the matrix tail candidate changes beyond a predetermined allowable value between consecutive frame images, it is stored as the matrix tail candidate based on the change direction and the stability after the change. And a farthest point determining means for determining a farthest point to be recognized as a true farthest point to be recognized as a true end of the matrix. An image processing device for matrix end recognition according to claim 1,
A waiting time estimating means for estimating a current matrix waiting time by comparing a matrix end position obtained from the image processing apparatus statistically in advance with a relationship between a matrix end position and a waiting time of the matrix; An apparatus for estimating a matrix waiting time.   The current waiting time is estimated for each of the multiple lane queues, and an overall waiting time corresponding to the multiple lanes is determined based on the waiting times. Matrix waiting time estimation device.