JP5731354B2 - Object number measuring device, object number measuring method, and object number measuring program - Google Patents

Description

  The present invention measures the number of objects (hereinafter referred to as the number of objects) in a captured image, and the number of objects in a station, bus stop, or other public place, and the flow of people on a road or a passage. The present invention relates to an object number measurement device, an object number measurement method, and an object number measurement program.

  As a method for measuring the number of objects of a specific object by performing image processing on images taken by a camera at a station, a bus stop, a road, a passage, or the like, a method described in Patent Document 1 is known. In this method, how much each pixel of an image contributes to the number of objects is determined by using a perspective projection relationship in which external parameters including the posture and position of the image input device are associated with three-dimensional points in space. A load value is set as a contribution rate expressed quantitatively, and the number of objects in the image is estimated from the product of the foreground region indicating the region where the object exists in the image and the load value. This method has an advantage that even when it is difficult to recognize and detect an object as a single object, the number of objects can be estimated as long as it can be detected as a foreground region.

JP 2009-294755 A

  In the prior art of Patent Document 1, the load value is calculated on the assumption that the size and shape of the target object is a specific object (specifically, a rectangle whose height and width are known). Therefore, when the size / shape of the target object is different from the specific size / shape described above, or when the object shape changes on the image due to false detection / non-detection of the foreground area or walking movement of the person Has a problem that a fraction is generated in the sum of the load values in the foreground region caused by one object, and an error occurs in the estimated number of objects. In particular, when the number of objects in the image is small, the influence of this error on the estimated number of objects becomes large.

  The present invention has been made in view of such circumstances, and uses a monocular surveillance camera or a fixed point camera installed indoors / outdoors to accurately measure the number of objects in the shooting range even when the number of objects is small. An object number measuring device, an object number measuring method, and an object number measuring program are provided.

  The present invention is an object number measuring apparatus for measuring the number of objects in an image, and stores a load value table in which a load value indicating a contribution ratio of each pixel of the image to the number of objects is defined. A foreground extraction means for extracting a foreground area indicating an area where an object is present from the image, a partial foreground area generation means for generating a spatially continuous partial foreground area in the foreground area, and the generated part Partial object number calculating means for calculating the number of objects from each pixel included in each of the foreground areas and the load value of the pixel obtained from the load value table, and the number of objects in the entire image from the calculated number of partial objects. And a total object number calculating means for calculating.

  The present invention is an object number measuring apparatus for measuring the number of objects in an image, and stores a load value table in which a load value indicating a contribution ratio of each pixel of the image to the number of objects is defined. And a foreground extraction means for extracting a foreground area indicating an area where an object is present from the image, and the load value table is referred to, and the foreground area weighted by the load value is projected in the vertical direction of the image. A foreground projection generating means for generating a foreground projection; a partial foreground projection generating means for generating a partial foreground projection in which the foreground projection continues from the generated foreground projection; and a number of objects from the generated partial foreground projection. A partial object number calculating means for calculating, and an overall object number calculating means for calculating the number of objects in the entire image from the calculated number of partial objects.

  The present invention provides a table storage means for storing a load value table in which a load value indicating a contribution ratio of each pixel of an image to the number of objects is defined in order to measure the number of objects in the image. A method for measuring the number of objects in a measuring apparatus, wherein a foreground extraction step for extracting a foreground area indicating an area where an object is present from the image, and a partial foreground area for generating a spatially continuous partial foreground area in the foreground area A generation step, a partial object number calculation step for calculating the number of objects from each pixel included in each of the generated partial foreground regions and the load value of the pixel obtained from the load value table, and each calculated portion And a total object number calculating step of calculating the number of objects of the entire image from the number of objects.

  The present invention provides a table storage means for storing a load value table in which a load value indicating a contribution ratio of each pixel of an image to the number of objects is defined in order to measure the number of objects in the image. A method for measuring the number of objects in a measuring apparatus, the foreground extraction processing step for extracting a foreground region indicating a region where an object exists from the image, and the foreground weighted by the load value with reference to the load value table A foreground projection generating step for generating a foreground projection in which a region is projected in the vertical direction of the image; a partial foreground projection generating step for generating a partial foreground projection in which the foreground projection continues from the generated foreground projection; A partial object number calculating step for calculating the number of objects from the partial foreground projection, and an overall object number calculating step for calculating the number of objects in the entire image from the calculated number of partial objects. And having a flop.

  The present invention is characterized by causing a computer to execute the object number measuring method.

  According to the present invention, it is possible to obtain an effect that the number of objects can be accurately measured even when the number of objects in the shooting range is small.

It is a block diagram which shows the structure of the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the apparatus shown in FIG. It is explanatory drawing which shows an example of partial foreground area | region production | generation. It is explanatory drawing which shows an example of the advantage by partial object number calculation. It is a block diagram which shows the structure of the 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the apparatus shown in FIG. It is explanatory drawing which shows an example of foreground area | region projection and partial foreground projection production | generation. It is explanatory drawing which shows an example of partial foreground projection production | generation.

<First Embodiment>
Hereinafter, an object number measuring apparatus according to a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the embodiment. In FIG. 1, reference numeral 1 denotes an image input unit which inputs, for example, an image sequence photographed in real time by a camera or an image sequence photographed and accumulated in advance. For image storage, either a form using a recording medium such as a hard disk, a RAID device, or a CD-ROM, or a form using a remote data resource via a network may be used. Reference numeral 2 denotes a background image generation unit that generates a background image (an image in which no moving object exists) from the image acquired by the image input unit 1. Reference numeral 3 denotes a foreground extraction processing unit that detects and extracts a foreground region from the image acquired by the image input unit 1 and the background image generated by the background image generation unit 2. Reference numeral 4 denotes a partial foreground region generation unit that divides the foreground region extracted by the foreground extraction processing unit 3 into partial foreground regions composed of spatially continuous portions in the foreground region.

  Reference numeral 5 denotes a load value table storage unit that stores a load value table that lists load values as contribution ratios that quantitatively represent how much each pixel of an image contributes to the number of objects. The load value table may be generated during processing, or may be generated in advance and recorded on a recording medium or the like. Reference numeral 6 denotes the number of partial objects for measuring the number of objects from each pixel included in the partial foreground area generated by the partial foreground area generation unit 4 and the load value of the pixel obtained from the load value table storage unit 5. It is a calculation part. Reference numeral 7 denotes an overall object number calculation unit that calculates the number of objects of the entire image from the number of partial objects calculated by the partial object number calculation unit 6. Reference numeral 8 denotes an overall object number output unit that outputs the overall object number calculated by the overall object number calculation unit 7.

  Each process in the background image generation unit 2, the foreground extraction processing unit 3, the partial foreground region generation unit 4, the partial object number calculation unit 6, and the total object number calculation unit 7 is executed by a computer, for example.

  Next, the processing operation of the object number measuring apparatus shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a flowchart showing the processing operation of the object number measuring apparatus shown in FIG. First, the image input unit 1 takes in the first one of the image series arranged in time series (step S1). Subsequently, the background image generation unit 2 generates a background image from the captured image (step S2). As a background image generation method, for example, a method of using the first captured image as a fixed background, a method of storing a certain number of captured images and taking an average pixel value for each coordinate as a background can be applied. . Note that the number of captured images stored in the background image generation method can be set according to, for example, the frame rate of the image series and the average pixel value in which time range (for example, (If the average pixel value for 5 seconds is taken at 10 [fps], it will be 50).

  Next, the foreground extraction processing unit 3 uses the background image to extract a foreground region as a region where a moving object exists from the image captured by the image input unit 1 (step S3). Foreground region extraction is described in literature: “C. Ridder, O. Munkelt and H. Kirchner,“ Adaptive Background Estimation and Foreground Detection Using Kalman-Filtering, ”Proc. Int'l Conf. Recent Advances in Mechatronics, ICRAM '95, pp. 193-199, 1995 ”, literature:“ KA Patwardhan, G. Sapiro, V. Morellas, “Robust Foreground Detection in Video Using Pixel Layers,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30, No. Various methods described in "4, 2008." are known, and known methods can be applied. The format of the data representing the foreground area is, for example, a binary image (hereinafter referred to as 1) with the same number of pixels as the image captured in step S1, with the pixel of the coordinate extracted as the foreground area being 1 and the other pixels being 0. This is referred to as a foreground image). In addition, in order to reduce the effects of foreground detection omissions and false detections, processing such as opening of morphology operations (removal of foreground areas) and closing (filling of foreground areas) are performed on the foreground area generated by this method. You may give it.

  Next, the partial foreground area generation unit 4 generates a partial foreground area that is a spatially continuous part of the foreground area based on the foreground area (step S4). As the definition of spatial continuity used here, various things such as 4-connection and 8-connection (see FIG. 3) can be applied, but this is not limited here. Further, as a specific partial foreground region generation algorithm, for example, reference: “What, Nest, Suzuki, Nakamura, Ito,“ High-speed twice-scanning labeling algorithm ”, Science theory (D), vol. D, no. 4, pp. 1016-1024, 2008 "and the like, and such a labeling process can be used.

  Next, the partial object number calculation unit 6 refers to the load value table stored in the load value table storage unit 5 and measures the number of partial objects for each partial foreground region generated by the partial foreground region generation unit 4. (Step S5). As the load value table, for example, one generated by the method described in Patent Document 1 can be used. However, the method is not limited to the method of Patent Document 1, and a load value in a broad sense that weights each pixel value based on a change in size of an object on the image (for example, an image photographed with an obliquely downward camera) In the near side (below the image), the object appears large, and the back side (above the image) appears small, and based on this change in size, each pixel of the image is weighted by some geometric model or geometric calculation ( A value designated so that the number of objects can be estimated by taking the sum of the pixels belonging to the foreground area corresponding to the object may be used.

  The calculation of the number of partial objects is performed, for example, by taking the sum of the load values corresponding to the coordinates that are the foreground included in the partial foreground area and rounding the value to an integer. Various methods are known as the rounding process, including rounding off, but the method is not limited here. For example, in a situation where an excessive foreground area is likely to be detected, such as when a shadow of the target object is likely to appear on the image, the cut-off amount is increased (for example, rounded down to less than 0.7, 0.7 is a threshold value), etc. Appropriate rounding processing may be implemented as appropriate.

  On the other hand, if the foreground area is likely to be missing (a situation where a part of the target object is difficult to detect as the foreground), rounding processing is performed to increase the round-up amount (for example, round up 0.4 or more). Also good. By calculating the number of objects for each partial foreground area and converting it to an integer, it is possible to prevent the accumulation of errors due to the fraction of the calculated number of objects, compared to the method of taking the sum of the load values corresponding to the foreground area of the entire image. . For example, as shown in FIG. 4, when the shadow of an object is detected as a foreground and the total load value per object is 1.2 to 1.3, the sum of the load values of the entire foreground area is calculated. If it is taken, it will be 6.2, and an error of +1 will be generated by rounding off. However, if the number of partial objects is calculated, it will be 1 in each partial foreground region, and thus the total number of objects will be correctly calculated as 5.

  Next, the total object number calculation unit 7 calculates the total number of objects by taking the sum of the number of partial objects corresponding to each of the partial areas (step S6). Subsequently, the total object number output unit 8 outputs the total object number calculated by the total object number calculation unit 7 (step S7). Then, it is determined whether or not the next processing target image exists (step S8). If it exists, the process returns to step S1, and if it does not exist, the process ends.

<Second Embodiment>
Next, an object number measuring apparatus according to a second embodiment of the present invention will be described. In the first embodiment, a labeling process is used when generating a partial foreground area based on the foreground area. When the labeling process is executed on a two-dimensional array as in the first embodiment, a large amount of calculation time is generally required. In the second embodiment, the labeling process is executed only on the one-dimensional array to reduce the calculation time. FIG. 5 is a block diagram showing the configuration of the embodiment. In this figure, the same parts as those in the object number measuring apparatus shown in FIG. The object number measuring device shown in FIG. 5 differs from the object number measuring device shown in FIG. 1 in that instead of the partial foreground region generation unit 4 and the partial object number calculation unit 6, a foreground region projection unit 40, a partial foreground projection generation unit. 41 and the partial object number calculation unit 60 are provided, and the foreground area projection unit 40 refers to the load value in the load value table storage unit 5.

  The foreground region projection unit 40 generates a foreground projection in which the foreground weighted by the load value is projected in the image vertical direction from the foreground region extracted by the foreground extraction processing unit 3 and the load value table 5. The partial foreground projection generation unit 41 generates a partial foreground projection including one continuous non-zero region from the foreground projection generated by the foreground region projection unit 40. The partial object number calculation unit 60 calculates the number of objects corresponding to the partial foreground projection for each partial foreground projection generated by the partial foreground projection generation unit 41.

  Next, the processing operation of the object number measuring apparatus shown in FIG. 5 will be described with reference to FIG. FIG. 6 is a flowchart showing the processing operation of the object number measuring apparatus shown in FIG. In FIG. 6, the same parts as those shown in FIG. The operation shown in FIG. 6 is different from the operation shown in FIG. 2 in that steps S40, S41, and S50 are provided instead of steps S4 and S5.

First, image acquisition, background image generation, and foreground region extraction (steps S1 to S3) are performed. After the foreground extraction processing unit 3 extracts the foreground region, the foreground region projection unit 40 includes the foreground extraction processing unit 3 With reference to the extracted foreground region and the load value table stored in the load value table storage unit 5, a foreground projection is generated by projecting the foreground weighted by the load value in the vertical direction of the image (step S40). When taking the coordinates x and y as shown in FIG. 7, the foreground projection P (x) is calculated by the following equation.

  Here, p (x, y) is a pixel value at coordinates (x, y) when the foreground area generated by the foreground extraction processing unit 3 is represented by a foreground image. w (x, y) is a load value at coordinates (x, y) of the load value table stored in the load value table storage unit 5.

  Next, the partial foreground projection generation unit 41 generates a partial foreground projection including one continuous non-zero region from the foreground projection generated by the foreground region projection unit 40 (step S41). The partial foreground projection is obtained by, for example, labeling a binarized object in which the foreground projection P (x) is binarized with 1 being a non-zero point and other points being 0, similarly to step S4 of the first embodiment. A method can be applied in which P (x) of corresponding coordinates for a partial region is used as one partial foreground projection.

  FIG. 7 shows an example in which partial foreground projection is set by this method. In FIG. 7, “partial foreground projection 1”, “partial foreground projection 2”, and “partial foreground projection 3” are the set partial foreground projections. Since the labeling here is a labeling for a one-dimensional array, the labeling can be executed in a shorter calculation time than the labeling for a two-dimensional array performed in the first embodiment. In addition, as shown in FIG. 8, a method of setting a partial foreground projection by dividing a region with a point where P (x) becomes a minimum as a boundary can be applied. When the number of objects in the image increases, there is a case where there is no point where P (x) = 0 between the two local maximum points in the projection as in the example shown in FIG. is there.

  Next, the partial object number calculation unit 60 measures the number of partial objects for each partial foreground projection generated by the partial foreground projection generation unit 41 (step S50). The number of partial objects is calculated, for example, by taking the sum of P (x) in the range included in the partial foreground projection and rounding the value to an integer. As a specific method of the rounding process, various methods can be applied as in step S5 of the first embodiment. Then, the total number of objects is calculated, the total number of objects is output, and the process is repeatedly determined (steps S6 to S8).

  As described above, in a time-series image, it is possible to detect the foreground area with respect to a moving object or background image, and even when it is difficult to recognize and detect the object as a single object, the number of objects can be measured. Even when there are few objects, it is possible to measure accurately.

  The program for realizing the functions of the processing units in FIGS. 1 and 5 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. A number measurement process may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Accordingly, additions, omissions, substitutions, and other modifications of components may be made without departing from the spirit and scope of the present invention.

  It can be applied to applications where it is indispensable to measure the number of single objects in a photographed image and grasp the number of people in a predetermined area and the flow of people on a road / passage.

  DESCRIPTION OF SYMBOLS 1 ... Image input part, 2 ... Background image generation part, 3 ... Foreground extraction process part, 4 ... Partial foreground area generation part, 5 ... Load value table storage part, 6 ... Partial object number calculation unit, 7 ... Total object number calculation unit, 8 ... Total object number output unit, 40 ... Foreground area projection unit, 41 ... Partial foreground projection generation unit, 60 ... Part Object number calculator

Claims (3)

An object number measuring device for measuring the number of objects in an image,
Table storage means for storing a load value table in which a load value indicating a contribution ratio of each pixel of the image to the number of objects is defined;
Foreground extraction means for extracting a foreground area indicating an area where an object exists from the image;
Referring to the load value table, foreground projection generating means for generating a foreground projection in which the foreground region weighted by the load value is projected in the vertical direction of the image;
Partial foreground projection generation means for generating a partial foreground projection in which the foreground projection is continuous from the generated foreground projection;
Partial object number calculating means for calculating the number of objects from the generated partial foreground projection;
An object number measuring device comprising: an entire object number calculating means for calculating the number of objects in the entire image from the calculated number of each partial object. In order to measure the number of objects in an image, an object in an object number measuring apparatus provided with a table storage means for storing a load value table in which a load value indicating a contribution ratio of each pixel of the image to the number of objects is defined A number counting method,
A foreground extraction processing step of extracting a foreground area indicating an area where an object exists from the image;
A foreground projection generating step of generating a foreground projection by projecting the foreground region weighted by the load value in the vertical direction of the image with reference to the load value table;
A partial foreground projection generation step for generating a partial foreground projection in which the foreground projection is continuous from the generated foreground projection;
A partial object number calculating step of calculating the number of objects from the generated partial foreground projection;
And a total object number calculating step of calculating the total number of objects from the calculated number of partial objects. An object number measurement program for causing a computer to execute the object number measurement method according to claim 2 .

Images