JP2013205412A - Method for detecting size of object in space monitoring apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method for detecting how much size an object existing in a monitoring space area has in an arbitrary monitoring space area.SOLUTION: A space area including a monitoring space area whose range is arbitrarily set is photographed by two digital cameras, parallax is found out from obtained digital images, a spatial coordinate position of an object projected to pixels is calculated, a set of adjacent pixels is discriminated and aggregated, and the size of the object is detected by the number of pixels included in the set. Namely, the size occupied by the object in the monitoring space area is detected from the number of pixels existing in the position included in the monitoring space area out of pixels included in the set of adjacent pixels.

Description

Detailed Description of the Invention

Field of Invention

The present invention relates to a method for detecting how large an object is in an apparatus for detecting whether or not an object is present in an arbitrarily set surveillance space area, and more specifically to an arbitrarily set space area. Is taken with two digital cameras, the parallax is obtained from the obtained digital image, the spatial coordinate position of the object shown in the pixel is calculated, the set of adjacent pixels is discriminated, and the size of the object is determined by the number of pixels included in the set. This is to detect the thickness.

The basics of the space monitoring device will be described with reference to FIG. 6 functional diagram of the space monitoring device, FIG. 7 structural diagram of the space monitoring device, and FIG. As shown in FIGS. 6 and 7, the space monitoring device includes a standard image camera 61, a reference image camera 62, a main control device 66, and a spatial distance calculation device 67.

The control signal 65a, 65b from the main controller 66 instructs the base image camera 61 and the reference image camera 62 to repeatedly shoot, and the taken base image data 63 and reference image data 64 are spatial. It is sent to the distance calculation device 67, where various calculation processes are performed, and an alarm signal 68. An image signal 69 is output.

The calculation processing method of the spatial distance calculation device 67 is described in FIG.
In step S801, the vertex coordinates of the three-dimensional shape of the monitoring space area are set in order to set the monitoring space area. As a setting method of the monitoring space area, it is possible to set all coordinates in the monitoring space area determined by the function of the side of the monitoring space area, the function of the surface, and the resolution of the monitoring device.

In step S802, standard image data 63 and reference image data 64 are acquired and stored from the standard image camera 61 and the reference image camera 62, respectively, for stereo matching processing.

In step S803, stereo matching processing is performed using the reference image data 64 in order to obtain the parallax of the pixels in the standard image data 63. The parallax of each pixel is held as data representing the position and parallax value of the pixel of the reference image data 63 as a parallax map.

In step S804, the spatial coordinate position indicated by each pixel that is equal to or greater than a specific parallax value is represented by the parallax value and the baseline lengths of the two cameras in order to confirm whether or not an object reflected in the pixel is present in the monitoring space area. Calculate using the focal length of the lens.

In step S805, in order to determine whether or not the object is included in the monitoring space area, the inclusion relationship between the space coordinate position of the object shown in each pixel calculated in step S804 and the monitoring space area is checked. If there is a pixel in the monitoring space area, it is determined that the object is in the monitoring space area, and an alarm signal is output in step S806. If no pixel exists, it is determined that there is no object in the monitoring space area.

In step S807, the standard image data 63, the reference image data 64, the parallax map, and the spatial coordinate position of the object are output as image information separately from the alarm for real-time confirmation and post-mortem analysis of the space monitoring situation.

In step S808, it is confirmed whether there is an instruction to end monitoring, and monitoring is ended if necessary.
The above is the basic flow of the arithmetic processing of the conventional space monitoring device, but this method always outputs an alarm if the object exists in the monitoring area regardless of the size of the detected object. It was not possible to realize the desire to avoid alarms for small objects such as insects or cats.

Problems to be solved by the invention

An object of the present invention is to shoot an arbitrary spatial region with two digital cameras arranged to generate parallax, obtain parallax values of all pixels from the obtained digital image by a stereo matching method, and obtain the obtained parallax value To calculate a spatial coordinate position of an object shown in each pixel, discriminate a pixel set in which the number of pixels in an adjacent spatial coordinate position is equal to or greater than a certain number, and monitor the pixels in the discriminated pixel set The processing speed is increased by detecting the size of the object to be detected by counting the pixels located in the region. A second problem is to provide a space monitoring device that can determine in advance the size of an effective object, that is, an object to be detected.

Means for solving the problem

According to the first aspect of the present invention, the parallax of all pixels is obtained by a stereo matching method using digital image data photographed by two digital cameras arranged to generate parallax in an arbitrary monitoring space region, In the space monitoring device that detects an object reflected in the image, a parallax value and a spatial coordinate position of each pixel are obtained, and a pixel having a specific parallax value is used as a nucleus, and a coordinate position above, right, below, and left of this nucleus pixel is determined. Find the difference in parallax value with the pixel, first gather the pixels where the difference in parallax value is less than or equal to a predetermined value from the top, right, bottom, and left, and then use the gathered pixels as adjacent confirmation target pixels sequentially The parallax value difference between the upper, right, lower, and left pixels is calculated, but in the same way, assembling of pixels whose parallax value difference is equal to or less than the predetermined value is advanced first, and finally it is not possible to perform the aggregation The number of pixels that have been assembled so far The size detection method is of an object and detecting the magnitude.

According to a second aspect of the present invention, in the first aspect, when the difference between the parallax values is equal to or less than a predetermined value, an average parallax value of all the aggregated pixels is obtained, and the average parallax value and the total number of pixels aggregated determine the object Calculate the area of the monitoring surface, and if the calculated area is less than the threshold value of the area to be determined as an effective object, determine that there is no detection object, and if the calculated area is equal to or greater than the threshold value of the area to be determined as an effective object This is a method for detecting the size of an object that is determined to be present.

Effect of the invention

In the present invention, the position of each pixel in the two-dimensional planar parallax map is calculated without calculating the three-dimensional coordinates based on the camera of the object that the pixel is imaged to determine whether or not they are three-dimensionally adjacent. Since the determination is made based only on the relationship (upper right lower left) and the difference between the parallax values, adjacent pixels can be aggregated at high speed, and the object detection time can be shortened.

Further, the area of the monitoring surface of the object captured by the aggregated pixels is calculated, and if the calculated area is less than the threshold value of the area to be determined as an effective object, it is determined that there is no detection object, and the calculated area is determined as an effective object. Since it is determined that there is a detected object if it is equal to or larger than the threshold value of the area to be determined, by changing the threshold value of the area that is determined to be an effective object, the size at which detection is desired can be freely set. Therefore, if the threshold value of the area determined as an effective object is increased to some extent, there is an effect that small objects such as insects and cats can be prevented from being detected.

Embodiment of the Invention

The flowchart shown in the flow chart of FIG. 1 spatial monitoring pixel discrimination processing is details of the pixel set discrimination processing. Pixel set discrimination is a collection of pixels on the parallax map as a core, calculates the size of the object from the average parallax value and the number of pixels in the pixel set, and extracts only those that are estimated to be larger than the specified size. It is processing to do. Hereinafter, it will be described together with an explanatory diagram of aggregation by confirmation of FIG. 3 adjacency. FIG. 3 shows an example of a state with a parallax map as a whole. Each rectangle represents a pixel in the parallax map, and the number in the rectangle is the parallax value of the pixel. A parallax value of 0 means no parallax, and the pixel does not represent a valid object or represents an object at infinity. For example, the pixel in column 1 and row 1 has a parallax value of 0, so it is not a pixel that captured an object, but the pixel in column 4 and row 2 has a parallax value of 5, and is a pixel that has captured an object. A portion surrounded by a thick line is a pixel set created by the following aggregation process.

In step S101, the first pixel is set as a candidate for the nucleus of the pixel set. In FIG. 3, the pixel in column 1 and row 1 is the first candidate for the nucleus of the pixel set, and then column 2 row 1, column 3 row 1, column 20 row 1, column 1 row 2, column 20 row 2,. It becomes a nuclear candidate in the order of line 20. Note that the core pixel is set to a specific parallax value, for example, 5 or more.

In step S102, it is checked whether there is a candidate pixel among the pixels belonging to the existing pixel set so that the nucleus candidate pixel is not registered in a plurality of pixel sets. In FIG. 3, when the pixel set 33 is created using the pixels in column 14 and row 15 as the nucleus candidates, the pixel set 31 is already set using the pixels in column 4 and row 2 as the nucleus, and the pixel set using the pixels in column 3 and row 3 as the nucleus. 32 is generated, it is confirmed whether or not the pixel set 31 and the pixel set 32 do not include the pixel of column 14 row 15.

If it does not belong to any pixel set, a new pixel set is created in step S103, a candidate pixel for a nucleus is determined as a nucleus pixel, and a target pixel (adjacency confirmation target pixel) for performing adjacent confirmation in step S104 To do. In the case of the pixel in the column 14 and row 15 in FIG. 3, since the pixel in the column 14 and row 15 does not exist in the pixel set 31 and the pixel set 32, a new pixel set 33 is generated using this pixel as a nucleus, and the adjacency confirmation is further performed. As a target pixel, pixel aggregation is performed by adjacency confirmation. That is, in step S104, in order to estimate the size of the object represented by the 14th row 15 of the pixel column of the pixel set, adjacent pixels are examined based on the 14th row 15 of the core pixel column, and the pixel set 33 is determined. create. Details of how to create a set (grouping of pixels) will be described later.

In step S105, in order to discriminate the size of the effective object, that is, the object to be detected, the average parallax value of the pixels of this pixel set is obtained, and the core pixel is copied from the average parallax value and the total number of pixels in the pixel set. Calculate the area of the monitoring surface of the object. If the calculated area is less than the threshold value of the area to be determined as an effective object, the pixel set is determined as a pixel set without a detected object in step S106. If the calculated area is equal to or greater than the threshold value of the area to be determined as an effective object, a pixel with a detected object is determined in step S107. Let it be a set. In the pixel set 33 in FIG. 3, when the number of pixels is 21, the average parallax value is 30.5, and the distance occupied by a pixel at the position of 3000 mm and 3000 mm is 100 square mm, the distance when the parallax value is 30.5 is 21. X100 = 2100 square mm. For example, if the threshold value of the area to be determined as an effective object is 1000 square mm, the pixel set 33 is a pixel set with a detected object.

In step S108, it is checked whether or not the confirmation of all the pixels on the parallax map has been completed. If completed, the confirmation of the pixel set is completed. In FIG. 3, it is checked whether all the pixels from column 1 row 1 to column 20 row 20 have been confirmed.

In step S109, when a pixel that can be a nucleus candidate remains in the parallax map, the pixel is set as a nucleus candidate of a new pixel set, and the process returns to step S102 to repeat the pixel grouping process. In addition, also when the confirmation of the pixel in column 14 and row 15 is completed in FIG. 3, the pixel grouping process is performed on the pixel in column 15 and row 15 from step S102.

FIG. 2 is a flowchart showing the details of the pixel grouping process performed in step S104. Pixel grouping is a procedure in which a certain pixel is used as a core, and whether or not the upper right lower left pixel is sequentially adjacent to each other in three dimensions. This process repeats recursively until there are no three-dimensional neighbors. However, even if the pixels are three-dimensionally adjacent to each other, the present invention is substantially equivalent to a state in which the difference between the parallax values between the pixels located on the two-dimensional parallax map in the vertical and horizontal directions is equal to or less than a predetermined value. Is used.

Fig. 4 shows the positional relationship between the monitoring device, the object, and the parallax map. Fig. 3 shows the positional relationship between the monitoring device, the object, and the parallax map in the monitoring space when the parallax map shown in Fig. 3 is obtained. ing. The parallax map 44 is shown as in FIG. An object 41 indicated by a one-dot chain line is copied as a pixel set 31 on the parallax map of FIG. 3, an object 42 indicated by a solid line is copied as a pixel set 32, and an object 43 indicated by a dotted line is copied as a pixel set 33. It is included. A dotted line coming out from the monitoring device 45 indicates a horizontal position of an object captured by each pixel, and is acquired as a parallax value of a pixel at a position intersecting with the parallax map 44. In other words, the column number of the parallax map in FIG. 3 indicates the position in the horizontal (width) direction when the object exists in the monitoring space as described above, and the object increases on the right side as the column number increases. It will exist. On the other hand, the row number in FIG. 3 indicates the position in the vertical (height) direction in the monitoring space, and the smaller the row number, the higher the object is located.
Further, the parallax value of a pixel that captures one object, for example, the object 42, is substantially the same value (here, 20 to 21) as in the pixel set 32 of FIG.

Here, as can be seen from the parallax map in FIG. 3, the vertical direction of the object 41 occupies the range from row 2 to row 20 of the parallax map. On the other hand, the object 42 is located in the range of the rows 3 to 6, and the object 43 is located in the range of the rows 15 to 19. Therefore, if FIG. 4 is a horizontal plane corresponding to row 3 on the parallax map of FIG. 3, the object 41 and the object 42 can be confirmed on the parallax map, but the object 43 at a low position cannot be confirmed. On the other hand, if FIG. 4 is a horizontal plane of the row 15 including the object 43, the object 41 and the object 43 can be confirmed, but the object 42 at a high position cannot be confirmed. For example, considering the case where FIG. 4 is a horizontal plane corresponding to row 3 of the parallax map of FIG. 3, the parallax corresponding to the horizontal column 3, column 4, column 5, and column 6 of the pixel set 32 for the object 42 is considered. A value is obtained, and for the object 41, parallax values corresponding to the columns 7 to 15 of the pixel set 31 are obtained.

Note that since FIG. 4 is a view seen from above, only the width direction can be confirmed, but the pixels that capture one object are also adjacent in the width direction of the parallax map, and are adjacent two-dimensionally. It becomes a state. Therefore, a simple two-dimensional calculation can be performed without a complicated three-dimensional calculation.

Next, a method for examining the upper, lower, left and right pixels will be described with reference to FIG. In step S201, in order to confirm the adjacency between the target pixel for adjacency confirmation and the pixel located thereabove, the difference between the parallax values of the target pixel for adjacency confirmation and the pixel above it is calculated. If the difference between the parallax values is equal to or smaller than a predetermined value of the difference between the parallax values to be aggregated, it is determined that the target pixel to be confirmed for adjacency and the pixel located thereon are adjacent.

If it is determined that the pixels are adjacent to each other, in step S202, the pixels located above are grouped together, and then the upper pixel is set as an adjacent confirmation target pixel, and pixel aggregation proceeds based on steps S201 to S208. If it is determined that the pixel above is not adjacent, the process proceeds to step S203. Until then, this process is performed recursively.

In step S203 to step S204, the adjacency with the pixel located on the right side is confirmed.
That is, in step S203, in order to confirm the adjacency between the above-described adjacent confirmation target pixel and the pixel located on the right side thereof, the difference between the disparity values between the adjacent confirmation target pixel and the right pixel is calculated, and the difference between the disparity values is calculated. Is equal to or less than a predetermined value which is a condition for aggregation, it is determined that the adjacent confirmation target pixel and the pixel located on the right side thereof are adjacent to each other.

If it is determined that the pixels are adjacent to each other, the pixels located on the right are assembled in step S204, and the right pixel is set as an adjacent confirmation target pixel. Then, the pixel aggregation proceeds based on steps S201 to S208. If it is determined that the right pixel is not adjacent, the process proceeds to step S205. Until then, this process is performed recursively.

In step S205 to step S206, the adjoining with the pixel located below is confirmed.
That is, in step S205, in order to confirm the adjacency between the above-described adjacent confirmation target pixel and the pixel located therebelow, the difference between the parallax values between the adjacent confirmation target pixel and the pixel below the pixel is calculated. If it is equal to or less than a predetermined value that is a condition for grouping, it is determined that the adjacent confirmation target pixel and the pixel located therebelow are adjacent.

If it is determined that the pixels are adjacent to each other, the pixels located below are assembled in step S206, and the lower pixel is set as an adjacent confirmation target pixel. Then, the pixel aggregation proceeds based on steps S201 to S208. If it is determined that the lower pixel is not adjacent, the process proceeds to step S207. Until then, this process is performed recursively.

In steps S207 to S208, the adjacent pixel to the left is confirmed.
That is, in step S207, in order to confirm the adjacency between the above-described adjacent confirmation target pixel and the pixel located on the left side, the difference between the parallax values between the adjacent confirmation target pixel and the left pixel is calculated. If it is equal to or less than a predetermined value which is a condition for aggregation, it is determined that the adjacent confirmation target pixel and the pixel located on the left side are adjacent to each other.

If it is determined that the pixels are adjacent to each other, the pixels located on the left in step S208 are assembled, and then the pixels on the left are set as adjacent confirmation target pixels, and the pixel aggregation is advanced based on steps S201 to S208 described above. If it is determined that the pixel to the left of the adjacent confirmation target pixel is not adjacent, the process proceeds to step S201. Until then, this process is performed recursively.

When the pixel in column 14 and row 15 in FIG. 3 is used as an adjacent confirmation target pixel, the difference in parallax value from the upper (column 14 and row 14) pixel is | 30-0 | = 30. For example, if the predetermined value of the difference between the parallax values to be aggregated is 1, it is determined that this state is not adjacent, so it is not included in the pixel set.
When the difference in the parallax value with the pixel on the right (column 15 row 15) is examined, the difference is | 30-30 | = 0, so this state is determined to be adjacent, and the pixel on the right (column 15 row 15) is determined. The position and parallax value included in the pixel set are stored. When a pixel included in the pixel set is found in this manner, the adjacent right (column 15 row 15) pixel is first connected before checking the pixel in column 14 and row 15 and the lower and left pixels. It is a rule to check. Therefore, this time, the adjacency confirmation is further performed using the pixel (column 15 row 15) as the adjacency confirmation target pixel. In other words, when the pixel at column 15 and row 15 is included in the pixel set, pixel aggregation processing (step S201 to step S208) is recursively performed with column 15 and row 15 as the adjacent confirmation target pixel.

FIG. 5 is an explanatory diagram of the assembling process based on the adjacency confirmation, and shows the assembling process of the pixel set 33 shown in FIG. Each table shows the pixels to be checked for adjacency. At the top of the table, the grouping order (A to U), columns, and rows of the pixels are shown. Next, the pixels for checking the adjacency on the top, right, bottom, and left are shown. Next, whether or not adjacent confirmation pixels can be assembled is shown. That is, “difference” indicates that the disparity value difference is large and cannot be aggregated, and “(A)” indicates that they are already aggregated in the order of (A), and an arrow appears. Pixels are aggregated, and the adjacency confirmation is performed as the adjacency confirmation target pixel at the tip of the arrow.
In FIG. 5, as indicated by the arrows in the pixel set 33 in FIG. 3, first, the pixels from (A) column 14 row 15 to (Q) column 16 row 19 are assembled, and then (R) column 14 row 19. The pixels in (S) column 13 and row 18 to (U) column 13 and row 16 are further gathered.

Flow chart of pixel discrimination processing for space monitoring Spatial monitoring pixel aggregation process flow diagram Explanatory diagram of grouping by checking neighbors The figure which shows the positional relationship of a monitoring apparatus, an object, and a parallax map Explanatory diagram of assembly process by checking neighbors Functional configuration diagram of the space monitoring device Structure diagram of space monitoring device Flow chart of overall processing of space monitoring

S101 Nuclei candidate pixel initialization step S102 Nuclei candidate pixel pre-aggregation confirmation step S103 New pixel set generation and adjacent confirmation target pixel setting step S104 Pixel aggregation step S105 Pixel aggregation detection area calculation step S106 Detected object invalidation step for pixel set S107 Detected object validation step for pixel set S108 Confirmation step for next candidate pixel in nucleus S109 Movement step 31 for candidate pixel in nucleus Pixel set 32 generated using column 4 and row 2 as a nucleus Pixel set generated with column 3 and row 3 as the nucleus 33 Pixel set generated with column 14 and row 15 as the nucleus

Claims (2)

Spatial monitoring to detect parallax of all pixels by stereo matching method using digital image data taken by two digital cameras arranged to generate parallax in an arbitrary surveillance space area and detect an object reflected in the pixel In the apparatus, the parallax value and the spatial coordinate position of each pixel are obtained, and the difference between the parallax value and the pixel at the upper, right, lower, and left coordinate positions is determined using the pixel having the specific parallax value as the nucleus. First, the pixels whose difference in parallax value is less than or equal to a predetermined value are first gathered from among the top, right, bottom, and left, and then the gathered pixels are sequentially selected as the adjacent confirmation target pixels, and the top, right, bottom, and left The difference in the parallax value from the pixel is calculated. Similarly, when the aggregation of the pixels where the difference in the parallax value is equal to or less than the predetermined value is first promoted and finally it becomes impossible to aggregate, until now The size of the object is detected by the number of pixels assembled in Size detection method of an object according to claim. In claim 1, when the difference between the parallax values is equal to or less than a predetermined value, the average parallax value of all the aggregated pixels is obtained, and the area of the monitoring surface of the object is calculated from the average parallax value and the total number of pixels aggregated, If the calculated area is less than the threshold value of the area to be determined as an effective object, it is determined that there is no detection object. If the calculated area is equal to or greater than the threshold value of the area to be determined as an effective object, the size of the object to be determined as having a detection object Detection method.

Images