JP2008276613A - Mobile body determination device, computer program and mobile body determination method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a mobile body determination device, for accurately determining the position and type of a mobile body, and to provide a computer program and a mobile body determination method therefor. <P>SOLUTION: A CPU 25 sets a pixel area, and sets a position of a target pixel. The CPU 25 extracts a characteristic quantity vector based on a correlation between the target pixel within the set pixel area and a specific pixel specified by a displacement direction from the target pixel. The pixel block size and frame number of the pixel area, the space between pixels and the frame space are changed according to the apparent size and moving speed of the mobile body in the position of the target pixel on an image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a moving body determination device that determines a type of a moving body based on a time-series captured image obtained by capturing an area including a road with an imaging device, and for realizing the moving body determination device with a computer. The present invention relates to a computer program and a moving object determination method.

  It is demanded to provide detailed traffic information in order to realize smooth traffic or prevent traffic accidents. For example, not only a device that can acquire macro traffic parameters such as the number of vehicles passing through a predetermined point on a road or an average speed, but also a device that can accurately measure the position or movement of each vehicle. Is required.

  These measuring devices are often installed outdoors, and stable measuring accuracy corresponding to various environmental changes is required. In order to spread social infrastructure using these devices, it is also essential to provide inexpensive devices.

  As an example of such an apparatus, an image-type vehicle detector using an image processing technique has been widely used. For example, a captured image obtained by capturing a road where no vehicle is present is created in advance, and a difference from the captured image captured when the vehicle travels on the road is calculated to extract information on only the vehicle. There has been proposed an apparatus for extracting an edge and detecting a vehicle position based on the distribution of the extracted edge (see Patent Document 1).

  In addition, there is a method in which a vehicle or other luminance distribution itself is used as a feature amount and the vehicle is discriminated by a neural network or the like. However, it is not realistic to prepare pattern data in various environments in advance. Since the processing time required for this is enormous, it has been very difficult to implement as an inexpensive device.

On the other hand, by extracting the correlation features in the local area around the pixel of interest and in the temporal direction based on the moving image, and taking into account the statistical distribution of motion features, the normal motion where the distribution is concentrated and its distribution A three-dimensional higher-order local autocorrelation feature extraction method has been proposed (see Non-Patent Document 1) for discriminating abnormal movements that greatly deviate from
JP-A-5-307695 Takuya Nanzato, 1 other, "Detection of abnormal motion from multiple moving images", IPSJ Research Report, 2004-CVIM-145, pp 179-186, September 11, 2004

  However, in the example of Patent Document 1, due to the influence of changes in the external environment such as sunshine changes and weather changes, the road surface shadows, road surface markings, road surface reflections, or tree shakes are erroneously measured as vehicles. There are problems and no radical solution has been made. Further, in the example of Non-Patent Document 1, although the presence or absence of normal operation or abnormal operation can be identified in the captured image, in which position on the captured image the object that performs the normal operation exists, In addition, it cannot be identified at which position on the captured image the target object that performs the abnormal operation exists. Therefore, even if the method of Non-Patent Document 1 is applied to traffic flow monitoring or the like, it is not possible to determine where a moving body such as a vehicle or a person exists.

  The present invention has been made in view of such circumstances, and sets a pixel region configured by arranging a plurality of pixel blocks in each captured image of a plurality of captured images at different capturing time points in time series, and the set pixels A feature quantity vector is extracted based on the correlation between the target pixel in the region and the pixel specified by the displacement information from the target pixel. The extracted feature vector is added to each pixel of interest in the pixel area, and the added feature vector is converted into a type determination value using a predetermined conversion coefficient to determine the type of moving object, thereby affecting the influence of external environment changes. It is an object of the present invention to provide a moving body determination device capable of accurately determining the position and type of a moving body without receiving a computer, a computer program for realizing the moving body determination device by a computer, and a moving body determination method. To do.

  A mobile body determination device according to a first aspect of the present invention is a mobile body determination device that determines a type of mobile body based on a time-series captured image obtained by imaging a region including a road. A setting unit that sets a pixel area configured by arranging a plurality of pixel blocks in each captured image of the image in time series, a target pixel in the pixel area set by the setting unit, and displacement information from the target pixel Extracting means for extracting a feature quantity vector based on the correlation with the pixel to be added, adding means for adding the feature quantity vector extracted by the extracting means for each target pixel in the pixel area, and features added by the adding means Conversion means for converting the quantity vector into a type determination value with a predetermined conversion coefficient, and configured to determine the type of the moving body based on the type determination value converted by the conversion means.

  In the mobile body determination device according to a second aspect of the present invention, in the first aspect, the setting means sets the size of the pixel block in the pixel area to be smaller / larger in an area corresponding to a long distance / short distance of the captured image. It is comprised by these.

  In the mobile object determination device according to the third invention, in the first invention or the second invention, the pixel interval between the pixel of interest and the pixel specified by the displacement information is small in a region corresponding to a long distance / short distance of the captured image. / It is characterized by being large.

  In the mobile object determination device according to a fourth aspect of the present invention, in any one of the first to third aspects, the setting means arranges the pixel areas in time series in an area corresponding to a long distance / short distance of the captured image. The present invention is characterized in that the number of pixel blocks is set to be large / small.

  According to a fifth aspect of the present invention, in any one of the first to fourth aspects, the time interval between the pixel block in which the pixel of interest is present and the pixel block in which the pixel specified by the displacement information is present is imaged. It is characterized by being large / small in an area corresponding to a long distance / short distance of an image.

  The moving body determination device according to a sixth aspect of the present invention is the normalization according to any one of the first to fifth aspects, wherein the feature vector added by the adding means is normalized by dividing by the number of pixels of interest in the pixel region. And the conversion means is configured to convert the feature quantity vector normalized by the normalization means.

  A moving body determination apparatus according to a seventh aspect of the present invention is the variation amount of any one of the first to sixth aspects, wherein the variation amount of each element of the feature amount vector added by the adding means is calculated over a plurality of imaging time points. And a calculating unit, wherein the extracting unit is configured to extract a feature vector by excluding an element corresponding to the variation when the variation calculated by the variation calculating unit is equal to or less than a predetermined threshold. It is characterized by being.

  A computer program according to an eighth invention is a computer program for causing a computer to determine a type of a moving body based on a time-series captured image obtained by imaging a region including a road. Specified by means for setting a pixel area configured by arranging a plurality of pixel blocks in each captured image of a plurality of different captured images in time series, and a target pixel in the set pixel area and displacement information from the target pixel Means for extracting a feature vector based on correlation with a pixel; means for adding a feature vector extracted for each pixel of interest in the pixel area; and a type determination value for the added feature vector with a predetermined conversion coefficient And a means for determining the type of the moving body based on the converted type determination value.

  A moving body determination method according to a ninth aspect of the present invention is the moving body determination method for determining the type of moving body based on a time-series captured image obtained by capturing an area including a road. Based on the correlation between the target pixel in the set pixel area and the pixel specified by the displacement information from the target pixel by setting a plurality of pixel blocks in each captured image of the image arranged in time series To extract the feature amount vector, add the feature amount vectors extracted for each pixel of interest in the pixel region, convert the added feature amount vector into a type determination value with a predetermined conversion coefficient, The type of the moving body is determined based on the above.

  In the first invention, the eighth invention, and the ninth invention, a pixel area configured by setting a plurality of pixel blocks in each captured image of a plurality of captured images at different imaging time points in time series is set, and the set pixel The position of the target pixel is set in the area. The set pixel region is a three-dimensional region in the two directions and the time direction on the captured image. A feature vector is extracted based on the correlation between the target pixel in the set pixel region and the pixel (specific pixel) specified by the displacement information from the target pixel. The pixel of interest is moved within the pixel area, and the feature vector extracted for each pixel of interest at a different position is added. The displacement information is the displacement direction of the specific pixel viewed from the target pixel. For example, the displacement direction can be 26 directions (26 ways) of a 3 × 3 × 3 pixel region centered on the target pixel. When obtaining the correlation feature quantity, the feature quantity is generated by the number of combinations in the displacement direction. For example, when the number of combinations is 251, the feature vector is a 251 dimensional vector. This increases the amount of features that can be extracted compared to conventional methods (for example, edge detection), and also enables the extraction of motion features due to changes over time. Extract well.

  The added feature vector is converted into a type determination value using a predetermined conversion coefficient. The conversion coefficient can be determined in advance. For example, the conversion coefficient is the learning data (for example, several hundred to several thousand frames, the frame rate is 30 frames per second) in which the type of the moving body (with vehicles, with people, without vehicles / people, etc.) is known. The feature quantity vector can be extracted by using multivariate analysis techniques such as regression analysis, principal component analysis, and discriminant analysis. The type of the moving object is determined based on the converted type determination value. Thereby, in the pixel block in the picked-up image at the time of image pick-up used as a reference of the pixel region, it is possible to determine the type of moving body such as the presence of a vehicle, the presence of a person, and the absence of a vehicle / person. Further, by repeating the same processing by allowing the entire pixel image to overlap with the pixel block in the pixel area, the position and type of the moving object on the captured image can be accurately determined.

  In the second invention, the size of the pixel block in the pixel region is set to be small / large in the region corresponding to the long / short distance of the captured image. When the moving body exists nearby, the apparent size on the captured image increases, so the size of the pixel block is increased. As a result, the size of the pixel region in the two directions on the captured image can be increased to extract a wide range of feature values, and the type of the moving object can be determined with high accuracy. In addition, when the moving body is located far away, the apparent size on the captured image is reduced, so the size of the pixel block is reduced. Accordingly, it is possible to reduce the processing effort by deleting unnecessary processing while accurately determining the type of the moving body by reducing the size of the pixel region in the two directions on the captured image.

  In the third invention, the pixel interval between the pixel of interest and the pixel specified by the displacement information is reduced / increased in a region corresponding to the long distance / short distance of the captured image. When the moving body is present in the vicinity, the apparent size on the captured image becomes large, so the pixel interval is increased to increase the length in the displacement direction. As a result, a wide range of feature quantities can be extracted, and the type of mobile object can be determined with high accuracy. In addition, since the apparent size on the captured image is reduced when the moving object is located far away, the pixel interval is reduced to shorten the length in the displacement direction. Thereby, according to the apparent magnitude | size of a moving body, the classification of a moving body can be determined with sufficient precision.

  In the fourth aspect of the invention, the number of pixel blocks arranged in time series of the pixel area in the area corresponding to the long distance / short distance of the captured image is set to be large / small. When the moving body exists nearby, the apparent moving speed on the captured image is fast, and the moving amount of the moving body is large even in a short time, so the number of pixel blocks is reduced. Thereby, the length of the pixel region in the time direction can be shortened, and the type of the moving body can be accurately determined according to the moving speed of the moving body. In addition, when the moving body exists far away, the apparent moving speed on the captured image is slow, and the moving amount of the moving body does not increase unless the time is long, so the number of pixel blocks is increased. Thereby, a feature-value can be extracted over a long time, and the classification of a mobile body can be determined with sufficient precision.

  In the fifth invention, the time interval between the pixel block in which the pixel of interest exists and the pixel block in which the pixel specified by the displacement information exists is increased / decreased in an area corresponding to the long / short distance of the captured image. To do. When the moving object is present in the vicinity, the apparent moving speed on the captured image is high, and the moving amount of the moving object is large even in a short time. Therefore, the time interval between the pixel blocks arranged in time series is shortened. As a result, the type of the moving body can be accurately determined according to the moving speed of the moving body. In addition, when the moving object is located far away, the apparent moving speed on the captured image is slow, and the moving amount of the moving object does not increase unless the time is long, so the time interval between the pixel blocks arranged in time series is increased. To do. Thereby, a feature-value can be extracted over a long time, and the classification of a mobile body can be determined with sufficient precision.

  In the sixth aspect of the invention, the added feature vector is normalized by dividing it by the number of target pixels in the pixel area (for example, the total number of target pixels). A type determination value is obtained by converting the normalized feature vector with a conversion coefficient. Thereby, the correlation feature amount independent of the size of the pixel region in the two directions or the time direction can be obtained.

  In the seventh invention, the fluctuation amount of each element of the added feature vector is calculated over a plurality of imaging time points. For example, using learning data in which the type of a moving object (with a vehicle, with a person, with a vehicle / without a person, etc.) is known in advance, a feature vector at a plurality of imaging points (for example, several hundred to several thousand frames) (A feature amount vector obtained by adding the feature amount vectors extracted for each pixel of interest in the pixel area) is extracted, and a variation amount of each element (251 elements in the case of a 251 dimensional vector) of the extracted feature amount vector is calculated. When the fluctuation amount is equal to or less than the predetermined threshold, that is, the element whose feature amount does not change regardless of the type of the moving body is removed. When determining the type of the moving body, a feature amount vector (partial feature amount vector) from which an element whose feature amount hardly changes regardless of the type of the moving body is removed is extracted. As a result, there is no need to obtain unnecessary feature amounts for the mobile body type determination, the processing amount can be reduced while maintaining the accuracy of the mobile body type determination, and the mobile body type determination is less expensive. Can be realized.

  In the present invention, it is possible to accurately determine the position and type of the moving body without being affected by changes in the external environment.

  Hereinafter, the present invention will be described with reference to the drawings illustrating embodiments. FIG. 1 is a schematic diagram showing an outline of mobile body determination using the mobile body determination apparatus according to the present invention. As shown in FIG. 1, the video camera 1 is installed at a position about 10 m on the road near the stop line P before the intersection. The video camera 1 images a range of about several tens of meters of an inflow path or an outflow path to an intersection as a determination area for determining the type of a moving body (for example, with a vehicle, with a person, without a vehicle / with a person).

  A moving body determination device 2 according to the present invention is connected to the video camera 1 via a required video cable. The moving body determination device 2 processes time-series captured images (moving images) obtained by capturing with the video camera 1 and determines the type of the moving body. The mobile body determination device 2 can communicate with the center device 3 installed in the traffic control center by wired or wireless communication. The mobile body determination device 2 outputs the determination result of the mobile body type to the center device 3. Note that the video camera 1 and the moving body determination device 2 may be integrated.

  FIG. 2 is a block diagram showing a configuration of the moving body determination device 2. In the figure, reference numeral 1 denotes a video camera. The video camera 1 outputs a captured image obtained by imaging to the image input unit 21 as a video signal (analog signal). The image input unit 21 outputs the acquired video signal to the A / D conversion unit 22.

  The A / D converter 22 converts the input video signal into a digital signal, and stores the converted digital signal in the image memory 23 as image data. The CPU 25 uses the captured image input from the video camera 1 via the image input unit 21 as image data in synchronization with the frame rate of the video camera 1 (interval at the time of imaging, for example, 30 frames per second). The data is stored in the image memory 23 in units of frames (for example, 480 × 640 pixels).

  The auxiliary storage unit 28 stores the computer program PG recorded on the CD-ROM 29 in the RAM 24 by inserting the CD-ROM 29 on which the computer program PG of the present invention is recorded. The medium for recording the computer program PG is not limited to the CD-ROM 29, but may be another recording medium, or can be downloaded from the Internet. The CPU 25 executes the computer program PG stored in the RAM 24. The moving body determination device 2 reads the computer program PG recorded in the CD-ROM 29 into the RAM 24 and executes the read computer program PG by the CPU 25 to perform the moving body type determination processing. Note that the mobile type determination process can be realized by a dedicated hardware circuit instead of the computer program.

  The image memory 23 stores the captured image acquired via the image input unit 21 as image data for each frame.

  The storage unit 27 stores the obtained feature vector extraction result and the moving body type determination processing result corresponding to each pixel of each frame by executing the computer program PG by the CPU 25. Further, the storage unit 27 stores a conversion coefficient calculated using the learning data. In addition, the storage unit 27 stores information related to elements that do not require extraction of the correlation feature quantity among the elements of the feature quantity vector.

  The communication unit 26 outputs information related to the position and type of the moving object determined by the moving object determination device 2 to the center device 3.

  The CPU 25 sets a plurality of pixel blocks having a required size on the captured image. The positions of the pixel blocks on the captured image can be arranged so that part of the pixel blocks overlap each other, can be arranged apart from each other, or there are mixed pixel blocks separated from the overlapping pixel blocks. You can also. The pixel block setting can be determined in advance instead of the CPU 25. The CPU 25 sets a pixel area configured by arranging a plurality of pixel blocks of a plurality of captured images at different imaging time points in time series, and sets the position of the target pixel in the set pixel area. The set pixel region is a three-dimensional region in the two directions and the time direction on the captured image.

  The CPU 25 extracts a feature vector based on the correlation between the target pixel in the set pixel region and the pixel (specific pixel) specified by the displacement information from the target pixel. The pixel of interest is moved within the pixel area, and the feature vector extracted for each pixel of interest at a different position is added. That is, the individual feature vectors extracted for each pixel of interest are added for all the pixels in the pixel region and the imaging time point. The displacement information is the displacement direction of a specific pixel in the pixel region viewed from the pixel of interest. For example, the displacement direction is 26 directions (26 ways) of a 3 × 3 × 3 pixel region centered on the pixel of interest. Can do. When extracting feature quantity vectors in the pixel area, feature quantities are generated as many as the number of combinations in the displacement direction. For example, when the number of combinations is 251, the feature vector is a 251 dimensional vector.

  The CPU 25 converts the added feature vector into a type determination value using a predetermined conversion coefficient. The CPU 25 determines the type of the moving body based on the converted type determination value. The conversion coefficient can be determined in advance. For example, the conversion coefficient is obtained by using learning data (for example, several hundred to several thousand frames, 30 frames per second) in which the type of moving body (with vehicles, with people, without vehicles / people, etc.) is known. Feature quantity vectors can be extracted and obtained by multivariate analysis techniques such as regression analysis, principal component analysis, and discriminant analysis. Thereby, in the pixel block in the picked-up image at the time of image pick-up used as a reference of the pixel region, it is possible to determine the type of moving body such as the presence of a vehicle, the presence of a person, and the absence of a vehicle / person. By repeating the same processing by allowing the pixel blocks in the pixel area to overlap and scanning the entire captured image, the position and type of the moving object on the captured image can be accurately determined.

  FIG. 3 is an explanatory diagram illustrating an example of a captured image. As shown in FIG. 3, the area corresponding to the short distance on the captured image is set as block A, the area corresponding to the long distance is set as block C, and the area corresponding to the middle distance in the middle is divided into three as block B. . As shown in FIG. 3, the block A is set to be larger than the block B, and the block B is set to be larger than the block C. As will be described later, the size of the pixel region is changed for each divided block. A plurality of pixel blocks are set for each of the pixel blocks A, B, and C. The positions of the pixel blocks on the captured image can be arranged so that part of the pixel blocks overlap each other, can be arranged apart from each other, or there are mixed pixel blocks separated from the overlapping pixel blocks. You can also.

  The number of divisions of the area corresponding to the distance of the captured image is not limited to 3, and can be divided into 4 or more. Further, the dividing line for dividing the captured image is not limited to the horizontal direction (horizontal scanning line direction), and can be appropriately set such as a vertical direction and an oblique direction according to the direction of the road to be captured and the road condition. .

  FIG. 4 is an explanatory diagram showing an example of a pixel region in each block. The pixel region is configured by arranging a plurality of pixel blocks of a plurality of captured images at different imaging time points in time series, and is a three-dimensional region in two directions and a time direction on the captured image. The size of the pixel region is set by the number of vertical pixels × the number of horizontal pixels (pixel block) × the number of frames. In addition, the pixel interval is, for example, a pixel interval between a target pixel in the pixel region and a specific pixel specified by displacement information (displacement direction) from the target pixel, and represents the length of the displacement on the captured image. Show. The frame interval is, for example, a frame interval between a frame in which a pixel of interest in the pixel area exists and a frame in which a specific pixel specified by displacement information (displacement direction) from the pixel of interest exists, and in the time direction The length of displacement at.

  As shown in FIG. 4, in block A (area corresponding to the short distance of the captured image), the size of the pixel area is 9 pixels × 9 pixels (9 × 9 pixel block) × 5 frames, The pixel interval between the target pixel and the specific pixel is 3 pixels, and the frame interval is 1 frame.

  FIG. 5 is an explanatory diagram showing a configuration example of the pixel area in the block A. As shown in FIG. 5, the pixel region is configured by arranging 9 pixel × 9 pixel pixel blocks in five captured images (5 frames) at time t,. Note that the time interval of each frame is 1/30 seconds if the frame rate is 30 frames / second.

  Further, as shown in FIG. 4, in the block B (area corresponding to the middle distance of the captured image), the size of the pixel area is 7 pixels × 7 pixels (7 × 7 pixel block) × 7 frames, The pixel interval between the target pixel and the specific pixel in the region is two pixels, and the frame interval is two frames.

  FIG. 6 is an explanatory diagram illustrating a configuration example of the pixel region in the block B. As shown in FIG. 6, the pixel area is configured by arranging 7 pixel × 7 pixel pixel blocks in 7 captured images (7 frames) at time t,..., T-6 for 7 frames in time series.

  As shown in FIG. 4, in block C (area corresponding to the long distance of the captured image), the size of the pixel area is 3 pixels × 3 pixels (3 × 3 pixel block) × 9 frames, The pixel interval between the target pixel and the specific pixel in the region is 1 pixel, and the frame interval is 3 frames.

  FIG. 7 is an explanatory diagram illustrating a configuration example of a pixel region in the block C. As shown in FIG. 7, the pixel area is configured by arranging 9 pixel blocks of 3 pixels × 3 pixels in nine captured images (9 frames) at time t,.

  Next, a feature vector extraction method will be described. In the following description, the pixel area in the block B will be described. However, the other block A and C have the same feature vector extraction method except that the size of the pixel area, the pixel interval, and the frame interval are different. Further, a pixel block of 5 pixels × 5 pixels in the pixel area is illustrated as an example, and the pixel interval is 2 pixels and the frame interval is 2 frames.

  FIG. 8 is an explanatory diagram showing an example of feature quantity vector extraction. As shown in FIG. 8, the frame interval is set to 2 frames (that is, every other frame), and 5 × 5 pixel blocks are arranged in time series for 3 frames (time t, t−2, t−4). It is configured. Further, the target pixel is at the center of the pixel block of the frame at time t−2.

  The CPU 25 extracts a feature vector Vij based on the formula (1) for each target pixel on the captured image.

  Here, i and j indicate the position of the target pixel on the captured image. R indicates the position of the target pixel in the pixel area, and r = (x, y, z), z = t, t−2, and t−4. Further, a1, a2,..., AN are displacement directions (displacement information) viewed from the target pixel, and N represents the number of displacement directions. r + a1, r + a2,..., r + aN represent the positions of the target pixel and the specific pixel for which the correlation feature amount is calculated. Further, f (r), f (r + a1),... Represent the luminance value of each pixel. Note that f (r), f (r + a1),... Can be values obtained by binarizing the time difference or the background difference result instead of the luminance value.

  The number N of the displacement directions can be, for example, 26 directions (26 ways) of a 3 × 3 × 3 pixel region centered on the target pixel. When extracting feature quantity vectors in the pixel area, feature quantities are generated for the number of combinations in the displacement direction corresponding to the number of selected pixels to be selected for obtaining the correlation feature quantity. For example, when binarized values are used as f (r), f (r + a1),..., When the number of selected pixels for obtaining the correlation feature amount is 1 to 3, when the number of selected pixels is 1, Since one target pixel is selected, the number of combinations is one. When the number of selected pixels is 2, the number of combinations of the target pixel and another specific pixel is 13. Further, when the number of selected pixels is 3, the number of combinations of the target pixel and the other two specific pixels is 237. Note that overlapping combinations are excluded by moving the pixel of interest within the pixel region. Thereby, the number of combinations becomes 251 and the feature vector becomes a 251 dimensional vector.

  That is, when binarized values are used as f (r), f (r + a1),..., There are 251 combinations that multiply f (r), f (r + a1),. become. Note that by binarizing the captured image in advance, the value obtained by multiplying f (r), f (r + a1),... Takes either 1 or 0. Further, the process of integrating with dr in equation (1) corresponds to moving the pixel of interest within the pixel area and obtaining a feature vector each time it is moved. Therefore, each element of the feature vector (251 element in the case of a 251 dimensional vector) is added every time the pixel of interest moves, and the feature vector after the addition process can be extracted.

  The feature amount vector after the addition process can be normalized by dividing by the number of target pixels (for example, the total number of target pixels) in the pixel region. Thereby, the extracted feature vector can be obtained as a value that does not depend on the magnitude of the two directions or the time direction on the captured image of the pixel region.

  Next, an example of calculating each element of the feature vector will be described. FIG. 9 is an explanatory diagram illustrating an example of calculating a feature vector when the number of selected pixels is two. As shown in FIG. 9, the target pixel r exists at the center of the pixel block of the frame at time t-2, and its coordinates are r (x, y, t-2). Further, the displacement direction a1 is set to a1 (−2, 0, 0). In this case, the length in the displacement direction, that is, the pixel interval when calculating the correlation feature amount is two pixels. The coordinate r + a1 of the specific pixel is r + a1 (x−2, y, t−2), and the element v1 of the feature vector is v1 = f (r) · f (r + a1).

  FIG. 10 is an explanatory diagram illustrating an example of calculating a feature vector when the number of selected pixels is three. As shown in FIG. 10, the target pixel r exists at the center of the pixel block of the frame at time t-2, and its coordinates are r (x, y, t-2). The displacement direction a1 is a1 (−2, 0, 0), and the displacement direction a2 is a2 (0, 2, 0). In this case as well, the length in the displacement direction, that is, the pixel interval when calculating the correlation feature amount is two pixels. The coordinate r + a1 of the specific pixel is r + a1 (x-2, y, t-2), and the coordinate r + a2 of the specific pixel is r + a2 (x, y + 2, t-2). The element v2 of the feature vector is v2 = f (r) · f (r + a1) · f (r + a2).

  FIG. 11 is an explanatory diagram illustrating a calculation example of the feature amount vector when displaced in the time direction. As shown in FIG. 11, the target pixel r exists at the center of the pixel block of the frame at time t-2. The displacement direction am is the direction of the specific pixel of the frame at time t as viewed from the target pixel, and the coordinate of the specific pixel is r + am. The displacement direction an is the direction of the specific pixel of the frame at time t-4 when viewed from the target pixel, and the coordinate of the specific pixel is r + an. In this case, the length in the displacement direction, that is, the frame interval for calculating the correlation feature amount is two frames. The element vk of the feature vector is vk = f (r) · f (r + am) · f (r + an).

  FIG. 12 is an explanatory diagram illustrating an example of a feature vector of a target pixel. As shown in FIG. 12, the feature vector Vij of the pixel of interest rij on the captured image can be extracted as Vij (v1, v2,..., V251) by performing the above-described processing. By scanning the entire pixel area and repeating the same processing, it is possible to extract a feature vector of each pixel in the pixel area.

  The feature amount vector Vij extracted for each pixel in the pixel region can be multiplied by a conversion coefficient to be converted into a moving object determination value. Expressions (2) and (3) are examples of conversion expressions.

  In Expression (2), A is a conversion coefficient, and when the feature quantity vector is a 251-dimensional vector, the conversion coefficient A can also be a 251-dimensional vector, and the mobile object determination value (type) as the conversion result The judgment value R can be a real number between 0 and 1, for example.

  For example, the conversion coefficient A is obtained by extracting a feature vector using learning data (for example, several hundred to several thousand frames) in which the type of the moving object (for example, there is a vehicle and no moving object) is known. It can be obtained by multivariate analysis techniques such as analysis, principal component analysis, and discriminant analysis.

  Also, in equation (3), B is a transform coefficient, and when the feature quantity vector is a 251 dimensional vector, the transform coefficient B can also be a 251 dimensional vector, and the mobile object determination value that is the transform result (Type determination value) S can be a real number between 0 and 1, for example.

  For example, the transformation coefficient B is obtained by extracting a feature vector using learning data (for example, several hundred to several thousand frames) in which the type of the moving object (for example, there is a person and no moving object) is known. It can be obtained by multivariate analysis techniques such as analysis, principal component analysis, and discriminant analysis.

  FIG. 13 is an explanatory diagram showing an example of determining the type of moving object. When the moving object determination value R obtained by the equation (2) is close to “1”, it can be determined that there is a vehicle. When the moving object determination value R is close to “0”, it is determined that there is no moving object. be able to.

  Similarly, when the moving object determination value S obtained by the equation (3) is close to “1”, it can be determined that there is a person. When the moving object determination value S is close to “0”, the moving object determination value S It can be determined that there is no body.

  As a result, in the pixel block in the captured image (time t in the above case) at the time of imaging serving as a reference for the pixel area, the type of the moving body such as the presence of a vehicle, the presence of a person, and the absence of a vehicle / person (moving body) be able to. By repeating the same processing by allowing the pixel blocks in the pixel area to overlap and scanning the entire captured image, the position and type of the moving object on the captured image can be accurately determined.

  The calculation example of the moving object determination value is an example, and is not limited to the above example. For example, the moving body determination value is obtained as a two-dimensional value by making the conversion coefficients A and B not a 251-dimensional vector but a 2 × 251 matrix (for example, the conversion matrix C), and the position on the two-dimensional coordinates. Depending on the situation, it may be determined that there is a vehicle, there is a person, and there is no moving body. Even in this case, the transformation matrix C is characterized by using learning data (for example, several hundred to several thousand frames) in which the type of the moving object (for example, with a vehicle, with a person, without a moving object) is known. A quantity vector can be extracted and obtained by a multivariate analysis technique such as regression analysis, principal component analysis, and discriminant analysis.

  FIG. 14 is an explanatory diagram illustrating an example of extracting feature quantity vectors in each frame. The pixel area will be described using the pixel area in the block B. Further, a pixel block of 5 pixels × 5 pixels in the pixel area is illustrated as an example, and the pixel interval is 2 pixels and the frame interval is 2 frames. As shown in FIG. 14A, when a frame at time t is input, a region for extracting a feature vector is a pixel block (5 pixels) of three frames at times t, t-2, and t-4. × 5 pixels). Extract a feature vector for the target pixel in the pixel area, move the target pixel in the pixel area, and add each element of the feature vector each time it is moved. Conversion to a moving body determination value is performed to determine the type of the moving body. After performing the same processing for all pixels on the captured image, a frame at time t + 1 is input as shown in FIG.

  At time t + 1, the region from which the feature vector is extracted is composed of pixel blocks (5 pixels × 5 pixels) of three frames at times t + 1, t−1, and t−3. Extract a feature vector for the target pixel in the pixel area, move the target pixel in the pixel area, and add each element of the feature vector each time it is moved. Conversion to a moving body determination value is performed to determine the type of the moving body. The same processing is performed for all the pixels on the captured image, and the next frame is input in the same manner, and the same processing is repeated.

  FIG. 15 is a flowchart showing a processing procedure for calculating a conversion coefficient. The CPU 25 acquires a captured image for learning (S11) and sets a pixel area (S12). In this case, the pixel area is set as shown in FIG. 4 according to which of the blocks A, B, and C the position of the pixel block on the captured image is. The CPU 25 sets the position of the pixel of interest within the pixel area (S13). Note that the scanning order of the target pixel can be set as appropriate, but as an example, scanning can be started from the upper left position of the pixel block to the lower right position. Further, the scanning order of the pixel of interest in the time direction can be set as appropriate. For example, scanning can be performed from a frame with an old time to a frame with a new time.

  The CPU 25 extracts the feature amount vector (after addition) by adding the feature amount vector extracted for the target pixel in the pixel region to all the target pixels in the pixel region (S14), and extracts the extracted feature amount vector. Each element is divided by the total number of pixels of interest in the pixel area and normalized (S15).

  The CPU 25 determines whether or not the processing for all the pixels of the captured image has been completed (S16). If the processing for all the pixels has not been completed (NO in S16), the processing from step S12 is repeated. When the processing for all the pixels has been completed (YES in S16), the CPU 25 determines whether the processing for all the frames has been completed (S17). If the processing for all the frames has not been completed (NO in S17), The processes after step S11 are repeated.

  When the processing for all the frames has been completed (YES in S17), the CPU 25 calculates the variation amount of each element of the extracted feature vector (S18). Note that the variation amount can be obtained as a difference between the maximum value and the minimum value for each element of the feature amount vector, but is not limited thereto.

  The CPU 25 determines whether or not the variation amount is equal to or less than a predetermined threshold (S19). If the variation amount is equal to or less than the predetermined threshold (YES in S19), the element of the feature amount vector corresponding to the variation amount is determined. It is set as a removal element (S20). That is, an element whose feature amount hardly changes regardless of the type of the moving body is removed. The CPU 25 calculates a conversion coefficient by multivariate analysis based on the extracted feature vector (S21), and ends the process. On the other hand, when the fluctuation amount is not equal to or smaller than the predetermined threshold (NO in S19), the CPU 25 performs the process of step S21. Note that the magnitude of the variation is determined for all elements.

  Using learning data, an element whose feature amount hardly changes regardless of the type of moving object can be specified in advance as a removal element. Thereby, when determining the type of the moving object, it is not necessary to obtain a feature amount unnecessary for determining the type of the moving object, and the processing amount can be reduced while maintaining the accuracy of the type determination of the moving object. It is possible to realize the type determination with a cheaper device.

  FIG. 16 is a flowchart showing a processing procedure for determining the type of the moving object. The CPU 25 acquires a captured image (S31) and sets a pixel area (S32). In this case, the pixel area is set as shown in FIG. 4 according to which of the blocks A, B, and C the position of the pixel block on the captured image is. The CPU 25 sets the position of the target pixel on the captured image (S33). Note that the scanning order of the target pixel can be set as appropriate, but as an example, scanning can be started from the upper left position of the pixel block to the lower right position. Further, the scanning order of the pixel of interest in the time direction can be set as appropriate. For example, scanning can be performed from a frame with an old time to a frame with a new time.

  The CPU 25 determines the presence / absence of a removal element (S34). If there is a removal element (YES in S34), the CPU 25 excludes the removal element and extracts all of the feature amount vector pixel areas extracted for the target pixel in the pixel area. A partial feature vector (after addition) is extracted by adding the pixel of interest (S35). Note that the dimension of the partial feature vector becomes less than 251 depending on the number of elements that can be excluded.

  When there is no removal element (NO in S34), the CPU 25 extracts the feature amount vector (after addition) by adding all the attention pixels in the feature amount vector pixel region extracted with respect to the attention pixel in the pixel region. (S36). The CPU 25 normalizes by dividing each element of the extracted feature vector (or partial feature vector) by the total number of pixels of interest in the pixel area (S37).

  The CPU 25 converts the feature vector into a moving object determination value using a conversion coefficient (S38), and determines the type of the moving object according to the moving object determination value (S39). The CPU 25 determines whether or not the processing of all the pixels of the captured image has been completed (S40). If the processing of all the pixels has not been completed (NO in S40), the processing from step S32 is repeated.

  When processing for all pixels is completed (YES in S40), the CPU 25 determines whether or not processing for all frames is completed (S41). When processing for all frames is not completed (NO in S41), The processes after step S31 are repeated. When the processing for all the frames is completed (YES in S41), the CPU 25 ends the processing.

  FIG. 17 is a schematic diagram showing another example of moving object determination using the moving object determination device according to the present invention. As shown in FIG. 17, the video camera 1 is installed at a position about 10 m on the road at the intersection angle. The video camera 1 captures an image in the intersection as a determination area for determining the type of moving object (for example, with a vehicle, with a person, without a vehicle / with a person).

  A moving body determination device 2 is connected to the video camera 1 via a required video cable. The moving body determination device 2 processes time-series captured images (moving images) obtained by capturing with the video camera 1 and determines the type of the moving body. The mobile body determination device 2 can communicate with the center device 3 installed in the traffic control center by wired or wireless communication. The mobile body determination device 2 outputs the determination result of the mobile body type to the center device 3. Note that the video camera 1 and the moving body determination device 2 may be integrated. Since the configuration and operation of the mobile body determination device 2 are the same as those in the above example, the description thereof is omitted.

  As described above, in the present invention, it is possible to determine the type of a moving body such as a vehicle, a person, and a vehicle / no person in a pixel block in a captured image at the time of imaging serving as a reference for a pixel region. it can. By repeating the same processing by allowing the pixel blocks in the pixel area to overlap and scanning the entire captured image, the position and type of the moving object on the captured image can be accurately determined.

  In addition, the size of the pixel block in the pixel area is set to be small / large in the area corresponding to the long distance / short distance of the captured image, and unnecessary types of processing are deleted while accurately determining the type of the moving object. Processing effort can be reduced. In addition, by reducing / increasing the pixel interval between the pixel of interest and the pixel specified by the displacement information in an area corresponding to the long / short distance of the captured image, a wide range of feature values can be extracted. In addition, it is possible to accurately determine the type of the moving body according to the apparent size of the moving body.

  In addition, by setting the number of pixel blocks arranged in time series in the pixel area in the area corresponding to the long distance / short distance of the captured image, the moving body can be accurately adjusted according to the apparent moving speed of the moving body. Can be determined. Further, by increasing / decreasing the time interval between the pixel block in which the pixel of interest exists and the pixel block in which the pixel specified by the displacement information exists in a region corresponding to the long / short distance of the captured image, The type of the moving body can be accurately determined according to the apparent moving speed.

  Also, by dividing the feature quantity vector by the total number of pixels of interest in the pixel area and normalizing it, it is possible to obtain a feature quantity vector that does not depend on the size in the two or temporal directions on the captured image of the pixel area. In addition, by extracting the partial feature quantity vector, it is not necessary to obtain a feature quantity unnecessary for the mobile body type determination, and the processing amount can be reduced while maintaining the accuracy of the mobile body type determination. It is possible to realize the type determination with a cheaper device.

  In the above-described embodiment, the size of the pixel block in the pixel region, the number of frames, the pixel interval, and the frame interval are merely examples, and are not limited thereto. The position of the target pixel on the captured image can be set as appropriate depending on whether the position corresponds to a long distance or a short distance.

  In the above-described embodiment, an element whose feature amount hardly changes regardless of the type of moving object is specified in advance as a removal element. However, the present invention is not limited to this, and the removal element is not specified in advance. It can also be configured. In this case, the processes in steps S19 and S20 illustrated in FIG. 15 are not necessary, and the processes in steps S34 and S35 illustrated in FIG. 16 are also unnecessary.

  The embodiments and examples disclosed above should be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above embodiments and examples but by the scope of claims, and is intended to include all modifications and variations within the meaning and scope equivalent to the scope of claims. .

It is a schematic diagram which shows the outline | summary of the mobile body determination using the mobile body determination apparatus which concerns on this invention. It is a block diagram which shows the structure of a moving body determination apparatus. It is explanatory drawing which shows an example of a captured image. It is explanatory drawing which shows an example of the pixel area | region in each block. It is explanatory drawing which shows the structural example of the pixel area | region in a block. It is explanatory drawing which shows the structural example of the pixel area | region in a block. It is explanatory drawing which shows the structural example of the pixel area | region in a block. It is explanatory drawing which shows the feature-value vector extraction example. It is explanatory drawing which shows the example of calculation of the feature-value vector in case the number of selection pixels is two. It is explanatory drawing which shows the example of calculation of the feature-value vector in case the number of selection pixels is 3. It is explanatory drawing which shows the example of calculation of the feature-value vector at the time of displacing in a time direction. It is explanatory drawing which shows the example of the feature-value vector of an attention pixel. It is explanatory drawing which shows the example of classification determination of a moving body. It is explanatory drawing which shows the example of extraction of the feature-value vector in each flame | frame. It is a flowchart which shows the process sequence which calculates a conversion coefficient. It is a flowchart which shows the process sequence which determines the classification of a moving body. It is a schematic diagram which shows the other example of the mobile body determination using the mobile body determination apparatus which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Video camera 2 Moving body determination apparatus 21 Image input part 22 A / D conversion part 23 Image memory 24 RAM
25 CPU
26 Communication Unit 27 Storage Unit 28 Auxiliary Storage Unit 29 CD-ROM

Claims (9)

In the mobile body determination device that determines the type of mobile body based on a time-series captured image obtained by imaging a region including a road,
Setting means for setting a pixel region configured by arranging a plurality of pixel blocks in each captured image of a plurality of captured images at different imaging time points in time series;
Extraction means for extracting a feature vector based on the correlation between the target pixel in the pixel region set by the setting means and the pixel specified by the displacement information from the target pixel;
Adding means for adding the feature quantity vector extracted by the extracting means for each target pixel in the pixel region;
Conversion means for converting the feature amount vector added by the addition means into a type determination value with a predetermined conversion coefficient, and
A mobile object determination apparatus configured to determine a type of a mobile object based on a type determination value converted by the conversion means. The setting means includes
The moving body determination apparatus according to claim 1, wherein the size of the pixel block of the pixel area is set to be small / large in an area corresponding to a long distance / short distance of a captured image.   The moving object determination according to claim 1 or 2, wherein a pixel interval between the pixel of interest and a pixel specified by the displacement information is small / large in a region corresponding to a long distance / short distance of a captured image. apparatus. The setting means includes
4. The configuration according to claim 1, wherein the number of pixel blocks arranged in time series of the pixel area in the area corresponding to the long distance / short distance of the captured image is set to be large / small. The moving body determination apparatus in any one of.   2. The time interval between a pixel block in which a pixel of interest exists and a pixel block in which a pixel specified by displacement information exists is large / small in a region corresponding to a long distance / short distance of a captured image. The moving body determination device according to claim 4. Normalizing means for normalizing by dividing the feature vector added by the adding means by the number of pixels of interest in the pixel region;
The converting means includes
6. The moving body determination apparatus according to claim 1, wherein the moving body determination apparatus is configured to convert the feature vector normalized by the normalizing unit. A variation amount calculating means for calculating a variation amount of each element of the feature amount vector added by the adding means over a plurality of imaging time points;
The extraction means includes
The feature amount vector is extracted by excluding an element corresponding to the variation amount when the variation amount calculated by the variation amount calculation means is equal to or less than a predetermined threshold value. The moving body determination apparatus in any one of Claim 6 thru | or 6. In a computer program for causing a computer to determine the type of a moving body based on a time-series captured image obtained by imaging a region including a road,
Means for setting a pixel area configured by arranging a plurality of pixel blocks in each captured image of a plurality of captured images at different imaging time points in time series;
Means for extracting a feature vector based on a correlation between a target pixel in a set pixel region and a pixel specified by displacement information from the target pixel;
Means for adding a feature vector extracted for each pixel of interest in the pixel region;
Means for converting the added feature vector into a type determination value with a predetermined conversion coefficient;
A computer program that functions as means for determining a type of a moving object based on a converted type determination value. In the moving object determination method for determining the type of moving object based on a time-series captured image obtained by imaging an area including a road,
Set a pixel area configured by arranging a plurality of pixel blocks in each captured image of a plurality of captured images at different capturing points in time series,
Extracting a feature vector based on the correlation between the target pixel in the set pixel region and the pixel specified by the displacement information from the target pixel;
Add the feature vector extracted for each pixel of interest in the pixel area,
The added feature vector is converted into a type determination value with a predetermined conversion coefficient,
A moving body determination method, wherein the type of a moving body is determined based on the converted type determination value.

Images