JP2008217330A - Speed estimation method and speed estimation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speed estimation method and a speed estimation program for precisely estimating the speed of an object moving at a high speed only with a camera without using any sensor or marker. <P>SOLUTION: This speed estimation method includes a step for acquiring a pickup image picked up by an interlaced movable camera means; a step for estimating the optical flow of an even-numbered scanning line image and an odd-numbered scanning line image in one frame of the pickup image; a step for processing an estimated flow vector as image data, and for resizing the flow vector into the same size as that of the pickup image; a step for superimposing the pickup image on the image of the resized flow vector; and a step for estimating the speed of a traveling object by using the time lag in scanning the even-numbered scanning line image and odd-numbered scanning line image of one frame and the image pattern of the flow vector. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a speed estimation method and a speed estimation program for a high-speed moving object equipped with camera means, and more particularly to a speed estimation method and a speed estimation program based on an intra-frame optical flow using an interlaced camera. .

  With the recent reduction in size and weight of computers, attention has been focused on wearable computing that always wears a computer and lives. One example of this is a human navigation system that uses this wearable computing technology to guide users in exhibitions and theme parks and provide the information required by the users. In addition, from a platform that can stay at a specific point represented by a helicopter as a technology related to mobile robots that play a role in nursing care robots, disaster relief, safety patrols, etc. in the medical field, and to grasp the details of vehicle dynamics There is also known a technique for acquiring a time-series image and analyzing it using a background difference value, optical flow information, and the like. From such a background, there is an increasing demand for accurately estimating the position and speed of the target (user). Conventionally, in outdoor position estimation, it is common to use a GPS (Global Positioning System) that can be executed in a wide range.

  However, various services using GPS cannot be used in places where radio waves from artificial satellites do not reach such as indoors. In view of this, research has been conducted on a position detection system as an alternative to GPS as means for obtaining position information indoors. As a method used in the position detection system, sensors and markers are placed on indoor ceilings, walls, floors, etc., and the position of the user is obtained by capturing them in an image and analyzing them, or receiving values from the sensors. You can see many things. There are many types of sensors and markers used, and various speed estimation techniques have been studied depending on the arrangement method.

  In the conventional speed estimation technique, the uncertainties in the calculation of the amount of movement when the target object moves at high speed is cited as an issue. This is because, in the case of the conventional speed estimation technique, it is assumed that the target object moves relatively slowly so that the camera image is not distorted. That is, when the target object with the camera moves at a relatively low speed, if there is a corresponding part between frames of the camera image, the speed and position can be estimated. When moving at a high speed, it is impossible to detect how much the camera image has been moved between frames and the speed and position cannot be estimated. Further, when the target object on which the camera is mounted moves at high speed, the camera image is distorted, so that it is difficult to detect the marker, and the calculation of the speed and position becomes inaccurate.

  In addition, there is known a method for detecting the moving speed of a subject by a simple calculation process using data of one frame without using a difference in subject image data between frames (Patent Document 1, Patent Document 1). 2). In this method, two-dimensional discrete wavelet transform is performed on non-interlaced image data formed from interlaced images of even-numbered scanning lines and odd-numbered scanning lines, and speed is determined from the obtained coefficient values. However, these techniques do not estimate the moving speed of an object or estimate the moving amount or position.

  Conventionally, as a technique for estimating the speed of a moving object without using a marker, an optical flow technique that is a speed field of each point on a screen, which is obtained from two consecutive images, is known. This optical flow is a velocity vector that indicates in what direction and how much a certain point or figure in the image moves at the next moment, and estimates the optical flow from the image captured by the camera means. Thus, the moving direction of the camera or the target object can be estimated.

  However, when an optical flow is obtained using two consecutive frames of images, there is a problem that accurate flow vector estimation cannot be performed when the camera moves faster than a certain speed. One reason is that the captured image is distorted when the camera is moved at high speed. This is due to image distortion caused by non-interlaced scanning of the camera means. Another reason is that there is no common part in the image in the local area between successive frames. In other words, it can be estimated if the same object is in the local area between consecutive frames, but it cannot be estimated if the object moves in such a high speed that there is no identical object in the local area between consecutive frames. It is. How fast the same object moves out of the local area when it moves depends on the scan time of the image, the size of the local area, and the size of the object. In a general environment, it is considered that the scan time between two frames takes 100 ms or more. When the camera is moved at high speed at such a speed that there is no common part in the image in the local region between two consecutive frames for 100 ms, accurate optical flow is not estimated.

JP 2004-126923 A JP 2004-159912 A

  As described above, as a feature of speed estimation using an optical flow obtained from images of continuous frames, a marker is not used, so it is excellent in versatility, but there is a problem that it does not support high-speed movement. is there. In other words, the conventional optical flow technology can cope with the case where the object moves at a relatively low speed that does not distort the camera image. However, if the object moves at a high speed, the camera image is distorted and the marker is moved. Cannot be detected, and there is a problem that the calculation of the speed and the moving amount becomes inaccurate due to the absence of a common part in two consecutive frames of images.

  Moreover, in order to use the technique for estimating the speed using the marker, it is necessary to arrange a large number of them indoors in advance. In an actual living environment, arranging a large number of sensors and markers indoors is not practical because it takes time, and there is also a problem that the landscape is damaged.

  In the conventional method of estimating the speed when the object or the camera body moves at high speed from the camera image, the corresponding part is found in the previous frame and the current frame, and the target motion between the camera and the object can be measured from the amount of change. . However, the current situation is that the measurement when the corresponding part is not found in the previous frame and the current frame is not well established. That is, in the prior art, a versatile speed estimation method that is compatible with high-speed movement and does not use a marker has not been established.

  Accordingly, an object of the present invention is to provide a speed estimation method and a speed estimation program that can accurately perform speed estimation during high-speed movement of a moving object equipped with a camera without requiring a special marker. And

  In order to achieve the above object, a speed estimation method according to a first aspect of the present invention includes a step in which an interlaced camera means is movable and images are captured by the camera means to obtain a captured image, and the imaging Using the step of estimating the optical flow of the even-numbered scan line image and the odd-numbered scan line image in one frame of the image, the time lag of the scan of the even-numbered scan line image and the odd-numbered scan line image of one frame, and the image pattern of the estimated optical flow vector And a step of estimating the speed and behavior of the moving object.

The present invention assumes that the object carrying the camera means moves at high speed, and moves the subject from blurring of the image in one frame by interlace scanning in which one frame of the camera image is scanned in the order of odd scan and even scan. Estimate speed.
When the object carrying the camera means moves at high speed, there are many cases where a common part cannot be found from the images of the previous and subsequent frames. In this case, the speed cannot be estimated because the correspondence between frames is unknown. Therefore, in the present invention, an odd-numbered scanned image and an even-numbered scanned image are acquired from one frame image, and the movement vector of the subject is determined by analyzing the degree of displacement of the subject in these two images.

  In other words, when an object equipped with interlaced camera means moves at high speed, the shift between the odd-numbered scan image and the even-numbered scan image that can be acquired from one frame image is used only from one frame of the camera image. The velocity vector is determined. When an object equipped with camera means moves, the object's speed vector is the speed vector of the subject in the camera image, the direction of which is opposite.

  In the conventional optical flow technique, since it is performed using two consecutive frames of images, it is difficult to cope with high-speed movement as it is. In general, the frame rate between two consecutive frames in a general-purpose computer is assumed to be about 100 ms or more although it depends on the program to be executed. Since the length of the frame rate leads to a speed / position estimation error, it is necessary to increase the length of the frame rate by a method other than the method of improving the performance of the computer. Therefore, interlaced camera means is used. The interlaced camera means scans an image of one frame divided into two scans of odd scan and even scan. The time (time lag) between the odd scan and the even scan is considerably faster than the frame rate between two frames (about 100 ms). Therefore, it is assumed that if the speed is estimated from the odd-numbered scan image and the even-number scan image within one frame, accurate estimation can be performed even when there is no common part of the image between two frames in 100 ms. Of course, if the camera is moved at such a high speed that the image flies during the time lag, it will be the limit of speed estimation, but by using this method, it moves at a much faster speed than the conventional optical flow method. It will be possible to cope with.

  Also, by using the flow vector image pattern, not only when the object equipped with the camera means moves in parallel, but also when the object equipped with the camera means moves up / down / rotation / diagonal directions, etc. The size and direction can be estimated.

  As the camera means, a USB (Universal Serial Bus) camera, a digital camera, a digital video camera, or the like can be used.

Further, the speed estimation method of the present invention is a method in which interlaced camera means is movable, the step of taking an image with the camera means to obtain a picked-up image, the even-numbered scan line image in one frame of the picked-up image, Estimating an optical flow of an odd scan line image, processing the estimated flow vector as image data and resizing the same size as the captured image, superimposing the captured image and the resized image of the flow vector, And a step of estimating the speed and behavior of the moving object using the time lag of the scan of the even scan line image and the odd scan line image of one frame and the image pattern of the flow vector.
The speed and behavior of a moving object are visualized by displaying it as an image on a display terminal of a computer.

Next, a speed estimation method according to a second aspect of the present invention includes a step of capturing a moving subject with an interlaced camera means to acquire a captured image, and even-numbered scan line images and odd-numbered scans in one frame of the captured image. Estimating the optical flow of a line image; estimating the velocity and behavior of a moving object using a scan time lag of an even scan line image and an odd scan line image of one frame and an image pattern of an estimated optical flow vector; , Provided.
Similarly, when visualizing the speed and behavior of a moving object, such as displaying it as an image on a display terminal of a computer, a step of capturing a moving subject with an interlaced camera means to obtain a captured image; A step of estimating an optical flow of an even-numbered scan line image and an odd-numbered scan line image in one frame of the captured image; a step of processing the estimated flow vector as image data and resizing to the same size as the captured image; A step of superimposing the image of the flow vector, and a step of estimating a speed and behavior of a moving object using a scan time lag of an even scan line image and an odd scan line image of one frame and an image pattern of the flow vector. It is characterized by being provided with.

The present invention assumes that the object to be imaged by the camera moves at high speed, and moves the subject from blurring of the image in one frame by interlace scanning in which one frame of the camera image is scanned in the order of odd scan and even scan. Estimate speed.
When a high-speed moving object is photographed with a camera, a common part cannot often be found from images of previous and subsequent frames. In this case, the speed cannot be estimated because the correspondence between frames is unknown. Therefore, in the present invention, an odd-numbered scanned image and an even-numbered scanned image are acquired from one frame image, and the movement vector of the subject is determined by analyzing the degree of displacement of the subject in these two images.

Here, in the step of estimating the optical flow of the speed estimation method according to claim 1 or 2, a predetermined local range is searched on the assumption that the movement of the gradation distribution pattern in the even-numbered scan line image and the odd-numbered scan line image is local. Then, it is preferable to perform matching of the shading pattern and estimate the flow vector.
By searching for a predetermined local range, matching shade patterns, and estimating a flow vector, the amount of calculation is small and the processing time can be shortened.

Further, in the step of capturing the captured image by capturing with the camera means of the speed estimation method according to claim 1 or 2, when the image captured by the camera means is a color image, the color image is converted into a grayscale image. It is preferable to process.
This is to avoid inconvenience of processing time delay caused by performing image processing with a color image.

Further, in the step of processing the flow vector estimated by the speed estimation method of claim 1 or 2 as image data and resizing to the same size as the captured image, resampling processing is performed using nearest neighbor interpolation, and estimation is performed. It is preferable to resize the image of the flow vector.
The nearest neighbor interpolation method is a method of assigning to the output pixel using the value of the nearest pixel, and can reduce the processing time.

  The present invention also provides a speed estimation program for causing a computer to execute each step of the speed estimation method, and a computer-readable recording medium on which the speed estimation program is recorded.

According to the speed estimation method and the speed estimation program of the present invention, when the camera or the subject is moving at high speed, the camera image is distorted, and the background is not clearly recognized or common to two consecutive frames. This solves the problem that the estimated optical flow is inaccurate because there is no portion, and has an effect that the speed of an object moving at high speed can be accurately estimated by using only the camera without using a sensor or a marker.
In the prior art, speed estimation was performed using a plurality of consecutive frames. However, the speed estimation method and speed estimation program of the present invention perform speed estimation using image blurring in one frame. Thus, there is an effect that it is possible to cope with high-speed movement of about 10 times or more.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the scope of the present invention is not limited to the following examples and illustrated examples, and various changes and modifications can be made.

  First, a schematic configuration diagram of a moving object speed estimation computer for realizing the speed estimation method of the present invention will be described with reference to FIG. The speed estimation computer 1 is mainly used for a central processing unit (CPU) 2 and a ROM 3 in which a program for causing the central processing unit to execute the steps of the speed estimation method of the present invention is used. A RAM 4, a keyboard 5 and mouse 6, which are human machine interfaces, a display 7, a camera 8 for photographing a subject, and a hard disk (HD) 9 for recording image data captured by the camera 8 and encoded data after encoding processing. Composed.

Next, the processing flow of the speed estimation method of the present invention will be described with reference to FIG. The speed estimation method according to the present invention includes a step in which an interlaced camera means is movable and images are captured by the camera means to obtain a captured image, and even-numbered scan line images and odd-numbered scans in one frame of the captured image. A step of estimating an optical flow of a line image, a step of processing the estimated flow vector as image data and resizing it to the same size as the captured image, a step of superimposing the captured image and the resized image of the flow vector, and one frame Estimating the speed of the moving object using the scan time lag of the even scan line image and the odd scan line image and the image pattern of the flow vector.
Hereinafter, each step will be described.

(1) An image is captured using interlaced camera means.
First, an interlaced camera unit is movable, and the step of capturing an image by capturing an image with the camera unit is a situation in which an object equipped with the interlaced camera unit is a landscape (for example, a ceiling, a floor, etc.) ) (Capture the image).
The image is captured at a rate of about once every 100 ms, depending on the performance of the speed estimation computer and the program contents. The captured image is, for example, an RGB color image of 480 × 480.
Here, the interlaced camera means will be described with reference to FIG. There are two types of camera scanning methods, a non-interlace method and an interlace method. The camera image is composed of a collection of dots called pixels. In the non-interlace method, scanning is performed line by line from the upper point, whereas in the interlace method, the odd-numbered scan lines are first scanned from the upper point and then the even-numbered lines. The image is displayed by scanning twice with the scanning line (see FIG. 2A). In both of these scanning methods, when a high-speed object is captured or when the camera body moves at high speed, the acquired image is distorted or blurred depending on the scanning condition. This is due to the scan execution time. In the case of the interlace method, as shown in FIG. 2B, the objects of the odd scan line image and the even scan line image are misaligned.

(2) If the captured image is a color image, it is converted to a grayscale image.
If the image captured in (1) is a color image, it is converted to a grayscale image. The luminance of the gray scaled image takes a value from 0 to 255.
Here, when the camera is moved at a certain speed or more, an image with blurring is obtained as shown in FIG. The blur is caused by the time lag between the odd-numbered scan and the even-numbered scan of the interlace camera.

(3) Divide the captured image of one frame into odd-numbered scan and even-numbered scan images.
The step of estimating the optical flow of the even-numbered scan line image and the odd-numbered scan line image in one frame of the captured image is performed by first converting the grayscale image obtained in the above (2) to an odd number as shown in FIG. Dividing into an image obtained by scanning and an image obtained by even scanning. At this time, since the image is divided by skipping one line in the vertical direction of the image, the obtained image size is 240 × 480 when the captured image is 480 × 480.

(4) Optical flow is estimated from two images divided into odd-numbered scan and even-numbered scan images.
The step of estimating the optical flow of the even-numbered scan line image and the odd-numbered scan line image in one frame of the captured image is then performed from the two odd-numbered scan images and even-numbered scan images obtained in (3) above, for example, Lucas- The optical flow from the even scan image to the odd scan image is estimated using the Kanade method. The size of the local area is appropriately set to 4 × 4 or 16 × 16.

(5) Save the estimated flow vector as an image.
In the step of processing the estimated flow vector as image data and resizing it to the same size as the captured image, first, the estimated flow vector is stored as an image. Here, since the optical flow is divided into odd and even 240 × 480 images in (3) above, the image size for storing the flow vectors is 240 × 480.

(6) Resize the image in which the flow vector is stored.
The step of processing the estimated flow vector as image data and resizing it to the same size as the captured image is the next step to draw the flow vector on the image captured in (1) above, so that the image size storing the flow vector is stored. Is resized to 480 × 480. At this time, resampling processing is performed using a nearest neighbor method. The nearest neighbor interpolation method is known as a method of assigning to the output pixel using the value of the nearest pixel.

(7) A flow vector at each point is drawn, and the resized flow vector image for obtaining the flow vector at each frame is shown on the captured image.
In the step of superimposing the captured image and the resized image of the flow vector, the flow vector is estimated at each point of one frame captured in the above (4). These are summed within one frame, and the flow vector in that frame is calculated.
For example, as shown in FIG. 3C, the optical flow at each point of the image can be estimated, and the average value thereof is shown as a thick line in the middle of the image. The thick line can be estimated as an optical flow in one frame of the captured image.

(8) The speed of the moving object is estimated using the scan time lag and the flow vector image pattern of the even scan line image and the odd scan line image of one frame.
The speed of the camera is estimated using the time lag of scanning one frame of even-numbered scan line images and odd-numbered scan line images of the interlaced camera means and the image pattern of the flow vector. As shown in FIG. 4, in the interlaced camera means, when one frame image is divided into two scans of odd scan and even scan, the time between the odd scan and even scan (time lag) Is considerably faster than the frame rate between two frames (about 100 ms). Therefore, if the speed is estimated from the odd-numbered scanned image and the even-numbered scanned image in one frame, accurate estimation is possible even when the speed is so high that there is no common part of the image between two frames in 100 ms. However, if the camera is moved at a high speed so that the image is skipped during the time lag, the speed is limited.

In addition, as shown in FIG. 5, the flow vector image pattern is characterized by the optical flow image pattern when the camera means is translated, zoomed, and rotated. It is possible to capture the movement of an object equipped with camera means.
By performing the above processing flow, the optical flow can be estimated from an image of one frame, and accurate speed estimation can be performed even when the camera moves at high speed.

When estimating the speed of a moving object, the speed estimation method of the present invention includes a step of capturing a moving subject with an interlaced camera means to acquire a captured image, and an even number in one frame of the captured image. A step of estimating an optical flow of a scanning line image and an odd number of scanning line images, a step of processing the estimated flow vector as image data and resizing to the same size as the captured image, and superimposing the captured image and the resized image of the flow vector And a step of estimating the speed of the moving object by using the time lag of the scan of the even-numbered scan line image and the odd-numbered scan line image of one frame and the image pattern of the flow vector.
The description of each step is the same as (1) to (8) above.

  In the following examples, using the speed estimation method and the speed estimation program according to the present invention, it is possible to accurately calculate the speed vector and the amount of movement of a high-speed moving object, which was inaccurate with the conventional method. While explaining.

In Example 1, the usefulness of the speed estimation method of the present invention will be described while showing data confirmed by conducting an evaluation experiment.
First, the system configuration for the evaluation experiment will be described with reference to FIG. FIG. 6 shows a schematic diagram of an evaluation experiment apparatus. A cylindrical tube to which a reproducible background image is attached is arranged at a position as shown in FIG. The cylinder has a circumference of 380 cm and can be moved manually or automatically using a motor or the like. Further, the camera is fixed at a distance of 30 cm upward from the outer peripheral surface of the cylinder, and data is acquired by rotating the cylinder. The camera used in the evaluation experiment is “Product Name: The CARD-7RL” manufactured by RF Corporation. This camera scans images in an interlaced manner.

The background image attached to the cylindrical tube uses a background in which three types of color images as shown in FIG. 7A are arranged in a positional relationship as shown in FIG. 7B. These three types of images are standard images often used in image processing, and are compressed in the JPEG format to a size of 303 pixels × 303 pixels (8.02 cm × 8.02 cm).
Since most of the speed estimation using the optical flow of the conventional method uses a non-interlace camera, in this evaluation experiment, it was decided to make a comparison similar to a scan with non-interlace. The camera used in this evaluation experiment is an interlaced camera, and is divided into an odd-numbered scan image and an even-numbered scan image in the processing steps of the speed estimation method of the present invention. Therefore, in this experiment, a method of using only odd-numbered scan images and estimating optical flow with odd-number scan images between two consecutive frames is compared with the speed estimation method of the present invention as a conventional method.

  A speed estimation accuracy evaluation experiment is performed using the evaluation experiment apparatus described above, and a comparison between the conventional method and the speed estimation method of the present invention is performed. In addition, since it is necessary to perform the comparison under the same conditions, both the conventional method and the velocity estimation method of the present invention are simultaneously performed in one experiment. The experimental procedure is described below.

(Experimental procedure 1) The tube attached with the background of FIG. In the experiment, it is moved in the positive direction of y with respect to the camera. In this case, the positive direction of y is the direction in which the optical flow should come out (see FIG. 8).
(Experimental procedure 2) In each image frame imaged with the camera, the optical flow of each point on the image and the total value thereof are estimated.
(Experimental procedure 3) After one round, in all frames (see FIG. 9), the average value of the total values of frames 1 to x) is handled as data of one experiment. At this time, the detection value of the optical flow is acquired separately for the x-direction component and the y-direction component.
(Experimental procedure 4) In both the conventional optical flow method and the speed estimation method of the present invention, the speed of moving the cylinder and the magnitude (detected value) of the optical flow are plotted on a graph, and the relationship between them is recorded.
(Experimental procedure 5) These operations are repeated 30 times. The frame rate under each condition is recorded.

As the speed of moving the camera (in the case of this experiment, the cylinder with the background attached) increases, the estimated optical flow value increases. Therefore, if the speed and the detected optical flow value are in a proportional relationship, the ideal result It is thought that is obtained.
In addition, since the cylinder with the background is moved in a certain direction (positive direction of y), when it is considered that there is no error, the detected value of the optical flow is only output in the positive direction of y. If a value in the x direction or a value in the negative direction of y appears, an error has occurred in the optical flow estimation, and it is considered that accurate velocity estimation cannot be performed. The experimental results were evaluated focusing on this point.

  Regarding the conventional optical flow method and the speed estimation method of the present invention, as shown in the following (a) to (d), the accuracy of the optical flow depends on the correlation between the moving speed of the camera and the optical flow, the limit speed, and the change in illuminance. Examine the correlation with respect to changes. In addition, as a feature of the optical flow, since there are a large number of points (coordinates) for estimating the optical flow in each frame, any change in accuracy can be seen when the number of points to be estimated is changed. To check.

(A) The correlation between the moving speed of the camera and the magnitude of the optical flow is examined.
(B) The limit speed (speed at which accurate speed estimation cannot be performed) is examined.
(C) Compare the accuracy of speed estimation due to changes in illuminance (1050LUX and 90LUX).
(D) Compare the accuracy of the optical flow when the number of points at which the optical flow is estimated in each frame is changed.

  First, measurement graphs when the illuminance is 1050 LUX are shown in FIGS. Based on these results, the critical speeds calculated for the conventional optical flow method (denoted as the conventional method in the figure) and the speed estimation method of the present invention (denoted as the proposed method in the figure) are summarized in Table 1 below. . 10 to 13 and Table 1, when the optical flow detection value in the x direction under each condition is seen, both the conventional optical flow method and the speed estimation method of the present invention have a value close to 0 when the speed is slower than the limit speed. You can see that However, the reason why the value does not completely become zero may be an error in the experimental process such as slight fluctuation of the experimental apparatus, but the detected value in the x direction is negligibly small compared to the detected value in the y direction.

Next, the correlation in the y direction will be described. For easy understanding, FIG. 14 shows a graph in which only the detected values in the y direction when the number of points is 3486 are picked up. FIG. 14 shows that the conventional optical flow method has a proportional relationship up to the limit speed of 4.38 (cm / sec), whereas the speed estimation method of the present invention has a limit speed of 63.62 (cm / sec). It is recognized that there is a proportional relationship up to (second).
Therefore, accurate speed estimation can be performed up to a speed of 4.38 (cm / second) with the conventional optical flow method, and 63.62 (cm / second) with the speed estimation method of the present invention. It can be understood that the method can cope with high-speed movement about 14.5 times as compared with the conventional optical flow method.

  Further, when the speed is further increased, it is recognized that accurate speed estimation cannot be performed since the conventional optical flow method and the speed estimation method of the present invention have no proportional relationship in the y direction.

  Even in cases where the number of points is other than 3486, in all cases from Table 1 above, the speed estimation method of the present invention can cope with high-speed movement 11 to 20 times as compared with the conventional optical flow method. Can understand

  Next, the accuracy due to the change in the number of points at which the optical flow is estimated in each frame is compared. As can be seen from FIGS. 10 to 13, it is recognized that the detection value of the optical flow varies as the number of points decreases. Depending on the background pattern, there are places where estimation errors are likely to occur, such as places where the same color is continuous. In the speed estimation method of the present invention, the sum of the flow vectors at all estimated points in one frame is treated as an optical flow in that frame, so the estimation error becomes more conspicuous as the number of points is smaller. I believe that Further, since there is almost no change in the frame rate by increasing the number of points for generating flow vectors, it is considered that the number of points for estimating the optical flow should be as large as possible.

  Next, the results when the illuminance is 90 LUX are shown in FIGS. As in the case of the illuminance of 1050 LUX, the limit speeds for the conventional optical flow technique and the speed estimation method of the present invention are summarized in Table 2 below. In the case of the illuminance of 90 LUX, it can be confirmed that both the conventional optical flow method and the speed estimation method of the present invention have performed accurate speed estimation until reaching the limit speed. Further, as can be seen from Table 2, it can be understood that the speed estimation method of the present invention can cope with a movement that is 14 to 20 times faster than the conventional optical flow technique.

  It is also recognized that the variation in accuracy due to the number of points for estimating the optical flow is smaller in the detection value as the number of points is larger, as in the case of illuminance 1050LUX.

  Next, the change in accuracy due to the difference in illuminance was examined. When comparing the case of illuminance 1050LUX (FIGS. 10 to 13) and the case of illuminance 90LUX (FIGS. 15 to 18), graphs having similar relationships are obtained, and therefore there is no change in accuracy due to illuminance. It is done. In addition, there was no significant change in the frame rate due to the difference in illumination.

  From the above, it has been confirmed numerically that speed estimation by the speed estimation method of the present invention can cope with a speed 10 times or more faster than the conventional optical flow technique. Here, FIGS. 19A to 19D show the state of an actual image until reaching the limit speed of the conventional optical flow method and the speed estimation method of the present invention, and the state at the limit speed, respectively. The downward direction with respect to the figure is the direction in which the optical flow should come out originally, and the thick line at the center is the optical flow in each frame. The thick thick line shows the optical flow of the conventional method, and the thin thick line shows the optical flow of the proposed method.

In FIG. 19A, an accurate optical flow is obtained for both the conventional optical flow method and the speed estimation method of the present invention. However, in FIG. 19B, the direction of the optical flow in the conventional optical flow method is wrong, and at this point, the conventional method cannot accurately estimate the speed. However, the speed estimation method of the present invention can perform accurate speed estimation even at a speed as shown in FIG. When the speed is increased to a speed as shown in FIG. 19D, speed estimation is impossible even with the speed estimation method of the present invention.
As can be seen from the images in FIGS. 19A to 19D, the speed estimation method of the present invention can cope with a considerably high speed movement as compared with the conventional optical flow technique. As described above, it can be seen from the experimental results that there is no change in accuracy due to illuminance, and the usefulness of the speed estimation method of the present invention has been confirmed.

The speed estimation method and speed estimation program of the present invention can be used in applications such as speed estimation of a mobile robot and speed estimation of a moving object such as a radio controlled helicopter.
Also, if the size of the target object and the distance to the camera are known, the speed can be estimated even if the object moves fast enough to blur the camera image, and sports players and balls (baseball, soccer, (Tennis, etc.) can also be used for moving speed measurement and automatic speed violation control equipment.
Furthermore, by constructing a system in which the mouse pointer moves in the direction in which the camera is moved as an application that uses the camera instead of the mouse, an operation interface that allows the mouse to be used even in the air can be realized.

Schematic configuration diagram of a computer that estimates the speed of a moving object Illustration of camera scan method Process flow diagram of speed estimation method of the present invention Explanatory drawing of the speed estimation method of the present invention Illustration of optical flow image pattern Schematic diagram of evaluation experiment equipment Illustration of standard image and background pattern for image processing used in evaluation experiment Figure showing the direction of moving the cylinder with attached background Illustration of how to obtain evaluation experiment data Measurement graph when illuminance is 1050LUX (when the number of points is 3486) Measurement graph when illuminance is 1050LUX (when the number of points is 231) Measurement graph when the illuminance is 1050 LUX (when the number of points is 45) Measurement graph when the illuminance is 1050 LUX (when the number of points is 18) A graph showing the correlation only in the y direction when the illuminance is 1050 LUX (when the number of points is 3486) Measurement graph when the illuminance is 90LUX (when the number of points is 3486) Measurement graph when the illuminance is 90LUX (when the number of points is 231) Measurement graph when the illuminance is 90LUX (when the number of points is 45) Measurement graph when illuminance is 90LUX (when the number of points is 18) The actual image until reaching the limit speed of the conventional optical flow method and the speed estimation method of the present invention, and the image at the limit speed

Explanation of symbols

1 Speed estimation computer 2 Central processing unit (CPU)
3 ROM
4 RAM
5 Keyboard 6 Mouse 7 Display 8 Camera 9 Hard disk (HD)

Claims (9)

  Interlace-type camera means is movable, the step of obtaining a captured image by imaging with the camera means, and the optical flow of the even-numbered scan line image and the odd-numbered scan line image in one frame of the captured image are estimated. And a step of estimating the speed and behavior of a moving object using a scan time lag of an even scan line image and an odd scan line image of one frame and an image pattern of an estimated optical flow vector. Speed estimation method.   A step of capturing a moving subject with an interlace camera means to acquire a captured image, a step of estimating an optical flow of an even-numbered scan line image and an odd-numbered scan line image in one frame of the captured image, and an even number of one frame A speed estimation method comprising: a step of estimating a speed and behavior of a moving object using a scan time lag of an scan line image and an odd scan line image and an image pattern of an estimated optical flow vector.   Interlace-type camera means is movable, the step of obtaining a captured image by imaging with the camera means, and the optical flow of the even-numbered scan line image and the odd-numbered scan line image in one frame of the captured image are estimated. A step of processing the estimated flow vector as image data and resizing to the same size as the captured image, a step of superimposing the captured image and the resized image of the flow vector, and an even scan line image of one frame, Estimating the speed and behavior of a moving object using a scan time lag of an odd scan line image and an image pattern of the flow vector.   A step of capturing a moving subject with an interlaced camera means to acquire a captured image, a step of estimating an optical flow of an even-numbered scan line image and an odd-numbered scan line image in one frame of the captured image, and an estimated flow vector Are processed as image data and resized to the same size as the captured image, a step of superimposing the captured image and the resized image of the flow vector, and a scan of an even scan line image and an odd scan line image of one frame. Estimating the speed and behavior of a moving object using a time lag and an image pattern of the flow vector.   In the step of estimating the optical flow, it is assumed that the movement of the light and shade distribution pattern in the even-numbered scan line image and the odd-numbered scan line image is local, and the flow vector is estimated by searching for a predetermined local range and matching the light and shade pattern. The speed estimation method according to claim 1, wherein the speed estimation method is performed.   5. In the step of obtaining an image captured by the camera means, when the image captured by the camera means is a color image, the color image is converted into a gray scale image. The speed estimation method according to any one of the above.   In the step of processing the estimated flow vector as image data and resizing to the same size as the captured image, resampling processing is performed using nearest neighbor interpolation, and the estimated flow vector image is resized. The speed estimation method according to claim 3 or 4.   The speed estimation program for making a computer perform each step of the speed estimation method in any one of Claims 1 thru | or 7.   A computer-readable recording medium on which the speed estimation program according to claim 8 is recorded.