JP2008058221A - Velocity of high-speed moving body, estimation method of position, estimation program, and estimation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately estimate the speed and position of a body or a body having a camera during high-speed movement when the camera is of either of an interlace type and non-interlace type. <P>SOLUTION: This estimation method comprises steps of: continuously imaging a moving subject with a camera means; calculating a velocity vector of the subject in one flame from the distortion or displacement of the shape of the subject in the camera image of the frame; and calculating a moving amount of the subject from the frame rate of the camera image and the velocity vector of the subject every frame and estimating the position. In the step of calculating the velocity vector, the velocity vector is calculated using the displacement of the subject by time lag of scan between an even-number scanning line image and an odd-number scanning line image when the camera means is of the interlace type, and the velocity vector is calculated using the vertical direction and lateral direction of the subject when the camera means is of the non-interlace type. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique, an estimation program, and an estimation system for estimating the speed and position of a high-speed moving object using camera images.

  In recent years, the processing power of computers has improved dramatically, and research on image processing and computer vision has been applied in a wide range of fields. In addition, with the widespread use of cameras mounted on mobile phones and PDAs (Personal Digital Assistance), the development of small, high-performance cameras has been promoted, and cameras have a close relationship with people's daily lives. It has become. As basic elements of image analysis in a camera image, there are a camera, an object, and a light source, and the analysis method varies depending on which of these elements moves.

  In addition, with the recent reduction in size and weight of computers, attention has been focused on wearable computing that always wears a computer and lives. Therefore, a lot of research on human navigation that guides the user's directions in exhibitions and theme parks using this wearable computing technology has been conducted, and the demand for accurately identifying the user's location is increasing. . Conventionally, in outdoor position estimation, it is common to use a GPS (Global Positioning System) that can be executed in a wide range.

  However, various services using GPS cannot be used in places where radio waves from artificial satellites do not reach such as indoors. In view of this, research has been conducted on a position detection system as an alternative to GPS as means for obtaining position information indoors. As a method used in the position detection system, sensors and markers are placed on indoor ceilings, walls, floors, etc., and the position of the user is obtained by capturing them in an image and analyzing them, or receiving values from the sensors. You can see many things. There are many types of sensors and markers used here, and the method of position estimation differs depending on the arrangement method. Furthermore, methods combining both a marker and a sensor have been studied, and various position estimation methods have conventionally existed, but both methods have advantages and disadvantages.

Among them, a method using an infrared sensor, an acceleration sensor, or the like can be cited as a method for estimating the position using a sensor. In the sensor method, it is not necessary to analyze the image captured by the camera as compared with the method using the marker, so that accuracy can be maintained even in a situation where visibility is poor such as darkness.
However, a lot of expensive sensors and receivers for analyzing the values from the sensors have to be arranged in advance in the indoor space, which is not very efficient.

Next, a method for performing position estimation using artificially arranged markers will be described. The artificially arranged markers can be classified into those having ID information for identification and those not having.
In the position estimation method using a non-ID marker, the marker itself does not hold information. Therefore, if the camera moves for a long time or moves at a high speed, errors related to position estimation are accumulated. There is a problem that the accuracy gradually decreases and it becomes impossible to follow the position at the end.

On the other hand, in the position estimation method using the ID marker, since the position information is registered in each marker, information on the state of the user is not required, and the marker itself is captured by the camera, and the registered ID information. It is possible to estimate the position of the user simply by analyzing.
However, in this method, when recognizing ID information, if the image quality is poor, the recognition rate is lowered, so a high-performance camera is required. In addition, since the amount of calculation for analyzing the recognized ID information is large, and it takes a certain amount of time to perform position estimation, this impedes work in real time.
Therefore, conventionally, the position is corrected by combining a sensor and an ID marker.

However, in the conventional method as described above, there is a problem that the calculation of the movement amount when the target object moves at a high speed is uncertain.
This is because, in the case of the conventional method, it is assumed that the target object moves relatively slowly so that the image of the camera is not distorted. That is, when a target object equipped with a camera moves at a relatively low speed, as shown in FIG. 1A, if there is a corresponding portion between frames of the camera image, the speed and position can be estimated. However, when the target object to which the camera is attached moves at high speed, as shown in FIG. 1B, it is impossible to detect how much the camera image has moved because the frame of the camera image has jumped. And the position cannot be estimated. Further, when the target object on which the camera is mounted moves at high speed, the camera image is distorted, so that it is difficult to detect the marker, and the calculation of the speed and position becomes inaccurate.

In addition, there is known a method for detecting the moving speed of a subject by a simple calculation process using data of one frame without using a difference in subject image data between frames (Patent Document 1, Patent Document 1). 2). In this method, two-dimensional discrete wavelet transform is performed on non-interlaced image data formed from interlaced images of even-numbered scanning lines and odd-numbered scanning lines, and speed is determined from the obtained coefficient values.
However, these techniques do not estimate the moving speed of an object or estimate the moving amount or position.

JP 2004-126923 A JP 2004-159912 A

In the conventional method of estimating the speed when the object or the camera body moves at high speed from the camera image, the corresponding part is found in the previous frame and the current frame, and the target motion between the camera and the object can be measured from the amount of change. . However, the current situation is that the measurement when the corresponding part is not found in the previous frame and the current frame is not well established. Therefore, an object of the present invention is to accurately estimate the speed and position when an object moves at high speed. In particular, it is assumed that estimation can be performed without requiring a special marker.
Another object of the present invention is to accurately estimate the speed and position of a moving object equipped with a camera when moving at high speed.
It is another object of the present invention to be able to estimate the speed and position of an object or a moving object equipped with a camera when moving at high speed, regardless of whether the camera is an interlace system or a non-interlace system.

  In order to achieve the above object, a method for estimating the speed and position of a high-speed moving object according to claim 1 of the present invention comprises a step of continuously imaging a moving subject by camera means and a shape of the subject in a one-frame camera image. A step of calculating the velocity vector of the subject in the frame from the distortion or displacement of the image, and a step of calculating the amount of movement of the subject from the frame rate of the camera image and the velocity vector of the subject for each frame and estimating the position It is characterized by that.

The present invention assumes a case where an object to be imaged by a camera moves at high speed, calculates a velocity vector from only one frame of a camera image by using distortion of the image of the camera, and calculates the obtained velocity vector and the previous frame. The amount of movement is calculated using the frame rate, and position estimation is performed.
When a high-speed moving object is photographed with a camera, a common part cannot often be found from images of previous and subsequent frames. In this case, the speed cannot be estimated because the correspondence between frames is unknown. Therefore, in the present invention, only the image of one frame is processed, the degree of distortion is analyzed, the velocity vector of the subject of the camera is determined, and the amount of movement is calculated using the frame rate of the obtained velocity vector and the previous frame. Is calculated and position estimation is performed.

  The present invention can determine the velocity vector of the subject by processing only one frame image and analyzing the degree of distortion even when there is no common part with the previous and next frame images. In addition, there are two types of scan methods, interlace method and non-interlace method, depending on the camera. In the present invention, as described later, a method for estimating a velocity vector that differs for each method is provided.

The movement amount is calculated using the frame rate of the obtained velocity vector and the previous frame. The velocity transition is estimated from the velocity sequence, and the estimated frame rate between the velocity transition and the previous frame is used. Is used to calculate the movement amount.
As the camera means, a USB (Universal Serial Bus) camera, a digital camera, a digital video camera, or the like can be used.

  Next, the speed / position estimation method for a high-speed moving object according to claim 2 of the present invention is a method in which the camera means is capable of high-speed movement. Calculating the velocity vector of the camera means in the frame from the distortion or deviation of the shape of the subject in the camera, and estimating the position by calculating the movement amount of the camera means from the frame rate of the camera image and the velocity vector of the camera means for each frame It is characterized by having a configuration comprising the steps of:

Assuming that the camera moves at high speed, the velocity vector is determined from the camera image distortion using only the camera image of one frame, and the amount of movement using the calculated velocity vector and the frame rate of the previous frame. Is calculated and position estimation is performed.
Here, a USB (Universal Serial Bus) camera, a digital camera, a digital video camera, or the like can be used as the camera means.
In the present invention, even when there is no common part with the image of the previous and subsequent frames, it is possible to determine the camera speed vector by processing only one frame image and analyzing the degree of distortion. Also, there are two types of scanning methods, interlaced and non-interlaced, depending on the camera, but the present invention provides a method for estimating a velocity vector that differs for each method.
Further, the present invention calculates a moving amount of a camera by estimating a speed transition from a speed sequence and using a frame rate between the estimated speed transition and the previous frame.

  Here, the speed / position estimation method of the high-speed moving object according to claim 3 of the present invention is the method of estimating the speed / position of the high-speed moving object according to claim 1 or 2, wherein the camera means is of an interlace type. The step of calculating the velocity vector is characterized in that the velocity vector is calculated by using the shift of the subject due to the time lag between the scanning of the even-numbered scanning line image and the odd-numbered scanning line image.

When the camera means is an interlace system, when the subject moves at a high speed or the camera means moves at a high speed, the subject is shifted in the camera image due to the time lag of the scan of the even-numbered scan line image and the odd-numbered scan line image.
The moving speed vector (moving speed and moving direction) of the subject can be obtained from this deviation.
If the camera means moves at a high speed, the speed vector of the camera means can be obtained by setting the direction of the subject to the reverse direction.

  According to a fourth aspect of the present invention, there is provided a method for estimating the speed / position of a high-speed moving object, wherein the camera means is a non-interlace type or an interlace type. The step of calculating the velocity vector is characterized in that the velocity vector is calculated using the vertical and horizontal distortions of the subject due to the scan time lag.

  When the camera means is a non-interlace system, if the subject moves at a high speed or the camera means moves at a high speed, the object will move during the scan because of the long scan time. Distortion occurs. The moving speed vector (moving speed and moving direction) of the subject can be obtained from the vertical and horizontal distortions of the subject generated in the camera image. If the camera means moves at a high speed, the speed vector of the camera means can be obtained by setting the direction of the subject to the reverse direction. Further, the non-interlace method using the distortion of the subject can also be used in the case of the interlace method. In other words, the subject may be distorted when only the odd line image or only the odd line image is viewed, and the subject is not fast enough to be detected by the above-described interlace method (that is, the subject does not appear to be shifted by the interlace method, Similarly, when the subject appears to be distorted, the moving speed vector (moving speed and moving direction) of the subject can be obtained from the vertical and horizontal distortion of the subject generated in the camera image.

  A speed / position estimation program for a high-speed moving object according to claims 5 to 9 of the present invention is a program for causing a computer to execute the steps of the method for estimating the speed / position of a high-speed moving object according to claims 1 to 4 or the program. Is a computer-readable recording medium on which is recorded.

  Next, a speed / position estimation system for a high-speed moving object according to a tenth aspect of the present invention is based on camera means capable of continuously imaging a moving subject, and distortion or misalignment of the shape of the subject in a one-frame camera image. The apparatus includes at least means for calculating the velocity vector of the subject in the frame and means for estimating the position by calculating the amount of movement of the subject from the frame rate of the camera image and the velocity vector of the subject for each frame.

  According to another aspect of the present invention, there is provided a system for estimating the speed and position of a high-speed moving object, wherein the camera means is capable of high-speed movement, the camera means capable of continuous imaging, and a subject in a one-frame camera image. Means for calculating the velocity vector of the camera means in the frame from the distortion or deviation of the shape of the camera, and means for estimating the position by calculating the movement amount of the camera means from the frame rate of the camera image and the speed vector of the camera means for each frame It is set as the structure provided with at least.

  The speed / position estimation system for a high-speed moving object according to claim 12 of the present invention is the system for estimating the speed / position for a high-speed moving object according to claim 10 or 11, wherein the camera means is of an interlace type. The means for calculating the velocity vector is characterized in that the velocity vector is calculated using a shift of the subject due to a time lag between scanning of the even-numbered scan line image and the odd-numbered scan line image.

  According to a thirteenth aspect of the present invention, there is provided a system for estimating the speed and position of a high-speed moving object according to the tenth or eleventh aspect, wherein the camera means is a non-interlace type or an interlace type. The means for calculating the velocity vector is characterized in that the velocity vector is calculated using the vertical and horizontal distortions of the subject due to the scan time lag.

  According to the estimation method, the estimation program, and the estimation system of the speed / position of the high-speed moving object according to the present invention, it is possible to estimate the velocity vector at the time of shooting the image frame using the distortion and displacement of the image in one frame, Further, there is an effect that the movement amount / position can be estimated with accuracy from the obtained velocity vector.

  First, there are two main types of camera scanning methods. One is a non-interlace method that scans one line at a time from one point above, and the other is an interlace that skips one line from one point above, that is, scans odd lines and then scans even lines that are skipped. It is a method. When scanning one screen, the scanning line in the horizontal direction is repeatedly scanned from top to bottom. At this time, the display of one screen (frame) is divided into two times, and an odd-numbered line is drawn at the first time and an even-numbered line is drawn at the second time is called “interlace”. A method of displaying one screen by one scan is called “non-interlace” (see FIG. 2).

In either of these two scanning methods, when a high-speed object is captured or when the camera body moves at high speed, the acquired image is distorted or the image is deviated. This distortion and misalignment of the image depends on the scan execution time. In the case of the non-interlace method, the object is stretched or shrunk as shown in FIG. That is, an image in a stationary state (circular image) as shown in FIG. 3A is distorted into an elliptical shape like the moving image in FIG. 3B due to the scan time lag. Utilizing this fact, the velocity vector is obtained from the scanning time lag by obtaining the extending direction and the extending distance as shown in FIG.
On the other hand, in the case of the interlace method, as shown in FIG. 4, the subject is shifted between the even-numbered scan line image and the odd-numbered scan line image. FIG. 4 shows a still image (circular image) as shown in FIG. 4A due to the time lag between the even-numbered scan line image and the odd-numbered scan line image, and the moving image shown in FIG. The image is misaligned. Utilizing this fact, the velocity vector is obtained from the scanning time lag by obtaining the extending direction and the extending distance as shown in FIG.

  5 and 6 show flowcharts of the speed / position estimation method for a high-speed moving object according to the present invention. FIG. 5 shows a case where the camera is an interlace system, and FIG. 6 shows a case where the camera is a non-interlace system.

First, a method for estimating the speed and position of a high-speed moving object when the camera is an interlace system will be described with reference to the flowchart of FIG.
Here, although the degree of high speed is not defined, a speed at which distortion and deviation can be confirmed in one frame of image data is considered as a high speed area.
As shown in the flowchart of FIG. 5, first, the subject is continuously imaged by the camera means (step S11). Next, one frame of image data is acquired (step S12). Then, the shift width of the subject due to the time lag of the scan of the even-numbered scan line image and the odd-numbered scan line image in one frame of image data is obtained (step S13). The moving speed of the subject is calculated from the shift width and the time lag (time) of the scan of the even-numbered scan line image and the odd-numbered scan line image (step S14). Next, the moving direction of the subject is obtained from the deviation direction (step S15). Then, the speed vector of the subject is estimated based on the obtained moving speed and moving direction of the subject (step S16).

The above steps 12 to 16 are repeated to calculate the velocity vector of the subject for each frame.
Then, the time (shooting interval) of each frame is obtained from the frame rate of the camera image, and the amount of movement of the subject is calculated from the velocity vector of the subject for each frame (step 17). Since the time (shooting interval) of each frame is short, the movement amount and direction are estimated from the shooting interval time of the frame and the position is estimated by adding them to the velocity vector in each frame ( Step 18).
FIG. 7 is an explanatory diagram of a method of calculating the movement amount from the speed. The distance is obtained linearly from the estimated speed from the acquired image data for each frame and the frame rate (that is, the shooting interval) with the previous frame. It is expected that a series of movements will eventually approach the actual movement amount. Furthermore, it is expected that the actual moving amount will be approached as much as possible by smoothly correcting the speed.

Next, a method for estimating the speed and position of a high-speed moving object when the camera is a non-interlace system will be described with reference to the flowchart of FIG.
As shown in the flowchart of FIG. 6, first, the subject is continuously imaged by the camera means (step S21). Next, one frame of image data is acquired (step S22). Then, the vertical and horizontal distortions of the subject in one frame of image data are obtained (step S23). This is because the extension of the subject shape in the scanning direction (usually from the top to the bottom of the camera image) is regarded as distortion, and the distorted shape is acquired. Then, the difference from the pre-registered object in a stationary state is calculated to obtain the extension width in the scanning direction. The moving speed of the subject is calculated from the distortion width (extension width) and the scan time lag (time) (step S24). Next, the moving direction of the subject is obtained from the distortion direction (step S25). Then, the speed vector of the subject is estimated based on the obtained moving speed and moving direction of the subject (step S26).

Step 22 to step 26 are repeated, and the velocity vector of the subject for each frame is calculated.
Then, the time (shooting interval) of each frame is obtained from the frame rate of the camera image, and the amount of movement of the subject is calculated from the velocity vector of the subject for each frame (step 27). Since the time (shooting interval) of each frame is short, the movement amount and direction are estimated from the shooting interval time of the frame and the position is estimated by adding them to the velocity vector in each frame ( Step 28).

  The above is a case where the subject moves at a high speed, but the same processing flow is performed when the camera is mounted on a high-speed moving object (for example, a high-speed moving radio-controlled helicopter) and the camera itself moves at a high speed. However, the vector direction is opposite to the moving vector direction of the subject.

  Next, FIG. 8 shows a schematic system configuration diagram of a speed / position estimation system for a high-speed moving object according to the present invention. A speed / position estimation system 1 includes a central processing unit (CPU) 2 and a ROM 3 that stores a program for executing the steps of the speed / position estimation method described above on the central processing unit. RAM 4 used at the time of execution, keyboard 5 and mouse 6 as a human machine interface, display 7, camera 8 for photographing the subject, image data photographed by camera 8, and code data after coding processing are recorded. A hard disk (HD) 9 is used.

  In the following embodiment, using the method for estimating the speed and position of a high-speed moving object according to the present invention, it is possible to accurately calculate the velocity vector and the movement amount of the high-speed moving object, which was inaccurate with the conventional method. This will be described with reference to experimental data.

In this embodiment, as one of indoor position identification systems, position estimation is performed by photographing a large number of same color and same shape markers arranged in a lattice pattern.
FIG. 9 shows the configuration of the experimental system used in this example. A circular marker having a black pattern, a diameter of 2 cm, the same color, and the same shape is arranged in a lattice pattern at a distance of 27.5 cm from the camera. The background is limited to white. Video taken by the camera is sent to the computer. The camera used in this system is "The CARD" manufactured by RL. Is used. This camera scans an image by an interlace method.

  Here, the procedure of the experiment will be described with reference to the flowchart of FIG. First, an image of 256 gradations taken with a camera is acquired. The size of the image is 640 horizontal pixels and 480 vertical pixels. Next, a binarization process is performed on the image to facilitate marker detection. The threshold value for binarization is set to 35 where black markers are easily detected under general brightness (illuminance 1000LUX-1300LUX). This value can be easily changed according to the brightness of the spot. The pixel value of each pixel in the binarized image is 255 for white and 0 for black. A histogram of the horizontal axis is created by dividing the total value of odd lines (even lines) in the vertical direction by 100. From there, a set of columns having a threshold value of 590 or less and a preceding column of 590 or less is taken, and an intermediate column is set as an x coordinate. When there are a plurality of sets, x coordinates are obtained by the number of sets.

  Five small windows with a width of 128 and a height of 96 are created around the derived x-coordinate, and pixel histograms of the horizontal axis and the vertical axis are created respectively. The method of making is the same as the method of making the histogram of the entire image described above. The total value of the pixel values of each pixel in the odd-numbered row (even-numbered row) vertical direction is divided by 100, and the histogram and pixels on the horizontal axis A vertical axis histogram is created by dividing the total value in the horizontal direction by 100. From there, a set of columns below the threshold on the horizontal axis is taken out, the middle column is taken as the x coordinate, a set of rows below the threshold on the vertical axis is taken out, and the middle row is taken as the y coordinate. Odd line (even line) points are extracted from the extracted x and y coordinates.

However, there are cases where a plurality of points are taken at the same marker as it is. In view of this, a point at a radius of about 40 pixels around one point of the marker is regarded as the same point. This value of radius 40 pixels is a value that can sufficiently secure the range of the same marker point set in an image acquired from a camera located at a distance of 27.5 cm from the marker.
As a result, as shown in FIG. 11, each marker has a point of one odd line and one even line.

Next, each odd marker and even marker are associated with each other. This association searches for an even marker within a radius of 100 pixels centering on the odd marker. This number is a value that the adjacent marker does not enter when stationary, and that the adjacent marker does not enter during movement or even during high-speed movement.
Then, the distance and direction of the associated odd marker even marker are obtained, and the average value of all the markers is taken. At this time, we decided to omit the extremely unnatural marker distance and direction. The velocity vector was made based on the data that had been experimented in advance from the average value. The amount of movement was calculated based on the frame rate with the previous image.

As an experimental environment, a card type small camera is set at a certain height using a marker having a diameter of 2 cm arranged on the floor every 10 cm, and the marker is moved in parallel.
In this experiment, a value for converting the number of pixels between the odd and even points of the marker in the acquired image into a velocity vector in real space is obtained. The procedure is as follows.

1) Translate over the marker in the experimental environment.
2) An image is recorded for each frame. At that time, an average value of pixels between odd and even points in the image and a frame rate between the previous image are recorded.
3) Take out consecutive frames close to constant speed.
4) The amount of movement in real space is obtained from the extracted images of successive frames, and the velocity vector is obtained from the frame rate.
5) The above operations 1) to 4) are repeated 60 times.
6) Plot the obtained velocity vector and the average number of pixels on a graph.
7) An approximate straight line is obtained from the graph, and a conversion value from the number of pixels to a velocity vector is obtained from the slope.

FIG. 12 is a diagram plotting the image speed and the number of pixels of 60 samples in the above experiment. In this speed estimation experiment, the converted value was 4.27716.

Next, experimental data for obtaining the variation between the actually moved distance and the movement amount calculated by the system will be described.
1) Translate 400 cm in the above experimental environment.
2) A velocity vector is obtained from an average value of pixels between odd and even points in the captured image.
3) The amount of movement from the previous image is calculated from the obtained velocity vector and the frame rate between the previous images.
4) Repeat the above steps 2) and 3) and stack them to calculate the total amount of movement.
5) Repeat steps 1) to 4) 30 times.
6) Graph the obtained values and examine the variation with the actual movement amount.
7) Figure 13 shows the graph obtained.

In the speed estimation experiment described above, as is clear from FIG. 12, the average number of pixels between the odd and even markers extracted from one image is proportional to the speed of the real space, and the speed is calculated from the number of pixels. It can be confirmed that it can be done.
FIG. 14 is an explanatory diagram of a speed estimation method from one frame image. FIG. 14B shows a limit region in which the correspondence relationship between adjacent markers is not known when going outside the circle outside the marker. Also, the center of the concentric circles is a scanned marker image of odd lines, and the scanned marker images of even lines are shifted to the right and shifted. The odd lines are scanned and then the even lines are scanned. In addition, a large circle in the center of the camera image in FIG. 14B indicates a reference for the first and last alignment for position estimation. Further, as is apparent from the evaluation experiment, a considerably high accuracy result is obtained that the error is −10 cm to +10 cm with respect to the movement of 400 cm. It is expected that a more accurate value can be obtained by improving the experimental environment and increasing the speed estimation experiment sample.

  In this experiment, the estimated speed was linearly complemented with the frame rate of the previous frame, and the amount of movement was calculated. However, as shown in FIG. 15, a position shift due to a rapid speed change between frames may occur. Therefore, it can be expected that the accuracy is further improved by adding the smooth inter-frame speed.

  In the second embodiment, an example of how to obtain a value when converting the number of pixels between an odd point and an even point of a marker in an acquired image into a velocity vector in real space is shown. As an experimental device, a circle is produced by connecting ends of a band in which markers having a diameter of 1.5 cm are spread at intervals of 10 cm. The circumference of the circle is 200 cm. In this device, when the handle attached to the device is rotated once, the circle itself is rotated three times. Therefore, the device handle is rotated once, and the moving distance of the circumferential portion is 600 cm. In addition, it is measured in advance to measure the time taken to make one round of the device.

  Next, a marker on the circumference is photographed from the outside of the circumference, and the camera image is recorded for each frame. At that time, an average value of pixels between odd and even points in the image and a frame rate between the previous image are recorded. The amount of movement in the real space is obtained from the extracted images of successive frames, and the average speed when the one-round device is turned is calculated using the time taken to make one turn of the device. This operation is repeated 60 times.

  The obtained velocity vector and the average value of the number of pixels are plotted on a graph. An approximate straight line is obtained from the graph, and a conversion value from the number of pixels to a velocity vector is obtained from the slope. The direction of the camera is changed so that the marker moves in eight directions with respect to the camera, and the same experiment is performed for each direction.

Next, a variation between the actually moved distance and the movement amount calculated by the system is obtained by the following experiment.
1) The 600 cm camera is moved in the experimental environment of Example 2. That is, the device is rotated once.
2) The amount of movement from the previous image is calculated using the conversion value obtained in the speed estimation experiment. 3) The amount of movement in real space is obtained from the extracted images of successive frames. 4) The total movement amount is calculated by repeatedly stacking the operations 2) and 3). 5) Repeat steps 1) to 4) above.
6) Graph the obtained values and evaluate the variation with the actual movement amount.

As speed / position estimation experiments in Example 2, experiments were performed in six directions (traveling direction, backward direction, right direction, left direction, left diagonally forward direction, and right diagonally backward direction). The results are shown in the graphs of FIGS. The conversion values obtained as a result were advancing direction: 2.4504, backward direction: 2.1805, right direction: 2.0606, left direction: 2.1377, left diagonally forward direction: 2.0623, right diagonally backward direction: 1.9508, respectively.
In either case, there is a correlation between the speed and the amount of pixel shift. The difference in the conversion value depending on the direction is the difference in the time required to finish scanning the marker, and a value consistent with the assumption is obtained.

Further, the result of the position estimation experiment in the traveling direction is shown in the graph of FIG.
Here, one round of the apparatus rotates in about 10 seconds. That is, the average speed is about 60 cm / s. The frame rate was about 7 frames / second. The error of the detected value is within −30 cm to +20 cm with respect to the movement of 600 cm, and it can be said that the accuracy of 5% or less was obtained. During the experiment, there were some frames where the marker could not be detected (about 5%), but this was mainly when the camera image had noise. It is considered that the performance is improved by improving the quality of the camera image or improving the noise resistance performance of the image processing.

  The method, the estimation program, and the estimation system for the speed and position of a high-speed moving object according to the present invention include the estimation of the position of a mobile robot, the estimation of the speed of a moving object such as a radio controlled helicopter, the estimation of the movement speed of a sports player, the ball It can be used for purposes such as estimating the movement speed of baseball, soccer, tennis, etc.

Explanatory diagram of problems in non-ID markers when the target object equipped with the camera moves at high speed Illustration of interlace method and non-interlace method Explanatory drawing about subject distortion in non-interlace method Explanatory diagram of subject displacement in case of interlace method Flowchart diagram of the method for estimating the speed and position of a high-speed moving object in the case of the interlace method Flowchart diagram of the method for estimating the speed and position of a high-speed moving object in the non-interlace method Illustration of how to calculate the amount of movement from speed Schematic system configuration diagram of a speed / position estimation system for a high-speed moving object according to the present invention Configuration diagram of the experimental system used in Example 1 Flow chart of experimental procedure of Example 1 Odd line even line point Real space speed and the average number of pixels of the image at that time System calculated movement amount and variation when moving the camera 400cm Explanatory drawing of speed estimation method from 1 frame image Explanatory drawing of the problem that position shift occurs due to sudden speed change between frames Graph of speed estimation experiment results (traveling direction) Graph of speed estimation experiment results (backward direction) Graph of speed estimation experiment results (right direction) Graph of speed estimation experiment results (left direction) Graph of speed estimation experiment result (left diagonally forward direction) Graph of speed estimation experiment results (right diagonally backward) Graph of position estimation experiment results (traveling direction)

Explanation of symbols

1 speed / position estimation system 2 central processing unit (CPU) 3 ROM 4 RAM 5 keyboard 6 mouse 7 display 8 camera 9 hard disk (HD)

Claims (13)

  A step of continuously imaging a moving subject by camera means, a step of calculating a velocity vector of the subject in the frame from a distortion or deviation of the shape of the subject in one frame of the camera image, a frame rate and a frame of the camera image A method for estimating the speed and position of a high-speed moving object, comprising a step of estimating a position by calculating a movement amount of the object from a speed vector of each subject.   The camera means is movable at high speed, and the velocity vector of the camera means in the frame is calculated from the step of continuously capturing images by the camera means and the distortion or displacement of the shape of the subject in the camera image of one frame. And a step of calculating the amount of movement of the camera means from the frame rate of the camera image and the velocity vector of the camera means for each frame and estimating the position of the speed and position of the high-speed moving object Estimation method.   The camera means is of an interlace system, and the step of calculating the velocity vector calculates the velocity vector using a shift of a subject due to a scan time lag between the even-numbered scan line image and the odd-numbered scan line image. The method for estimating the speed and position of a high-speed moving object according to claim 1 or 2.   The camera means is of a non-interlace type or of an interlace type, and the step of calculating the velocity vector calculates the velocity vector using the vertical and horizontal distortions of the subject due to the scan time lag. The method for estimating the speed and position of a high-speed moving object according to claim 1 or 2.   A step of inputting a camera image continuously captured by the camera means, a step of calculating a velocity vector of the subject in the frame from a distortion or deviation of the shape of the subject in the camera image of one frame, and a frame rate of the camera image And a speed / position estimation program for a high-speed moving object for causing a computer to execute a step of calculating the amount of movement of the subject from the velocity vector of the subject for each frame and estimating the position.   The camera means is capable of moving at high speed, and the step of inputting a camera image continuously taken by the camera means and the distortion or misalignment of the shape of the subject in the camera image of one frame, the camera in the frame High-speed movement for causing a computer to calculate a speed vector of the means and a step of calculating a movement amount of the camera means from a frame rate of the camera image and a speed vector of the camera means for each frame and estimating a position An object speed / position estimation program.   The camera means is of an interlace system, and the step of calculating the velocity vector calculates the velocity vector using a shift of a subject due to a scan time lag between the even-numbered scan line image and the odd-numbered scan line image. The speed / position estimation program for a high-speed moving object according to claim 5 or 6.   The camera means is of a non-interlace type or of an interlace type, and the step of calculating the velocity vector calculates the velocity vector using the vertical and horizontal distortions of the subject due to the scan time lag. The speed / position estimation program for a high-speed moving object according to claim 1 or 2.   A computer-readable recording medium in which the high-speed moving object speed / position estimation program according to any one of claims 5 to 8 is recorded.   Camera means capable of continuously capturing a moving subject, means for calculating a velocity vector of the subject in the frame from the distortion or deviation of the shape of the subject in one frame of the camera image, the frame rate of the camera image and each frame A speed / position estimation system for a high-speed moving object, comprising at least means for calculating the amount of movement of the subject from the velocity vector of the subject and estimating the position.   The speed vector of the camera means in the frame is calculated from the camera means that can move at high speed and can continuously capture images, and the distortion or displacement of the shape of the subject in the camera image of one frame. Means for calculating the moving amount of the camera means from the frame rate of the camera image and the velocity vector of the camera means for each frame and estimating the position of the speed and position of the high-speed moving object Estimation system.   The camera means is of an interlace type, and the means for calculating the speed vector calculates the speed vector using a shift of a subject due to a scan time lag between the even-numbered scan line image and the odd-numbered scan line image. The speed / position estimation system for a high-speed moving object according to claim 10 or 11. The camera means is of a non-interlace type or an interlace type, and the means for calculating the speed vector calculates a speed vector using distortion in the vertical and horizontal directions of the subject due to a scan time lag. The speed / position estimation system for a high-speed moving object according to claim 10 or 11.