JP6349737B2 - Moving object tracking device and moving object tracking method - Google Patents

Description

  The present invention relates to a moving body tracking device and a moving body tracking method.

  A method has been proposed in which a moving body moving at a high speed at a long distance point is imaged by an optical sensor and the course is predicted. For example, in Japanese Patent Application Laid-Open No. 2008-112210, a moving body detection unit that acquires a plurality of images of a moving body, detects the moving body from these images, and calculates a movement vector thereof, a calculated movement vector, and the like And a position predicting means for predicting a future position using the above.

  When predicting the future position, it is assumed that the moving body is moving linearly, and the position of the moving body is predicted using the movement vector based on the assumption.

JP 2008-112210 A

  However, in the method according to Japanese Patent Application Laid-Open No. 2008-112210 described above, a plurality of images having different times are necessary when obtaining the movement vector. For this reason, there are cases where the accuracy of estimation of the traveling direction is low in a moving body whose traveling direction changes.

  In addition, since it is assumed that the moving body moves linearly, there is a problem that the future position cannot be properly predicted when the moving body is moving non-linearly.

Therefore, a main object of the present invention is to provide a mobile tracking device and a mobile tracking method that can improve the prediction accuracy of the future position of the mobile.


Therefore, the main object of the present invention is to provide a mobile tracking device and a mobile device that can predict the future position of the mobile body with high accuracy even when the mobile body moves in a non-linear motion without requiring many images. It is to provide a body tracking method.

  In order to solve the above-described problems, the invention according to the moving body tracking device captures an image of the moving body at the current position as image information and compares it with a template registered in advance, thereby determining the traveling direction of the moving body as the traveling direction. The future direction where the moving body moves along the traveling direction by acquiring the traveling direction acquisition unit that calculates / outputs the information and the current position information and traveling direction information that is the current position of the moving body A position prediction unit.

  In addition, the invention relating to the moving body tracking method calculates and outputs the moving direction of the moving body as the traveling direction information by capturing the image of the moving body at the current position as image information and comparing it with a pre-registered template. A future direction prediction procedure for acquiring a current position information and a current direction information that are the current position of the mobile body, and predicting a future position when the mobile body has moved along the travel direction; It is characterized by including.

  According to the present invention, the traveling direction of the moving object is acquired from at least one captured image, and the future position is predicted using the traveling direction predicted using the moving direction, so that the prediction accuracy can be improved. Become.

It is a figure explaining the principle regarding a tracking method, (a) is a figure explaining the tracking method concerning this invention, (b) is a figure explaining the well-known tracking method. It is a block diagram of the mobile body tracking device concerning an embodiment. It is a flowchart shown in a mobile body tracking method. It is explanatory drawing of image information, speed information, and distance information, (a) is explanatory drawing of image information, (b) is explanatory drawing of speed information, (c) is explanatory drawing of distance information. It is the figure which illustrated the control process. It is explanatory drawing of visual field direction information. It is a figure explaining derivation of advancing speed. It is a figure explaining the error of the predicted future position. It is a block diagram of the mobile body tracking apparatus of the structure replaced with the mobile body tracking apparatus of FIG.

<Principle>
First, the principle of the moving body tracking method according to the present embodiment will be described. 1A and 1B are diagrams for explaining the tracking principle of a moving object, where FIG. 1A is a principle according to the present embodiment (hereinafter referred to as the principle of the present invention), and FIG. 1B is a conventional principle (hereinafter referred to as a known principle). It is the figure which showed the principle which concerns. In the figure, the moving body M is shown in the captured image F. A symbol P indicates the position (pixel position) of the moving body M in the coordinate system set in the captured image F. Therefore, the fact that the moving body M is located at the position P is described as M (P). The time series information is indicated by subscripts i, j, and k, where i is the past, j is the present, and k is the future.

  In order to predict the future position of the mobile body, information on the current position, speed, and traveling direction of the mobile body is required. According to the known principle, it is assumed that the moving body M does not change the traveling direction, assuming that the moving body M is moving at a constant linear velocity. Then, the speed, the current position, and the traveling direction of the moving body M are obtained by a radar or the like, and the future position is predicted. At this time, the traveling direction is obtained from the past position and the current position of the moving body. That is, as shown in FIG. 1B, the traveling direction is obtained from the past position Pi and the current position Pj, and the speed is obtained from the distance between the position Pi and the position Pj and the required time t1. Then, the position Pk after the time t2 from the current position Pj is predicted using the traveling direction and the speed. Therefore, if the imaging apparatus is directed toward the predicted position Pk and the image is taken, the moving object M can be imaged (tracked).

  However, as described above, such a tracking method is established under the assumption that the moving body M is moving at a constant linear velocity. Therefore, in the case of the moving body M that moves in a non-linear manner, after the time t2, it is not located at the position Pk but is located at the position Pk ′. Therefore, the moving body is imaged (tracking) even if the imaging device is directed to the predicted position Pk. Can not do it.

  In contrast, according to the principle of the present invention, the traveling direction is obtained from the current captured image Fj as shown in FIG. Further, the speed and the current position are obtained using information from the radar or the like, and the future position Pk is predicted. Therefore, since the future position Pk can be predicted regardless of whether the moving body M is moving linearly or non-linearly, it can be tracked.

  Next, the mobile body tracking device 2 based on the principle of such a mobile body tracking method will be described. FIG. 2 is a block diagram of the moving body tracking device 2 according to the present embodiment. The moving body tracking device 2 includes a traveling direction acquisition unit 11, a current position calculation unit 12, and a future position calculation unit 13 as main components.

  Note that the moving body tracking device 2 illustrated in FIG. 2 does not include the measurement unit 4 including the imaging device 4a and the radar 4b (radio wave radar, laser radar, or the like). This is because the imaging device 4a and the radar 4b are installed for various purposes, and the mobile tracking device 2 according to the present embodiment uses the existing equipment (the imaging device 4a, the radar 4b, etc.). This is just an example. However, including the imaging device 4a and the radar 4b in the moving body tracking device 2 does not depart from the spirit of the present invention.

  The imaging device 4a is equipped with a CCD camera or the like, images a moving moving body, and outputs it as image information G1. Further, the radar 4b detects information (hereinafter referred to as speed information) G2 related to the speed of the moving object and information (hereinafter referred to as distance information) G3 related to the distance to the moving object. As information regarding the speed of the moving body, a Doppler signal can be exemplified, but the present embodiment is not limited to the Doppler signal.

  The traveling direction acquisition unit 11 includes a template storage unit 11a and an extracted image analysis unit 11b.

  The template storage unit 11a stores templates in advance. For example, when the moving body is an aircraft, this template can be exemplified by an image of three-dimensional data of an aircraft including positional relationships such as a cockpit, a main wing, and a tail wing as feature points, or numerical data composed of three-dimensional information.

  The extracted image analysis unit 11b extracts an image of only the moving object from the image information G1. Hereinafter, an image obtained by the extraction process is referred to as an extracted image, and information regarding the extracted image is referred to as extracted image information G4. Then, the extracted image analysis unit 11b performs a contrast process between the extracted image and the template, determines the traveling direction of the moving body with respect to the imaging device 4a with three-dimensional coordinates, and outputs this as traveling direction information G5.

  As a method for acquiring the traveling direction information, for example, a method disclosed in Japanese Patent Application Laid-Open No. 09-9-33649 can be applied. In this method, a plurality of points are designated on an ISAR image obtained by processing radar echoes, and the relative positional relationship on the two-dimensional surface between each point is obtained. Then, corresponding points corresponding to a plurality of points on the ISAR image are obtained for a three-dimensional shape model arbitrarily selected from the target three-dimensional shape model database, and a relative positional relationship in the three-dimensional space between the corresponding points is obtained. Then, the direction of travel is estimated.

  Of course, the method of acquiring the traveling direction information is not limited to the above-described method, and other known methods can be applied. As described above, the template image may be an image of three-dimensional data or numerical data including three-dimensional information. In this case, the template image is three-dimensionally rotated and scaled, and the difference between the template image and the extracted image at that time is obtained. Then, it is possible to obtain the traveling direction from the amount of rotation when the difference is the smallest.

  The current position calculation unit 12 includes a target orientation analysis unit 12a and a target position analysis unit 12b.

  The target orientation analysis unit 12a determines the viewing direction of the moving object from the pixel position of the moving object included in the extracted image and the conditions such as the directing direction (gaze direction), the angle of view, and the number of pixels of the imaging device 4a. This is output as visual field direction information G6. The visual field direction is a direction formed by a line segment connecting the observation point and the moving body, and here, the viewing direction of the moving body as viewed from the imaging device 4a.

  Based on the distance information G3 from the radar 4b and the visual field azimuth information G6 from the target azimuth analysis unit 12a, the target position analysis unit 12b obtains the current position of the moving object in three-dimensional coordinates and outputs this as current position information G7. To do.

  The future position calculation unit 13 includes a future position prediction unit 13a and a target position error analysis unit 13b.

  The future position prediction unit 13a receives speed information G2 from the radar 4b, travel direction information G5 from the extracted image analysis unit 11b, and current position information G7 from the target position analysis unit 12b. Therefore, the future position prediction unit 13a predicts the future position of the mobile object using these pieces of information, and outputs the prediction result as future position information G8.

  When predicting the future position of the moving body, the current position, traveling direction, and speed of the moving body are required. The current position can be obtained from the current position information G7, and the traveling direction can be obtained from the traveling direction information G5.

  However, the speed indicated by the speed information G2 (hereinafter referred to as the line-of-sight speed) is the speed in the line-of-sight direction of the moving body (the direction of the moving body viewed from the radar), and the speed in the traveling direction of the moving body (hereinafter referred to as the progress) Not described as speed). Therefore, the traveling speed having the visual line speed indicated by the speed information G2 as a speed component is obtained. That is, the traveling speed is calculated so that the line-of-sight direction component of the traveling speed becomes the line-of-sight speed.

  By the way, the position error correction information G9 is also inputted to the future position prediction unit 13a from the target position error analysis unit 13b. Therefore, the future position prediction unit 13a corrects at least one information of the traveling direction information G5, the speed information G2, and the current position information G7 based on the position error correction information G9, and uses the corrected information to determine the traveling speed. Ask.

  The target position error analysis unit 13b obtains an error of the future position information G8 based on the current position information G7 from the target position analysis unit 12b and the future position information G8 from the future position prediction unit 13a. However, if the future position information G8 is the position of the moving object at time t1, the position of the moving object indicated by the current position information G7 is also the position at time t1. Thus, from the difference between the predicted position (the future position of the mobile body indicated by the future position information G8) and the actual position (the current position indicated by the current position information G7), the traveling direction information G5, the speed information G2, and the current A correction amount for the position information G7 is calculated. Then, as described above, the future position prediction unit 13a obtains the corrected traveling speed of the moving body using the correction amount.

  The correction process is preferably performed with priority on the traveling direction information G5. This is because the speed information G2 depends on the performance of the radar 4b, and the current position information mainly includes errors derived from the measuring device depending on the performance of the radar 4b and the imaging device 4a. This is because the analysis error depending on the control accuracy of the control process is main.

  The target position error analysis unit 13b stores the calculated correction value for several times, and determines whether the correction amount tends to converge. And when it is in a convergence tendency, it outputs to the future position prediction part 13a, without changing a correction amount, and when not in a convergence tendency, a correction calculation process is continued.

  Next, a detailed configuration of the mobile tracking device 2 described above will be described with reference to FIG. FIG. 3 is a flowchart showing the moving body tracking method.

  Step S1: The moving body tracking device 2 takes in measured values (image information G1, velocity information G2, distance information G3) from the measurement unit 4 such as the imaging device 4a or the radar 4b. FIGS. 4A to 4C are explanatory diagrams of the image information G1, the speed information G2, and the distance information G3. FIG. 4A illustrates the image information G1. In this image information G1, an aircraft is shown as a moving body, and clouds and the like are shown as a background. FIG. 4B is an explanatory diagram of the speed information G2, and illustrates a case where the moving body is approaching at a line-of-sight speed of 100 m / s in the line-of-sight direction of the radar 4b. Further, FIG. 4C is an explanatory diagram of the distance information G3, and illustrates a case where the moving body is present at a position 10 km away from the radar 4b in the line-of-sight direction.

  Step S2: Therefore, the extracted image analysis unit 11b creates extracted image information G4 from the image information G1. The extracted image information G4 is created by subtracting the background from the image information G1 as shown in FIG. As a result, the moving object is extracted from the image information G1, and an image is obtained.

  Step S3: After creating such extracted image information G4, the extracted image analysis unit 11b outputs the extracted image information G4 and obtains the traveling direction information G5 from the extracted image information G4 by a contrast process.

  This contrast process is a process of reading a template from the template storage unit 11a and comparing it with the extracted image information G4. FIG. 5 is a diagram illustrating such a control process. An arbitrary template is read from the template storage unit 11a and compared (contrast) with the extracted image information G4. When the difference between the extracted image information G4 and the template is smaller than a preset threshold value, it is determined that the compared template and the extracted image information G4 have the same type. The same type corresponds to determining that the moving body included in the extracted image information G4 is an aircraft when the template is an aircraft image. Hereinafter, a case where the type of the moving body is an aircraft will be described as an example.

  Step S4: Next, the extracted image analysis unit 11b determines the position of the feature point corresponding to the type of the moving object from the extracted image information G4. For example, when the moving body is an aircraft, the position of preset characteristic parts such as a cockpit, main wing, and tail wing is determined from the image of the extracted image information G4. And the advancing direction of a moving body is determined with a three-dimensional coordinate from the positional relationship of a characteristic part.

  Since the template is a three-dimensional image, the aircraft in the template can be superimposed on the aircraft in the extracted image information G4 by rotating and enlarging / reducing the template. The traveling direction of the moving body can be obtained from the rotation amount of the template at this time.

  Step S5: As shown in FIG. 6, the target azimuth analyzing unit 12a determines the azimuth of the moving body from the position of the moving body in the extracted image information G4, and outputs it as visual field azimuth information G6.

  For example, in the case illustrated in FIG. 6, if the extracted image information G4 is image information having a horizontal angle of 640 pixels and an angle of view of 2 °, and a vertical angle of 480 pixels and an angle of view of 1.5 °, the moving object is centered in the horizontal axis direction. 160 pixels, 120 pixels from the center in the vertical axis direction. At this time, assuming that the center of the extracted image information G4 is the current directivity direction (gaze direction) of the imaging device 4a, the moving body is about 0.5 ° from the gaze direction in the horizontal axis direction and about 0 in the vertical axis direction. It can be determined that the head is located in a 38 ° azimuth.

  Step S6: Next, the target position analysis unit 12b calculates the distance to the moving body using the distance information G3 and the visual field orientation information G6, and outputs the current position information G7.

  The distance information G3 from the radar 4b only indicates the distance between the moving body and the radar 4b, and does not include information regarding the direction. Therefore, the current position of the moving body (a position that is separated from the radar 4b by a distance Δ □) is calculated from the distance information G3 and the visual field direction information G6.

  Step S7: The traveling direction information G5, speed information G2, and current position information G7 acquired by the above process are input to the future position prediction unit 13a. The future position prediction unit 13a obtains the traveling speed in the traveling direction of the moving object from the speed information G2.

  FIG. 7 is a two-dimensional plan view for explaining the derivation of the traveling speed. It is assumed that the traveling direction of the moving body is 60 ° in the direction approaching the radar 4b with respect to the viewing direction from the traveling direction information G5, and the viewing speed is 100 m / s in the direction approaching the radar 4b from the speed information G2. At this time, the traveling speed of the moving body is 200 m / s when the line-of-sight speed is projected in the traveling direction.

  Step S8: Since the traveling speed has been calculated in this way, the future position predicting unit 13a uses the traveling speed, the current position, and the traveling direction to move from the current position to the direction along the traveling direction at the traveling speed. When this is done, it is calculated which position (future position) is located after a predetermined time. The calculated future position is output as future position information G8. The future position prediction unit 13a stores a correction value from the target position error analysis unit 13b input in the procedure described later. The future position prediction unit 13a performs a correction based on the correction value on at least one of the traveling speed, the current position, and the traveling direction, and predicts the future position.

  Step S9: The target position error analysis unit 13b calculates a future position error from the difference between the future position information G8 and the moving body current position information G7, and obtains a correction amount based on the error.

  FIG. 8 is a diagram for explaining the error. The point α indicates the future position indicated by the future position information G8, and the point β indicates the current position indicated by the moving body current position information G7. In FIG. 8, it is assumed that main error is included in the traveling direction information G5. That is, the traveling direction calculated based on the traveling direction information G5 is an angle of 60 ° with respect to the visual field direction, but the actual traveling direction is an angle of 50 °. Therefore, the target position error analysis unit 13b sets the angle difference (= 10 °) in the traveling direction as a correction value.

  Steps S10 and S11: The target position error analysis unit 13b stores the correction value for several times and determines its convergence. In this case, if the correction value approaches a constant value, it is determined that there is convergence, and this correction value is output to the future position prediction unit 13a. The future position prediction unit 13a replaces the correction value with the correction value that has already been stored. That is, the correction value is updated.

  On the other hand, if the correction value increases, the process returns to step S9. Returning to step 9 means that the correction value stored in the future position prediction unit 13a is not updated.

  As described above, in predicting the future position, information such as the past position of the mobile object is not required, and the future position is predicted only with the current information. For this reason, it becomes possible to track a moving body that moves non-linearly with high accuracy without using a complicated tracking algorithm or the like.

  In the above description, the current position of the moving object is calculated with high accuracy by using the viewing direction information G6 and the distance information G3. However, when the radar can acquire the current position of the moving object using, for example, an array antenna, it is added that the current position calculation unit can be omitted as shown in FIG.

DESCRIPTION OF SYMBOLS 2 Moving body tracking apparatus 4 Measurement unit 4a Imaging apparatus 4b Radar 11 Traveling direction acquisition unit 11a Template memory | storage part 11b Extracted image analysis part 12 Current position calculation unit 12a Target direction analysis part 12b Target position analysis part 13 Future position calculation unit 13a Future position Prediction unit 13b Target position error analysis unit

Claims (8)

A moving direction analysis unit that takes in an image of a moving body at a specific timing as image information and compares it with a template registered in advance, and calculates the moving direction of the moving body as traveling direction information;
Speed acquisition means for acquiring the speed of the moving body;
Moving body position acquisition means for acquiring position information of the moving body;
The traveling direction information at a specific timing of the mobile body, the speed of the mobile body at the specific timing acquired by the speed acquisition means, and the mobile body at the specific timing acquired by the mobile body position acquisition means position information of the position prediction means for predicting the position of the movable body when the predetermined period of time has elapsed from a specific timing using a
The mobile body position acquisition means acquires position information of the mobile body when a predetermined period has elapsed from the specific timing, and the position of the mobile body predicted by the position prediction means and the mobile body when the predetermined period has elapsed from the specific timing. A moving body tracking device comprising a moving body position error analyzing means for calculating a correction value for correcting the traveling direction information by calculating a prediction error by comparing with position information . A moving direction analysis unit that takes in an image of a moving body at a specific timing as image information and compares it with a template registered in advance, and calculates the moving direction of the moving body as traveling direction information;
Speed acquisition means for acquiring the speed of the moving body;
Moving body position acquisition means for acquiring position information of the moving body;
The traveling direction information at a specific timing of the mobile body, the speed of the mobile body at the specific timing acquired by the speed acquisition means, and the mobile body at the specific timing acquired by the mobile body position acquisition means A mobile body tracking device comprising position prediction means for predicting the position of the mobile body when a predetermined period has elapsed from a specific timing using the position information of
The position predicting means uses the traveling direction information and the line-of-sight speed information as the line-of-sight direction of the traveling speed when the speed in the traveling direction is the traveling speed and the speed of the moving body in the radar line-of-sight direction is the line-of-sight The moving body tracking device according to claim 1, wherein the traveling speed is calculated so that a velocity component in the line coincides with the line-of-sight speed . The mobile tracking device according to claim 1 ,
When the position predicting means uses the speed in the traveling direction as the traveling speed and the speed in the radar line-of-sight direction of the moving body as the line-of-sight speed, the position predicting means uses the traveling direction information and the line-of-sight speed information to calculate the position The moving body tracking device , wherein the traveling speed is calculated so that a speed component of the traveling speed in the line-of-sight direction matches the line-of-sight speed . The mobile tracking device according to any one of claims 1 to 3,
The traveling direction analyzing means is
Template storage means for storing a plurality of the templates in advance;
The extracted image information obtained by extracting the moving object from the image information is created, and the extracted image information is compared with the template read from the template storage unit, thereby obtaining the traveling direction of the moving object as the traveling direction information. A moving body tracking device comprising extracted image analysis means . A moving direction analysis procedure for calculating the moving direction of the moving body as moving direction information by taking an image of the moving body at a specific timing as image information and comparing it with a pre-registered template, and the speed of the moving body To get the speed acquisition procedure,
A moving object position acquisition procedure for acquiring position information of the moving object;
The traveling direction information at a specific timing of the mobile body, the speed of the mobile body at the specific timing acquired by the speed acquisition procedure, and the mobile body at the specific timing acquired by the mobile body position acquisition procedure position information of the position prediction procedure predicts the position of the movable body when the predetermined period of time has elapsed from a specific timing using a
The mobile body position acquisition procedure acquires the position information of the mobile body at the time when the predetermined period has elapsed from the specific timing, and the position of the mobile body predicted by the position prediction procedure and the mobile body at the time when the predetermined period has elapsed from the specific timing. A moving body tracking method comprising: a moving body position error analysis procedure for calculating a correction value for correcting the traveling direction information by calculating a prediction error by comparing with position information . A moving direction analysis procedure for calculating the moving direction of the moving body as moving direction information by taking an image of the moving body at a specific timing as image information and comparing it with a pre-registered template, and the speed of the moving body To get the speed acquisition procedure,
A moving object position acquisition procedure for acquiring position information of the moving object;
The traveling direction information at a specific timing of the mobile body, the speed of the mobile body at the specific timing acquired by the speed acquisition procedure, and the mobile body at the specific timing acquired by the mobile body position acquisition procedure A position tracking procedure for predicting the position of the moving object when a predetermined period has elapsed from the specific timing using the position information of
In the position prediction procedure, when the speed in the traveling direction is set as the traveling speed and the speed in the radar line-of-sight direction of the moving body is set as the line-of-sight speed, the position prediction procedure is calculated from the traveling direction information and the line-of-sight speed The moving body tracking method , wherein the traveling speed is calculated so that a speed component of the traveling speed in the line-of-sight direction matches the line-of-sight speed . The mobile tracking method according to claim 5 ,
In the position prediction procedure, when the speed in the traveling direction is set as the traveling speed and the speed in the radar line-of-sight direction of the moving body is set as the line-of-sight speed, the position prediction procedure is calculated from the traveling direction information and the line-of-sight speed. The moving body tracking method , wherein the traveling speed is calculated so that a speed component of the traveling speed in the line-of-sight direction matches the line-of-sight speed . A moving body tracking method according to any one of claims 5 to 7,
The traveling direction analysis procedure is:
A template storage procedure for storing a plurality of the templates in advance;
Including an extracted image analysis procedure for creating extracted image information obtained by extracting the moving object from the image information and comparing the extracted image information and the template to determine the traveling direction of the moving object as traveling direction information. The moving body tracking method characterized by this.

Images