JP2013055474A - Mobile object tracking device and reference point tracking method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mobile object tracking device for precisely grasping a position of a point detached by a prescribed length in a direction rotated by a prescribed angle from a travel speed direction of a mobile object to a prescribed point on the mobile object.SOLUTION: A prescribed part of the mobile object is imaged as an image tracking point, and a tracking error showing a difference between a tracking original point coordinate for tracking and a coordinate of an image tracking point of the mobile object is obtained from image data of the acquired image tracking point. On the basis of the tracking error, a visual axis angle speed obtained by correcting follow-up delay of an imaging device to the mobile object is calculated for tracking angular speed. A relative distance between the mobile object tracking device and the mobile object and travel speed of the mobile object tracking device itself are measured. A reference point tracking angle for tracking a reference point detached by the prescribed length in a direction rotated by a prescribed angle from the travel speed direction of the mobile object from the image tracking point is calculated on the basis of the obtained visual axis angle, the relative distance, and the travel speed of the mobile object tracking device itself.

Description

  Embodiments described herein relate generally to a moving object tracking device and a moving object reference point tracking method.

  In general, in a tracking device that tracks a moving object by an image tracking method, if the shape of the entire moving object to be tracked can be imaged and its shape, feature amount, etc. can be identified, a certain point on the moving object The position on the captured image can be calculated.

  2. Description of the Related Art Conventionally, there is one that extracts the principal axis of inertia of a moving object from a captured image by image processing or the like and calculates a specific point on the moving object based on the extracted direction (see, for example, Patent Document 1 for a flying object). ).

  However, when the dynamic range of the brightness value of the pixel of the moving object in the captured image exceeds the dynamic range of the brightness value of the image sensor, or due to the pixel having a brightness value above a certain threshold for image processing and noise removal. When only images (including the case of binarization) can be obtained, the shape of the entire moving object to be tracked can be obtained from the captured image even though the pixel information having the luminance value necessary for tracking can be obtained. It may be difficult to obtain.

Japanese Patent No. 4412799

  The problem to be solved by the present invention is that a moving object tracking capable of accurately grasping the position of a predetermined distance in a direction rotated by a predetermined angle from the moving speed direction of the moving object with respect to a certain point on the moving object It is an object of the present invention to provide a device and a moving object reference point tracking method.

The moving object tracking device according to the embodiment captures image data of an image tracking point acquired by the imaging device that images a predetermined part of the moving object as an image tracking point, a drive mechanism that drives the imaging device, and the imaging device. An image processor that inputs and calculates a tracking error representing a difference between the tracking origin coordinates for tracking in the captured image and the coordinates of the image tracking point of the moving object;
A control processor for calculating a visual axis angular velocity obtained by correcting a tracking delay of the imaging device with respect to the moving object to a tracking angular velocity of the imaging device based on a tracking error obtained by the image processor; A distance measuring sensor that measures the relative distance between them, a speed measuring device that measures the moving speed of the moving object tracking device itself, an angular velocity obtained by the control processor, and a relative distance obtained by the distance sensor. And a reference point separated by a predetermined length in a direction rotated by a predetermined angle from the moving speed direction of the moving object from the image tracking point based on the moving speed of the moving object tracking device itself obtained by the speed measuring device. A reference point tracking angle calculation unit that calculates a reference point tracking angle for tracking the reference point.

It is a figure which shows the structure of the moving object tracking apparatus which concerns on embodiment. It is a figure which shows the coordinate system of the moving object seen from the imaging device. It is a figure which shows the granting relationship of the signal with what simplified the function implement | achieved by an image pick-up device, an image processor, and a control processor to an image tracking control system and an angular velocity control system, and a drive mechanism part.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same portions are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a diagram illustrating a configuration of a moving object tracking device 1 according to the embodiment. The moving object tracking device 1 according to the present embodiment directs the imaging device 10 in the direction of the moving object 100 and tracks the moving object 100 based on the captured image. A certain point on the moving object 100 for tracking an image, for example, a part where the luminance value of a pixel is high in a captured image or a part where a characteristic shape is seen is set as a tracking point 100a, and the moving object 100 with respect to the tracking point 100a. A point separated by a predetermined length L in a direction rotated by a predetermined angle Δδ from the moving speed V direction is defined as a reference point 100b. In the present embodiment, the moving object 100 is tracked when the moving object 100 moves straight ahead while maintaining the inclination by the angle Δδ.

  Since the reference point 100b of the moving object 100 cannot be directly grasped, it is indirectly grasped based on the tracking point 100a. As shown in FIG. 1, the moving object tracking device 1 includes an imaging device 10, an image processor 20, a control processor 30, a drive mechanism unit 40, a reference point tracking angle calculation unit 50, and a distance measuring sensor 60. And a speed measuring instrument 70.

  The imager 10 is intended to image the tracking point 100 a on the moving object 100 and is mounted on the movable side of the drive mechanism unit 40. The imager 10 can capture the moving object 100 within the field of view by appropriately moving the drive mechanism 40. For example, an infrared camera is often used as the imaging device 10 when a part having a temperature difference compared to other parts is captured as the tracking point 100a. Image data including the image of the moving object 100 captured by the image capturing device 10 is sent to the image processor 20. The image processor 20 calculates a tracking error signal described later based on the image data. Further, a sensor signal including angle θ information in the moving object direction viewed from the moving object tracking device 1 is sent from the drive mechanism unit 40 to the control processor 30. Then, the moving object direction angle θ signal obtained by processing the sensor signal is sent from the control processor 30 to a reference point tracking angle calculator 50 described later.

  The coordinates of the tracking point 100a on the moving object 100 in the captured image, and the predetermined length L in the direction that is on the moving object 100 and rotated from the tracking point 100a by the predetermined angle Δδ from the moving speed V direction of the moving object 100. There is a difference from the coordinates on the captured image of the distant reference point 100b. FIG. 2 is a diagram schematically showing a captured image when the moving object 100 is captured in the field of view of the imager 10, and also shows a coordinate system. On the captured image, it is sufficient that an image having a luminance value capable of capturing at least the tracking point 100a on the moving object 100 is obtained, and an image having a luminance value capable of capturing the entire shape of the moving object 100 is sufficient. May not be obtained. As shown in FIG. 2, the difference between the tracking origin coordinates representing coordinates on the captured image in the visual axis direction of the imaging device 10 or the tracking reference direction and the coordinates on the captured image of the tracking point 100 a on the moving object is shown. The representing signal is a tracking error signal. The tracking error is shown separately for a component in the horizontal direction (X direction) of the captured image and a component in the vertical direction (Y direction) of the captured image.

  The tracking error signal calculated by the image processor 20 is sent to the control processor 30.

  The control processor 30 generates a drive signal corresponding to the tracking error signal and sends it to the drive mechanism unit 40. For example, as shown in FIG. 3 to be described later, the drive mechanism unit 40 can be configured by an amplifier / motor 40a, a drive mechanism 40b, an angular velocity sensor 40c, and an angle sensor 40d.

  When the drive signal for driving the drive mechanism unit 40 is generated in the control processor 30, the sensor signals of the angle sensor 40d and the angular velocity sensor 40c attached to the drive mechanism unit 40 are fed back and controlled at the same time. Thus, the imaging device 10 can be pointed in the direction of the moving object 100 and tracked more accurately and quickly.

  The distance measuring sensor 60 measures a relative distance R between the moving object 100 and the moving object tracking device 1. The relative distance signal obtained by the distance measuring sensor 60 is sent to the reference point tracking angle calculation unit 50.

The speed measuring device 70 measures the moving speed V ₀ of the moving object tracking device 1 itself. The moving speed V ₀ of the moving object tracking device 1 itself obtained by the speed measuring device 70 is sent to the reference point tracking angle calculation unit 50.

  FIG. 3 is a control block diagram in the embodiment, in which functions realized by the image pickup device 10, the image processor 20, and the control processor 30 are represented as an image tracking control system 30a and an angular velocity control system 30b, and a drive mechanism unit 40. It is a figure which shows the grant relationship of the signal. Information on the moving direction and angle of the moving object 100 is supplied to the image tracking control system 30 a as a tracking error signal calculated by the image processor 20 based on the image data captured by the imager 10. The tracking error signal obtained by the image tracking control system 30a is supplied to the angular velocity control system 30b. Sensor signals from the angle sensor 40d and the angular velocity sensor 40c are supplied to the angular velocity control system 30b, and a drive signal for driving the drive mechanism unit 40 by performing feedback control is generated and supplied to the drive mechanism unit 40.

  From the angular velocity control system 30b, information on the visual axis angular velocity described later is supplied to the reference point tracking angle calculation unit 50.

From the distance measuring sensor 60, a relative distance signal is supplied to the reference point tracking angle calculation unit 50. From the speed measuring device 70, the moving speed V ₀ of the moving object tracking device 1 itself is supplied to the reference point tracking angle calculation unit 50.

  The reference point tracking angle calculation unit 50 calculates a reference point tracking angle Δθ that is an apparent angle difference between the tracking point 100a and the reference point 100b.

  Here, the visual axis angular velocity and the tracking angular velocity Ω will be described.

  The visual axis angular velocity is an angular velocity at which the visual axis direction of the image pickup device 10 moves, and is calculated based on an angular velocity signal from an angular velocity sensor 40c mounted on the drive mechanism 40. Note that, depending on the method of the angular velocity sensor 40c and the attachment site, the calculation is also performed using an angle signal from the angle sensor 40d mounted on the drive mechanism 40.

  The tracking angular velocity Ω is the angular velocity of the moving object 100 as viewed from the moving object tracking device 1. If the visual axis direction of the image pickup device 10 is always directed to a certain point (tracking point) on the moving object 100 without any error, the tracking angular velocity Ω matches the visual axis angular velocity. There is a difference (tracking error). However, for example, when the tracking control system by the image tracking control system 30a and the angular velocity control system 30b is configured by a type 1 servo system, when the moving object 100 moves at a constant angular velocity, the tracking error is constantly The visual axis angular velocity is equal to the tracking angular velocity Ω. Therefore, if the moving object 100 moves at an equal angular velocity, or can be considered to move approximately or at a constant angular velocity for a short time, the tracking angular velocity Ω may be replaced with the visual axis angular velocity.

  Further, when the target value of the moving angular velocity of the driving mechanism 40 for tracking the moving object 100 is generated inside the tracking control system, the target value may be regarded as the tracking angular velocity Ω. For example, a case where a value proportional to the tracking error is set as a target value of the moving angular velocity of the drive mechanism 40 is applicable. In this case, fluctuations up to a frequency higher than the frequency band of the entire tracking control system can be followed. However, there may be a risk of being susceptible to errors and noise.

  On the other hand, when the time change rate of the moving angular velocity (tracking angular velocity Ω) of the moving object 100 is significant, a large delay occurs in the follow-up of the imaging device 10. Therefore, when calculating the reference point tracking angle Δθ, the reference point 100b can be obtained more accurately based on the information on the visual axis angular velocity that takes into account (corrects) the tracking delay of the imaging device 10. Here, detailed description of the follow-up delay correction is omitted.

  The reference point tracking angle calculation unit 50 calculates the reference point tracking angle Δθ based on the value of the visual axis angular velocity.

ここで，
L ： 移動物体上での画像追跡点と参照点との間の実際の長さ
Δδ ： 移動物体上での画像追跡点に対する参照点の方向の角度
（移動物体の移動速度方向基準）
Ω ： 移動物体追跡装置が移動物体を追跡する角速度
V ： 移動物体の移動速度
R ： 移動物体追跡装置と移動物体との間の相対距離
V₀ ： 移動物体追跡装置自身の移動速度
θ ： 移動物体追跡装置から見た移動物体方向の角度
（移動物体追跡装置自身の移動速度方向基準）
である。
なお，移動物体１００の移動速度Vの値は，0でないとする。 In the moving object tracking device 1 configured as described above, the reference point tracking angle calculation unit 50 calculates the reference point tracking angle Δθ by the following equation.

here,
L: Actual length between the image tracking point on the moving object and the reference point Δδ: Angle of the reference point direction with respect to the image tracking point on the moving object
(Movement speed direction reference for moving objects)
Ω: Angular velocity at which moving object tracking device tracks moving object
V: Moving speed of moving object
R: Relative distance between moving object tracking device and moving object
V ₀ : Movement speed of the moving object tracking device itself θ: Angle of moving object direction as seen from the moving object tracking device
(Moving speed direction standard of moving object tracking device itself)
It is.
It is assumed that the moving speed V of the moving object 100 is not 0.

  As shown in FIG. 2, since the reference point tracking angle Δθ can be expressed by an X direction component Δθx and a Y direction component Δθy, the reference point tracking angle Δθ is calculated by applying the above formula for each direction component. To do.

  The tracking angular velocity Ω at which the moving object tracking device 1 tracks the moving object 100 can be calculated by using the visual axis angular velocity of the imager 10 mounted on the movable part of the drive mechanism unit 40 as described above. The visual axis angular velocity can be obtained as the output of the control processor 30 based on the angular velocity of the drive mechanism 40 and the angle if necessary.

  After the reference point 100b on the moving object 100 is grasped as described above, for example, even if the moving object 100 continues to move when the light wave is irradiated in the direction of the reference point 100b, the moving object 100 is always on the moving object 100. The reference point 100b can be illuminated from the outside.

  Note that the tracking point 100a on the moving object only needs to be moved together with the moving object 100 without changing the relative positional relationship with respect to the moving object 100, and is not necessarily a point that actually exists on the moving object 100. No (for example, in the case where a region where hot combustion gas is injected is imaged and tracked with an infrared camera as a tracking point).

  In FIG. 1, the moving object 100 is shown to have a width in the direction orthogonal to the moving speed V direction. However, the moving object 100 is long and narrow, and the reference point 100b is longer than the tracking point 100a. The same relationship holds true when the moving object 100 is separated by a predetermined length L in the vertical direction and the speed direction of the moving object 100 is deviated by a predetermined angle Δδ with respect to the length direction of the moving object 100.

  In the above description, the tracking error signal is calculated by the image processor 20, but the image processor 20 calculates the coordinates of the tracking point 100a on the captured image and sends it to the control processor 30, and the tracking error signal is processed by the control processing. A method of calculating by the device 30 may be used.

(Modification 1 of embodiment)
In the first embodiment described above, the actual length L between the image tracking point 100a and the reference point 100b on the moving object 100 and the angle of the direction of the reference point 100b with respect to the image tracking point 100a on the moving object. The case where each value is known has been described.

  On the other hand, when the identity of the moving object 100 is not clear, the values of the length L and the angle are unknown. In such a case, it is possible to use estimated values for the length L and angle values from the specifications of the known moving object 100.

(Modification 2 of embodiment)
There is a case where the moving object tracking device 1 is always used in a stationary state. In such a case, as a configuration of the moving object tracking device 1, there is one that does not include the speed measuring device 70. In such a case, V ₀ = 0 in the above equations (1) and (2).

  As described above, according to the present embodiment, on the captured image, a predetermined length in a direction rotated by a predetermined angle from the moving speed direction of the moving object with respect to the point on the moving object being tracked (tracking point). The difference between the position of the remote point (reference point) on the captured image and the position of the tracking point on the captured image, that is, the reference point tracking angle can be calculated. Then, after the reference point 100b on the moving object 100 can be grasped, for example, even if the moving object 100 continues to move when the light wave is irradiated in the direction of the reference point 100b, the reference point 100b on the moving object 100 is always moved. Can be illuminated from the outside.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

1 ... Moving object tracking device,
10: Imager,
20: Image processor,
30 ... Control processor,
40: Drive mechanism,
50: Reference point tracking angle calculation unit,
60 ... Distance sensor,
70: Speed measuring device.

Claims (7)

An imager for imaging a predetermined part of a moving object as an image tracking point;
A drive mechanism for driving the imager;
An image processor for inputting image data of an image tracking point acquired by the image pickup device and calculating a tracking error representing a difference between the tracking origin coordinate for tracking in the picked-up image and the image tracking point coordinate of the moving object; ,
Based on the tracking error obtained by the image processor, a control processor that calculates a visual axis angular velocity in which the tracking delay of the imager with respect to the moving object is corrected to the tracking angular velocity of the imager;
A distance measuring sensor for measuring a relative distance between the moving object;
A speed measuring device that measures the moving speed of the moving object tracking device itself;
Based on the visual axis angular velocity obtained by the control processor, the relative distance obtained by the distance measuring sensor, and the moving velocity of the moving object tracking device itself obtained by the velocity measuring device, the movement from the image tracking point is performed. A reference point tracking angle calculation unit that calculates a reference point tracking angle for tracking a reference point that is separated by a predetermined length in a direction rotated by a predetermined angle from the moving speed direction of the object;
A moving object tracking device.   The moving object tracking according to claim 1, wherein the visual axis angular velocity is calculated based on one or both of a sensor signal of a directivity angle and a moving angular velocity of the drive mechanism and a tracking error obtained by the image processor. apparatus.   The reference point tracking angle is calculated from values of an actual length between a tracking point and a reference point on the moving object, and an angle of a reference point direction with respect to an image tracking point on the moving object. The moving object tracking device according to claim 1 or 2.   Each value of the actual length between the tracking point on the moving object and the reference point and the angle of the direction of the reference point with respect to the image tracking point on the moving object is an estimated value estimated from the known moving object The moving object tracking device according to claim 3, wherein:   The reference point tracking angle calculating unit uses the relative distance and the time change rate of the relative distance obtained by the distance measuring sensor, the moving speed of the moving object tracking apparatus itself, and the values of the visual axis angular speed and the moving object. The moving object tracking device according to claim 1, wherein the reference point tracking angle is calculated after calculating the moving speed of the moving object. Capture moving objects with an imager,
The coordinates on the captured image of the tracking point on the moving object are calculated based on the image data acquired by the imager, and the coordinates on the captured image in the direction of the visual axis of the imager or the reference direction for tracking are represented. Calculating a tracking error representing a difference between the tracking origin coordinates and the tracking point coordinates on the moving object;
Based on the tracking error, generates a drive signal to a drive mechanism unit that drives the imager to capture the moving object in the field of view of the imager;
A visual axis angular velocity is calculated based on one or both of a sensor signal of a directivity angle and a moving angular velocity from the drive mechanism unit, and the tracking error,
Measure the relative distance to the moving object,
Measure the moving speed when tracking the moving object,
Based on the visual axis angular velocity, the relative distance, and the moving speed when tracking the moving object, the tracking point on the moving object is separated by a predetermined length in a direction rotated by a predetermined angle from the moving speed direction of the moving object. Calculate a reference point tracking angle for tracking the reference point
Reference point tracking method for moving objects.   The reference point tracking angle is calculated after calculating the moving speed of the moving object using each value of the time change rate of the relative distance, the moving speed when tracking the moving object, and the visual axis angular velocity. 6. A moving object reference point tracking method according to 6.

Images