JP2873339B2 - Moving object recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving object recognition apparatus. A moving object recognition apparatus often employs a TV camera as an image input unit into which an image of an external object is input. That is, in many cases, a moving object is recognized by capturing an external object with a TV camera and digitally processing an input image signal. If the TV camera is used as the image input unit, visual information equivalent to humans can be input, so that the application range is extremely wide.

2. Description of the Related Art Generally, an image processing apparatus is a device capable of processing visual information similar to that of a human being, and is widely applied to an FA inspection, an unmanned monitoring device, and a visual device for an unmanned mobile vehicle. The potential demand is even higher, and research and development is being actively pursued. Although the image processing apparatus initially targeted still image processing, in recent years the emphasis has shifted to moving image processing.
In particular, the recognition of the moving object is performed by the entire image processing apparatus.
It came to occupy a large specific gravity.

[0003] In general, a moving object recognizing device is used in a situation where an image input unit of the moving object recognizing device and an object are relatively moving.
It recognizes the shape and relative movement of the moving object. That is, even when the object is not actually moving, if the image input unit of the moving object recognition apparatus is moving, the shape of the stationary object and the movement of the image input unit are recognized. For example, in a visual device for an unmanned mobile vehicle, the image input unit is mounted on the mobile vehicle, and the moving object recognition device recognizes the environment in which the mobile vehicle runs.

In a moving object recognizing device, small size and high speed are required. The compactness is important for mounting on a moving vehicle, particularly when applied to a visual device for an unmanned moving vehicle. In addition, as for high-speed processing, real-time processing similar to that performed by human eyes is required.

[0005]

2. Description of the Related Art A conventional moving object recognition apparatus captures an object using two image input units. The feature points of the object in the two images captured by the two image input units are associated with each other, and the shape of the object is obtained for each time point according to the principle of triangulation, and then the movement of the object is calculated.

FIG. 30 shows the concept of a conventional moving object recognition apparatus. The first image input unit 2 and the second image input unit 3 input an image of the object 1. The moving object recognizing unit 4 detects the feature points of the target object in the two images, and
After associating the images, the position of each feature point is calculated according to the principle of triangulation. Here, the feature point is a point indicating a specific portion of the target object, such as a vertex or an edge point, or a pattern point or a color boundary point when there is a pattern. The moving object recognizing unit 4 obtains
Further, the motion of the feature point and the motion of the object are calculated. The positions and movements of these feature points and the movement of the object are output as a recognition result 5.

FIG. 31 shows the configuration of a conventional moving object recognition apparatus. The image input from the first image input unit 2 has its feature points extracted by a first feature point extraction unit 6, and is input to a feature point association unit 8. Similarly, the image input from the second image input unit 3 has its feature points extracted by the second feature point extraction unit 7 and is input to the feature point association unit 8.
The feature points extracted in 6 and 7 are associated with the same feature point in a feature point association unit 8. In the feature point position calculation unit 9, the positions of the feature points are obtained based on the installation positions of the first image input unit 2 and the second image input unit 3 and the associated feature points, and the feature point position storage is performed. It is stored in the unit 10. The positions of the feature points at a plurality of time points stored in the feature point position storage unit 10 are sent to the object movement calculation unit 11, where the movement of the object is calculated, and the object movement storage unit 1
2 is stored.

[0008]

However, FIG.
The conventional moving object recognition device described in 0, 31
There were problems (a) and (b). (a) Since it is necessary to individually extract feature points from images input by the two image input units, the process of extracting feature points is as follows:
This is twice as large as when one TV camera is used. In FIG. 31, one feature point extraction unit is required for each image input unit. (b) Processing for associating feature points extracted from two images with different appearances is required. In FIG. 31, the feature point associating unit 8 is required. Since the installation positions of the two image input units are different, the appearance of the object is different, and it is difficult to associate the feature points of the object with each other. That is, the closer the position of the image input unit is, the easier it is to associate the feature points of the target object, but the lower the recognition accuracy of the shape of the target object is.

In a typical case where the processing is difficult in (b), the feature points of the object captured by one image input unit are not captured by the other image input unit, and the feature points are May be impossible. For example, although there are two feature points in FIG. 32, both feature points can be captured only by one image input unit, and cannot be captured by the other image input unit.

An object of the present invention is to recognize a moving object by extracting feature points at a plurality of times input from one image input unit.

[0011]

FIG. 1 is a block diagram showing the principle of the present invention. The figure uses only one image input unit, a feature point extraction unit 16 for extracting feature points in an image input from the image input unit 15, and a feature point storage unit for storing the extracted feature points. 17 is a block diagram showing the principle of a moving object recognizing device for recognizing the movement of a moving object by using known position information of the characteristic points, for example, X coordinates of a plurality of characteristic points of the moving object moving on the XY plane. It is.

In FIG. 1, the shape / motion recognition means 18
Is, for example, a shape / motion recognition unit. In an image obtained by capturing an object that moves on a plane while rotating from a direction perpendicular to the rotation axis, a right angle is extracted from the feature points of the object. Known position information at two time points of the three feature points to be formed,
For example, from the X coordinate of the feature point of the object moving on the XY plane, the actual position and movement of the feature point of the object in the three-dimensional space are calculated.

[0013]

The present invention is based on a new theory submitted and proved by the present applicant for the first time. According to this new theory, even if there is only one image input unit, if the feature points at several points can be extracted, the shape (position of the feature points) and the motion of the target object under certain limited conditions Indicates that calculation is possible. First, this new theory will be described in detail.

First, according to the present invention and the present theory, an object rotates around a fixed rotation axis, and the image input unit
It is assumed that the observation is made from the vertical direction of the rotation axis of the object. Under this assumption, there are many applications. For example, the object is various objects such as a side wall on a road,
This is a case where a TV camera is mounted on a self-propelled vehicle as an image input unit. In this case, relative to the image input unit that is the observation system
From a point of view, the object side wall rotates in any vertical direction
Treat as rotating and moving around an axis
Can be.

Further, the present invention is based on the premise that the objects have planes perpendicular to each other. This is based on the premise that three characteristic points are orthogonal to the projection plane perpendicular to the rotation axis of the object. It means being in a relationship.

Even if this premise is added, it can be applied in many cases. For example, when observing the side wall on the side of the road with the TV camera mounted on the above-described self-propelled vehicle, the side wall is applicable because the side wall is formed of surfaces having a right angle relationship. Not only the side wall but also the artificial building is composed of planes whose corners are in a right angle relationship, and can be applied to the case where observation is performed by a TV camera mounted on a self-propelled vehicle.

Further, the image from the image input unit is approximated by orthographic projection. The approximation by orthographic projection gives a good approximation in the case of radiography or when the image input unit is a TV camera and the angle of view of the object captured by the image input unit is narrow.
Further, the present theory assumes that three feature points of the object have a right-angled relationship on a projection plane perpendicular to the rotation axis of the object.

FIG. 2 shows the relationship between the object and the observation surface of the image input unit.
Shown in The Z axis on the image plane is set in the direction of the rotation axis of the target object, and the X axis is set in a direction perpendicular to the Z axis. That is, the X axis is a direction parallel to the movement of the feature point. The origin on the image plane is the point where the X axis and the Z axis intersect. The Y axis is perpendicular to the image plane and intersects the origin on the image plane. According to the orthographic projection condition, the XYZ axis coordinate system can be considered to translate in any direction.

Further, the distance in the depth Y-axis direction is unknown due to the orthogonal projection condition, and the movement amount in the X-axis direction is the same as the observation amount on the image plane. Therefore, as shown in FIG. 3, it can be considered that one feature point 0 is fixed to the origin O.

To calculate the shape and motion information of the target part,
The movement amount of one associated feature point 0 and the rotation amount around the feature point may be calculated. As described above, since the movement amount of the feature point 0 can be obtained clearly, the rotation amount around the feature point 0 may be calculated.

In the moving object recognition apparatus of the present invention, in addition to obtaining the X coordinate and the Z coordinate from the feature points of the input image,
The Y coordinate of the feature point when the feature point 0 is moved to the origin and the amount of rotation about the origin of the object are calculated.

In the present invention, the observation axis (X) of three feature points on a rigid body that moves while rotating in a two-dimensional space (XY)
The purpose is to recognize the motion and structure of the rigid body from the orthogonal projection information on the axis). However, it is assumed that the three feature points of the rigid body have an orthogonal relationship. Further, it is assumed that the correspondence of points on the X axis has been established.

The absolute distance in the depth direction (Y direction) cannot be restored in principle. Also, the amount of parallel movement in the observation axis direction is obvious. Therefore, the problem can be considered by moving the right-angle point of the three feature points on the rigid body to the origin of the coordinate axes.
That is, when the rigid body is rotating around the origin and the X coordinates of two points other than the origin are observed, there is a problem of obtaining the Y coordinate and the amount of rotation.

By associating three feature points on two screens, a moving object can be recognized (Theorem 1 described later). However, one of the feature points is taken on a right-angled edge and moves to the origin of the coordinate axis. The other two feature points are on the plane sandwiching the right angle point.
Strictly speaking, it is only necessary that the three feature points have a right angle relationship on the XZ plane.

In order to observe the condition of having a right-angled edge, three feature points are needed. Further, the shape of the moving object is not determined on one screen. Therefore, three feature points and two screens are also necessary conditions.

FIG. 4 shows that a solution cannot be determined even if a single screen has a right-angled edge. Feature point 1 and feature point 2
Are on the straight line of the observed X coordinate. If the feature point 1 is arbitrarily determined, one position of the feature point 2 that satisfies the condition is determined. This is because the triangle formed by the feature point 1, the origin (the feature point 0), and the feature point 2 may be a right triangle. That is, there are countless combinations of feature point 0 and feature point 1 that satisfy the condition.

FIG. 5 shows a situation where the actual position and movement of two characteristic points other than the origin of the object are obtained from the orthogonally projected points on the screen axis at two times. The symbols in FIG. 5 will be described.

Feature point 0: a feature point on the object that has moved to the origin O. Feature point 1, Feature point 2: Feature points on the object other than feature point 0. u ₁ , u ₂ : two-dimensional vectors on the XY plane from the origin at the first time point to the feature points 1 and 2 (the same order) v ₁ , v ₂ : from the origin at the second time point to the feature points 1 and 2 Two-dimensional vector on XY plane (in order)

[0029]

[Equation 11]

Rotation R: Two-dimensional rotation on the XY plane about the origin from the first time point to the second time point of the object.

[0031]

(Equation 12)

U ₂ is a vector obtained by rotating u ₁ by π / 2,
Or a vector rotated by -π / 2. The solution in these two cases has the relationship of Lemma 1. [Lemma 1] The following solutions (1) and (2) are obtained by mirror-inverting the other with respect to the X axis. (1) solutions when u ₂ is a vector obtained by rotating [pi / 2 and u ₁ (2) solutions when u ₂ is a vector the u ₁ is rotated - [pi] / 2 Figure 6 is thus Two sets of solutions that are mirror-inverted with respect to the X axis are shown. This is the feature point 0 in the orthogonally projected image.
At the origin, there are cases where the Y coordinate of other feature points becomes positive as shown in FIG. 6 and cases where it becomes negative. This also corresponds to the fact that the surface of the object is either convex or concave when viewed from the TV camera side as the image input unit.

In the present invention, the problem is solved by assuming that u ₂ is a vector obtained by rotating u ₁ by π / 2, and the solution is set as a recognition solution <1>. U ₂ turns u ₁ by −π / 2. The solution (recognition solution <2>), which is a vector obtained, is obtained by mirror-inverting the former solution with respect to the X axis.

Therefore,

[0035]

(Equation 13)

[0036] Also,

[0037]

[Equation 14]

[0038] It is. When these are related, v _i = Ru _i (i =
1, 2,). Therefore, the problem can be formulated as follows.

[Recognition of an object having a right-angled edge on a plane] u _i (i = 1, 2); two-dimensional vector, first component is known u ₂ is a vector obtained by rotating u ₁ by π / 2 R: 2 dimension rotation matrix _{v i (i = 1,2);} 2 -dimensional vector, when the first component is known _{v i = Ru i (i =} 1,2) this, u _i, v _i of (i = 1, 2) Find the second component and R.

The second component of v _i (i = 1, 2) is R and u
_i from the equation v _i = Ru _i, obtained immediately. Therefore,
It is sufficient to find the second component of u _i (i = 1, 2) and R. θ, d ₁ , d ₂ , and α may be obtained.

The following Theorem 1 gives a condition under which the problem can be solved. [Theorem 1] The conditions for solving the above motion / structure recognition problem are
v ₁₁ ≠ ± u ₁₁ or v ₂₁ ≠ ± u ₂₁ .

... With respect to coordinates observable from the X axis, it is possible to determine whether or not a solution can be uniquely obtained by determining.

The following Theorem 2 gives the meaning of the condition of Theorem 1. [Theorem 2] The condition of Theorem 1 is the same as the following condition.

[0044]

U ₁₁ v ₂₁ + u ₂₁ v ₁₁ ≠ 0... Means that the rotation angle of the object is not 0 degrees or 180 degrees. In order to clarify the meaning of, consider the case of α + θ = nπ−α, which is not the case. α + θ is 2π
(360 degrees) Since the state is the same even when shifted, the case of α + θ = −α and the case of α + θ = −α + π may be considered. These states are shown in FIG.

In the case of α + θ = −α, the feature point 1 moves symmetrically about the X axis and the feature point 2 moves symmetrically about the Y axis. When α + θ = -α + π, α + θ
= -Α when rotated 180 degrees.

Is used in Theorem 3. Next, θ, d ₁ , d
_2. Begin to derive the formula for α. In order to perform a specific numerical calculation, first, a range of values of α and sinθ is narrowed down.

In Lemma 2, the value of α is narrowed down. [Lemma 2] u _i (i = 1, 2); a two-dimensional vector from the origin, and u ₂ ; a vector obtained by rotating u ₁ by π / 2, the rotation angle of u _{1 from} the X axis The value of α is u _i (i =
The sign of the X coordinates u ₁₁ and u ₂₁ of (1, 2) is determined within the range of π / 2 as shown in FIG.

In Lemma 2, by replacing u ₁ with v ₁ and α with β, the value of β = α + θ is also narrowed down using FIG. In Lemma 3, using the result of Lemma 2,
The value of sinθ is narrowed down.

[Lemma 3] From FIGS. 8 and 9, (π / 2) m ≦
β <(π / 2) (m + 1), (π / 2) n ≦ α <(π /
2) Integers m and n (= 0, 1, 2) satisfying the two equations (n + 1)
2, 3) are obtained.

By using the comparison between the values of m and n and the values of u ₁₁ and v ₁₁ , the sign of sin θ can be determined from the following (1) and (2). (1) When mn is an odd number p = (mn-1) / 2. p is an integer.

The sign of sin θ is determined from FIG. (2) When mn is an Even Number The sign of sin θ is determined from FIG. When the sign is satisfied in the conditional expression, sinθ = 0.

Under the preparation of the lemmas 2 and 3, theorem 3 which gives a formula for calculating the solution can be shown. [Theorem 3] Under the conditions of Theorems 1 and ₂ , u ₂ reduces u ₁ by π / 2.
The solution for the rotated vector can be calculated as follows:

Θ, d ₁ , d ₂ and α can be calculated by the following equations. cos θ = (u ₁₁ u ₂₁ + v ₁₁ v ₂₁ ) / (u ₁₁ v ₂₁ + u ₂₁ v
₁₁ ) sin θ = ± (1−cos ² θ) ^1/2 , and the sign is determined by Lemma 3. (1) When u ₁₁ ≠ 0 and u ₂₁ ≠ 0 When tan α = (u ₁₁ cos θ−v ₁₁ ) / u ₁₁ sin θ, and the range of α obtained from Lemma 2, the value of α is determined to be one.

D ₁ = u ₁₁ / cos α d ₂ = −u ₂₁ / sin α (2) In the case of u ₁₁ = 0

[0056]

(Equation 15)

(3) When u ₂₁ = 0

[0058]

(Equation 16)

A solution in which u ₂ is a vector obtained by rotating u ₁ by −π / 2 is obtained by inverting the solution of Theorem 3 with respect to the X axis as described in Lemma 1. System 3.1
Describes this concretely.

[System 3.1] The solution in which u ₂ is a vector obtained by rotating u ₁ by −π / 2 can be obtained as shown in (1) and (2).

[0061] (1) a solution u ₂ of Theorem 3 is a vector obtained by rotating [pi / 2 to _{u 1, R, u i,} v i (i = 1,
Assuming 2), the solution that is a vector rotated by -π / 2 is

[0062]

[Equation 17]

Is as follows. (2) If the solution in which u _{2 in} Theorem 3 is a vector obtained by rotating u ₁ by π / 2 is θ, d ₁ , d ₂ , α, the solution in which −2 is rotated by −π / 2 is − θ, d ₁ , d ₂ , −α.

Next, the proof of Lemma 1-3, Theorem 1-3, and Corollary 3.1 will be described. Solution when [proof of Lemma 1] (1) u ₂ is a vector obtained by rotating [pi / 2 to u ₁ a (2) u ₂ is u ₁ in solution Figure 6 when a vector obtained by rotating - [pi] / 2 As shown, the solutions (1) and (2) are obtained by mirror-inverting the other with respect to the X axis.

[0065]

(Equation 18)

Since v ₁ and v ₂ are vectors obtained by rotating u ₁ and u ₂ by θ,

[0067]

[Equation 19]

U _i , v _i (i = 1, 2); The condition that the first component is known is expressed by the following equations (1) to (4). d ₁ cos α = u ₁₁ (1) -d ₂ sin α = u ₂₁ (2) d ₁ cos (α + θ) = v ₁₁ (3) -d ₂ sin (Α + θ) = v ₂₁ (4) In (1) to (4), the right side is a known value.

For the proofs of Lemma 2-3 and Theorem 1-3,
The following Proposition 1, Theorem 2, Lemma 2-3, and Theorem 3 are shown in this order. [Proposition 1] In the theorem 2,, is a necessary condition for the solution to be determined.

[Proof of Proposition 1] It is shown that the solution is not determined unless (1) is a proof that it is a necessary condition .

Substituting θ = nπ into (3) and (4), ± d ₁ cos α = v ₁₁ (3) ′ − + d ₂ sin α = v ₂₁ (4) The signs in '(3)' and (4) 'are the same.

(3) 'and (4)' are obtained from (1) and (2) as follows:
(3) ′, (4) ′, (1) to (4) can be transformed to (1)
, (2), (3) ″, (4) ″. Therefore, (1) to (4)
Is equivalent to (1), (2), (3) ″, (4) ″.

D ₁ cos α = u ₁₁ (1) −d ₂ sin α = u ₂₁ (2) v ₁₁ = ± u ₁₁ (3) ″ v ₂₁ = ± u ₂₁ ... (4) ″ (3) ″, (4) ″ have the same sign.

That is, the conditional equations are (1) and (2), the unknown variables are d ₁ , d ₂ and α, and the solution is not determined. This shows that the solution is not determined unless (2) is a proof that it is a necessary condition .

Α + θ = nπ-α (n: integer) is expressed by (3),
(4), d ₁ cos (α + θ) = ± d ₁ cos α−d ₂ sin (α + θ) = ± d ₂ sin α ± d ₁ cos α = v ₁₁ (3) ′ ± d ₂ sinα = v ₂₁ ... (4) ′ Therefore, (1) to (4) are equivalent to (1), (2), (3) ″, (4) ″.

D ₁ cos α = u ₁₁ (1) −d ₂ sin α = u ₂₁ (2) v ₁₁ = ± u ₁₁ (3) ″ v ₂₁ = − + u ₂₁ ... (4) ″ That is, the conditional equations are (1) and (2), the unknown variables are d ₁ , d ₂ and α, and the solution is not determined.

[End of Proof] [Proof of Theorem 2] First, it is shown that "or, or or is equivalent."

Prove the even number of this proposition. The even number of the proposition is v ₁₁ = ± u ₁₁ , v ₂₁ = ± u ₂₁ (in no particular order) ⇔θ = nπ
Or α + θ = nπ−α

[0079]

(Equation 20)

Are shown by the following (1) and (2). It is proved in (3) that and are equivalent. (1) v ₁₁ = ± u ₁₁ , v ₂₁ = ± u ₂₁ (same order) ⇔ θ =
(3) ″ and (4) ″ are derived from the proof (1) of nπ proof (←) Proposition 1. (→) Use the formula number in Proof 1 (1).

(3) "and (4)". Deformation using (1) and (2) yields (3) 'and (4)'. Combined with (3) and (4),

[0082]

(Equation 21)

When sin θ is calculated using this equation, sin θ = sin ｛(α + θ) −α｝ = sin (α + θ) cosα−cos (α + θ) sinα = ± ｛sinα cosα−cosα sinα｝ = 0 Therefore, θ = Nπ (2) v ₁₁ = ± u ₁₁ , v ₂₁ = − + u ₂₁ (the same applies) ⇔ Proof of α + θ = nπ−α (←) Proof of Proposition 1 (2). (→) Proposition 1
Use the formula number in the proof (2).

Assuming (3) ″ and (4) ″, (3) ′ and (4) ′ hold. Combined with (3) and (4),

[0085]

(Equation 22)

[0086] (3) When a is substituted into prove _{u 11 v 21 + u 21 v} 11 that are equivalent to (1) ~ (4), u 11 v 21 + u 21 v 11 = -d 1 d 2 {cosα sin (α + θ) + sinα cos (α
_{+ Θ)} = -d 1 d} 2 sin (2α + θ) α + θ = nπ-α (n; integer) ⇔2α + θ = nπ (n ; integer) ⇔ sin (2α + θ) = 0 ⇔u 11 v 21 + u 21 v 11 = 0 α + θ ≠ nπ-α (n; integer) ⇔u ₁₁ v ₂₁ + u ₂₁ v ₁₁ ≠ 0 [End of proof] [Proof of Lemma 2] When the value of α is α = 0,0 <Α <π / 2, α
= Π / 2, π / 2 <α <π, α = π, π <α <3/2
When π and α = 3 / 2π, 3 / 2π <α <2π, FIG. 8 is obtained by examining the signs of u ₁₁ and u ₂₁ respectively.

[End of Proof] [Proof of Lemma 3] (π / 2) m ≦ β <(π / 2) (m +
1), (π / 2) n ≦ α <(π / 2) (n + 1) m and n are integers and satisfy 0 ≦ m ≦ 3 and 0 ≦ n ≦ 3.

Since θ = β−α, (π / 2) (mn) −π / 2 <θ <(π / 2) (mn) + π / 2 (1) ) When mn is an odd number mn = 2p + 1, p; can be written as an integer.

That is, p = (mn-1) / 2. Substituting this into (*), pπ <θ <(p + 1)
Therefore, FIG. 10 is established. (2) In the case where mn is an even number From 0 ≦ m ≦ 3, 0 ≦ n ≦ 3, −3 ≦ mn ≦ 3, but since mn is an even number, mn = 0, It can be classified into three cases of 2 and -2.

For these three cases, m and n respectively
12 are obtained by obtaining conditions for determining the sign of sin θ. In the conditional expression of FIG. 12, when the sign is satisfied, sin θ = 0.

FIG. 11 is a summary of this drawing. Hereinafter, a process for obtaining FIG. 12 will be described. Basically, m
According to the values of n and n, the position range of u ₁ and v ₁ is schematically represented,
The magnitude relation between u ₁₁ and v _{11 that} can restrict θ was obtained. (a) When mn = 0 From (*), -π / 2 <θ <π / 2. 13 in FIG.
May be street.

The value of θ can be restricted by comparing the values of u ₁₁ and v ₁₁ . (b) When mn = 2 From (*), π / 2 <θ <3π / 2. 14-2
May be street.

The value of θ can be restricted by comparing the values of | u ₁₁ | and | v ₁₁ |. (c) When mn = −2 From (*), −3π / 2 <θ <−π / 2. FIG.
There are two cases. The value of θ can be restricted by comparing the values of | u ₁₁ | and | v ₁₁ |.

[End of proof] [Proof of Theorem 3] When (1) and (3) are multiplied by crossing, u ₁₁ cos (α + θ) = v ₁₁ cos α (5) (2) When (4) is multiplied by crossing, u ₂₁ sin (α + θ) = v ₂₁ sin α (6) (1) When u ₁₁ ≠ 0, u ₂₁ ≠ 0 From FIG. Since cosα ≠ 0 and sinα ≠ 0 (5) are expanded and both sides are divided by cosα, u ₁₁ (cos θ−tan α · sin θ) = v ₁₁ −u ₁₁ tan α · sin θ = v ₁₁ -u ₁₁ expand cosθ ····· (5) '(6 ), dividing both sides by _{sinα, u 21 (cosθ + cotα} · sinθ) = v 21 u 21 cotα · sinθ = v 21 -u 21 cosθ ····· (6) '(5 )', (6) multiplying both sides of ^{_{', -u 11 u 21 sin 2}} θ = (v 11 -u 11 cosθ) (v 21 -u 21 cosθ) _{(u 11 v 21 + u 21} v 11) cosθ = u 11 u 21 + v 11 v 21 ····· (7) u 11 v 21 because it is _{u 21 v 11 ≠ 0 ····· (} 5) cosθ = (u 11 u 21 + v 11 v 21) / (u 11 v 21 + u 21 v 11) ····· (8) sinθ = ± (1−cos ² θ) ^1/2 (9) The sign of (9) is obtained from Lemma 2.

Since (5) ′ and sin θ ≠ 0, tan α (u ₁₁ cos θ−v ₁₁ ) / u ₁₁ sin θ When combined with the value range of α obtained from Lemma 2, the value of α is determined to be one.

From (1), d ₁ = u ₁₁ / cos α (10) From (2), d ₂ = −u ₂₁ / sin α (11) (2) When u ₁₁ = 0 If

[0097]

(Equation 23)

Cos α = 0 and sin α = ± 1.
(1) are substituted into ~ (4), 0 = u 11 ····· (1) '- + d 2 = u 21 ····· (2)' - + d 1 sinθ = v 11 ·· ··· (3) - = '+ d 2 cosθ = v 21 ····· (4)' (7) of the left side = d ₁ d ₂ sinθ cosθ follow the right-hand side of (7), also in this case (7) Holds, and (8) and (9) hold.

In (3) ′, since sin θ ≠ 0 (∵), d ₁ = | v ₁₁ / sin θ | (10) ′ (2) ′ gives d ₂ = | u ₂₁ | ... (11) ' (3) When u ₂₁ = 0

[0100]

(Equation 24)

Cos α = ± 1, sin α = 0, which is
Substituting into (1) to (4), ± d ₁ = u ₁₁ ... (1) ″ 0 = u ₂₁ ... (2) ″ ± d ₁ cosθ = v _11. (3) ″ − + d ₂ sin θ = v ₂₁ ... (4) ″ Left side of (7) = − d ₁ d ₂ sin θ cos θ = right side of (7) Therefore, also in this case, (7) is Holds, and (8) and (9) hold.

(1) ″ From d ₁ = | u ₁₁ | (10) ″ (4) ″ Since sin θ ≠ 0 ((), d ₂ = | v ₂₁ / sin θ | (11) ″ [End of proof] The new theory used in the present invention has been described in detail above. The shape / motion recognition means 18 shown in FIG. By applying the new theory, the actual positions and movements in the three-dimensional space of these feature points are calculated based on the extraction results of the three feature points forming a right angle at the two points in the image, and the moving object is calculated. Can be recognized.

[0103]

FIG. 16 is an explanatory view of the concept of the moving object recognition apparatus of the present invention. In this embodiment, the movement of the object is recognized using only one image input unit.
It is assumed that the image input unit is moving while rotating around two rotation axes, for example, the Z axis, and that the image input unit is observing the object from a direction perpendicular to the rotation axis. As an example of this situation, there is a case where the object is a side wall on the side of a road, and a TV camera is mounted on a self-propelled vehicle as an image input unit. In this case, relative to the image input unit that is the observation system
In the figure, the object side wall is an arbitrary vertical rotation axis.
Can be considered as moving while rotating around
it can.

In FIG. 16, the image of the object 20 is input to the image input unit 21 and sent to the moving object recognizing unit 22.
The moving object recognition unit 22 extracts feature points of the target object from the image,
The shape (position of the feature point) and the movement of the object are recognized from the positions of the feature points at a plurality of time points. The recognition result is output from the moving object recognition unit 22 as a recognition result 23.

FIG. 17 is a block diagram showing the overall configuration of an embodiment of the moving object recognition apparatus according to the present invention. In the figure, an input image input to an image input unit 24 is output to a feature point extraction unit 25, and the feature point extraction unit 25 extracts feature points from the input image, and the position information of the feature points in the image is Feature point storage unit 26
Is output to The known position information of the feature points at a plurality of time points stored in the feature point storage unit 26 is output to the shape / motion recognition unit 27, and the shape / motion recognition unit 27 outputs the plurality of feature points on the actual three-dimensional coordinate space. Calculates the position and movement of the object at
The resulting movement of the object is stored in the object movement storage unit 28, and the three-dimensional position of the feature point is stored in the feature point position storage unit 29.

In FIG. 17, the shape / motion recognition unit 27 calculates the actual position of the feature point and the motion of the object using the new theory described above. FIG. 18 is a detailed configuration block diagram of the shape / motion recognition unit 27. In FIG. 17, a shape / motion recognition unit includes a known information input unit 31 to which known information relating to the movement of an object from a sensor is input.
In, the position of the feature point stored in the feature point storage unit 26 in the image is input, and when a feature point that is a right-angled vertex among the three feature points is set at the origin on the three-dimensional coordinate space, the other feature points are obtained. A feature point position normalization unit 32 for obtaining a relative position of a point, that is, a normalized position, and a shape for determining whether or not the motion of the target object can be recognized using the output of the feature point position normalization unit 32 When the motion and motion determining unit 33 and the shape and motion determining unit 33 determine that the motion of the object is recognizable, the rotation of the object around the rotation axis is performed using the output of the feature point position normalizing unit 32. The motion calculation unit 34 and the motion calculation unit 3 that calculate the amount
4 and the output of the feature point position normalization unit 32, the shape calculation unit 35, the motion calculation unit 34, and the shape calculation unit 35 that determine unknown position information of feature points other than the feature point located at the origin.
And a feature point that outputs the movement of the object to the object movement storage unit 28 using the output of the feature point position normalization unit 32 and outputs the position of the feature point in the three-dimensional coordinate space to the feature point position storage unit 29. It comprises a position restoring unit 36.

FIGS. 19 to 21 are flow charts of an embodiment of the whole processing of the shape / motion recognition section 27 in FIG. When the process is started in the figure, in step (S) 40, it is determined whether or not any one or more of formulas (1) to (5) has been input to the known information input unit 31 in FIG. Here, for example, it is generally the case that the formula is first determined to be satisfied after the calculation of the rotation matrix R, but various sensors not described here are attached to the moving object recognizing device of the present invention. Based on the signal from the sensor, it is determined whether the rotation of the object from the first time point to the second time point is 0 degree or 180 degrees, for example. Note that, for example, dashes in the formulas 'to' are shown in Theorem 2
This corresponds to rewriting the expression using a sign.

If it is determined in S40 that any one or more of the expressions '〜' has been input, the known information input unit 31 outputs a non-activation signal to the feature point position normalizing unit 32 in S41. At the same time, the unrecognizable information is stored in the motion calculating unit 34, and the process proceeds to the unrecognizable process.

FIG. 22 is a flowchart showing an embodiment of the unrecognizable process. When the unrecognizable process is started, first, in S42, the unrecognizable information is transferred from the motion calculating unit 34 to the feature point position restoring unit 36. The unrecognizable information is stored in the point position storage unit 29, and the process ends.

If it is determined in S40 of FIG. 19 that none of the formulas (1) to (4) has been input to the known information input unit 31, the three feature points on the screen at two time points are stored in the feature point storage unit 26 in S44. It is determined whether or not the position of the point has been stored. If the position has not been stored yet, the determination in S44 is continued. If it is determined that the data is stored, a start signal is sent from the feature point storage unit 26 to the feature point position normalization unit 32 in S45, and the feature point position normalization unit 32 is started.

Subsequently, in S46 of FIG. 20, the positions on the screen of the three feature points 0, 1 and 2 stored in the feature point storage unit 26 by the feature point position normalization unit 32 are stored. but u _i, v first component of _i (i = 1, 2) as the X coordinate minus the screen coordinates of the feature point 0 from the other screen coordinates of the feature point 1 and 2 such that the origin is determined. S
At 47, the shape and motion determination unit 3 for these first components
3 or whether the equation holds. If neither holds, the unrecognizable information is sent from the shape and motion determining unit 33 to the motion calculating unit 34 in S48, and the process proceeds to the unrecognizable process in FIG.

As explained in Theorem 2, or the expression is the same as the expression, the check in S47 and the S
Checking at 40 can be said to be double. However,
Since the check in S40 can be performed by the sensor,
Such processing is performed. Shape and motion determination unit 33
If it is determined that the recognition is not possible, the subsequent calculation is not performed.

When it is determined in S47 that one of the equations is satisfied, the motion calculating unit 34 is activated by the shape and motion determining unit 33 in S49, and the processing shifts to the processing of the motion calculating unit and the processing of the shape calculating unit. . These processes are performed by using Lemma 3 and Theorem 3.

FIG. 23 is a flowchart of a processing example of the motion calculation unit. When the process is started in FIG.
At 50, an integer n that determines the range of α is determined using FIG. 8, and at S51, an integer m that determines the range of β is determined using FIG. 9, and the process proceeds to the process of determining the sign of sin θ.

FIG. 24 is a flow chart of the embodiment of the sign determination processing of sinθ. Referring to FIG.
It is determined whether n is an odd number or an even number. If it is an odd number, the value of an integer p is obtained in S53, and the sign of sin θ is determined in S54 according to whether the integer p is an even number or an odd number using FIG. Determined. When the integer mn is an even number in S52, S5
5, the sign of sin θ is determined using FIG.

Subsequently, in S56 of FIG. 23, all components of the rotation matrix R are obtained. In S57, R and one of its inverse rotation matrices satisfy the known information of the object input to the known information input unit 31. It is determined whether or not it is satisfied. If the condition is not satisfied, the process proceeds to a non-recognizable process in FIG. If the condition is satisfied, the information of the angle α and the matrix R is sent from the motion calculation unit 34 to the shape calculation unit 35 and the feature point position restoration unit 36 in S58.

The known information on the object is known information on the movement of the object, which is input from another sensor or the like which is not described in the moving object recognition apparatus of the present invention as described above. For example, when an object is stationary on a plane and the T
Assuming a situation in which the V camera moves on the same plane, the moving vehicle may slip, and the movement of the TV camera is not exactly known, but the direction of the relative movement of the moving vehicle with respect to a stationary object is known. I do. The relative motion direction of the stationary object is input to the external memory as known information of the object, and is provided to the known information input unit 31 via the external memory. The motion calculation unit obtains two solutions of a rotation matrix R and its inverse rotation matrix R ', and one of the solutions is selected as a correct solution according to the constraint condition of the relative motion direction.

FIG. 25 is a flowchart of a processing example of the shape calculating unit. When the process is started in FIG.
At 0, it is determined whether both the values of u ₁₁ and u ₂₁ are not 0, and if both are not 0, the process proceeds to the shape calculation process.

FIG. 26 is a flowchart of an embodiment of the shape calculation process. Referring to FIG.
_The angle α at which ₁ forms the X axis is obtained, and the absolute values d ₁ and d ₂ of the two-dimensional vectors u ₁ and u ₂ are obtained in S62.

If either u ₁₁ or u ₂₁ is 0 in S60 of FIG. 25, it is determined in S63 whether u ₁₁ is 0. If it is 0, the flow shifts to shape calculation processing. FIG. 27 is a flowchart of the embodiment of the shape calculation process. In the figure, it is determined in S64 whether u ₂₁ is smaller than 0,
If it is smaller, α is set to π / 2 in S65, and if it is not smaller, α is set to 3π / 2 in S66, and the absolute values d ₁ and d ₂ of the two-dimensional vector are obtained in S67.

[0121] u ₁₁ in S63 of FIG. 25 shifts to shape calculation process when not 0 is determined. FIG. 28 is a flowchart of the embodiment of the shape calculation process. In the figure, if u ₁₁ at S70 is 0 is determined is greater than or not is, alpha in S71 if large 0, not larger S7
After α is set to π in ₂ , the values of d ₁ and d ₂ are obtained in S73.

In FIG. 25, after execution of the shape calculation processing or the second component of the two-dimensional vectors u ₁ and u ₂ is calculated in S74, the second components of v ₁ and v ₂ are calculated in S75, and these are calculated in S76. Is the feature point position restoration unit 3
Sent to 6.

In FIG. 20, when the processing up to the processing of the shape calculation unit 35 is executed, the processing shifts to the processing of FIG. In FIG. 21,
In S77, the feature point position restoring unit 36 determines the transferred solution as a recognition solution <1>, determines a recognition solution <2> in S78, and determines in S79 the known information among the recognition solutions <1> and <2>. It is determined whether or not there is something that satisfies the known information input to the input unit 31, and if there is no such solution, the process proceeds to a non-recognizable process.

In order to explain an example of the known information here,
A TV set with an object stationary on a plane and mounted on a mobile vehicle
Consider a situation where the camera moves on the same plane. Here, it is assumed that it is known in advance whether the surface shape of the stationary object viewed from the TV camera side is convex or concave. For example, a telephone pole has a convex surface shape. The unevenness information of this stationary object is
The known information of the object is provided to a known information input unit via an external memory or the like. In the shape calculation unit, two solutions having a mirror relationship with respect to the observation surface are obtained, but only one solution is selected as a correct solution according to the unevenness information of the object.

FIG. 29 is a flowchart showing an example of the unrecognizable process. In the figure, the unrecognizable information is stored in the object motion storage unit 28 and the feature point position storage unit 29 from the feature point position restoration unit 36 in S81, and the process ends.

If there is a solution that satisfies the known information in S79 of FIG. 21, such a solution is selected, and the rotation matrix R of the selected recognition solution and the feature point are obtained from the feature point position restoration unit 36 in S82. The coordinates of the feature point 0 on the screen stored in the position normalization unit 32 are stored in the object motion storage unit 28, and S8
3 and the feature points 0, 1, and 2 stored in the feature point position normalizing unit 32.
Are stored in the feature point position storage unit 29, and the process is terminated.

In this embodiment, the recognition solution <1> is first obtained, but there is another embodiment in which the recognition solution <2> is first obtained.
To do so, a recognition solution <2> can be obtained by performing the above processing with the direction of the X axis reversed. The recognition solution <1> is obtained by mirror-inverting the solution of the recognition solution <2> with respect to the X axis.

As described above, in the present invention, the movement of the object can be recognized from the positions on the screen of the three characteristic points forming a right angle at two points in time of the object moving on the plane while rotating. . Also, as described with reference to FIG. 20, the shape and motion determination unit determines the feature point from the position on the screen.
Whether or not motion can be recognized can be determined immediately.

[0129]

As described above in detail, according to the present invention, the shape and motion of an object are determined based on feature points extracted from an image of the object captured by one image input unit. Be recognized. Accordingly, the number of image input units and the number of feature point extraction units can be halved as compared with the case where two image input units are used, and the apparatus scale can be reduced.
Further, it is not necessary to associate feature points on the image of the object captured from different positions as in the case of using two image input units, and this processing can be omitted, and the processing time can be reduced. Is big.

[Brief description of the drawings]

FIG. 1 is a principle block diagram of the present invention.

FIG. 2 is a diagram illustrating a general relationship between an object and an observation surface of an image input unit.

FIG. 3 is a diagram illustrating a relationship between an object and an observation surface when a feature point 0 is fixed to an origin.

FIG. 4 is a diagram illustrating the existence of countless solutions on one screen.

FIG. 5 is a diagram showing orthogonally projected points of feature points on an XY plane.

FIG. 6 is a diagram illustrating an example of two solutions that are mutually mirror-inverted.

FIG. 7 is a diagram illustrating the meaning of an expression.

FIG. 8 is a diagram illustrating a method of determining a range of an angle α.

FIG. 9 is a diagram illustrating a method of determining a range of an angle β.

FIG. 10 is a diagram illustrating a method of determining the sign of sin θ when mn is an odd number.

FIG. 11 is a diagram for explaining a method of determining the sign of sin θ when mn is an even number.

FIG. 12 is a diagram showing the relationship between m, m and the sign of sin θ when mn is an even number.

FIG. 13 is a diagram for explaining sign determination of sin θ when mn = 0.

FIG. 14 is a diagram for explaining sign determination of sin θ when mn = 2.

FIG. 15 is a diagram illustrating the sign determination of sin θ when mn = -2.

FIG. 16 is a diagram illustrating the concept of the moving object recognition device of the present invention.

FIG. 17 is a block diagram showing the overall configuration of the moving object recognition device of the present invention.

FIG. 18 is a block diagram illustrating a configuration of an embodiment of a shape / motion recognition unit.

FIG. 19 is a flowchart (part 1) of an overall processing example of a shape / motion recognition unit.

FIG. 20 is a flowchart (part 2) of the overall processing example of the shape / motion recognition unit.

FIG. 21 is a flowchart (part 3) of an overall processing example of the shape / motion recognition unit.

FIG. 22 is a flowchart illustrating an example of an unrecognizable process.

FIG. 23 is a flowchart of a processing example of a motion calculation unit.

FIG. 24 is a flowchart of an embodiment of a sign determination process for sin θ.

FIG. 25 is a flowchart of a processing example of a shape calculating unit.

FIG. 26 is a flowchart illustrating an example of a shape calculation process.

FIG. 27 is a flowchart illustrating an example of a shape calculation process.

FIG. 28 is a flowchart illustrating an example of a shape calculation process.

FIG. 29 is a flowchart of an embodiment of a non-recognizable process.

FIG. 30 is a conceptual diagram of a conventional moving object recognition device.

FIG. 31 is a block diagram illustrating a configuration of a conventional moving object recognition device.

FIG. 32 is a diagram illustrating an example of a feature point that cannot be associated with a conventional moving object recognition device.

[Explanation of symbols]

 15, 21, 24 Image input unit 16, 25 Feature point extraction unit 17, 26 Feature point storage unit 18 Shape / motion recognition unit 27 Shape / motion recognition unit 28 Object motion storage unit 29 Feature point position storage unit 31 Known information input Unit 32 feature point position normalization unit 33 shape and motion determination unit 34 motion calculation unit 35 shape calculation unit 36 feature point position restoration unit

──────────────────────────────────────────────────続 き Continued on front page (58) Field surveyed (Int.Cl. ⁶ , DB name) G06T 7/20

Claims (16)

(57) [Claims] 1. A feature point extraction unit (16) for extracting feature points in an image input from an image input unit (15), and a feature point storage unit (17) for storing the extracted feature points. A moving object recognizing device for recognizing a movement of a moving object using known position information of the feature point, wherein the moving object moves on a plane relative to the image input unit (15) while rotating. From the known position information at two time points of three feature points forming a right angle among the feature points of the object in an image captured from a direction perpendicular to the rotation axis, A moving object recognizing device comprising a shape / motion recognizing means (18) for calculating an actual position and a motion of a feature point. 2. The method according to claim 1, wherein the shape / motion recognition means processes an image obtained by capturing the object from a direction perpendicular to a rotation axis as an orthographic image of the object. Item 2. The moving object recognition device according to Item 1. 3. The shape / movement recognizing means (18) calculates a right-angled vertex point among three characteristic points of the object at each of the two time points as an origin of a three-dimensional coordinate space. A feature point position normalization unit that obtains a relative position of another feature point at the time of setting as normalized known position information, and a position of a feature point of the moving object using an output of the feature point position normalization unit. A shape and motion determining unit that determines whether or not the motion can be recognized; and a shape and motion determining unit that is activated when the position and the motion of the feature point of the moving object are determined to be recognizable, Using the output of the point position normalization unit, a motion calculation unit that calculates the amount of rotation of the object around the origin, using the outputs of the motion calculation unit and the feature point position normalization unit,
A shape calculation unit that obtains unknown position information other than the known position information of the three feature points of the moving object; a rotation amount around the origin output by the motion calculation unit; and a rotation amount output from the feature point position normalization unit. The position information of the feature point located at the origin is combined with the position information in the image as the movement of the object, and the unknown position information output from the shape calculation unit and the output from the feature point position normalization unit are output. 3. The moving object recognition apparatus according to claim 1, further comprising: a feature point position restoring unit that outputs the three feature points together with position information in the image as a feature point position. 4. The three features forming the right angle, wherein the shape / movement recognizing means (18) further comprises a known information input section for inputting known information on the movement of the moving object from the outside including a sensor. Among the points, the vertices are at right angles. The feature point located at the origin is the feature point 0, the other feature points are the feature points 1 and 2, the image plane is the XZ plane, the X axis is the translation direction of the feature point, Z A direction parallel to the axis of rotation of the object,
Y axis is a direction perpendicular to the image plane, Two-dimensional vector on the XY plane of the u _1, u ₂ applied to the feature point 1 from the origin at the first time point by,
v ₁ and v ₂ are two-dimensional vectors on the XY plane from the origin at the second time point to the feature points 1 and 2 Is a rotation matrix representing a two-dimensional rotation on the XY plane about the origin from the first time point to the second time point of the object, and v _i = Ru _i (i = 1,2) Equation 3] Shall be denoted as recognition solutions a solution representative of the actual position and movement of the feature point when u ₂ is a vector obtained by rotating [pi / 2 to u ₁ <1>, u ₂ is the u ₁ - [pi] When the solution obtained when the vector is rotated by ２ is set as the recognition solution <2>, the known information input unit inputs θ = nπ (n is an integer) or α + θ = nπ−α or u ₁₁ v ₂₁ + u ₂₁ v ₁₁ = 0, the known information input unit outputs a non-activation signal to the feature point position normalization unit and stores the unrecognizable information in the motion calculation unit. The apparatus for recognizing a moving object according to claim 3. 5. The feature of the moving object, wherein the shape and motion judging unit sets v ₁₁ ≠ u ₁₁ or v ₂₁ ≠ ± u ₂₁ as a necessary condition, and at least one of the two conditions as a sufficient condition. The moving object recognition device according to claim 4, wherein the position and movement of the point can be determined. 6. The motion calculation unit according to claim 1, wherein said recognition solution <1> is: And the range of the angle α formed by u ₁ with the X axis according to the signs of u ₁₁ and u ₂₁ (π / 2) n ≦ α <(π / 2) (n + 1)
An integer n (= 0, 1, 2, 3) that determines , The range (π / 2) m ≦ β <(π /) of the angle β (= α + θ) formed by v ₁ with the X axis according to the signs of v ₁₁ and v ₂₁
2) The moving object recognition device according to claim 4, wherein an integer m (= 0, 1, 2, 3) that determines (m + 1) is obtained. 7. The motion calculation unit calculates an integer p = (mn−1) / 2 for the recognition solution <1> when mn is an odd number with respect to the values of m and n. , When p is even, sinθ
A sign + between the time codes of odd - and then, m-n compares the u ₁₁ and v ₁₁ when an even number, TABLE 3 The moving object recognition device according to claim 6, wherein the sign of sinθ is determined by the following equation, and sinθ is set to 0 when the sign is satisfied in the two inequalities in Table 3. 8. The motion calculation unit calculates a rotation matrix R with respect to the recognition solution <1> as follows: The moving object recognition device according to claim 4, wherein the calculation is performed by: 9. The motion calculation unit obtains an integer p = (mn−1) / 2 for the recognition solution <1> when mn is an odd number with respect to the values of m and n. , When p is even, sinθ
The sign of is +, the sign of an odd number is-, and when mn is an even number, u ₁₁ and v ₁₁ are compared. The moving object recognizing device according to claim 8, wherein the sign of sinθ is determined by the following equation, and the value of sinθ is uniquely determined according to the determined sign. 10. The method according to claim 1, wherein the shape calculation unit calculates the recognition solution <1>.
Tan α = (u ₁₁ cos θ−v ₁₁ ) / u ₁₁ sin θ d ₁ = u ₁₁ / cos α d ₂ = −u ₂₁ / sin α when u ₁₁ ≠ 0 and u ₂₁ ≠ 0, u ₁₁ In the case of = 0, Thus, when u ₂₁ = 0, 9. The moving object recognition device according to claim 8, wherein α, d ₁ , and d ₂ are calculated according to: 11. The motion calculation section according to claim 1, wherein the recognition solution <1>
Table 5 And the range of the angle α formed by u ₁ with the X axis according to the signs of u ₁₁ and u ₂₁ (π / 2) n ≦ α <(π / 2) (n + 1)
11. An integer n (= 0, 1, 2, 3) that determines the value of .alpha. Is determined, and the shape calculation unit determines only one value of .alpha. According to the determined range of .alpha. Moving object recognition device. 12. The method according to claim 1, wherein the shape calculation unit calculates the recognition solution <1>.
, The two-dimensional vectors u ₁ , u ₂ , v ₁ , and v
The second component u ₁₂ of the _2, u _22, v _12, and v ₂₂ to be calculated by _{u 12 = d 1 sinα, u} 22 = d 2 cosα v 12 = R 2 u 1, v 22 = R 2 u 2 10. The method according to claim 10, wherein
2. The moving object recognition device according to 1. 13. The recognition solution <2> is obtained by performing a process on the recognition solution <1> by reversing the direction of the X-axis to obtain a recognition solution <2>. 10, 11,
Or the moving object recognition device according to 12. 14. The feature point position restoring unit recognizes the recognition solution <1> obtained by the motion calculation unit and the shape calculation unit as a recognition solution <2> that is mirror-inverted with respect to the X-axis. The inverse rotation matrix R ′ of the rotation matrix R for 1> is expressed by The mirror inversion u _i ′, v _i ′ of the two-dimensional vector u _i , v _i with respect to the X-axis is given by The moving object recognition device according to claim 4, wherein the calculation is performed by: 15. The recognition solution <1>, wherein the feature point position restoring unit recognizes the recognition solution <2> obtained by the motion calculation unit and the shape calculation unit as a recognition solution <1> that is mirror-inverted with respect to the X axis. The inverse rotation matrix R ′ of the rotation matrix R with respect to 2> is given by By, mirroring inverted u _i in the X-axis of the two-dimensional vectors _{u i, v i ', v} i' Equation 10] a The moving object recognition device according to claim 13, wherein the calculation is performed by: 16. The feature point position restoring unit and the motion calculating unit may be configured to calculate a position and a movement of a feature point of the object and calculate a motion of the object input to the known information input unit. 5. The moving object recognition device according to claim 4, wherein only the result that does not contradict known information is output as the feature point position and the movement of the object.