JPH07262382A - Object recognition and image interpolating device for image processing system - Google Patents

Abstract

PURPOSE:To execute the calculation of the three-dimensional position or motion of an object in the case of image interpolation and the generation of an interpolated image considering the three-dimensional motion and further to add the three-dimensional effect of shining expression to the interpolated image by using the three-dimensional position or motion of the object even when the shape of the object is not a rigid body. CONSTITUTION:This device is composed of an image input part 1 for inputting and holding the plural pieces of applied source images, feature point input part 2 for holding feature points on the designated source images, motion calculating part 3 for designating a coordinate transformation formula to link the plural pieces of source images and for calculating the position and motion of the object based on the feature points and the designated coordinate transformation formula, motion interpolating part 5 for hourly continuously changing the motion of the object, image projecting part 6 for generating the projected image of the object at the time between the images, and image output part 7 for outputting the projected image generated by the image projecting part.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object recognition and image interpolation device in an image processing system. Techniques for object recognition and image interpolation are, for example, automatic production by robots, TV
Image processing of video, computer graphics (C
G) It is used in various fields such as technology, and its progress is remarkable. The present invention further improves this object recognition and image interpolation technique.

Here, the object recognition is a technique for obtaining the three-dimensional position and movement of an object in the image from several shot images. Obtaining the three-dimensional position and movement of an object in an image is important for FA (factory automation) and autonomous vehicles, for example. Image interpolation is a technique for synthesizing images at a predetermined time between several images when several images are given. The image interpolation technique is, for example, an image database, T
It is widely used in image processing systems such as V-phones and animations.

The apparatus of the present invention targets an image taken by a camera or the like in which the three-dimensional position or movement of an object in the image is unknown. In order to obtain a three-dimensional position and movement, at least two images of the same object taken from different directions are required. Although the following description will be made assuming that the number of original images is two, the present invention is also applicable to the case where there are three or more original images.

The general processing for performing image interpolation using the object recognition technology is as follows. First, the three-dimensional motion of the obtained object is virtually changed temporally smoothly, and the position of the object at the time between images is re-projected on the image to generate an image at the time. In the image generated by CG, the three-dimensional positions of the objects in the image are all known (that is, stored in the computer), so the present invention is not necessary.

[0005]

2. Description of the Related Art FIG. 6 is an explanatory diagram of image interpolation, and FIG. 7 is an explanatory diagram of image interpolation using an object recognition technique. As shown in FIG. 6, image interpolation is a technique of synthesizing images at a predetermined time between several images when given several images. Further, as shown in FIG. 7, a general process of performing image interpolation using an object recognition technique is such that when the original image 1 is changed to the original image 2, the three-dimensional movement of the object is virtually and temporally. By changing smoothly (virtual movement) and re-projecting the position of the object at the time between,
An image (interpolation image) at a time between is generated.

By the way, in order to obtain the three-dimensional position and movement of an object in an image, the following techniques 1 to 3 are conventionally known. 1. Method of Obtaining Two-Dimensional Correspondence Between Images Two-dimensional correspondence between images means designating correspondence between original images of representative points (feature points) in the images. The feature points can be dealt with by automatically associating points such as the corners of an object that are easy to automatically extract, manually by the operator, and the amount of movement (optical flow) between them based on the image differences. A method of calculating is known. Then, the optical flow calculation is performed under the approximation that the distance between the feature points is sufficiently small when the feature points are automatically matched.

2. Method of setting some assumptions on the three-dimensional shape of an object In general, a projected image lacks information in the depth direction of the object, and therefore a solution cannot be determined unless some assumption is made on the object shape. The commonly used assumptions are shown below. -Flatness: It is assumed that an object is composed of one plane or a plurality of planes with specified boundaries. In this case, the position and movement of the object can be calculated if there are four sets of feature points associated with one plane.

Rigidity: An object is assumed to be one rigid body. In this case, the position and movement of the object can be calculated if the correspondence of the eight sets of feature points is attached. Which of these assumptions is used is determined in consideration of the purpose of use and the labor of processing. 3. Method for calculating three-dimensional position and motion of object Three-dimensional position and motion of object are calculated under the above assumptions. However, assuming only the rigid body, the depth of the object at each feature point is determined, and the depths at other points on the image are not determined. Will be required. A simple weighted average is often used for this estimation. On the other hand, if flatness is assumed, the depth of the object at all points is determined. Also, three-dimensional position (depth)
At the same time, it is possible to calculate how the object moved.

Next, in order to perform image interpolation, the following (a)
The techniques (b) and (b) have been conventionally known. (A) There are roughly two types: a method in which the object recognition technology is not used and (b) a method in which it is used. (B) Method that does not use object recognition technology-Image interpolation is performed by only one affine transformation (translational movement / rotational movement / enlargement / reduction).

This is one in which image interpolation is performed by one affine transformation (that is, transformation in which a pixel value is translated / rotated / scaled and moved). This process is simple and can be automated because the operator does not need to specify the feature points. However, image interpolation is not possible when the object is moving in three dimensions. This example is, so to speak, a method of connecting a pair of images by moving a rubber film on the two-dimensional image.

Image interpolation is performed by combining a number of affine transformations. This is because the operator specifies a large number of affine transformations so that the corresponding feature points are associated with each other by designating corresponding feature points between the original images, and then the coefficients are changed continuously in time. Is used for interpolation. In this example, a rubber film whose periphery is pressed by a pin is moved on a two-dimensional image, so to speak, to connect two images. Similar to the above, image interpolation is not possible when an object is moving in three dimensions.

(B) Method using object recognition technology: Image interpolation is performed by one projective transformation (that is, transformation including three-dimensional motion in addition to affine transformation). Assuming that the object shape is a plane, the position and motion of the plane are first calculated from the correspondences between the four sets of feature points that are automatically or manually given, and then this is temporally continuous in three-dimensional space. It is a method of changing the image and re-projecting it on the image. Although it can represent a three-dimensional motion, it has a drawback that the object shape is limited to a plane. Even if the object is composed of one front surface and the other side surface portion, it is possible to extend it so that image interpolation can be performed, but there are still many restrictions on the object shape.

Image interpolation performed by a combination of a number of projective transformations (provided that the object is rigid) The object shape is assumed to be a rigid polygon (a set of planes) and is automatically or manually given 8 From the correspondence between the set of feature points, first calculate the motion of the rigid body and the depth of each feature point,
Next, the depths of all points are calculated by using the above-mentioned “planarity”. Then, it is a method of changing these continuously in time in a three-dimensional space and re-projecting them on an image.
This method is often used when the shape of an object can be regarded as a rigid polygon, but there is a problem in that an image (morphing) in which a person changes into a fish cannot be generated.

Image interpolation performed by combining a large number of projective transformations (not limited to rigid bodies) Automatically or manually assuming that the object shape is a polygon (a set of planes) that is not necessarily a rigid body. From the correspondence between the four sets of feature points given for each surface, first, the position and movement of each plane are calculated, and then they are continuously changed in time in the three-dimensional space and re-projected on the image. Is the way. Although three-dimensional motion can be expressed to some extent and there is an advantage that the restrictions on the object shape are loose, it is not always possible to move each plane in a three-dimensional space because the rigid body is not assumed. There is a risk that objects will fall apart at different times. In this method, in order to avoid this, if the positional relationships between the planes are averaged in two dimensions and the planes are connected, it can be used for image interpolation purposes.
There is a problem that three-dimensional position and movement are not required.

FIGS. 8A and 8B are explanatory views of conventional morphing. The method (A) is a method already proposed and put into practical use by the inventor of the present application, and the method (B) is a method conventionally used. In the method (A), the object is designated for each divided area surrounded by a rectangle composed of feature points (each ●), whereas in the method (B), the area is not divided and Specify the correspondence with a polygonal line. The method (A) has a remarkable effect that the number of feature points is smaller than that of the method (B). That is, in (A), since the projective transformation is determined for each quadrangle, the influence range of one feature point is limited, and as a result, there is an effect that the feature point can be easily specified. On the other hand, in the method of (B), how a certain feature point changes is influenced by all the surrounding line segments, so that the influence of one feature point extends over a wide range.

The method (A) will be described in more detail.
This method is divided into a plurality of regions surrounded an object with feature points, performs interpolation processing for each divided region, the time of the division, are classified in advance which region when directing feature point Also specify As the interpolation processing for each area, first, the area of each original image is converted into an image to be output, and then the interpolation image is generated by referring to the pixel value of the original image. Further, techniques such as projective transformation, affine transformation, and weighting of transformation coefficients are disclosed as transformation methods for each region of the original image.

[0017]

As is apparent from the above, the problem of the conventional object recognition technique is that "the conditions for the object shape are too restricted". For example, the condition that an object is a rigid body is too restrictive. In addition, the problem of the conventional image interpolation technique is, in the end, that "image interpolation can be performed even when the object is not a rigid body, but three-dimensional motion cannot be expressed, and further, the position and motion are obtained. I can't do that. "

The object of the present invention is to enable the object recognition from a given image to determine the three-dimensional position and movement of the object in the image even when the object shape is not rigid. Especially. Further, in image interpolation, it is possible to both calculate a three-dimensional position and motion of an object and generate an interpolated image in consideration of the three-dimensional motion. Furthermore, it is possible to add a three-dimensional effect such as gloss expression to the interpolated image by using the three-dimensional position and movement of the object.

[0019]

One aspect of the present invention is an object recognizing device in an image processing system using a computer, which inputs and holds a plurality of given original images as shown in FIG. The image input unit 1 to be used, for example, the feature point input unit 2 that holds the feature points on the original image designated by the operator, and the coordinate conversion formula for connecting a plurality of given original images, A motion calculation unit 3 that obtains the position and motion of an object based on a point and a specified coordinate conversion formula
And a motion output means for outputting the position and motion of the object as the calculation result.

Here, as the coordinate conversion formula, translational motion, deformation matrix, and depth for each feature point can be used. Further, the object recognition apparatus of the present invention further comprises means for designating a plurality of sets of feature points, and means for switching the coordinate conversion formula for connecting the original images for each designated set of feature points.

Further, there is further provided means for making a coordinate conversion formula for connecting the original images for each set of specified feature points continuous on the feature points belonging to a plurality of sets. On the other hand, another aspect of the present invention is an image interpolating device in an image processing system, and as shown in FIG. 4, an image input unit 1 for inputting and holding a plurality of given original images and a designated original image. The feature point input unit 2 that holds the feature points on the image and the coordinate conversion formula for connecting the given original images are specified.
A motion calculation 3 for determining the position and motion of an object based on a feature point and a designated coordinate conversion formula, a motion interpolation 5 for continuously changing the motion of the determined object in time, and a motion interpolation unit 5
The image projection unit 6 that inputs the result of (3) and generates a projection image of the object at the time between images, and the image output unit 7 that outputs the projection image generated by the image projection unit 6.

Here, as a method of continuously changing the obtained motion of the object in time, the translational velocity and the deformation matrix are collectively regarded as a deformation matrix in a four-dimensional space, and it is changed at a constant rate in time. Let Further, as a method of continuously changing the obtained motion of the object in time, the translation velocity is changed linearly and the deformation matrix is changed at a constant rate in time.

Further, it further comprises means for adding gloss to the interpolated image depending on the positional relationship between the light source and the object.

[0024]

The function of the present invention will be described in detail below. First, in object recognition from a given image, in the present invention, the condition of the rigid body conventionally used is relaxed, and the object is made from a plurality of quasi-rigid bodies (in this sense, tentatively referred to as "according to rigid body"). It is configured. And 3 of each area
Obtain the dimensional position and motion under the constraint that adjacent quasi-rigid bodies do not separate.

Next, the parameter representing the obtained three-dimensional motion is continuously changed in time to calculate the position of the time between the interpolations, and the image is reprojected to calculate the image. Interpolate. FIG. 1 is an explanatory diagram of a coordinate relationship between an observation object and a projection surface. The principle of object recognition from an image according to the present invention will be described below. First, a motion of an object and a model for projecting it on an image plane will be described. As shown in the figure, first, a coordinate system is introduced into the three-dimensional space and the viewpoint is set at the origin. The projection plane is set to Z = 1, and a moving object in a three-dimensional space is centrally projected on this plane for observation.

From this, it can be seen that the point (x, y) on the projection plane onto which the point (X, Y, Z) on the object is projected can be expressed as follows.

[0027]

[Equation 1]

Here, an image 1 is taken as an observation result at a certain time (t = 0), and after a lapse of a certain time from t = 0 (t
Image 2 is the observation result of = 1). Now, assuming that the object is one rigid body, the translation velocity vector of the object between image 1 and image 2 is u, and the rotation matrix is R. Then, t
A point X ₀ = (X ₀ , Y ₀ , Z ₀ ) ^t on the object at = 0,
The point X ₁ = (X ₁ , Y ₁ , Z on the corresponding object at t = 1
₁ ) The following relation holds for ^t .

[0029]

[Equation 2]

By the way, a point X ₀ = on the object at t = 0
The point projected on the image of (X ₀ , Y ₀ , Z ₀ ) ^t is x _0.
= (X ₀ , y ₀ , 1) ^t , the corresponding point X ₁ = at t = 1
The point projected on the image of (X ₁ , Y ₁ , Z ₁ ) ^t is x ₁
= (X ₁ , y ₁ , 1) ^t , the relationship between x ₀ and x ₁ is calculated. This means that points corresponding to each other on the image 1 and the image 2 are obtained. From equations (1) and (2),

[0031]

[Equation 3]

It becomes Therefore, given the correspondence of feature points between the original images 1 and 2, the problem of finding the three-dimensional position and movement of the object in the image from them is the problem when x ₀ and x ₁ are given. Z ₀ , Z ₁ , u, R satisfying the expression (3)
Can be formulated as On the other hand, there are various possible ways to extend the model so that it can handle cases where the object is not a rigid body, but in this example, the rotation matrix R A method of expanding the matrix A of 3 × 3 is adopted. Here, various variations are obtained depending on how the possible values of the matrix A are limited. For example, the rotation matrix R
Is an orthogonal matrix R ^t R = I ₃ and the determinant has a value of 1. Therefore, if A is restricted to this extent, it will return to the case of a rigid body. Therefore, since the overall scale of the equation is indefinite and the absolute size is unknown, only the limitation that the value of the determinant is 1 is given. Below, this expanded matrix A is tentatively referred to as a “deformation matrix”, and the region that moves by one A is referred to as a quasi-rigid body. In this case, equation (3) becomes

[0033]

[Equation 4]

It is expanded as follows. In order to handle the case where an object is composed of multiple quasi-rigid bodies, the following symbols are defined. Quasi-rigid body with serial number l (lowercase "L") (l
= 1 ,. ． ． , L), and the deformation matrix A _{l, for} each quasi-rigid body
It is assumed that a translational motion u _l is performed. Also, serial numbers j (j = 1, ..., J) are assigned to the feature points on the image,
It is assumed that each point has a depth Z _{0, j,} Z _{1, j} . Feature point j
There are generally several quasi-rigid bodies to which
_{Let j, k} (k = 1, ..., k _j ). In this case, equation (4) can be expressed as follows.

[0035]

[Equation 5]

This is a quadratic simultaneous equation with respect to Z _{0, j,} Z _{1, j} , A _{l and} u _l . The total degree of freedom of unknowns is (2
J + 111L), the number of equations is 3Σ _j k _j (≧ 3J). Therefore, if the number L of quasi-rigid bodies is sufficiently smaller than the total number J of feature point pairs, there is a possibility that no solution satisfying all equations exists. is there. Therefore, the following evaluation function K is considered in order to find a solution that satisfies Expression (5) in the meaning of least squares.

[0037]

[Equation 6]

Z that minimizes K in equation (6)
_{0, j, Z 1, j} , A l, by obtaining u _l numerically the exhaustive search or the like, the depth Z ₀ at the feature point is part of a three-dimensional position of the _{object, j,} Z _{1, j} And parameters A _{l and} u _l representing the motion of the object are obtained. This procedure 1-5
Will be described below. 1. Initialization Put appropriate values in Z _{0, j,} Z _{1, j} , A _{l and} u _l .

2. Calculation of Evaluation Function K The value of the evaluation function K with respect to the current value is calculated by the equation (6). 3. Comparison of the value of the evaluation function K Compared with the value of the previous K, and if it has not decreased, the process ends. 4. Correction of values The values of Z _{0, j,} Z _{1, j} , A _{1 and} u ₁ are slightly changed in the direction in which the value of the evaluation function K decreases.

5. The evaluation function is calculated again. Next, the depths Z _{0, j,} Z _{1, j} at the feature points are spatially averaged to obtain the depth Z ₀ (x), at each image point X.
Calculate Z ₁ (x). Thereby, the three-dimensional position of the object is obtained. The procedures 1 to 5 will be described below. 1. Initialization The current focus is on the upper left of the screen. Prepare two images for distance writing.

2. Weight calculation For all feature points within a certain distance from the point of interest at present, the reciprocal of the distance to the feature point and the depth there are averaged. This process is performed for each original image. 3. Distance writing The averaged value is regarded as the depth at that point, and is written in the corresponding portion of the image for distance writing.

4. Moving the point of interest Move the point of interest currently by one to the right. If you cannot go further to the right, go to the left end one step below. If it does not go further down, it ends. 5. The weight is calculated again. Note: The solution that minimizes equation (6) is the adjacent quasi-rigid bodies l, l '.
Three-dimensional coordinates Z _{0, j} x _0j, Z _{1, j} x _{1j of} feature points j between
Note that satisfies the constraint that is the same for both quasi-rigid bodies.

Next, the principle of image interpolation using the object recognition technique according to the present invention will be described. As described above, the object recognition technique is the correspondence x _0j, x _{1j of} given feature points,
Given an index l _{j, k} that specifies which quasi-rigid body l it belongs to, then the motion of each quasi-rigid body (deformation matrix A _l , translational velocity u _l ) and the respective original points at each feature point. The depth Z _{0, j} , Z _{1, j} on the image is obtained.
Furthermore, by averaging the depth of each feature point, each point x in the image
It is also possible to calculate the depths Z ₀ (x) and Z ₁ (x) at. These represent the three-dimensional position of the object.

Image interpolation using the object recognition technique according to the present invention is performed in the following steps 1 to 3. 1. The movement of an object is changed continuously in time. A parameter transformation matrix A _l representing the motion of each quasi-rigid body l and a translational velocity u _l at t = 0 are I ₃ (showing a unit matrix and not deformed) and o (0 vector, translation amount 0), and t = 1
Then, the time t is continuously changed so that the calculated value is obtained. This makes it possible to calculate what the three-dimensional coordinates of the feature point j at the time t between the images become.

2. Generate a projected image of the object at the time between images. The brightness and pattern (texture) of the original image are coordinate-converted into the position of the object at the time between the images calculated above. It is then projected onto the Z = 1 projection plane. 3. The projection result is output as an interpolation image. It is not possible to determine what the object will look like at the time between images from the original image alone, so there are various possible methods for continuously changing the motion of the object. There are two methods.

A method in which the transformation matrix A and the translational motion u are collectively considered as a four-dimensional transformation matrix, and the transformation matrix is changed at a constant rate in time. The feature of this method is that it has a similar change to the dynamic system when the frictional force is large enough (for example, when the leaves of a tree fly in the air). Therefore, if the original object is assumed to have such properties, this method should be used.

A method in which the translational velocity u is changed linearly and the transformation matrix A is changed at a constant rate in time. The feature of this method is that a uniform linear motion and a uniform rotary motion can be expressed. Therefore, this method should be used if the original object is assumed to have such simple movements.

The transformation matrix A (t) is changed at a constant rate over time so that it matches I ₃ (or I ₄ ) at t = 0 and A obtained by the object recognition technique at t = 1 as follows. It results in solving the undetermined coefficient linear differential equation (7).

[0049]

[Equation 7]

Note that G is a constant matrix which is not yet determined. This solution can be expressed by the following equation (8).

[0051]

[Equation 8]

The index t represents the power of the matrix. A (0) =
I, A (1) = A holds. If A is an ordinary real number, this means that A (t) changes at a constant rate with time. On the other hand, changing the translational velocity u linearly means that the translational velocity u (t) at time t is u.
(T) = tu.

By the method described above, it becomes possible to change the motion parameter continuously in time. Next, generation of the projected image of the object at the time between will be described.
The value of the motion parameter in quasi-rigid l at time _{t A l (t), u} l (t), the brightness at the point x of the original image I
Generation procedure 1 of the projected image I _t (x) as ₀ (x), I ₁ (x)
5 will be described below.

1. Initialization Point x, which is currently focused on, is the upper left corner of the image. Prepare one image for image interpolation output. 2. Calculation of corresponding points at time t For a quasi-rigid body 1 to which x belongs, Z _t (y) y = A
Z _t (y) and y satisfying _l (t) (Z ₀ (x) x + u _l (t) are obtained, which is the corresponding point on the image at time t.

3. Calculation of Image Brightness at Corresponding Point A point P on the original image 2 corresponding to x is calculated, and (1-t)
I ₀ (x) + t I ₁ (p) is the brightness I of the image at the corresponding point y.
The image for interpolation output is written as _t (y). 4. Moving the point of interest Moves the point of interest x to the right by one. If it cannot move further to the right, it moves one position down to the left end. If it cannot go further down, it ends.

5. The corresponding points are calculated again. Further, it becomes possible to add a three-dimensional effect such as expression of gloss to the interpolated image by using the calculated depth Z _t (x) of the object. Here, as an example, steps 1 to 6 for adding gloss of perfect diffuse reflection will be described below. 1. Initialization Point x, which is currently focused on, is the upper left corner of the image.

2. Calculation of the normal vector of the object at the point of interest At present, the normal vector n of the object at the point of interest x
Calculate (x). The calculation method is, for example, taking the points y and z in this vicinity,

[0058]

[Equation 9]

It is assumed that Here, x represents an outer product. 3. Calculation of the cosine of the angle between the light source and the normal vector of the object This is when the coordinate of the light source is l.

[0060]

[Equation 10]

It is 4. Calculation of Gloss A value proportional to the appropriate power of the cosine of the angle is added to the value I _t (x) of the interpolation image at the point x of interest. 5. Moving the point of interest Moves the point of interest x to the right by one. If it cannot move further to the right, it moves one position down to the left end. If it cannot go further down, it ends.

6. The normal vector is calculated again.

[0063]

2 is a block diagram of an embodiment of the present invention for recognizing an object from an image, and FIG. 3 is a processing flowchart of the configuration of FIG. The flow of processing will be described below. First, two given original images are stored in the image input unit 1 (S1). Further, the result of the instruction of the characteristic points by the operator is stored in the characteristic point input unit 2 (S2). The feature point designation includes both correspondence between original images and designation of which set the feature point belongs to.
Next, the motion calculation unit 3 calculates the position and motion of the object (S
3). Finally, the motion output unit 4 outputs the position and motion of the object (S4).

FIG. 4 is a block diagram of an embodiment of the present invention for performing image interpolation, and FIG. 5 is a processing flowchart of the configuration of FIG. The flow of processing will be described below. First given 2
The original images are stored in the image input unit 1 (S11). Further, the result of the instruction of the feature point by the operator is stored in the feature point input unit 2 (S12). The feature point designation includes both correspondence between original images and designation of which set the feature point belongs to. Next, the motion calculation unit 3 calculates the position and motion of the object (S13). Furthermore, the motion interpolation unit 5 continuously changes the motion of the object in time (S14). The result of the motion interpolation unit 5 is input to the image projection unit 6, the image projection unit 6 generates a projected image of the object at the time between images (S15), and the image output unit 7 outputs the interpolated image (S16). ).

[0065]

As described above, according to the present invention,
In object recognition from a given image, it is possible to obtain the three-dimensional position and movement of an object in the image even when the object shape is not rigid. Further, in image interpolation, it is possible to perform both calculation of a three-dimensional position and movement of an object and generation of an interpolation image considering the three-dimensional movement. Furthermore, by using the calculated three-dimensional position of the object, it becomes possible to add a three-dimensional effect such as a gloss expression at the time of image interpolation.

[Brief description of drawings]

FIG. 1 is an explanatory diagram of a coordinate relationship between an observation object and a projection surface for explaining the present invention.

FIG. 2 is a configuration diagram of an embodiment of an object recognition device according to the present invention.

FIG. 3 is a processing flowchart of the configuration of FIG.

FIG. 4 is a configuration diagram of an embodiment of an image interpolation device of the present invention.

5 is a processing flowchart of the configuration of FIG. 4. FIG.

FIG. 6 is an explanatory diagram of image interpolation.

FIG. 7 is an explanatory diagram of image interpolation using an object recognition technique.

FIG. 8 is an explanatory diagram (A) and (B) of conventional morphing.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Image input part 2 ... Feature point input part 3 ... Motion calculation part 4 ... Motion output part 5 ... Motion interpolation part 6 ... Image projection part 7 ... Image output part

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI technical display location 9365-5L G06F 15/72 350

Claims (8)

[Claims] 1. An object recognition device in an image processing system, comprising image input means for inputting and holding a plurality of given original images, and characteristic point input means for holding characteristic points on a specified original image. And a motion calculation unit that specifies a coordinate conversion formula for connecting a plurality of given original images, and obtains the position and motion of the object based on the specified coordinate conversion formula and the feature point, and the object that is the calculation result. Object recognition apparatus in an image processing system, comprising: a motion output unit that outputs the position and motion of the. 2. The object recognition device according to claim 1, wherein a translational motion, a deformation matrix, and a depth for each feature point are used as the coordinate conversion formula. 3. The method according to claim 1, further comprising means for designating a plurality of sets of the feature points, and means for switching a coordinate conversion formula for connecting the original images for each of the designated sets of feature points. The object recognition device described. 4. The object recognizing apparatus according to claim 3, further comprising means for making a coordinate conversion formula for connecting the original images for each set of specified feature points continuous on the feature points belonging to a plurality of sets. . 5. An image interpolating device in an image processing system, comprising image input means for inputting and holding a plurality of given original images, and characteristic point input means for holding characteristic points on a specified original image. And a coordinate transformation formula for connecting the given original images, and a motion calculation means for obtaining the position and movement of the object based on the feature points and the designated coordinate transformation formula, and A motion interpolation means for continuously changing the motion in time, an image projection means for inputting the result of the motion interpolation means and generating a projection image of an object at a time between images, and an image projection means generated by the image projection means An image interpolation device in an image processing system, comprising: an image output unit that outputs a projected image. 6. As a method of continuously changing the obtained motion of an object in time, the translational velocity and the deformation matrix are collectively regarded as a deformation matrix of a four-dimensional space, and it is changed at a constant rate with time. The image interpolation device according to claim 5, wherein 7. The method of changing the obtained motion of an object continuously in time, wherein the translation velocity is changed linearly and the deformation matrix is changed at a constant rate in time. The described image interpolating device. 8. The image interpolating apparatus according to claim 5, further comprising means for adding gloss to the interpolated image according to the positional relationship between the light source and the object.