JP2991878B2 - Optical flow operation circuit - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to real-time determination of an optical flow, which is a movement vector of an object, from two images taken at different times, that is, different in time, taken by a video camera or the like. And an optical flow arithmetic circuit used for image recognition and the like.

[0002]

2. Description of the Related Art As a method of obtaining an apparent motion field (optical flow) from two images, "Determining Optical Flood" is used.
w) ", Artificial Intelligence ( Artif
icial Intelligence) Vol. 17 No. 1-3 (198
1) pp. There are those proposed in 185-203.

[0003] The principle of the method is described below. The density of a certain coordinate (x, y) in the image at time t is represented by I (x, y,
When expressed by t), the approximate expression (1) is established from the first assumption that the density of the object is invariant with time.

[0004] _{I x u + I y v +} I t = 0 ··· (1)

Here, I _x and I _y are in the spatial direction (x direction,
concentration gradient in the y-direction), I _t is the concentration gradient of the time direction, u,
v is the component of the optical flow in the x and y directions. I _x, I _y, I _t is given by the following equation.

I _x = ｛I (i + 1, j, t) -I (i, j, t) + I (i + 1, j + i, t) -I (i, j + 1, t) + I (I + 1, j, t + 1) -I (i, j, t + 1) + I (i + 1, j + 1, t + 1) -I (i, j + 1, t + 1)｝ / 4 (2)

I _y = ＩI (i, j + 1, t) -I (i, j, t) + I (i + 1, j + i, t) -I (i + 1, j, t) + I (I, j + 1, t + 1) -I (i, j, t + 1) + I (i + 1, j + 1, t + 1) -I (i + 1, j, t + 1)｝ / 4 (3)

[0008] _{I t = {I (i,} j, t + 1) -I (i, j, t) + I (i + 1, j, t + i) -I (i + 1, j, t) + I (i, j + 1, t + 1) -I (i, j + 1, t) + I (i + 1, j + 1, t + 1) -I (i + 1, j + 1, t) ｝ / 4 (4)

Further, when the second assumption that the local speed change is smooth is added to the equation (1), u and v that minimize the following error function E become optical flows.

E = ∬ (E _b ² + α ² E _c ) dxdy (5)

Here, E _b and E _c are respectively (6)
Where E _b is an error function according to the first assumption, and E _c is an error function according to the second assumption. Also, α
Is a weight coefficient between the two error functions, and is a small value when the observation data is reliable, and is a large value when the observation data is not reliable due to noise or the like in order to increase the smoothing effect by the second assumption.

[0012] _{E b = I x u + I} y v + I t ··· (6) E c = (∂u / ∂x) 2 + (∂u / ∂y) 2 + (∂v / ∂x) 2 + (∂v / ∂y) ²・ ・ ・ (7)

U and v that minimize the equation (5) can be obtained by calculating the following iterative equation.

[0014] ^{u (n + 1) = u} av (n) - {[I x (I x u av (n) + I y v av (n) + I t)] / [α 2 + I x 2 + I y 2] ｝ (8) v ^{(n + 1)} = v _av ⁽ⁿ⁾ − ｛[I _y (I _x u _av ⁽ⁿ⁾ + I _y v _av ⁽ⁿ⁾ + I _t )] / [α ² + I _x ² + I _y ² ]｝ (9) where u _av ⁽ⁿ⁾ is an average value of eight neighboring pixels of u ⁽ⁿ⁾ , v _av
⁽ⁿ⁾ is the average value of the eight neighboring pixels of v ⁽ⁿ⁾ , and n is the number of repetitions.

From the above, according to the method described in the above document, when two images are input at different times, the spatial density gradient I is calculated according to the equations (2) to (4).
_x, computes the I _y and time gradient I _t, with their concentration gradients (8), the optical flow u, v is determined by performing a suitable number of times iterative calculations of equation (9).

FIG. 12 is a block diagram showing a configuration of an optical flow arithmetic circuit for executing the above processing. In the figure, reference numerals 11 and 12 indicate two temporally different images. Then, the gradient calculation circuit 13 calculates the spatial density gradient I
_x, computes the I _y and time gradient I _t to store them in the gradient memory 21. Then, the motion field u, v operation circuit 22
Calculates u ^{(n + 1)} and v ^{(n + 1 )} by inputting u ⁽ⁿ⁾ and v ⁽ⁿ⁾ in the motion field u and v memory 23. Then, u ^{(n + 1)} and v ^{(n + 1)} are moved to the motion field u,
Stored in v-memory 23. Note that there is a prior art disclosed in Japanese Patent Application Laid-Open No. 3-74782.

[0017]

Since the conventional optical flow arithmetic circuit is constructed as described above, n +
In order to calculate the value of the first optical flow, the n-th value must be calculated in advance over the entire screen, and there is a problem that the calculation time becomes longer in proportion to the number of repetitions. Further, a gradient memory 21 for storing a density gradient for each pixel and a motion field u, v memory 23 for storing an n-th value are indispensable, and there is a problem that the circuit scale increases.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and an optical flow operation circuit which does not require a memory and can perform an optical flow operation in a shorter time than the conventional one. The purpose is to gain.

[0019]

According to a first aspect of the present invention, an optical flow calculation circuit receives two images at different times and calculates a spatial density gradient and a time density gradient between these images. A gradient operation circuit, a coefficient operation circuit for calculating the coefficient of the optical flow repetition operation from each output of the gradient operation circuit, and a column for calculating a motion field representing the optical flow using the output of the coefficient operation circuit. A plurality of motion field u, v operation circuits
And a delay circuit for delaying the output of the coefficient operation circuit and synchronizing the output with the operation of each motion field u, v operation circuit.

The optical flow arithmetic circuit according to the second aspect of the present invention is the optical flow arithmetic circuit according to the first aspect of the present invention, wherein the optical flow arithmetic circuit is provided before the gradient arithmetic circuit.
The image processing apparatus includes a position shift correction unit that detects a position shift of a background between images and corrects the position shift.

According to a third aspect of the present invention, there is provided the optical flow operation circuit according to the first aspect, wherein the gradient operation circuit calculates an addition value between pixels at the same position in two images. After calculating the subtraction value, the spatial density gradient and the time density gradient between the images are calculated.

The optical flow arithmetic circuit according to the invention according to claim 4 is the optical flow arithmetic circuit according to claim 1, wherein each of the gradient arithmetic circuits is provided between the gradient arithmetic circuit and the coefficient arithmetic circuit. A bit number reduction circuit for reducing the number of bits between the output and the weight constant of the coefficient calculation is provided.

[0023]

According to the first aspect of the present invention, a plurality of motion fields u,
The v operation circuit executes the repetition operation of the optical flow in a pipeline manner. At that time, the coefficient calculation circuit calculates in advance each coefficient required for the repetition calculation,
Enables such pipelined operations.

[0024] The misregistration correction unit according to the second aspect of the present invention adjusts the background between two images to prevent the occurrence of an optical flow due to the misregistration of the background.

The gradient calculating circuit according to the third aspect of the present invention calculates an addition value and a subtraction value between pixels at the same position in two images in advance to simplify the product-sum operation.

The bit number reducing circuit according to the fourth aspect of the present invention reduces the number of bits of each data handled by the coefficient calculating circuit, reduces the number of bits for internal calculation in the coefficient calculating circuit, and simplifies the configuration of the coefficient calculating circuit. Become

[0027]

【Example】

Embodiment 1 FIG. FIG. 1 is a block diagram showing a configuration of an optical flow calculation circuit according to one embodiment of the present invention. In the figure, reference numerals 11 and 12 denote two images, and reference numeral 13 denotes a gradient operation circuit similar to the conventional one. Reference numeral 14 denotes a coefficient operation circuit that preliminarily calculates a coefficient required for an optical flow repetition operation. Here, α is a weight constant required for the coefficient calculation. Reference numeral 15 denotes an initial value of the repetition operation.

16a is a motion field u, v for performing the first calculation.
An operation circuit, 16b is a motion field u, v operation circuit for performing the second operation, 16c is a motion field u, v operation circuit for performing the third operation, and 17a is a motion field u, v for the output of the coefficient operation circuit 14. A delay circuit 17b for giving a delay corresponding to the operation time in the v operation circuit 16a, a delay circuit 17b for giving a delay corresponding to the operation time in the motion field u, v operation circuit 16b to the output of the delay circuit 17a, and a delay circuit 17c 17b
Is a delay circuit that gives a delay corresponding to the operation time in the motion field u, v operation circuit 16c to the output of

Next, the operation will be described. First image 11
Is an image at time t + 1, and the second image 12 is an image at time t, and is input to the gradient calculation circuit 13 by raster scan from two image buses. As shown in FIG. 2, the first image 11 is delayed by one line by the line memory 31a,
And a delay of two rows by the line memory 31b. The second image 12 is delayed by one line by the line memory 31c, and is delayed by two lines by the line memory 31c and the line memory 31d. The outputs of the line memories 31a to 31d are input to three 4 × 2 product-sum operation circuits 32a to 32c.

Here, assuming that the data in the line memory 31b is in the j-th row, the data in the line memory 31a is in the j + 1-th row. Also, the line memory 31d
Is the data of the j-th line, the line memory 31
The data in c is for the (j + 1) th row. Therefore, 4 ×
The two-sum-of-products operation circuit 32a outputs the input data
By multiplying the coefficient shown in (a) and taking the sum of the results,
The concentration gradient _Ix is calculated by the equation (2). Note that FIG.
In (a), the four coefficients in the right column are coefficients for the i-th pixel, and the four coefficients in the left column are coefficients for the (i + 1) -th pixel.

Similarly, the 4 × 2 product-sum operation circuit 32b multiplies the input data by the coefficient shown in FIG. 3 (b) and obtains the sum of the results to obtain the density gradient I _{y according} to the equation (3). calculated, 4 × 2 sum operation circuit 32c is multiplied by the coefficient shown in FIG. 3 (c) with respect to the input data, by taking the sum of the results to calculate the concentration gradient I _t by (4).
The repetition formulas of the formulas (8) and (9) are (10) and (11)
It can be transformed like an equation.

[0032] ^{u (n + 1) = {} (α 2 + I y 2) u av (n) -I x I y v av (n) -I x I t} / (α 2 + Ix 2 + Iy 2) ·· ^{· (10) v (n +} 1) = {(α 2 + I x 2) v av (n) -I x I y u av (n) -I y I t} / (α 2 + Ix 2 + Iy 2) ·・ ・ (11)

By setting the coefficients a to e required for the repetition operation as follows, the repetition formulas are expressed by (17) and (1).
8) can be written as

A = (α ² + I _y ² ) / (α ² + Ix ² + Iy ² ) (12) b = I _x I _y / (α ² + Ix ² + Iy ² ) (13) c = _{^{(α 2 + I x 2)}} / (α 2 + Ix 2 + Iy 2) ··· (14) d = I x I t / (α 2 + Ix 2 + Iy 2) ··· (15) e = I y I t / (Α ² + Ix ² + Iy ² ) (16)

[0035] ^{u (n + 1) = au} av (n) -bv av (n) -d ··· (17) v (n + 1) = cv av (n) -bu av (n) -e ·・ ・ (18)

The coefficient operation circuit 14 is composed of (12) to (16)
Performs a formula operation. FIG. 4 is a block diagram showing a configuration of the coefficient operation circuit 14. As shown in FIG.
51c performs alpha ^2, computation of I _x ^2, I _y ^2, respectively. Also, the multiplier 52a~52c each I _x I
_y, I _x I _t, the calculation of I _y I _t do. Then, the adder 53 calculates α ² + I _x ² + I _y ² , and
4a to 54b represent α ² + I _x ² and α ² + I _y ^{2 respectively.}
Is calculated.

FIG. 5 is a block diagram showing the structure of the motion field u, v operation circuit 16a. The configuration of the motion field u, v operation circuits 16b to 16c is the same as that shown in FIG.
In the motion field u, v operation circuit 16a, initial values u ⁽⁰⁾ ,
When v ⁽⁰⁾ is input, the average value calculation circuit 61a outputs u ⁽⁰⁾
The average value u _av ⁽⁰⁾ in the neighborhood of 8 is calculated, and the average value calculation circuit 6
1b is v ⁽⁰⁾ average 8-neighborhood of v _av ⁽⁰⁾ Is calculated.

The average value calculation circuit 61a is configured as shown in FIG. 6, for example, and u ⁽⁰⁾ is used for storing the line memories 71a-71.
c gives a delay of one to three rows. Then, the outputs of the line memories 71 a to 71 c are input to the 3 × 3 product-sum operation circuit 72. The 3 × 3 product-sum operation circuit 72 multiplies each pixel by the coefficient shown in FIG. 7 for the output of each of the line memories 71a to 71c, and sums the results to obtain an average value u.
Calculate _av ⁽⁰⁾ . The same applies to the average value calculation circuit 61b.
BR> a configuration of calculating the v mean value for _{^{(0) v av (0)}} .

In this operation, u _av ⁽⁰⁾ , v _av ⁽⁰⁾
Are output from the average value calculation circuits 61a and 61b with a delay of two rows and one pixel with respect to u ⁽⁰⁾ and v ⁽⁰⁾ . Therefore, the delay circuit 17a calculates a delay corresponding to this delay by coefficients a, b,
After being given to c, d, and e, it is output to the motion field u, v operation circuit 16a. The multiplier 62a in the motion field u, v calculation circuit 16a receives the average value u _av ⁽⁰⁾ and the coefficients a, au _av
Calculate ⁽⁰⁾ . The multiplier 62b calculates the average value v _av
⁽⁰⁾ and the coefficient b are input, and bv _av ⁽⁰⁾ is calculated. The multiplier 62c receives the average value v _av ⁽⁰⁾ and the coefficient c,
Calculate _av ⁽⁰⁾ . The multiplier 62d calculates the average value u _av
⁽⁰⁾ and coefficient b are input, and bu _av ⁽⁰⁾ is calculated.

Then, the uv calculator 63a performs the operation of the equation (19 ⁾ to calculate u ⁽¹⁾ . The uv operation unit 63
b calculates v ⁽¹⁾ by performing the operation of equation (20).

U ⁽¹⁾ = au _av ⁽⁰⁾ −bv _av ⁽⁰⁾ −d (19) v ⁽¹⁾ = cv _av ⁽⁰⁾ −bu _av ⁽⁰⁾ −e (20 ⁾ )

The calculated u ⁽¹⁾ and v ⁽¹⁾ are given to the next stage motion field u and v operation circuit 16b. Further, the coefficients a, b, c, d, and e delayed by the second-stage delay circuit 17b are also provided to the motion field u, v operation circuit 16b.

Thus, the motion field u, v operation circuit 16b
It operates in the same manner as the motion field u, v operation circuit 16a,
⁽²⁾ and v ⁽²⁾ are output. Motion field u, v operation circuit 16c
U ⁽²⁾ , v ⁽²⁾ and the coefficients a, b, c, d, and e delayed by the third-stage delay circuit 17c are input to u ⁽³⁾ , v
⁽³⁾ is calculated.

As described above, the optical flow calculation with the number of repetitions of 3 is realized. If it is desired to increase the number of repetitions, the number of stages of the motion field u, v operation circuit and the delay circuit may be increased.

Embodiment 2 FIG. In the above embodiment, an optical flow is directly calculated from two images at different times. However, for example, when a position shift occurs in the background due to the movement of the camera or the like, not only the optical flow of the moving object but also the optical flow due to the relative movement of the background is detected. Then, it is difficult to extract the optical flow of the moving object.

Therefore, as shown in FIG.
And the second image 12 are temporarily stored in the first frame memory 91 and the second frame memory 92, respectively. At the same time, the first image 11 and the second image 12 are input to the positioning circuit 93. Positioning circuit 93
Detects the amount of background displacement. Then, the position for reading data from the second frame memory 92 to the gradient calculation circuit 13 is controlled by the position shift correction signal 94 so that the position shift of the background is eliminated. In this way, the occurrence of an optical flow due to the displacement of the background is prevented,
An optical flow using only a moving object is obtained. Note that the misalignment correction unit includes the first frame memory 91 and the second frame memory 91.
This is realized by a frame memory 92 and a positioning circuit 93. The configuration of the portion after the gradient calculation circuit 13 is the same as the configuration shown in FIG.

Embodiment 3 FIG. In the first embodiment, the gradient operation circuit 1
3, the first image 11 and the second image 12 are input, and 4 ×
The two-product sum calculation circuits 32a to 32c have calculated the spatial density gradient and the time density gradient. However, equations (2) to (4) are
Since the sum and the difference of the same pixel positions different in time are included, they can be expressed by the equations (21) to (25).

I _x = {− g (i, j) + g (i + 1, j) −g (i, j + 1) + g (i + 1, j + 1)} / 4 (21) _{I y = {- g (i} , j) -g (i + 1, j) + g (i, j + 1) + g (i + 1, j + 1)} / 4 ··· (22) I t = {H (i, j) + h (i + 1, j) + h (i, j + 1) + h (i + 1, j + 1)} / 4 (23)

G (i, j) = I (i, j, t + 1) + I (i, j, t) g (i + 1, j) = I (i + 1, j, t + 1) + I (i + 1, j, t) g (i, j + 1) = I (i, j + 1, t + 1) + I (i, j + 1, t) g (i + 1, j + 1) = I (i + 1, j + 1, t + 1) + I (i + 1, j + 1, t) (24)

H (i, j) = I (i, j, t + 1) −I (i, j, t) h (i + 1, j) = I (i + 1, j, t + 1) −I (i + 1, j, t) h (i, j + 1) = I (i, j + 1, t + 1) −I (i, j + 1, t) h (i + 1, j +1) = I (i + 1, j + 1, t + 1) -I (i + 1, j + 1, t) (25)

That is, if the sum and difference of the two images are calculated in advance, the product-sum operation circuit in the gradient operation circuit 13 may be 2 × 2. FIG. 9 is a block diagram showing a configuration of the gradient operation circuit 13 in the optical flow operation circuit according to the third embodiment of the present invention based on such a concept. As shown in FIG.
Performs the operation of Expression (24), and the subtractor 102 performs the operation of Expression (25).

The output of the adder 101 is supplied to the line memory 31
a and the line memory 31b are delayed by one or two rows. The output of the subtracter 102 is delayed by one or two lines in the line memories 31c and 31d. The outputs of the line memories 31a and 31b are 2 × 2 multiply-accumulate circuits 103a and 103
b, the output of the line memory 31c and the output of the line memory 31d are input to the 2 × 2 product-sum operation circuit 103c.

Then, the 2 × 2 product-sum operation circuit 103a, 1
03b and 103c are shown in FIGS. 10 (a), (b),
By multiplying by the coefficient shown in (c) and taking the sum of the results,
I _x , by equations (21), (22) and (23), respectively.
I _y, to calculate the I _t.

Embodiment 4 FIG. In the first embodiment, the coefficient calculation circuit 1
No. 4 calculates coefficients according to the equations (12) to (16). Here, assuming that the first image and the second image are each represented by 8 bits, the output of the gradient calculation circuit 13 is a 9-bit signed integer. Also, if the constant α is input with a maximum of 8 bits, α is considered in consideration of the sign.
² , I _x ² and I _y ² are each 16 bits, I _x I _y ,
_{I x I t, I y I} t are each a 17-bit integer. Further, α ² + I _x ² + I _y ² is 18 bits, α ²
Each of + I _x ² and α ² + I _y ² requires 17 bits, and the internal data amount increases.

Here, paying attention to the fact that the calculation in the coefficient calculation circuit 14 is finally a ratio calculation, and considering that the value of the optical flow should have the same gradation as that of the input image, each coefficient is calculated. Need only have an effective digit of about 8 bits. _{Accordingly, α, I x, I y} , may be shifted each value as absolute value of I _t is maximum is that of figures about 4 bits. That is, the bit number reduction circuit shown in FIG. 11 may be placed between the gradient operation circuit 13 and the coefficient operation circuit 14 in the configuration shown in FIG.

[0056] The maximum value detecting circuit 121 in the number of bit reduction circuit, alpha, absolute value of I _x, I _y, I _t detects the maximum one. Then, the maximum value is determined by the shift amount determination circuit 1
22. The shift amount determining circuit 122 determines a shift amount such that the highest-order "1" of the maximum value is at the fourth bit, and supplies the shift amount to each of the shift circuits 124a to 124d. Each shift circuit 124a~124d according shift amount, respectively _{α, I x, I y,} s α obtained by subtracting the number of bits by shifting the _{I t, I xs, I ys} , and outputs the I _ts.

[0057] In this case, the coefficient calculation circuit 14 shown in FIG. 4, alpha, I _x, I _y, instead of I _t α _s, I
_xs, performs coefficient computation with I _ys, the I _ts. Therefore, the multipliers 51a to 51c and 52a to 52c and the adder 53,
The number of bits of 54a and 54b is reduced. Further, the number of bits is reduced for both the numerator and denominator of the division in the dividers 55a to 55e, and the configuration is simplified. Note that the division may be realized by a conversion table using a ROM in which the numerator and denominator are input addresses and the division result is output, and in that case, the processing speed is further increased.

[0058]

As described above, according to the first aspect of the present invention, the optical flow arithmetic circuit calculates the necessary coefficients in advance by the coefficient arithmetic circuit, and the delay circuit delays those coefficients by the delay circuit. Since the motion field u, v operation circuit is configured to be capable of pipeline operation, no memory is required, and the optical flow operation can be executed faster than in the conventional case, and the optical flow can be executed in real time. There is an effect that what can be obtained is obtained.

According to the second aspect of the present invention, the optical flow calculation circuit has a configuration in which the position shift correction unit corrects the position shift of the background between the two images. This has the effect of preventing occurrence and obtaining an optical flow of only the moving object in the image.

According to the third aspect of the present invention, the optical flow calculation circuit calculates an addition value and a subtraction value between pixels at the same position in two images and then calculates the spatial density gradient between the images. Since the configuration for calculating the time-density gradient is used, there is an effect that the configuration of the gradient calculation circuit can be simplified.

According to the fourth aspect of the present invention, the optical flow operation circuit has a configuration in which the bit number reduction circuit reduces the number of bits between each output of the gradient operation circuit and the weight constant of the coefficient operation. There is an effect that a configuration that can simplify the configuration of the coefficient operation circuit is obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an optical flow operation circuit according to one embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a gradient calculation circuit.

FIG. 3 is an explanatory diagram showing coefficients in a 4 × 2 product-sum operation circuit.

FIG. 4 is a block diagram illustrating a configuration of a coefficient operation circuit.

FIG. 5 is a block diagram showing a configuration of a motion field u, v operation circuit.

FIG. 6 is a block diagram illustrating a configuration of an average value calculation circuit.

FIG. 7 is an explanatory diagram showing coefficients in a 2 × 2 product-sum operation circuit of the average value operation circuit.

FIG. 8 is a block diagram showing a main part of an optical flow operation circuit according to another embodiment of the present invention.

FIG. 9 is a block diagram showing another configuration of the gradient operation circuit.

FIG. 10 is an explanatory diagram showing coefficients in a 2 × 2 product-sum operation circuit in FIG. 9;

FIG. 11 is a block diagram illustrating a configuration of a bit number reduction circuit.

FIG. 12 is a block diagram showing a configuration of a conventional optical flow operation circuit.

[Explanation of symbols]

 Reference Signs List 13 gradient calculation circuit 14 coefficient calculation circuit 16a to 16c motion field u, v calculation circuit 17a to 17c delay circuit 91 first frame memory (position shift correction section) 92 second frame memory (position shift correction section) 93 positioning circuit ( Position shift correction unit) 121 Maximum value detection circuit (bit number reduction circuit) 122 Shift amount determination circuit (bit number reduction circuit) 124a to 124d Shift circuit (bit number reduction circuit)

Claims (4)

(57) [Claims] 1. A gradient calculation circuit for inputting two images at different times and calculating a spatial density gradient and a time density gradient between the images, and repeating an optical flow from each output of the gradient calculation circuit A coefficient calculation circuit for calculating a coefficient of calculation, a plurality of motion fields u and v calculation circuits connected in tandem for calculating a motion field representing an optical flow using an output of the coefficient calculation circuit; An optical flow arithmetic circuit including a delay circuit for delaying an output and synchronizing the output with the operation of each motion field u, v arithmetic circuit. 2. A position shift correcting unit for inputting two images at different times, detecting a position shift of a background between the images, and correcting the position shift, and correcting the position shift by the position shift correcting unit. A gradient calculation circuit for inputting a plurality of images and calculating a spatial density gradient and a time density gradient between the images, and a coefficient calculation circuit for calculating a coefficient of an optical flow repetition calculation from each output of the gradient calculation circuit And a plurality of motion fields u, v connected in cascade for calculating a motion field representing an optical flow using an output of the coefficient operation circuit.
An optical flow calculation circuit comprising: a calculation circuit; and a delay circuit for delaying an output of the coefficient calculation circuit and synchronizing the output with the calculation of each motion field u, v calculation circuit. 3. After two images at different times are input, an addition value and a subtraction value between pixels at the same position in the images are calculated, and a spatial density gradient and a time density gradient between the images are calculated. , A coefficient operation circuit for calculating the coefficient of the optical flow repetition operation from each output of the gradient operation circuit, and a motion field representing the optical flow using the output of the coefficient operation circuit. An optical system comprising: a plurality of motion field u, v operation circuits connected in tandem; and a delay circuit for delaying the output of the coefficient operation circuit and synchronizing the output with the operation of each motion field u, v operation circuit. Flow operation circuit. 4. A gradient calculation circuit for inputting two images at different times and calculating a spatial density gradient and a time density gradient between the images, and outputs of the gradient calculation circuit and weights for coefficient calculation. A bit number reduction circuit that reduces the number of bits with a constant, a coefficient calculation circuit that calculates a coefficient of an optical flow repetition calculation from each output of the bit number reduction circuit, and an optical flow using the output of the coefficient calculation circuit. A plurality of motion field u, v operation circuits connected in a column for calculating the motion field to be represented; and a delay for delaying the output of the coefficient operation circuit and synchronizing the output with the operation of each motion field u, v operation circuit. An optical flow operation circuit comprising a circuit.