JP2796221B2 - Correlation processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a correlation processing method,
In particular, the present invention relates to a correlation processing method for obtaining a relationship between graphic features between two images based on a correlation parameter which is a result of the correlation processing.

[0002]

2. Description of the Related Art In order to control the movement and autonomous operation of a robot, it is necessary to three-dimensionally grasp the environment in which the robot operates. Such stereoscopic grasping includes “binocular stereopsis”, which measures depth based on the principle of triangulation from the difference in the appearance of two eyes (parallax) on the left and right, and “motion, There is "stereoscopic vision". Since we perform stereoscopic vision with two eyes, development of binocular stereoscopic vision has been attempted for a long time. However, in binocular stereopsis, it is necessary to extract corresponding parts from the images captured by the left and right eyes, and it is difficult to extract such corresponding points, and it has not been practical yet.

FIG. 60 explains the principle of binocular stereopsis, and it is assumed that an object is placed at two points A and B on a plane. Each eye can see only the directions of A and B, but the intersection of the line of sight of the left eye looking at A and the line of sight of the right eye looking at A can tell the depth of A, and the depth of B can be similarly determined. Perceived. That is, as shown in FIG. 61 , assuming that the distance between the eyes is d, and the angles between the line of sight of the left and right eyes and the vertical line are ρ _L and ρ _R , the depth D is represented by the following equation: D = d / (tan ρ _L + tan ρ) _R ).

[0004]

However, the intersection of the line of sight can be changed. The line of sight of the left eye looking at A also intersects with the line of sight of the right eye looking at B. This intersection is a false point β, and similarly, a false point α is generated, and it is necessary to remove this false point in binocular stereopsis. In our cerebrum, the ability to see shape (morphology) has been developed, so we can easily remove false points α and β,
It is difficult with conventional binocular stereoscopic technology, and it is a problem as a corresponding point problem.

As described above, a first object of the present invention is to provide a correlation processing method capable of accurately determining corresponding points of a plurality of figures with a simple procedure and a small amount of processing, and realizing a binocular stereoscopic function. It is. A second object of the present invention is to track a moving object in association with a temporally different image,
An object of the present invention is to provide a correlation processing method capable of measuring the moving direction and the moving speed. A third object of the present invention is to provide a correlation processing method that can be applied to texture analysis or the like for examining the degree of the same pattern in one screen. A fourth object of the present invention is to provide a correlation processing method that further ensures binocular stereopsis, moving object gaze, and texture analysis by performing color correlation processing for each of the three primary colors or each of the three elements of color. It is.

A fifth object of the present invention is to obtain a corresponding tangent (line / gap, edge) in a plurality of figures,
Moreover, it is an object of the present invention to provide a correlation processing method capable of quantitatively obtaining the position, the direction, the parallax, and the speed. The sixth object of the present invention is capable of performing accurate filtering, and is suitable for application when performing “extraction of a feature that looks the same with the left and right eyes”, “tracking of the same feature as the previous image”, and the like. It is to provide a simple correlation processing method. A seventh object of the present invention is to provide a correlation processing method capable of performing binocular stereoscopic vision and tracking of an object having an indistinct outline using clues such as gradual changes in brightness and hue.

[0007]

FIG. 1 is a diagram illustrating the principle of the present invention. 14 and 24 are polar conversion units that perform polar conversion processing on the first and second image data, 16 and 26 are one-dimensional filters that perform one-dimensional filter processing on the polar conversion results, and 17 and 27 are filter processing results ( Ρ- that stores the hypercolumn image)
A dual plane (hypercolumn memory) 30 of θ is a correlation processing unit that performs a correlation process between data mapped on each dual plane.

[0008]

The polar converter 1 converts the first and second image data.
The polar conversion processing is performed at 4, 24 to make the dual planes 17, 27
Alternatively, the result of the polar conversion processing is further filtered by one-dimensional filters 16 and 26 to be mapped to dual planes 17 and 27, respectively, and the correlation processing unit 30 determines the position of the mapped data on the dual plane ( ρ, θ) and the interval (shift amount) between the mapping data to be subjected to the correlation processing is used as an element parameter to obtain a correlation amount (correlation parameter), and a point having a characteristic correlation parameter (for example, a maximum value) is obtained. Variables (parallax, moving direction, speed, etc.) that define the relationship between the graphic features (eg, tangents) of each image are measured based on the values of the element parameters that provide the various correlation parameters. By the polar transformation, the contour tangent (the straight line itself in the case of a straight line) included in the first and second images is transformed by reducing the dimension to a point. This can be replaced with a point correspondence problem, and as a result, the corresponding portion of a plurality of figures can be easily and accurately determined with a small amount of processing by correlation processing, and a binocular stereoscopic function can be realized.

The amount of processing can be significantly reduced by performing polar conversion processing, filter processing, and correlation processing on a receptive field image obtained when one screen is divided into small receptive fields. Furthermore, when each receptive field image belongs to a different screen captured by two or three cameras, the function of binocular stereoscopic vision and trinocular stereoscopic vision can be realized, and each receptive field image belongs to a temporally different screen. In this case, it is possible to measure the moving direction and moving speed of features (lines, corners, etc.) in the receptive field, so that it is possible to move the object while capturing it at the center of the field of view, and to use a mobile robot or unmanned It can be expected to be applied to running vehicles.

Further, by making each receptive field image an image of a different receptive field on the same screen, or by making the same receptive field image,
Texture analysis or the like for checking the degree of the same pattern in one screen can be performed. Furthermore, the correlation processing between a plurality of receptive field images is performed for each color, or for each color difference signal,
Alternatively, binocular stereopsis, moving object gaze, and texture analysis can be performed more reliably by performing each of the three color components.

Further, by performing one-dimensional Gaussian filter processing after polar conversion, or performing two-dimensional Gaussian filter processing before polar conversion processing, it is possible to obtain corresponding lines or gaps in a plurality of figures. ,
Also, by performing one-dimensional gradient filter processing and one-dimensional Gaussian filter processing after polar conversion, or performing two-dimensional Gaussian filter processing before polar conversion processing and performing one-dimensional gradient filter processing after polar conversion processing , Corresponding edges in a plurality of figures can be obtained, and furthermore, the positions, orientations,
Parallax, moving direction, and moving speed can be obtained.

Further, when a plurality of receptive field images belong to spatially different screens, a correlation parameter C (ρ,
θ, σ), or (ρ, θ)
By calculating the plane correlation parameter C (ρ, θ, σ ₁ , σ ₂ ), it is possible to extract a tangent that moves while changing the azimuth. When a plurality of receptive field images belong to temporally different screens, the correlation parameter C (ρ,
θ, τ), the position, azimuth, and velocity of the tangent that translates can be quantitatively determined.
By calculating the axial correlation parameter C (ρ, θ, τ), the position, azimuth, azimuth rotation speed, and the like of the tangent passing through the center of the receptive field can be quantitatively obtained.
θ) plane correlation parameter C (ρ, θ, τ ₁ , τ ₂ )
By calculating, the position, azimuth, moving speed, rotation speed, and the like of the tangent that moves while changing the direction can be quantitatively measured.

Further, the correlation parameter C (ρ, θ, σ) is projected in the σ-axis direction indicating the spatial shift amount σ, or projected in the ρ-axis direction indicating the tangential position, or the θ-axis direction indicating the tangential direction. , Or by projecting in any of these two directions, the memory capacity for storing the correlation parameter can be reduced by one or two axes, and the tangent position, parallax, A desired value among the azimuths and the like can be obtained. Also, the correlation parameter C
(Ρ, θ, τ) is projected in the τ-axis direction indicating the temporal shift amount τ, or projected in the ρ-axis direction indicating the tangential position, or projected in the θ-axis direction indicating the tangential direction, or any of these. By projecting in two axial directions, the memory capacity for storing correlation parameters can be reduced by one or two axes, and by selecting the projection direction, the position, orientation, translation speed, rotation speed, etc. of the tangent can be determined. A desired value can be obtained.

Further, the receptive field image is subjected to a polar transformation process to be mapped on a dual plane of ρ-θ, and the mapping data a (ρ, θ) of the dual plane and a mapping set on another dual plane. By extracting a set of data b (ρ, θ) whose coordinate values are shifted by a predetermined amount in the ρ-axis direction and calculating the sum of the products, accurate filtering can be performed.
It is suitable to be applied when performing “extraction of a feature that looks the same with the left and right eyes”, “tracking of the same feature as the previous image”, and the like.

Further, the mapping data a (ρ,
θ), the other mapping data b (ρ, θ) is shifted in the ρ-axis direction or the θ-axis direction and then subtracted.
The shift amount is sequentially changed and subtracted, and the obtained subtraction result is used as a correlation parameter. Can be done.

[0016]

【Example】

(A) Outline of the present invention In binocular stereopsis, conventionally, a corresponding "figure"
I was exploring. For this reason, it has been difficult for current computers that are vulnerable to shape processing to make a stable response decision. Therefore,
The simplest feature, the “tangent” that makes up the outline of a figure
(The element when the contour is cut by a minute length, and in the case of a straight line, the straight line itself) is used to determine the correspondence. this"
In the "tangent" method, it is sufficient to compare tangents (straight lines) appearing in both eyes, and the procedure is simple and stable, and correspondence can be determined.
In the following description, the terms tangent line, contour tangent line, and contour line appear, but when considering a receptive field image in a receptive field, which is a small region described later, these terms are synonyms. Such a tangent comparison makes the figure to be handled simpler, but requires a corresponding tangent to be determined on a two-dimensional screen, which takes a long time to process. By the way, when the input image is polar-transformed and mapped onto a dual plane having the line direction θ and the position ρ as coordinate axes, the “tangents” in the left and right images are converted into reduced “dimensions” into “points”.
Therefore, the problem of tangent correspondence in two dimensions can be solved in one dimension, and the processing amount can be greatly reduced.

FIG. 2 explains the principle of binocular stereoscopic vision of the present invention.
Things. 1. The "tangent" in the space is the input image IM of the left and right eyes_L, IM_R
Then SL_L, SL_RIt appears to be shifted in parallel as shown by. 2. On the other hand, hyper-colla obtained by polar transformation of the input image
Ρ-θ dual plane HCP_L, HCP_R
Then, the inclination is θ_P“Tangent” (parallel line) is θ = θ_PΡ axis
It is converted to the above "dot sequence". 3. Therefore, the input image IM of the left and right eyes_L, IM_RNot parallel
"Tangent line" SL_L, SL_RIs a dual plane
HCP_L, HCP_RThen the same θ_PPoints P on the ρ axis of_L, P_R
Is converted to 4. Distance σ between the two points_PAnd the left and right images IM_L,
IM_R"Tangent" SL at _L, SL_RThe amount of translation of
Parallax is determined. 5. The σ_PAnd the distance between the eyes, the spatial depth of the "tangent"
And the binocular stereoscopic function has been realized.
You. From the above, by performing polar transformation on the input image,
Comparing "tangents" with "comparing" points "on the ρ-axis
Dimensional processing '', and the correspondence can be determined with a small amount of processing.
I can.

The "polar transformation" and the "receptive field" required later will be described below. The polar conversion is a polar conversion on an arbitrary curved surface such as a polar conversion on a spherical surface, a polar conversion on a cylinder, or a polar conversion on a plane.
FIG. 3 is an explanatory diagram of a spherical mapping (polar transformation on a spherical surface). As shown in FIG. 3 (a), the spherical mapping is an arbitrary point P on the spherical surface.
Is converted to a great circle R (which is the largest circle on the spherical surface and corresponds to the equator) having that as a pole. According to this spherical mapping, as shown in FIG. 3B, the points P ₁ , P
₂ , P _3, ..., The great circles R ₁ , R ₂ ,... Having the points P ₁ ′, P ₂ ′, P ₃ ′.
When R ₃ ... Are drawn, each great circle always intersects at one point S. The intersection S is a unique point corresponding to the line segment L on a one-to-one basis. The longer the line segment L, the greater the number of points that are elements of L, and therefore the greater the number of great circles, and the greater the degree of overlap of great circles in S. In this way, the line segment is extracted as a point on the spherical surface corresponding to the line segment, and the length of the line segment can be measured by taking a histogram of the degree of intersection of the great circle at each point.
Note that the geometric meaning of the point S corresponding to the line segment L is “line segment L
Is a pole whose projection L ′ onto the spherical surface is a great circle. ”

FIG. 4 is an explanatory diagram of pole conversion on a cylinder. In the polar transformation on a cylinder, a straight line (intersection line of an ellipse) and a point change with each other. That is, the straight line S on the plane PL passing through the origin O
Is projected onto a cylinder, the pole P is converted into a pole P, which is the intersection of the cylinder with the normal NL of the plane PL at the origin O, and the pole P is converted back into an ellipse ELP. If this cylinder is cut open parallel to the axis, it can be developed into a plane. On this plane, a straight line (sinusoidal curve) and a point alternate with each other. From the above, the outline of the image input from the camera is extracted and written into the projection image memory, the outline point is converted into a great circle or a sine curve by the polar converter, and the information of the great circle or the sine curve is written into the mapping memory. In the same manner, all the contour points are polar-converted into a great circle or a sine curve, and then each cell of the mapping memory is scanned to find the peak position of the count value. And a line segment can be extracted. Each cell of the mapping memory is constituted by, for example, a counter or the like, so that the storage content is increased each time data is written.

Receptive field method In the method of polar conversion of the entire screen to extract line segments, the processing amount increases, and high-speed processing cannot be performed. The reason is that each point in the input image is pole-transformed to extract line segments,
In other words, there is an object in which the dimension is enlarged and mapped to a great circle or a sinusoidal curve on a spherical surface. Input image size N × N
If, to convert each point on the great circle or sinusoids of length N in polar transformation is an obstacle for speed requires processing N times N ³ of the input image. Therefore, when the input image is divided into images (receptive field images) for each receptive field, which is a small area, and the polar transform is performed on each receptive field image, the processing amount is greatly reduced, and the High-speed pole conversion can be performed with hardware.

FIG. 5 is a diagram for explaining the principle of the receptive field method. Reference numeral 1 denotes an input memory for storing a target object image (input image) IMG of N × N size projected on a predetermined input plane, and 2 denotes an input plane of m. When divided into receptive fields which are small areas of × m, a receptive field memory for storing an image (receptive field image) in each receptive field, a polar conversion unit 3 for performing polar conversion on each receptive image, and a polar conversion unit 4 for performing polar conversion Ρ-θ dual plane (hypercolumn memory) for mapping the image (hypercolumn image).

The target object image IMG of the N × N size stored in the input memory 1 is divided into receptive fields, which are small areas of m × m, and each receptive field image is stored in the receptive field memory 2 in order, and pole conversion is performed. The unit 3 performs polar transformation on each receptive field image, maps the output (hypercolumn image) on the dual plane 4, and extracts features such as line segments based on the polar transformation output.

The specific procedure of the receptive field method will be described with the simplest polar transformation, that is, “plane projection input + cylindrical pole transformation”. That is, "the pixels in each receptive field obtained by dividing the input image into mxm are pole-to-cylinder transformed, and the obtained curve (sine wave in the pole-to-cylinder transformation) is drawn on a dual plane". The above is the case of “plane projection input + cylindrical pole conversion”. However, in the case of “spheric projection input + spherical top pole conversion”, the sine curve may be changed to a great circle in the above. The main flow is shown below. (a) Divide the image projected on the sphere into small areas (receptive fields). (b) Polarize each pixel in the receptive field on the sphere (draw a large circle corresponding to the pixel). (c) Entire receptive field Is pole transformed into a band on a sphere, but the receptive field is generally small, so this band is developed into a plane. This is a dual plane (hypercolumn plane) corresponding to each receptive field. The polar transformation of (b) is approximated as “pixel → sine wave”, and the transformation between the receptive field and the hypercolumn is “convert each pixel of the receptive field to a corresponding sinusoidal curve on the dual plane”. . Each axis in the dual plane indicates the position ρ and direction θ of the line in the receptive field, and the line segment in the image plane (receptive field) is concentrated at the intersection of the sine waves, and the intersection coordinates ρ ₀ and θ ₀ are set in the receptive field. Indicates the position and direction of the line segment at.

Referring to FIG. 6, a detailed explanation will be given.
The point P in C is mapped to a great circle (straight line) R by increasing the dimension by polar transformation. By repeating this, the entire receptive field is mapped to the band BLT on the spherical surface (FIG. 6 (a)). When this band is cut out and developed on a plane (FIG. 6 (b)), a rectangular grid having a line direction θ and a line position ρ as coordinate axes is obtained. By this polar transformation, each pixel in the receptive field is multiply mapped to a plurality of grid points (sinusoidal shape) of the dual plane. By repeating this, the line segment in the receptive field is extracted as the "intersection of the sine wave group" in which the point sequence constituting the line segment is pole-transformed. That is, the intersection Q becomes a concentrated line segment L (FIG. 6 (c)) in the receptive field, and its θ-axis and ρ-axis coordinate values θ ₀ and ρ ₀ indicate the direction and position of the line segment in the receptive field. become.

In the receptive field method, as described above, it is only necessary to perform polar transformation on an image in an m × m receptive field. In the case of polar transformation on a spherical surface, each pixel in the receptive field has a length m. need only draw a great circle, processing amount becomes m × N ^2,
Greatly reduced compared to conventional draw N great circles (N required throughput of ^3), moreover it is possible to downsize the hardware. The details of the polar transformation are described in Japanese Patent Application No. 3-327723, filed by the present applicant (title of image processing method,
Filing date: December 11, 1991).

(B) Basic Configuration of the Present Invention In summary, the basic configuration of binocular stereopsis is input image → polar conversion → one-dimensional correlation processing. As described above, the polar conversion is an operation of converting a “tangent” into a “point” in order to perform a tangent comparison in one dimension. This is an operation to extract the corresponding “point”. The pole-transformed mapping data in the dual plane of both eyes is represented by L (ρ,
θ), R (ρ, θ), interval between mapping data (shift amount)
The correlation amount (referred to as a correlation parameter) is calculated by the following equation: C (ρ, θ, σ) = L (ρ, θ) · R (ρ + σ, θ) (1) Then, when the values (ρ _P , θ _P , σ _P ) of the element parameters ρ, θ, σ at which the correlation amount C (ρ, θ, σ) is maximized are determined based on the values of the element parameters. The tangent is determined, and the position, orientation, and parallax of the tangent are quantitatively determined. The correlation parameter can be calculated by the asymmetric correlation operation of the following equation in addition to the asymmetric correlation operation of the equation (1). C (ρ, θ, σ) = L (ρ−σ, θ) · R (ρ + σ, θ) (1) ′ However, hereinafter, the correlation parameter is calculated by the asymmetric correlation calculation of equation (1). It will be described as. In the following, the terms correlation amount, correlation parameter, and correlation result are used together, but are synonymous.

(C) Embodiment of the Present Invention FIG. 7 shows an embodiment of the present invention for finding the position, orientation, and parallax of the corresponding tangent lines included in two images by input image → receptive field division → polar conversion → one-dimensional correlation processing. It is an example block diagram. In the figure,
11 and 21 input memory for storing input image IM _L, IM _R of the right and left eyes, 12 and 22 when dividing the input plane receptive field is a small area of m × m, sequential receptive field of the image (receptive field Receptive field extracting circuit for extracting and outputting an image), 1
Reference numerals 3 and 23 denote receptive field memories for storing images (receptive field images) in each receptive field, 14 and 24 denote polar conversion circuits for performing polar conversion on each receptive field image, and 15 and 25 denote polar-converted dual planes (hyperplanes). Image on column plane) (hyper column image)
Is a pole conversion hyper column memory for storing the data. The pole conversion hypercolumn memories 15 and 25 are composed of ρmax in the ρ direction, θmax in the θ direction, and a total of ρmax × θmax storage areas (hypercolumn cells).

Reference numerals 16 and 26 denote one-dimensional filter circuits for performing one-dimensional filter processing on the hypercolumn images obtained by the polar transformation, and reference numerals 17 and 27 denote dual planes (hypercolumn memory) for storing the filtered hypercolumn images. ) And 30 are correlation processing units. In the correlation processing unit 30, reference numeral 31 denotes a correlation operation unit that performs a one-dimensional correlation operation according to the equation (1), and 32 denotes a ρ- that stores a correlation value (correlation parameter).
A correlation parameter storage unit having a dimension of σ-θ, and a peak detection unit 33 that scans data stored in the correlation parameter storage unit and detects local maximum points (ρ _P , θ _P , and σ _P ).

The polar conversion circuits 14 and 24 polar-convert each pixel in the receptive field, that is, convert the pixel into a corresponding great circle and store it in the polar-converted hypercolumn memories 15 and 25.
In practice, the polar conversion is approximated as “pixel → sine wave”, and each pixel in the receptive field is converted into a corresponding sine curve on the hypercolumn and stored in the polar conversion hypercolumn memories 15 and 25.

The one-dimensional filter circuits 16 and 26 are for extracting sharp lines, edges and gaps with a small amount of processing. In order to perform contour enhancement and feature extraction, usually, a two-dimensional convolution filter is applied to image data, and then polar conversion is performed. However, according to the convolution method, the process of a ² filter sizes for each input point When a is required, the processing amount with increasing filter size increases. However, since “two-dimensional convolution filter + polar transformation” is equivalent to “performing one-dimensional filtering in the ρ direction after polar transformation”, the embodiment of FIG. 7 applies a one-dimensional filter after polar transformation. . By doing so, the processing amount is as small as a for each input point because of the one-dimensional filter, and the processing amount is reduced to about 2 / a as compared with the convolution method.

Correlation operation The correlation operation unit 31 includes a mapping data L (ρ, θ) in the left-eye dual plane 17 and a right-eye dual plane shifted σ in the ρ-axis direction from the mapping data L (ρ, θ). 27, and multiplies it by the mapping data R (ρ + σ, θ) and stores the result in the correlation parameter storage unit.
To σmax, ρ to 0 to ρmax (receptive field width), θ to 0
Each multiplication result is stored in the correlation parameter storage unit 32 while being changed to θmax (reception field width).

FIG. 8 is a flow chart of the correlation calculation process, where 0 → θ, 0 → ρ, 0 → ρ are set in the correlation calculation (steps 101 to 103). Next, the correlation parameter is calculated by equation (1) and stored in the correlation parameter storage unit 32.
(Steps 104 and 105). Then, σ is incremented, and it is determined whether σ> σmax (steps 106 and 107).
4 and the subsequent processing is repeated. If σ> σmax, ρ is incremented and it is determined whether ρ> ρmax (steps 108 and 109), and ρ ≦ ρmax
If so, the process returns to step 103 and the subsequent processing is repeated. If ρ> ρmax, θ is incremented and it is determined whether θ> θmax (steps 110 and 11).
1) If θ ≦ θmax, the process returns to step 102 and the subsequent processes are repeated. If θ> θmax, the correlation calculation process ends.

The polar conversion circuit polar transformation circuit 14 (also polar transformation circuit 24), as shown in FIG. 9, read sequentially pixel by pixel receptive field image (amplitude) from the address of the receptive field memory 13,該番point and amplitude A read control unit 14a that outputs a value, a polar conversion unit 14b that performs a polar conversion (a conversion from a pixel address to a sine wave address) for each pixel,
A write control unit 14c is provided for writing the read amplitude to a plurality of storage locations of the pole conversion hypercolumn memory 15 indicated by the sine wave address obtained by the pole conversion.

The polar conversion unit 14b is provided with an address conversion table memory 14b-1 for making the pixel addresses of the receptive field memory 13 correspond to a plurality of addresses of the polar conversion hypercolumn memory 15, and a polar conversion hypercolumn memory for converting the address of the receptive field memory 13. An address conversion circuit 14b-2 for converting the address into an address of 15 is provided. The address conversion table stored in the address conversion table memory 14b-1 is used to polar-convert a point in the receptive field image into a sine wave on the hyper column plane. The conversion into a number of addresses on the pole conversion hyper column memory where each point of the point sequence forming the sine wave is located.

The read control unit 14a includes the receptive field memory 1
3, the amplitude data is read from the first address, and the amplitude data and the address data (first address) are input to the pole conversion unit 14b. The pole conversion unit 14b is the first address (pixel) of the receptive field memory.
Is converted into a number of addresses of a sine wave point sequence in the pole conversion hyper column memory 15 and the addresses and amplitudes are output. The write control unit 14c adds the amplitude data input from the pole conversion unit 14b to the contents (initial value is 0) of each address of the pole conversion hypercolumn memory 15 similarly input, and writes the data to the address. Thereafter, the reception area memory 14
When the above-described processing is performed for all the addresses, the polar conversion for the receptive field image is completed.

[0036] From one-dimensional filter circuit one-dimensional filter circuit 16 (also one-dimensional filter circuit 26), as shown in FIG. 10, polar transformation Hypercolumn certain ρmax number of addresses θ in the memory 15 (e.g. see the hatched portion) A read control unit 16a for reading and outputting the amplitude, a one-dimensional filter unit 16b for performing one-dimensional filtering processing on each read amplitude, and storage of a one-dimensional filtering processing result in a dual plane (hyper column memory) 17 Write control unit 16c for writing to the area
It has.

The one-dimensional filter section 16b has, for example, an FIR digital filter configuration, and has a one-dimensional filter memory 16b-1 and a product-sum circuit 16b-2. The one-dimensional filter memory 16b-1 discretely stores one-dimensional filter characteristic values (coefficients) with the horizontal axis being the ρ axis. That is, ρ
A one-dimensional filter characteristic value is stored for each position of -ρmax / 2 to ρmax / 2 in the direction, and the one-dimensional filter unit 16b is used for the one-dimensional primary differential filter and the one-dimensional secondary differential filter based on the characteristic value. It can be. The width and value of the filter are appropriately determined as needed. Product-sum circuit 16
b-2 is ρmax read from the pole conversion hypercolumn 15
Are multiplied by the corresponding ρmax characteristic values stored in the one-dimensional filter memory 16b-1 to output the sum (amplitude) of the multiplication results and shift the correspondence by one pixel at a time. And outputs the operation result.

For example, first, all addresses (ρ
ρmax amplitudes are read from (max addresses). Then
Filter characteristic ρ in one-dimensional filter memory 16b-1 =
The position of 0 is associated with ρmax leftmost addresses A _0i . Thereafter, the corresponding amplitude value is multiplied by the characteristic value, the sum Σ _{0i of the} multiplication result is calculated, and the sum is written to the hyper column memory 17.
_Is written to the address A _0i .

When the above processing is completed, the filter characteristic value
The position of ρ = 0 is the second address A from the left._1iCorresponding to
The corresponding amplitude value is multiplied by the characteristic value, and the product is multiplied.
Sum of calculation results Σ_1iThe write controller 16c
-Address A of column memory 17 _1iWrite to. After that, the same
And the position of the filter characteristic value ρ = 0
Ground A_{max, i}If the above processing is completed in accordance with
Repeat the above product-sum operation as + 1 → i to perform one-dimensional fill
Performs a tarring process.

Simulation Results FIGS. 11 to 13 are explanatory diagrams of simulation results when a one-dimensional Gaussian filter is used as the one-dimensional filters 16 and 26 in FIG. 7 to extract lines. Images IM _L , I by polar transformation and one-dimensional Gaussian filtering
The “line” in M _R is converted into a “point” in a dual plane with the coordinate ρ of the line position and the azimuth θ of the line. FIG.
(a), the circular part C _L in (b), C _R are the left and right eyes receptive field, the image of the circular is receptive field images caught on the right and left eyes, cross-hairs of the portion surrounded by a scene in a square in the plant The figure of is reflected.
In all data, the signal strength is indicated by contour lines.

FIGS. 12 (a) and 12 (b) are hypercolumn images on the dual plane of ρ-θ which have been subjected to the above-mentioned “polar conversion + one-dimensional Gaussian filter” processing. Are two points P _L1 on the ρ axis of 90 ⁰ and 180 ⁰ , respectively.
P _L2 ; converted to P _R1 and P _R2 . Incidentally, those peaks are sharpened by one-dimensional filtering, and as a side effect thereof, negative sub-peaks P _L1 ′ and P _L2 ′ are provided on both sides;
P _R1 ', P _R2' is accompanied by. This sub-peak is very effective in sharpening the next correlation process.

FIG. 13 shows the result of calculating the correlation amount (correlation parameter) C (ρ, θ, σ) of the equation (1) from the dual plane data of the left and right eyes, where the horizontal axis is ρ (position) and the vertical axis is σ (parallax),
Depth developed in ρ-σ-θ plane and theta (orientation), indicates a correlation value by contour lines, for convenience of description, the top is shown [rho-theta plane theta = 90 ⁰ . When (ρ _P , θ _P , σ _P ) at which the correlation parameter C (ρ, θ, σ) reaches a peak (maximum) is obtained, the correspondence between the crosshairs reflected in the left and right eyes is quantitatively obtained, and ρ _P , Θ _P , and σ _P indicate the position, orientation, and parallax of the line, respectively. When the two lines constituting the cross line vertical bar (theta = 90 ⁰⁾ is try seeking parallax visible left and right eyes, a 1 ^0, calculate the distance to the vertical bars the distance between both eyes as a 6cm Then, it becomes 360 cm, and it can be confirmed that binocular stereoscopic vision has been accurately performed.

Evaluation As described above, the following effects can be obtained by the basic processing (polar conversion + one-dimensional correlation processing). Effect of comparing the features of figures with tangents It is difficult to find the same figure from the left and right images because the current image processing technology is weak in the processing of "shape". Therefore, if the correspondence is determined by the tangents constituting the figure outline, the simplest linear figure is obtained, and the correspondence can be determined by a simple procedure.

Effect of Processing Two-Dimensional Correspondence of Tangent Lines in One Dimension To determine tangent correspondences in the left and right images, two-dimensional processing is required. However, when the polar transformation process is performed before the correspondence process and the image is mapped onto the dual plane, the parallel lines are transformed into a sequence of points on the ρ axis. The problem is reduced to a large amount, and the processing amount is greatly reduced, and the stability of the response is also improved.

Effect of Advanced Filtering After the polar conversion, an advanced one-dimensional filtering process, which has already been filed, can be performed. As a result, the configuration of the polar conversion → one-dimensional filter → one-dimensional correlation processing ensures a certainty. Correspondence can be determined, and various correlation parameters described later can be measured accurately. The types and functions of one-dimensional filtering are as follows. Edge determination by odd function filtering: Edges (brightness boundaries), which are the most important features of an image but difficult to extract with a conventional two-dimensional convolution filter, can be extracted, and edge correspondence can be determined. This is because an odd function filter has been made possible by performing polar conversion in advance. -Stable correspondence determination by multi-filtering: A large number of filters having different widths can be simultaneously applied to pole-converted data (multi-filter). This allows
"Blur feature" mixed in images, which is difficult with conventional methods
And "fine features" can be extracted at the same time, and stable correspondence determination can be made. High-speed processing by skeleton filtering: The most efficient skeleton filter can be applied to pole-converted data. Thereby, the above-mentioned odd function filter and multi-filter can be realized at high speed. For details of the odd function filter, the multi-filter, the skeleton filter, etc., refer to Japanese Patent Application No.
No. 7722 (Title of Invention: Image Processing Method, Filing Date: December 11, 1991).

(D) Extension of the Present Invention The principle and effects of correlation filtering have been described above with reference to binocular stereovision for ease of understanding. However, the essence is to measure and judge the degree of correlation between graphics between images, and not only for correspondence with both eyes, but also for use in tracking moving objects in association with temporally different images, It can be widely used for texture analysis to check the degree of the same pattern in one screen. Hereinafter, the extension of the correlation filtering of the present invention will be described in detail.

(A) Basic Flow FIG. 14 shows a basic flow of the correlation filtering, in which the input data A and B are subjected to polar conversion processing (steps 51a and 51a).
1b), performing a one-dimensional filtering process on the result of the polar transformation and mapping the result to a correlation dual plane (steps 52a and 52).
b) Correlation processing is performed between the data mapped on each dual plane (step 53). It is desirable that input data be divided into receptive fields, but it is not always necessary to divide into receptive fields. The one-dimensional filter processing may be performed as needed, and the type thereof may be a filter disclosed in Japanese Patent Application No. 3-327722. Further, all of the reciprocal transformation and the orthogonal complement space can be applied instead of the polar transformation. Further, the input data is not limited to the image from the camera. For example, data obtained by polar-converting a camera image may be input, and the input data is not limited to two.
There may be three or more.

In the following, a description will be given assuming that receptive field division and one-dimensional filter processing are performed. (b) Correlation action screen Action screen, that is, input data A,
B has the following cases, and each can measure a unique correlation parameter. (b-1) Correlation between different images Correlation between spatially different screens Input data A and B are image data of a spatially different screen, for example, a screen captured by another camera. This is a case where filtering is performed, and a feature correlation parameter between spatially different screens can be extracted. Binocular stereovision and trinocular stereovision are particularly useful examples.

FIG. 15 is a flowchart of correlation filtering between screens spatially different from each other. FIG. 15 (a) shows a screen input from two cameras, and FIG. It is. In the case of a screen input from two cameras (FIG. 15
(a)) When there are two cameras, the left camera CML and the right camera CMR
In the image data IM _L of one screen captured by dividing the IM _R to receptive field images (step 50a, 50b), each receptive field image I
Polar conversion processing is performed on M _L ′ and IM _R ′ (step 51 a,
51b) The one-dimensional filtering process is performed on the polar conversion result to map the result onto a dual plane (steps 52a and 52).
b) Correlation processing is performed between the data mapped on each dual plane (step 53). The above-described binocular stereoscopic vision corresponds to this.

In the case of a screen input from three cameras (FIG. 15
(b)) In binocular stereopsis, since there are two viewing angles, there is a case where the correspondence determination is wrong. The illusion of surprised houses is due to two eyes. Engineeringly, three eyes are possible, and the illusion decreases with each step when viewed from three angles by adding another camera. Placing three cameras at the vertices of the triangle can further reduce the illusion in all directions and further improve. If the camera is three, divided left camera CML, center camera CMC, the image data IM _L of one screen captured by the right camera CMR, IM _R, the IM _C to receptive field image (step 50 a to 50 c), each receiving Field image IM _L ', I
The polar conversion process is performed on M _R ′ and IM _C ′ (steps 51a to 51a).
51c), performing a one-dimensional filtering process on the polar conversion result and mapping the result to a dual plane (steps 52a to 52).
c) A correlation process is performed between the data mapped on each dual plane (step 53).

Correlation between temporally different screens Input data A and B in the basic flow are image data of a temporally different screen, for example, a screen captured while moving a camera, and the input data is subjected to correlation filtering. Is the case. With this correlation filtering,
For example, it is possible to measure the direction and speed of movement of features (corners, contours, etc.) in the receptive field, so that it is possible to move the object while capturing it at the center of the field of view. It can be expected to be applied to running vehicles. Even in the primary visual cortex of the cerebrum, there are cells that detect the direction of movement of the tangent line reflected in the receptive field, and a signal is sent to the eye movement control unit (upper hill or lateral knee) to view the object In addition to capturing at the center, signals are sent to the higher visual cortex in parallel, and the movement of the scene in the field of view is three-dimensionally measured.

FIG. 16 is a flow chart of correlation filtering between screens different in time. Image data IM _L of one screen captured by the left camera CM is not delayed, while the right camera CMR image data IM _R for one screen captured by is delayed by the delay unit (step 49), the image data which is not delayed I
'It is divided into each receptive field image (step 50a, 50a' M _L and the delayed image data IM _R), each receptive field image IM
_L and IM _R ′ are subjected to polar conversion processing (steps 51 a and 51).
a '), the result of the polar conversion is subjected to one-dimensional filtering, and mapped to a dual plane (steps 52a and 52a).
a '), a correlation process is performed between the data mapped on each dual plane (step 53). That is, the input image data is divided into those that are delayed in time and those that are not delayed, and these are inputted to the basic correlation filter in FIG. 14, and a temporal correlation parameter is obtained. Therefore, the simulation can be executed in the same manner as the above-described binocular stereoscopic simulation except for the delay unit.

Correlation between temporally and spatially different screens This is a case in which correlation filtering is performed on a screen captured while moving a plurality of cameras. In the case of two cameras, a binocular stereoscopic spatial correlation filter is used. And a time correlation filter. Figure 17 is a flow diagram of the correlation filtering between the time and space different screen, delays the image data IM _L of one screen captured by the left camera CML delay unit (step 49a), not delayed image The data IM _L and the delayed image data IM _L ′ are each divided into receptive field images (steps 50a and 50a).
a '), polar conversion processing is performed on each of the receptive field images IM _L , IM _L ' (steps 51a and 51a '), and the result of the polar conversion is subjected to one-dimensional filtering processing to be mapped onto a dual plane (step 52a, 52a). 52a '). In parallel with the above, image data I for one screen taken by the right camera CMR
M _R is delayed by the delay unit (step 49b), and the image data IM _{R which} is not delayed and the delayed image data IM _R 'are respectively divided into receptive field images (steps 50b and 50b'), and each receptive field image IM _R , The polar conversion process is performed on IM _R '(steps 51b and 51b'), the one-dimensional filtering process is performed on the result of the polar conversion, and the results are mapped onto dual planes (steps 52b and 52b '). Then, a correlation process is performed between the data mapped on each dual plane (step 5).
3). By tracking the target object while extracting parallax by binocular stereovision by the correlation filtering between screens that are temporally and spatially different, a composite function like a human becomes possible. The existence of such a filter that combines time and space is known in psychology as a "Spatiotempora filter", but this item was first formulated to contribute to engineering.

(B-2) Correlation between Same Images In the above, correlation filtering between different screens has been described, but correlation within the same screen is also effective. Correlation between Receptive Fields FIG. 18 is a flowchart of correlation filtering between receptive fields on the same screen. An image for one screen captured by the camera CM is stored in the input memory IMM, and cut out for each of the receptive fields A, B,...
_{M A, IM B, ·· IM} N and is (step 48), each receptive field image _{IM A, IM B, ·· IM} N pole conversion processing on (step 51A～51n), primary to each pole conversion result An original filtering process is performed to map to dual planes (steps 52a to 52n), and a correlation process is performed between data mapped to each dual plane (step 53). In the case of such a correlation between different receptive fields on the same screen,
Image features receptive field image IM _A receptive field image IM _B, ·· IM _N
Can be accurately compared with the above-mentioned feature, and a texture analysis such as a texture condition in the image can be performed. This processing is performed by changing the left and right eyes of the binocular stereopsis to each receptive field, and can be executed in the same manner as in the above-described simulation.

Correlation in Receptive Field The correlation between receptive fields has been described above, but correlation in the same receptive field on the same screen is also possible. FIG. 19 is a flowchart of correlation filtering in the same receptive field on the same screen, in the case of correlation filtering between ρ axes. The receptive field image is polar-transformed and stored in a ρ-θ dual plane (hyper column memory) HCM, and the cutout control unit controls the ρ axis A,
ρ axis B,..., are cut out for each ρ axis N (step 4
8 '), a polar conversion process is performed on the hypercolumn image of each of the ρ axes A to N (steps 51a' to 51n '), and a one-dimensional filtering process is performed on each of the polar conversion results to be mapped onto a dual plane (step 52a). 'To 52n'), a correlation process is performed between the data mapped on each dual plane (step 53). With this correlation filter, a more detailed texture analysis in the receptive field can be performed, and the binocular stereoscopic simulation can be similarly executed for the same reason as in the above case. Although the correlation between the ρ axes has been described, the correlation in the θ axis direction is also possible, and the correlation in the (ρ, θ) plane is also effective as a two-dimensional correlation.

(B-3) Correlation Between Different Color Images The correlation filtering of the intensity signal such as the brightness of the input image has been described above. If the correlation filtering of FIG. In addition, a more detailed filtering can be performed by detecting a subtle difference in color. The input color information can be color information that has been subjected to various processes according to the problem. The basic color information is as follows. Correlation with Three Primary Colors The correlation between the three primary colors is the most basic color correlation, and FIG. 20 is a flow chart of color correlation filtering between the three primary colors between different images. The color images for one screen taken by the left camera CML are respectively red, blue, and green filters (RFL, BFL, GF).
Red through L), separated blue, green image, after which each of red, blue, green receptive field image IM _R, IM _B, divided into IM _G (step 60 a to 60 c), each receptive field image IM _R , I
M _B, subjected to polar transformation processing IM _G (step 61a~61
c), performing a one-dimensional filtering process on the result of the polar conversion and mapping the result to a dual plane (steps 62a to 62).
c). Similarly, although not shown, color images for one screen captured by the right camera are passed through red, blue, and green filters, respectively.
The image is separated into blue and green images, then divided into red, blue, and green receptive field images, respectively, subjected to polar conversion processing on each receptive field image, and subjected to one-dimensional filtering processing on the polar conversion result to form a dual plane. Map to Then, a red correlation process, a blue correlation process, and a green correlation process are performed between the data mapped to the dual planes for red, blue, and green (steps 63a to 63).
c). By this color correlation filtering, correlation parameters of each of the three primary colors can be measured.

Correlation with Three Elements of Color (Lightness, Saturation, Hue) Next, the basic color correlation is the amount by which humans subjectively see physical three primary colors, that is, brightness (brightness), crispness ( This is a method of performing correlation filtering after conversion into saturation (color saturation) and color condition (hue). With this filter, it is possible to measure the correlation parameters of the colors arranged psychologically. FIG.
Is a flow of color correlation filtering for three elements between different images. One screen of color image captured by the left camera CML is separated into red, blue, and green images by red, blue, and green filters, respectively, and then the color conversion unit converts the color information of the three primary colors into lightness, saturation, and hue. three elements IM _V color, converts IM _S, the IM _H (step 59), receptive field image IM _V each lightness, saturation, and hue ', IM _S', divided into IM _H '(step 60
a'~60c '), each receptive field image _{IM V', IM S ',} IM
_H ′ is subjected to polar conversion processing (steps 61 a ′ to 61 a).
c ′), the result of the polar conversion is subjected to one-dimensional filtering processing and mapped to a dual plane (steps 62 a ′ to 62 a).
c '). Similarly, although not shown, color images for one screen captured by the right camera are respectively subjected to red, blue, and green filters by red, blue, and green filters.
The color conversion unit converts the color information of the three primary colors into three elements of lightness, saturation, and hue, and divides them into receptive field images of lightness, saturation, and hue. The polar image is subjected to a polar conversion process, the result of the polar conversion is subjected to a one-dimensional filtering process, and the result is mapped to a dual plane. Then, a brightness correlation process, a saturation correlation process, and a hue correlation process are performed between the data mapped to each dual plane with lightness, saturation, and hue (steps 63a 'to 63c').

Correlation with Color Difference Signal In order to accurately measure the correlation parameter at the boundary between colors, such as the transition between red and green, it is effective to perform a correlation process on the color difference signal. In particular, it is effective to use the three primary colors as a difference signal in a complementary color relationship. FIG. 22 is a flowchart of color correlation filtering using a color difference signal between different images. The color images for one screen captured by the left camera CML are red,
Separate into red, blue and green images with blue and green filters, and then
Red and green, green and blue, and blue and red difference signals are calculated (steps 58a to 58c), and the receptive field image I of each difference signal is calculated.
The image is divided into M _RG , IM _GB and IM _BR and each receptive field image I
Polar conversion processing is performed on M _RG , IM _GB , and IM _BR (steps 61a ″ to 61c ″), and the result of the polar conversion is subjected to one-dimensional filtering processing to be mapped onto a dual plane (steps 62a ″ to 62c ″). Similarly, although not shown, the color image for one screen captured by the right camera is separated into red, blue, and green images by red, blue, and green filters, respectively, and then, red, green,
The difference signals between green and blue and between blue and red are respectively operated, polar conversion processing is performed on the receptive field image of each difference signal, one-dimensional filtering processing is performed on the polar conversion result, and the result is mapped to a dual plane. Then, a red-green correlation process, a green-blue correlation process, and a blue-red correlation process are performed between the data mapped to the respective dual planes with the corresponding color difference signals (steps 63a "to 63c").
Note that instead of taking the difference, the difference after division or logarithmic conversion is also effective. Also, the difference or division may be performed after the polar conversion or the one-dimensional filter. In the above color correlation, the color correlation between different images has been explained.However, the color correlation method can be combined with the various correlation methods described in (b-1) and (b-2) above, and binocular stereo vision and movement Correlation filtering such as object gaze and texture analysis can be made more reliable.

(C) Working direction of correlation In the example of binocular stereovision (see FIG. 7), the orientation of the dual plane
By combining data in the ρ-axis direction with the same θ,
However, it is not necessary to limit to the ρ axis.
Can be generalized. (c-1) Correlation in the ρ-axis direction Correlation in the ρ-axis with the same θ This is the case of binocular stereopsis etc.
The meaning of “same ρ-axis correlation” means that both screens
To accurately measure the amount of deviation between parallel lines
You. FIG. 23 shows the flow of the ρ-axis correlation filtering with the same θ.
The input data A and B are divided into receptive field images.
(Steps 50a and 50b), polar conversion processing for each receptive field image
(Steps 51a and 51b), and
Perform dimensional filtering and map to dual plane
(Steps 52a and 52b), from each dual plane
θ₀, Θ₁Select data in the ρ-axis direction for each
Step 54a ₀, 54a₁54b₀, 54b₁,
・), And performs correlation processing between data having the same θ value (step
Step 53₀, 53₁, ...)). FIG. 23 shows a parallel view for each θ value.
In this case, one correlation calculation unit is provided.
The correlation calculation of each θ value can be performed sequentially by
Is the same as the flowchart in FIG.

Correlation of ρ axis with different θ This correlation is filtering to measure the correlation parameter between tangents with different orientations. This filtering can extract not only the correlation of parallel lines but also the extraction of tangents that move while changing the azimuth, and can be expected to be applied to vehicles running on rough roads. FIG. 24 is a flowchart of such ρ-axis correlation filtering with different θ. The input data A and B are divided into receptive field images (steps 50a and 50b), and a polar conversion process is performed on each receptive field image (steps 51a and 51b).
The polar transformation result is subjected to one-dimensional filtering processing to be mapped to dual planes (steps 52a and 52b), and θ _i , θ _{i + 1} ,..., Θ _j , θ _{j + 1} ,. Is selected in the ρ-axis direction (steps 54a ₀ , 54
_{a 1, ··; 54b 0,} 54b 1, ··), different θ values subjected to correlation processing between data (step 53 ₀ ', 5
3 ₁ ', ...).

(C-1) Correlation in the θ-axis direction The meaning of the correlation parameter between θs having the same ρ is explained with reference to FIG. 25. In the θ correlation of ρ = 0 (FIG. 25 (a)), The rotation amount of the tangent line SL passing through C can be extracted. Also, ρ ≠
With the θ-axis correlation of 0, the rotational movement amount of the tangent line SL tangent to the circle CIR having the radius ρ sharing the receptive field center C can be detected. FIG. 26 is a flow chart of the θ-axis correlation filtering with the same ρ. Each input data A and B is divided into receptive field images (steps 50 a and 50 b), and the polar conversion process is performed on each receptive field image (step 51 a). , 51b), applying a one-dimensional filtering process to the result of the polar transformation and mapping the result to a dual plane (steps 52a, 52b). From each dual plane, ρ ₀ , ρ ₁ ,
..Select data in the ρ-axis direction for each (step 55)
a ₀ , 55a ₁ ,...; 55b ₀ , 55b ₁ ...), and performs correlation processing between data having the same ρ value (steps 53 ₀ , 5).
3 ₁ ...). Although FIG. 26 shows the case where the correlation operation is performed in parallel for each ρ value, one correlation operation unit can be provided to sequentially perform the correlation operation for each ρ value.

(C-3) Correlation in the (ρ, θ) plane In the above description, the one-dimensional correlation in the ρ-axis or θ-axis direction has been described, but two-dimensional correlation in the (ρ, θ) plane is also possible. . With this correlation, it is possible to accurately perform the correlation of the tangents that move while changing the direction. FIG. 27 is a flowchart of the correlation filtering in the (ρ, θ) plane. Each of the input data A and B is divided into receptive field images (step 50a,
50b), performing polar conversion processing on each receptive field image (step 5)
1a, 51b), one-dimensional filtering is performed on the result of the polar transformation, and the result is mapped to a dual plane (steps 52a, 51b).
52b), a two-dimensional correlation process is performed on each dual plane (step 53). Note that the two-dimensional correlation amount C (ρ, θ,
σ ₁ , σ ₂ ) represents the data on each dual plane as a (ρ,
θ), b (ρ, θ), the shift amounts of the dual plane data in the ρ-axis direction and the θ-axis direction are σ ₁ , σ ₂ , and the following equation C (ρ, θ, σ ₁ , σ ₂ ) = a (ρ, θ) · b (ρ + σ ₁ , θ + σ ₂ ) (2)

(D) Correlation Parameter In the example of binocular stereopsis, for simplicity of description, a method of obtaining a corresponding tangent using the correlation amount (correlation parameter) in equation (1) has been described. Various correlation parameters can be defined, and characteristic correlation processing can be performed for each. (d-1) Basic Correlation Parameter The basic correlation parameter is generally C
(Ρ, θ, σ) are represented in three dimensions, and the three-dimensional space of ρ, θ, σ is as shown in FIG. In the figure, τ is an inter-data shift amount (moving speed) introduced in correlation between temporally different screens, which will be described later. Correlation parameter C
A point (ρ _P ,
θ _P , σ _P ), the correspondence between the binocular stereopsis and the moving tangent are determined, and the position of the tangent from each coordinate value (element parameter value) that gives the maximum point, Direction, parallax or speed can be measured quantitatively.
In the two-dimensional correlation of the (ρ, θ) plane, the correlation parameter is a parameter C (ρ, θ, σ ₁ , σ ₂ ) of a four-dimensional space.
Becomes

Spatial Correlation Parameter The case where the action screen of the correlation is a spatially different screen or receptive field (cases (b-1) and (b-2) described above), and the following maximum points of the correlation parameter Thus, the correlation amount indicating the degree of correspondence such as the parallax of the corresponding tangent can be measured. (1) Correlation parameter C (ρ, θ, σ) in the ρ direction: a parameter indicating the degree of spatial correlation of a group of parallel lines. The aforementioned correlation parameter for binocular stereopsis is an example of this, and C (ρ,
When ρ _P , θ _P , and σ _{P at} which (θ, σ) are maximal are determined, a corresponding tangent is determined from the parallel line group, and the position, orientation, and parallax of the tangent can be quantitatively measured. Incidentally, the correlation parameter is obtained by converting the data on each dual plane into a (ρ, θ), b
(Ρ, θ), assuming that the shift amount in the ρ-axis direction is σ, the following equation can be obtained: C (ρ, θ, σ) = a (ρ, θ) · b (ρ + σ, θ).

(2) The correlation parameter Cθ (ρ,
θ, σ): parameters indicating the degree of spatial correlation of a tangent group tangent to a circle having a radius ρ sharing the center of the receptive field. ρ = 0
In the θ correlation processing of the above, ρ at which Cθ (ρ, θ, σ) becomes a maximum
_P, theta _P, from sigma _P, corresponding tangents are determined from the radiation group, the position of the tangent direction can quantitatively measure the orientation change amount. The correlation parameter is given by the following expression C θ (ρ, θ, σ) = a (ρ, θ) · b (ρ, θ + σ) (3), where σ is the shift amount in the θ-axis direction.

(3) Correlation parameter C (ρ, θ, σ ₁ , σ ₂ ) of (ρ, θ) plane: a parameter indicating the spatial correlation degree of a tangent group moving while changing its direction, and is a dual plane Can be evaluated in more detail than the one-dimensional correlation parameter described above. The correlation parameter is expressed by the equation (2), that is, the following equation: C (ρ, θ, σ ₁ , σ ₂ ) = a (ρ, θ) · b (ρ + σ ₁ ,
θ + σ ₂ ).

Temporal Correlation Parameter When the operation screen of the correlation is a temporally different screen or a receptive field (such as (b-1)), the moving tangent of the moving tangent is determined from the following maximum points of the correlation parameter. It can measure the moving direction and moving speed. (1) Correlation parameter C (ρ, θ, τ) in the ρ-axis direction: a parameter indicating a temporal correlation degree of a group of parallel lines. An example to be described later, that is, a correlation parameter when tracking a moving tangent is an example of this, and from ρ _P , θ _P , and τ _P at which C (ρ, θ, τ) is maximum, position,
Direction and speed can be measured quantitatively. The correlation parameter is given by the following equation, where τ is the moving speed in the ρ-axis direction: C (ρ, θ, τ) = a (ρ, θ) · b (ρ + τ, θ) (4) Incidentally, a (ρ, θ) = a t (ρ, θ), b
(Ρ + τ, θ) = a t + DELT A (ρ + τ, θ) the correlation parameter a is expressed as the following equation C (ρ, θ, τ) = a t (ρ, θ) · a t + DELTA ( ρ + τ, θ) (4) '. That is, in step 53 of FIG. 16, the correlation processing of equation (4) 'is executed. Here, DELTA represents the delay time, and the velocity V of the tangential line that moves in parallel can be obtained as follows: V = (τ _P · Δρ) / DELTA where Δρ is the resolution in the ρ direction. This is the “Direction” of the living cerebral visual cortex.
It models the function of "al Selectivity simple cells". The cells exhibit the unique property of responding to stimuli moving in one direction but not responding to stimuli in the opposite direction. This can be realized by limiting τ in the above equation to be positive or negative, and reflects the restriction that the nerves of the living body cannot send both positive and negative information at the same time. This cell is, to be precise, a “non-linear response type” Directional Selectivity simple cell. Incidentally, the temporal correlation parameter can be calculated by the symmetric correlation operation of the following equation in addition to the asymmetric correlation operation of the equation (4). C (ρ, θ, τ) = a (ρ−τ, θ) · b (ρ + τ, θ) (4) ″ However, hereinafter, the temporal correlation parameter is calculated by the asymmetric correlation calculation of the equation (4). It will be described as what is done.

The correlation parameter Cθ (ρ, θ,
τ): a parameter indicating a temporal correlation degree of a tangent group tangent to a circle having a radius ρ sharing the center of the receptive field. ρ = 0 θ
In the correlation processing, ρ _P , at which Cθ (ρ, θ, τ) is maximized,
From θ _P and τ _P , the position, azimuth, and azimuth rotation speed of a tangent passing through the center of the receptive field can be quantitatively measured. Note that the correlation parameter is given by the following expression Cθ (ρ, θ, σ) = a (ρ, θ) · b (ρ, θ + τ) (5), where τ is the moving speed in the θ-axis direction. Incidentally, a (ρ, θ) = a t (ρ, θ), b
(Ρ, θ + τ) = a t + DELT A (ρ, θ + τ) correlation parameters and expressed by the following equation Cθ (ρ, θ, τ) = a t (ρ, θ) · a t + DELTA ( ρ, θ + τ) (5) '. That is, in step 53 of FIG. 16, the correlation processing of equation (5) 'is executed. Here, DELTA represents a delay time, and the rotation speed ω is obtained as ω = (τ _P · Δρ) / DELTA. There is a cell in a living body that detects rotation, which is modeled.

(3) Correlation parameter C (ρ, θ, τ ₁ , τ ₂ ) of (ρ, θ) plane: A parameter indicating a temporal correlation degree of a tangent group moving while changing its direction. C
Ρ _P , θ _P , τ _1P , τ at which (ρ, θ, τ ₁ , τ ₂ ) is maximized
_{From 2P} , the position, azimuth, movement speed, and rotation speed of the tangent passing through the center of the receptive field can be measured in more detail than the one-dimensional correlation parameter. The correlation parameter is given by the following equation: C (ρ, θ, τ ₁ , τ ₂ ) = a (ρ, θ) · b (ρ + τ ₁ , θ + τ ₂ ) (6) That is, in step 53 of FIG. 16, the correlation processing of equation (6) is executed. In addition, a (ρ, θ) = a t
(Ρ, θ), b (ρ + τ ₁ , θ + τ ₂ ) = at _{+ DEL TA} (ρ
+ Tau _1, the correlation parameter a is expressed as θ + τ ₂₎ by the following equation _{C (ρ, θ, τ 1} , τ 2) = a t (ρ, θ) · a t + DELTA (ρ + τ 1, θ + τ 2) ··· ( 6) ′. That is, in step 53 of FIG. 16, the correlation processing of equation (6) 'is executed. Here, DELTA represents a delay time, and the translation speed V and the rotational speed ω are obtained as follows: V = (τ _1P · Δρ) / DELTA ω = (τ _2P · Δρ) / DELTA

(D-2) Projection of basic correlation parameters In the above basic correlation parameters, the position of the tangent (ρ _P ),
Although all the correlation parameters of the azimuth (θ _P ) and the parallax / velocity (σ _P / τ _P ) have been determined, there is a problem that a large correlation parameter storage unit of three dimensions or more is required. Hereinafter, the definition and characteristics of the correlation parameter that can reduce the number of parameters for the correlation determination as necessary and reduce the capacity of the correlation parameter storage unit will be described below. Spatial correlation parameter C (ρ, θ, σ) and temporal correlation parameter C
(Ρ, θ, τ) will be described. Correlation parameters C (ρ, θ, σ ₁ , σ ₂ ), C (ρ,
θ, τ ₁ , τ ₂ ) can be similarly defined, and more detailed correlation measurement can be performed than in the case of the one-dimensional correlation parameter.

When the position ρ and direction θ of the corresponding tangent are important in the σ or τ direction and the parallax σ (movement speed τ) does not need to be measured directly, the basic correlation parameter C (ρ, θ, σ ) Or C (ρ, θ, τ) in the σ or τ direction, C _PRJ− σ (ρ, θ) or C
_PRJ- τ (ρ, θ) is effective, and the correlation parameter storage unit is reduced to two dimensions of ρ−θ, the same as the input dual plane. C _PRJ- σ (ρ, θ) = ΣC (ρ, θ, σ) (where σ = 0,1,2,... Σmax) (7) C _PRJ- τ (ρ, θ) = ΣC (ρ , Θ, τ) (where τ = 0, 1, 2, ... τmax) (8)

FIG. 29 shows a basic correlation parameter C (ρ, θ,
It is a flowchart of the filtering process which projects (sigma).
The input data A and B are divided into receptive field images (step 50).
a, 50b), perform polar conversion processing on each receptive field image (steps 51a, 51b), apply one-dimensional filtering processing to the polar conversion result, and map the result to a dual plane (step 52).
a, 52b), the data of each dual plane is subjected to the correlation processing of equation (1) ′ (step 53), and the correlation parameter C (ρ, θ, σ) obtained by the correlation processing is integrated according to equation (7). (Step 71) The integration result is stored in the ρ-θ correlation parameter memory (Step 72).

FIG. 30 is a flowchart of the process of calculating the correlation parameter and projecting in the σ direction. 0 →
θ, 0 → ρ, 0 → σ (Steps 201 to 20)
3). Then, according to the equation (1) ', the correlation parameter C
(Ρ, θ, σ) is calculated (step 204), and the correlation parameters are integrated according to equation (7) (step 20).
5). Then, while σ is incremented, σ> σ
max (steps 206 and 207), and σ
If .ltoreq..sigma.max, the process returns to step 204 and the subsequent processes are repeated. When σ> σmax, the integrated value is ρ,
It is stored at the matrix intersection of the correlation parameter storage unit indicated by θ (step 208). Thereafter, ρ is incremented and it is determined whether ρ> ρmax (step 20).
9, 210), if ρ ≦ ρmax, the process returns to step 203 and the subsequent processes are repeated. If ρ> ρmax, θ is incremented and it is determined whether θ> θmax (steps 211 and 212). If θ ≦ θmax, the process returns to step 202 and the subsequent processing is repeated, and
Then, the correlation calculation / projection processing ends.

When the azimuth θ of the corresponding tangent and the parallax σ (or the moving speed τ) are important and the position ρ does not need to be measured directly, the basic correlation parameter C (ρ, θ, σ ) _Is added in the ρ direction, and the correlation parameter C _PRJ- ρ (θ, σ) or C _PRJ- ρ (θ, τ) is effective, and the correlation parameter storage unit is the same as the input dual plane. Reduced to dimensions. C _PRJ- ρ (θ, σ) = ΣC (ρ, θ, σ) (where ρ = 0,1,2,... Ρmax) (9) C _PRJ- ρ (θ, τ) = ΣC (ρ , Θ, τ) (where ρ = 0, 1, 2,... Ρmax) (10) This correlation parameter is effective when detecting a contour moving in the visual field, and The tangent direction θ and the moving speed τ can be measured independently of the line position ρ. There are cells that perform the same function in the hypercolumn of the cerebrum, fire strongly when a straight line moves in the field of view, and emit no signal when stopped, and are known as "DirectionalSelectivity complex cells". This is based on “optical flow measurement” and “binocular stereoscopic vision from random dots” described later.
Play an important role.

When the position ρ of the corresponding tangent and the parallax σ (or the moving speed τ) are important and the azimuth θ does not need to be measured directly, the basic correlation parameter C (ρ, θ, σ ) _Are added in the θ direction, and the correlation parameter C _PRJ- θ (ρ, σ) or C _PRJ- θ (ρ, τ) given by the following equation is effective, and the correlation parameter storage unit stores the same two parameters as the input dual plane. Reduced to dimensions. C _PRJ- θ (ρ, σ) = ΣC (ρ, θ, σ) (where θ = 0,1,2,... _Θmax ) (11) C _PRJ- θ (ρ, τ) = ΣC (ρ , Θ, τ) (however, θ = 0, 1, 2, ... θmax) (12) This correlation parameter is effective when tracking a moving contour while capturing near the receptive field. . The characteristic is that the distance ρ and the moving speed τ of the contour from the center of the receptive field can be measured irrespective of the direction of the line. Similar cells are found in retinas such as frogs and are used to track prey such as flies.

Projection in the ρσ or ρτ direction The tangent θ of the corresponding tangent is important, and the position ρ and the parallax σ (movement
If it is not necessary to measure the velocity τ) directly,
Parameter C (ρ, θ, σ) in the ρσ direction (or ρτ direction)
The correlation parameter C given by_PRJ-ρσ
(Θ) or C _PRJ-ρτ (θ) is valid and the correlation parameter
The meter storage is reduced to one dimension. C_PRJ-ρσ (θ) = ΣΣC (ρ, θ, σ) (However, ρ = 0,1,2, ... ρmax, σ = 0,1,2, ... σmax) (13) C_PRJ-ρτ (θ) = ΣΣC (ρ, θ, τ) (where ρ = 0,1,2, ... ρmax, σ = 0,1,2, ... σmax) (14) This correlation parameter is A contour line in a certain direction in the receptive field
It is effective for detecting that
The orientation θ of the contour line regardless of the position of the contour line or the moving speed
It has the characteristic that it can be measured at any time.

When the parallax ρ (moving speed τ) of the corresponding contour is important and it is not necessary to directly measure the position ρ and the azimuth θ, the basic correlation parameter C (ρ, θ, σ) is calculated. The correlation parameter C _PRJ- ρθ (σ) or C given by the following equation added in the ρθ direction:
_PRJ- ρθ (τ) is effective, and the correlation parameter storage is reduced to one dimension. C _PRJ− ρθ (σ) = ΣΣC (ρ, θ, σ) (However, ρ = 0,1,2,... Ρmax, θ = 0,1,2,... Θmax) (15) C _{PRJ - ρθ (τ) = ΣΣC (} ρ, θ, τ) ( where, ρ = 0,1,2, ··· ρmax, θ = 0,1,2, ··· θmax) (16) this correlation parameter This is effective when it is desired to detect the contour moving in the receptive field together with its speed, and in the case of equation (16), the moving contour speed τ can be measured regardless of the position and orientation of the contour. It has the characteristic of.

It is only important whether or not there is a corresponding contour line in the receptive field projected on θρσ or θρτ.
When it is not necessary to measure the position ρ, the azimuth θ, and the parallax σ (moving speed τ), it is given by the following equation in which the basic correlation parameter C (ρ, θ, σ) is added in the ρθσ direction (or ρθτ direction). The correlation parameter C _PRJ- θρσ or C _PRJ- θρτ is effective, and the correlation parameter storage unit stores 0 for only one cell.
Reduced to dimensions. C _PRJ− θρσ = ΣΣΣC (ρ, θ, σ) (where ρ = 0, 1, 2,... Ρmax, θ = 0, 1, 2,..., Θmax, σ = 0, 1, 2,.・ ・ Σmax) (17) C _PRJ- θρτ = ΣΣΣC (ρ, θ, τ) (However, ρ = 0,1,2 ... ρmax, θ = 0,1,2, ... θmax, σ = 0,1,2,. And has the characteristic that measurement can be performed irrespective of the position, orientation, or moving speed of the contour line. Cells similar to this are commonly found in the retina of lower animals such as frogs, and in order to quickly detect whether or not a prey such as a fly has entered the receptive field, the cells are extremely The wisdom that constitutes a small number of processing systems is presumed. Engineeringly, use I
In a self-propelled vehicle for planetary exploration where the number of C is very limited,
This is effective when grasping the approximate movement status in receptive field units.

Projection in the oblique direction of the (ρ, τ) or (ρ, σ) plane In the above, in order to reduce the capacity of the correlation parameter storage unit,
Basic correlation parameters C (ρ, θ, σ), C (ρ, θ,
τ) and the like, that is, in the direction of the contour position ρ, azimuth θ, parallax σ, and moving speed τ. Here, a method of reducing the capacity of the correlation parameter storage unit while maintaining the properties of the above-described element parameters will be described. Basic correlation parameters C (ρ, θ, σ), C
(Ρ, θ, τ) is replaced by (ρ, τ) or (ρ, τ) in FIG.
σ) When projected in any diagonal direction of the plane, ρ, τ,
The correlation parameter storage unit can be reduced two-dimensionally while retaining the property of σ. The oblique projection, (ρ, σ) is described in example projected in 45 ⁰ direction. When an axis perpendicular to the projection direction and xi], correlation parameters projected on 45 ⁰ direction C _PRJ-45 sigma
(Θ, ξ) is given by the following equation, and the correlation parameter C _PRJ-45
When θ _P and ξ _{P at} which σ (θ, ξ) are maximal are obtained, the azimuth θ _P of the corresponding contour line and the “parameter ξ _P combining the position ρ and the parallax σ” are measured. C _PRJ-45 σ (θ, ξ) = ΣC (ρ + i, θ, σ + i) (where i = 1, 2,...) (19)

(D-3) Convolution filter C ₁ (ρ, θ) of natural filter Output data of the dual plane are a (ρ, θ), b (ρ,
The following projection correlation parameter C ₁ (ρ, θ) as θ) constitutes an interesting filter. C ₁ (ρ, θ) = Σa (ρ + i, θ) · b (i, θ) (where i = 1, 2,...) ), The one-dimensional filter convolution operation output C ₂ (ρ,
θ) is given by the following equation, where g (ρ) is a one-dimensional filter function. C ₂ (ρ, θ) = Σa (ρ + i, θ) · g (i) (where i = 1, 2,...) (21) Comparing both equations, the correlation parameter C ₁ (ρ, θ)
Is obtained by changing the filter function g (ρ) of the convolution output C ₂ (ρ, θ) into correlation data b (i, θ). From this, the correlation parameter C ₁ (ρ, θ) is calculated using natural data b instead of the artificial filter function g (ρ).
It can be considered as a “natural filtering” result obtained by performing a convolution operation with (i, θ).

Considering both filtering functions,
The "artificial filtering" of the equation (21) in which the filter function g (ρ) is known can be employed when an action such as differentiation can be defined in advance. When tracking the same feature as the previous image, a convolution operation with natural data is required, and such feature extraction and feature tracking are impossible. On the other hand, the “natural filtering” of the equation (20) can perform an accurate filter even in the above case where the filter pattern cannot be defined in advance.

FIG. 31 is a diagram showing the configuration of a natural filter according to the present invention. Reference numeral 81 denotes a receptive field division unit for cutting out and outputting input data (image data for one screen) for each receptive field. The pole conversion unit 83 that performs the conversion process, and 83 is a pole conversion result a
A ρ-θ dual plane (hypercolumn memory) 84 for storing (ρ, θ) is a natural filter according to the present invention, and includes a receptive field division unit 84a that cuts out and outputs the natural filter data for each receptive field, A polar conversion unit 84b that performs a polar conversion process on the receptive field image, a dual plane 84c of ρ-θ that stores the polar conversion result b (ρ, θ), a multiplication unit 84d that performs the operation of Expression (20), and an integration unit 84e.

The multiplying unit 84d includes dual planes 83 and 84c.
A (ρ + i, θ), b (i, θ) output from
The following multiplication a (ρ + i, θ) · b (i, θ) is performed on (ρ, θ, i is 0) and output to the integration unit 84e, and i is incremented and the next Data a (ρ + i, θ), b (i,
The same multiplication is performed on θ) until i> imax (the filter width is appropriately set). Integrator 84e
Multiplies the multiplication result, and if i> imax, the integration result is ρ
-Store in a θ plane memory (not shown). Thereafter, if the product-sum operation is performed for all (ρ, θ), the natural filtering process ends.

Application to Binocular Stereopsis As described above, detailed contours can be corresponded in three dimensions by the basic correlation parameter C (ρ, θ, σ). Plane (ρ, θ)
Enables the contour to be handled. Specifically, the aforementioned C
_By obtaining the maximum of ₁ (ρ, θ), the position ρ and the direction θ of the corresponding contour line can be determined independently of the parallax σ by both eyes. Cells having a similar function exist in the monkey cerebral area 18, and the corresponding contour lines are detected by both eyes.

Although the present invention has been applied to the tracking of a moving contour line, a natural filter between receptive fields spatially different from each other has been described, but a temporal natural filter is also effective. Temporal correlation parameter C _time (ρ, θ) is the input data temporally different _{a t (ρ, θ),} a t + DELTA (ρ, θ) as similarly following equation C _time (ρ, θ _{) = Σa t (ρ + i} , θ) · a t + DELTA (i, θ) ( however, i = 0,1 ···) is given by (22). As described above, in a three-dimensional space, the position, azimuth, and moving speed of a moving contour can be measured by the simultaneous basic correlation correlation parameter C (ρ, θ, τ). However, this natural filter has the characteristic that the position ρ and azimuth θ of the moving contour can be determined on the two-dimensional plane independently of the moving speed τ by obtaining the maximum of C _time (ρ, θ). Similar cells exist in the visual cortex of the cerebrum.

(D-4) Basic correlation of difference type In the above, the basic correlation parameters are C (ρ, θ, σ) = a (ρ, θ) · b (ρ + σ, θ) C (ρ, θ, τ) _{= a t (ρ, θ)} · a t + DELTA (ρ +
.tau., .theta.), but the same effect can be obtained by adding this. In this case, linearity is guaranteed between the signals, and is known as "linear simple cell" in living organisms. Furthermore, by subtracting the multiplication in the above expression, it becomes possible to perform binocular stereoscopic vision and tracking of an object having an indistinct outline using clues such as gradual changes in brightness and hue.

Spatial correlation In the case of the correlation in the ρ-axis direction, the basic correlation parameter is given by C ′ (ρ, θ, σ) = a (ρ, θ) −b (ρ + σ, θ) (23) . In this correlation, C ′ (ρ, θ, σ) becomes zero at the same position of the input data. This property enables binocular stereoscopic viewing of an object whose contour is not clear. For example,
Looking at the large round pillar, there is no clear outline except at both ends, but its brightness changes gradually in relation to the incident angle. C '(ρ, θ, σ) where the brightness is equal between the left and right eyes
When it is found from the property that becomes zero, both eyes can be handled, and the surface state of the cylinder can be measured stereoscopically.

FIG. 32 is a flow chart of the difference type correlation processing. The overall basic flow is the same as that of FIG. 14, and the difference is that the calculation of the correlation parameter is performed by equation (23). 0 → θ, 0 → ρ, 0 → σ for correlation calculation
(Steps 301 to 303). Next, the correlation parameter is calculated by equation (23) and stored in the correlation parameter storage unit (steps 304 and 305). Then, σ is incremented, and it is determined whether σ> σmax (steps 306 and 307). If σ ≦ σmax, the process returns to step 304 to repeat the subsequent processing. And σ
> Σmax, ρ is incremented and ρ>
ρmax is determined (steps 308 and 309),
If ρ ≦ ρmax, the process returns to step 303 and the subsequent processes are repeated. If ρ> ρmax, θ is incremented and it is determined whether θ> θmax (step 31).
0, 311), if θ ≦ θmax, the process returns to step 302 and the subsequent processes are repeated. If θ> θmax, the correlation calculation process ends.

Temporal correlation In the case of the ρ direction, the basic correlation parameter is given by the following equation: C ′ (ρ, θ, τ) = a (ρ, θ) −b (ρ + τ, θ) (23) . Incidentally, a (ρ, θ) = a t (ρ, θ), b
(Ρ + τ, θ) = a t + DELT A (ρ + τ, θ) the correlation parameter a is expressed as the following equation C '(ρ, θ, τ ) = a t (ρ, θ) -a t + DELTA (Ρ + τ, θ) (23) ′. Also in this correlation, C ′ (ρ, θ, τ) becomes zero at a position where the input data is equal. This property enables tracking of an object whose contour is not clear. In the same manner as in the case of, the place where the brightness becomes the same is represented by
If the detection is performed from the property that τ) becomes zero, stable tracking can be performed even in a portion where a clear contour line cannot be detected, such as a cylindrical surface or a part of a human face. This cell is called "Directio
nal Selectivity "Linear response type cell" belonging to "simple cell".

(E) Specific Methods and Embodiments (a) Binocular Stereo Vision As a specific example of a spatially different operation screen, a binocular stereo vision system and a simulation example will be described. A tangent has the characteristics of a line, a gap, and an edge.
The “edge” is a boundary line between a bright portion and a dark portion, and the “gap” is a dark thin band opposite to the line. Although shown for explanation of the principle, the binocular stereoscopic view of other features including the line will be systematically described here.
7722), line extraction filter, edge extraction filter,
Although the gap extracting filters have been disclosed, binocular stereoscopic vision of lines and gaps and binocular stereoscopic vision of edges can be realized by using these filters.

(A-1) Configuration of Various Filters Line Extraction Filter FIG. 33 is a configuration diagram of a "line" extraction filter, and (a) shows "receptive field method (receptive field division + polar transformation) + one-dimensional gas filter". (B) is a line extraction filter having a “two-dimensional gas filter + receptive field method” configuration. The line extraction filter of (a) divides the input image into receptive fields in a receptive field dividing unit,
The polar conversion unit performs a polar conversion for each receptive field image, and a one-dimensional gas filter performs a one-dimensional secondary differential filtering process on the polar conversion output to extract a line segment. The line extraction filter (b) is a two-dimensional gas filter that performs a two-dimensional second-order differentiation process on the input image, then divides the image for each receptive field, performs polar transformation on each receptive field image, and extracts lines. It is.

Edge Extraction Filter FIG. 34 is a block diagram of various "edge" extraction filters.
Is the configuration of a basic filter that extracts the "edge" by applying one-dimensional gr filter processing to the output of the receptive field method (receptive field division + polar transformation), and (b) shows a further one-dimensional filter after the basic filter of (a) (c) is a configuration of an edge extraction filter in which a two-dimensional gas filter is further connected in front of the basic filter of (a). The edge extraction filter of (a) divides the input image for each receptive field at the receptive field division unit, performs polar transformation on each receptive field image at the polar transformation unit, and performs a first differentiation process on the polar transformation output with the one-dimensional gr filter. To extract the “edge”. The edge extraction filter of (b) divides the input image for each receptive field in the receptive field dividing unit, performs polar transformation on each receptive field image in the polar transforming unit, and performs one-dimensional linear transformation to a polar transformed output with a one-dimensional gr filter. Differential processing is performed, and then one-dimensional second-order differential processing is performed using a two-dimensional gas filter to extract “edges”. (c)
The edge extraction filter performs a two-dimensional second differentiation process on the input image with a two-dimensional gas filter, then divides the input image into receptive fields by a receptive field dividing unit, and polarizes each receptive field image by a polar transform unit. After performing the transformation, the polar transformation output is subjected to a one-dimensional first-order differentiation process by a one-dimensional gr filter to extract an “edge”.

Gap Extraction Filter The gap extraction filter is obtained by inverting the sign of the line extraction filter. Therefore, if a sign inversion unit is connected to the subsequent stage of each line extraction filter shown in FIG. 33, a gap extraction filter is obtained. FIG. 35 is a configuration diagram of the gap extraction filter, in which a sign inversion unit is provided.

(A-2) Binocular stereoscopic view of lines and gaps Binocular stereoscopic view of lines and gaps with a one-dimensional filter The “polar conversion + one-dimensional filter processing” in the basic flow of FIG. ) Or “receptive field division + polar transformation + one-dimensional Gaussian filter (+ sign inversion) processing” shown in FIG. 35 (a), and by inputting the left and right eye images, It becomes possible to see. The simulation results have already been described with reference to FIGS. FIG.
, The sharp peak of C (ρ, θ, σ) indicates the corresponding point of the vertical line, and the parallax is correctly measured as 5 pixels from the value of the σ axis.

Binocular stereoscopic viewing of lines and gaps using a two-dimensional filter (biological binocular stereoscopic viewing) “Polar transformation + one-dimensional filtering” in the basic flow of FIG. 14 is performed as shown in FIG. 33 (b) or FIG. In addition to “two-dimensional Gaussian filter + receptive field division + polar conversion (+ sign inversion) processing” shown in b), binocular stereoscopic view of lines and gaps becomes possible by inputting left and right eye images. . This method is
The input images of the left and right eyes are subjected to a two-dimensional second-order differentiation process (two-dimensional convolution filter process) using a two-dimensional gas filter, and thereafter, are divided into receptive fields, and polar transforms are performed on each receptive field image. . The two-dimensional gas filter processing is equivalent to “one-dimensional filter processing after polar transformation”, and the simulation results are omitted, but are the same as FIGS. 11 to 13. This method is the same as human stereoscopic vision, and has the advantage that the result of contour enhancement using a two-dimensional gas filter can be commonly used for shape recognition and the like.

(A-3) Binocular Stereoscopic View of Edge Binocular Stereoscopic View of Edge with One-Dimensional Filter “Polar transformation + one-dimensional filter processing” in the basic flow of FIG. Receptive field division + polar transformation + one-dimensional gradient filter + one-dimensional Gaussian filter processing "and inputting images of the left and right eyes enables binocular stereoscopic viewing of edges. The simulation results are shown in FIGS. FIG. 36 (a), the circular part C _L in (b), C _R is receptive field, the image of the circular right and left eyes are receptive field images caught on the right and left eyes, the scene in the square S in a plant
The figure of the edge of the part surrounded by Q is shown. In FIG. 37, all data are indicated by the contour lines of the signal intensities, and the polar transformation result and the intensity distribution of the basic correlation parameter C (ρ, θ, σ) at θ = 168 ⁰ are indicated by contour lines. ing. The sharp peak of C (ρ, θ, σ) indicates the corresponding point of the edge, and the parallax is correctly measured as one pixel from the value of the σ axis.

A binocular stereoscopic view of an edge in which a two-dimensional filter is mixed (biological binocular stereoscopic view) “Polar transformation + one-dimensional filter processing” in the basic flow of FIG. In addition to “Gaussian filter + receptive field division + polar transformation + one-dimensional gradient filter processing”, by inputting images of the left and right eyes, binocular stereoscopic viewing of the edge becomes possible. In this method, the input images of the left and right eyes are subjected to a two-dimensional second-order differentiation process using a two-dimensional gas filter, and thereafter divided into receptive fields, and polar images are applied to each receptive field image. The two-dimensional gas filter processing is equivalent to “one-dimensional filter processing after polar conversion”, and the simulation result is omitted, but is the same as FIG. This method is the same as human stereoscopic vision, and has the advantage that the result of contour enhancement using a two-dimensional gas filter can be commonly used for shape recognition and the like. (a-4) Binocular stereoscopic view of polygonal figures, curved figures, random dots, and textures: Since this binocular stereoscopic view is related to the description in the measurement of the moving direction and speed described later, (c) ) Will be explained again.

(B) Measurement of the moving direction and the moving speed As a specific example of the operation screen which is different in time, the inside of the screen is
In order to track a moving object, the moving direction Φ and the speed V
The measuring method will be described. (b-1) Measurement of the moving direction and speed of the contour tangent The contour of the object is approximated by a straight line (tangent), and the tangent
The moving direction and speed can be measured by the following method. Current time image
And the image at the next time is "polar transformation + one-dimensional filter processing"
Data a_t(Ρ, θ), a_{t + DELTA}From (ρ, θ),
The basic correlation parameter C (ρ, θ, τ) is expressed by the following equation: C (ρ, θ, τ) = a_t(Ρ, θ) · a_{t + DELTA}(Ρ + τ, θ) Calculated from (24) and the point at which C (ρ, θ, τ) is maximal (ρ
_P, Θ_P, Τ_P), The tangent movement direction Φ = θ_P+90⁰  Moving speed of tangent V = (τ_P.DELTA..rho.) / DELTA... (25), and the tangent can be tracked from the data. here,
Δρ is the resolution in the ρ direction.

[0099] The direction and speed, resulting in errors when the moving direction V ₀ and the tangent of the azimuth are not orthogonal. The reason will be described with reference to FIG. 38. The tangent L measured by the equation (25)
Is the “direction orthogonal to the tangent direction” and “the speed V in the orthogonal direction”, respectively, and does not match the true direction vector V ₀ . A correct moving direction and speed measuring method in which this error is corrected will be described below, but this method is sufficient for tracking an object.
This is because the tracking is performed at short time intervals, and the error of the above equation is small every time and does not cause a problem.

FIG. 39 is a flowchart for measuring the moving direction and moving speed of the tangent. Delay image data (Step 4
9) The image data of one screen at the current time and the image data of one screen before a predetermined time obtained by the delay are divided into receptive field images (steps 50a and 50a '), and each receptive field image IM, IM ′ is subjected to polar conversion processing (step 51a,
51a '), one-dimensional filtering is performed on the result of the polar transformation, and the result is mapped onto a dual plane (steps 52a and 52a).
a '), the correlation processing of equation (24) is performed between the data mapped on each dual plane (step 53), and the calculated correlation parameter C (ρ, θ, τ) is stored in the correlation parameter storage unit ( Step 72) After the completion of the correlation calculation, the correlation parameter storage is scanned to detect (ρ _P , θ _P , τ _P ) at which C (ρ, θ, τ) becomes maximum (step 73), and finally,
Calculate the moving direction Φ and the moving speed V of the tangent by the formula (25).
(Step 74).

(B-2) Measuring the direction and speed of the corner movement To accurately measure the direction and speed of the corner, two or more
The top contour is needed. The target object is shown in FIG.
In general, a corner consisting of two contour tangent lines Li and Lj
It has CN, and it is accurate by correlation processing of this corner.
It can measure various moving directions and speeds. First, the case of (b-1)
Similarly, the correlation processing is performed by the equation (24). That is,
The image of the time and the image of the next time are referred to as “polar conversion + one-dimensional
Filtered data a _t(Ρ, θ), a_{t + DELTA}(Ρ,
θ) and the basic correlation parameter C (ρ, θ, τ) to (2
The point where C (ρ, θ, τ) becomes the maximum, calculated from equation 4)
(Ρ_P, Θ_P, Τ_P). Next, the maximum value search
The points corresponding to the two extracted tangent lines Li and Lj are (ρ_i,
θ_i, Τ_i), (Ρ_j, Θ_j, Τ_j), ...
The exact moving direction Φ and speed V are given by the following formula Φ = arctan [(τ_isinθ_j−τ_jsinθ_i) / (Τ_icosθ_j−τ_jcosθ_i)] (26) V = (Δρ · τ_i) / (Sin (Φ−θ_i) ・ DELTA) Measured by (27). Where DELTA is the difference between the current time and the next time
Time interval.

Equations (26) and (27) are derived as follows. That is, the tangent L measured by the equation (25) (FIG. 38)
The moving direction and the speed of the reference direction are a “direction orthogonal to the tangent direction” and a “speed V in the orthogonal direction”, respectively. Therefore, when moving in the Φ ₀ direction in FIG. 38, the speed measured by the equation (25) becomes V ₀ cosξ. From _{this, V 0 cosξ = (τ P} · Δρ) / DELTA (28) is satisfied. Incidentally, assuming that the clockwise direction is positive in FIG. 38, the angle ξ is given by the following equation: ξ = 90 ⁰ − (Φ ₀ −θ _P ) (29) (where θ _P is the direction of the tangent line L). (28)
By substituting the equation (9), V ₀ cos [90 ^0- (Φ ₀ -θ _P )] = (τ _P · Δρ) / DELTA (30). When transformed, V ₀ sin (Φ ₀ -θ _P ) = ( τ _P · Δρ) / DELTA (30) ′. Here, when τ _P = τ _i and θ _P = θ _i and V ₀ is obtained, equation (27) is derived. If τ _P is represented by τ and θ _P is represented by θ, and k = DELTA / Δρ, and the equation (30) is modified, the following equation is obtained: τ = k · V ₀ · cos [90 ^0- (Φ ₀ -θ)] (31), and this relational expression is drawn on the θ-τ plane, as shown in FIG.
As shown in (b), a sine wave (sinusoidal firing pattern) is obtained. In other words, each side Li, Lj of the figure is concentrated to one point on a sine wave on the θ-τ plane.

Now, by transforming equation (31), τ = k · V ₀ · sin (Φ ₀ -θ) (32) In this equation (32), (θ, τ) = (θ _i ,
_{τ i), (θ, τ} ) = (θ j, when the tau _j), the following equation _{τ i = k · V 0 ·} sin (Φ 0 -θ i) (32) 'τ j = k · V 0 · sin (Φ ₀ -θ _j ) (32) ″. From the equations (32) ′ and (32) ″, τ _i / τ _j = sin (Φ ₀ -θ _i ) / sin (Φ ₀ -θ _j ) is Holds, tanΦ ₀ = (τ _i sinθ _j −τ _j sinθ _i ) / (τ _i cosθ _j −τ _j cos
θ _i ) is derived, and equation (26) is derived. Note that the corner is composed of two contours, but if the feature is composed of three or more contours, the sine wave of FIG. 40 (b) can be determined more accurately.
Therefore, the measurement accuracy of (Φ ₀ , V ₀ ) can be improved.

(B-3) Measurement of moving direction / speed from polygon / curve In the above, measurement of moving direction / speed from two lines (corners) has been described. Focusing on the configured “polygon / curve”, “movement direction / velocity measurement” with higher reliability can be performed. The correlation parameter C _PRJ- ρ (θ, θ,
τ) C _PRJ- ρ (θ, τ) = ΣC (ρ, θ, τ) (ρ = 1,
2, ...) play an important role.

Measurement of Moving Direction and Speed from Polygon As shown in FIG. 41, the response of this parameter to a polygon figure is such that the response of each side of the polygon is distributed in a sine wave shape. In the case of an N-sided polygon, N peaks are arranged in a sine wave shape, and a polygon having a high speed is a sine wave having a large amplitude. From this, the moving direction and speed of the entire polygon can be accurately measured. Assuming that the angle between the azimuth of the straight line and the moving direction is ξ as shown in FIG. 38, τ _{P at} which the correlation parameter C (ρ, θ, τ) reaches a peak is given by τ _P = (V ₀ · DELTA) from equation (28). / Δρ) · cosξ, and C _PRJ− ρ (θ, τ) _obtained by projecting this in the ρ direction becomes a sine wave (sinusoidal firing pattern) in FIG. 41. When the maximum point (θmax, τmax) is obtained, the moving direction Φ ₀ = θmax−90 ⁰ (33a) The true moving speed V ₀ = (τmax · Δρ) / DELTA (33b) can be calculated. Compared to the aforementioned corner method (FIG. 40 (b)),
The basic principle is the same, but many points (N points) on the sine wave
Is distributed, so that there is a great feature that the peak (point at which the maximum amplitude is obtained) can be calculated accurately.

The extraction of the sine wave can be performed using polar transformation. In other words, pole transformation on cylinder (Hough
In the transformation, the points are converted into a sine wave, and when each point on the sine wave is further pole-transformed (inverse Hough transformation), each point is converted into a straight line intersecting at one point. Therefore, a sine wave can be extracted from the point. Specifically, each point of the C _PRJ- ρ (θ, τ) plane is converted into a straight line satisfying the following relationship τ = −Vy · cos θ + Vx · sin θ (34), and the intersection CP (Vx, Vy) (See FIG. 42). The direction and distance from the Vx, Vy coordinate origin to the intersection CP are θma in the equations (33a) and (33b).
x, τmax. The direction and distance to the intersection CP are (33
The reasons for θmax and τmax in equations a) and (33b) are as follows. (34), when deformation with true velocity V ₀ and direction ^{Φ τ = (√ (Vx 2} + Vy 2)) · sin (θ-Φ) = V 0 · (DELTA / Δρ) · sin (θ −Φ) (34) ′. Here, Φ = arctan (Vy / Vx). Therefore, C
_{PRJ- ρ (θ, τ)} sine wave peaks of the planes becomes _{τmax = V 0 · (DELTA /} Δρ) or ^{= √ (Vx 2 + Vy 2} ) θmax = Φ-90 0. From this, the true speed and direction can be calculated by the following equation: true speed V ₀ = (Δρ / DELTA) √ (Vx ² + Vy ² ) (35a) True direction Φ ₀ = arctan (Vy / Vx) (35b) . This method corresponds to “inverse Hough transform”.

FIG. 43 is a flowchart for measuring the moving direction and the moving speed as described above. The image data for one screen at the current time and the next time is divided into receptive field images (step 5).
0a, 50a '), polar conversion processing is performed on each receptive field image IM, IM' (steps 51a, 51a '), and the result of the polar conversion is subjected to one-dimensional filtering to be mapped on a dual plane (steps 52a, 52a). '), The correlation processing of equation (24) is performed between the data mapped on each dual plane (step 53), and the calculated correlation parameter C (ρ, θ, τ)
Is stored in the correlation parameter storage unit (step 72).
Then, the following equation C _PRJ- ρ (θ, τ) = ΣC (ρ, θ, τ) (ρ = 1,
2,...), The correlation parameter C _PRJ- ρ projected in the ρ direction
(Θ, τ) is determined (step 75), and stored in the correlation parameter storage unit on the θ-τ plane (step 76). Next, a polar conversion process is performed on the correlation parameter C _PRJ- ρ (θ, τ) according to equation (34) (step 77), and a peak point (intersection) on the Vx, Vy plane (velocity plane) is obtained (step 78). The direction and the distance from the coordinate origin to the intersection are obtained, and the true velocity V ₀ and the true direction Φ are calculated based on the equations (35a) and (35b).
₀ is calculated (step 79). Measurement of moving direction and speed from curved figures Although polygons have been described above, highly reliable measurements can be performed with curved figures as well. In the “receptive field method + polar conversion”, the tangent of the curve can be accurately extracted, and C _PRJ− ρ (θ, τ) may be similarly calculated from the data.

(B-4) Measurement of Moving Direction and Speed from Random Dots and Textures In the above, it has been described that the moving direction and speed can be measured from figures (polygons and curves) composed of straight lines and tangents.
Next, it is expanded to a figure composed of random points. The moving direction and speed of this figure can be measured by exactly the same processing as in (b-3). The reason is that "pair of points can be extracted as a straight line" by "receptive field method + one-dimensional filter",
This is because it is considered the same as a polygon (see FIG. 44).
According to this method, the moving direction and the speed of the “texture” graphic can be measured, as a matter of course, because the random dot graphic is composed of fine patterns. This expansion has a very great effect.

[0109] Figure 45, Figure 46 is a simulation result diagram in the direction of movement, speed measurement of the random dot shape, per second, the result of when moving 6√2 pixel 45 ⁰ direction, the delay DELTA 1 Set to seconds. The random dot figure is a random dot stereogram generated by a computer, and is drawn with one dot = 1 pixel, and has a density of 50%. In FIG. 45, I
M and IM ′ are receptive field images at the current time (no delay) and the next time (with a delay), respectively, and HCIM and HCIM ′ perform polar conversion processing on each receptive field image IM and IM ′, and the result of the polar conversion Is a hyper-column image obtained by performing a one-dimensional filtering process on a dual plane of ρ-θ. In FIG. 46, PRIM performs a correlation process between hypercolumn images, and obtains a correlation parameter C (ρ, θ, τ) in a θ-τ plane projected in the ρ direction by using a correlation parameter C _PRJ- ρ.
(Θ, τ), HGIM is the correlation parameter C _PRJ- ρ (θ,
τ) is a pole conversion result on the Vx and Vy planes after the pole conversion processing. On theta-tau plane, and it appears sinusoidal pattern SWV, also 45 ⁰ direction on speed plane, a sharp peak PK in 6√2 pixel positions are extracted. From this peak point (Vx, Vy), the true speed and the true moving direction can be calculated based on the equations (35a) and (35b).

In the cerebral hypercolumn, “Directional S
There are cells called "elasticity simple cells" and "Directional Selectivity complex cells", which detect the direction and speed of the moving contour line, and their functions are very similar to this method. In particular, on the fifth and sixth layers of the hypercolumn, there are complex cells that feed back signals to the superior hill or outer geniculate body. It plays an important role in controlling the eyeball to capture objects in the center of the visual field. From an engineering point of view, it is necessary for a mobile robot or the like to capture an object at the center of the visual field and perform operations such as approaching and avoiding the object, and this method is effective. In recent years, research on measuring the moving direction and speed of an object has been actively conducted, but these methods include "a method for calculating the moving direction and speed from a temporal change in brightness" and "a change due to movement of a graphic feature." Method of obtaining moving direction and speed from the above). However, the former has a practical problem that it is a differential treatment and is susceptible to changes in illumination and vibration, and the latter is important for the correspondence of graphic features. There is a limit to the application of the image processing technology which is weak in the application. To address these optical flow issues, the present method has the characteristic that the former is "an integral operation that converts the point cloud that composes a figure into a straight line by polar transformation and integrates it, and is resistant to noise." doing. The latter is characterized by the fact that the simplest figure, that is, it can be broken down into contour tangents and the tangents can be processed one-dimensionally by polar transformation, so that even complex figures can be reliably measured. have. In other words, this method can be said to be a "new optical flow method capable of one-dimensional processing of an integral type."

(B-5) Measurement of moving direction and speed of line / gap Moving of line / gap including simulation results below
The direction and speed measurement method will be described. Tools for moving direction and speed
The physical measurement method was (b-1) as an example, but (b-2) to (b-4)
But you can do the same. Measurement of moving direction and velocity with one-dimensional filter "Receptive field division + polar transformation + one-dimensional" in the flow of Fig. 39
The “filter process” is performed as shown in FIG. 33 (a) or FIG. 35 (a).
Binno division + polar transformation + one-dimensional Gaussian filter (+ sign
Inversion) process ”, the movement direction of the line / gap
The speed can be measured. The simulation results
As shown in FIG. Vertical line (θ ≒ 90⁰) 7 strokes horizontally
The correlation pattern obtained by inputting the two images before and after the elementary movement
The intensity of the parameter C (ρ, θ, σ) is indicated by a contour line. Basic
The sharp peak of the correlation parameter C (ρ, θ, σ) is the vertical line
And the corresponding point is denoted by (ρ_P, Θ_P, Τ_P)
And the moving direction = θ_P+90⁰= 182⁰  (Moving speed / DELTA) = τ_P= 6 images and the correct result was obtained.

Measurement of Movement Direction and Velocity with Two-Dimensional Filter (Biological) “Receptive field division + polar transformation + one-dimensional filter processing” in the flow of FIG. 39 is shown in FIG. 33 (b) or FIG. By performing the “two-dimensional Gaussian filter + receptive field division + polar transformation (+ sign inversion) processing” shown in FIG. In this method, an input image is subjected to a two-dimensional second-order differentiation process (two-dimensional convolution filter process) using a two-dimensional gas filter, and then divided into each receptive field, and each receptive field image is subjected to polar transformation. is there. Two-dimensional gas filter processing is "one-dimensional filter processing after polar transformation"
Although the simulation results are omitted, the same measurement results of the moving direction and the speed as in FIG. 47 are obtained. This method is the same as human stereoscopic vision, and has the advantage that the result of contour enhancement by a two-dimensional gas filter can be commonly used for shape recognition and the like.

(B-6) Measurement of Edge Moving Direction and Velocity The following is the most frequently used image including simulation results.
Of moving direction and speed of edge, which is an image feature appearing on the screen
The formula will be described. The specific method of measuring the moving direction and speed is
(b-1) is used as an example, but (b-2) to (b-4) can be executed in the same way.
You. Measurement of edge moving direction and velocity by one-dimensional filter "Receptive field division + polar transformation + one-dimensional"
The filter processing is performed as shown in FIG. 34 (b).
Transform + one-dimensional gradient filter + one-dimensional Gaussian
Filter processing ”, the edge moving direction and speed
Degree measurement is possible. Figure of the simulation result
48. 120⁰Edge moved 7 pixels horizontally
Correlation parameter C obtained by inputting two images before and after
The intensity of (ρ, θ, σ) is shown by contour lines. Basic correlation parameters
The sharp peak of data C (ρ, θ, σ) indicates the corresponding point of the edge.
And the point is (ρ_P, Θ_P, Τ_P), The moving direction = θ_P+90⁰= 210⁰  (Moving speed · DELTA) / Δρ = τ_P= 7 images and the correct result was obtained. Where Δρ is the decomposition in the ρ direction
Noh, DELTA is the delay time.

Measurement of moving direction and velocity of edge with two-dimensional filter mixed (biological binocular stereopsis) "Receptive field division + polar transformation + one-dimensional filter processing" in the flow of Fig. 39 is shown in Fig. 34 (c). By using “two-dimensional Gaussian filter + receptive field division + polar transformation + one-dimensional gradient filter processing” shown in (1), it is possible to measure the moving direction and speed of the edge. This method uses two-dimensional
The gas filter performs two-dimensional second-order differentiation processing (two-dimensional convolution filter processing), and thereafter, divides each receptive field, and performs polar transformation on each receptive field image. Two dimensions
The gas filter processing is equivalent to “one-dimensional filter processing after polar transformation”, and the simulation results are omitted.
The same movement direction and speed measurement results as in 1 are obtained. This method is the same as human stereoscopic vision, and has the advantage that the result of contour enhancement using a two-dimensional gas filter can be commonly used for shape recognition and the like.

(C) Binocular stereoscopic view of polygonal figure, curved figure, random dot, and texture The calculated correlation parameter C _PRJ- ρ (θ, σ) C _PRJ- ρ (θ, σ) = ΣC (ρ, θ, σ) (ρ = 1,
By using (2,...), Stereoscopic measurement of polygonal figures and curved figures becomes possible. The method is based on the time correlation C _PRJ- ρ in (b).
It is only _necessary to replace (θ, τ) with the binocular correlation C _PRJ- ρ (θ, σ). In this method, the reliability is improved by the number of sides as compared with binocular stereoscopic vision from one straight line.

[0116] Now, a straight line SL _L in FIG. 49 in the input image of the right and left eyes appear shifted as parallel indicated by SL _R.
The difference between the straight line positions is obtained as a displacement σ in a direction orthogonal to the azimuth θ of the straight line on the hypercolumn plane of ρ−θ. Therefore, the binocular parallax d in the horizontal direction can be expressed by the following equation: d = σ / sin θ. From the above, when the horizontal parallax (binocular parallax) is d, the displacement σ of the straight line in the θ direction is given by the following equation: σ = d · sin θ (36) When this relational expression is drawn on the θ-σ plane, it becomes a sine wave as shown in FIG. In other words, each side Li, Lj, Lk of the figure shown in FIG. 50 (a) captured by the left and right eyes is pole-transformed, and the correlation between the hypercolumn images obtained by the pole transformation is calculated based on the equation (1). Perform processing to obtain correlation parameter C
(Ρ, θ, σ), and calculates the correlation parameter C (ρ,
θ, σ) is projected in the ρ direction according to equation (9) to obtain a correlation parameter C _PRJ- ρ (θ, σ), and when displayed on the θ-ρ plane, each side Li, Lj, Lk becomes Points Li ', Lj', Lk '
Is concentrated.

Therefore, as in the case of (b-3), C _PRJ- ρ (θ, σ) is calculated from the binocular input image, and it is pole-transformed by the equation (34). As shown, each point on the sine wave is converted into a straight line passing through one point PK on the Vx and Vy planes.
When this peak point PK (Vx, Vy) is obtained, a true parallax σ _{0 is obtained.}
Can be calculated by the following equation: σ ₀ = Δρ · √ (Vx ² + Vy ² ) (37) The direction of parallax is always the direction connecting both eyes.
(Horizontal direction). Further, assuming that the direction in which the arbitrary figure can be seen is ξ and the distance between the eyes is d, the distance D to the arbitrary figure is calculated by the following equation: D = d · sin (σ ₀ + ξ) / sin (σ ₀ ) (38) it can.

FIG. 51 is a diagram for explaining a method of calculating the distance to an arbitrary figure by binocular stereoscopic vision, where L is a straight line, and E _L and E _R are the left and right eyes. Assuming that the parallax obtained as described above is σ ₀ , the direction in which the arbitrary figure can be seen is ξ, the distance between the eyes is d, and the distance from the left and right eyes to the arbitrary figure is D ₁ and D ₂ , the following equation is given: D ₁ sinξ = D ₂ sin (ξ + σ ₀ ) D ₁ cosξ−D ₂ cos (ξ + σ ₀ ) = d. When obtaining the D ₁ from the simultaneous equations (38) is obtained.

FIG. 52 is a flowchart for measuring the parallax and the distance to an arbitrary figure by binocular stereovision. The left and right eye image data are each divided into receptive field images (steps 50a and 50a).
0a '), polar conversion processing is performed on each receptive field image IM, IM' (steps 51a, 51a '), the result of the polar conversion is subjected to one-dimensional filtering processing, and mapped to a dual plane (steps 52a, 52a'). The correlation processing of the equation (1) is performed between the data mapped on each dual plane (step 5).
3) The calculated correlation parameter C (ρ, θ, σ) is stored in the correlation parameter storage unit (step 72). Then, according to equation (9), the correlation parameter C projected in the
_PRJ- ρ (θ, σ) is obtained (step 75 ′) and stored in the correlation parameter storage unit on the θ-σ plane (step 7).
6 '). Then, the correlation parameter C _PRJ- ρ (θ, σ)
The pole conversion processing is performed according to the equation (34) (step 77 ').
A peak point (intersection) on the x, Vy plane is determined (step 78 '), and a distance from the coordinate origin to the intersection is determined.
A binocular disparity σ ₀ and a distance D to an arbitrary figure are calculated based on the equations (37) and (38) (step 79 ′).

FIGS. 53 and 54 are explanatory diagrams of simulation results of binocular stereopsis of random dot figures, in which random dot figures different in six pixels are input to the left and right eyes. Note that this random dot figure is a random dot stereogram generated by a computer.
The dot is drawn with one pixel, and the density is 50%. In FIG. 53, IM and IM 'are receptive field images of the left and right eyes, respectively, and HCIM and HCIM' are receptive field images IM and IM ', respectively.
Is a hyper-column image obtained by performing a polar conversion process on the resultant data, performing a one-dimensional filtering process on the result of the polar conversion, and mapping the result to a dual plane of ρ-θ. Further, in FIG. 54, PRIM is subjected to a correlation processing between hypercolumn image, resulting correlation parameter C (ρ, θ, σ) correlation parameters in projected theta-sigma plane to [rho direction C _PRJ- ρ (θ , Σ), HGI
M is a polar conversion result on the Vx, Vy plane _{obtained by performing a} polar conversion process on the correlation parameter C _PRJ- ρ (θ, σ). A sine wave pattern SWV appears on the θ-σ plane, and Vx,
On the Vy plane, sharp peaks PK are extracted at six pixel positions in the horizontal direction from the origin. This peak point (Vx, Vy)
Thus, the binocular disparity σ ₀ and the distance D to an arbitrary figure can be calculated based on the equations (37) and (38).

Binocular Stereoscopic View of Random Dot Texture The above-described replacement (that is, the same method as above) enables binocular stereoscopic view of a “random dot graphic or texture graphic”. This makes it possible to easily perform binocular stereoscopic viewing of a plane having a fine pattern. This is the psychologist Julesz
Is a cell-level model of the fact that "we can binocularly view even a random dot figure whose shape is unknown".

(D) Motion stereoscopic vision Linear motion stereoscopic vision The method for determining the speed and moving direction of a moving object has been described in (b). With the same processing, when the image capturing means (camera or the like) moves, the distance (depth) to a straight line in the space can be measured. That is, the correlation parameter C (ρ, θ, τ) is obtained by the equation (4) ′.
, And the peak points (ρ _P , θ _P ,
τ _P ), the disparity between the current image (the image after the movement) and the previous image (the image before the movement) is calculated by τ _P · Δρ.
The direction in which the straight line can be seen is ξ, and the moving speed of the image capturing means is V
If the delay of s and (4) ′ is DELTA, the distance to the straight line is
(38) below and it can be calculated by similarly following equation D = (Vs · DELTA) · sin (τ P · Δρ + ξ) / sin (τ P · Δρ) (39).

FIG. 55 is a flowchart for measuring the depth to a straight line in the space by the motion stereoscopic vision. The image data before the movement is delayed (step 49), and the image data before the movement a _t (ρ, θ) and the image data after the movement a _{t + DELTA} (ρ +
τ, θ) are divided into receptive field images (step 50).
a, 50a '), perform a polar conversion process on each receptive field image IM, IM' (steps 51a, 51a '), apply a one-dimensional filtering process to the result of the polar conversion, and map the result to a dual plane (steps 52a, 52a). '), The correlation processing of the equation (4)' is performed between the data mapped on each dual plane (step 53), and the calculated correlation parameters C (ρ, θ, τ)
Is stored in the correlation parameter storage unit (step 72).
Next, the peak point (ρ _P , θ _P , τ _P ) of the correlation parameter C (ρ, θ, τ) is determined (step 73), and the parallax τ _P
The depth to a straight line in the space is calculated by Expression (39) using Δρ (step 91).

Motion Stereoscopic View of Arbitrary Figure When the image capturing means (camera or the like) moves, the distance (depth) to the arbitrary figure in space can be measured. That is, the same processing as described in (b-3) is performed on each of the images before and after the image capturing means moves to determine the peak points on the Vx and Vy planes, and the equations (35a) and (35b) are used. To calculate the true speed V ₀ and the moving direction. Assuming that the direction in which the figure can be seen is ξ, the moving speed of the image capturing means is Vs, and the delay in equation (4) 'is DELTA, the distance to the arbitrary figure is the following equation as in equation (38): D = (Vs · DELTA)・ Sin (V ₀ · DELTA + ₀ ) / sin (V ₀ · DELTA) (40)

FIG. 56 is a flowchart for measuring the depth to an arbitrary figure in the space by the stereoscopic movement. The image data before the movement is delayed (step 49), and the image data before the movement a _t (ρ, θ) and the image data after the movement a _{t + DELTA} (ρ
+ Τ, θ) is divided into receptive field images (step 50).
a, 50a '), perform a polar conversion process on each receptive field image IM, IM' (steps 51a, 51a '), apply a one-dimensional filtering process to the result of the polar conversion, and map the result to a dual plane (steps 52a, 52a). '), The correlation processing of the equation (4)' is performed between the data mapped on each dual plane (step 53), and the calculated correlation parameters C (ρ, θ, τ)
Is stored in the correlation parameter storage unit (step 72).
Next, the correlation parameter C _PRJ- ρ (θ, τ) projected in the ρ direction is obtained by equation (10) (step 75), and θ-τ
It is stored in the plane correlation parameter storage unit (step 7).
6). Next, the correlation parameter C _PRJ- ρ (θ, τ) is _calculated as (3
The pole conversion processing is performed by the equation (4) (step 77), and Vx, V
Find a peak point (intersection) on the y-plane (step 7)
8) The true velocity V ₀ and the true direction Φ ₀ are calculated based on the equations (35a) and (35b) (step 79). Finally, the depth to the arbitrary figure is calculated based on the equation (40).

(E) Generalization By generalizing the above, the correlation processing method of the present invention can be generalized as shown in FIG. That is, the input data is subjected to a polar conversion process and mapped to a dual plane of ρ-θ (step 501), and data a (ρ, θ) obtained by the polar conversion is obtained.
Is subjected to new parameter addition processing, correlation processing, projection with arbitrary parameters or polar transformation processing (projection in the 0 ⁰ or 90 ⁰ direction is the same as polar transformation), thereby including a sinusoidal firing pattern on a plane. Data a (ξ ₁ ,
ξ ₂ ,...), or on a spherical surface, data including a firing pattern of a great circle (step 502), and a sine-wave or great-circle firing pattern included in the data is extracted by a reverse pole conversion process. And output useful data (step 503). In the equations (1) and (4), the “correlation processing” is performed by “adding a new parameter σ or τ”. Also, (9), (1
In equation (0), “projection along parameter ρ” is performed. Further, in the equation (19) is carried out 45 ⁰ direction of the projection in ρσ plane, the projection "([rho, sigma) polar transformation plain"
To the same as that removed 45 ⁰ direction component.

In the measurement of the moving direction and the speed described in (b-2) and (b-3), two screen data different in time are subjected to a polar conversion process to obtain the data a in the dual plane of ρ-θ. (Ρ, θ) and b (ρ, θ) are obtained (step 501). Next, the correlation processing is performed by the equation (4) by introducing the moving speed parameter τ in the ρ-axis direction, and the correlation parameter C (ρ, θ, τ) obtained by the correlation processing is projected in the ρ direction by the equation (10). And a sinusoidal firing pattern (FIG. 40,
The projection data C _PRJ- ρ (θ, τ) including the projection data (see FIG. 41) is obtained (step 502). Then, the projection data C _PRJ-
ρ (θ, τ) is subjected to reverse pole conversion processing to extract a sinusoidal firing pattern (point giving a maximum point) and output useful data (moving direction and speed) (step 50).
3). `` Reverse pole transformation '' to extract sinusoidal firing patterns
Is expressed as follows. That is, assuming that δ () is a delta function and τ _X and τ _Y are velocity parameters in the X and Y axis directions, output (τ _X , τ _Y ) = {θΣτ ｛C _PRJ- ρ (θ, τ) · δ
(Τ + τ _X cos θ + τ _Y sin θ)｝. Since the delta function δ () has meaning only at the point of 0, if the above equation is modified, the output (τ _X , τ _Y ) = { _{θΣC PRJ-} ρ (θ, _{-τ X} cos θ-τ
_Y sinθ)｝, and the content of the reverse pole conversion is clearly understood. This calculation is performed in the embodiment.

In the measurement of binocular stereopsis described in (c), two screen data captured by both eyes are subjected to a polar conversion process, and data L (ρ, θ) in a ρ-θ dual plane is obtained. ) And R (ρ, θ) (step 50)
1). Next, the parallax σ in the ρ-axis direction is introduced, the correlation processing is performed by the equation (1), and the correlation parameter C (ρ, θ, σ) obtained by the correlation processing is projected in the ρ direction by the equation (9). Projection data C _PRJ- ρ (θ, σ) including a sinusoidal firing pattern (see FIG. 50) is obtained (step 502). Thereafter, the projection data C _PRJ- ρ (θ, σ) is subjected to _{reverse pole} conversion processing to extract a sine wave-like firing pattern (a point giving a maximum point) and output useful data (parallax) (step 503). ). The expression "reverse pole transformation" for extracting a sinusoidal firing pattern is expressed as follows. That is,
Assuming that δ () is a delta function and σ _X and σ _Y are parallax parameters in the X and Y axis directions, output (σ _X , σ _Y ) = {θΣσ ｛C _PRJ- ρ (θ, σ) · δ
(Σ + σ _X cos θ + σ _Y sin θ)｝. Since the delta function δ () has meaning only at the point of 0, if the above equation is transformed, the output (σ _X , σ _Y ) = { _{θΣC PRJ-} ρ (θ, _{-σ X} cos θ-σ
_Y sin θ)｝. This calculation is performed in the embodiment.

In the above description, in step 502 of FIG. 57, a sinusoidal firing pattern is obtained by performing correlation processing or the like. However, a sinusoidal firing pattern can be obtained only by polar conversion without performing correlation processing or the like. May be possible. In such a case, in step 502, the (ρ, θ) data obtained by the polar conversion in step 501 is subjected to conversion processing such as shift or rotation, and the (ρ, θ) data obtained by the conversion processing is subjected to step 503. By extracting the sinusoidal firing pattern (the point giving the maximum point) by performing the inverse pole conversion processing of, useful data can be output.
For example, when the circle of the radius R shown in FIG.
As shown in (b), a sine wave shifted by R in the ρ direction is obtained. Assuming that the center of the circle is (x ₀ , y ₀ ), this sine wave is given by: ρ = R−β ₀ cos (θ−α ₀ ) (41) β ₀ = √ (x ₀ ² + y ₀ ² ) α ₀ = arctan (x ₀ / y ₀ ). Therefore, the data obtained by the polar conversion is shifted by R in the ρ direction, and the data a obtained by the shift process
If the sinusoidal firing pattern (the point giving the maximum point) is extracted by performing the inverse pole conversion process on (θ, σ), the center of the circle can be output (step 503).

FIG. 59 is an explanatory diagram of a simulation result of circle detection. Laplacian filter processing is applied to a black circle drawn on a white sheet to obtain input image data. (Ρ, θ) in a ρ-θ plane having a sinusoidal firing pattern
θ) data, the (ρ, θ) data is shifted by R in the ρ direction, and the data obtained by the shift processing is subjected to the inverse polar transformation. In addition, the intensity of ignition is represented by a contour line, and the positive part is distinguished by shading. In FIG. 59 (b), two parallel sine waves SN1 and SN2 are obtained by the polar transformation in consideration of the physiological knowledge of the hypercolumn “the direction θ is in the range of 0 to π” (41). This is because the range of π to 2π in the equation is inverted to the negative side of ρ. As is apparent from FIG. 59 (d), one sine wave fires as a sharp "negative peak PK" corresponding to the center of the circle, and the other sine wave fires as a weak ring RG having a radius of 2R. ing. As described above, the present invention has been described with reference to the embodiments. However, the present invention can be variously modified in accordance with the gist of the present invention described in the claims, and the present invention does not exclude these.

[0131]

As described above, according to the present invention, the input data is subjected to polar conversion processing and mapped on a dual plane, or the result of the polar conversion processing is further filtered and mapped on dual planes. Thereafter, a configuration is adopted in which a correlation process is performed between the mapping data on the dual plane to measure a variable defining the relationship between the graphic features (for example, tangents) of the input data. Can be easily and accurately performed with a small amount of processing, and binocular stereoscopic functions (parallax, depth measurement, measurement of moving speed of object, measurement of moving direction, etc.) can be realized. Further, according to the present invention, since the polar conversion process, the filter process, and the correlation process are performed on the receptive field image when one screen is divided into the receptive fields, which are small regions, the processing amount is significantly reduced. it can.

Further, according to the present invention, when the receptive field image belongs to different screens captured by two or three cameras, the functions of binocular stereoscopic vision and trinocular stereoscopic vision can be realized. If the images belong to different screens in time, the moving direction and the moving speed of the feature (line, corner, etc.) appearing in the receptive field can be measured. For this reason, it is possible to move while capturing the target object at the center of the field of view, and it is possible to apply to a mobile robot, an unmanned self-propelled vehicle, and the like. Further, when each receptive field image belongs to the screen before and after the movement of the image capturing means such as a camera, the function of the stereoscopic movement can be realized and the depth to the target object can be measured. According to the present invention, each receptive field image is an image of a different receptive field on the same screen,
Alternatively, by using the same receptive field image, a texture analysis or the like for examining the degree of the same pattern in one screen can be performed.

Further, according to the present invention, the correlation processing between a plurality of receptive field images is performed for each same color or for each color difference signal.
Alternatively, since it is performed for each of the three elements of color, binocular stereopsis, moving object gaze, and texture analysis can be performed more reliably. According to the present invention, a corresponding line or gap in a plurality of figures is obtained by performing one-dimensional Gaussian filter processing after polar conversion or performing two-dimensional Gaussian filter processing before polar conversion processing. One-dimensional gradient filter processing and one-dimensional Gaussian filter processing after polar conversion, or two-dimensional Gaussian filter processing before polar conversion processing and one-dimensional gradient filter processing after polar conversion processing , It is possible to obtain corresponding edges in a plurality of figures, and further obtain the position, orientation, parallax, moving direction, and moving speed of these figure elements.

Furthermore, according to the present invention, when a plurality of receptive field images belong to spatially different screens, the correlation parameter Cθ (ρ, θ, σ) in the θ-axis direction is calculated, or (ρ , Θ) plane correlation parameter C (ρ,
By calculating θ, σ ₁ , σ ₂ ), it is possible to extract a tangent that moves while changing the direction. According to the present invention, when each receptive field of a plurality of image data belongs to a screen different in time, the correlation parameter C (ρ, θ,
By calculating τ), the position of the tangent line that translates,
The azimuth and speed can be determined quantitatively, and the position, azimuth and azimuth rotation speed of the tangent passing through the center of the receptive field can be quantified by calculating the correlation parameter Cθ (ρ, θ, τ) in the θ-axis direction. By calculating the correlation parameter C (ρ, θ, τ ₁ , τ ₂ ) of the (ρ, θ) plane, the position, azimuth, and moving speed of the tangent moving while changing the direction can be obtained. In addition, the rotational speed can be quantitatively measured.

Further, according to the present invention, the correlation parameter C
(Ρ, θ, σ) is projected in the σ-axis direction indicating the spatial shift amount σ, or projected in the ρ-axis direction indicating the tangential position, or projected in the θ-axis direction indicating the tangential direction, or any of these. By projecting in the two-axis direction, the memory capacity for storing the correlation parameter can be reduced by one or two axes, and by selecting the projection direction, a desired value among the tangent position, parallax, and azimuth can be obtained. Obtainable.
Also, according to the present invention, the correlation parameter C (ρ, θ, τ)
Is projected in the τ-axis direction indicating the temporal shift amount (moving speed) τ, in the ρ-axis direction indicating the tangential position, or in the θ-axis direction indicating the tangential direction, or in any two-axis direction. , The memory capacity for storing the correlation parameter can be reduced by one or two axes,
Moreover, by selecting the projection direction, it is possible to obtain desired values among the position, azimuth, translation speed, rotation speed, and the like of the tangent.

Further, according to the present invention, a polar transformation process is performed on the receptive field image to map it onto a dual plane of ρ-θ, and the mapping data a (ρ, θ) of the dual plane and another dual plane An accurate filtering can be performed by taking out a set of mapping data b (ρ, θ) whose coordinate values are shifted by a predetermined amount in the ρ-axis direction from the mapping data b (ρ, θ) set on the plane, and calculating the sum of the products. Extract features that look the same with the left and right eyes "
It is suitable to be applied when performing “tracking of the same feature as the previous image” or the like. Further, according to the present invention, the mapping data a (ρ, θ) of the dual plane is compared with the other mapping data b (ρ, θ).
θ) is subtracted after being shifted in the ρ-axis direction or the θ-axis direction, and the shift amount is sequentially changed and subtracted.
By using the obtained subtraction result as a correlation parameter,
It is possible to perform binocular stereoscopic vision and tracking of an object whose outline is unclear using clues such as gradual changes in brightness and hue as clues.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the principle of the present invention.

FIG. 2 is a diagram illustrating the principle of binocular stereoscopic vision according to the present invention.

FIG. 3 is an explanatory diagram of a spherical mapping (polar conversion of a spherical surface).

FIG. 4 is an explanatory diagram of pole conversion on a cylinder.

FIG. 5 is a diagram illustrating the principle of the receptive field method.

FIG. 6 is an explanatory diagram of pole conversion in a hyper column.

FIG. 7 is a configuration diagram of an embodiment of the present invention.

FIG. 8 is a flowchart of a correlation process.

FIG. 9 is a configuration diagram of a pole conversion circuit.

FIG. 10 is a configuration diagram of a one-dimensional filter.

FIG. 11 is a first explanatory diagram illustrating a simulation result.

FIG. 12 is a second explanatory diagram illustrating a simulation result.

FIG. 13 is a third explanatory diagram illustrating a simulation result.

FIG. 14 is a basic flowchart of correlation filtering.

FIG. 15 is a flowchart of correlation filtering between spatially different screens.

FIG. 16 is a flowchart of correlation filtering between temporally different screens.

FIG. 17 is a flowchart of correlation filtering between screens different in time and space.

FIG. 18 is a flowchart of correlation filtering between receptive fields on the same screen.

FIG. 19 is a flowchart of correlation filtering in the same screen and the same receptive field.

FIG. 20 is a flowchart of correlation filtering between different color images (three primary colors).

FIG. 21 is a flowchart of correlation filtering between different color images (three elements of color).

FIG. 22 is a flowchart of correlation filtering between different color images (color difference signals).

FIG. 23 is a flowchart of ρ-axis correlation filtering with the same θ.

FIG. 24 is a flowchart of ρ-axis correlation filtering with different θ.

FIG. 25 is an explanatory diagram of the θ direction correlation.

FIG. 26 is a flowchart of θ-axis correlation filtering with the same ρ.

FIG. 27 is a flowchart of correlation filtering in the ρ-θ plane.

FIG. 28 is an explanatory diagram of a correlation parameter space.

FIG. 29 is an overall flowchart of a filtering process of projecting a correlation parameter.

FIG. 30 is a flowchart of a process of projecting a correlation parameter in the σ direction.

FIG. 31 is a configuration diagram of a natural filter.

FIG. 32 is a flowchart of a difference type correlation process.

FIG. 33 is a diagram illustrating various configurations of a line extraction filter.

FIG. 34 is a diagram illustrating various configurations of an edge extraction filter.

FIG. 35 is a diagram showing various configurations of a gap extraction filter.

FIG. 36 is a first explanatory diagram (binocular stereoscopic view of an edge) showing a simulation result.

FIG. 37 is a second explanatory diagram (binocular stereoscopic view of an edge) showing a simulation result.

FIG. 38 is an explanatory diagram of a moving direction and a speed of a tangent line.

FIG. 39 is a flowchart for measuring a moving direction and a moving speed.

FIG. 40 is an explanatory diagram of corner moving direction and speed detection.

FIG. 41 is an explanatory diagram of a response of C _PRJ- ρ (θ, τ) to a polygon.

FIG. 42 is an explanatory diagram of extraction of a sine wave by the inverse Hough transform.

FIG. 43 is a flowchart for measuring a moving direction and a moving speed.

FIG. 44 is an explanatory diagram of pole conversion from a random dot figure.

FIG. 45 is an explanatory diagram of a simulation result in measuring the moving direction and speed of a random dot graphic.

FIG. 46 is an explanatory diagram of a simulation result in measuring the moving direction and speed of a random dot graphic.

FIG. 47 is an explanatory diagram of a simulation result (moving direction and speed of a line).

FIG. 48 is an explanatory diagram of a simulation result (moving direction and speed of an edge).

FIG. 49 is an explanatory diagram showing a relationship between a line shift amount and parallax.

FIG. 50 is an explanatory diagram of binocular stereoscopic viewing of an arbitrary graphic.

FIG. 51 is an explanatory diagram of a distance calculation method based on binocular stereovision.

FIG. 52 is a flowchart of parallax / distance calculation by binocular stereopsis.

FIG. 53 is an explanatory diagram of a simulation result of binocular stereoscopic viewing of a random dot figure.

FIG. 54 is an explanatory diagram of a simulation result of binocular stereoscopic viewing of a random dot figure.

FIG. 55 is a flowchart showing a depth calculation flow up to a straight line in motion stereoscopic vision.

FIG. 56 is a flowchart showing a depth calculation flow up to an arbitrary figure in a motion stereoscopic view.

FIG. 57 is a flowchart in the case of generalization.

FIG. 58 is an explanatory diagram of circle detection.

FIG. 59 is an explanatory diagram of a simulation result of circle detection.

FIG. 60 is an explanatory diagram of a problem in binocular stereopsis.

FIG. 61 is a diagram illustrating depth calculation for binocular stereopsis.

[Explanation of symbols]

14, 24 polar conversion unit 16, 26 one-dimensional filter 17, 27 dual plane of ρ-θ (hyper column memory) 30, correlation processing unit

Continuation of the front page (56) References JP-A-5-165957 (JP, A) JP-A-5-165957 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) G06T 7 / 00 G06T 7/20 JICST file (JOIS)

Claims (20)

(57) [Claims] 1. A polar conversion process is performed on input data to obtain ρ−θ
To the dual plane, and perform correlation processing on the (ρ, θ) data obtained by polar transformation.
Or (ρ, θ) data obtained by polar transformation
Shift processing so that the firing pattern is sinusoidal on the plane.
The data including the turn, the firing pattern of the great circle on the spherical surface
And obtaining useful data by extracting a sine-wave or great-circle firing pattern included in the data by an inverse-polar transformation process. 2. A plurality of input data are respectively subjected to a polar conversion process to be mapped onto a dual plane of ρ−θ, or a result of the polar conversion process is further subjected to a filtering process to perform a dual plane of ρ−θ. A correlation processing method characterized by performing a correlation process between data mapped onto each dual plane, and obtaining a relationship between graphic features of each input data based on the correlation result. 3. The correlation processing method according to claim 2, wherein the input data is image data corresponding to a receptive field when one screen is divided into small receptive fields. 4. A plurality of input data are respectively subjected to a polar conversion process to be mapped onto a dual plane of ρ-θ, or a result of the polar conversion process is further subjected to a filtering process to perform a dual plane of ρ-θ. The correlation result is obtained by performing a correlation process using the position (ρ, θ) of the mapping data on the dual plane and the shift amount between the mapping data as element parameters, and obtaining a point having a characteristic value among the correlation results. And calculating a variable that defines the relationship between the graphical features of each input data based on the value of the element parameter that gives the characteristic correlation result. 5. The input data is image data of a receptive field when one screen is divided into receptive fields, which are small areas, and performs polar conversion processing on the receptive field image data to map them onto a dual plane. Then, by sequentially changing the shift amount in the ρ-axis direction or the shift amount in the θ-axis direction between the respective mapping data, correlation processing in the ρ-axis direction, correlation processing in the θ-axis direction, or (ρ, θ)
The correlation processing method according to claim 4, wherein the correlation processing is performed on a plane. 6. The mapping data a (ρ,
θ) is multiplied after shifting other mapping data b (ρ, θ) in the ρ-axis direction or the θ-axis direction,
6. The correlation processing method according to claim 5, wherein the shift amount is sequentially changed and multiplication is performed, and an obtained multiplication result is used as a correlation result. 7. The mapping data a (ρ,
θ), the other mapping data b (ρ, θ) is shifted by a predetermined amount in the ρ-axis direction or the θ-axis direction and then subtracted, and the shift amount is sequentially changed and subtracted. 6. The correlation processing method according to claim 5, wherein the subtraction result is a correlation result. 8. A polar conversion process is performed on input data representing a receptive field image when one screen is divided into small receptive fields, and the input data is mapped onto a dual plane of ρ-θ, or a polar conversion process is performed. The result is further filtered to obtain ρ-θ
Of the dual plane mapping data a (ρ, θ) and the mapping data b (ρ, θ) set in another dual plane, the coordinate values of which are shifted by a predetermined amount in the ρ-axis direction. Take out the pair
The product sum Ci (ρ, θ) = Σa (ρ + i, θ) · b (i, θ) (where i is added) is calculated as a correlation parameter, and the mapping data of the input data is calculated using these correlation parameters. A correlation processing method characterized by extracting graphic features corresponding to b (ρ, θ). 9. The mapping data a (ρ, θ), b (ρ,
9. The correlation processing method according to claim 8, wherein θ) is obtained by performing a polar transformation on spatially different input data such as binocular stereoscopic vision and mapping the input data on a dual plane of ρ−θ. 10. The mapping data a (ρ, θ), b
9. The correlation processing method according to claim 8, wherein (ρ, θ) is obtained by performing a polar transformation on input data different in time and mapping the input data to a dual plane of ρ−θ. 11. A receptive field image obtained by dividing an image input from two image input means into receptive fields, respectively, to perform polar transformation processing to map onto a dual plane of ρ-θ, or to perform polar transformation. The processing result is further subjected to filter processing and mapped to dual planes of ρ-θ, respectively. For the mapping data a (ρ, θ) of the dual plane,
The other mapping data b (ρ, θ) is shifted in the ρ-axis direction and then multiplied. In addition, the shift amount σ is sequentially changed and multiplied, and the obtained multiplication result is used as the correlation parameter C (ρ, θ).
θ, σ), a correlation tangent between each receptive field image is determined from the maximum point of the correlation parameter, and the parallax is measured based on σ giving the maximum point. 12. The tangent line is obtained as a line or a gap by performing one-dimensional Gaussian filter processing after polar conversion or performing two-dimensional Gaussian filter processing before polar conversion processing. The correlation processing method according to claim 11. 13. One-dimensional gradient filter processing and one-dimensional Gaussian filter processing after polar conversion, or two-dimensional Gaussian filter processing before polar conversion processing, and one-dimensional gradient filter processing after polar conversion processing. 12. The correlation processing method according to claim 11, wherein the tangent is obtained as an edge by performing the following. 14. A receptive field image obtained by dividing an image input from two image input means into receptive fields is subjected to polar transformation processing to map the image onto a dual plane of ρ-θ, or polar transformation. The processing result is further subjected to filter processing and mapped to dual planes of ρ-θ. One mapped data a (ρ, θ) of the dual plane is mapped to the other mapped data b (ρ, θ) Are shifted in the ρ-axis direction and then multiplied, and the shift amount σ is sequentially changed to be multiplied.
After converting the correlation parameter C (ρ, θ, σ) into C _PRJ −ρ (θ, σ) projected in the ρ direction, the sine in the (θ, σ) plane A correlation processing method wherein a point having a maximum amplitude of a wave pattern is extracted by inverse polar transformation, and binocular disparity and / or a depth to a target straight line are calculated using σ which gives the maximum amplitude. 15. A polar transformation process is applied to receptive field images obtained by dividing two temporally different images into receptive fields to map them onto a dual plane of ρ-θ, or a result of the polar transformation process. further performs a filtering process to map the dual plane, respectively [rho-theta, present time of the mapping data a _t of the dual plane relative to (ρ, θ)
To the mapping data at _{+ 1} (ρ, θ) at the next time
After shifting by a predetermined amount in the axial direction, multiplication is performed, and the shift amount τ is sequentially changed to perform multiplication. The obtained multiplication result is used as a correlation parameter C (ρ, θ, τ). Determine the corresponding tangent between the receptive field images from the element parameter values ρ _P , θ _P , τ _P that give (ρ _P , θ _P , τ _P ) and measure the moving direction and moving speed of the corresponding tangent. A correlation processing method characterized by the following. 16. The following equation: Φ = θ_P+90⁰  V = k₁・ Τ_P (k₁Is a constant) to calculate the moving direction Φ and the moving speed V.
16. The correlation processing method according to claim 15, wherein: 17. The parameters of the maximum points of two tangents forming the corner of the receptive field image are (ρ _i , θ _i , τ _i ),
When (ρ _j , θ _j , τ _j ), the following equation Φ = arctan {(τ _i sin θ _j −τ _j sin θ _i ) / (τ _i cosθ _j −τ
17. The corner moving direction Φ and the moving speed V are calculated by _j cos θ _i )｝ V = k ₂ · τ _i / sin (Φ−θ _i ) (k ₂ is a constant). Correlation processing method. 18. The two images are an image in which the image capturing means captures a target straight line at a certain time and an image in which the image capturing means captures the target straight line a predetermined time after moving. 16. The correlation processing method according to claim 15, wherein the depth to the target straight line is calculated using _P and the moving speed Vs of the image capturing unit. 19. A receptive field image obtained by dividing two temporally different images into receptive fields, respectively, is subjected to a polar transformation process to be mapped on a dual plane of ρ-θ, or the result of the polar transformation process is obtained. further performs a filtering process to map the dual plane, respectively [rho-theta, present time of the mapping data a _t of the dual plane relative to (ρ, θ)
To the mapping data at _{+ 1} (ρ, θ) at the next time
After shifting by a predetermined amount in the axial direction, multiplication is performed, and the shift amount τ is sequentially changed to perform multiplication. The obtained multiplication result is used as a correlation parameter C (ρ, θ, τ). , Θ, τ) are converted to C _PRJ −ρ (θ, τ) projected in the ρ direction, and the (θ, τ)
A correlation processing method in which a point having a maximum amplitude of a sine wave pattern in a plane is extracted by inverse pole conversion, and a moving direction Φ and a moving speed V are calculated using θ and τ that give the maximum amplitude. 20. The two images are an image in which the image capturing unit captures the target graphic at a certain time and an image in which the image capturing unit captures the target graphic a predetermined time after moving. 20. The correlation processing method according to claim 19, wherein a depth up to a target straight line is calculated using the moving speed Vs of the image capturing means.