JPH0644364A - Correlation processing system - Google Patents

Abstract

PURPOSE:To provide the function binocular stereoscopic vision by easily deciding the correspondent parts of plural graphics with correlation processing and further performing the decision with a little processing. CONSTITUTION:First and second image data are mapped to paired planes 17 and 27 by performing pole transformation processing at pole conversion parts 14 and 24. Otherwise, the pole converted results are mapped to paired planes by further performing filter processing at one-dimensional filters 16 and 26 respectively. A correlation processing part 30 calculates a correlative amount (correlation data) with positions (rho,theta) of the mapped data on the paired planes and a gap between the mapped data to perform the correlation processing as element parameters, calculates a point having any feature correlative parameter (such as a maximal value, for example), and measures a parameter (parallax, moving direction or velocity) to specify the relation between the graphical features (such as tangents, for example,) of respective reception field images based on the value of the element parameter to apply the correlative parameter.

Description

Detailed Description of the Invention

[0001]

The present invention relates to a correlation processing method,
In particular, the present invention relates to a correlation processing method for obtaining a relationship between graphic features between two images based on a correlation parameter which is a result of the correlation processing.

[0002]

2. Description of the Related Art In order to control the movement and autonomous operation of a robot, it is necessary to three-dimensionally grasp the environment in which the robot operates. For such stereoscopic grasping, "binocular stereoscopic vision", in which the depth is measured based on the principle of triangulation based on the difference in the appearance (parallax) of the two left and right eyes, and "movement that captures a stereoscopic effect from the motion parallax that occurs when one moves There is "stereoscopic vision". Since we perform stereoscopic vision with two eyes, development of binocular stereoscopic vision has been attempted for a long time. However, in binocular stereoscopic vision, it is necessary to extract corresponding portions from the images captured in the left and right eyes, and it is difficult to extract such corresponding points, which has not yet been put into practical use.

FIG. 57 illustrates the principle of binocular stereoscopic vision, and it is assumed that an object is placed at points A and B on the plane. Only the directions of A and B can be seen by each eye, but the depth of A can be known by the intersection of the line of sight of the left eye looking at A and the line of sight of the right eye looking at A. Is perceived. That is, as shown in FIG. 58, if the distance between both eyes is d and the angles between the lines of sight of the left and right eyes are vertical lines ρ _L and ρ _R , the depth D is expressed by the following equation: D = d / (tan ρ _L + tan ρ _R ).

[0004]

However, there may be other points of intersection of the lines of sight. The line of sight of the left eye looking at A intersects with the line of sight of the right eye looking at B. This intersection is a false point β, and similarly, a false point α occurs, and it is necessary to remove this false point in binocular stereoscopic vision. In our cerebrum, the ability to see shapes (morphoscopic vision) has developed, so we can easily remove false points α and β.
It is difficult to use conventional binocular stereoscopic technology, and it is a problem as a corresponding point problem.

From the above, a first object of the present invention is to provide a correlation processing method capable of accurately determining corresponding points of a plurality of figures with a simple procedure and a small processing amount, and realizing a binocular stereoscopic function. Is. A second object of the present invention is to track a moving object in association with each other from temporally different images,
It is to provide a correlation processing method capable of measuring the moving direction and moving speed. A third object of the present invention is to provide a correlation processing method that can be applied to texture analysis or the like for examining the degree of the same pattern in one screen. A fourth object of the present invention is to provide a correlation processing method that further secures binocular stereoscopic vision, moving object gaze, and texture analysis by performing color correlation processing for each of the three primary colors or three color elements. Is.

A fifth object of the present invention is to obtain corresponding tangent lines (lines / gaps, edges) in a plurality of figures,
Moreover, it is to provide a correlation processing method capable of quantitatively obtaining the position, direction, parallax, and speed. The sixth object of the present invention is suitable for performing accurate filtering, and is suitable for performing "extraction of features that look the same between the left and right eyes" and "tracking of features that are the same as the previous image". To provide a simple correlation processing method. A seventh object of the present invention is to provide a correlation processing method capable of performing binocular stereoscopic vision and tracking of an object with an unclear contour by using a gradual change in brightness or hue as a clue.

[0007]

FIG. 1 shows the principle of the present invention. Reference numerals 14 and 24 are polar conversion units that perform polar conversion processing on the first and second image data, 16 and 26 are one-dimensional filters that perform one-dimensional filter processing on the polar conversion results, and 17 and 27 are filter processing results ( Memorize hyper column image)
A dual plane of θ (hyper column memory), 30 is a correlation processing unit for performing a correlation process between the data mapped to each dual plane.

[0008]

The polar converter 1 for the first and second image data
4 and 24 are used to perform the polar conversion processing and the dual planes 17 and 27
To the dual planes 17 and 27, and the correlation processing unit 30 positions the mapping data on the dual plane ( ρ, θ) and the interval (shift amount) between mapping data to be subjected to correlation processing are used as element parameters to obtain a correlation amount (correlation parameter), and a point having a characteristic correlation parameter (for example, a maximum value) is obtained. Variables (parallax, moving direction, speed, etc.) that define the relationship between the graphic features (for example, tangent lines) of each image are measured based on the values of the element parameters that give different correlation parameters. By the polar transformation, the contour tangent lines (the straight line itself in the case of a straight line) included in the first and second images are transformed into points by reducing the dimension, so that the problem of tangent line correspondence in two dimensions can be solved in one dimension. This can be replaced by a point correspondence problem, and as a result, the corresponding portions of a plurality of figures can be determined easily and accurately with a small amount of processing by the correlation processing, and the binocular stereoscopic function can be realized.

Further, the processing amount can be remarkably reduced by performing polar conversion processing, filter processing, and correlation processing on the receptive field image when one screen is divided into receptive fields which are small areas. Further, when each receptive field image belongs to a different screen captured by two or three cameras, the functions of binocular stereoscopic vision and trinocular stereoscopic vision can be realized, and each receptive field image belongs to a temporally different screen. In this case, it is possible to measure the moving direction and moving speed of the features (lines, corners, etc.) that appear in the receptive field, which makes it possible to move while keeping the target object in the center of the visual field. It can be expected to be applied to running vehicles.

Further, by setting each receptive field image as an image of a different receptive field on the same screen, or by using the same receptive field image,
It is possible to perform a texture analysis or the like to check the degree of the same pattern on one screen. Furthermore, correlation processing between a plurality of receptive field images is performed for each same color or for each color difference signal.
Alternatively, binocular stereoscopic vision, moving object gaze, and texture analysis can be more reliably performed by performing the process for each of the three color elements.

Further, by performing the one-dimensional Gaussian filter processing after the pole conversion or by performing the two-dimensional Gaussian filter processing before the pole conversion processing, it is possible to obtain the corresponding lines or gaps in a plurality of figures. ,
Also, by performing one-dimensional gradient filter processing and one-dimensional Gaussian filter processing after polar conversion, or by performing two-dimensional Gaussian filter processing before polar conversion processing and performing one-dimensional gradient filter processing after polar conversion processing. , It is possible to obtain the corresponding edges in a plurality of figures, and further, the positions and orientations of these figure elements,
The parallax, the moving direction, and the moving speed can be obtained.

Further, when a plurality of receptive field images belong to spatially different screens, the correlation parameter C (ρ, ρ,
θ, σ), or (ρ, θ)
By calculating the correlation parameter C (ρ, θ, σ ₁ , σ ₂ ) of the plane, it is possible to extract the tangent line that moves while changing the azimuth. When a plurality of receptive field images belong to different screens in terms of time, the correlation parameter C (ρ, ρ,
By calculating θ, τ), it is possible to quantitatively determine the position, orientation, and velocity of the tangent line that moves in parallel.
By calculating the correlation parameter C (ρ, θ, τ) in the axial direction, the position, azimuth, azimuth rotation speed, etc. of the tangent line passing through the receptive field center can be quantitatively obtained.
θ) plane correlation parameter C (ρ, θ, τ ₁ , τ ₂ )
By calculating, it is possible to quantitatively measure the position, orientation, moving speed, rotational speed, etc. of the tangent line that moves while changing the direction.

Further, the correlation parameter C (ρ, θ, σ) is projected in the σ axis direction showing the spatial shift amount σ, or in the ρ axis direction showing the tangent position, or in the θ axis direction showing the tangent direction. Or by projecting in any of these two axis directions, the memory capacity for storing the correlation parameter can be reduced by one axis or two axes, and by selecting the projection direction, the tangent position, parallax, It is possible to obtain a desired value in the azimuth and the like. Also, the correlation parameter C
(Ρ, θ, τ) is projected in the τ axis direction showing the temporal shift amount τ, in the ρ axis direction showing the tangent position, or in the θ axis direction showing the tangential direction, or any of these By projecting in the two-axis direction, the memory capacity for storing the correlation parameter can be reduced by one axis or two axes, and by selecting the projection direction, the tangent position, azimuth, parallel movement speed, rotation speed, etc. Of these, the desired value can be obtained.

Further, the receptive field image is subjected to polar conversion processing to be mapped on a dual plane of ρ-θ, and mapping data a (ρ, θ) of the dual plane is set to another dual plane. Accurate filtering can be performed by extracting a set of coordinate values deviated by a predetermined amount in the ρ-axis direction from the data b (ρ, θ) and calculating the sum of the products,
It is suitable to be applied when performing “extraction of features that look the same to the left and right eyes” and “tracking of features that are the same as the previous image”.

Further, the mapping data a (ρ,
other mapping data b (ρ, θ) with respect to θ) is shifted in the ρ-axis direction or the θ-axis direction and then subtracted,
By sequentially changing the shift amount and subtracting, and using the obtained subtraction result as a correlation parameter, binocular stereoscopic vision and tracking of an object with an unclear contour can be used as a clue to a gentle change in brightness or hue. Can be done by

[0016]

【Example】

(A) Outline of the Invention Conventionally, in binocular stereoscopic vision, the corresponding "graphic" on the screen is used.
Was exploring. Therefore, it is difficult for the current computer, which is weak in shape processing, to make a stable correspondence decision. Therefore,
The simplest feature, the "tangent" that makes up the outline of the figure
Correspondence is determined by (the element when the contour is cut with a minute length, in the case of a straight line, the straight line itself). this"
In the "tangent" method, tangents (straight lines) appearing in both eyes can be compared, and the procedure is simple and stable correspondence can be determined.
In the following description, the terms tangent line, contour tangent line, and contour line appear, but when considering the receptive field image in the receptive field, which is a small region described later, these terms are synonymous. Although such a tangent line comparison simplifies the figure to be handled, it is necessary to determine the corresponding tangent line on the two-dimensional screen, which requires a long processing time. By the way, when the input image is pole-transformed and mapped onto a dual plane having the line direction θ and the position ρ as the coordinate axes, the “tangent” in the left and right images is transformed into “points” with reduced dimensions.
Therefore, the problem of tangential line correspondence in two dimensions can be performed in one dimension, and the amount of processing can be greatly reduced.

FIG. 2 illustrates the principle of binocular stereoscopic vision according to the present invention.
It is something. 1. The "tangent" in the space is the input image IM of the left and right eyes_L, IM_R
Then SL_L, SL_RIt appears to be displaced in parallel as shown by. 2. On the other hand, a hyper-colla obtained by polar transformation of the input image
H-CP dual plane HCP_L, HCP_R
Then, the inclination is θ_P"Tangent line" (parallel line) is θ = θ_PΡ axis
Converted to the above "point sequence". 3. Therefore, the input images IM of the left and right eyes_L, IM_RNot parallel to
The "tangent" SL that is visible_L, SL_RIs a dual plane
HCP_L, HCP_RThen the same θ_PTwo points P on the ρ axis of_L, P_R
Is converted to. 4. Distance between the two points σ_PTo find the left and right images IM_L，
IM_R"Tangent" SL at _L, SL_RThe amount of translation of
The parallax is determined. 5. That σ_PAnd the space between both eyes, the spatial depth of the "tangent"
And the binocular stereoscopic function has been realized.
It From the above, the two-dimensional
"Comparison of" tangents "" in "Comparison of" points "on the ρ axis
Dimension processing "can be simplified, and correspondence determination can be performed with a small amount of processing.
I can.

The "polar transformation" and the "receptive field" that will be required later will be described below. The polar transformation includes polar transformation on an arbitrary curved surface such as polar transformation on a spherical surface, polar transformation on a cylinder, or polar transformation on a plane.
FIG. 3 is an explanatory diagram of a spherical map (polar conversion on a spherical surface). As shown in Fig. 3 (a), the spherical map is an arbitrary point P on the sphere.
Is an operation to convert R into a great circle having the pole as its pole (the largest circle on the sphere and corresponding to the equator). According to this spherical map, as shown in FIG. 3 (b), the points P ₁ and P forming the line segment L are formed.
₂ , P _3, ... Great circles R ₁ , R ₂ , with their points P ₁ ′, P ₂ ′, P ₃ ′ ...
As you draw a R ₃ ···, each great circles always intersect at one point S. This intersection S is a unique point corresponding to the line segment L on a one-to-one basis. Then, the longer the line segment L, the greater the number of points that are elements of L, and thus the number of great circles also increases, and the degree of overlap of great circles at S increases. In this way, the line segment is extracted as a point on the spherical surface corresponding to it, and the length of the line segment can also be measured by taking a histogram of the degree of intersection of the great circles at each point.
The geometrical meaning of the point S corresponding to the line segment L is "line segment L
Is a pole whose projection L'is a great circle.

FIG. 4 is an explanatory diagram of the pole conversion on the cylinder. In the polar transformation on a cylinder, a straight line (intersection line of an ellipse) and a point move to each other. That is, the straight line S on the plane PL passing through the origin O
The ellipse ELP obtained by projecting on the cylinder is pole-transformed into a pole P which is an intersection of the normal NL of the plane PL at the origin O and the cylinder, and the pole P is inversely transformed into an ellipse ELP. When this cylinder is cut open parallel to the axis, it can be developed into a plane. On this plane, a straight line (sinusoidal curve) and a point move to each other. From the above, the contour of the image input from the camera is extracted and written in the projection image memory, the contour points are converted into a great circle or a sine curve by the polar conversion unit, and the information of the great circle or the sine curve is written in the mapping memory. Similarly, all contour points are pole-transformed to be transformed into a great circle or a sine curve, and then each cell of the mapping memory is scanned to find the peak position of the count value. It becomes the pole of and the line segment can be extracted. Each cell of the mapping memory is composed of, for example, a counter, and the stored content is increased each time it is written.

Receptive field method By the way, in the method of extracting the line segment by polar-transforming the entire screen, the processing amount increases, and the high speed processing cannot be performed. The reason is that each point in the input image is pole-transformed to extract a line segment,
That is, it is to enlarge the dimension and map it to a great circle or a sine curve on a spherical surface. Input image size is N × N
Then, in order to convert each point into a great circle or a sine curve having a length N by the polar conversion, it is necessary to process N ³ times N times the input image, which is an obstacle to speeding up. Therefore, if the input image is divided into images (receptive field images) for each receptive field, which is a small region, and the polar transformation is performed on each receptive field image, the processing amount is greatly reduced and a small size High-speed polar conversion can be performed with software.

FIG. 5 is an explanatory view of the principle of the receptive field method. Reference numeral 1 is an input memory for storing an N × N size target object image (input image) IMG projected onto a predetermined input plane, and 2 is an input plane. When divided into receptive fields, which are small areas of × m, a receptive field memory that stores the image (receptive field image) in each receptive field, 3 is a polar transformation unit that performs polar transformation on each receptive image, and 4 is polar transformed. It is a dual plane (hyper column memory) of ρ-θ that maps an image (hyper column image).

The N × N size target object image IMG stored in the input memory 1 is divided into receptive fields which are m × m small areas, and each receptive field image is sequentially stored in the receptive field memory 2 and the polar transformation is performed. The unit 3 polarizes each receptive field image to map the output (hyper column image) onto the dual plane 4, and extracts features such as line segments based on the polar conversion output.

The specific procedure of the receptive field method will be described below with reference to the simplest pole transformation, that is, "plane projection input + cylindrical pole transformation". That is, “the pixels in each receptive field obtained by dividing the input image into m × m are cylindrically pole-transformed, and the obtained curve (sinusoidal wave in the cylindrical pole-transformation) is drawn on the dual plane”. The above is the case of “planar projection input + cylindrical polar transformation”, but in the case of “spherical projection input + spherical polar transformation”, the sine curve may be changed to a great circle in the above. The main flow is shown below. (a) Divide the image projected on the sphere into small areas (receptive field) (b) Pole transform each pixel in the receptive field on the sphere (draw a great circle corresponding to the pixel) (c) Receptive field overall Is transformed into a band on the sphere, but the receptive field is generally small, so this band is expanded to a plane. This is a dual plane (hyper column plane) corresponding to each receptive field. The polar transformation of (b) is approximated to "pixel → sine wave", and the transformation between the receptive field and the hypercolumn is "convert each pixel in the receptive field into the corresponding sine curve on the dual plane". . Each axis in the dual plane indicates the position ρ of the line in the receptive field and the direction θ, and the line segments in the image plane (receptive field) are concentrated at the intersections of the sine waves, and the intersection coordinates ρ ₀ , θ ₀ are in the receptive field. Indicates the position and direction of the line segment at.

Explaining in detail with reference to FIG. 6, the receptive field R
The point P in C is mapped to a great circle (straight line) R with an increased dimension by polar transformation. By repeating this, the entire receptive field is mapped to the band BLT on the spherical surface (Fig. 6 (a)). When this band is cut open and developed on a plane (FIG. 6 (b)), a rectangular grid is formed with the line direction θ and the line position ρ as coordinate axes. By this polar transformation, each pixel in the receptive field is multivalued mapped to a plurality of grid points (sinusoidal) on the dual plane. By repeating this, the line segment in the receptive field is extracted as the "intersection point of the sine wave group" in which the point sequence that constitutes it is pole-transformed. That is, the intersection point Q is a concentration of the line segment L (FIG. 6 (c)) in the receptive field, and its θ axis and ρ axis coordinate values θ ₀ , ρ ₀ indicate the direction and position of the line segment in the receptive field. become.

In the receptive field method, it is only necessary to perform the polar transformation on the image in the m × m receptive field in this way. Therefore, in the case of the polar transformation on the spherical surface, each pixel in the receptive field has a length m. All you have to do is draw a big circle, and the processing amount becomes m × N ² ,
Compared with the conventional example that draws a large circle of N (the processing amount of N ³ is required), it is significantly reduced and the hardware can be downsized. For details of the polar conversion, the applicant of the present application has already filed Japanese Patent Application No. 3-327723 (Title of the invention: image processing method,
Filing date: December 11, 1991).

(B) Basic Configuration of the Present Invention In summary, the basic configuration of binocular stereoscopic vision is input image → polar conversion → one-dimensional correlation processing. As described above, the polar transformation is an operation that transforms a "tangent" into a "point" in order to perform tangential comparison in one dimension. One-dimensional correlation processing is performed from a "point sequence" on the ρ axis with the same θ. This is an operation for extracting the corresponding "point". The mapping data on the dual planes of both eyes that have been polar-transformed are respectively denoted by L (ρ,
θ), R (ρ, θ), interval between mapping data (shift amount)
Where σ is σ, the correlation amount (referred to as correlation parameter) is calculated by the following expression C (ρ, θ, σ) = L (ρ, θ) · R (ρ + σ, θ) (1). Then, when the values (ρ _P , θ _P , σ _P ) of the element parameters ρ, θ, σ that maximize the correlation amount C (ρ, θ, σ) are obtained, the corresponding values are obtained based on the values of the element parameters. The tangent line is determined, and the position, orientation, and parallax of the tangent line are quantitatively obtained. The correlation parameter can be calculated by the symmetric correlation calculation of the following formula in addition to the asymmetric correlation calculation of formula (1). C (ρ, θ, σ) = L (ρ−σ, θ) · R (ρ + σ, θ) (1) ′ However, in the following, the correlation parameter is calculated by the asymmetrical correlation calculation of equation (1). As described below. Further, hereinafter, the terms correlation amount, correlation parameter, and correlation result are used in a mixed manner, but they are synonyms.

(C) Embodiment of the present invention FIG. 7 is a diagram showing an embodiment of the present invention for obtaining the position, orientation and parallax of corresponding tangent lines included in two images by input image → receptive field division → polar transformation → one-dimensional correlation processing. It is an example block diagram. In the figure,
Reference numerals 11 and 21 denote input memories for storing input images IM _L and IM _R for the left and right eyes. Reference numerals 12 and 22 denote images in the receptive field (receptive field) when the input plane is divided into receptive fields which are small areas of m × m. Receptive field clipping circuit for clipping and outputting (image) 1
3, 23 are receptive field memories that store images (receptive field images) in each receptive field, 14 and 24 are polar conversion circuits that perform polar conversion on each receptive field image, and 15 and 25 are dual polar planes (hyper planes). Image on column plane) (Hyper column image)
It is a polar transformation hyper column memory that stores The polar conversion hypercolumn memories 15 and 25 are composed of ρmax in the ρ direction, θmax in the θ direction, and a total of ρmax × θmax storage areas (hypercolumn cells).

Reference numerals 16 and 26 denote one-dimensional filter circuits for performing one-dimensional filter processing on the hyper-column image obtained by the polar transformation. Reference numerals 17 and 27 denote dual planes (hyper-column memory) for storing the filtered hyper-column image. ) And 30 are correlation processing units. In the correlation processing unit 30, 31 is a correlation calculation unit that performs one-dimensional correlation calculation according to the equation (1), and 32 is a correlation value (correlation parameter) that stores ρ−
A correlation parameter storage unit having a dimension of σ−θ, 33 is a peak detection unit that scans the storage data of the correlation parameter storage unit and detects the maximum points (ρ _P , θ _P , σ _P ).

The polar conversion circuits 14 and 24 polar-convert each pixel in the receptive field, that is, convert the pixel into a corresponding great circle and store it in the polar-conversion hyper-column memories 15 and 25.
Actually, the polar conversion is approximated to “pixel → sine wave”, and each pixel in the receptive field is converted into a corresponding sine curve on the hyper column and stored in the polar conversion hyper column memories 15 and 25.

The one-dimensional filter circuits 16 and 26 are for extracting sharp lines, edges and gaps with a small processing amount. In order to perform edge enhancement and feature extraction, image data is usually subjected to a two-dimensional convolution filter and then subjected to polar transformation. However, according to such a convolution method, when the filter size is a, it is necessary to process a ² for each input point, and the processing amount increases as the filter size increases. However, "two-dimensional convolution filter + pole conversion" is equivalent to "performing one-dimensional filter processing in the ρ direction after pole conversion". Therefore, in the embodiment of FIG. 7, a one-dimensional filter is applied after pole conversion. . By doing so, the processing amount is as small as a for each input point because it is a one-dimensional filter, and the processing amount is reduced to about 2 / a as compared with the convolution method.

Correlation Calculation The correlation calculation unit 31 maps the mapping data L (ρ, θ) on the dual plane 17 of the left eye and the dual plane of the right eye σ shifted from the mapping data L (ρ, θ) in the ρ-axis direction. 27 and the mapping data R (ρ + σ, θ) in 27 are stored in the correlation parameter storage unit, and thereafter, the shift amount σ is sequentially set to 0.
To σmax, ρ to 0 to ρmax (receptive field width), and θ to 0
The multiplication result is changed up to θmax (receptive field width), and each multiplication result is stored in the correlation parameter storage unit 32.

FIG. 8 is a flow chart of the correlation calculation processing, where 0 → θ, 0 → ρ, and 0 → ρ are set in the correlation calculation (steps 101 to 103). Then, the correlation parameter is calculated by the equation (1) and stored in the correlation parameter storage unit 32.
(Steps 104 and 105). Then, σ is incremented, and it is determined whether σ> σmax (steps 106 and 107). If σ ≦ σmax, step 10
Returning to step 4, the subsequent processing is repeated. When σ> σmax, ρ is incremented and it is determined whether ρ> ρmax (steps 108 and 109), and ρ ≦ ρmax
If so, the process returns to step 103 to repeat the subsequent processing. If ρ> ρmax, θ is incremented and it is determined whether θ> θmax (steps 110 and 11).
1) If θ ≦ θmax, the process returns to step 102 and the subsequent processes are repeated. If θ> θmax, the correlation calculation process ends.

Pole conversion circuit The pole conversion circuit 14 (similarly to the pole conversion circuit 24) reads the receptive field image (amplitude) from the address of the receptive field memory 13 one pixel at a time as shown in FIG. A read control unit 14a that outputs a value, a polar conversion unit 14b that performs polar conversion (conversion from pixel address to sine wave address) for each pixel,
A write control unit 14c for writing the read amplitudes into a plurality of storage positions of the polar conversion hyper column memory 15 indicated by the sine wave address obtained by the polar conversion is provided.

The pole conversion unit 14b converts the pixel address of the receptive field memory 13 to a plurality of addresses of the polar conversion hyper column memory 15 and the address conversion table memory 14b-1 and the receptive field memory 13 to polar conversion hyper column memory. An address conversion circuit 14b-2 for converting into 15 addresses is provided. The address conversion table stored in the address conversion table memory 14b-1 is for polarizing a point in the receptive field image into a sine wave on the hyper column plane, and an address corresponding to the point in the receptive field image is This is a conversion to a large number of addresses on the polar conversion hyper column memory where each point of the point sequence forming the sine wave is located.

The read control unit 14a uses the receptive field memory 1
The amplitude data is read from the first address of No. 3, and the amplitude data and the address data (first address) are input to the pole conversion unit 14b. The pole converter 14b is the first address (pixel) of the receptive field memory.
Is converted into a large number of addresses in the sine wave point sequence of the polar conversion hyper column memory 15, and the addresses and amplitudes are output. The write control unit 14c adds the amplitude data input from the pole conversion unit 14b to the contents (initial value is 0) of each address of the pole conversion hyper column memory 15 which is also input and writes the added data to the address. After that, sequentially, receptive field memory 14
If the above process is performed for all the addresses, the polar transformation for the receptive field image ends.

One-Dimensional Filter Circuit The one-dimensional filter circuit 16 (similarly to the one-dimensional filter circuit 26), as shown in FIG. A read control unit 16a that reads and outputs the amplitude, a one-dimensional filter unit 16b that performs a one-dimensional filtering process on each read amplitude, and a one-dimensional filtering process result stored in a dual plane (hyper column memory) 17. Write control unit 16c for writing in the area
Is equipped with.

The one-dimensional filter section 16b has, for example, a FIR digital filter configuration, and has a one-dimensional filter memory 16b-1 and a product-sum circuit 16b-2. The one-dimensional filter memory 16b-1 discretely stores one-dimensional filter characteristic values (coefficients) with the horizontal axis as the ρ axis. That is, ρ
The one-dimensional filter characteristic value is stored for each position of −ρmax / 2 to ρmax / 2 in the direction, and the one-dimensional filter unit 16b is configured to use the one-dimensional first-order differential filter and the one-dimensional second-order differential filter according to the characteristic value. Can be The width and the value of the filter are appropriately determined as needed. Sum of products circuit 16
b-2 is ρmax read from the polar conversion hypercolumn 15
And the corresponding ρmax characteristic values stored in the one-dimensional filter memory 16b-1 are respectively multiplied, and the sum (amplitude) of the multiplication result is output. The sum of products is calculated and the calculation result is output.

For example, first, all addresses (ρ
ρ max amplitudes are read out from max addresses). Then,
Filter characteristic ρ in one-dimensional filter memory 16b-1
The position of 0 is associated with the ρmax leftmost addresses A _0i . Thereafter, the corresponding amplitude value and characteristic value are multiplied, and the sum Σ _{0i of the} multiplication results is calculated.
Write at address A _0i .

When the above processing is completed, the filter characteristic value
The position of ρ = 0 is the second address A from the left._1iCorresponding to
Multiply the corresponding amplitude value and characteristic value, and then multiply
Sum of calculation results Σ_1iWrite controller 16c
-Address A of column memory 17 _1iWrite in. After that, the same
And perform the appropriate processing to set the position of the filter characteristic value ρ = 0 to the rightmost end.
Ground A_{max, i}If the above process is completed in response to
One-dimensional fill by repeating the above product-sum operation as + 1 → i
Perform the tarring process.

Simulation Results FIGS. 11 to 13 are explanatory diagrams of simulation results when a one-dimensional Gaussian filter is used as the one-dimensional filters 16 and 26 in FIG. 7 for extracting lines. The image IM _L , I is obtained by the polar transformation and the one-dimensional Gaussian filter processing.
The "line" in M _R is converted into a "point" in the dual plane having the line position ρ and the line direction θ as coordinate axes. Figure 11
(a), the circular part C _L in (b), C _R are the left and right eyes receptive field, the image of the circular is receptive field images caught on the right and left eyes, cross-hairs of the portion surrounded by a scene in a square in the plant The figure is shown.
All the data show the signal strength by contour lines.

FIGS. 12 (a) and 12 (b) are hypercolumn images on the dual plane of ρ-θ, which have been subjected to the above-mentioned “polar transformation + one-dimensional Gaussian filter” processing, and which form two crosshair lines. Are two points P _L1 on the ρ axis of 90 ⁰ and 180 ⁰ , respectively.
It has been converted into P _L2 ; P _R1 and P _R2 . Incidentally, those peaks are sharpened by one-dimensional filtering, and as a side effect thereof, negative sub-peaks P _L1 ′, P _L2 ′ on both sides are obtained.
With P _R1 ′ and P _R2 ′. This subpeak is very effective in sharpening the next correlation process.

FIG. 13 shows the result of obtaining the correlation amount (correlation parameter) C (ρ, θ, σ) of the equation (1) from the dual plane data of the left and right eyes, with the horizontal axis representing ρ (position) and the vertical axis representing. σ (parallax),
It is developed on a ρ-σ-θ plane having a depth of θ (azimuth) and the correlation values are shown by contour lines. For convenience of explanation, the ρ-θ plane of θ = 90 ⁰ is shown at the top. . When the correlation parameter C (ρ, θ, σ) reaches a peak (maximum) (ρ _P , θ _P , σ _P ), the correspondence between the crosshairs in the left and right eyes is quantitatively obtained, and ρ _P , Θ _P and σ _P respectively indicate the position, direction and parallax of the line. When the two lines constituting the cross line vertical bar (theta = 90 ⁰⁾ is try seeking parallax visible left and right eyes, a 1 ^0, calculate the distance to the vertical bars the distance between both eyes as a 6cm Then, it becomes 360 cm, and it can be confirmed that the binocular stereoscopic vision was accurately performed.

Evaluation As described above, the following effects can be obtained by the basic processing (polar conversion + one-dimensional correlation processing). Effect of tangential comparison of features of figure It is difficult to find the same figure from left and right images because current image processing technology is weak in processing of "shape". Therefore, if the correspondence is determined by the tangent line that constitutes the figure contour, the correspondence is determined by a simple procedure because the simplest straight-line figure is obtained.

Effect of Processing Two-Dimensional Correspondence of Tangent Lines in One Dimension Two-dimensional processing is required to determine the correspondence of tangent lines in the left and right images. However, when the polar transformation process is performed before the correspondence process and mapped onto the dual plane, the parallel line group is transformed into a sequence of points on the ρ-axis. The problem is reduced to a large amount, the processing amount is greatly reduced, and the stability of the response is improved.

Effect of advanced filtering After the polar transformation, the advanced one-dimensional filtering processing already applied for can be performed, and as a result, the polar transformation-> one-dimensional filter-> one-dimensional correlation processing ensures reliable operation. Correspondence can be determined, and various correlation parameters described later can be precisely measured. The types and functions of one-dimensional filtering are as follows. Edge correspondence determination by odd function filtering: Edges (brightness boundaries), which are the most important feature of an image but are difficult to extract with a conventional two-dimensional convolution filter, can be extracted, and edge correspondence determination is possible. This is because the odd function filter has become possible by performing pole transformation in advance.・ Stable correspondence determination by multi-filtering: A large number of filters with different widths can be applied simultaneously to pole-transformed data (multi-filter). This allows
"Blur feature" mixed in the image, which is difficult with the conventional method
And "fine features" can be extracted at the same time, and stable correspondence can be determined. High-speed processing by skeleton filtering: The most efficient skeleton filter can be applied to pole-transformed data. As a result, the odd function filter and the multi-filter can be realized at high speed. The details of the odd function filter, the multi-filter, the skeleton filter, etc. are described in Japanese Patent Application No.
See No. 7722 (Title of Invention: Image Processing Method, Application Date: December 11, 1991).

(D) Expansion of the Invention For the sake of clarity, the principle and effect of correlation filtering have been described above by taking binocular stereoscopic vision as an example. However, the essence is to judge by measuring the degree of correlation of figures between images, and not only the correspondence between both eyes but also the use of tracking moving objects by associating them with temporally different images, and It can be widely used for texture analysis to check the degree of the same pattern in one screen. Hereinafter, the extension of the correlation filtering of the present invention will be described in detail.

(A) Basic Flow FIG. 14 is a basic flow of correlation filtering, in which polar conversion processing is performed on the input data A and B (steps 51a, 5).
1b), the one-dimensional filtering process is performed on the polar transformation result, and the result is mapped to the correlation dual plane (steps 52a, 52).
b), Correlation processing is performed on the data mapped on each dual plane (step 53). It is desirable to divide the input data into receptive fields, but it is not always necessary to divide into receptive fields. Further, the one-dimensional filter processing may be performed as necessary, and the type thereof may be the filter disclosed in the above-mentioned Japanese Patent Application No. 3-327722. Further, instead of polar transformation, reciprocal transformation, orthogonal complementary space, etc. can be applied. Further, the input data is not limited to the image from the camera, for example, the data obtained by polar conversion of the camera image may be input, and the input data is not limited to two,
There may be three or more.

In the following, description will be made assuming that receptive field division and one-dimensional filter processing are performed. (b) Correlation action screen Action screen, that is, input data A in the basic flow,
B has the following cases, and the respective unique correlation parameters can be measured. (b-1) Correlation between different images Correlation between spatially different screens Input data A and B are image data of screens that are spatially different, for example, screens captured by different cameras, and are correlated to this input data. This is a case where filtering is performed, and the feature correlation parameter between the screens spatially different can be extracted. Binocular stereoscopic vision and tricular stereoscopic vision are particularly useful examples.

FIG. 15 is a flow chart of correlation filtering between spatially different screens. (A) is a screen input from two cameras, (b) is a screen input from three cameras. Is.・ In the case of a screen input from two cameras (Fig. 15)
(a)) When there are two cameras, the left camera CML and the right camera CMR
The image data IM _L , IM _R for one screen captured in step S1 is divided into receptive field images (steps 50a and 50b), and each receptive field image I is divided.
A polar conversion process is performed on M _L ′ and IM _R ′ (step 51a,
51b), performs one-dimensional filtering processing on the polar transformation result and maps it on a dual plane (steps 52a, 52).
b), correlation processing is performed between the data mapped to each dual plane (step 53). The above-mentioned binocular stereoscopic vision corresponds to this.

In the case of a screen input from three cameras (see FIG. 15)
(b)) Since there are two viewing angles in binocular stereoscopic vision, the correspondence decision may be incorrect. Even in us, the illusion in surprised houses is due to two eyes. From an engineering point of view, three eyes are possible, and with one more camera, the illusion decreases from three angles. Placing the three cameras at the vertices of the triangle further reduces the illusion in all directions, which is a further improvement. When there are three cameras, the image data IM _L , IM _R , and IM _C for one screen captured by the left camera CML, the central camera CMC, and the right camera CMR are divided into receptive field images (steps 50a to 50c), and each receptive field image is received. Field image IM _L ′, I
A polar conversion process is performed on M _R ′ and IM _C ′ (steps 51a to
51c), performs a one-dimensional filtering process on the polar conversion result and maps it to a dual plane (steps 52a to 52).
c), correlation processing is performed between the data mapped to each dual plane (step 53).

Correlation Between Time-Different Screens Input data A and B in the basic flow are image data of time-different screens, for example, screens taken while moving the camera, and the input data is subjected to correlation filtering. This is the case. By this correlation filtering,
For example, it is possible to measure the moving direction and moving speed of the features (corner, contour line, etc.) of the object in the receptive field, which makes it possible to move while keeping the object in the center of the field of view. It can be expected to be applied to running vehicles. Even in the primary visual cortex of the cerebrum, there are cells that detect the direction of movement of the tangent line reflected in the receptive field, and signals are sent to the eye movement control section (upper colliculus or lateral geniculate body) to view the object. In addition to capturing it in the center, it also sends signals to the higher visual cortex in parallel to three-dimensionally measure the movement of the scene in the visual field.

FIG. 16 is a flow chart of correlation filtering between screens which are temporally different. The image data IM _{L for} one screen imaged by the left camera CM is not delayed, while the image data IM _{R for} one screen imaged by the right camera CMR is delayed by the delay unit (step 49) and is not delayed. I
M _L and the delayed image data IM _R ′ are divided into receptive field images (steps 50a and 50a ′), and each receptive field image IM is divided.
_A polar conversion process is performed on _L and IM _R ′ (steps 51a, 51
a ′), the polar transformation result is subjected to one-dimensional filtering processing and mapped to a dual plane (steps 52a, 52).
a '), correlation processing is performed between the data mapped to each dual plane (step 53). That is, the input image data is divided into one that is delayed in time and one that is not delayed, and these are input to the basic correlation filter of FIG. 14, and a temporal correlation parameter is obtained. Therefore, the simulation can be performed in the same manner as the above-described binocular stereoscopic simulation except for the delay unit.

Correlation between screens temporally and spatially different Correlation filtering is applied to screens taken while moving a plurality of cameras. In the case of two cameras, binocular stereoscopic spatial correlation filter Is equivalent to a combination of a time correlation filter and FIG. 17 is a flow chart of such correlation filtering between screens that differ temporally and spatially. The image data IM _L for one screen captured by the left camera CML is delayed by the delay unit (step 49a), and the image is not delayed. The data IM _L and the delayed image data IM _L ′ are divided into receptive field images (steps 50a, 50).
a ′), the receptive field images IM _L , IM _L ′ are subjected to polar conversion processing (steps 51a and 51a ′), and the polar conversion results are subjected to one-dimensional filtering processing to be mapped to dual planes (steps 52a, 52a, 51a, 51a ′). 52a '). In parallel with the above, one screen of image data I captured by the right camera CMR
The delay unit delays M _R (step 49b) and divides the non-delayed image data IM _R and the delayed image data IM _R ′ into receptive field images (steps 50b and 50b ′), and receptive field images IM _R , The IM _R ′ is subjected to polar conversion processing (steps 51b and 51b ′), and the polar conversion result is subjected to one-dimensional filtering processing to be mapped to dual planes (steps 52b and 52b ′). Then, correlation processing is performed between the data mapped to each dual plane (step 5).
3). By tracking the object while extracting the parallax by binocular stereoscopic vision by the correlation filtering between the screens that are temporally and spatially different, a human-like composite function becomes possible. The existence of such a filter that fuses time and space is known in psychology as a "Spatiotempora filter", but this item was formulated for the first time in order to contribute to engineering.

(B-2) Correlation between the same images The correlation filtering between different screens has been described above, but the correlation within the same screen is also effective. Correlation between receptive fields FIG. 18 is a flow chart of correlation filtering between receptive fields on the same screen. An image for one screen captured by the camera CM is stored in the input memory IMM, and is cut out for each of the receptive fields A, B, ...
_{M A, IM B, ·· IM} N and is (step 48), each receptive field image _{IM A, IM B, ·· IM} N pole conversion processing on (step 51A～51n), primary to each pole conversion result The original filtering process is performed to map each to the dual plane (steps 52a to 52n), and the correlation process is performed to the data mapped to each dual plane (step 53). In the case of such a correlation between different receptive fields on the same screen,
Image features receptive field image IM _A receptive field image IM _B, ·· IM _N
Can be compared accurately with the features of the above, and texture analysis such as the texture of the image can be performed. The processing is performed by changing the right and left eyes of binocular stereoscopic vision to respective receptive fields, and can be executed in the same manner as the above-described simulation.

Correlation Within Receptive Field The correlation between receptive fields has been described above, but the correlation within the same receptive field on the same screen is also possible. FIG. 19 is a flow chart of correlation filtering in the same receptive field on the same screen, which is the case of correlation filtering between ρ axes. The receptive field image is pole-transformed and stored in a dual plane (hyper column memory) HCM of ρ−θ, and the clipping control unit controls the ρ axis A,
Each ρ-axis B, ..., P-axis N is cut out (step 4
8 '), polar transformation processing is performed on each hypercolumn image of each ρ-axis A to N (steps 51a' to 51n '), one-dimensional filtering processing is performed on each polar transformation result, and each is mapped to a dual plane (step 52a). '~ 52n'), correlation processing is performed between the data mapped to each dual plane (step 53). This correlation filter enables a more detailed texture analysis in the receptive field, and the binocular stereoscopic simulation can be similarly performed for the same reason as in the case. Although the correlation between the ρ axes has been described, the correlation in the θ axis direction is also possible, and the correlation in the (ρ, θ) plane is also effective as a two-dimensional correlation.

(B-3) Correlation between different color images In the above, the correlation filtering of the intensity signal such as the brightness of the input image has been described. If the input data is used as the color information and the correlation filtering of FIG. 14 is performed. , It is possible to detect subtle differences in color and perform more detailed filtering. The color information to be input can be color information that has been subjected to various processes depending on the problem, but the basic color information is as follows. Correlation in Three Primary Colors The correlation between the three primary colors is the most basic color correlation, and FIG. 20 is a flow chart of color correlation filtering in the three primary colors between different images. Color images for one screen captured by the left camera CML are respectively filtered with red, blue and green filters (RFL, BFL, GF).
Red through L), separated blue, green image, after which each of red, blue, green receptive field image IM _R, IM _B, divided into IM _G (step 60 a to 60 c), each receptive field image IM _R , I
M _B, subjected to polar transformation processing IM _G (step 61a~61
c), the polar transformation result is subjected to one-dimensional filtering processing and mapped to a dual plane (steps 62a to 62).
c). Similarly, although not shown, one screen of color image captured by the right camera is passed through the red, blue, and green filters to produce red,
It is separated into blue and green images, and then divided into red, blue, and green receptive field images, and each receptive field image is subjected to polar conversion processing, and the polar conversion result is subjected to one-dimensional filtering processing to obtain a dual plane. Map to. Then, the red correlation processing, the blue correlation processing, and the green correlation processing are performed on the data mapped to the dual planes for red, blue, and green (steps 63a to 63).
c). By this color correlation filtering, the correlation parameter of each of the three primary colors can be measured.

Correlation in Three Elements of Color (Lightness, Saturation, Hue) Next, the basic color correlation is an amount by which a human can visually perceive the three primary physical colors, that is, brightness (brightness), vividness ( This is a method of performing correlation filtering after converting into saturation and color condition (hue). This filter can measure psychologically organized color correlation parameters. Figure 21
Is a flow of color correlation filtering with three elements between different images. A color image for one screen captured by the left camera CML is separated into red, blue, and green images by the red, blue, and green filters, respectively, and then the color conversion unit converts the color information of the three primary colors into lightness, saturation, and hue. It is converted into three color elements IM _V , IM _S , and IM _H (step 59) and divided into receptive field images IM _V ′, IM _S ′, and IM _H ′ of lightness, saturation, and hue (step 60).
a ′ to 60c ′), each receptive field image IM _V ′, IM _S ′, IM
Subjected to polar transformation process in _H '(step 61a'~61
c '), the one-dimensional filtering process is applied to the polar conversion result, and the result is mapped to the dual plane (steps 62a'-62).
c '). Similarly, although not shown, a color image for one screen taken by the right camera is red, blue, and green with red, blue, and green filters, respectively.
After separating it into a green image, the color conversion unit converts the color information of the three primary colors into three elements of lightness, saturation, and hue, and divides them into lightness, saturation, and hue receptive field images. A pole transformation process is performed on the field image, and a one-dimensional filtering process is performed on the pole transformation result to map it on a dual plane. Then, the lightness correlation processing, the saturation correlation processing, and the hue correlation processing are performed between the data mapped to the dual planes for the lightness, the saturation, and the hues (steps 63a 'to 63c').

Correlation with Color Difference Signal In order to accurately measure the correlation parameter at the color boundary, such as the transition between red and green, it is effective to perform correlation processing on the color difference signal. In particular, it is effective to use the difference signals of the three primary colors in the relationship of complementary colors. FIG. 22 is a flow chart of color correlation filtering with a color difference signal between different images. Color images for one screen taken with the left camera CML are red,
Separated into red, blue and green images with blue and green filters, and then
The difference signals of red and green, green and blue, and blue and red are calculated (steps 58a to 58c), and the receptive field image I of each difference signal is calculated.
It is divided into M _RG , IM _GB , and IM _BR , and each receptive field image I
The polar conversion process is performed on M _RG , IM _GB , and IM _BR (steps 61a ″ to 61c ″), and the polar conversion result is subjected to one-dimensional filtering process to be mapped on the dual plane (steps 62a ″ to 62c ″). Similarly, although not shown, one screen of color image captured by the right camera is separated into red, blue, and green images by red, blue, and green filters, respectively.
The difference signals of green and blue, and blue and red are respectively calculated, the receptive field image of each difference signal is subjected to polar conversion processing, and the polar conversion result is subjected to one-dimensional filtering processing to be mapped on the dual plane. Then, the red-green correlation processing, the green-blue correlation processing, and the blue-red correlation processing are performed between the data mapped to the dual planes by the difference signals of the corresponding colors (steps 63a "to 63c").
Instead of taking the difference, the difference after division or logarithmic conversion is also effective. Further, the difference or division may be performed after the polar conversion or the one-dimensional filter. Although the color correlation between different images has been described in the above color correlation, the color correlation method can be combined with the various correlation methods described in (b-1) and (b-2) above to enable binocular stereoscopic vision and movement. Correlation filtering such as object gaze and texture analysis can be made more reliable.

(C) Direction of Correlation In the example of binocular stereoscopic vision (see FIG. 7), the azimuth of the dual plane
Correlation is performed by combining data in the ρ-axis direction where θ is the same.
However, it is not necessary to limit to the ρ axis, and even within the ρ axis
Can be generalized. (c-1) Correlation in the ρ-axis direction Correlation in the same ρ-axis with the same θ This is the case of the above-mentioned binocular stereoscopic vision, etc.
The meaning of "same ρ-axis correlation" is as shown in FIG.
To accurately measure the amount of deviation of parallel lines in the
It Fig. 23 shows the flow of ρ-axis correlation filtering with the same θ.
It is a diagram, and each input data A, B is divided into receptive field images
(Steps 50a, 50b), polar conversion processing for each receptive field image
(Steps 51a and 51b), and
Dimensional filtering is applied to map to dual planes.
(Steps 52a and 52b), from each dual plane
θ₀, Θ₁, ... Select the data in the ρ-axis direction for each (step
54a ₀, 54a₁, ...; 54b₀, 54b₁・ ・ ・
・), Correlation processing is performed between data with the same θ value (step
P53₀, 53₁, ...). Note that FIG. 23 is parallel for each θ value.
In the case of performing the correlation calculation on the
It is also possible to perform correlation calculation of each θ value sequentially in that case.
Is the same as the flow chart of FIG.

Correlation of ρ-axis with different θ This correlation is filtering for measuring a correlation parameter between tangent lines with different directions. This filtering can extract not only the correlation of parallel lines of but also the tangents that move while changing the direction, and can be expected to be applied to automobiles running on rough roads. FIG. 24 is a flow chart of ρ-axis correlation filtering with different θ. The input data A and B are divided into receptive field images (steps 50a and 50b), and polar conversion processing is performed on each receptive field image (steps 51a and 51b).
The polar conversion result is subjected to one-dimensional filtering processing to be mapped to a dual plane (steps 52a and 52b), and θ _i , θ _{i + 1} , ...; θ _j , θ _{j + 1} , ... Data in the ρ-axis direction is selected (steps 54a ₀ , 54
_{a 1, ··; 54b 0,} 54b 1, ··), different θ values subjected to correlation processing between data (step 53 ₀ ', 5
3 ₁ ', ...).

(C-1) Correlation in the θ-axis direction The meaning of the correlation parameter between θs having the same ρ is explained with reference to FIG. 25. In the θ correlation with ρ = 0 (FIG. 25 (a)), the center of the receptive field is The rotation amount of the tangent line SL passing through C can be extracted. Also, ρ ≠
With the θ axis correlation of 0, the rotational movement amount of the tangent line SL tangent to the circle CIR having the radius ρ sharing the receptive field center C can be detected. FIG. 26 is a flowchart of the θ-axis correlation filtering with the same ρ. The input data A and B are divided into receptive field images (steps 50a and 50b), and each receptive field image is subjected to polar conversion processing (step 51a). , 51b), performs a one-dimensional filtering process on the polar transformation result and maps it to a dual plane (steps 52a, 52b), and outputs ρ ₀ , ρ ₁ , from each dual plane.
.. Select data for each ρ axis direction (step 55
_{a 0, 55a 1, ··;} 55b 0, 55b 1 ··), subjected to a correlation processing between the data of the same ρ value (step 53 _0, 5
3 ₁ ...). Although FIG. 26 shows a case where the correlation calculation is performed in parallel for each ρ value, one correlation calculation unit may be provided to sequentially perform the correlation calculation for each ρ value.

(C-3) Correlation in (ρ, θ) plane In the above, one-dimensional correlation in the ρ-axis or θ-axis direction is explained, but two-dimensional correlation in the (ρ, θ) plane is also possible. . With this correlation, it is possible to accurately perform the correlation of the tangent line that moves while changing the azimuth. FIG. 27 is a flowchart of correlation filtering on the (ρ, θ) plane. Each input data A, B is divided into receptive field images (step 50a,
50b), and polar conversion processing is applied to each receptive field image (step 5).
1a, 51b), one-dimensional filtering processing is performed on the polar conversion result, and the result is mapped to a dual plane (steps 52a, 52a, 51b).
52b), two-dimensional correlation processing is performed on each dual plane (step 53). The two-dimensional correlation amount C (ρ, θ,
σ ₁ , σ ₂ ) is the data on each dual plane a (ρ,
θ), b (ρ, θ), and the shift amounts in the ρ-axis direction and the θ-axis direction of the dual plane data are σ ₁ and σ ₂ , the following equation C (ρ, θ, σ ₁ , σ ₂ ) = a (ρ, θ) · b (ρ + σ ₁ , θ + σ ₂ ) (2)

(D) Correlation Parameter In the example of binocular stereoscopic vision, the method of finding the corresponding tangent line by the correlation amount (correlation parameter) of the equation (1) has been described in order to simplify the description. Various correlation parameters can be defined, and characteristic correlation processing can be performed for each. (d-1) Basic correlation parameter The basic correlation parameter is generally C
It is expressed in three dimensions such as (ρ, θ, σ), and the three-dimensional space of ρ, θ, σ is as shown in FIG. In the figure, τ is the shift amount (moving speed) between data that is introduced in the correlation between screens that differ in time, which will be described later. Correlation parameter C
The point in the three-dimensional space where (ρ, θ, σ) is maximum (ρ _P ,
θ _P , σ _P ) is obtained, correspondence determination of binocular stereoscopic vision, correspondence determination of moving tangents, and the like are performed. From each coordinate value (value of element parameter) that gives the maximum point, the position of the tangent, Azimuth, parallax or speed can be measured quantitatively.
In the two-dimensional correlation of the (ρ, θ) plane, the correlation parameter is the parameter C (ρ, θ, σ ₁ , σ ₂ ) of the four-dimensional space.
Becomes

Spatial Correlation Parameter When the action screen of the correlation is a spatially different screen or receptive field (cases (b-1) and (b-2) described above), the following maximum points of the correlation parameters From this, it is possible to measure the correlation amount indicating the degree of correspondence such as parallax of the corresponding tangent line. (1) Correlation parameter C (ρ, θ, σ) in the ρ direction: This is a parameter indicating the degree of spatial correlation of the parallel line group. The correlation parameter of the binocular stereoscopic vision described above is an example of this, and
When ρ _P , θ _P , and σ _{P at} which θ, σ) becomes maximum, the corresponding tangent line is determined from the parallel line group, and the position, orientation, and parallax of the tangent line can be quantitatively measured. As the correlation parameter, the data on each dual plane is a (ρ, θ), b
(Ρ, θ), where σ is the shift amount in the ρ-axis direction, it is given by the following formula C (ρ, θ, σ) = a (ρ, θ) · b (ρ + σ, θ).

(2) Correlation parameter in the θ direction Cθ (ρ,
θ, σ): A parameter indicating the degree of spatial correlation of a tangent group tangent to a circle having a radius ρ sharing the center of the receptive field. ρ = 0
In the θ correlation processing of, Cθ (ρ, θ, σ) is the maximum ρ
_The corresponding tangent line is determined from the radiation groups from _P , θ _P , and σ _P , and the position, azimuth, and azimuth change amount of the tangent line can be quantitatively measured. The correlation parameter is given by the following equation C θ (ρ, θ, σ) = a (ρ, θ) · b (ρ, θ + σ) (3), where σ is the shift amount in the θ-axis direction.

(3) Correlation parameters C (ρ, θ, σ ₁ , σ ₂ ) of the (ρ, θ) plane: parameters showing the spatial correlation of the tangent group moving while the direction changes, and the dual plane The degree of two-dimensional correlation can be evaluated in more detail than the one-dimensional correlation parameter described above. The correlation parameter is expressed by equation (2), that is, the following equation C (ρ, θ, σ ₁ , σ ₂ ) = a (ρ, θ) · b (ρ + σ ₁ ,
given by θ + σ ₂ ).

Temporal Correlation Parameter When the action screen of the correlation is a screen that is temporally different or a receptive field (in the case of (b-1) etc.), the moving tangent of It can measure the moving direction and moving speed. (1) Correlation parameter C (ρ, θ, τ) in the ρ-axis direction: This is a parameter indicating the degree of temporal correlation of parallel line groups. An example to be described later, that is, a correlation parameter in the case of tracking a moving tangent line, is an example of this, and ρ _P , θ _P , τ _P where C (ρ, θ, τ) has a maximum value, position,
Azimuth and speed can be measured quantitatively. The correlation parameter is given by the following equation C (ρ, θ, τ) = a (ρ, θ) · b (ρ + τ, θ) (4) where τ is the moving speed in the ρ-axis direction. Note that a (ρ, θ) = a _t (ρ, θ), b
When expressed as (ρ + τ, θ) = a _{t + DELT A} (ρ + τ, θ), the correlation parameter is expressed by the following equation C (ρ, θ, τ) = a _t (ρ, θ) · at _{+ DELTA} ( ρ + τ, θ) (4) ′. That is, the correlation processing of the equation (4) 'is executed in step 53 of FIG. Here, DELTA represents a delay time, and the velocity V of the tangential line moving in parallel is calculated as V = (τ _P · Δρ) / DELTA with Δρ being the resolution in the ρ direction. This is the "Direction" of the cerebral visual cortex of the living body.
It is a model of the function of "al Selectivity simple cell". The cells exhibit the unique property of responding to stimuli that move in a particular direction but not to stimuli in the opposite direction. This can be realized by limiting τ in the above equation to positive or negative, and reflects the constraint that the nerves of the living body cannot send positive and negative information at the same time. To be precise, this cell is a “non-linear response type” Directional Selectivity simple cell. By the way, the temporal correlation parameter can be calculated not only by the asymmetrical correlation calculation of equation (4) but also by the symmetrical correlation calculation of the following equation. C (ρ, θ, τ) = a (ρ−τ, θ) ・ b (ρ + τ, θ) (4) ″ However, in the following, the temporal correlation parameter is calculated by the asymmetrical correlation calculation of equation (4). It will be explained as what is done.

Correlation parameter C θ (ρ, θ,
τ): This is a parameter indicating the degree of temporal correlation of a tangent group tangent to a circle having a radius ρ sharing the center of the receptive field. θ with ρ = 0
In the correlation processing, ρ _P , where Cθ (ρ, θ, τ) becomes maximum,
From θ _P and τ _P , it is possible to quantitatively measure the position, orientation, and orientation rotation speed of the tangent line passing through the center of the receptive field. The correlation parameter is given by the following expression Cθ (ρ, θ, σ) = a (ρ, θ) · b (ρ, θ + τ) (5) where τ is the moving speed in the θ-axis direction. Note that a (ρ, θ) = a _t (ρ, θ), b
(Ρ, θ + τ) = a t + DELT A (ρ, θ + τ) correlation parameters and expressed by the following equation Cθ (ρ, θ, τ) = a t (ρ, θ) · a t + DELTA ( ρ, θ + τ) (5) ′. That is, the correlation processing of the equation (5) 'is executed in step 53 of FIG. Here, DELTA represents a delay time, and the rotation speed ω is obtained as ω = (τ _P · Δρ) / DELTA. There are cells that detect rotation in the living body, which are modeled.

(3) Correlation parameter C (ρ, θ, τ ₁ , τ ₂ ) of (ρ, θ) plane: This is a parameter indicating the temporal correlation of a tangent group moving while the direction changes. C
Ρ _P , θ _P , τ _1P , τ where (ρ, θ, τ ₁ , τ ₂ ) is maximum
_{From 2P} , the position, orientation, moving speed, and rotation speed of the tangent line passing through the center of the receptive field can be measured in more detail than the one-dimensional correlation parameter. The correlation parameter is given by the following equation C (ρ, θ, τ ₁ , τ ₂ ) = a (ρ, θ) · b (ρ + τ ₁ , θ + τ ₂ ) (6). That is, the correlation processing of the equation (6) is executed in step 53 of FIG. In addition, a (ρ, θ) = a t
(Ρ, θ), b (ρ + τ ₁ , θ + τ ₂ ) = a _{t + DEL TA} (ρ
+ Tau _1, the correlation parameter a is expressed as θ + τ ₂₎ by the following equation _{C (ρ, θ, τ 1} , τ 2) = a t (ρ, θ) · a t + DELTA (ρ + τ 1, θ + τ 2) ··· ( 6) ′. That is, in step 53 of FIG. 16, the correlation processing of the equation (6) ′ is executed. Here, DELTA represents a delay time, and the parallel movement speed V and the rotation speed ω are obtained as V = (τ _1P · Δρ) / DELTA ω = (τ _2P · Δρ) / DELTA.

(D-2) Projection of Basic Correlation Parameter With the above basic correlation parameters, the position of the tangent (ρ _P ),
Although all the correlation parameters of the azimuth (θ _P ) and the parallax / velocity (σ _P / τ _P ) can be determined, on the other hand, there is a problem that a large correlation parameter storage unit having three dimensions or more is required. Hereinafter, the definition and characteristics of the correlation parameter that can reduce the number of parameters for the correlation determination as necessary and reduce the capacity of the correlation parameter storage unit will be described below. Spatial correlation parameter C (ρ, θ, σ) and temporal correlation parameter C
(Ρ, θ, τ) will be described. Correlation parameters C (ρ, θ, σ ₁ , σ ₂ ) and C (ρ,
The same can be defined for the cases of θ, τ ₁ , τ ₂ ), and more detailed correlation measurement is possible than with the one-dimensional correlation parameter.

When the position ρ and the azimuth θ of the tangent line corresponding to the projection in the σ or τ direction are important and the parallax σ (moving speed τ) does not have to be directly measured, the basic correlation parameter C (ρ, θ, σ ) Or C (ρ, θ, τ) is added in the σ or τ direction as C _PRJ- σ (ρ, θ) or C
_PRJ- τ (ρ, θ) is effective, and the correlation parameter storage unit is reduced to two dimensions of ρ−θ, which is the same as the input dual plane. C _PRJ- σ (ρ, θ) = ΣC (ρ, θ, σ) (where σ = 0, 1, 2, ... σmax) (7) C _PRJ- τ (ρ, θ) = ΣC (ρ , Θ, τ) (However, τ = 0,1,2, ・ ・ ・ τmax) (8)

FIG. 29 shows the basic correlation parameter C (ρ, θ,
It is a flowchart of the filtering process which projects (σ).
The input data A and B are divided into receptive field images (step 50
a, 50b), polar transformation processing is performed on each receptive field image (steps 51a, 51b), and one-dimensional filtering processing is performed on the polar transformation result to map it to a dual plane (step 52).
a, 52b), the correlation processing of the equation (1) ′ is applied to the data of each dual plane (step 53), and the correlation parameter C (ρ, θ, σ) obtained by the correlation processing is integrated according to the equation (7). (Step 71), the integration result is stored in the correlation parameter memory of ρ-θ (step 72).

FIG. 30 is a flow chart of the process of calculating the correlation parameter and projecting it in the σ direction. 0 for correlation calculation
θ, 0 → ρ, 0 → σ (steps 201 to 20)
3). Then, using the equation (1) ′, the correlation parameter C
(Ρ, θ, σ) is calculated (step 204), and the correlation parameters are integrated according to the equation (7) (step 20).
5). Then σ is incremented and σ> σ
It is determined whether it is max (steps 206 and 207), and σ
If ≦ σmax, the process returns to step 204 and the subsequent processes are repeated. If σ> σmax, the integrated value is ρ,
It is stored at the matrix intersection of the correlation parameter storage indicated by θ (step 208). Thereafter, ρ is incremented and it is determined whether ρ> ρmax (step 20
9, 210), and ρ ≦ ρmax, the process returns to step 203 and the subsequent processes are repeated. If ρ> ρmax, θ is incremented and it is determined whether θ> θmax (steps 211 and 212).
If so, the correlation calculation / projection process ends.

Projection in ρ direction The azimuth θ of the corresponding tangent line and the parallax σ (or moving speed τ) are important, and when the position ρ does not have to be directly measured, the basic correlation parameter C (ρ, θ, σ ) _Is added in the ρ direction, the correlation parameter C _PRJ− ρ (θ, σ) or C _PRJ− ρ (θ, τ) given by the following equation is effective, and the correlation parameter storage unit stores the same two parameters as the input dual plane. Reduced to dimension. C _PRJ- ρ (θ, σ) = ΣC (ρ, θ, σ) (where ρ = 0,1,2, ... ρmax) (9) C _PRJ- ρ (θ, τ) = ΣC (ρ , Θ, τ) (however, ρ = 0,1,2, ・ ・ ・ ρmax) (10) This correlation parameter is effective when detecting a contour line moving in the visual field, The azimuth θ of the tangent line and the moving speed τ can be measured regardless of the position ρ of the line. Some cells in the cerebral hypercolumn work in the same way. When a straight line moves in parallel in the field of view, it ignites strongly and ceases to give a signal when stopped, and is known as a "Directional Selectivity complex cell". This is described later in "Optical flow measurement" and "Binocular stereoscopic vision from random dots."
Play an important role in.

When the position ρ of the corresponding tangent line and the parallax σ (or moving speed τ) are important and the azimuth θ does not have to be directly measured, the basic correlation parameter C (ρ, θ, σ ) In the θ direction, the correlation parameter C _PRJ− θ (ρ, σ) or C _PRJ− θ (ρ, τ) given by the following equation is effective, and the correlation parameter storage unit stores the same two parameters as the input dual plane. Reduced to dimension. C _PRJ- θ (ρ, σ) = ΣC (ρ, θ, σ) (where θ = 0, 1, 2, ... θmax) (11) C _PRJ- θ (ρ, τ) = ΣC (ρ , Θ, τ) (however, θ = 0, 1, 2, ... θmax) (12) This correlation parameter is effective for tracking a moving contour line while capturing it in the vicinity of the receptive field. . The characteristic is that the distance ρ of the contour line from the center of the receptive field and the moving speed τ can be measured independently of the direction of the line. Similar cells exist in the retina of frogs and are used to track prey such as flies.

Projection in the ρσ or ρτ direction The azimuth θ of the corresponding tangent is important, and the position ρ and the parallax σ (movement
When it is not necessary to directly measure the velocity τ), the basic correlation pattern
Parameter C (ρ, θ, σ) in ρσ direction (or ρτ direction)
Correlation parameter C given by_PRJ-ρσ
(Θ) or C _PRJ-ρτ (θ) is effective and the correlation parameter
The meter storage is reduced to one dimension. C_PRJ-ρσ (θ) = ΣΣC (ρ, θ, σ) (where ρ = 0,1,2, ... ρmax, σ = 0,1,2, ... σmax) (13) C_PRJ-ρτ (θ) = ΣΣC (ρ, θ, τ) (where ρ = 0,1,2, ... ρmax, σ = 0,1,2, ... σmax) (14) This correlation parameter is The outline of a certain direction in the receptive field
It is effective in detecting that you have moved,
The azimuth θ of the contour line is independent of the position and movement speed of the contour line.
It has the characteristic of being able to measure.

When the parallax ρ (moving speed τ) of the contour line corresponding to the projection on ρθ is important and it is not necessary to directly measure the position ρ and the azimuth θ, the basic correlation parameter C (ρ, θ, σ) is set. Correlation parameter C _PRJ- ρθ (σ) or C given by the following equation added in the ρθ direction
_PRJ- ρθ (τ) is effective, and the correlation parameter storage unit is reduced to one dimension. C _PRJ- ρθ (σ) = ΣΣC (ρ, θ, σ) (where ρ = 0,1,2, ... ρmax, θ = 0,1,2, ... θmax) (15) C _{PRJ - ρθ (τ) = ΣΣC (} ρ, θ, τ) ( where, ρ = 0,1,2, ··· ρmax, θ = 0,1,2, ··· θmax) (16) this correlation parameter , It is effective when you want to detect the contour line moving in the receptive field together with its velocity. In the case of equation (16), the moving contour line velocity τ can be measured irrespective of the position and direction of the contour line. It has the characteristic of

Only whether or not there is a contour line corresponding to the projected receptive field in θρσ or θρτ is important.
When it is not necessary to measure the position ρ, the azimuth θ, and the parallax σ (moving speed τ), the basic correlation parameter C (ρ, θ, σ) is added in the ρθσ direction (or ρθτ direction) by the following formula. The correlation parameter C _PRJ- θρσ or C _PRJ- θρτ is effective, and the correlation parameter storage unit stores 0 for only one cell.
Reduced to dimension. C _PRJ- θρσ = ΣΣΣC (ρ, θ, σ) (where ρ = 0,1,2, ... ρmax, θ = 0,1,2, ... θmax, σ = 0,1,2, ...・ ・ Σmax) (17) C _PRJ- θρτ = ΣΣΣC (ρ, θ, τ) (However, ρ = 0,1,2, ... ρmax, θ = 0,1,2, ... θmax, σ = 0,1,2, ... σmax) (18) Such a case is effective when it is desired to detect only whether or not there is a contour line that moves within the receptive field, using Equation (18) as an example. Therefore, it has a characteristic that measurement can be performed regardless of the position and direction of the contour line and the moving speed. Cells similar to this exist in common in the retinas of lower animals such as frogs, and are extremely few cells that quickly detect whether prey such as flies has entered the receptive field, The wisdom that makes up the small number of processing systems can be inferred. Engineering, use I
In a self-propelled vehicle for planetary exploration where the number of C is very limited,
This is effective when grasping the rough movement status on a receptive field basis.

Projecting in the diagonal direction of the (ρ, τ) or (ρ, σ) plane Above, in order to reduce the capacity of the correlation parameter storage unit,
Basic correlation parameters C (ρ, θ, σ), C (ρ, θ,
τ) and other element parameters, that is, the contour line position ρ, azimuth θ, parallax σ, and moving speed τ. Here, a method will be described in which the capacity of the correlation parameter storage unit is reduced while the above-described properties of each element parameter remain. Basic correlation parameters C (ρ, θ, σ), C
(Ρ, θ, τ) is represented by (ρ, τ) or (ρ,
σ) When projected in any diagonal direction of the plane, ρ, τ,
The correlation parameter storage unit can be two-dimensionally reduced while leaving the property of σ. The oblique projection, (ρ, σ) is described in example projected in 45 ⁰ direction. Letting ξ be the axis orthogonal to the projection direction, the correlation parameter C _PRJ-45 σ projected in the 45 ⁰ direction.
(Θ, ξ) is given by the following equation, and the correlation parameter C _PRJ-45
When θ _P and ξ _{P at} which σ (θ, ξ) becomes maximum are obtained, the azimuth θ _P of the corresponding contour line and the “parameter ξ _P that combines position ρ and parallax σ” are measured. C _PRJ-45 σ (θ, ξ) = ΣC (ρ + i, θ, σ + i) (where i = 1, 2 ...) (19)

(D-3) Convolution filter C ₁ (ρ, θ) of natural filter The output data of the dual plane is a (ρ, θ), b (ρ, θ)
The following projected correlation parameter C ₁ (ρ, θ), taken as θ), constitutes an interesting filter. C ₁ (ρ, θ) = Σa (ρ + i, θ) · b (i, θ) (where i = 1, 2 ...) (20) On the other hand, the aforementioned prior patent (Japanese Patent Application No. 3-327722) No.) one-dimensional filter convolution operation output C ₂ (ρ,
θ) is given by the following equation, where the one-dimensional filter function is g (ρ). C ₂ (ρ, θ) = Σa (ρ + i, θ) · g (i) (where i = 1, 2 ...) (21) Comparing both equations, the correlation parameter C ₁ (ρ, θ)
Is the filter function g (ρ) of the convolution output C ₂ (ρ, θ) changed to the correlation data b (i, θ). From this, the correlation parameter C ₁ (ρ, θ) is the natural data b instead of the artificial filter function g (ρ).
It can be considered as a result of "natural filtering" that is obtained by performing a convolution operation with (i, θ).

Considering both filtering functions,
The "artificial filtering" of the equation (21), in which the filter function g (ρ) is known, can be adopted when the action such as differentiation can be specified in advance, but "features that look the same between the left and right eyes" can be extracted or In the case of tracking "the same feature as the previous image", convolution operation with natural data is required, and such feature extraction and feature tracking are impossible. On the other hand, the “natural filtering” of the equation (20) can perform an accurate filter even in the above case where the filter pattern cannot be defined in advance.

FIG. 31 is a block diagram of the natural filter of the present invention. Reference numeral 81 is a receptive field dividing section for cutting out input data (image data for one screen) for each receptive field and outputting, and 82 is a receptive field image. A polar conversion unit that performs conversion processing, 83 is a polar conversion result a
A ρ-θ dual plane (hyper column memory) for storing (ρ, θ), 84 is a natural filter according to the present invention, and a receptive field dividing unit 84a that cuts out natural filter data for each receptive field and outputs the data. A polar conversion unit 84b that performs a polar conversion process on the receptive field image, a ρ-θ dual plane 84c that stores the polar conversion result b (ρ, θ), a multiplication unit 84d that performs the operation of the equation (20), and an integration unit. 84e.

The multiplication unit 84d is a dual plane 83, 84c.
Data a (ρ + i, θ), b (i, θ)
(The initial value of ρ, θ, i is 0) The following multiplication a (ρ + i, θ) · b (i, θ) is executed and output to the integrating unit 84e, and i is incremented to the next value. Data a (ρ + i, θ), b (i,
The same multiplication is performed on θ) until i> imax (filter width is set appropriately). Accumulator 84e
Multiply the multiplication results, and if i> imax,
-Θ Plane memory (not shown). After that, if the sum of products operation is executed for all (ρ, θ), the natural filtering process ends.

Application to binocular stereoscopic vision As described above, it is possible to perform detailed contour correspondence in three dimensions by the basic correlation parameter C (ρ, θ, σ). Plane (ρ, θ)
The contour can be handled with. Specifically, the above-mentioned C
_By obtaining the maximum of ₁ (ρ, θ), the position ρ and the azimuth θ of the corresponding contour line for both eyes can be determined regardless of the parallax σ. There are cells with similar functions in the cerebral cortex 18 of the monkey, and the corresponding contour lines are detected by both eyes.

Application to tracking of moving contour line The above is a natural filter between spatially different receptive fields, but a temporal natural filter is also effective. Temporal correlation parameter C _time (ρ, θ) is the input data temporally different _{a t (ρ, θ),} a t + DELTA (ρ, θ) as similarly following equation C _time (ρ, θ ) = Σa _t (ρ + i, θ) · a _{t + DELTA} (i, θ) (where i = 0, 1 ...) (22). As described above, in a three-dimensional space, the position, direction, and moving speed of the moving contour line can be measured by the simultaneous basic correlation correlation parameters C (ρ, θ, τ). However, this natural filter has a characteristic that the position ρ and the direction θ of the moving contour can be determined independently of the moving speed τ on the two-dimensional plane by obtaining the maximum of C _time (ρ, θ). Similar cells exist in the visual cortex of the cerebrum.

(D-4) Difference Type Basic Correlation In the above, the basic correlation parameter is C (ρ, θ, σ) = a (ρ, θ) · b (ρ + σ, θ) C (ρ, θ, τ) = A _t (ρ, θ) · at _{+ DELTA} (ρ +
(τ, θ) is constructed by multiplication, but the same effect can be obtained by adding it. In this case, linearity is guaranteed between the signals, and it is known as "linear simple cell" in the living body. Furthermore, by using the multiplication of the above equation as subtraction, it becomes possible to perform binocular stereoscopic vision and tracking of an object with an unclear contour by using a gradual change in brightness or hue as a clue.

Spatial Correlation In the case of the ρ-axis direction correlation, its basic correlation parameter is given by C '(ρ, θ, σ) = a (ρ, θ) -b (ρ + σ, θ) (23) . In this correlation, C '(ρ, θ, σ) becomes zero at the same place in the input data. This property enables binocular stereoscopic viewing of an object whose contour is not clear. For example,
Looking at a large round pillar, there is no clear contour except at both ends, but its brightness gradually changes in relation to the incident angle. C '(ρ, θ, σ) where the left and right eyes have the same brightness
If we find out from the property of zero, it is possible to correspond to both eyes, and the surface condition of the cylinder can be measured stereoscopically.

FIG. 32 is a flow chart of the difference type correlation processing. The entire basic flow is the same as that in the case of FIG. 14, and the difference is that the calculation of the correlation parameter is performed by the equation (23). 0 → θ, 0 → ρ, 0 → σ in correlation calculation
(Steps 301 to 303). Then, the correlation parameter is calculated by the equation (23) and stored in the correlation parameter storage unit (steps 304 and 305). Then, σ is incremented, and it is determined whether σ> σmax (steps 306 and 307). If σ ≦ σmax, the process returns to step 304 and the subsequent processing is repeated. And σ
> Σmax, ρ is incremented and ρ>
It is determined whether or not ρmax (steps 308 and 309),
If ρ ≦ ρmax, the process returns to step 303 and the subsequent processes are repeated. If ρ> ρmax, θ is incremented and it is determined whether θ> θmax (step 31
0, 311), and θ ≦ θmax, the process returns to step 302 and the subsequent processes are repeated. When θ> θmax, the correlation calculation process ends.

Temporal correlation Describing in the case of ρ direction, the basic correlation parameter is given by the following equation C '(ρ, θ, τ) = a (ρ, θ) -b (ρ + τ, θ) (23) . Note that a (ρ, θ) = a _t (ρ, θ), b
(Ρ + τ, θ) = a t + DELT A (ρ + τ, θ) the correlation parameter a is expressed as the following equation C '(ρ, θ, τ ) = a t (ρ, θ) -a t + DELTA (Ρ + τ, θ) is given by (23) ′. Even in this correlation, C ′ (ρ, θ, τ) becomes zero at the same place in the input data. This property enables tracking of an object whose contour is not clear. In the same way as in the case of, the place where the brightness is the same is C ′ (ρ, θ,
By detecting from the property that τ) becomes zero, stable tracking can be performed even in a part where a clear contour line cannot be detected such as the surface of a cylinder or a part of a human face. This cell is
nal Selectivity “Simple cells” are “linear response cells”.

(E) Concrete Method and Example (a) Binocular Stereoscopic As a concrete example of a spatially different action screen, a binocular stereoscopic method and a simulation example will be described. The tangent has the characteristics of lines, gaps, and edges, and the "line" is a bright band.
The "edge" is a boundary line between a bright portion and a dark portion, and the "gap" is a dark thin band which is the opposite of the line. Although shown for explaining the principle, here, binocular stereoscopic vision of other features including lines will be systematically described.
7722), line extraction filter, edge extraction filter,
Although the gap extraction filters have been disclosed, using these filters enables binocular stereoscopic vision of lines and gaps and binocular stereoscopic vision of edges.

(A-1) Various Filter Configurations Line Extraction Filter FIG. 33 is a configuration diagram of a “line” extraction filter, and (a) is “receptive field method (receptive field division + polar transformation) + one-dimensional gas filter”. The line extraction filter of the configuration, (b) is the line extraction filter of the "two-dimensional gas filter + receptive field method" configuration. The line extraction filter in (a) divides the input image into receptive fields by the receptive field dividing unit,
The polar transformation unit performs polar transformation for each receptive field image, and the one-dimensional gas filter performs one-dimensional second derivative filter processing on the pole transformation output to extract line segments. The line extraction filter in (b) is a two-dimensional gas filter that applies two-dimensional second-order differential processing to the input image, then divides it into receptive fields, and polarizes each receptive field image to extract lines. Is.

Edge Extraction Filter FIG. 34 is a block diagram of various “edge” extraction filters.
Is a basic filter configuration that extracts "edges" by applying a one-dimensional gr filter process to the output of the receptive field method (receptive field division + pole transformation), and (b) is a further one-dimensional structure after the basic filter in (a). A configuration of an edge extraction filter connected with a gas filter, (c) is a configuration of an edge extraction filter further connected with a two-dimensional gas filter before the basic filter of (a). The edge extraction filter in (a) divides the input image into receptive fields by the receptive field dividing unit, performs polar transformation on each receptive field image in the polar transforming unit, and performs a first-order differential processing on the polar transform output with the one-dimensional gr filter. Is applied to extract the "edge". The edge extraction filter in (b) divides the input image into receptive fields by the receptive field dividing unit, performs polar transformation on each receptive field image in the polar transforming unit, and uses a one-dimensional gr filter to obtain a one-dimensional primary The "edge" is extracted by performing differential processing and then performing one-dimensional secondary differential processing with a two-dimensional gas filter. (c)
The edge extraction filter is a two-dimensional gas filter that performs two-dimensional second-order differential processing on the input image, then the receptive field division unit divides the input image into receptive fields, and the polar transformation unit polarizes each receptive field image. The transformation is performed, and the one-dimensional gr filter is used to perform one-dimensional first-order differential processing on the pole transformation output to extract the "edge".

Gap Extraction Filter The gap extraction filter is an inversion of the sign of the line extraction filter. Therefore, if a sign inversion unit is connected after each line extraction filter shown in FIG. 33, it becomes a gap extraction filter. FIG. 35 is a configuration diagram of the gap extraction filter, in which a sign inversion unit is provided.

(A-2) Binocular stereoscopic view of line / gap Binocular stereoscopic view of line / gap with one-dimensional filter "Polar transformation + one-dimensional filter processing" in the basic flow of FIG. ) Or the “receptive field division + polar conversion + one-dimensional Gaussian filter (+ sign inversion) processing” shown in FIG. 35 (a), and by inputting the images of the left and right eyes, the binocular stereo of the line / gap It becomes possible to see. The simulation result has already been described with reference to FIGS. FIG.
The sharp peak of C (ρ, θ, σ) in ∘ indicates the corresponding point of the vertical line, and the parallax is correctly measured as 5 pixels from the value of the σ axis.

Binocular stereoscopic view of lines and gaps by two-dimensional filter (biological binocular stereoscopic view) The "polar conversion + one-dimensional filter processing" in the basic flow of FIG. By performing the "two-dimensional Gaussian filter + receptive field division + polar conversion (+ sign inversion) processing" shown in b) and inputting the images of the left and right eyes, stereoscopic vision of lines and gaps is possible. . This method
The input images of the left and right eyes are subjected to two-dimensional second-order differential processing (two-dimensional convolution filter processing) with a two-dimensional gas filter, and then divided into receptive fields, and polar transformation is applied to each receptive field image. . The two-dimensional gas filter processing is equivalent to “one-dimensional filter processing after pole conversion”, and the simulation result is omitted, but the same as in FIGS. 11 to 13. This method is similar to human stereoscopic vision, and has the advantage that the results of contour enhancement by a two-dimensional gas filter can be commonly used for shape recognition and the like.

(A-3) Binocular stereoscopic view of the edge Binocular stereoscopic view of the edge with the one-dimensional filter "Polar transformation + one-dimensional filter processing" in the basic flow of FIG. 14 is shown in FIG. 34 (b). Receptive field division + polar transformation + one-dimensional gradient filter + one-dimensional Gaussian filter processing ”and by inputting images of the left and right eyes, binocular stereoscopic vision of the edge becomes possible. The simulation results are shown in FIGS. 36 and 37. The circular parts C _L and C _R in FIGS. 36 (a) and 36 (b) are receptive fields for the left and right eyes, and the images in the circle are receptive field images for the left and right eyes.
The figure on the edge of the part surrounded by Q is shown. In FIG. 37, the strength of the signal of all data is shown by contour lines, and the result of the pole conversion and the intensity distribution of the basic correlation parameter C (ρ, θ, σ) at θ = 168 ⁰ are shown by contour lines. ing. The sharp peak of C (ρ, θ, σ) indicates the corresponding point of the edge, and the parallax is correctly measured as 1 pixel from the value of the σ axis.

Binocular stereoscopic vision at the edge where two-dimensional filters are mixed (biological binocular stereoscopic vision) "Polar conversion + one-dimensional filter processing" in the basic flow of FIG. 14 is shown in FIG. Gaussian filter + receptive field division + polar transformation + one-dimensional gradient filter processing ”and by inputting the images of the left and right eyes, binocular stereopsis of the edge becomes possible. In this method, the input images of the left and right eyes are subjected to a two-dimensional quadratic differential process by a two-dimensional gas filter, and then divided for each receptive field, and a polar transformation is applied to each receptive field image. The two-dimensional gas filter processing is equivalent to “one-dimensional filter processing after pole conversion”, and the simulation result is omitted, but it is the same as in FIG. 36. This method is similar to human stereoscopic vision, and has the advantage that the results of contour enhancement by a two-dimensional gas filter can be commonly used for shape recognition and the like. (a-4) Polygonal figure / curve figure / random dot / texture binocular stereoscopic vision: Since this binocular stereoscopic vision is related to the description in the measurement of the moving direction / speed described later, (c ) Will be explained again.

(B) Measurement of moving direction and moving speed As a concrete example of an action screen that is temporally different,
In order to track a moving object, the moving direction Φ and velocity V
The measuring method will be described. (b-1) Measurement of moving direction and velocity of contour tangent The contour of the object is approximated by a straight line (tangent), and the tangent shift
The moving direction and speed can be measured by the following methods. Current time image
And the image at the next time "polar conversion + one-dimensional filter processing"
Data a_t(Ρ, θ), a_{t + DELTA}From (ρ, θ),
The basic correlation parameter C (ρ, θ, τ) is calculated by the following equation C (ρ, θ, τ) = a_t(Ρ, θ) ・ a_{t + DELTA}(Ρ + τ, θ) Calculated from (24), and the point (ρ) at which C (ρ, θ, τ) is maximum
_P, Θ_P, Τ_P) Is obtained, the tangent moving direction Φ = θ_P+90⁰  Tangent moving speed V = (τ_P・ Δρ) / DELTA ・ ・ ・ (25) is obtained, and the tangent line can be tracked from the data. here,
Δρ is the resolution in the ρ direction.

This direction and velocity cause an error when the azimuth of the tangent line and the moving direction V ₀ are not orthogonal. The reason will be described with reference to FIG. 38. The tangent line L measured by the equation (25) is
This is because the moving direction and the velocity of are the “direction orthogonal to the tangential direction” and the “velocity V in the orthogonal direction”, respectively, and do not match the true direction vector V ₀ . A correct method of measuring the moving direction and velocity in which this error is corrected will be described below, but this method is sufficient for tracking an object.
Because the tracking is performed at short time intervals, the error in the above equation is small and does not pose a problem.

FIG. 39 is a flow chart for measuring the moving direction and moving speed of a tangent line. Delay image data (step 4
9) The image data for one screen at the current time and the image data for one screen before the predetermined time obtained by the delay are divided into receptive field images (steps 50a, 50a '), and each receptive field image IM, IM ′ is subjected to polar conversion processing (step 51a,
51a '), the one-dimensional filtering process is performed on the polar transformation result, and the result is mapped to a dual plane (steps 52a, 52).
a '), the correlation processing of the equation (24) is performed between the data mapped to the dual planes (step 53), and the calculated correlation parameter C (ρ, θ, τ) is stored in the correlation parameter storage unit ( Step 72), after completion of the correlation calculation, the correlation parameter storage unit is scanned to detect (ρ _P , θ _P , τ _P ) where C (ρ, θ, τ) becomes maximum (step 73), and finally
Calculate the moving direction Φ of the tangent and the moving speed V using the equation (25).
(Step 74).

(B-2) Measurement of direction and speed of corner movement In order to accurately measure the direction and speed of movement, two or more corners should be placed in the receptive field.
The top line is required. The object is shown in Fig. 40 (a).
A corner which generally consists of two contour tangents Li, Lj
It has CN and is accurate by the correlation processing of this corner.
It can measure the moving direction and speed. First, in case of (b-1)
Similarly, the correlation processing is performed by the equation (24). That is, the present
The image of the time and the image of the next time are “polar conversion + one-dimensional fi
Data processed " _t(Ρ, θ), a_{t + DELTA}(Ρ,
θ) is used to calculate the basic correlation parameter C (ρ, θ, τ) by (2
The point at which C (ρ, θ, τ) is maximized, calculated from equation (4)
(Ρ_P, Θ_P, Τ_P). Next, the maximum value search
The points corresponding to the two extracted tangents Li and Lj are (ρ_i，
θ_i, Τ_i), (Ρ_j, Θ_j, Τ_j), ...
The accurate moving direction Φ and velocity V are calculated by the following formula Φ = arctan [(τ_isin θ_j−τ_jsin θ_i) / (Τ_icos θ_j−τ_jcos θ_i)] (26) V = (Δρ ・ τ_i) / (Sin (Φ−θ_i) ・ DELTA) (27). Where DELTA is the current time and the next time
It is a time interval.

The expressions (26) and (27) are derived as follows. That is, the tangent line L measured by the equation (25) (see FIG.
The moving direction and the velocity (see the reference) are the “direction orthogonal to the tangential direction” and the “velocity V in the orthogonal direction”, respectively. Therefore, when moving in the Φ ₀ direction in FIG. 38, the speed measured by the equation (25) becomes V ₀ cos ξ. From this, V ₀ cos ξ = (τ _P · Δρ) / DELTA (28) holds. By the way, the angle ξ is given by the following equation ξ = 90 ⁰ − (Φ ₀ −θ _P ) (29) (where θ _P is the direction of the tangent line L) when the clockwise direction in FIG. 38 is positive. In equation (28), (2
Substituting the equation (9), V ₀ cos [90 ^0- (Φ ₀ -θ _P )] = (τ _P · Δρ) / DELTA (30), and when transformed, V ₀ sin (Φ ₀ -θ _P ) = ( τ _P · Δρ) / DELTA (30) ′. Here, when τ _P = τ _i and θ _P = θ _i, and V ₀ is obtained, the equation (27) is derived. If τ _P is represented by τ and θ _P is represented by θ, and equation (30) is transformed with k = DELTA / Δρ, the following equation τ = k · V ₀ · cos [90 ⁰ − (Φ ₀ −θ)] (31) becomes, and if this relational expression is drawn on the θ-τ plane,
As shown in (b), it becomes a sine wave (sinusoidal firing pattern). In other words, each side Li, Lj of the figure is concentrated at one point on the sine wave on the θ-τ plane.

By modifying the equation (31), τ = k · V ₀ · sin (Φ ₀ −θ) (32) can be obtained. In this equation (32), (θ, τ) = (θ _i ,
τ _i ), (θ, τ) = (θ _j , τ _j ), the following equation τ _i = k · V ₀ · sin (Φ ₀ −θ _i ) (32) ′ τ _j = k · V ₀ · sin (Φ ₀ -θ _j ) (32) ″. From (32) ′ and (32) ″, τ _i / τ _j = sin (Φ ₀ -θ _i ) / sin (Φ ₀ -θ _j ) is TanΦ ₀ = (τ _i sin θ _j −τ _j sin θ _i ) / (τ _i cos θ _j −τ _j cos
θ _i ) is derived and Eq. (26) is derived. Note that the corner is composed of two contour lines, but if it is a feature composed of three or more contour lines, the sine wave of FIG. 40 (b) can be determined more accurately,
Therefore, the measurement accuracy of (Φ ₀ , V ₀ ) can be improved.

(B-3) Measurement of Moving Direction / Velocity from Polygon / Curve In the above, measurement of moving direction / velocity from two lines (corner) was described, but from more lines and tangents. Focusing on the "polygons / curves" that are configured, more reliable "movement direction / speed measurement" becomes possible. The correlation parameter C _PRJ- ρ (θ,
τ) C _PRJ- ρ (θ, τ) = ΣC (ρ, θ, τ) (ρ = 1,
2, ...) plays an important role.

Measurement of Moving Direction / Velocity from Polygon The response of this parameter to a polygonal figure is that the response of each side of the polygon is distributed in a sine wave shape, as shown in FIG. In the case of an N-sided polygon, N peaks are arranged in a sinusoidal shape, and a polygon having a high velocity becomes a large-amplitude sinusoidal wave. From this, the moving direction and speed of the entire polygon can be accurately measured. When the angle between the azimuth of the straight line and the moving direction is ξ as shown in FIG. 38, τ _{P at} which the correlation parameter C (ρ, θ, τ) becomes a peak is expressed by the equation (28) as τ _P = (V ₀ · DELTA / Δρ) · cosξ, and C _PRJ− ρ (θ, τ) _obtained by projecting this in the ρ direction becomes the sine wave (sinusoidal firing pattern) of FIG. 41. When the maximum point (θmax, τmax) is obtained, the moving direction Φ ₀ = θmax−90 ⁰ (33a) The true moving speed V ₀ = (τmax · Δρ) / DELTA (33b) can be calculated. Compared with the above-mentioned corner method (Fig. 40 (b)),
The basic principle is the same, but many points (N points) on the sine wave
Has a great feature that its peak (point at which the maximum amplitude) can be accurately calculated.

The extraction of this sine wave can be performed by using the polar transformation. That is, the polar transformation on the cylinder (Hough)
In (conversion), the points are converted into sine waves, and when each point on the sine wave is further pole-converted (inverse Hough conversion), each point is converted into a straight line intersecting at one point. Therefore, a sine wave can be extracted from this point. Specifically, each point of the C _PRJ- ρ (θ, τ) plane is converted into a straight line that satisfies the following relation τ = −Vy · cos θ + Vx · sin θ (34), and its intersection point CP (Vx, Vy) Is calculated (see FIG. 42). The direction and distance from the origin of the Vx and Vy coordinates to this intersection CP are θma in the equations (33a) and (33b).
x, τmax. The direction and distance to the intersection CP are (33
The reasons for θmax and τmax in equations (a) and (33b) are as follows. When the equation (34) is transformed by using the true velocity V ₀ and the direction Φ, τ = (√ (Vx ² + Vy ² )) · sin (θ−Φ) = V ₀ · (DELTA / Δρ) · sin (θ −Φ) (34) ′. However, Φ = arctan (Vy / Vx). Therefore, C
The peak of the sine wave on the _PRJ- ρ (θ, τ) plane is τmax = V ₀ · (DELTA / Δρ) or = √ (Vx ² + Vy ² ) θmax = Φ−90 ⁰ . From this, the true speed and direction can be calculated by the following formula: true speed V ₀ = (Δρ / DELTA) ・ √ (Vx ² + Vy ² ) (35a) true direction Φ ₀ = arctan (Vy / Vx) (35b) . This method corresponds to "inverse Hough transform".

FIG. 43 is a flow chart for measuring the moving direction and moving speed as described above. Image data for one screen at the current time and the next time is divided into receptive field images (step 5).
0a, 50a '), the receptive field images IM, IM' are subjected to polar transformation processing (steps 51a, 51a '), and the polar transformation result is subjected to one-dimensional filtering processing to be mapped on a dual plane (steps 52a, 52a'). ′), The correlation processing of equation (24) is performed between the data mapped to the dual planes (step 53), and the calculated correlation parameter C (ρ, θ, τ)
Is stored in the correlation parameter storage unit (step 72).
Then, the following expression C _PRJ- ρ (θ, τ) = ΣC (ρ, θ, τ) (ρ = 1,
2, ...), the correlation parameter C _PRJ− ρ projected in the ρ direction
(Θ, τ) is obtained (step 75) and stored in the correlation parameter storage unit on the θ-τ plane (step 76). Then, the correlation parameter C _PRJ- ρ (θ, τ) is subjected to a pole conversion process by the equation (34) (step 77), and a peak point (intersection point) on the Vx, Vy plane (velocity plane) is obtained (step 78). , The direction and distance from the coordinate origin to this intersection are obtained, and the true velocity V ₀ and the true direction Φ are calculated based on the equations (35a) and (35b).
₀ is calculated (step 79). Measurement of moving direction and speed from curved figure The above is a polygon, but a curved figure can also perform highly reliable measurement. In the “receptive field method + polar transformation”, the tangent line of the curve can be accurately extracted, and C _PRJ− ρ (θ, τ) can be calculated from the data.

(B-4) Measurement of Moving Direction / Velocity from Random Dots and Textures Above, it has been described that the moving direction and velocity can be measured from a figure (polygon, curve) composed of straight lines and tangent lines.
Next, it expands to a figure composed of random points. The movement direction and velocity of this figure can be measured by the same process as in (b-3). The reason is that the "receptive field method + one-dimensional filter" can be extracted as a "line of points",
This is because it can be considered the same as a polygon (see FIG. 44).
By this method, the random dot figure is naturally composed of a fine pattern, and it becomes possible to measure the moving direction / velocity of the "texture" figure, and this expansion has an extremely large effect.

45 and 46 are explanatory diagrams of simulation results in the moving direction and speed measurement of a random dot figure, which are the results when 6√2 pixels are moved in the 45 ⁰ direction in 1 second, and the delay DELTA is 1 Set to seconds. It should be noted that this random dot figure is a computer-generated random dot stereogram and is drawn with 1 dot = 1 pixel, and the density is 50%. In FIG. 45, I
M and IM 'are receptive field images at the current time (no delay) and the next time (with delay), and HCIM and HCIM' are subjected to polar conversion processing on the respective receptive field images IM and IM ', and the polar conversion results are obtained. Is a hyper-column image obtained by performing a one-dimensional filtering process on the and to map it to a dual plane of ρ-θ. Also, in FIG. 46, PRIM performs correlation processing between hypercolumn images, and the obtained correlation parameter C (ρ, θ, τ) is projected in the ρ direction, and the correlation parameter C _PRJ- ρ on the θ-τ plane.
(Θ, τ), HGIM is the correlation parameter C _PRJ- ρ (θ,
τ) is the result of polar conversion on the Vx and Vy planes that has been subjected to polar conversion processing. A sine wave pattern SWV appears on the θ-τ plane, and a sharp peak PK is extracted at 6√2 pixel positions in the 45 ⁰ direction on the velocity plane. From this peak point (Vx, Vy), the true speed and the true moving direction can be calculated based on the equations (35a) and (35b).

[0110] The cerebral hypercolumn shows "Directional S
There are cells called "electivity simple cells" and "Directional Selectivity complex cells", which detect the moving direction and speed of the moving contour line, and their functions are very similar to this method. Especially, in the 5th and 6th layers of the hyper column, there are complex cells that feed back signals to the superior colliculus and the lateral geniculate body, and these complex cells are measured by measuring the moving direction and moving speed data of the contour line. It plays an important role in controlling the eyeball so that an object is captured in the center of the visual field. From an engineering point of view, it is necessary for mobile robots to capture an object in the center of the field of view and perform actions such as approaching and avoiding it, and this method is effective. Also, researches for measuring the moving direction and speed of an object have become popular in recent years, but those methods are "method of obtaining moving direction and speed from temporal change of brightness" and "change with movement of figure feature". The method of calculating the moving direction and the speed can be roughly classified. However, the former has a big problem in practical use that it is handled differentially and is weak against changes and vibrations of lighting, and the latter is the point of correspondence of graphic features, but the current "shape processing" There is a limit to the application of the weak image processing technology. In order to solve these optical flow problems, this method has the characteristic that the former method is "integral operation that changes the point group that composes a figure into a straight line by polar transformation and integrates it, and is resistant to noise". is doing. For the latter, the feature is that "the simplest figure, that is, the tangent line to the contour can be decomposed and the tangent line can be processed in a one-dimensional manner by polar transformation, so that reliable measurement can be performed even for complicated figures." have. In other words, this method can be said to be "a new optical flow method capable of integral one-dimensional processing".

(B-5) Measurement of moving direction and speed of line / gap Moving line / gap including simulation results below
The method of measuring the direction and speed will be described. Direction and speed tool
The physical measurement method used (b-1) as an example, but (b-2) to (b-4)
But you can do the same. Measurement of moving direction and velocity with one-dimensional filter "Receptive field division + pole transformation + one-dimensional" in the flow of FIG.
"Filter processing" is shown in Fig. 33 (a) or 35 (a).
Yano field division + pole transformation + one-dimensional Gaussian filter (+ sign
"Reversal) process", so that
It is possible to measure the speed. The simulation result
It shows in FIG. Vertical line (θ≈90⁰) In the horizontal direction
The correlation pattern obtained by inputting the two images before and after the elementary movement
The intensity of the parameter C (ρ, θ, σ) is shown by contour lines. Basic
The sharp peak of the correlation parameter C (ρ, θ, σ) is the vertical line.
Corresponding points of (ρ_P, Θ_P, Τ_P) And
And moving direction = θ_P+90⁰= 182⁰  (Movement speed / DELTA) = τ_P= 6 images and correct results were obtained.

Measurement of moving direction and velocity by two-dimensional filter (biological) “Receptive field division + polar conversion + one-dimensional filter processing” in the flow of FIG. 39 is shown in FIG. 33 (b) or FIG. 35 (b). By performing the “two-dimensional Gaussian filter + receptive field division + polar conversion (+ sign inversion) processing” shown, it becomes possible to measure the moving direction and speed of the line / gap. In this method, the input image is subjected to two-dimensional second-order differential processing (two-dimensional convolution filter processing) with a two-dimensional gas filter, then divided into receptive fields, and polar transformation is applied to each receptive field image. is there. Two-dimensional gas filter processing is "one-dimensional filter processing after pole transformation"
Although the simulation result is omitted, the same measurement result of the moving direction and speed as in FIG. 47 can be obtained. This method is similar to human stereoscopic vision, and has the advantage that the results of contour enhancement with a two-dimensional gas filter can be commonly used for shape recognition and the like.

(B-6) Measurement of edge moving direction and speed The following are the most frequent images, including simulation results.
Of moving direction and speed of edge, which are image features appearing on the screen
The formula will be described. The specific measuring method of moving direction and speed is
Although (b-1) is used as an example, the same procedure can be performed for (b-2) to (b-4).
It Measurement of edge moving direction and velocity with one-dimensional filter "Receptive field division + pole transformation + one-dimensional" in the flow of FIG.
Filter processing "is shown in Fig. 34 (b)," Receptive field division + pole
Transform + One-dimensional gradient filter + One-dimensional Gaussian
Filter processing ", the edge moving direction and speed
It is possible to measure the degree. Figure of the simulation result
48. 120⁰Moved the edge of 7 pixels horizontally
Correlation parameter C obtained by inputting two images before and after
The intensity of (ρ, θ, σ) is shown by contour lines. Basic correlation parameter
The sharp peak of data C (ρ, θ, σ) indicates the corresponding point of the edge.
The point (ρ_P, Θ_P, Τ_P), Moving direction = θ_P+90⁰= 210⁰  (Movement speed / DELTA) / Δρ = τ_P= 7 images and correct results were obtained. Where Δρ is the decomposition in the ρ direction
Noh, DELTA is the delay time.

Measurement of the moving direction and speed of the edge in which two-dimensional filters are mixed (biological binocular stereoscopic view) "Receptive field division + polar conversion + one-dimensional filter processing" in the flow of FIG. 39 is shown in FIG. 34 (c). By the "two-dimensional Gaussian filter + receptive field division + pole conversion + one-dimensional gradient filter processing" shown in (3), the moving direction and speed of the edge can be measured. This method is a two-dimensional
Two-dimensional second-order differential processing (two-dimensional convolution filter processing) is performed by the gas filter, and thereafter, each receptive field is divided, and each receptive field image is subjected to polar transformation. Two dimensions
Although the gas filter processing is equivalent to “one-dimensional filter processing after pole conversion”, the simulation result is omitted, but FIG.
The same moving direction and speed measurement results as 1 can be obtained. This method is similar to human stereoscopic vision, and has the advantage that the results of contour enhancement by a two-dimensional gas filter can be commonly used for shape recognition and the like.

(C) Polygonal figure / curve figure / random dot / texture binocular stereoscopic vision Polygonal figure / curve figure binocular stereoscopic vision Moving direction / speed measurement Similar to (b), projection in ρ direction Correlated parameter C _PRJ- ρ (θ, σ) C _PRJ- ρ (θ, σ) = ΣC (ρ, θ, σ) (ρ = 1,
2, ...) enables stereoscopic measurement of polygonal figures and curved figures. The method is the time correlation C _PRJ- ρ in (b).
It _suffices to replace (θ, τ) with the binocular correlation C _PRJ- ρ (θ, σ). In this method, the reliability is improved by the number of sides as compared with the binocular stereoscopic view from one straight line.

Now, the straight lines appear to be shifted in parallel in the input images of the left and right eyes as shown by SL _L and SL _R in FIG.
The difference between the positions of the straight lines is obtained as a displacement σ in the direction orthogonal to the azimuth θ of the straight line on the hypercolumn plane of ρ−θ. Therefore, the binocular parallax d in the horizontal direction can be expressed by the following expression d = σ / sin θ. From the above, when the horizontal parallax (binocular parallax) is d, the displacement amount σ of the straight line in the θ direction is given by the following equation σ = d · sin θ (36). When this relational expression is drawn on the θ-σ plane, it becomes a sine wave as shown in FIG. In other words, the sides Li, Lj, and Lk of the figure shown in FIG. 50 (a) captured by the left and right eyes are polar-transformed, and the hypercolumn images obtained by the polar transformation are correlated based on the equation (1). Correlation parameter C after processing
(Ρ, θ, σ) is calculated, and the correlation parameter C (ρ,
θ, σ) is projected in the ρ direction according to the equation (9) to obtain the correlation parameter C _PRJ- ρ (θ, σ) and displayed on the θ-ρ plane. Points Li ', Lj', Lk '
Is concentrated to.

Therefore, as in the case of (b-3), C _PRJ- ρ (θ, σ) is calculated from the binocular input image, and it is pole-transformed by the equation (34). As shown, each point on the sine wave is converted into a straight line passing through one point PK on the Vx and Vy planes.
If this peak point PK (Vx, Vy) is calculated, the true parallax σ ₀
Can be calculated by the following formula σ ₀ = Δρ · √ (Vx ² + Vy ² ) (37). The direction of parallax is always the direction that connects both eyes.
(Horizontal direction). If the viewing direction of an arbitrary figure is ξ and the distance between both eyes is d, the distance D to the arbitrary figure is calculated by the following equation: D = d ・ sin (σ ₀ + ξ) / sin (σ ₀ ) (38) it can.

FIG. 51 is a diagram for explaining a method of calculating a distance to an arbitrary figure by binocular stereoscopic vision, where L is a straight line and E _L and E _R are left and right eyes. If the parallax obtained above is σ ₀ , the viewing direction of an arbitrary figure is ξ, the distance between both eyes is d, and the distances from the left and right eyes to the arbitrary figure are D ₁ and D ₂ , the following equation D ₁ sin ξ = D ₂ sin (ξ + σ ₀ ) D ₁ cos ξ−D ₂ cos (ξ + σ ₀ ) = d holds. Equation (38) is obtained by finding D ₁ from this simultaneous equation.

FIG. 52 is a flow chart for measuring parallax and the distance to an arbitrary figure by binocular stereoscopic vision. The left and right eye image data are divided into receptive field images (steps 50a, 5a).
0a '), the receptive field images IM, IM' are subjected to polar conversion processing (steps 51a, 51a '), and the polar conversion result is subjected to one-dimensional filtering processing to be mapped on a dual plane (steps 52a, 52a'). , The correlation processing of the equation (1) is performed between the data mapped to the dual planes (step 5).
3) The calculated correlation parameter C (ρ, θ, σ) is stored in the correlation parameter storage unit (step 72). Then, using the equation (9), the correlation parameter C projected in the ρ direction
_PRJ- ρ (θ, σ) is obtained (step 75 ′) and stored in the correlation parameter storage unit on the θ−σ plane (step 7).
6 '). Then, the correlation parameter C _PRJ- ρ (θ, σ)
The polar conversion process is performed by the equation (34) (step 77 ′), and V
Find the peak point (intersection) on the x and Vy planes (step 78 ') and find the distance from the coordinate origin to this intersection.
The binocular parallax σ ₀ and the distance D to the arbitrary figure are calculated based on the equations (37) and (38) (step 79 ′).

FIG. 53 and FIG. 54 are explanatory diagrams of the simulation result of the random dot figure by binocular stereoscopic vision, in the case where the random dot figure different by 6 pixels is input to the left and right eyes. This random dot figure is a computer generated random dot stereogram.
The dot is drawn with one pixel, and the density is 50%. In FIG. 53, IM and IM ′ are receptive field images of the left and right eyes, and HCIM and HCIM ′ are receptive field images IM and IM ′.
Is a hyper-column image obtained by subjecting the result of the pole transformation to a two-dimensional plane of ρ-θ by subjecting the result of the pole transformation to one-dimensional filtering. Further, in FIG. 54, PRIM performs correlation processing between hypercolumn images, and the obtained correlation parameter C (ρ, θ, σ) is projected in the ρ direction, and the correlation parameter C _PRJ- ρ (θ , Σ), HGI
M is a polar transformation result on the Vx and Vy planes obtained by subjecting the correlation parameter C _PRJ- ρ (θ, σ) to the polar transformation process. A sine wave pattern SWV appears on the θ-σ plane, and Vx,
On the Vy plane, sharp peaks PK are extracted at 6 pixel positions in the horizontal direction from the origin. This peak point (Vx, Vy)
Thus, the binocular parallax σ ₀ and the distance D to an arbitrary figure can be calculated based on the equations (37) and (38).

Binocular Stereoscopic View of Random Dot Texture The above replacement (that is, the same method as above) enables binocular stereoscopic view of “random dot figure or texture figure”. As a result, it is possible to easily perform a binocular stereoscopic view of a plane having a fine pattern. This is psychologist Julesz
It is a model at the cell level of the fact that we can stereoscopically view binocular stereoscopic vision even if the shape is not known as random dots.

(D) Motion stereoscopic vision Linear motion stereoscopic vision The method for obtaining the velocity and moving direction of the moving object has been described in (b). With the same processing, when the image capturing means (camera or the like) moves, the distance (depth) to a straight line in space can be measured. That is, the correlation parameter C (ρ, θ, τ) is calculated by the equation (4) ′.
And the peak points (ρ _P , θ _P ,
When τ _P ) is obtained, the parallax between the current image (image after movement) and the previous image (image before movement) is calculated by τ _P · Δρ.
Ξ is the direction in which the straight line is visible, and V is the moving speed of the image capturing means.
If the delay of s and (4) ′ is DELTA, the distance to the straight line is
(38) below and it can be calculated by similarly following equation D = (Vs · DELTA) · sin (τ P · Δρ + ξ) / sin (τ P · Δρ) (39).

FIG. 55 is a flow chart for measuring the depth to a straight line in space by stereoscopic motion. The image data before the movement is delayed (step 49), and the image data a _t (ρ, θ) before the movement and the image data at _{t + DELTA} (ρ +) after the movement are delayed.
(τ, θ) is divided into receptive field images (step 50).
a, 50a '), the receptive field images IM, IM' are subjected to a polar conversion process (steps 51a, 51a '), and the polar conversion result is subjected to a one-dimensional filtering process to be mapped to a dual plane (steps 52a, 52a). ′), The correlation processing of the equation (4) ′ is performed between the data mapped to each dual plane (step 53), and the calculated correlation parameter C (ρ, θ, τ)
Is stored in the correlation parameter storage unit (step 72).
Then, the peak point (ρ _P , θ _P , τ _P ) of the correlation parameter C (ρ, θ, τ) is obtained (step 73), and the parallax τ _P ·
Using Δρ, the depth to the straight line in the space is calculated by the equation (39) (step 91).

Movement of arbitrary figure Moving stereoscopic image capturing means (camera, etc.) can measure the distance (depth) to an arbitrary figure in space. That is, the same processing as described in (b-3) is performed on each image before and after the image capturing means moves to determine the peak points on the Vx and Vy planes, and equations (35a) and (35b) are used. The true velocity V ₀ and the moving direction are calculated by. Assuming that the viewing direction of the figure is ξ, the moving speed of the image capturing means is Vs, and the delay of the equation (4) ′ is DELTA, the distance to the arbitrary figure is the following equation D = (Vs · DELTA) as in the equation (38).・ Sin (V ₀ · DELTA + ξ) / sin (V ₀ · DELTA) (40) can be used for calculation.

FIG. 56 is a flow chart for measuring the depth up to an arbitrary figure in space by motion stereoscopic vision. The image data before the movement is delayed (step 49), and the image data a _t (ρ, θ) before the movement and the image data at _{+ DELTA} (ρ) after the movement are delayed.
+ Τ, θ) is divided into receptive field images (step 50).
a, 50a '), the receptive field images IM, IM' are subjected to a polar conversion process (steps 51a, 51a '), and the polar conversion result is subjected to a one-dimensional filtering process to be mapped to a dual plane (steps 52a, 52a). ′), The correlation processing of the equation (4) ′ is performed between the data mapped to each dual plane (step 53), and the calculated correlation parameter C (ρ, θ, τ)
Is stored in the correlation parameter storage unit (step 72).
Then, the correlation parameter C _PRJ- ρ (θ, τ) projected in the ρ direction is obtained from the equation (10) (step 75), and θ−τ
The data is stored in the plane correlation parameter storage unit (step 7).
6). Then, the correlation parameter C _PRJ- ρ (θ, τ) is set to (3
The polar conversion process is performed according to equation (4) (step 77), and Vx, V
Find the peak point (intersection) on the y-plane (Step 7
8), the true velocity V ₀ and the true direction Φ ₀ are calculated based on the equations (35a) and (35b) (step 79). Finally, the depth to the arbitrary figure is calculated based on the equation (40).

(E) Generalization By generalizing the above, the correlation processing method of the present invention can be generalized as shown in FIG. That is, the input data is subjected to a polar conversion process and mapped to a dual plane of ρ-θ (step 501), and the data a (ρ, θ) obtained by the polar conversion is obtained.
Is subjected to new parameter addition processing, correlation processing, projection with arbitrary parameters, or polar transformation processing (projection in the 0 ⁰ or 90 ⁰ directions is the same as polar transformation), and as a result, a sinusoidal firing pattern is included on the plane. Data a (ξ ₁ ,
ξ ₂ , ...) or data including a firing pattern of a great circle on a spherical surface (step 502), and the firing pattern of a sine wave or a great circle included in the data is extracted by inverse pole transformation processing. And useful data is output (step 503). In addition, in the equations (1) and (4), "addition of a new parameter σ or τ" is performed for "correlation processing". Also, (9), (1
In the equation (0), “projection along the parameter ρ” is performed. Furthermore, in equation (19), projection in the 45 ⁰ direction on the ρσ plane is performed, and this projection is “polar transformation of the (ρ, σ) plane”.
And the 45 ⁰ direction component is extracted.

In the measurement of the moving direction and the velocity described in (b-2) and (b-3), the polar conversion processing is performed on two screen data which are different in time, and the data a in the dual plane of ρ−θ is obtained. (Ρ, θ) and b (ρ, θ) are obtained (step 501). Next, the moving speed parameter τ in the ρ-axis direction is introduced, the correlation processing is performed by the equation (4), and the correlation parameter C (ρ, θ, τ) obtained by the correlation processing is projected in the ρ direction by the equation (10). And a sinusoidal firing pattern (Fig. 40,
Projection data C _PRJ- ρ (θ, τ) including (see FIG. 41) is obtained (step 502). After that, projection data C _PRJ-
Reverse polarity conversion processing is performed on ρ (θ, τ) to extract a sinusoidal firing pattern (point giving a maximum point) and useful data (moving direction and velocity) is output (step 50).
3). "Inverse pole transformation" to extract sinusoidal firing patterns
When is expressed as an expression, it becomes as follows. That is, when δ () is a delta function and τ _X and τ _Y are velocity parameters in the X and Y axis directions, output (τ _X , τ _Y ) = ΣθΣτ {C _PRJ− ρ (θ, τ) · δ
It can be expressed as (τ + τ _X cos θ + τ _Y sin θ)}. Since the delta function δ () has meaning only at 0, the output (τ _X , τ _Y ) = Σθ {C _PRJ- ρ (θ, _{-τ X} cos θ-τ
_Y sin θ)}, and the contents of the inverse pole transformation are well understood. This calculation is performed in the embodiment.

Further, in the measurement of binocular stereoscopic vision described in (c), the polar conversion processing is performed on the two screen data captured by both eyes, and the data L (ρ, θ) in the dual plane of ρ−θ is obtained. ), R (ρ, θ) is calculated (step 50
1). Then, the parallax σ in the ρ-axis direction is introduced, the correlation processing is performed by the equation (1), and the correlation parameter C (ρ, θ, σ) obtained by the correlation processing is projected in the ρ direction by the equation (9). Projection data C _PRJ- ρ (θ, σ) including a sinusoidal firing pattern (see FIG. 50) is obtained (step 502). Thereafter, the projection data C _PRJ- ρ (θ, σ) is subjected to _{inverse pole} conversion processing to extract a sinusoidal firing pattern (point giving a maximum point) and useful data (parallax) is output (step 503). ). The "reverse pole transformation" for extracting a sinusoidal firing pattern is expressed as follows. That is,
When δ () is a delta function and σ _X and σ _Y are parallax parameters in the X and Y axis directions, output (σ _X , σ _Y ) = ΣθΣσ {C _PRJ− ρ (θ, σ) · δ
It can be expressed as (σ + σ _X cos θ + σ _Y sin θ)}. Since the delta function δ () has meaning only at 0, if the above equation is transformed, the output (σ _X , σ _Y ) = Σθ {C _PRJ- ρ (θ, _{-σ X} cos θ-σ
_Y sin θ)}. This calculation is performed in the embodiment.

In the above, in step 502 of FIG. 57, correlation processing or the like is performed to obtain a sinusoidal firing pattern. However, without performing correlation processing or the like, a sinusoidal firing pattern can be obtained only by polar conversion. May be possible. In such a case, in step 502, conversion processing such as shift or rotation is performed on the (ρ, θ) data obtained by the pole conversion in step 501, and the (ρ, θ) data obtained by the conversion processing is subjected to step 503. Useful data can be output by performing the inverse pole conversion process of and extracting the sine wave-shaped firing pattern (point giving the maximum point).
For example, if the circle of radius R shown in FIG.
As shown in (b), a sine wave shifted by R in the ρ direction is obtained. This sine wave has the center of the circle as (x ₀ , y ₀ ). Ρ = R−β ₀ cos (θ−α ₀ ) (41) β ₀ = √ (x ₀ ² + y ₀ ² ) α ₀ = arctan (x ₀ / y ₀ ). Therefore, the data obtained by the polar transformation is shifted by R in the ρ direction, and the data a obtained by the shift processing is
The center of the circle can be output by subjecting (θ, σ) to inverse polarity conversion processing to extract a sine-wave firing pattern (point giving a maximum point) (step 503).

FIG. 59 is a diagram for explaining a simulation result of circle detection. A black circle drawn on a white paper is subjected to Laplacian filter processing to obtain input image data, and the input image is divided into receptive field images to obtain a receptive field image. In the ρ-θ plane that has a sinusoidal firing pattern after the conversion process, (ρ,
θ) data is obtained, then the (ρ, θ) data is shifted by R in the ρ direction, and the data obtained by the shift processing is subjected to reverse polarity conversion. The intensity of ignition is represented by contour lines, and the positive part is shaded. In FIG. 59 (b), two parallel sine waves SN1 and SN2 are obtained by the polar transformation, considering the hypercolumn physiological knowledge “azimuth θ is in the range of 0 to π” (41). This is because the range of π to 2π in the formula is inverted to the negative side of ρ. As is clear from FIG. 59 (d), one sine wave fires as a sharp "negative peak PK" corresponding to the center of the circle, and the other sine wave fires as a weak ring RG with a radius of 2R. ing. Although the present invention has been described above with reference to the embodiments, the present invention can be variously modified according to the gist of the present invention described in the claims, and the present invention does not exclude these.

[0131]

As described above, according to the present invention, the polar transformation processing is performed on the input data to map it to the dual plane, or the polar transformation processing result is further filtered and mapped to the dual plane. After that, it is configured to measure the variables that define the relationship between the graphical features (eg, tangents) of the input data by performing correlation processing between the mapping data on the dual plane, and thus determining the corresponding parts of multiple graphics. Can be performed easily and accurately with a small amount of processing, and a binocular stereoscopic function (parallax, depth measurement, moving speed of an object, measurement of moving direction, etc.) can be realized. Further, according to the present invention, the polar conversion processing, the filter processing, and the correlation processing are performed on the receptive field image when one screen is divided into receptive fields which are small areas, so that the processing amount is remarkably reduced. it can.

Further, according to the present invention, when the receptive field image belongs to different screens captured by two or three cameras, the functions of binocular stereoscopic vision and trinocular stereoscopic vision can be realized, and the receptive field image can be realized. If the images belong to different screens in time, the moving direction and moving speed of the features (lines, corners, etc.) appearing in the receptive field can be measured. Therefore, it becomes possible to move while keeping the object in the center of the field of view, and it is possible to apply it to a mobile robot or an unmanned self-propelled vehicle. Further, when each receptive field image belongs to the screen before and after the movement when the image capturing means such as the camera is moved, the function of motion stereoscopic vision can be realized and the depth to the object can be measured. Further, according to the present invention, each receptive field image is an image of a different receptive field on the same screen,
Alternatively, by using the same receptive field image, it is possible to perform texture analysis for examining the degree of the same pattern on one screen.

Further, according to the present invention, the correlation processing between a plurality of receptive field images is performed for each same color or for each color difference signal.
Alternatively, since it is performed for each of the three color elements, it is possible to more reliably perform binocular stereoscopic vision, moving object gaze, and texture analysis. Further, according to the present invention, the corresponding line or gap in a plurality of figures is obtained by performing one-dimensional Gaussian filter processing after the pole conversion or by performing two-dimensional Gaussian filter processing before the pole conversion processing. It is also possible to perform one-dimensional gradient filter processing and one-dimensional Gaussian filter processing after the pole transformation, or two-dimensional Gaussian filter processing before the pole transformation processing and one-dimensional gradient filter processing after the pole transformation processing. By carrying out, it is possible to obtain the corresponding edges in the plurality of figures, and further it is possible to obtain the positions, orientations, parallaxes, moving directions, and moving speeds of these figure elements.

Furthermore, according to the present invention, when a plurality of receptive field images belong to spatially different screens, the correlation parameter Cθ (ρ, θ, σ) in the θ axis direction is calculated, or (ρ , Θ) plane correlation parameter C (ρ,
By calculating θ, σ ₁ , σ ₂ ) it becomes possible to extract the tangent line that moves while changing the azimuth. Further, according to the present invention, when the receptive fields of the plurality of image data belong to different screens in terms of time, the correlation parameter C (ρ, θ,
By calculating τ), the position of the tangent line that moves in parallel,
The azimuth and velocity can be quantitatively obtained, and the position, azimuth, and azimuth rotation velocity of the tangent line passing through the receptive field center can be quantified by calculating the correlation parameter Cθ (ρ, θ, τ) in the θ-axis direction. By further calculating the correlation parameter C (ρ, θ, τ ₁ , τ ₂ ) of the (ρ, θ) plane, the position, azimuth, and moving speed of the tangent line that moves while changing the direction. The rotation speed can be measured quantitatively.

Furthermore, according to the present invention, the correlation parameter C
(Ρ, θ, σ) is projected in the σ-axis direction indicating the spatial shift amount σ, in the ρ-axis direction indicating the tangent position, or in the θ-axis direction indicating the tangential direction, or By projecting in the two-axis direction, the memory capacity for storing the correlation parameter can be reduced by one axis or two axes, and moreover, by selecting the projection direction, a desired value of the tangent position, parallax, azimuth, etc. can be obtained. Obtainable.
Further, according to the present invention, the correlation parameter C (ρ, θ, τ)
Is projected in the τ-axis direction indicating the temporal shift amount (moving speed) τ, in the ρ-axis direction indicating the tangent position, or in the θ-axis direction indicating the tangential direction, or in any of these two axis directions. , The memory capacity for storing the correlation parameter can be reduced by one or two axes,
Moreover, by selecting the projection direction, it is possible to obtain a desired value among the tangent line position, azimuth, parallel movement speed, rotation speed, and the like.

Further, according to the present invention, the receptive field image is subjected to the polar transformation process to be mapped on the dual plane of ρ−θ, and the mapping data a (ρ, θ) of the dual plane and another dual plane. Precise filtering can be performed by extracting a set of coordinate data that is deviated by a predetermined amount in the ρ-axis direction from the mapping data b (ρ, θ) set in the plane, and calculating the sum of the products, Extraction of features that look the same with the left and right eyes "
It is suitable to be applied when performing “tracking the same feature as the previous image” or the like. Further, according to the present invention, for the mapping data a (ρ, θ) of the dual plane, another mapping data b (ρ,
θ) is shifted in the ρ-axis direction or the θ-axis direction and then subtracted, and the shift amount is sequentially changed and subtracted,
By using the obtained subtraction result as a correlation parameter,
It is possible to perform binocular stereoscopic vision and tracking of an object with an unclear contour by using a gradual change in brightness or hue as a clue.

[Brief description of drawings]

FIG. 1 is a diagram illustrating the principle of the present invention.

FIG. 2 is a diagram illustrating the principle of binocular stereoscopic vision according to the present invention.

FIG. 3 is an explanatory diagram of a spherical mapping (polar conversion of a spherical surface).

FIG. 4 is an explanatory diagram of pole conversion on a cylinder.

FIG. 5 is an explanatory view of the principle of the receptive field method.

FIG. 6 is an explanatory diagram of pole conversion in a hyper column.

FIG. 7 is a configuration diagram of an embodiment of the present invention.

FIG. 8 is a flowchart of correlation processing.

FIG. 9 is a configuration diagram of a pole conversion circuit.

FIG. 10 is a configuration diagram of a one-dimensional filter.

FIG. 11 is a first explanatory diagram illustrating a simulation result.

FIG. 12 is a second explanatory diagram illustrating a simulation result.

FIG. 13 is a third explanatory diagram illustrating a simulation result.

FIG. 14 is a basic flow chart of correlation filtering.

FIG. 15 is a flowchart of correlation filtering between spatially different screens.

FIG. 16 is a flowchart of correlation filtering between screens that are temporally different.

FIG. 17 is a flow diagram of correlation filtering between screens that differ temporally and spatially.

FIG. 18 is a flowchart of correlation filtering between receptive fields on the same screen.

FIG. 19 is a flowchart of correlation filtering within the same screen and within the same receptive field.

FIG. 20 is a flowchart of correlation filtering between different color images (three primary colors).

FIG. 21 is a flowchart of correlation filtering between different color images (three color elements).

FIG. 22 is a flow chart of correlation filtering between different color images (color difference signals).

FIG. 23 is a flowchart of ρ-axis correlation filtering with the same θ.

FIG. 24 is a flowchart of ρ-axis correlation filtering with different θ.

FIG. 25 is an explanatory diagram of θ-direction correlation.

FIG. 26 is a flowchart of θ-axis correlation filtering with the same ρ.

FIG. 27 is a flowchart of correlation filtering on the ρ-θ plane.

FIG. 28 is an explanatory diagram of a correlation parameter space.

FIG. 29 is an overall flowchart of filtering processing for projecting a correlation parameter.

FIG. 30 is a flowchart of a process of projecting a correlation parameter in the σ direction.

FIG. 31 is a configuration diagram of a natural filter.

FIG. 32 is a flowchart of difference type correlation processing.

FIG. 33 is a diagram showing various configurations of a line extraction filter.

FIG. 34 is a diagram showing various configurations of an edge extraction filter.

FIG. 35 is a diagram showing various configurations of a gap extraction filter.

FIG. 36 is a first explanatory diagram (binocular stereoscopic view of edges) showing simulation results.

FIG. 37 is a second explanatory diagram (a binocular stereoscopic view of edges) showing a simulation result.

FIG. 38 is an explanatory diagram of a moving direction and speed of a tangent line.

FIG. 39 is a flowchart for measuring a moving direction and a moving speed.

FIG. 40 is an explanatory diagram of detecting a moving direction of a corner and speed.

FIG. 41 is an explanatory diagram of a response of C _PRJ- ρ (θ, τ) to a polygon.

FIG. 42 is an explanatory diagram of extracting a sine wave by inverse Hough transform.

FIG. 43 is a flow chart for measuring a moving direction and a moving speed.

FIG. 44 is an explanatory diagram of polar conversion from a random dot figure.

FIG. 45 is a diagram illustrating a simulation result in measuring the moving direction / velocity of a random dot figure.

FIG. 46 is a diagram illustrating a simulation result in measuring the moving direction / velocity of a random dot figure.

FIG. 47 is an explanatory diagram of a simulation result (movement direction / speed of a line).

FIG. 48 is an explanatory diagram of simulation results (edge moving direction / velocity).

[Fig. 49] Fig. 49 is an explanatory diagram of a relationship between a line shift amount and parallax.

FIG. 50 is an explanatory diagram of binocular stereoscopic vision of an arbitrary figure.

FIG. 51 is an explanatory diagram of a distance calculation method by binocular stereoscopic vision.

[Fig. 52] Fig. 52 is a parallax / distance calculation flowchart for binocular stereoscopic vision.

[Fig. 53] Fig. 53 is an explanatory diagram of a simulation result by binocular stereoscopic vision of a random dot figure.

FIG. 54 is an explanatory diagram of a simulation result by binocular stereoscopic vision of a random dot figure.

[Fig. 55] Fig. 55 is a flow chart of depth calculation up to a straight line in motion stereoscopic vision.

FIG. 56 is a depth calculation flow chart up to an arbitrary figure by motion stereoscopic vision.

FIG. 57 is a flowchart in the case of generalization.

FIG. 58 is an explanatory diagram of circle detection.

FIG. 59 is a diagram illustrating a simulation result of circle detection.

FIG. 60 is a diagram illustrating a problem of binocular stereoscopic vision.

FIG. 61 is an explanatory diagram of depth calculation for binocular stereoscopic vision.

[Explanation of symbols]

14, 24 ·· Pole conversion unit 16, 26 ·· One-dimensional filter 17, 27 · · Dual plane of ρ-θ (hyper column memory) 30 · · Correlation processing unit

Claims (20)

[Claims] 1. ρ−θ obtained by subjecting input data to a polar conversion process.
Is mapped to the dual plane of and the (ρ, θ) data obtained by the polar transformation is subjected to additional processing of new parameters, correlation processing, projection with arbitrary parameters, or polar transformation processing, and a firing pattern with a sine wave shape on the plane. Correlation processing characterized by obtaining a firing pattern of a great circle on a sphere, and extracting a firing pattern of a sine wave or a great circle included in the data by inverse pole transformation processing to obtain useful data. method. 2. A plurality of input data are each subjected to a polar conversion process and mapped to a ρ-θ dual plane, or a polar conversion result is further filtered to obtain a ρ-θ dual plane. The correlation processing method is characterized by performing a correlation process on the data mapped on each dual plane and determining the relationship between the graphical features of each input data based on the correlation result. 3. The correlation processing method according to claim 2, wherein the input data is image data corresponding to a receptive field when one screen is divided into receptive fields which are small areas. 4. A plurality of input data are each subjected to a polar conversion process and mapped to a ρ-θ dual plane, or a polar conversion process result is further filtered to obtain a ρ-θ dual plane. , The position of the mapping data on the dual plane (ρ, θ) and the shift amount between the mapping data are used as element parameters to perform correlation processing to obtain the correlation result. And a variable that defines the relationship between the graphical features of each input data is measured based on the value of the element parameter that gives the characteristic correlation result. 5. The input data is image data of a receptive field when one screen is divided into receptive fields which are small areas, and the receptive field image data is subjected to polar conversion processing to be mapped on a dual plane. Then, by sequentially changing the shift amount in the ρ-axis direction or the shift amount in the θ-axis direction between each mapping data, the correlation process in the ρ-axis direction, or the correlation process in the θ-axis direction, or (ρ, θ)
The correlation processing method according to claim 4, wherein the correlation processing is executed in a plane. 6. The mapping data a (ρ,
θ) and other mapping data b (ρ, θ) are shifted in the ρ-axis direction or the θ-axis direction and then multiplied, and
6. The correlation processing method according to claim 5, wherein the shift amount is sequentially changed and multiplied, and the obtained multiplication result is used as a correlation result. 7. The mapping data a (ρ,
Other mapping data b (ρ, θ) is shifted by a predetermined amount in the ρ-axis direction or the θ-axis direction with respect to θ) and then subtracted, and the shift amount is sequentially changed and subtracted. The correlation processing method according to claim 5, wherein the subtraction result is used as a correlation result. 8. An input data representing a receptive field image when one screen is divided into receptive fields, which are small areas, is subjected to a polar conversion process and mapped to a dual plane of ρ−θ, or a polar conversion process. The result is further filtered and ρ−θ
Of the mapping data a (ρ, θ) of the dual plane and the mapping data b (ρ, θ) set in another dual plane with coordinate values deviated by a predetermined amount in the ρ-axis direction. Take out the existing pair,
The sum of products Ci (ρ, θ) = Σa (ρ + i, θ) · b (i, θ) (where i is changed and added) is calculated as a correlation parameter, and the mapping data of the input data is calculated using these correlation parameters. A correlation processing method characterized by extracting a graphical feature according to b (ρ, θ). 9. The mapping data a (ρ, θ), b (ρ,
9. The correlation processing method according to claim 8, wherein θ) is obtained by performing polar transformation on spatially different input data such as binocular stereoscopic vision and mapping the data into a dual plane of ρ−θ. 10. The mapping data a (ρ, θ), b
9. The correlation processing method according to claim 8, wherein (ρ, θ) is obtained by performing polar transformation on input data different in time and mapping the data into a dual plane of ρ−θ. 11. A receptive field image obtained by dividing an image input from two image input means into receptive fields is subjected to a polar conversion process to map it to a dual plane of ρ−θ, or a polar conversion. The processing result is further filtered and mapped to a dual plane of ρ−θ, and the mapping data a (ρ, θ) of the dual plane is
The other mapping data b (ρ, θ) is shifted in the ρ-axis direction and then multiplied, and the shift amount σ is sequentially changed and multiplied, and the obtained multiplication result is correlated parameter C (ρ,
θ, σ), the corresponding tangent line between each receptive field image is determined from the maximum point of the correlation parameter, and the parallax is measured based on σ that gives the maximum point. 12. The tangent line is obtained as a line or a gap by performing a one-dimensional Gaussian filter process after the polar conversion or a two-dimensional Gaussian filter process before the polar conversion process. The correlation processing method according to claim 11. 13. A one-dimensional gradient filtering process and a one-dimensional Gaussian filtering process after the polar transformation, or a two-dimensional Gaussian filtering process before the polar transformation process and a one-dimensional gradient filtering process after the polar transformation process. The correlation processing method according to claim 11, wherein the tangent line is obtained as an edge by applying 14. A receptive field image obtained by dividing an image input from two image input means into receptive fields is subjected to a polar transformation process to be mapped on a dual plane of ρ−θ, or a polar transformation. The processing result is further filtered and mapped to dual planes of ρ−θ, and one mapping data a (ρ, θ) of the dual plane is mapped to the other mapping data b (ρ, θ). Is shifted in the ρ-axis direction and then multiplied, and the shift amount σ is sequentially changed and multiplied, and the obtained multiplication result is correlated to the correlation parameter C.
(Ρ, θ, σ), and after converting the correlation parameter C (ρ, θ, σ) into C _PRJ −ρ (θ, σ) projected in the ρ direction, the sine in the (θ, σ) plane A correlation processing method characterized in that a point having the maximum amplitude of a wave pattern is extracted by inverse pole transformation, and σ giving the maximum amplitude is used to calculate the binocular parallax and / or the depth to a target straight line. 15. A receptive field image obtained by dividing two temporally different images into receptive fields is subjected to a polar conversion process and mapped to a dual plane of ρ−θ, or a result of the polar conversion process. To the dual plane of ρ−θ, and the mapping data a _t (ρ, θ) at the current time of the dual plane is obtained.
, The mapping data a _{t + 1} (ρ, θ) at the next time is ρ
After shifting by a predetermined amount in the axial direction, multiplication is performed, and the shift amount τ is sequentially changed and multiplied, and the obtained multiplication result is used as a correlation parameter C (ρ, θ, τ), and the maximum point C of the correlation parameter is obtained. _{(ρ P, θ P, τ} P) values [rho _P element parameter that gives, theta _P, and determines the corresponding tangent between the receptive field images from tau _P, to measure a moving direction and moving speed of the corresponding tangent Correlation processing method characterized by. 16. The following formula Φ = θ_P+90⁰  V = k₁・ Τ_P (k₁The moving direction Φ and the moving speed V are calculated by
16. The correlation processing method according to claim 15, wherein: 17. The parameters of local maximum points of two tangents forming a corner of the receptive field image are (ρ _i , θ _i , τ _i ),
When (ρ _j , θ _j , τ _j ), the following equation Φ = arctan {(τ _i sin θ _j −τ _j sin θ _i ) / (τ _i cos θ _j −τ
The moving direction Φ and the moving speed V of the corner are calculated by _j cos θ _i )} V = k ₂ · τ _i / sin (Φ-θ _i ) (k ₂ is a constant). Correlation processing method. 18. The two images are an image in which an image capturing unit captures a target straight line at a certain time and an image in which the image capturing unit captures a target straight line after a predetermined time has passed, and the element parameter τ 16. The correlation processing method according to claim 15, wherein the depth to the target straight line is calculated using _P and the moving speed Vs of the image capturing means. 19. A receptive field image obtained by dividing two temporally different images into receptive fields is subjected to a polar transformation process and mapped to a dual plane of ρ−θ, or a result of the polar transformation process. To the dual plane of ρ−θ, and the mapping data a _t (ρ, θ) at the current time of the dual plane is obtained.
, The mapping data a _{t + 1} (ρ, θ) at the next time is ρ
After shifting by a predetermined amount in the axial direction, multiplication is performed, and the shift amount τ is sequentially changed and multiplied, and the obtained multiplication result is set as a correlation parameter C (ρ, θ, τ), and the correlation parameter C (ρ , Θ, τ) is converted into C _PRJ −ρ (θ, τ) projected in the ρ direction, and then (θ, τ)
A correlation processing method characterized in that a point having the maximum amplitude of a sine wave pattern in a plane is extracted by inverse pole transformation, and a moving direction Φ and a moving speed V are calculated using θ and τ which give the maximum amplitude. 20. The two images are an image in which the image capturing means captures the target graphic at a certain time and an image in which the image capturing means captures the target graphic after a predetermined time has passed, and the moving speed V 20. The correlation processing method according to claim 19, wherein the depth to the target straight line is calculated by using the moving speed Vs of the image capturing means.