JPH0159619B2 - - Google Patents

Description

[Detailed Description of the Invention] [Technical Field of the Invention] The present invention relates to a mapping-based information extraction method for extracting information such as reproduction of line segments and intersections of line segments from an original image by mapping, and in particular, a method for extracting information by mapping after mapping. This invention relates to an information extraction method using mapping that is improved so that it is easier to extract the correlation between mapping functions on a mapping surface.

[Technology background]

In recent years, object recognition technology has been actively developed to enable machines to perform functions similar to those of the human eye.

In such object recognition, a method is used in which the object is grasped as an image and information such as the object's characteristics is extracted from this image to recognize the object.

For this reason, it is generally necessary to image the object with an imaging means such as a television camera, capture the original image, and perform electrical processing based on this image to extract information.

On the other hand, such original images often contain noise, blur, or blur, and it is necessary to remove these noise components and extract structural lines and the like that constitute the original image.

[Conventional technology and problems]

As a method for extracting such an image, an information extraction method using mapping has been known.

Mapping is the concept of generating a line by applying a certain function (mapping function) to a point. When mapping is performed on a point group, a line group is generated, and from the mutual relationship of the line group, the mutual relationship of the point group is Information is extracted by finding relationships. Therefore, since mapping can condense dispersed information into a single point, it is widely used in the field of image processing, such as for detecting straight lines.

As methods for these mappings, mapping methods using Hough transform, spherical mapping methods, and the like have been proposed.

Information extraction by such mapping is performed as shown in FIG. That is, points are extracted from the target surface, a mapping function SF corresponding to the extracted point P is generated, and the generated mapping function SF is written on the mapping surface. After mapping is performed for all target points on the target surface, information is extracted by examining the mutual function of the mapping functions on the mapping surface, that is, the intersection of the mapping functions.

In this type of information extraction method using mapping, a method is used in which the mapping surface on which the mapping function is written (recorded) has a structure (quantization structure) and only the intersections of the frame of that structure and the mapping function are recorded. Ta. For example, when the mapping plane has a quantization structure of an orthogonal coordinate system as shown in FIG. 2A, the intersection of the mapping function SF with the frame of this coordinate system is recorded. However, in such conventional methods, due to quantization errors on the mapping surface, even though the mapping functions SF1 and SF2 intersect as shown in FIG. 2B, the intersection point of the intersection is not recorded well on the mapping surface. There was a problem in that it was difficult to reproduce and extract the intersections of mapping functions from the mapping surface.

[Purpose of the invention]

SUMMARY OF THE INVENTION An object of the present invention is to provide an information extraction method using mapping that allows easy extraction of intersections between mapping functions in consideration of the quantum structure of the mapping surface.

[Structure of the invention]

In order to achieve the above object, a first aspect of the present invention generates a mapping function corresponding to the coordinates of the extraction point on the target surface for each extraction point, writes the generated mapping function on the mapping surface, and An information extraction method using mapping that extracts information meant by each extraction point from the mutual relationship of mapping functions written on a mapping surface, wherein the mapping function is written with a predetermined width on the mapping surface. It is characterized by

Further, the embodiments of the first aspect of the present invention are listed as follows.

In order to give the mapping function a predetermined width, the generated mapping function is characterized in that the generated mapping function has a predetermined width.

When writing the mapping function having the width on the mapping plane, the writing is performed at the end point position of the width.

In order to write to the end point position of the width, two mapping functions indicating the end point positions are generated as the generated mapping functions.

Or, the method is characterized in that a differential value is given to the end point position.

The method is characterized in that the differential value written on the mapping surface is integrated and rewritten on the mapping surface.

In order to write the mapping function with a predetermined width, the coordinate frame of the mapping plane has a width.

Next, in the second aspect of the present invention, a mapping function corresponding to the coordinates of the extraction point on the target surface is generated for each extraction point, the generated mapping function is written on the mapping surface, and the mapping function is written on the mapping surface. This is an information extraction method using mapping that extracts information meant by each extraction point from the mutual relationship between mapping functions, in which the mapping function has a predetermined width, and on both sides of the width there are areas with opposite polarities to the area of the width. It is characterized in that writing is performed on the mapping surface with a suppression area of .

Further, the embodiments of the second invention of the present invention are listed as follows.

In order to make the mapping function have a predetermined width region and a suppression region, the generated mapping function is provided with a predetermined width region and a suppression region.

When writing the mapping function having the width region and the suppression region on the mapping plane, the writing is performed at end point positions of the width and suppression region.

In order to write to the end point positions of the width and suppression area, four mapping functions indicating the end point positions are generated as the generated mapping functions.

Or, the method is characterized in that a differential value is given to the end point position.

The method is characterized in that the differential value written on the mapping surface is integrated and rewritten on the mapping surface.

[Embodiments of the invention]

Hereinafter, the present invention will be explained in detail with reference to Examples.

3 and 4 are explanatory views of the principle of the first aspect of the present invention. Conventionally, the reason why intersection points cannot be reproduced (extracted) well is because only one point is recorded for each mapping function in each coordinate system frame. Therefore, in the present invention, the intersection of the frame of the coordinate system and the mapping function is not set as a point, but is given a range. That is, the mapping function allows recording of multiple points in each coordinate frame.

One way to provide this range is to give the mapping function SF itself a width H and write it on the mapping surface, as shown in FIG. 3A. By doing this, the intersection point CP with mapping functions SF1 and SF2 is written by both mapping functions SF1 and SF2, as shown in Figure 3B, so it has a larger weight than the others, as shown in the double mark. , the intersection point CP can be extracted well depending on the size of the weight.

Another method is to virtually give a width H _x to the frame of the coordinate system of the mapping plane shown in Figure 4A, and use the mapping function SF
If it passes within the width of this frame, recording is performed in that frame, and the mapping function is recorded with a width. By doing this, the intersection of the mapping functions SF1 and SF2 in FIG. 4B is the grid point CP1 and CP2.
Both mapping functions SF1 and SF2 are written to the grid points CP1 and CP2, and intersection points can be easily extracted in the same way.

FIG. 5 is a diagram showing the recording state of such a mapping plane. When the mapping function SF crosses Y1 to Y4 of the X-Y coordinate system as shown in FIG. 5A, the mapping plane
- In the memory 10 of the Y coordinate system, mapping functions are provided in the X direction for each coordinate frame Y1 to Y4 of the Y axis, as shown in FIG. 5B.
A plurality of bits (four in the figure) corresponding to the width of SF are written according to their positions. In this way, by writing the mapping function on the mapping plane with a width, if one mapping function is captured, each coordinate frame Y1 to Y4 on the Y axis that the mapping function crosses will have multiple points instead of one point. A recording is made to facilitate replay of the intersection.

On the other hand, recording (writing) a plurality of points in this way requires more writing time, and recording the entire width of the mapping function is not preferable from the viewpoint of execution speed, recording amount, etc. Regarding the improved version of this point, the 6th section
This will be explained with reference to FIGS. 7 and 8.

As shown in Figure 6A, if the coordinate frame Yn with the Y coordinate of the X-Y coordinate has a width from Xl to , since a large amount of recording is required, information is recorded only at the end points of the width, as shown in FIG. 6B. In other words, if the weight of the mapping function is "1", "+1" is added to the start point (position) Xl among the end points of the width as start information (weighted addition), and end information (weighted addition) is added to the end point (position) Xn. Write “-1” as the addition). That is,
Write the differential value obtained by differentiating FIG. 6A. As a result, regardless of the width, only two points need to be written, and increases in execution speed and recording amount can be minimized. In order to reproduce the state shown in FIG. 6A using this recorded content, the state shown in FIG. 6A can be reproduced as shown in FIG. 6C by sequentially integrating the state shown in FIG. 6B from the left.

FIG. 7 is an explanatory diagram of an example in which a plurality of mapping functions are written in the same manner. As shown in Figure 7A, the coordinate frame Yn is
written to Xl, X _l+1 , Xm, X _o-1 , Xn, and
In the case _where X _l+1 , Write "+1" to both ends Xl, "-1" to X _o-1 , and write the mapping function SF
All you have to do is write "+1" to both ends of 2, Xl ₊₁ , and "-1" to Xn. If we integrate the data in Figure 7C sequentially from the left in the same way as Figure 6C, Figure 7C shows Figure 7A.
state can be played.

Similarly, it can be applied to multivalued mapping functions.

FIG. 8 is an explanatory diagram when applied to a multi-value _example , and as shown in _FIG .
If they intersect at Yn, the coordinate frame Yn will be in the state as shown in Figure 8B, but according to the end point recording method shown in Figures 6 and 7, as shown in Figure 8C, "+w ₁ " is set at the starting end point of sequential mapping function SF1, "+w ₂ " is set at the starting end point of mapping function SF2, mapping function SF1
What is necessary is to record "-w ₁ " at the end point of mapping function SF2 and "-w ₂ " at the end end point of mapping function SF2.

Instead of writing the end point explained in Figures 6 to 8, the value (weight) of the area (between the start end point and the end end point) and the size of the area (from the start end point to the end end point) are written to (the position of) the start end point. You may also write the number of bits up to

FIG. 9 is a block diagram for realizing the end point recording method shown in FIGS. 6 to 9. In the figure, reference numeral 10 is a mapping memory, which is the same as that described in FIG. 5 and subsequent figures. Reference numeral 11 denotes a mapping function generation circuit, which generates a corresponding mapping function from the extraction point P, and calculates the coordinates of the starting end point and ending end point of the width for each of the coordinate frames Y ₁ to Yn of each mapping function. It stores coordinates.

To explain the operation of the block configuration in FIG. 9, the mapping function corresponding to the coordinates of each extraction point P is stored in the mapping function generation circuit 11, so when the extraction point P is input, the corresponding mapping function is selected. mapping memory 10 for each coordinate frame of the mapping function.
The coordinates of the upper start end point and end end point are sequentially given to the mapping memory 10, and start or end information is written to the corresponding coordinates in the mapping memory 10. In the case of a weighted mapping function, separately given weighted end point information is written.

FIG. 10 is a diagram for explaining the effects of the present invention, and shows an example in which three mapping functions intersect in the coordinate frame Yn.

In the conventional mapping function writing method shown in FIG. If the mapping function of the invention is written with a width, the peak level will rise near the intersection as shown in FIG. 10B, and the slope will become steeper, making it easier to detect the peak position. Incidentally, FIG. 10C is according to the second aspect of the present invention and will be described later.

Next, the second invention of the present invention will be explained.

FIG. 11 is an explanatory diagram of the principle of the second invention of the present invention. As shown in FIGS. 11A and 11B, suppression regions SFa and SFb of opposite polarity to the mapping function SF are provided on both sides of the mapping function SF having a width. In other words, there are suppression regions SFa on both sides of the original SF that has width as a mapping function,
Generate the one with SFb added.

Then, as shown in FIG. 11C, the writing result of multiple mapping functions with widths and suppression areas is as follows:
At the intersection (focal point), it is possible to obtain the effect of having a width as described above, and around the intersection, the mapping functions cancel each other out due to the suppression area, so that it is possible to obtain a mapping result in which the intersection is emphasized. In other words, since the mapping function SF has a width, line-to-line interference between the mapping functions occurs, and the mapping function is also written in extra locations below the intersection point. A suppression area is provided to offset the results. In FIG. 11C, around the intersection point CP, cancellation is performed in the dotted line region of the figure by the suppression region.

As shown in FIG. 11A, the weight of this suppression area is determined by the suppression areas SFa and SFb having the same width as the original area SF.
If the weight w of the original region SF is given a weight of −w/2, the increase in the weight of the mapping function SF provided with the width can be offset as a whole.

To explain the effect of the second invention with reference to FIG. 10C, by providing the suppression area, FIG. 10B
Compared to the mapping function in which a width is simply provided as shown in FIG. 1, the offset level is reduced and a steeper peak with emphasized intersection points can be obtained. Therefore, it becomes easier to extract intersection points. Writing of such a mapping function with suppression region will be explained with reference to FIG.

As shown in FIG. 12A, the left suppression region SFa added to the mapping function in the coordinate frame Yn is from Xl to X _n-1 ,
The right suppression region SFb is set to X _p+1 to Xn, and its weight is set to -a. On the other hand, the domain SF of the original mapping function
is from Xm to Xp and the weight is p.

As in the cases explained in FIGS. 6 to 8, writing in all these areas increases the writing time and recording amount, so as shown in FIG. 12B, the differential form of FIG. write to. That is, the weight of the suppression area is set to the coordinate Xl, which is the starting point of the suppression area.
a” to the coordinates Xm that is the starting point of the mapping function
+b", "-(a+b)" is written to the coordinate Xp which is the end point of the region SF of the mapping function, and "a" is written to the coordinate Xn which is the end point of the suppression region SFb. This means that
The principle is the same as the endpoint recording method explained in FIGS. 6 to 8, and values based on differentiation are recorded in FIGS. 6 to 8 as well. However, since the end point of the suppression area SFa and the start point of the mapping function area SF are adjacent to each other, the ending point of the suppression area SFa is not recorded, and instead the differential value is recorded at the starting point of the mapping function area SF. I am writing. Starting point of suppression region SFb and region of mapping function
The relationship with the end point of SF is based on the same principle, and the differential value is written at the end point of area SF.

By doing this, as a mapping function, the starting point (address) of one suppression area SFa,
Four points are generated: the start point and end point (address) of the width region SF of the mapping function, and the end point (address) of the other suppression region SFb, and a differential value is separately assigned and written to the mapping memory. Just put it in.

FIG. 13 is a block diagram for realizing the second invention, and the same parts as in FIG. 9 are indicated by the same symbols. 11' is a mapping function generation circuit;
It stores the mapping functions corresponding to the coordinates of the extraction point P, and the start address Xl of the suppression area, the start address Xm of the width area, the end address Xp of the suppression area in each coordinate frame Y ₁ to Yn of each mapping function, and the start address Xl of the suppression area It stores the end address Xn.

To explain the operation of the configuration shown in FIG. 13, the extraction point P
When the coordinates of are given to the mapping function generation circuit 11',
The corresponding mapping function is selected, and for each coordinate frame of the mapping function, the start address Xl of the suppression area on the mapping memory 10, the start and end addresses Xm of the width area,
Xp and the end address Xn of the suppression area are given to the mapping memory 10, and the above-mentioned differential value is written to the corresponding address (coordinates) of the mapping memory 10.

Next, examples in which the present invention is carried out by spherical mapping, mapping by Hough transform, etc. will be explained.

Typical information extraction methods using mapping include one using Hough transform and one using spherical transformation. Use a circle. The mapping procedure is to convert each point on the target surface into a mapping function, write it on the mapping surface, and calculate the information contained in each point on the target surface from the mutual relationship of the written mapping functions (for example, their intersection points). For example, it extracts whether a straight line is formed or the position, direction, etc. of the straight line.

Below, the case using spherical mapping will be explained first, and then the case using Hough transform will be explained.

The basic procedure of spherical mapping will be explained below. first,
Project the original image onto a spherical surface (spherical projection). next,
For each point of the original image projected onto the spherical surface, a great circle (mapping function) is drawn with that point as the center. Furthermore, information is extracted from the interrelationships of the drawn great circle groups.

By following these steps, it is possible to obtain the normal vector of the plane that includes each point of the original image and the center of the sphere, and it is also possible to extract simultaneous line segments and to reproduce interrupted or distorted line segments. Become. Furthermore, by mapping the change information of the projection on the spherical surface to one point on the spherical surface, the locus of the point-like object moving in a straight line is extracted. Furthermore, the projections of a plurality of line segments on the spherical surface are each mapped to one point on the spherical surface, and then these mapped points are mapped again to reproduce the original intersection points of the line segments.

FIGS. 14 and 16 are explanatory diagrams of spherical projection, and FIGS. 17 and 18 are explanatory diagrams of the principle of spherical mapping.

For example, as shown in FIG. 14, while plane projection projects an external solid 31 onto a plane 32, spherical projection projects the solid 31 onto a spherical surface 33. When the solid body 31 is photographed using a camera with an ordinary lens, it becomes a planar projection, but when a spherical camera with a spherical lens such as a fisheye lens is used, it becomes a spherical projection onto a spherical surface 33. In addition,
In this case, the camera can be considered to be located at the center of the spherical surface 33. In the following description, the radius of the sphere formed by the spherical surface 33 is assumed to be 1, that is, the radius is projected onto a unit sphere.

As shown in FIG. 15, when the straight line 36 on the Euclidean plane 35 is projected onto the spherical surface 33, the spherical surface 3
3, it becomes a part of the great circle 37. large circle 37
In other words, can be defined as a straight line on the spherical surface.
Similarly, a point on the plane 35 corresponds to a pole on the spherical surface 33, which is defined as a point. Due to this spherical projection, spherical geometry, especially Riemann's spherical non-Euclidean geometry, is applied instead of so-called Euclidean geometry.

According to this spherical projection, as shown in FIG. 16A, parallel lines 38 that do not intersect on the Euclidean plane 35 intersect within a finite range at an intersection 39 on the spherical surface 33. Of course, as shown in FIG. 16B, the straight line 40 intersecting on the plane 35 is
They also intersect at the top. Due to the above properties,
According to spherical projection, the intersection of parallel lines at infinity can be measured in terms of latitude and longitude. This property that the intersection of parallel lines at infinity can be measured is the key point of three-dimensional measurement using spherical mapping, which will be explained below.

In the case of the present invention, the image data captured by the spherical camera is not only directly used as measurement data as described above, but also processed by converting the data by dual mapping on the spherical surface, that is, spherical mapping, as described below. be made to do.

FIG. 17 shows an example in which points are made to correspond to straight lines (great circles on the spherical surface) by spherical mapping. As shown in FIG. 17A, a point P in the three-dimensional space is projected onto a point P' on a spherical surface 33 by spherical projection. Point P' is further mapped onto straight line l, which is a great circle, by spherical mapping. The camera, which is the robot's eye, is located at the origin O.
It is located in The mapping to the straight line l is shown in Figure 17B.
As shown in the figure, it is given as corresponding to the cut of the spherical surface 33 by a plane perpendicular to the line segment connecting the point P' and the origin O. In this way, by spherical mapping, the external point P is expressed as the straight line l on the spherical surface 33.
It will be mapped to .

FIG. 18 shows an example in which straight lines are made to correspond to points determined by a group of straight lines using spherical mapping. The straight line L in the three-dimensional space is projected onto the straight line L' on the spherical surface 33 by spherical projection. Points P ₁ , P ₂ , on straight line L
P ₃ ,... are projected to points P ₁ ′, ₂ ′, P ₃ ′, etc. on the straight line L′, respectively. Each of these points P ₁ ′, P ₂ ′, P ₃ ′... is
By spherical mapping, the straight line groups l ₁ , l ₂ , l ₃ , which are great circle groups,
..., but these great circles intersect at two points (diagonal points) that are symmetrical about the origin O. These two points are 1
Think of it as a point, let it be represented by one side, and call it the S point hereafter in the sense of mapping. In other words, the information on the straight line L is concentrated into one point S by spherical mapping. Note that the above P ₁ , P ₂ , P ₃ , ...P ₁ ′, P ₂ ′, P ₃ ′, ... and l ₁ , l ₂ , l ₃ , ... are conceptually continuous, but
In reality, it corresponds to discrete positions depending on the configuration of the light receiving element and image memory.

If the vector from the origin O to the point S is represented by S, then S is the normal vector of the plane containing the straight line L and the origin O. With this method, S can be measured,
Since this S includes three-dimensional information, it has a great feature that other information extraction methods such as Hough transform or least squares method do not have.

FIG. 19 is a diagram for explaining line segment extraction based on the spherical mapping. As shown in FIG. 19, according to spherical mapping, a line segment L' onto which an external line segment L is projected is mapped to point S. The arrow in FIG. 19 means the mapping from the line segment L', which is a part of the great circle, to the point S. Therefore, if the presence or absence of a line segment is detected by the presence or absence of the S point on the mapping sphere, the line segment can be extracted with a high SN ratio.

FIGS. 20 and 21 are diagrams for explaining an example of reproducing interrupted or distorted line segments. Images captured by a camera may be distorted due to noise, or may become partially blurred due to the background. As shown below, information is concentrated using spherical mapping, and line segments with discontinuities or distortions are reproduced. If the input image is a projection of a line segment that is part of an external straight line, even if there are some breaks or distortions, these spherical maps will be true, as shown in Figure 20, for example. They will gather near the S point. That is, the great circle for each point on the line segment passes near point S. Therefore, by regarding the position of the peak where the great circles are most concentrated as point S and performing inverse mapping from the point to the line as shown in FIG. 21, the line segment will be reconstructed. In addition, the 21st
In the figure, a line segment in the three-dimensional space corresponding to a line segment on the spherical surface 33 is shown, but this direction vector cannot be determined. It is reproduced only on the spherical surface 33. This direction vector can be determined, for example, by eye movement or parallel illumination, as described below.

FIG. 22 is a diagram for explaining detection of a moving object. For example, projected images are stored in an image memory at regular time intervals, and the difference between the latest one and the one moments before is taken. In this way, changes in the image can be extracted.
Once the changes in the projected image on the spherical surface 33 have been extracted, the mappings of the points of change are accumulated one after another. Now, if the point-like object is moving in a straight line as shown in FIG. 22, the cumulative signal mapped near the S point increases. Therefore, the peak position is regarded as the S point, and the same inverse mapping as in the case of FIG. 21 is performed. As a result, the locus of the moving object on the spherical surface 33 can be extracted. Furthermore, with this method,
Multiple moving objects can be detected simultaneously.
It also has the characteristic that the more the object moves, the larger the peak becomes, which improves the signal-to-noise ratio.

Similar to the reproduction of line segments, the reproduction of intersection points is also possible as described below with reference to FIG. Second
3. As shown in Figure 3, point S of line segment P is S _P and point S of line segment Q is
Let the point be S _Q. Also, let l be the great circle that passes through S _P and S _Q. If we set the normal of the plane containing l at the origin O, we get
It intersects at the intersection of P' and Q' on the spherical surface 33. This point will be called SS in the sense of mapping of mappings. That is, by mapping the S point of each line segment again, the intersection point SS can be reproduced. Information on each line segment is accumulated at each S point, and the information on these S points is
Since the points are accumulated at the SS point, it can be considered that all the constituent points of the original line segment are accumulated at the SS point. Therefore, it is possible to reproduce the intersection with a high signal-to-noise ratio. Note that the same applies when three or more line segments intersect at one point.

As a special case, FIG. 24 shows an example where the line segments are parallel. In the world of artifacts,
For example, the sash on the window, the bookshelf, and the appearance of a building.
There are many line segments that are parallel to each other. Parallel lines can generally be said to intersect at a point at infinity. It is difficult to find such an intersection of points at infinity using normal means. However, according to this method, as can be seen from FIG. 24, the intersection point SS can be found on the spherical surface 33 in the same way as in the case of FIG. 23. Furthermore, the greater the number of line segments that are parallel to each other, the higher the signal-to-noise ratio can be to reproduce the intersection. In addition,
Whether or not the intersection point SS is an actual parallel line can be determined as follows.

The camera, which is the robot's eye, is moved in parallel.
Since the point at infinity is constant, it corresponds to parallel lines
The SS point does not move even if the eye is moved in parallel.
Therefore, if you move your eyes in parallel and the SS point does not move, you can say that the lines are parallel. Conversely, if the SS point moves due to parallel movement of the eyes, it can be determined that the lines are not parallel. Note that the same determination can be made using two cameras instead of moving the eyes. If the SS points are the same in two cameras, they correspond to parallel lines.

Additionally, the following knowledge can be used to determine parallel lines. For example, in the world of artificial objects, there are many rectangular parallelepiped shapes, so the number of edges that intersect at one point is often three or less. That is, four line segments do not intersect at corners of a rectangular parallelepiped. Therefore, 4
When more than one line segment intersects at one point, it is determined that they are parallel and there is almost no error. Furthermore, the intersection points of the edges, which are the vertices of the polyhedron, are in contact with the surfaces that sandwich those edges. Therefore, when the SS point is not more than a certain distance away from the region that sandwiches the corresponding line segment, it may be determined that the line segments are not parallel but are edges that intersect at one vertex.

FIG. 25 is an explanation of the basic procedure of the above-mentioned spherical mapping. First, examine the spherical surface 33 (actually memory),
A point P on the spherical surface 33 is extracted. Next, a corresponding mapping function (great circle) is generated from the position of point P, and great circle l is written on another spherical surface 33' (actually, memory). The intersection point of each great circle on the spherical surface 33' is found, and the relationship information between each point on the spherical surface 33 is extracted based on this position.

If the spherical surface 33 is a spherical projection of the original image in this way, line segments can be extracted and reproduced as shown in FIGS. 18 to 21, and the spherical surface 33 is mapped to
3', it is possible to extract intersection points and surfaces of line segments as shown in FIGS. 22 and 23. Next, the principle of mapping function generation used in the present invention will be explained.

As described above, information can be easily extracted using spherical mapping, but for this purpose it is necessary to generate a mapping function from points on the spherical surface. This mapping function is the 26th
As shown in the figure, when points P ₁ to _{P 4} are different, points L ₁ to _{L 4} are different. Therefore, it is necessary to generate a different mapping function for each point, and if the coordinates of a great circle were calculated one by one from the coordinates of each point, it would take time to generate the mapping function, making it possible to achieve high-speed mapping. Can not.

For this reason, it is possible to register the pattern of the mapping function (great circle) in advance in memory (ROM) as shown in Figure 27, but it is possible to register the mapping function for all possible points on the original image. This requires a huge amount of memory, which is not a good idea.

Therefore, in the mapping function generation method used in the present invention, the mapping speed is improved by devising the configuration of the mapping surface, and the memory capacity is reduced.

28 to 33 are diagrams explaining the principle of mapping function generation used in the present invention.

As shown in FIG. 28A, in the present invention, a spherical surface 33', which is a mapping surface, is connected to concentric latitude lines ₁ to 1 with the pole as the center.
Divide into r _o . Therefore, the mapping plane memory is also divided into concentric circles as shown in FIG. 28B. That is, it is set in polar coordinate format (r, θ). In this polar coordinate format, if the great circle corresponding to point P ₁ is l ₁ as shown in Fig. 29A, then point P ₁ as shown in Fig. 29B
The great circle _l2 at point _P2 , which is on the same concentric circle as , is the great circle _l1 rotated around the origin O. Furthermore, the amount of rotation depends on the distance (rotation angle) between points P ₁ and P ₂ on the concentric circle. Therefore, great circles corresponding to points on the same concentric circle can be obtained by rotating one great circle.

Therefore, by setting a reference line as shown in FIG. 30A, setting the intersection of the reference line and each concentric circle r ₁ to _{r o} as a reference point Pma, and registering one great circle lma corresponding to this reference point Pma, P(r, θ) at a position rotated by θ on the same concentric circle from the reference point Pma as shown in Figure 30B.
The great circle l corresponding to the point is found by rotating the reference great circle lma by θ.

The pattern of this reference great circle lma can be registered by storing the intersection points (circles in the figure) with each of the concentric circles r ₁ to _{r o} as shown in FIG. 31A. For example, if each concentric circle is divided equiangularly and quantized, and each concentric circle is expanded into a straight line, a quantized surface of a matrix of concentric circles r ₁ to _{r o} directions and the θ direction as shown in FIG. 31B is obtained. Then, by marking the intersection points with each concentric circle, the pattern of the reference great circle can be registered. For example, in the great circle lma, there is no intersection with concentric circles r ₁ to r ₃ , so no mark is attached, and since there is one intersection with r ₄ , a mark is placed at that position, and the same goes for the corresponding positions up to r _o . A mark will be added. Since the processing is actually done on the memory side, the bit “1” is used instead of the mark.
is written. If this reference great circle pattern is shifted in the θ direction, another great circle l pattern shown in FIG. 30B can be obtained.

On the other hand, as shown in Fig. 32, the radius of the reference point Pma,
That is, since the great circles have different shapes depending on the radius of each concentric circle r ₁ to _{r o} , the great circles l _1a to _l oa corresponding to the reference points P _1a to P _oa of each concentric circle r ₁ to _{r o} Register. Therefore, if n is the number of concentric circles and q is the number of bits (number of pixels) required to store one great circle, then the capacity of the ROM for storing the great circle pattern is q×n bits. On the other hand, as described above, if a great circle pattern is stored for each point, the total number of points will be q×, so a large reduction in capacity can be achieved. For example 256×256
On the pixel mapping plane, the number of points is π(256/2) ² , which requires 13 megawords, but in the present invention, the number of concentric circles is
256/2 = 128, which means it only takes 32 kilowords, and the capacity is only 128 x π ≒ 1/403.

In addition to registering the great circle pattern as a bit pattern as shown in FIG. 31B, it may also be registered as a value of θ. FIG. 33 shows this example. When the entire circumference of concentric circles is divided into 17 pixels (bits), the coordinates of the mark positions are stored. Note that for concentric circles without intersections, a special code is written in place of the coordinates to avoid writing on the mapping plane. For example, a special code can be selected that is an impossible value (for example, 20) that is greater than or equal to the maximum value of the coordinates (17 in this example).

In the above description, concentric circles and rotational positions were used as the address of point P, but other division formats can also be used.

FIG. 34 is a diagram illustrating an example of another format.
In the case of rotational position, the number of divisions of each concentric circle is equal, and the inner concentric circle has a higher resolution and therefore a smaller pixel area. To avoid this, the third
As shown in FIG. 4B, the length L from the reference line in the concentric circle is used instead of the rotational position θ, and (r, L) is used instead of (r, θ). By doing this,
As shown in FIG. 34A, the pixel length Δx is the same for each concentric circle. Moreover, if the width Δ of each concentric circle is constant, the pixel area of each concentric circle is approximately uniform, and the resolution is constant.

Application of the first aspect of the present invention to spherical mapping using the mapping function generation method as described above will be described.

FIG. 35 is an explanatory diagram of the first aspect of the present invention in which the aforementioned mapping function generation method is used for spherical mapping.

As explained above with reference to FIGS. 6 and 9, in the first invention, a width is provided as a mapping function, and its end points are generated and recorded. Therefore, two mapping functions Rlma and Llma indicating the end points of each extraction point may be generated as mapping functions (great circles). That is, it is sufficient to generate locus addresses on each concentric circle of the mapping function indicating both end points. In order to generate this, only the reference mapping functions indicating the two end points at the reference point of each concentric circle are stored as explained in FIG. Two mapping functions at the extraction point are generated by shifting the function on polar coordinates.

FIG. 36 is an explanatory diagram for calculating coordinates for generating the two mapping functions.

As shown in FIG. 36A, 2 corresponding to the extraction point P
Let the two mapping functions be Llma and Rlma, and Llma and Rlma
Let W be the width of the mapping function (that is, the width of the mapping function).

Here, as shown in FIG. 36B, the locus addresses of the two mapping functions Rlma and _Llma on the concentric circle r _n are defined _as
shall be. As shown in FIG. 36C, let the angle formed by the extraction point P centering on the origin O of the sphere be φ, and the angle formed by the concentric circles r _n be φ. With this definition, each locus address (X _R , Y) and (X _L , Y) is determined using FIG. 36D, with the height of the concentric circle r _n being the Y address.

_{Y = sinφ R} = X _c −△X = sinφtan−W/2sin X _L =X _c +△X sinφtan+W/2sin.

That is, the angle formed by the point P and the angle φ formed by the concentric circle r _n
If the predetermined width W of the mapping function is obtained, the locus address in each concentric circle is uniquely determined by the XY coordinates suitable for the memory from the above equation. Therefore, it is sufficient to obtain the locus addresses of the two reference mapping functions corresponding to the reference point in advance from the above-mentioned formula and store them as data in the generation memory.

FIG. 37 is a block diagram of an embodiment for realizing the method shown in FIG. Among the reference mapping functions having a width corresponding to the reference point, the reference mapping function Rlma indicating one end point is stored in the ROM 40-1, and the reference mapping function Llma indicating the other end point is stored in the ROM 40-2, as described above. Each reference mapping function is stored in the format of ( _XL , Y) (X _R , Y) for each concentric circle, and 41-1 and 41-2 are adders, and ROMs 40-1 and 40 respectively - Adds the output of 2 and the output L of the rotation ROM described later, 4
2 is a mapping memory in which the mapping function is written in the XY coordinate system, and 43 is a rotation ROM (read only memory), which stores the rotational position of the extraction point from the reference point as explained in Fig. 34. This converts θ into a distance L.

Next, the operation of the block configuration shown in FIG. 37 will be explained. First, the coordinates of the extraction point P are given in polar coordinate format of a concentric circle position r and a rotational position θ from a reference point on the concentric circle. The coordinate r of this point P is given to the reference mapping functions ROM40-1, 40-2, mapping functions Rlma and Llma corresponding to the reference point of the concentric circle r are selected, and the locus address (X _L , Y) of each concentric circle is ,
(X _R , Y) are output to adders 41-1 and 41-2, respectively. On the other hand, the rotational position θ of point P is rotation ROM
43, converted to distance L, and added to adder 41-
Output to 1,41-2. Adder 41-1, 41
-2, add X _L +L and X _R +L, respectively.
The outputs become (X _L +L, Y) and (X _R +L, Y) and are input to the mapping memory 42 as a write address. In the mapping memory 42, similar to FIG. 9 (X _L +
"+1" to the address of (L, Y), (X _L +L, Y)
"-1" is written to the address. Similarly, the locus address is similarly shifted for each concentric circle and written into the mapping memory 42, and a mapping function (great circle) having a width corresponding to the extraction point P is written. When the writing of the mapping function having the width of one extraction point P is completed, writing of the mapping function having the width of the next target extraction point is sequentially performed in the same manner.

Next, regarding the second invention of the present invention, the 38th section describes an example in which the mapping function generation method described above is used for spherical mapping.
This will be explained using figures.

As explained in FIGS. 11 to 13, in the second invention, suppression regions are provided on both sides of the width region of the mapping function, and the end points of the suppression regions are generated and recorded. Therefore, as mapping functions, in addition to the two mapping functions Rlma and Llma that indicate the end points of the width region, as in FIG.
Just generate Flma. Therefore, as in the first invention described above, it is sufficient to generate locus addresses on each concentric circle of the mapping function indicating these end points, and as in the case of the first invention, four reference mappings at the reference point of each concentric circle are generated. By storing only the functions and shifting the four reference mapping functions according to the distance L between the reference point and the extraction point, mapping functions having widths and suppression regions at the extraction point, that is, four mapping functions Rlma,
Generate Llma, Slma, Flma.

FIG. 39 is a block diagram of an embodiment for realizing the method of FIG. 38, and the same parts as in FIG.
are the reference mapping functions ROM, and the mapping functions Rlma and Flma for the suppression area are respectively ROM40-1 and ROM40-1 and Flma.
Stored in the same manner as 0-2, 41-3,
41-4 are adders, and ROM40-3,
It is provided corresponding to adder 40-4, and is the same as adder 41-1 and 41-2.

Next, to explain the operation of the block configuration in FIG. 39, the coordinates of the extraction point P are given as polar coordinates (r, θ), and the concentric coordinates r are
40-1, 40-2, 40-3, and 40-4, and the rotational position coordinate θ is provided to the rotation ROM 43. From the reference mapping function ROMs 40-1 and 40-2, the end point address ( _XL , Y) of the width area of the mapping function corresponding to the reference point of the concentric circle r, as shown in FIG.
(X _R , Y) is output, and the reference mapping function ROM40
-3 and 40-4 output the addresses (X _S , Y) and (X _F , Y) of the reference mapping functions Slma and Flma for the suppression region corresponding to the reference point of the concentric circle r.
As in FIG. 37, each adder 41-1 to 41-4 performs addition of X _L +L, X _R +L, X _S +L, and X _F +L, respectively, and the outputs thereof are (X _L +L, Y), (X _R +L,
Y), (X _S +L, Y), (X _F +L, Y) and are input to the mapping memory 42 as a write address. In the mapping memory 42, the same as in Fig. 13 (X _L +L, Y)
"a+b" to the address (X _n in Figure 13),
" _- (a+b)" is added to the address _{( X}
In FIG. 3, "-a" is written to X _l ), and "+a" is written to the address (X _F +L, Y) (X _o in FIG. 13).

Similarly, the locus address is shifted for each concentric circle and written into the mapping memory 42, and a mapping function having a width and suppression area corresponding to the extraction point P is written.

When the writing of the mapping function having the width and suppression area for one extraction point is completed, writing of the mapping function having the width and suppression area of the next target extraction point is sequentially performed in the same manner.

40 schematically shows the state of the mapping memory in which the mapping function is written according to the configuration of FIG. 37 related to the first invention, and FIG. 41 shows the state of the mapping memory according to FIG. 39 related to the second invention. This is a diagram schematically showing the state of a mapping memory in which a mapping function is written according to the configuration shown in FIG.

From Figure 40, the peak point is an intersection with sufficient height.
Extraction of CP becomes easy. Furthermore, as shown in FIG. 41, the peak point becomes even steeper, making it easier to extract the intersection point CP.

FIG. 42 is a block diagram of an embodiment of a spherical mapping processor for realizing the method of the present invention. In the figure, the same parts as in FIGS. It is a counter that sequentially generates the addresses of concentric circles r ₁ to _{r n} (i.e., Y ₁ to _{Y n} ) and provides them to the great circle ROM (mapping function memory) 40, rotation (shift) ROM (shift amount memory) 43, etc. 45 is the concentric circle maximum value ROM, and if the coordinate format is (r, L) as explained in Figures 33 and 34 above, the rotation direction of each concentric circle is different because the number of divisions of each concentric circle is different. The maximum value of 46 is stored, and the corresponding maximum value is read out according to the address of the concentric counter 44.
is a normalization circuit, and the great circle from the adder 41
When the sum output of ROM40 and shift ROM43 exceeds 360° from the reference point in terms of rotation angle (that is, the maximum value of concentric circles), this is normalized, and the maximum value of concentric circle maximum ROM45 is converted into rotation angle.
It is normalized as 360°. 47a is an absent great circle detection circuit, and as explained in FIG. 33, the reference great circle detects a value (for example, 512) that cannot exist in a concentric circle without an intersection that is greater than or equal to the maximum value of the largest concentric circle (for example, 460). is written in the great circle ROM 40, so the output of the great circle ROM 40 is monitored, and when the value is a value that cannot exist, a write prohibition signal is output because it is a concentric circle with no intersection.
A write control circuit 7b prohibits writing to the mapping memory 42 indicated by the write address from the normalization circuit 46 in response to a write inhibit signal from the absent great circle detection circuit 47a. 48
A is a latch circuit that latches the contents read from the mapping memory 42, and 48b is an adder that adds an added value from a switching circuit, which will be described later, to the contents of the latch circuit 48a and performs mapping via the bus BUS. 49a is a switching circuit that writes to the memory 42, and an adder 48b selectively writes the weight W of point P from the bus BUS or a value from an integration register to be described later.
49b is an integration register, which integrates the end point recorded as explained in Figs. 6 and 7 above into the values in Fig. 6C and Fig. 7C. It is for. Reference numeral 49c is a suppression counter, which generates the aforementioned four-point addresses in order to generate the suppression areas SFa and SFb. 49d is a multiplication circuit, which outputs the weight W from the bus BUS to the output of the suppression counter 49c. Accordingly-
It converts into w/2, 3/2w, -3/2w, and w/2 and outputs.

Next, the operation of the embodiment configuration shown in FIG. 42 will be explained. Coordinates of each extraction point (r _n ,
θ _n ) and the weight W of that point are given. large circle
ROM40 is ROM40-1 to 40- in Figure 39.
4, that is, it stores a mapping function with a width and a suppression area in which the addresses of the four points mentioned above are stored as locus addresses for each concentric circle, and the output of the suppression counter 49c Accordingly, the addresses of these four points are sequentially output. Coordinates r _n in the concentric direction are input to the great circle ROM 40, and four reference great circles Slma, Rlma, Llma, corresponding to r _n are input.
Flma is selected, while the rotational direction coordinate θ _n is input to the shift ROM 43, and the shift distance group L _n corresponding to θ _n is selected. The concentric circle counter 44 sequentially outputs the address of each concentric circle, and the maximum concentric circle value is
Give it to ROM45, great circle ROM40, and shift ROM43. The maximum concentric circle value ROM 45 outputs the maximum value of the concentric circle, and the great circle ROM 40 outputs the coordinates (addresses) of the intersections of the four reference great circles l _n with the concentric circle in sequence according to the output of the suppression counter 49. and from the shift ROM 43, the distance group
The shift distance in the concentric circle among L _n is output. Therefore, the adder 41 adds the four intersection coordinates from the great circle ROM 40 and the shift distance from the shift ROM 43, and inputs the result to the normalization circuit 46.
The normalization circuit 46 compares the maximum value from the concentric circle maximum value ROM 45 and the added value from the adder 41,
If the added value is smaller than the maximum value, the added value is given to the mapping memory 42 as a write address, and if the added value is larger than the maximum value, (added value - maximum value) is given to the mapping memory 42 as a write address. On the other hand, the large circle
The four intersection point coordinates from the ROM 40 are given to the absent great circle detection circuit 47a, and when the detection circuit 47a detects the above-mentioned impossible value as the intersection point coordinate, it is determined that the reference great circle is the output of a concentric circle having no intersection point. It is determined that the write inhibit signal is given to the write control circuit 47b, and writing to the mapping memory 42 at the write address is prohibited. Conversely, if the write inhibit signal is not output, writing to the mapping memory 42 is performed at the write address.

Next, in order to weight the written contents, the contents at the writing position are read from the mapping memory 42 and latched into the latch circuit 48a via the bus BUS. On the other hand, the multiplication circuit 49d applies the weight explained in FIG. 12B to each end point _Xl , _Xn , _XP , _Xo according to the output of the suppression counter 49c.
It outputs w/2, 3/2w, -3/2w, and w/2 and applies it to the switching circuit 49a. The switching circuit 49a is a multiplication circuit 49d.
Since the adder 48b is switched to provide the output of the latch circuit 48 to the adder 48b, the adder 48b
The output of the multiplier circuit 49d is added to the contents of the write position of a, and the result is applied to the bus BUS, and the result is written to the corresponding write position of the mapping memory 42. The information of the multiplication circuit 49d sequentially outputs -w/2, 3/2w, -3/2w, w/2 according to the output of the suppression counter 49c, while the large circle RO
The address of each end point is sequentially outputted from M40 according to the output of the suppression counter 49c, outputted by the adder 41, sequentially written into the mapping memory 42, and simultaneously read out to the latch circuit 48a and multiplied. It is synchronized with the sequential output of the circuit 49d. Therefore, the adder 48b outputs a value obtained by adding -w/2 to the contents of _Xl , which is the starting point of the suppression area, and writes it to the corresponding starting point _Xl of the mapping memory 42, and then writes it to the starting point Outputs the value obtained by adding 3/2w to the content of point X _n , writes it to the corresponding starting point X _n of the mapping memory 42, and then adds -3/2w to the content of X _P , which is the end point of the width area. The added value is output and the corresponding end point of the mapping memory 42 is
Write to X _P and finally X _o which is the end point of the suppression region
The value obtained by adding w/2 to the contents of is output and written to the corresponding end point X _o of the mapping memory 42.

In this way, when the writing of the four end points for one concentric circle is completed, the great circle is
The end point addresses of the four points of the next concentric circle are sequentially outputted from the ROM 40, shifted by the adder 41, similarly weighted, and stored in the mapping memory 42.

When writing of the great circle to the mapping memory 42 for one point P is completed, the coordinates and weight of the next point are given via the bus BUS, and the great circle is written to the mapping memory 42 in the same way.

In this way, when great circles for all target points are written into the mapping memory 42, the same written content as explained in FIG. 12B is obtained. Next, a peak point is searched from the contents of the mapping memory 42. This peak point is the intersection of the aforementioned great circles.

For this reason, it is first necessary to integrate the end point recorded contents explained in FIGS. 6B and 7B in the mapping memory 42 to restore the original level distribution explained in FIGS. 6C and 7C.

Therefore, the switching circuit 49a is connected to the integration register 4.
9b side so that the contents of the integration register 49b are given to the adder 48b. Next, the contents of the mapping memory 42 are scanned and read out in the X direction for each Y axis, inputted to the latch circuit 48a, and added to the contents of the integration register 49b by the adder 48b, and the addition result is sent via the bus BUS. Mapping memory 42
It is written to the corresponding read position and set in the integration register 49b. If this operation is sequentially repeated in the X direction of the mapping memory 42, the adder 48b
The output is the integration result, and the mapping memory 42 stores the integrated weights of the end points, that is, FIGS. 40 and 41.
The content of the original level distribution of the diagram is obtained.

Next, in order to detect a peak point from this level distribution, the output of the mapping memory 42 is sliced at a predetermined slice value. Therefore, the integral register 49
A slice value is set in b.

After this setting, the contents of the mapping memory 42 are read out and sequentially input to the latch circuit 48a via the bus BUS and latched.The latched contents and the slice value are added by the adder 48b, and the adder 48b adds the added value. If it overflows, it will generate a carry signal, otherwise it will not generate any output. Here, if the slice value is set to such a value that no carry occurs at any point other than the peak point, no carry will occur at any point other than the peak point, but only at the peak point. That is, a slicing operation based on addition is performed. The output of the adder 48b is written to the read position of the mapping memory 42 via the bus BUS, and the register 49b is not updated. Therefore, if this operation is performed on all addresses in the mapping memory 42, only the position (address) corresponding to the peak point in the mapping memory 42 will be rewritten to "1", and the others will be rewritten to "0".

In this way, the mapping memory 42 stores contents in which "1" is stored at the position of the peak point S and "0" is stored at the other positions, and information extraction by mapping is completed when the position of this point S is known. become.

In the above description, the generation and writing of a mapping function having the width and suppression area of the second invention in which the suppression counter 49c outputs four end point addresses has been explained, but the width area of the first invention is To generate and write the mapping function, the ROM 40 is configured with ROMs 40-1 and 40-2 shown in FIG. 37, and the 4-bit output suppression counter 49c is changed to a 2-bit output width counter. configuration can be used.

Next, a three-dimensional measurement system for an object will be described as an application example of the information extraction method using mapping. FIG. 43 is a configuration diagram of a three-dimensional measurement system that is an application example of the present invention. In the figure, 2 is a television camera, which is composed of a spherical camera to be described later, 4a and 4b are mapping processors, 43, 50 is an analog-to-digital (AD) converter which converts the video signal from the television camera 2 into a multivalued digital image signal, 51 is an outline extraction unit, This is a well-known method that draws a contour based on the image signal from the converter 50, and the image signal from which the contour has been extracted is stored in a frame memory (to be described later), and the coordinates and weights of each point of the contour obtained as a result of contour extraction are stored in the original image (to be described later). 52 is a frame memory for outputting to memory; 53 is a frame memory for storing the image signal from which contours have been extracted for display;
54 is a television monitor for displaying the contents of the frame memory 52; 54 is an original image memory for storing the coordinates and weights of each point extracted by the contour extraction unit 51; 55 and 56, respectively. A mapping memory is used to store the mapping results obtained by the mapping processors 4a and 4b, and 57 is a parallel interface that connects the contour extraction unit 51, the mapping processor 4a,
4b is for exchanging commands and data. A recognition unit 60 controls the entire system and recognizes the three-dimensional image captured by the television camera 2 using the mapping results, and is comprised of components 61 to 66. 61
62 is a processor that controls and recognizes the entire system by executing programs.
63 is the main memory, which is composed of RAM (random access memory) and stores data etc. for the operation of the processor, and 63 is the interface, which exchanges data and commands with the outside and the keyboard, which will be described later. something to do,
64 is a printer interface, which exchanges data and commands with the printer, which will be described later. 65 is a floppy disk controller, which is used to control the floppy disk device 66 as an external storage. 67 is a printer. 68 is a keyboard for inputting commands and data. 70 is a multi-bus, which includes a processor 61 of the recognition unit 60, an AD converter 50, a frame memory 52, and an original image memory 5.
4, mapping memories 55, 56, parallel interface 57, and floppy disk controller 65 are connected to exchange commands and data.

Next, the operation of the embodiment configuration shown in FIG. 43 will be explained. In this embodiment, the television camera 2 obtains a spherically projected image (FIG. 14) of the object. This will be explained with reference to FIGS. 44 to 47.

Figure 44 is an explanatory diagram of the spherical camera,
FIG. 5 shows the configuration of an embodiment of the image input section, FIG. 46 is a diagram for explaining the embodiment shown in FIG. 45, and FIG. 47 is a diagram for explaining the characteristics of the equidistant projection lens.

The camera used in this example consists of a lens that forms an image at a position whose arc distance from the optical axis is proportional to the incident angle of the light, that is, an equidistant projection lens, and a light receiving element with a homogeneous two-dimensional positional resolution. It has the following. Alternatively, instead of the equidistant projection lens, a normal lens, a flat light receiving element with a homogeneous two-dimensional positional resolution, and an electronic coordinate conversion section are provided, and the light is received by the equidistant projection lens. The coordinates of the input image information may be transformed so that they become substantially similar.

The above-mentioned spherical camera virtually consists of a pinhole 20 and a spherical light receiving element 2, as shown in FIG. 44A.
It can be considered that the camera is equivalent to the camera configured by 1 and 1. In reality, the pinhole 20 has the problem of insufficient light quantity and blurring of the image, so the pinhole 20 is replaced with a lens system. Furthermore, since the spherical light receiving element 21 is difficult to manufacture, a flat light receiving element is used. Therefore, the fourth
As shown in FIG. 4B, a difference occurs between the ideal spherical surface 22 and the flat light receiving element 23. Note that the reference numeral 24 in the figure represents a lens system including a lens or a combination of a plurality of lenses.

For example, flat photodetectors such as MOS and CCD are
Normally, the light-receiving cells are arranged at equal intervals in the vertical and horizontal directions, and the resolution is constant regardless of the light-receiving position. In addition, continuous plane light receiving elements such as so-called vidicon etc.
Dimensional position resolution is homogeneous. Therefore, it is necessary to remove the deviation between the ideal spherical surface 22 and the flat light receiving element 23 as shown in FIG. 44B by using a lens or a lens system 24.

For example, as shown in FIG. 44c, let y be the height of the image formed on the light-receiving surface located at a focal length f from the center of the lens 24. If the angle of the incident light with respect to the optical axis of the lens is taken as the angle, in a normal lens, there is a relationship of y=f tan . . . . However, in order to form an image equivalent to the spherical light receiving element 21 of a pinhole camera as shown in FIG. 44A on the flat light receiving element 23 shown in FIG. 44B, a projection method in which y is proportional to ,
In other words, a lens that satisfies y=f... is used. Such a lens is a type of fisheye lens and is known as a so-called equidistant projection lens.

Note that even without using the special projection lens described above, a spherical camera can be constructed using a normal lens or another projection lens as shown below. As shown in FIG. 45, this spherical camera includes a light receiving element 26, a memory 27, and a coordinate conversion control circuit 28.
and an arithmetic circuit 29.

The light receiving element 26 outputs a signal obtained by sequentially scanning a two-dimensional image one-dimensionally. Although not shown, of course, when the subsequent processing is handled as a digital signal, the output video signal of the light receiving element 26 is
Analog/digital conversion is performed by an A/D converter. This video signal is temporarily written into the image memory 27. The coordinate conversion control circuit 28 controls an arithmetic circuit 29 that writes and reads data into and from the image memory 27 and performs arithmetic operations on output data from the image memory 27.

For example, when a normal lens is used, the image y' formed by the normal lens is y'=f tan . . . from the following equation. Since we want to convert this into the characteristic of the formula, we can substitute the formula into the formula to obtain y=f = f tan ^-1 (y'/f)... That is, the coordinate transformation control circuit 28 and the calculation circuit 29 electronically perform the coordinate transformation of the formula (1).

The image memory 27 stores image data in one-to-one correspondence with the light-receiving cells of the light-receiving element 26, as shown by white circles in FIG. Since these lens characteristics are different from the above formula, they are not spaced at equal angular intervals. Grid points corresponding to equal angular intervals are determined, for example, as shown by black circles in FIG. 46, depending on the lens characteristics. Note that when the lens is a normal lens, the position of the black circle is determined by the above formula. Therefore, the data at the position of the black circle may be interpolated from the four neighboring white circles using a weighted average according to the distance. Alternatively, when the intervals between the white circles are close, that is, when the resolution of the light receiving element 26 is sufficiently fine, the value of the white circle that is approximately closest to the black circle may be adopted as the output.

The angular resolution of the normal lens and the angular resolution of the equidistant projection lens are illustrated in FIG. 47, for example.

The angular resolution of a normal lens is obtained by differentiating the formula by dy/d=f/cos ² . On the other hand, the angular resolution of the equidistant projection lens is obtained by differentiating the formula by dy/d=f... FIG. 47 is a plot of the equation and the equation. As can be seen from this, the angular resolution of an equidistant projection lens is constant regardless of the angle of incidence, whereas with a normal lens, the smaller the angle of incidence, the lower the angular resolution.

That is, a spherical camera using an equidistant projection lens or a mechanism that performs conversion equivalent thereto has the characteristics that the angular resolution is homogeneous regardless of the incident angle, and the field of view is large. Therefore, when this spherical camera is applied to robot vision, it becomes possible to perform accurate three-dimensional measurement due to the homogeneity of angular resolution. In addition, because the field of view is large, it is possible to obtain a wide range of visual information.
For example, it is possible to present a realistic image even to an operator located in a remote location.

Based on the image information captured by the spherical camera 2, three-dimensional measurement using spherical mapping is performed as follows.

The video signal of the image captured by the television camera 2, which is a spherical camera, is input to the AD converter 50 and converted into a multivalued digital image signal.
given to 1. The contour extraction unit 51 extracts contours from the multivalued digital image signal, and the contour-extracted image signal is given to the frame memory 52, which displays the image captured by the television camera 2 on the television monitor 53 and performs contour extraction. The coordinates and weights of the extracted points obtained as a result are given to the original image memory 54.

When the coordinates and weights of the extraction points related to the original image memory 54 are stored, the mapping processor 4a stores them in the original image memory 54.
4, the coordinates and weights of each extraction point are read out, and as explained in FIG. 43, a great circle corresponding to the coordinates of the extraction point is generated, written into its own mapping memory 42, and stored with weighting. The mapping processor 4a successively reads out the coordinates and weights of each extraction point from the original image memory 54, and similarly writes the corresponding great circle into its own mapping memory 42. In this way, for all target extraction points, the self mapping memory 42
When the writing is completed, the peak point (S point) is searched for as explained in FIG. 43, and the mapping memory 42
Rewrite. Thereafter, the contents of the mapping memory 42 are transferred to the mapping memory 55. Line segments can be extracted based on the contents of this mapping memory 55.

When the mapping result of the mapping processor 4a is stored in the mapping memory 55, the mapping processor 4b reads the coordinates of point S from the mapping memory 55, and writes the corresponding great circle into its own mapping memory as described above. . It is assumed that the image captured by the television camera 2 has a plurality of line segments, and a plurality of S (peak) points are stored in the mapping memory 55 as a mapping result of the mapping processor 4a. Mapping processor 4b
As described above, once the great circles corresponding to all target S points are written into its own mapping memory, the peak point (SS point) is searched for and the mapping memory is rewritten. That is,
As explained in FIG. 33, inverse mapping is performed based on the coordinates of point S, and thereby the direction of the line segment and the point of intersection can be determined. Mapping processor 4b transfers the contents of its own mapping memory to mapping memory 56. Furthermore, although not shown, if a mapping processor that performs further inverse mapping based on the contents of the mapping memory 56 and a mapping memory that stores this mapping result are provided, information on the direction of the plane can be provided as the mapping result. It will be done.

A series of operations of these mapping processors is performed by the recognition unit 6.
This is done by the 0 processor 61 exchanging commands and data with each mapping processor via the bus 70 and parallel interface 57.

In this way, the line segments, the intersections of the line segments, the directions, and the directions of the plane formed by the line segments are extracted by the three mapping processors and stored in the three mapping memories. 61
reads out these three mapping memories via the bus 70, uses the main memory 62 to identify line segments, intersections, and planes of the image captured by the television camera 2,
The object is reconstructed from these line segments and planes, and compared with data of various objects stored in the floppy disk device 66 to identify and recognize the object.

The recognized result is printer interface 64
The data is printed out to the printer 67 via the printer 67. If the robot controller is connected via the interface 63, the recognition results are given to the robot controller, and the robot controller causes the robot to run without hitting obstacles or to grip a specific object. be able to.

Next, the present invention will be described using a Hough transform as a mapping.

FIG. 49 is an explanatory diagram in which the present invention is applied to the Hough transform. The Hough transform is used as a method for concentrating certainty when extracting straight lines or line segments from an input image. The Hough transform is the point P (X, Y) on the X-Y coordinate plane in Figure 48A.
, is converted into the sine wavefront shown in FIG. 48B using the following equation.

r=Xsin+y cos... That is, it can be interpreted as mapping the two-dimensional coordinates (X, Y) onto the polar coordinate plane (r,).

When the sine waves corresponding to each point on a straight line as shown in FIG. point). The rotation angle θ ₀ and height r ₀ of this point are determined by the length r ₀ of a perpendicular line drawn from the origin to the straight line formed by each point in FIG. 48A and the rotation point θ ₀ . Therefore, information about the straight line formed by each point can be extracted. That is, extraction of straight lines or line segments,
It becomes possible to reproduce broken or distorted line segments.

Then, as shown in FIG. 48B, the first
By applying the invention described above and giving a width SF to the sine wave (mapping function) for one point, it becomes easy to extract the intersection point. Similarly, as shown in FIG. 48C, by applying the second invention of the present invention, the width SF and suppression region are
If SFa and SFb are provided, a steeper intersection can be obtained as described above, and extraction of the intersection becomes easier and more accurate. The same mapping function generation method as explained for the spherical transformation can be used in this Hough transformation as well. FIG. 49, FIG. 50, and FIG. 51 are diagrams for explaining mapping function generation for this Hough transform.

When the above equation is transformed, |r|=√ ² + ² ...-1 = tan ^-1 Y/X ...-2.

That is, points P 1 and P 2 having the same |r|, that is, points P ₁ and P ₂ on the same concentric circle of radius |r| centered on the origin O as shown in FIG. 49A, differ only in phase.
Therefore, according to the principle explained in FIG. 29, the sine wave (mapping function) at point P ₁ as shown in FIG.
By simply shifting l ₁ , the sine wave l ₂ at point P ₂ can be obtained. Similar to the spherical transformation described above (e.g.
If a reference sine wave (with the intersection of the axis and the concentric circle as the reference point) is provided, sine waves at points on the same concentric circle can be obtained by shifting the reference sine wave.

Therefore, as shown in Figure 50, the XY coordinate plane is r ₁ ~ _{r o}
By dividing it into concentric circles and expressing it in polar coordinate format (r,), a mapping function can be generated using the same method as explained for spherical transformation.

FIG. 51 is a diagram for explaining the method of registering a reference sine wave for Hough transform, and as explained in FIG. 31A, the vertical axis (radial direction) of the mapping plane is
The pattern can be registered by dividing it into r ₁ to r _o and storing the intersection points (circles in the figure) of each concentric circle and the reference sine wave. The pattern is expressed using the value (address) method described in FIG. 33.

If we apply this mapping function registration to the first invention and the second invention, the first invention only needs to have two sine waves corresponding to one point, and the second invention only needs to have four sine waves. .

FIG. 52 is a block diagram of an embodiment in which the present invention is applied to the Hough transform, and is a block diagram for the first invention. In the figure, the same parts as in Fig. 37 are indicated by the same symbols, 40'-1, 40'-
2 is a reference sine wave ROM which stores the coordinates of the intersection of the reference sine wave of each concentric circle with each concentric circle as shown in Fig. 51, and the ROM 40'-1 stores the reference sine wave indicating the starting point of the width area. However, ROM40'-2
A reference sine wave indicating the end point is stored in . 41'-1 and 41'-2 are adders,
42, which is the same as that in FIG. 37, is a mapping memory in which a write address is specified by the outputs of adders 41'-1 and 41'-2, and a sine wave pattern is written therein. Note that the configuration in Figure 52 is
Standard great circle ROM40-1, 40-2 with the configuration shown in Figure 7
is replaced with the reference sine wave ROMs 40-1 and 40'-2.

Next, the operation of the embodiment shown in FIG. 52 will be described. The coordinates of point P(x, y) are converted into polar coordinate format, ie, concentric circle position |r| and phase, by the following equation. Concentric circle position |r| is the reference sine wave
The reference sine wave pattern corresponding to the concentric circles |r| is selectively output by inputting to the ROMs 40'-1 and 40'-2. On the other hand, the phase is given to the rotation ROM 43 and converted into a distance L. Each adder 41'-1, 4
1'-2 is output by adding the distance L to the intersection point on each concentric circle of the selected reference sine wave pattern.

As a result, the reference sine wave pattern is shifted by the phase (distance L) to become a sine wave pattern corresponding to point P, which is divided into concentric circles using the outputs of adders 41'-1 and 41'-2 as addresses. A sine wave corresponding to point P is written into mapping memory 42'.
In this way, the sine wave at each point is converted to the reference sine wave
Generated based on the reference sine wave by ROM 40',
Mapping is performed by writing into the mapping memory 42'.

Note that the extraction of the intersection point is the same as that explained in FIG. 42, so the explanation will be omitted.

Although the above-mentioned FIG. 52 is related to the application of the first invention, it can be similarly modified to the application of the second invention. That is, as shown in Fig. 39, the reference sine wave
This can be achieved in the same way by providing four ROMs.
Furthermore, the end point recording method can also be applied in the same manner.

Furthermore, the present invention can be applied to other mapping methods. This will be explained with reference to FIG.

As shown in FIG. 53A, the origin O(a, b) is set on the two-dimensional plane (X, Y) of the point sequence P ₁ to P ₅ .

Next, as shown in FIG. 53B, circles C1 to C5 are drawn corresponding to each point _P1 to _P5 . The meaning of this circle is the 53rd
To explain using Figure C, a straight line passing through point P can lie in a direction of 360 degrees with point P in the center. On the other hand, if we examine the intersections of the straight lines with the perpendicular lines CL1 and CL2 that connect each straight line (for example L1, L2) centered on these points P and the origin O, we can find that the line segment connecting the point P and the origin O has a diameter and exists on a circle C passing through point P and origin O. That is, the circle C is the locus of the intersections of each straight line centered on the point P and the perpendicular line from the origin O. Therefore, if a circle C is drawn, it will show the representative points of all possible straight lines centered on the point P.

In this way, each point P ₁ to _{P 5} as shown in FIG. 53B
Draw circles C ₁ to C ₅ corresponding to .

Next, as shown in Figure 53D, C1 to C of these circles
Find the intersection point like the dotted line in 5. As mentioned above, circle C1
~C5 is the representative point of all the straight lines centered on each point P ₁ ~ _{P 5} , so these intersection points are each point P ₁ ~ _{P 5}
It is a representative point of a common straight line among straight lines centered at . When these intersection points coincide in each circle C1 to C5, it means that points P ₁ to _{P 5} are completely on the straight line indicated by the intersection point, and generally all points P ₁ to _{P 5} are on the straight line indicated by the intersection point. Since this is rare, the intersection points do not coincide in each circle C1-C5. For this purpose, a point with a large number of matching circles is set as an intersection point.

If the intersection point m is found in this way as shown in FIG. 53E, the straight line L defined by the points P ₁ to P ₅ is a line perpendicular to the straight line connecting the origin O and the intersection point m as shown in FIG. 53F, By drawing this, the straight line L can be extracted and reproduced.

The first and second aspects of the present invention can also be applied to information extraction methods using such mapping, and since these are the same as the above-mentioned examples of spherical mapping and mapping using Hough transform, the explanation will be omitted.

Although the present invention has been described above with reference to examples, the present invention can be modified in various ways according to the gist of the present invention.
These are not excluded from the present invention.

〔Effect of the invention〕

As explained above, according to the first aspect of the present invention, a mapping function corresponding to the coordinates of the extraction point on the target surface is generated for each extraction point, the generated mapping function is written on the mapping surface, and the mapping function corresponding to the coordinate of the extraction point on the target surface is generated. An information extraction method using mapping that extracts information meant by each extraction point from the mutual relationship of mapping functions written on a mapping surface, wherein the mapping function is written with a predetermined width on the mapping surface. As a result, the level of the peak point, which is the intersection of mapping functions on the mapping surface, increases and the slope becomes steeper, which has the effect of making it easier to extract the intersection, and therefore information can be extracted easily. And it's accurate. In addition, since it is only necessary to give a width to the mapping function, it is easy to implement, contributing to the spread of information extraction methods using such mapping, and further contributing greatly to the implementation of three-dimensional measurement systems.

Furthermore, according to the second aspect of the present invention, a mapping function corresponding to the coordinates of the extraction point on the target surface is generated for each extraction point, and the generated mapping function is written on the mapping surface,
An information extraction method using mapping that extracts information meant by each extraction point from the mutual relationship of mapping functions written on the mapping surface, the mapping function having a predetermined width and a method of extracting information on both sides of the width. Since it is characterized in that a suppression area having the opposite polarity to the area of the width is written on the mapping plane, in addition to the effect of the first invention described above, the suppression area makes it possible for the mapping functions to be similar except at the intersection. Since these cancel each other out, the offset level is reduced, and the intersection points are further emphasized, making it easier to extract the intersection points, and the position of the intersection point can also be extracted accurately. Also plays. Moreover, it is easy to realize, and contributes to the spread of information extraction methods using mapping.

[Brief explanation of drawings]

Fig. 1 is a basic procedure diagram of an information extraction method using mapping, Fig. 2 is an explanatory diagram of a conventional method of writing a mapping function, and Figs. 3 and 4 are a method of writing a mapping function according to the first invention of the present invention. An explanatory diagram of, Figure 5 is the third
Explanatory diagram of the recording state of the mapping plane by the method shown in the figure, No. 6
Figures 7 and 8 are explanatory diagrams showing how the mapping function in Figure 3 is written using the endpoint recording method, and Figure 9 is an explanatory diagram of writing the mapping function in Figure 3 using the end point recording method.
A block diagram of an embodiment for carrying out the method shown in the figure by the endpoint recording method, FIG. 10 is an explanatory diagram for explaining the effects of the present invention, and FIG. 11 is for explaining the second invention of the present invention. Fig. 12 is an explanatory drawing of the method shown in Fig. 11 performed by the end point recording method.
FIG. 3 is a block diagram of an embodiment for carrying out the second invention method of the present invention by the endpoint recording method, and FIG.
Figures 15 and 16 are explanatory diagrams of spherical projection when the present invention is applied to spherical mapping, and Figures 17 and 18 are
Figures 19, 20, and 21 are explanatory diagrams of spherical mapping to which the present invention is applied, and Figures 22, 23,
Fig. 24 is an explanatory diagram of the information extraction method using spherical mapping to which the present invention is applied, Fig. 25 is an explanatory diagram of the procedure of spherical mapping to which the present invention is applied, and Figs. 26 and 27 are diagrams for explaining the spherical mapping. Explanatory diagrams for conventional mapping function generation, Fig. 28, Fig. 29, Fig. 30, Fig. 31
32 is an explanatory diagram of an embodiment of mapping function generation used for spherical mapping to which the present invention is applied, and FIG.
34 is an explanatory diagram of registration of mapping functions used in spherical mapping to which the present invention is applied; FIG. 34 is an explanatory diagram of coordinate division used in spherical mapping to which the present invention is applied;
Figure 5 shows the first invention of the present invention and Figures 28 to 34.
Fig. 36 is an explanatory diagram of the method of calculating the locus address in Fig. 35, and Fig. 37 is an illustration of the case where the generation of the mapping function according to the figure is applied. One embodiment block diagram,
FIG. 38 is an explanatory diagram when the generation of mapping functions according to FIGS. 28 to 34 is applied to the second invention of the present invention, FIG. 39 is a block diagram of an embodiment for realizing FIG. 38, 40 is a content distribution diagram in the mapping memory shown in FIG. 37, FIG. 41 is a content distribution diagram in the mapping memory shown in FIG. 39, and FIG. 42 is a block diagram of an embodiment for realizing the method of the present invention. Fig. 43 is a block diagram in which the method of the present invention is applied to a three-dimensional measurement system; Figs. 44, 45, 46, and 47 are explanatory diagrams of the television camera having the configuration shown in Fig. 43;
49, 50, and 51 are explanatory diagrams when the method of the present invention is applied to mapping by Hough transform,
FIG. 52 is a block diagram of an embodiment in which the present invention is applied to mapping by Hough transform, and FIG. 53 is an explanatory diagram of another information extraction method by mapping to which the present invention is applied. In the figure, SF, SF1, SF2...mapping functions, SFa,
SFb... Suppression area, 10... Mapping memory (mapping surface), 11, 11'... Mapping function generation circuit.

Claims (1)

[Claims] 1. A mapping function corresponding to the coordinates of the extraction point on the target surface is generated for each extraction point, the generated mapping function is written on the mapping surface, and the mapping function written on the mapping surface is An information extraction method using mapping for extracting information meant by each of the extraction points from mutual relationships, the method characterized by writing the mapping function with a predetermined width on the mapping plane. . 2. The information extraction method by mapping according to claim 1, wherein the generated mapping function is made to have a predetermined width so that the mapping function has a predetermined width. 3. The information extraction method by mapping according to claim 1, characterized in that when writing the mapping function having the width on the mapping surface, the writing is performed at an end point position of the width. 4. The information extraction method by mapping according to claim 3, characterized in that, in order to write at the end point positions of the width, two mapping functions indicating the end point positions are generated as the generated mapping functions. 5. An information extraction method using mapping according to claim 3 or 4, characterized in that a differential value is assigned to the end point position. 6. The information extraction method by mapping according to claim 5, wherein the differential value written on the mapping surface is integrated and rewritten on the mapping surface. 7. The information extraction method by mapping according to claim 1, characterized in that in order to give the mapping function a predetermined width, the coordinate frame of the mapping plane has a width. 8 Generate a mapping function corresponding to the coordinates of the extraction point on the target surface for each extraction point, write the generated mapping function on the mapping surface, and calculate each extraction from the correlation of the mapping functions written on the mapping surface. An information extraction method using mapping that extracts information meant by a point, in which the mapping function has a predetermined width, and suppression regions with opposite polarities to the region of the width are provided on both sides of the width to create the mapping surface. An information extraction method using mapping, which is characterized by writing into. 9. According to the mapping according to claim 8, wherein the mapping function to be generated has a predetermined width region and a suppression region in order to make the mapping function have a predetermined width region and a suppression region. Information extraction method. 10. According to the mapping according to claim 8, wherein when writing the mapping function having the width region and the suppression region on the mapping plane, writing is performed at end point positions of the width and suppression region. Information extraction method. 11. Information by mapping according to claim 10, characterized in that in order to write to the end point positions of the width and suppression area, four mapping functions indicating the end point positions are generated as the generated mapping functions. Extraction method. 12. The information extraction method by mapping according to claim 10 or 11, characterized in that a differential value is assigned to the end point position. 13. The information extraction method by mapping according to claim 12, wherein the differential value written on the mapping surface is integrated and rewritten on the mapping surface.