JP3852704B2 - Method and apparatus for recognizing three-dimensional shape of object - Google Patents

Description

  The present invention relates to a method and apparatus for recognizing a three-dimensional shape of an object without using a conventional image processing technique based on two-dimensional shape extraction, and is used for product inspection, as a peripheral object information acquisition device mounted on a vehicle, or for crime prevention. The present invention relates to a method and apparatus for recognizing a three-dimensional shape of an object that can be applied as a camera for monitoring traffic and that can estimate a three-dimensional shape of a recognized object and simulate changes in the near future. .

  For the three-dimensional shape recognition by photographing with a camera, a method of estimating the distance of an object shape pattern extracted using a pattern recognition method based on images from two cameras is usually used.

  In this method, first, a plurality of images from a camera are prepared, and then a common pattern is extracted by performing image processing based on the dot image of the above image using a dedicated image processing device composed of a CPU and a memory, and its line of sight The distance estimated from the crossing relationship is given as an attribute to the extracted pattern. The pattern used here is a two-dimensional geometric pattern extracted from a luminance change or a color pattern change, and gives an average distance from the camera in the plane to the geometric pattern. Even when three or more cameras are used, the basic processing method is the same, and all of them require an image processing apparatus that extracts a two-dimensional geometric pattern object shape from each captured image.

  By the way, the measurement of the pattern processing speed and the distance to the object depends on the speed of the CPU. Of course, in the three-dimensional shape recognition using a high resolution camera with a large number of pixels, it takes time proportional to the square of the number of pixels. Met. Furthermore, in an image in which a plurality of objects having different distances exist, a larger amount of time is required because there are a plurality of patterns to be recognized. In addition, a small object that is far away is captured in the image of one camera, and the other camera is hidden in the shadow of a nearby large object and does not exist as a pattern in the image, or the pattern is cut off. In such a case, the pattern cannot be recognized, and naturally the distance cannot be estimated.

  On the other hand, as a method not using a camera, a method using a millimeter wave radar and a laser radar is known. These methods have high reliability under adverse conditions and have been put into practical use in an in-vehicle auto cruise control system. However, it has the disadvantage of having a low ability to detect small non-metallic obstacles such as tires and wooden boxes.

  In addition, as a system using only one camera, a method using an optical flow based on continuous images acquired during traveling with a camera mounted on the vehicle on the assumption that the vehicle travels has been proposed. Since an optical flow vector obtained from an image of a flat road surface follows a specific formula, if there is a point that does not apply to this rule, it is a method of recognizing an object having a height different from that of the road surface. Furthermore, a method has been proposed in which this continuous image is subjected to image processing in accordance with the movement of the car (see Patent Document 1).

However, since this method also requires an image processing device equipped with a CPU, there are disadvantages that a large amount of processing time is required when there are a large number of objects to be image-processed or high-resolution captured images. It was. In addition, when the vehicle is slow, the optical flow vector becomes small, making it impossible to recognize three-dimensionally, or requiring a separate device for holding background three-dimensional information.
JP 2001-266160 A

  The present invention has been made paying attention to such problems, and the problem is that the two-dimensional shape pattern extraction that increases the processing time due to an increase in the number of pixels and an increase in objects to be recognized. An object of the present invention is to provide a method and apparatus for recognizing a three-dimensional shape of an object that can recognize a three-dimensional shape without using a conventional image processing method.

That is, the invention according to claim 1
Assuming a method of recognizing the three-dimensional shape of an object based on continuous images taken by the camera with the direction from the camera to the object being photographed as the Z direction,
A continuous image is obtained by continuously photographing the object while moving the camera in one direction or a plurality of directions directly or indirectly in a plane perpendicular or substantially perpendicular to the Z direction, and a sweep signal of each image Obtaining a continuous image signal comprising:
Among the obtained continuous image signals, the proportional coefficient (α / L) of the following formula (1) with respect to the displacement (ΔP) of the position of the characteristic signal strength and the movement distance (ΔD) of the camera is characteristic. Obtaining surface distribution data in a three-dimensional space in an object having a Z value related to a distance in the Z direction from the continuous image signal in association with an initial position of signal intensity;
Each step of specifying an object by partial matching search from the obtained surface distribution data of the three-dimensional space is provided.

ΔP = (α / L) · ΔD = α · (1 / L) ΔD (1)
However, L is the distance from the camera to the object to be photographed, and α is a coefficient of (1 / L) ΔD in the above formula (1) determined by the image receiving system.

The invention according to claim 2
Based on the method for recognizing the three-dimensional shape of an object according to the invention of claim 1,
The continuous image signal is a continuous image difference signal obtained by a difference between sweep signals in each image,
The invention according to claim 3
Based on the method for recognizing the three-dimensional shape of an object according to the invention of claim 1 or 2,
Obtaining the continuous image signal or the continuous image difference signal using a horizontal synchronous sweep signal in each image,
The invention according to claim 4
Based on the method for recognizing the three-dimensional shape of an object according to the invention of claim 1 or 2,
The camera is periodically or directly moved in one or more directions within a plane perpendicular or substantially perpendicular to the Z direction,
The invention according to claim 5
Based on the method for recognizing the three-dimensional shape of the object according to the invention of claim 1, 2 or 4,
The camera is moved directly or indirectly in a circular direction or an elliptical direction within a plane perpendicular or substantially perpendicular to the Z direction.

Next, the invention according to claim 6 is:
Assuming a device that recognizes the three-dimensional shape of an object based on continuous images taken by the camera, with the direction from the camera to the object being imaged as the Z direction,
A continuous image is obtained by continuously photographing the object while moving the camera in one direction or a plurality of directions directly or indirectly in a plane perpendicular or substantially perpendicular to the Z direction, and a sweep signal of each image Means for obtaining a continuous image signal comprising:
Among the obtained continuous image signals, the proportional coefficient (α / L) of the following formula (1) with respect to the displacement (ΔP) of the position of the characteristic signal strength and the movement distance (ΔD) of the camera is characteristic. Means for obtaining surface distribution data of a three-dimensional space in an object having a Z value related to a distance in the Z direction from the continuous image signal by associating with an initial position of signal intensity;
The main part is constituted by means for specifying an object by partial matching search from the obtained surface distribution data of the three-dimensional space,
ΔP = (α / L) · ΔD = α · (1 / L) ΔD (1)
However, L is the distance from the camera to the object to be photographed, and α is a coefficient of (1 / L) ΔD in the above formula (1) determined by the image receiving system.

The invention according to claim 7 provides:
Assuming an apparatus for recognizing the three-dimensional shape of an object according to the invention of claim 6,
The continuous image signal is a continuous image difference signal obtained by a difference between sweep signals in each image.

According to the method and apparatus for recognizing the three-dimensional shape of an object according to the present invention,
The above object is continuously photographed while moving the camera directly or indirectly in one or more directions in a plane perpendicular or substantially perpendicular to the Z direction, and continuous images are acquired. Obtaining a continuous image signal comprising, or means;
Among the obtained continuous image signals, the proportional coefficient (α / L) of the following formula (1) with respect to the displacement (ΔP) of the position of the characteristic signal strength and the movement distance (ΔD) of the camera is characteristic. Obtaining surface distribution data of a three-dimensional space in an object having a Z value related to a distance in the Z direction from the continuous image signal by associating with an initial position of signal intensity;
ΔP = (α / L) · ΔD = α · (1 / L) ΔD (1)
In order to provide each step or means of performing the object identification by partial match search from the obtained surface distribution data of the three-dimensional space,
It is possible to recognize a three-dimensional shape without using a conventional image processing technique based on two-dimensional shape pattern extraction that increases the processing time by increasing the number of pixels or the number of objects to be recognized.

  Accordingly, when the method or apparatus according to the present invention is applied to the field of inspecting small product parts manufactured in large quantities such as connectors of electronic equipment, for example, it becomes possible to recognize the direction of each small product part. It is possible to significantly reduce the robot workload when assembling the robot.

  In addition, when the apparatus according to the present invention is mounted on, for example, a traveling car, the shape can be recognized including the shadowed part that is not shown in the image by identifying the direction of the other stopped car. It becomes possible to specify all positions of the door. In addition, by recognizing the object ahead through the car glass, it is possible to cut out an object other than the matched shape (car) first, and to recognize the cut out part by matching the partial shape. Pedestrians can be recognized. Therefore, there is an effect that it is possible to bridge to a function for predicting / inferring the danger of the door opening and the possibility of the pedestrian jumping out.

  Hereinafter, the present invention will be described in detail with reference to the drawings.

  An apparatus for recognizing a three-dimensional shape according to the present invention comprises a horizontally movable camera, its driving mechanism and a signal conversion unit, means for acquiring a continuous image signal of an object, and a Z direction direction from the obtained continuous image signal. The main part is composed of means for obtaining surface distribution data in a three-dimensional space for an object having a Z value related to distance, and means for specifying an object by partial match search from the obtained surface distribution data in the three-dimensional space. ing.

  First, the means for obtaining the continuous image signal and the means for obtaining surface distribution data in a three-dimensional space will be described.

  There are analog and digital imaging cameras, but in either case, signals are sent from a two-dimensionally arranged pixel matrix. In any case, the analog signal is a combination of a horizontal sweep signal and a vertical synchronization signal. The luminance is taken as an example as a characteristic signal among the transmitted signals.

An image of a certain cuboid object (see FIG. 1B. FIG. 1B shows the object viewed from a slight angle) (however, this image is an image of the object taken from the front, If the horizontal sweep signal has a black background and the object is whitish, the color signal at the pixel position corresponding to the object position is shown in FIG. It appears in a trapezoidal shape as shown in A). If the signal position rising in this trapezoidal shape is now P and the signal is acquired again when the camera moves by ΔD, this signal position P ₀ is the distance from the camera to the object as L, and ΔP in the following equation (1). Shift.

ΔP = α · (1 / L) ΔD (1)
In addition, Formula (1) is a relational expression obtained from the relation of (L: ΔD = α: ΔP) from the ratio of triangles shown in FIG. In addition, the coefficient α in the formula (1) is a constant characteristic of photographing determined by an image receiving system such as a camera lens system and an image receiving mechanism, and is constant for an object photographed at the same time.

  Therefore, the change coefficient dP / dD = α (1 / L) with respect to the camera movement amount of the signal position P is related to the proximity of the distance.

  If the optical signal from the subject does not change over time, or if the contribution due to the change in the horizontal displacement of the camera is sufficiently greater than the temporal change of the optical signal from the subject, the signal from the background is clearly Since the movement of the pattern of the signal and the movement of the pattern of the signal from the object surface at the distance L are collectively different by ΔP, the displacement ΔP is obtained by tracking the boundary signal (here, the rising signal on the trapezoid). Can be measured and the distance (L / α) is calculated backward. The distance (L / α) is a distance in which α is a unit length.

  If the horizontal direction corresponding to the moving direction of the camera in the image plane is X, the horizontal signal is for a certain Y value, and the signal position P indicates the X value in the image plane. Thus, a three-dimensional component set of (X, Y, Z) can be calculated with the distance (L / α) as the Z value for the point (X, Y) in the image plane. Furthermore, since the camera displacement ΔD continuously changes, ΔP can be obtained more accurately, and plane distribution data in a three-dimensional space for an object having a Z value can be obtained.

  Hereinafter, a method of calculating a three-dimensional component set of (X, Y, Z) with a distance (L / α) as a Z value with respect to a point (X, Y) in the image plane will be specifically described.

  First, a case where there is one rectangular parallelepiped object will be described.

  As shown in FIGS. 3A to 3C, a horizontal synchronous sweep signal (continuous image signal) is obtained from three images. In the continuous image signal, h is a rectangular parallelepiped side luminance, i is a rectangular parallelepiped front luminance, j is a background luminance, and k is an edge line luminance.

  Among these obtained continuous image signals, the displacement (ΔP) of the position of the characteristic signal intensity (for example, the position of h, i, j, etc. in FIGS. 3A to 3C) and the moving distance of the camera ( By associating the proportionality coefficient (α / L) of equation (1) with respect to ΔD) with the initial position of the characteristic signal intensity, a three-dimensional space in an object having a Z value related to the distance in the Z direction from the continuous image signal Can be obtained.

  Further, a case will be described in which a small rectangular parallelepiped is nearby and hidden behind an object (large rectangular parallelepiped) during shooting as shown in FIG.

  As shown in FIG. 4, when the situation of moving the camera from the left to the right in the drawing is displayed in eight stages, when a horizontal synchronous sweep signal (continuous image signal) is obtained from each of the obtained eight images, Corresponding image signals having the shapes shown in FIGS. 5A to 5H are obtained, and the small rectangular parallelepiped is hidden behind the large rectangular parallelepiped at the time shown in FIG.

  An example of a method for obtaining the Z value related to the distance in the Z direction from the continuous image signals in FIGS. 5E and 5F will be described below.

  First, four characteristic signal lines are observed in the image signal (sweep signal) in FIG. 6A (corresponding to FIG. 5E). When this signal is traced to FIG. 6B (corresponding to FIG. 5F), signals 1 and 4 change slowly and signals 2 and 3 change fast or large.

Accordingly, if the amount of change of each signal line is ΔP ₁ , ΔP ₂ , ΔP ₃ , ΔP ₄ ,
ΔP _{1 to} ΔP ₄ <ΔP _{2 to} ΔP ₃ (2)
It becomes the relationship shown by the inequality (2). “˜” indicates that they are substantially equivalent.

And when obtaining the Z value, that is, the distance to the object surface, based on the above-described formula (1),
Z _{1 to} Z ₄ > Z _{2 to} Z ₃ (3)
The inequality (3) is obtained. It should be noted that “˜” also indicates that they are substantially equivalent.

  Since there is no feature line for obtaining the background Z value, if the background Z value is infinite, these Z values are illustrated on the horizontal sweep line H in FIG. 6A as shown in FIG. , The Z value is obtained. It is assumed that interpolation is performed between the Z values.

  Here, the broken line in FIG. 7 is another possibility (solution) obtained by processing with one horizontal sweep. In fact, if the front and side surfaces are painted in a uniform color, If there is no characteristic signal due to a stain or the like, it cannot be distinguished.

  However, by processing the entire image using the entire horizontal line, the Z value of the closed region such as the cube surface or side surface can be estimated, and which solution can be determined.

  Many of the actual images are a set of various lines and points, and each of them can be processed as a feature signal, so that each has a Z value, and the Z value interpolation within the surface becomes easy.

  Here, α in Equation (1) is a coefficient characterized in the image receiving system as described above, and is a proportionality constant that links ΔP and (1 / L) ΔD.

  When the image receiving system is a simple image receiving system such as a film or an imaging tube that is simply imaged through a lens (not shown by a camera lens), α is projected from the lens as shown in FIG. It is equal to the distance to (film, imaging tube, etc.).

  In addition, when using an image obtained by projecting on a large screen as a continuous image, the magnification of the image projected on the projection plane in the case of a simple image receiving system is set to the distance in the case of a simple image receiving system. The product is α.

  In addition, when an image converted into an analog signal is used as a continuous image, ΔP is used as a horizontal signal. Therefore, the unit is voltage (volt), and a coefficient for converting the unit is used. The product is α. In the case of a digital signal, the signal is the number of dot units (integer and dimensionless but in units), so α multiplied by the length corresponding to each dot (piece) is α. .

  In any case, α is a coefficient determined by the image receiving system that does not depend on the size or position (L) of the object (subject) to be imaged. This coefficient can be determined before use by calibrating in advance using a standard subject.

  By the way, although the above explanation was the recognition of the Z value by a simple luminance signal, a case where a background and an object pattern overlap most difficult with conventional techniques is considered in consideration of a more general image. For example, there are bright and dark striped patterns on the background wall, and there are also bright and dark striped patterns on the surface of the rectangular parallelepiped that is the object. . This is a state in which the conventional image processing method using two-dimensional shape pattern extraction cannot be identified because the patterns overlap. If the background wall surface is sufficiently far away from the position of the rectangular parallelepiped (distance L), the signal position is shifted by ΔP only when the camera is displaced by ΔD, and therefore, as shown in FIGS. As described above, the rectangular parallelepiped is separated from the background wall surface in the three-dimensional XYZ space by assigning (X, Y, Z) values to each pixel by the method described above. In addition, since there is a background wall portion observed in the vicinity of the cuboid that disappears due to the movement of the cuboid range, it can be determined that the cuboid portion and the background portion are not continuous, and each is recognized as an independent object. It is also possible to do.

  Such a horizontal signal processing makes it possible to send a distance signal (Z) in pairs with each horizontal signal. In the digital processing, the distance signal (Z) can be added to the storage memory area as new data in addition to the luminance or color signal (RGB) of the two-dimensionally distributed (X, Y) pixels.

  These processes can be applied in the same manner even when a plurality of objects exist at different distances as described above, and a Z value is obtained for each pixel (X, Y), so that each object is illuminated from the camera. It is possible to obtain a result like a process in which only the reflected part is taken out by throwing.

  In the description of the present invention, an example in which the camera is moved is taken up, but it is obvious that the same effect can be obtained when the camera is fixed and the image is displaced using a mirror. In addition, although the explanation has been given using an example in which the camera movement plane is perpendicular to the camera focal direction, the camera movement is considered as a vector having a direction and a magnitude, and the vector is projected onto the vertical plane (camera focal direction). It is clear that the same effect can be obtained with the components. That is, it is not necessary to move the camera strictly on the vertical plane, and the same effect can be obtained even if there is a deviation of several tens of degrees due to the error range of the drive mechanism in the camera and the accuracy of the focal direction. This is equivalent to the fact that an object projected on the camera may not have the object at the center of the camera.

  Further, the moving direction of the camera is not limited to one direction and may be a plurality of directions. When there are multiple camera movement directions, the same result can be obtained by tracking the pixel signal in the direction of the line segment of this orbit, for example, the camera revolves around the circle without rotating the camera lens. I can do it. Since a plurality of directions are used, the three-dimensional surface distribution data becomes more accurate.

  Next, pixel data having a three-dimensional value (three-dimensional pixel data) obtained by signal conversion is transferred to a means for specifying an object, that is, a recognition processing device. The recognition processing apparatus is not different from a normal computer having a CPU and a memory. Therefore, all the processing from now on will be described with respect to the algorithm constituting the program.

  As described above, since pixel data having a three-dimensional value (three-dimensional pixel data) can be separated as an independent object, if the object of interest is an object, the three-dimensional pixel data of the object thus obtained is It becomes surface data and forms part of the surface data of an actual three-dimensional object. However, the X value and Y value used in the three-dimensional pixel data (X, Y, Z) are the values obtained by the image receiving system from the direction axis of the photographing center according to the Z value (L) (L / α). Use only stretched ones.

  At this stage, not only the projected cross-sectional shape of the object that can be confirmed in the direction of the line of sight from the camera (focal direction), but also its three-dimensional shape and arrangement can be acquired as pixel data having three-dimensional values (three-dimensional pixel data). . Here, the projected cross-sectional shape is the same as the result of the object shape (two-dimensional image) obtained by the conventional image processing technique based on the two-dimensional shape pattern extraction.

  The present invention realizes the above processing at high speed without using a conventional image processing method based on two-dimensional shape pattern extraction, and further enables to have a three-dimensional extent of an object shape as three-dimensional pixel data. As well as a device.

  Since a part of an actual object is extracted as a three-dimensional part by using the three-dimensional pixel data, it must match the actual object as a three-dimensional spatial spread. This is a function realized because the pattern shape is not simply a two-dimensional projection.

  Therefore, it is possible not only to recognize the object shape as a projection pattern shape as in the prior art, but also to newly recognize a real object. Since the 3D pixel data according to the present invention must match a corresponding part of an actual object, by preparing 3D CAD model data of the object, a partial shape match can be obtained from the surface data. It becomes possible to search with. Here, when the partial shapes match, it means that it can be recognized which part of the actual object matches, so in the (X, Y, Z) space where the three-dimensional pixel data exists, the three-dimensional CAD model. By arranging data in the space, it is possible to adjust the arrangement so that a portion where the shape of the three-dimensional CAD model data matches the pixel data portion belonging to the object is connected. As a result, in this space, not only the 3D pixel data in the range visible from the camera, but also the 3D CAD model data reproduces the part of the object disappeared due to the shape of the back side and the shadow of the object in the foreground. It becomes possible to make it.

  The partial shape matching can be realized by various methods. One is using the three-dimensional CAD model data that is an ideal surface function and the three-dimensional pixel data (measured value) obtained from observation (imaging), and using the square sum of each corresponding distance as an evaluation function. In this method, the least square method is implemented in a three-dimensional space. In addition, since the projection pattern of the target surface can be easily created from various directions because it is configured as three-dimensional pixel data, it is also possible to newly adopt pattern processing here. However, the pattern processing employed here is a process step that does not reach the conventional pattern processing using a two-dimensional captured image.

  Furthermore, if the time change of these three-dimensional pixel data is tracked, the movement of the object can be observed, and if it is a recognized object, the movement of the object can be extracted. That is, the object moves on the background surface data created by other unrecognized 3D pixel data by embedding 3D CAD data in the spatial area of the 3D pixel data of the object that can be recognized. It is also possible to draw background pixel data and object three-dimensional CAD data on a computer monitor. Thereby, it can be easily understood that this method is an algorithm for realizing a three-dimensional projection from an actual captured image onto a virtual space in a computer.

  When a digital image signal is used, processing can be performed with respect to an arbitrary camera image direction without using a horizontal sweep signal. Specifically, if the camera is displaced in the direction of a straight line indicated by the relationship “y = ax” on the XY plane formed by the plane of the image, it can be realized by two methods.

For example, as shown in FIG. 9A, the same processing as the analog horizontal sweep signal processing can be realized by repeating pixel data acquisition in an oblique direction. However, the swept portion deviated from the region by the oblique processing is subjected to the remainder processing [that is, the remainder x ′ obtained by dividing x by x _max for the portion _exceeding the maximum value x _max in the horizontal direction x of the image data region. This is a process for acquiring image data of coordinates (x ′, y ′). However, the minimum value x _min in the horizontal direction x _min = 0. , And supply data of the non-swept portion in the area [ie, move the data of the upper left blank triangular portion to a position beyond x _max on the right side as shown in FIG. 9B. In addition, the periodic boundary portion (line on x _max ) is not treated as the same object even if it has the same Z value between adjacent data across the boundary.

  The other is that the camera image is rotated until the straight line “y = ax” becomes horizontal, and as shown in FIG. 10, the rotation coordinate system (x ′, y ′) obtained from the horizontal minimum value to the maximum value is obtained. In this method, the vertical direction is also swept from the minimum value to the maximum value. In this case, the above-described processing on the boundary line is not necessary, for example, adjacent pixels having the same Z value are not regarded as the same object, and the processing in the sweep in the region outside the image area (or boundary line) is ignored. Just do it.

  That is, sequentially proceeding with the relationship point indicated by “y = ax” in the digital signal is the same as the sweep in the “y = ax” direction, and the image signal corresponding thereto is also at the position (x, y). Correspondence can be achieved by interpolation (for example, weighted average) of data (luminance, etc.) held by pixels from surrounding pixel points. As a result, a continuous signal that can be handled in the same manner as the horizontal sweep signal used in the analog description is obtained. The term “continuous” as used herein means that adjacent position data are connected, and whether analog or digital is a particularly dense set of pixels necessary for the processing of the present invention is not a problem.

  By the way, in the description so far according to the present invention, the surface distribution data of the three-dimensional space in the object is obtained using the continuous image signal composed of the sweep signal of each image. You may obtain | require using the following continuous image difference signals obtained by the difference of a sweep signal. Hereinafter, a method using a differential signal instead of the signal of FIG.

  Here, the difference is the difference in signal intensity between adjacent pixels in the case of a digital signal, and in the case of an analog signal, a signal obtained as a result of passing through a so-called analog differentiation circuit or an AD (Analog to Digital) converter. It is the signal which took the difference of the adjacent value of the data digitized through. Therefore, the differential signal may be expressed as a differential signal as long as the signal intensity is relatively handled.

  The signal shape shown in FIG. 1A of the continuous image signal composed of the sweep signal of each image is the same as the signal shape shown in the graph of FIG. 11A when the continuous image difference signal is used. Become. The boundary portion becomes a whisker-like signal in the range of the width of the boundary line in the positive and negative directions. By using this signal as a characteristic signal, ΔP can be obtained in the same manner as described above. In the case of a difference, since it is a difference in signal intensity between adjacent pixels, a step between the background and the cube surface appears as a difference peak, and after passing the boundary, the difference value becomes zero and becomes pulse-like (whisker-like) ) Signal.

  When the signal passes through an analog differentiation circuit, the signal shape is as shown in FIG. As a circuit for realizing this, there is a well-known circuit, for example, the simplest differentiation circuit (FIG. 11C) including a capacitor C and a resistor R.

  In this way, the position and orientation of the actual object can be known through the step of obtaining the surface distribution data of the three-dimensional space of the object based on the continuous image signal or the continuous image difference signal and the step of specifying the object.

  Accordingly, when the above-described method or apparatus according to the present invention is applied to the field of inspecting small-sized product parts manufactured in large quantities such as connectors of electronic devices, for example, it becomes possible to recognize the direction of each small-product part. The robot work load when assembling the parts can be significantly reduced.

  For example, when mounted on a running car, the shape of the car can be recognized by identifying the direction of other cars that are stopped, including shadows that are not reflected in the image. It becomes possible to specify. In addition, by recognizing the object ahead through the car glass, it is possible to cut out an object other than the matched shape (car) first, and to recognize the cut out part by matching the partial shape. Pedestrians can be recognized. Therefore, it is possible to bridge to the function of predicting / inferring the danger of opening the door and the possibility of pedestrian jumping.

FIG. 1A is a graph showing a relationship between a horizontal sweep signal constituting a continuous image signal and signal intensity, and FIG. 1B is an explanatory diagram of an image obtained by photographing a rectangular parallelepiped object. Explanatory drawing which shows the relationship between the object and to-be-photographed object. 3A to 3C are explanatory diagrams of three images obtained by photographing a rectangular parallelepiped object, and a graph showing a relationship between a horizontal sweep signal and luminance corresponding to each image. Explanatory drawing which shows the state which images two large and small objects which are to-be-photographed objects. 5A to 5H are explanatory diagrams of eight images obtained by photographing two large and small objects, and graphs showing the relationship between the horizontal sweep signal corresponding to each image and the luminance. FIGS. 6A to 6B are explanatory diagrams of two images obtained by photographing two large and small objects, and a graph showing a relationship between a horizontal sweep signal corresponding to each image and luminance. FIG. 7 is a graph showing a relationship between a Z value obtained based on the continuous image signal shown in FIGS. 6A to 6B and a horizontal sweep signal. FIGS. 8A to 8B are graphs showing the relationship between the horizontal sweep signal of the image and the luminance when the pattern of the object to be imaged and the background pattern overlap. FIGS. 9A to 9B are explanatory diagrams showing a sweep method that does not use a horizontal sweep signal for a digital image signal. Explanatory drawing which shows the sweep method which does not use a horizontal sweep signal with respect to a digital image signal. FIG. 11A is a graph showing the relationship between the horizontal sweep signal constituting the continuous image difference signal and the difference signal intensity, and FIG. 11B is the input signal and the differential when the analog continuous image difference signal is used. FIG. 11C is an explanatory diagram of a signal, and FIG. 11C is a configuration explanatory diagram of a differentiating circuit when an analog continuous image difference signal is used.

Claims (7)

In the method of recognizing the three-dimensional shape of an object based on continuous images taken by the camera with the direction from the camera to the object being imaged as the Z direction,
A continuous image is obtained by continuously photographing the object while moving the camera in one direction or a plurality of directions directly or indirectly in a plane perpendicular or substantially perpendicular to the Z direction, and a sweep signal of each image Obtaining a continuous image signal comprising:
Among the obtained continuous image signals, the proportional coefficient (α / L) of the following formula (1) with respect to the displacement (ΔP) of the position of the characteristic signal strength and the movement distance (ΔD) of the camera is characteristic. Obtaining surface distribution data in a three-dimensional space in an object having a Z value related to a distance in the Z direction from the continuous image signal in association with an initial position of signal intensity;
A method for recognizing a three-dimensional shape of an object, comprising the steps of specifying an object by partial match search from the obtained surface distribution data of the three-dimensional space.
ΔP = (α / L) · ΔD = α · (1 / L) ΔD (1)
However, L is the distance from the camera to the object to be photographed, and α is a coefficient of (1 / L) ΔD in the above formula (1) determined by the image receiving system.   2. The method for recognizing a three-dimensional shape of an object according to claim 1, wherein the continuous image signal is a continuous image difference signal obtained by a difference between sweep signals in each image.   3. The method for recognizing a three-dimensional shape of an object according to claim 1, wherein the continuous image signal or the continuous image difference signal is obtained using a horizontal synchronous sweep signal in each image.   3. A method for recognizing a three-dimensional shape of an object according to claim 1 or 2, wherein the camera is periodically or directly moved in one or a plurality of directions in a plane perpendicular or substantially perpendicular to the Z direction. .   5. The method for recognizing a three-dimensional shape of an object according to claim 1, wherein the camera is moved directly or indirectly in a circular direction or an elliptical direction within a plane perpendicular or substantially perpendicular to the Z direction. In an apparatus for recognizing the three-dimensional shape of an object based on continuous images taken by the camera with the direction from the camera to the object to be photographed as the Z direction,
A continuous image is obtained by continuously photographing the object while moving the camera in one direction or a plurality of directions directly or indirectly in a plane perpendicular or substantially perpendicular to the Z direction, and a sweep signal of each image Means for obtaining a continuous image signal comprising:
Among the obtained continuous image signals, the proportional coefficient (α / L) of the following formula (1) with respect to the displacement (ΔP) of the position of the characteristic signal strength and the movement distance (ΔD) of the camera is characteristic. Means for obtaining surface distribution data of a three-dimensional space in an object having a Z value related to a distance in the Z direction from the continuous image signal by associating with an initial position of signal intensity;
An apparatus for recognizing a three-dimensional shape of an object, characterized in that a main part is constituted by means for specifying the object by partial matching search from the obtained surface distribution data of the three-dimensional space.
ΔP = (α / L) · ΔD = α · (1 / L) ΔD (1)
However, L is the distance from the camera to the object to be photographed, and α is a coefficient of (1 / L) ΔD in the above formula (1) determined by the image receiving system. The apparatus for recognizing a three-dimensional shape of an object according to claim 6, wherein the continuous image signal is a continuous image difference signal obtained by a difference between sweep signals in each image.

Images