JP3415868B2 - Apparatus for detecting external shape of three-dimensional object - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [0001] BACKGROUND OF THE INVENTION 1. Field of the Invention
Automatically detects the contour of the overlapping part of the sewing object such as
With this, it is used for sewing with a sewing machine,
It can also be used to generate the suture locus data
Without using previously prepared reference data, etc.
More effectively detect the external shape of the three-dimensional object
For detecting the external shape of a three-dimensional object that can be output
I do. [0002] 2. Description of the Related Art LSI (large scale integrated circu)
it) design technology and digital circuits embedded in LSI
Various digital technologies such as design technology of LSI and LSI manufacturing technology
Technological progress is remarkable. For example, in recent years
Is a very high performance CPU (central processing
 unit) is one chip. Also, image processing, etc.
An arithmetic circuit that performs relatively special processing exclusively
An LSI is also provided. Furthermore, the storage of the semiconductor storage device
There is also a remarkable increase in capacity, and digital
Technology is rapidly evolving. In addition, the processing capability of the CPU and the like as described above is improved.
As the image processing technology advances, for example, in factories,
Parts inspection, blood cell inspection in medical field, cytology, etc.
, The number of parts in the image to be inspected, blood cells, cells
Counting the number of
I have. [0004] For inspection and object recognition by image processing,
For example, there is a technique called template matching. Obedience
The template matching that has been performed since then
Inspection of a two-dimensional template prepared in advance
Up and down in the two-dimensional image to be recognized or recognized
Moves left and right to find similarity with the template
It is to evaluate. For example,
Based on the similarity assessment performed while moving the template
And, for example, the magnitude of the value of the correlation function obtained by this
Inspecting and recognizing target images
It is. Further, a three-dimensional object can be
Examination and recognition of the body are also performed. For example, by multi-view
Multiple images obtained, for example, shot with a binocular TV camera
3D object inspection and recognition
There is something to do insight. This is, for example,
Therefore, for each of the multiple images obtained,
In addition, by performing
Of multiple template matchings obtained as
By integrating the results, the target 3D object
Inspection and recognition are performed. [0006] Inspection and recognition of such a three-dimensional object
For example, data on template matching
By comparison with the reference data
For the three-dimensional object,
Can be recognized. Also, for such a three-dimensional object,
In the inspection and recognition, so-called artificial intelligence
The use of the law is also practiced. [0007] SUMMARY OF THE INVENTION However, as described above,
Inspection and recognition of conventional three-dimensional objects, such as
Standards prepared in advance, such as data on partner matching
It was to use data and the like. For example,
Anything that is not template matching like this
The reference data was used. For example, three-dimensional
For the target object, a predetermined pattern, for example, a stripe shape
And shoot it with a binocular TV camera.
Of the external shape of a three-dimensional object using a pair of stereo images
In the past, some reference planes for 3D objects
Using data prepared in advance for
It is. As described above, the template matching
Tertiary using reference data prepared in advance, such as data to perform
There are various problems in inspecting and recognizing the original object. For example, for a target three-dimensional object
Must be specified in advance and
Data must be obtained in advance and the inspection
The objects are limited. Also within the inspection
The contents and the recognition contents are also limited. example
For example, the contour of the overlapping part of the sewing object such as leather of arbitrary shape
Automatically detects and stitches with sewing machine
To generate the suture trajectory data
It was difficult. [0010] Further, a template match as described above.
The time until the result is uncertain
And the amount of calculation increases.
For example, moving a template and evaluating its similarity
The value of the correlation function is greater than or equal to a predetermined value.
Iterative processing must be performed until a location is found.
No. The present invention is to solve the above-mentioned conventional problems.
Data on template matching.
Data without using reference data prepared in advance.
More effectively detecting the external shape of a three-dimensional target object
To provide a three-dimensional object external shape detection device capable of
With the goal. [0012] SUMMARY OF THE INVENTION The present invention providesOf three-dimensional objects
In the outer shape detection device,For a three-dimensional target object,
An illumination device for projecting a predetermined pattern, and the predetermined pattern
Is photographed of the three-dimensional object on which
An imaging device for obtaining an image, the three-dimensional object, and the imaging
The lighting device, while maintaining a relative positional relationship with the device,
Projection of the predetermined pattern from the object onto the three-dimensional object
The three-dimensional object and the imaging to change the state
Position of the lighting device with respect to the deviceIn the Z-axis directionChange
LightingTransferMoving device and the three-dimensional pair for the imaging device
The relative position of the elephant objectA predetermined amount of movement in the X-axis directionchange
The target object moving device to be
And saidClear dressPlaceMove in Z axis directionShooting while letting
Storing a plurality of the two-dimensional images obtained from the photographing device;
A first image memory, and the second image memory stored in the first image memory.
Said object which was not activated at this time after obtaining the dimensional image
Object moving deviceActivated only once,
The relative positional relationship of the three-dimensional target object
Changes once by the fixed travel distanceThe predetermined continuous shooting pattern
In the above,Clear dressPlaceMove in Z axis directionShooting while letting
And storing a plurality of the two-dimensional images obtained from the photographing device.
A second image memory, and the first image memory and the second image memory.
When obtaining a plurality of two-dimensional images stored in a two-image memory
Controlling the illumination driving device and the target object moving device
Movement control device, and the first image memory
The two-dimensional image, and stored in the second image memory
The two-dimensional imageOf performing differential processing between pixels
Calculating means for performing a product-sum operation on the differential processing result.
And output the external shape data of the three-dimensional target object
Sum of productsWith arithmetic meansBy achieving the above task
WasThings. [0013] The present invention is based on a paper including one of the inventors of the present invention.
Of the theory published in the literature, the actual shape of the three-dimensional object
This is embodied as a detection device. The argument
As a sentence, "Reconstruction of stereo images using differential
Current System ”Shigeru Ando, Transactions of the Society of Instrument and Control Engineers, 23-
4, 319/325, 1987 (hereinafter referred to as Paper A)
"Velocity vector component using spatio-temporal differentiation of image
Cloth Measurement System "Shigeru Ando, Transactions of the Society of Instrument and Trajectory Control, 2
2-12, 1330/1336, 1986
"Sentence B"); "Differential binocular stereo to detect binocular intensity difference
Visualization and Its Application to Gloss Perception "Shigeru Ando, Measurement Automatic Control
Transactions, 23-6, 619/624, 1987 (hereinafter
"The dynamic three-dimensional cumulative synthesis mechanism
Differential binocular method with Shigeru Ando and Toshiyuki Tabei, automatic measurement
Transactions of the Society of Control Engineers, 24-6, 628/634, 1988
Year (hereinafter referred to as thesis D). In the present invention, the template map
Reference data prepared in advance, such as data on
More effective external shape of 3D object without using
The theory published in the above-mentioned paper to detect
In addition, as described in detail later, a first image group and a second image group
Multiple sets of images from groups or more
I use it. These image groups are
TransferLighting device by a moving device or a target object moving device described later
ToMoveWhile moving or moving the 3D object
It is obtained. As a result, the previously prepared
Targeted with higher accuracy without using quasi-data
The external shape of the three-dimensional object can be detected. FIG. 1 is a device layout diagram showing the gist of the present invention.
is there. In FIG. 1, a three-dimensional object 1 is
On the other hand, an illumination device used in the external shape detection device of the present invention.
The arrangement of the device 12 and the photographing device 14 is shown. The one or more lighting devices 12 are tertiary.
A predetermined pattern is projected onto the original target object 1.
is there. The predetermined pattern projected by the illumination device 12
Therefore, the present invention does not specifically limit this.
The predetermined pattern is detected when one external shape is detected.
It is only necessary that the cycle is constant. For example, the predetermined pattern is a grid-like pattern.
Pattern, stripe pattern, polka dot pattern,
Or some random pattern with no periodicity
You may. The predetermined pattern to be projected is determined by detecting the outer shape.
The shape of the three-dimensional object to be output (the shape of the main object)
And arrangement position, lighting described laterTransferOf lighting system by moving device
Direction of movement of the target object moving device
It is preferable to determine in consideration of the above. For example,
LightTransferThe direction of movement of the lighting device by the
The change in the shading of the pattern
It is desirable that fineness is ensured. Note that the predetermined
For the turn, consider the resolution of the photographing device 14
Is determined. The photographing device 14 is connected to the lighting device 12.
The three-dimensional object on which the predetermined pattern is projected
A two-dimensional image is obtained by photographing the body. The photographing device
14 is, for example, a CCD (charge coupled device).
e) A television camera or the like using the method can be used. The above-mentioned place to the three-dimensional object 1 is
While projecting a fixed pattern, the second
When obtaining the original image, in the present invention,
Based on the theory indicated by D, the position of the lighting device 12
Alternatively, the tilt angle is changed. That is, the position of the lighting device 12
Is changed by the photographing device 14,
This is to obtain an original image. Got these multiple
The two-dimensional image, therefore, is
The positions of the positions 12 are different from each other, and the three-dimensional pair
The states of the patterns projected on the elephant object 1 are also different from each other.
For example, in the Z-axis direction shown in FIG.
2 to obtain a plurality of two-dimensional images
You. Further, in the present invention, the aforementioned
As described above, which is performed while changing the position of the lighting device 12.
The photographing of a plurality of two-dimensional images is performed by the photographing device 14.
Changing the relative positional relationship of the three-dimensional target object 1
I try to do it multiple times. Such two-dimensional shadows
Performing image capturing a plurality of times is described in the above articles A to D.
It has not been announced. For example, in a predetermined continuous photographing pattern,
illuminationTransferMotion deviceWorkWhile moving, with the photographing device 14
First, a plurality of two-dimensional images obtained by shooting
Store in memory. Do not activate this at this time
Operating the target object moving device by a fixed amount,
In the predetermined continuous shooting pattern, the illuminationTransferActivate the moving device
While photographing with the photographing device 14,
Storing a number of said two-dimensional images in a second image memory.
To Further, in the present invention, the first image
A plurality of the two-dimensional images stored in a memory;
2 For a plurality of the two-dimensional images stored in the image memory
I.e. the lightingTransferOf motionWorkMoving or moving the target object
For two-dimensional images obtained with the operation of the motion device
Thus, differentiation processing between predetermined pixels is performed.
Also, a predetermined product-sum operation is performed on such a differential processing result.
And output the external shape data of the three-dimensional object.
I am trying to. As described above, in the present invention, a plurality of 2
According to the aforementioned predetermined continuous photographing pattern for obtaining a two-dimensional image,
At least two times, and differentiate the results
By performing logical and product-sum operations, the three-dimensional object
To obtain a gradient pattern. For example,
The change amount in the Z-axis direction in FIG. 1, that is, for example, the height
The pattern of the amount of change is determined. The gradient
A turn is a relative quantity, for example, an absolute height in the Z-axis direction.
It is different from the numerical value indicating the value. In the present invention, the predetermined continuous
Performs a shooting pattern multiple times and differentiates the results
And a gradient with relative value property obtained by multiply-accumulate operation
By obtaining layout pattern data, for example,
Bases prepared in advance, such as data on rate matching
Detects the external shape of a three-dimensional object without using quasi-data.
To be able to get out. In the present invention, the pre-prepared
Since no reference data is required, for example,
The shape of a three-dimensional object can also be detected. For example,
The outline of the overlapping part of the sewing object such as leather of arbitrary shape
Even if this is detected automatically, the
Also generate the suture locus data used for
Can be used at the time. That is, in this case,
Regarding the shape of the sewing material such as
There is no need to register in advance. [0027] BRIEF DESCRIPTION OF THE DRAWINGS FIG.
I will tell. FIG. 2 is a diagram showing the outer shape of a three-dimensional object according to the present invention.
It is a block diagram showing composition of an example of an output device. As shown in FIG. 2, the outer shape of the present embodiment
The shape detection device includes the imaging device 14, ShineLight moving device 16
A target object moving device 18, a first image memory 22,
Second image memory 24, movement control device 26, differential operation
And a product-sum calculator 34.
Further, the external shape detecting device of the present embodiment is shown in FIG.
The illumination device for projecting the three-dimensional object 1 as described above.
A device 12 is also provided. The photographing device 14 of the present embodiment is
As shown in FIG. 1, the above-mentioned not shown in FIG.
The three-dimensional target object 1 is photographed. The shooting range is about 25cm
× 16 cm. Further, the photographing device 14 and the three-dimensional pair
The distance from the elephant 1 is about 50 cm. Of the photographing device 14
The number of effective pixels is 640 x 480, with 8 bits for each pixel.
Output the brightness data of the The obtained image is (2 × 2
= 4) For each pixel, average to 8-bit 1-pixel brightness data
And shrink. This is due to the interlaced scanning of video imaging.
Averaging to reduce the effect of In the present embodiment, the lighting device 12
Prescribed 2mm pitch stripe pattern on the surface
Is projected onto the three-dimensional target object 1. stripe
The predetermined pattern has a stripe shape of X in FIG.
It is orthogonal to the axial direction. In the present embodiment, the illumination moving device
16 is a phase between the three-dimensional object and the photographing device 14.
While keeping the positional relationship constant, the lighting device 12
The projection state of the predetermined pattern on the three-dimensional object is changed.
In order to make the lighting device 12
In the direction v. Specifically, before
Each time one frame is photographed by the photographing device 14, about 200
μm (about 200 μm / frame
Speed). The target object moving device 18 includes the photographing device.
The relative positional relationship of the three-dimensional object 3 with respect to the
In order to change, in the X-axis direction of FIG.
The target object 1 is moved by a predetermined movement amount. Ingredient
Physically, the predetermined movement amount is about 0.25 mm. The pair
The elephant object moving device 18 is, specifically, the three-dimensional object.
The table 18a and the table 18a shown in FIG.
And a driving device for moving the table 18a in the X-axis direction.
And a control device for this purpose (mobile disassembly
About 5 μm). The movement control device 26 includes the illumination device 1
2 at a constant speed of v
A plurality of two-dimensional images at a predetermined position,
Controls shadow patterns. The movement control device 2
6 shows such a predetermined continuous photographing pattern,
With the body moving device 18,
When performing several times, the movement of the target object moving device 18 is started.
U. In addition, such an illumination moving device 16 or the object
Along with the control of the body moving device 18, the predetermined continuous shooting as described above is performed.
A plurality of the two-dimensional images obtained by the shadow pattern
Stored in the first image memory 22 or the second image memory 24
It also controls the timing to make it work. The first image memory 22 is provided as described above.
For the photographing device 14 performed in a predetermined continuous photographing pattern,
For storing a plurality of two-dimensional images obtained by shooting
It is. Further, the second image memory 24 stores
After recording the two-dimensional image in the first image memory 22,
Further, the target object moving device 18 moves the predetermined amount of movement.
After moving the three-dimensional target object 1,
A plurality of images obtained from the photographing device 14 in a continuous photographing pattern
Is stored. In the following,
The plurality of two-dimensional images stored in the first image memory 22
The image is referred to as a first image group. On the other hand, the second image memo
A plurality of the two-dimensional shadows as described above stored in the
The image is hereinafter referred to as a second image group. In this embodiment, the first image group and the
A predetermined differential between pixels with respect to the second image group
Perform processing. That is, the first obtained as described above
The image group and the second image group will be described later (20).
A predetermined differentiation process as shown in Expressions (22) to (22) is performed.
U. Further, the product-sum calculator 34 calculates the differential operation.
A predetermined product is added to the differential processing result
Performs a sum operation and outputs the external shape data of the three-dimensional object
I do. Specifically, the product-sum calculator 34 will be described later.
By performing the processing as shown in Expression (23),
The three-dimensional target object 1 viewed from the photographing device 14
Data on the gradient pattern at each position. FIG. 3 is a view showing the outside of a three-dimensional object in this embodiment.
It is a flowchart of a shape detection process. In the flowchart of FIG.
In step 104, the first image group is photographed.
U. This means that the illumination device 12 projects a predetermined pattern.
Meanwhile, the lighting device is
The device 12 is moved from the predetermined moving start point S at the moving speed v.
While moving, a plurality of the two-dimensional
That is, a shadow image can be taken. For example, in this embodiment
Then, the first image group is referred to as the second imaging group.
Every time the illumination device 14 is moved by about 200 μm.
Frame shooting, and finally, for each imaging group,
A two-dimensional image of a total of 128 frames is captured. Thereafter, at step 106, the illumination transfer is performed.
The return operation of the moving device 16 is performed. The return operation
Thus, the position of the lighting device 12 is shifted to the movement start point S.
Is restored. Next, in step 108, the target object
The moving device 18 moves the three-dimensional target object 1 by a predetermined amount.
Move by the amount. That is, in the X-axis direction in FIG.
It is moved by a constant moving amount, about 0.25 mm in this embodiment. In step 112, the second imaging group is set.
Then, the lighting device 12 is
From the movement start point S to the movement speed v (same as the first imaging group)
While moving at about 200μm / frame)
Also, while projecting a predetermined pattern by the lighting device 12,
A plurality of the two-dimensional images are obtained by the photographing device. An example
For example, in the present embodiment, a total of 12
An eight-frame two-dimensional image is obtained. The secondary obtained at this time
The original image is stored in the second image memory as the second image group.
24. Subsequently, at step 114, the differential operation is performed.
The first image group and the second image group are calculated by an arithmetic unit 32.
Is subjected to a predetermined differentiation process. Also, at step 116
Is the product-sum operation unit for such a differential processing result.
34, a predetermined product-sum operation is performed, and the three-dimensional object
Obtain the external shape data of the object. In the next step 118, the above step
Shape data obtained in step 116, that is, the direction gradient pattern
By comparing the data of the
The step is detected. This is, for example, the sewing of certain leather
What is sewn, such as other leather, is superimposed on the upper surface of the object
To obtain the contour of the upper sewn object
is there. Also, in step 122, step 1
Sewing on the sewing machine based on the contour of the step
Generate the suture locus data to be used
You. Hereinafter, the principle of the present embodiment will be described with reference to mathematical expressions.
Will be described. First, as shown in FIG.
The device 12 sends the three-dimensional object 1 to the
By projecting a tripe-like pattern, the illumination
Light projected from the device 12 forms a three-dimensional distribution of light and dark
I do. That is, illumination (projection or projection) from the illumination device 12 is performed.
In front of the lighting device 12, which is shadowed, the three-dimensional object
Not only on the surface where the object 1 is placed, but also in that space,
According to the above-mentioned stripe pattern to be projected
A three-dimensional distribution of light and dark is formed. Further, such a three-dimensional distribution of light and darkness of light is as follows.
It is moved with the movement of the lighting device 12. That is,
By moving the illumination device 12 in the Z-axis direction,
In parallel with this, move the three-dimensional distribution of light intensity as described above
I do. Here, the illumination moving device 16 is located at the original position.
(Movement start point S), and the target object moving device 18
In the initial state, ie, the movement start time t = 0.
And the three-dimensional distribution of light intensity as described above is i (x, y
 , Z). Also, the illumination moving device 16 controls the illumination.
Let v be the speed at which the lighting device 12 moves in the Z-axis direction.
Then, the three-dimensional distribution of light brightness at a certain time t is i
(X, y, z−vt). Also this
Between the three-dimensional distribution of light and darkness of the light and the three-dimensional object 1
Intersection, that is, the third order represented by i (x, y, z−vt)
Surface of the three-dimensional target object 1 determined by the original distribution
The three-dimensional object according to the brightness of the space at the upper position
The intensity of light incident on the surface of No. 1 is determined. Here, the landmark on the surface of the three-dimensional object 1 is
The height distribution is expressed as a function of the XY coordinates serving as a reference plane, h
(X, y). Also, the surface of the three-dimensional object 1
X-y coordinate, which is also the reference plane
Let r (x, y) be a function of I mentioned earlier
According to the altitude distribution of the surface of the three-dimensional object 1,
The intensity of light incident on the surface of the three-dimensional target object 1 is i (x,
y, z −vt−h (x, y)). Here, the surface of the three-dimensional object 1 is completed.
The total diffusion surface, that is, the incident light
Even if it is a thing, its reflection is uniformly reflected in all directions
And the lighting device 1
2, the projection direction of the predetermined pattern is θ
Assuming that it is an angle, from the lighting device 12
Of the three-dimensional object 1 by projection of the predetermined pattern
Surface luminance distribution f₁(X, y, t) is expressed as
Can be [0052]      f₁(X, y, t)           = I (x, y, z − vt − h (x + Δcos θ, y + Δsin θ))             × r (x + Δcos θ, y + Δsin θ) (1a) In the above equation (1a), Δ is sufficiently small.
When the derivative in the θ direction is taken, the following equation is obtained. [0054]      f₁(X, y, t) = i_z(X, y, z)           × (z −vt− h_{dif ・ deg}(x + Δcos θ, y + Δsin θ))           × r (x + Δcos θ, y + Δsin θ) + i (x, y, z)           × r_{dif ・ deg}(X + Δcos θ; y + Δsin θ)           = −i_z h_{dif ・ deg}+ I r_{dif ・ deg}              … (1b) Therefore, the first order term of the Taylor expansion is taken.
Then f₁By approximating (x, y, t),
Can be expressed as follows. [0056]      f₁(X, y, t) to ir + Δ (−i_z h_{dif ・ deg}+ I r_{dif ・ deg})           = Ir-Δi_zrh_{dif ・ deg}+ Δir_{dif ・ deg}            … (2) The above equations (1a), (1b) and (2)
In, the subscript z represents the Z-direction derivative. Subscript dif
deg represents the derivative in the θ direction. Also, in the above formula (2)
Ir is the zero-order term of the Taylor expansion. Δ (− i_z h
_{dif ・ deg}+ I r_{dif ・ deg}) Is the first order term. Tables represented by the above equations (1) and (2)
Surface luminance distribution f₁(X, y, t) is replaced by the first
It corresponds to one image group. At this time, the three-dimensional
After moving the target object 1 by a predetermined amount of movement in the Z-axis direction
Surface luminance distribution, that is, the surface luminance distribution f of the second image group
_Two(X, y, t) can be expressed as
You. [0059]      f_Two(X, y, t)           = I (x, y, z − vt − h (x − Δcos θ, y − Δsin θ))             × r (x −Δcos θ, y −Δsin θ) (3) In the above equation (3), the above equation (1) is also used.
A similar approximation can be made. That is, the above equation (3) is
It can be approximated by the following equation. [0061]      f_Two(X, y, t)-ir + Δi_zrh_{dif ・ deg}−Δir_{dif ・ deg}  … (4) In the above equations (1) to (4),
Is the photographing device 1 based on the altitude distribution h (x, y).
Disregard the displacement in the two-dimensional image obtained in step 4.
I have to. The surface of the three-dimensional object 1 is at the elevation
As shown by the distribution h (x, y), the surface
are doing. Therefore, the second order obtained by the photographing device 14
In the original image, before that position relative to the XY reference plane
The position of the surface of the three-dimensional target object 1 is different.
However, assuming that the photographing device 14 is sufficiently far away,
Distortion is a secondary problem and should be ignored
are doing. Next, the sum of equations (2) and (4) is
Then, f₊Is defined. F₁+ F_Two= F₊      … (5a) Further, the value of f shown in the above equation (5a)₊Is
It can be expressed as an equation. F₊= 2ir ... (5b) Further, the difference between the above equation (2) and the above equation (4) is obtained.
For f_-Is defined. [0068] f₁−f_Two= F_-                    … (6a) Further, the value of f shown in the above equation (6a)_-Is
It can be expressed as an equation. [0070]      f_-= -2Δi_zrh_{dif ・ deg}+ 2Δir_{dif ・ deg}      … (6b) First, f (x, y, t) is differentiated as follows.
I do. [0072]      f_t(X, y, t) = (δ / δt) f (x, y, t)           = (Δ / δt) ｛(x, y, z −vt−h (x, y))           × r (x, y)｝           = I_z(X, y, z−vt−h (x, y)) × (−v) × r (x, y)           = −v i_z(X, y, z−vt−h (x, y)) × r (x, y)                                                                 … (7) In the above equation (7), h (x, y) is expressed as h
(X ± Δcos θ, y ± Δsin θ), and r (x, y
 ) Is replaced by r (x ± Δcos θ, y ± Δsin θ)
Is f_1t(X, y, t) or f_2t(X, y, t)
It is. These f_1t(X, y, t) and f_2t(X, y,
t), approximation from the above equation (1) to the above equation (2)
Performs an approximation to the first order of the Taylor expansion, as in
And the following equation. [0074]      f_1t(X, y, t) ~ -vi_zr + Δv i_zz h_{dif ・ deg}                         −Δv i_z r_{dif ・ deg}              … (7a)      f_2t(X, y, t) ~ -vi_zr + Δv i_zz h_{dif ・ deg}                         + Δv i_z r_{dif ・ deg}              … (7b) Therefore, f_1t+ F_2tTo f_tIs defined as
It can be calculated as follows. [0076] f_t= F_1t+ F_2t~ -2 v i_z r… (8) The above (6b) is sequentially changed as follows.
Type. [0078]      f_-~ Δ (-2 vi_zr) h_{dif ・ deg}+ Δ (2ir) (r_{dif ・ deg}/ R)           = Δf_t h_{dif ・ deg}+ Δv f₊(R_{dif ・ deg}/ R)           = ΔT f_t h_{dif · deg +}ΔT f₊ρ_{dif ・ deg}        … (9) In the above equation (9), ΔT and ρ
Are as follows. ΔT = Δ (10a) ρ = log r… (10b) Thus, f₊And f_-And f_tDefine
After that, while using the covariance matrix,
In order. Here, the time trajectory group (spatio-temporal gradient
Cloth) into a three-dimensional vector (f₊, F_t, F_-)^TAs
Define. Also, do not move the illumination device 12 in the Z-axis direction.
All the time during which the photographing by the photographing device 14 is performed,
And the observation window に 関 す る for the pixels near the point of interest.
The covariance matrix (vector S) for this is
Can be represented. [0082] (Equation 1) The above-mentioned method of selecting the observation window 前述
Depends on the application, such as the characteristics of the three-dimensional object 1
It was done. In the following description, the illumination device 12
Space to be a point of interest, with a sufficiently long travel time
Is treated as a single pixel, and the emphasis is placed on the spatial resolution of the measurement.
Case. The S shown in the above equation (11)₊₊, S_ttas well as
S_+-By the judgment amount J_DETIs defined as follows.
Also, the determination amount J_DETAnd the predetermined threshold J_MINAnd
Form a significant distribution by comparing
Is determined. That is, a three-dimensional vector (f₊, F_t, F
_-)^TForm a significant distribution. [0085] J_DET= S₊₊S_tt-S^Two _+-> J_MIN      … (12) When rejected by the above equation (12),
In the examples, labels for indeterminate (measurable) areas
Then, the processing at the point of interest ends. On the other hand, the above equation (12)
If determined to be significant in the given plane
When forming the distribution, f₊, F_t, F_-What is in between
An equal linear relationship exists. When such a linear relationship exists, the following equation is obtained.
As shown in FIG.
And the unevenness gradient h_{dif ・ deg}And the logarithmic reflection coefficient gradient ρ
_{dif ・ deg}Can be solved. [0088] (Equation 2) The solution solved by the above equation (13) is
Using the matrix element of the (vector S) described above,
Can be represented. [0090]     ΔTh_{dif ・ deg}=-(S₊₊S_t--S_{+ t}S_+-) / J_DET      … (14)     Δρ_{dif ・ deg}=-(S_ttS_+--S_{+ t}S_t-) / J_DET        … (15) Here, ΔTh expressed by the above equation (14)
_{dif ・ deg}Represents a directional gradient pattern.
On the other hand, ΔTρ_{dif ・ deg}Is a table of the three-dimensional object 1
It represents the reflection coefficient at the surface. Here, J_ERRIs defined as
You. [0093]     J_ERR= (ΔTh_{dif ・ deg}S_t-+ Δρ_{dif ・ deg}S_+-+ S_-) S / J_DET                                                             … (16a) Further, it is determined whether or not to form a specific plane distribution.
Constant, that is, whether or not the area is an effective area is determined by ΔTh
_{dif ・ deg}Error variance estimator J_ERRAnd threshold J_MAXAnd by
Therefore, the following equation can be used. J_ERR<J_MAX      … (16b) Note that the judgment shown in the above equation (16b)
And J_DETAnd J_ERRAnd the threshold
J_MAXFor the calculation method of
As mentioned. The surface distribution is expressed by the above equation (16b).
First, the cause that is determined not to be
In the object moving device 18, the three-dimensional target object 1 is
When moved by 2Δ, f₁(X, y, t) to f_Two
Shift image change 2Δi to (x, y, t)_zrh
_{dif ・ deg}Exceeds the range of the first order approximation. Here, f (x, y, t + Δt) ′ is expressed by the following equation.
Is defined as [0099]     f (x, y, t) '= f (x, y, t + .DELTA.t) (17a) Further, the illumination device 12 is moved in the Z-axis direction.
Time of image sequence consisting of multiple 2D images obtained while performing
The shift of the axis by Δt is caused by shifting the surface of the three-dimensional object 1
Can be said to be equal to the shift v Δt of the altitude in the Z-axis direction.
For this reason, f (x, y, t + Δt) and f (x, y, t
 ) 'Can be expressed as follows: [0101]     f (x, y, t) '         = F (x, y, t + Δt)         = I (x, y, z −vt− (h (x, y) + vΔt)) r (x, y)                                                             … (17b) Therefore, the planar distribution is formed as described above.
F to prevent loss₁And f_TwoMark
It is effective to shift the realization times from each other. in this way
Shifting can easily cause excessive shifting as described above.
Can be offset. Further, Δt is changed while the lighting device 12 is not moved.
To an integral multiple of the time interval for capturing two-dimensional images
So that the selection of a plurality of obtained two-dimensional images
Can respond. That is, replacement of frame numbers etc.
You only need to do this, there is no need to retake. For this reason, rejection is made in the determination of the above equation (16).
For the point of interest, the time t is shifted sequentially.
Search for the position where the flatness of the distribution appears.
You. That is, Δt with respect to time t is calculated as shown in the following equation.
While gradually increasing, from the equation (12), the equation (16b)
The surface distribution is formed by the above equation (16b).
Repeat until it is determined that
Search for the appearance position. When flatness is found in this way,
Add intentional shift amount v Δt to detected unevenness gradient
Then, an unevenness gradient is obtained as shown in the following equation. [0106]     Δt = ± Δt₀, ± 2Δt₀, ..., ± NΔt₀        … (18)     ΔTh_{dif ・ deg}'= ΔTh_{dif ・ deg}+ V Δt (19) As described above, in the present embodiment,
According to the equation (15) or the formation of the plane distribution is immediately
If not determined by the above (19),
Unevenness gradient ΔTh of the three-dimensional target object 1_{dif ・ deg}(I want
Is ΔTh_{dif ・ deg}'). Also,
Unevenness gradient ΔTh_{dif ・ deg}By integrating
Convex gradient map h_{dif ・ deg}(X, y) can also be calculated
Noh. Further, a logarithmic reflectance gradient map ρ
_{dif ・ deg}(X, y) and measurement error dispersion map J
_ERR(X, y) can also be determined. The measurement error
Dispersion map J_ERR(X, y) is the final obtained
Asperity gradient ΔTh_{dif ・ deg}Or concave as described above
Convex gradient map h_{dif ・ deg}Also the reliability of that value
(Precision). Hereinafter, using the above equations (1) to (19),
The theory of the outer shape detection performed in the present embodiment described above
Based on the computer processing actually performed in this embodiment
Will be described. First, in this embodiment, the lighting device
12 while moving it in the Z-axis direction,
1st frame to kth frame, (k + 1) th frame to
The above-mentioned shooting of a plurality of two-dimensional images of 128 frames
This is performed by the device 14. Each two-dimensional image obtained in this way
In FIG. 4, the image is composed of m × n pixels.
(Actually, as described above, the number of effective pixels is 640 × 480.
Is averaged from 2 × 2 pixels to 1 pixel). Follow
For a particular pixel in a particular frame
Can be represented by (i, j, k). In the present embodiment, as shown in FIG.
Obtaining a two-dimensional image of eight frames is based on the three-dimensional object.
After moving the object 1 in the X-axis direction by a predetermined movement amount,
As described above, the operation is performed twice in total. That is,
The first one obtained in this way is the first one described above.
The second one is the second video as described above.
Group. Therefore, a certain frame in the first video group
Some pixels have f₁(I, j, k)
You. In the second video group, there is a certain frame.
For pixels, f_Two(I, j, k)
Wear. In the present embodiment, as described above using equations
It is based on theory, in particular, adjacent to a particular pixel.
For the observation window Γ as described above for pixels,
First, the difference method performed in the computer processing as shown in
By f₊, F_-And f _tAsk for. [0112]      f₊~ F₁(I, j, k + 1) + f₁(I, j, k)                 + F_Two(I, j, k + 1) + f_Two(I, j, k) ... (20)      f_-~ F₁(I, j, k + 1) + f₁(I, j, k)                 −f_Two(I, j, k + 1)-f_Two(I, j, k) ... (21)      f_t~ F₁(I, j, k + 1)-f₁(I, j, k)                 + F_Two(I, j, k + 1)-f_Two(I, j, k) ... (22) In the above equation, f₊, F_-And f_tIs required
Then, it corresponds to the product-sum operation as shown in the above equation (11).
Perform the operation to be performed. The operation corresponding to the equation (11) is
In the embodiment, a simple multiply-accumulate operation as shown in the following equation is used.
Is performed as a computer process. [0114] (Equation 3) The product-sum operation shown in the above equation (23)
In the following, the dynamics of the data in the multiplication operation
Since the cleansing is doubled, it is necessary to
Therefore, in this embodiment, a 32-bit integer type or a single
Arithmetic processing is performed with a precision real number type. Again
By the expression (11) and the expression (23), S₊₊,
S_{+ t}, S_+-… Etc. are required, the above (12)
By performing an operation corresponding to Expressions (19) to (19),
The constant gradient ΔTh_{dif
・ Deg}And the unevenness gradient map h_{dif ・ deg}(X
 , Y) and so on. FIG. 5 shows that f₁
FIG. 4 is a diagram showing a pixel of interest and a pixel adjacent thereto.
is there. FIG. 6 shows the f_Twoof
FIG. 3 is a diagram illustrating a target pixel and pixels adjacent thereto. In FIGS. 5 and 6, reference numeral 5 is used.
Pixels adjacent to and adjacent to the target pixel to be
Each of the six pixels is shown. Reference numeral 3 is
It is a step on the three-dimensional target object 1. The f shown in FIG.₁The noted pixel 5 of
And f shown in FIG._TwoAnd the noted pixel 5 of
It has become a thing corresponding to. That is, for example,
5 f₁6 of the k-th frame of FIG.
The f_TwoThe noted pixel 5 of the k-th frame
And its photographing time t. Therefore, the slit
Are also identical to each other. In addition, FIG.
 f₁The pixel of interest 5 of the (k + 1) th frame of the
F in Figure 6_TwoOf the pixel of interest 5 in the (k + 1) th frame
However, for the position of the pixel and the shooting time t,
They correspond to each other. In FIG._TwoAbout the shooting time t
As the time elapses, the dark part of the slit moves. or,
F in FIG._TwoFor the kth frame and the (k +
1) With the frame, as the shooting time t elapses,
The position of the dark part of the slit is moving. FIG. 7 shows the three-dimensional object of the present embodiment.
It is a top view showing an example. FIG. 8 shows the three-dimensional object.
It is a perspective view of a body. The three-dimensional data shown in FIG. 7 and FIG.
The target object 1 is an example of the target of the present embodiment.
And the leather sewn object 1b and the upper surface of the leather sewn object 1a.
And 1c. In addition, these leather sewing products
1b and 1c follow the contour of the overlap
The sewing is performed on the leather sewing object 1a. Again
As described above with reference to the flowchart of FIG.
The example three-dimensional object outer shape detection device uses such a sewing method.
To generate the suture trajectory data for performing
This is to automatically detect the contour of the overlapping part.
You. FIG. 9 shows an example of the three-dimensional object.
It is a top view showing a step difference detection result. In FIG. 9, FIG. 7 and FIG.
The three-dimensional target object 1 targeted by the present embodiment shown in FIG.
The example was obtained by the external shape detection device of this embodiment.
An indented step is shown. In FIG. 9, a dashed line
u and w are steps that are higher on the left than on the right. Broken line 1v
Is a step on the left side lower than the right side. The result of such step detection is shown in FIG.
The actual 3D target object 1 shown in FIG.
Therefore, accurate detection of the step is performed. FIG. 10 shows an example of the three-dimensional object.
The unevenness gradient ΔTh obtained for the example _{dif ・ deg}And integrate
Bumpy map h_{dif ・ deg}(X, y) are shown. The unevenness map h shown in FIG.
_{dif ・ deg}As for (x, y), FIG.
Matches the actual shape of the three-dimensional object 1 shown in FIG.
It has become. Also, in FIG.
The surface condition of the leather sewn object 1b or 1c
There is also shown a situation where there are more irregularities than the surface condition of a. As described above, the three-dimensional object of this embodiment
According to the external shape detection device of the present invention,
A first video group consisting of a plurality of two-dimensional images
Using a second image group consisting of a number of two-dimensional images,
The differential processing by the arithmetic unit 32 and the product-sum arithmetic unit 34
In the product-sum operation, the irregularity gradient ΔTh_{dif ・ deg}Ask for etc.
Can be Also, based on this,
Automatic detection of the outline of the overlapped part of the leather sewing product
can do. Also, based on this, sewing with a sewing machine
Generate the suture locus data to be used for joining
It is possible. The data used in this embodiment is mainly
The first image group obtained by the photographing device 14 and the second image group
This is data on the image group, and requires
Such as data on template matching
It does not require reference data prepared in advance such as data. or,
As described above, the unevenness map h_{dif ・ deg}(X, y)
For example, when using
It does not require reference data prepared in advance. Therefore, according to the present embodiment, as described above,
The shape of the superimposed leather sewn product may be arbitrary,
The shape, data similar to the shape,
There is no need to prepare the data in advance. Therefore, according to this embodiment,
For example, the contour of the overlapping part of the sewing object such as leather of arbitrary shape
Automatic detection with high efficiency and improved work efficiency
Can be done. Further, according to the test of the present embodiment,
Even if the leather sewn items superimposed on
As a result, it has been confirmed that the step can be recognized.
The shape of the step that can be detected can be any shape such as a straight line or a curve.
Shape can be detected effectively
Have been. Further, according to the present embodiment, such leather sewing
It can detect not only the contour of an object but also the outer shape of a free-form surface
That has been confirmed. [0131] As described above, according to the present invention, the
Template matching data, etc.
3D target object outline without using reference data
Excellent shape can be detected more effectively
The effect can be obtained.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a side view showing an arrangement of a lighting device and a photographing device used in a three-dimensional object outer shape detection device of the present invention. FIG. 2 is a three-dimensional object outer shape detection device of the present invention. FIG. 3 is a block diagram showing a configuration of an outer shape detection process in the embodiment; FIG. 4 is a diagram showing a pixel configuration of a two-dimensional image obtained by the photographing apparatus in the embodiment; 5 is a diagram showing a pixel of interest with respect to f ₁ of the embodiment. FIG. 6 is a diagram showing a pixel of interest with respect to f ₂ of the embodiment. FIG. 7 is an example of a three-dimensional target object which is a target of the embodiment. FIG. 8 is a perspective view of the three-dimensional target object. FIG. 9 is a plan view showing a step detection result of the three-dimensional target object in this embodiment. FIG. 10 is a plan view of the three-dimensional object. Diagram showing the concavo-convex map obtained by the above description [Description of reference numerals] 1. Part 5: Target pixel 12: Illumination device 14: Imaging device 16: Illumination moving device 18: Target object moving device 22: First image memory 24 ... Second image memory 26: Movement control device 32: Differential calculator 34: Sum of products Arithmetic unit

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-4-147001 (JP, A) JP-A-3-209112 (JP, A) JP-A-57-24810 (JP, A) (58) Field (Int.Cl. ⁷ , DB name) G01B 11/00-11/30

Claims (1)

(57) Claims 1. An illuminating device for projecting a predetermined pattern onto a three-dimensional target object, photographing the three-dimensional target object on which the predetermined pattern is projected, and A photographing device that obtains a shadow image, and while changing a relative position relationship between the three-dimensional target object and the photographing device, to change a projection state of the predetermined pattern from the lighting device to the three-dimensional target object, the position of the lighting device with respect to the three-dimensional object and said imaging device and an illumination moving device for varying the Z-axis direction, a predetermined amount of movement of the relative positional relationship of the three-dimensional object in the X-axis direction with respect to the imaging device changes and the target object moving device which, at a predetermined continuous shooting pattern, the irradiation AkiraSo location in the Z-axis direction
After obtaining a first image memory for storing a plurality of the two-dimensional images obtained from the image capturing apparatus while taking a picture while moving the image capturing apparatus, and obtaining the two-dimensional image to be stored in the first image memory,
The object moving apparatus once that did not operate at this time
Only actuate the three-dimensional object with respect to the imaging device
Change the relative positional relationship of the object once in the X-axis direction by a predetermined movement amount
It is of at predetermined continuous shooting pattern, the irradiation AkiraSo location shot while moving in the Z axis direction, and a second imaging memory for storing a plurality of the two-dimensional shadow image obtained from the imaging device, the first When obtaining a plurality of the two-dimensional images stored in the first image memory and the second image memory, a movement control device that controls the illumination driving device and the target object moving device, and is stored in the first image memory. the two-dimensional imaging, and, in the two-dimensional imaging to be stored in the second imaging memory
On the other hand, a calculating means for performing a differentiation process between pixels, and a product-sum calculation process on the result of the differentiation process,
Outer shape detection device of a three-dimensional object, characterized by comprising: a product-sum operation means for outputting the external shape data of a three-dimensional object.