JP2010266202A - Image processing apparatus for photogrammetry, on-site recording system using the same, baggage recording system, and laser beam machining system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus for photogrammetry capable of performing photogrammetry of a building structure or the like by the use of a camera, and improving quality control for the construction process. <P>SOLUTION: The image processing apparatus for photogrammetry includes: a camera; a measuring implement; and an information calculation terminal having a display means, an input means, a calculation means, a storage means, and a data transfer means. In the processor; the measuring implement is so structured as to be a rectangular flat plate or the like having four reference points for photogrammetry forming the vertexes of a rectangle, and is arranged adjacently to an object to be measured; and the information calculation terminal stores the inner orientation parameters of the camera calculated previously by the storage means, captures by the data transfer means an image comprising in the visual field of the camera, the object to be measured and the measuring implement photographed by using the camera, calculates information on correlations between a camera image and actual coordinates and exterior orientation parameters of the camera by the calculation means from the captured image, and measures two-dimensional coordinate positions and distances on a plane which the measuring implement forms at least at two points or more on the image, a plane parallel with the plane, and planes having known angles of intersection with those planes. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus for photo measurement that performs measurement without providing a reference point for photo measurement on an object to be measured, and measures the position and interval of arbitrary designated points on the object to be measured by image processing, and uses the same. The present invention relates to the technology of the on-site recording system, package recording system, competition content recording system, and laser processing system.

Conventional photo measurement is based on a bundle calculation method for calculating the position of a target point by performing bundle calculation based on images taken from many viewpoints with one camera, or measuring with two cameras whose positions are fixed in advance. A stereo camera system is used.
First, in the case of the bundle calculation method, there is a problem in that it takes time for measurement because the flash photography is performed with a large number of targets (reflective markers) attached to the measurement target. Further, when a distance such as length or interval is required, a scale bar or the like is required.

On the other hand, in the case of the stereo camera system, it is possible to know the three-dimensional coordinates of a point on the measurement target by designating corresponding points of two photographs taken from different angles. Since the corresponding point images of the two photos appear at different points on the photo, the difference in position (parallax) on the photo can be measured with each image, and this parallax includes distance information Thus, distances such as length and interval can be automatically calculated.
However, in the case of the stereo camera system, since the camera is fixed, the photographing range is limited, and it is difficult to photograph from a free direction. Furthermore, in the case of the stereo camera system, it is necessary to manufacture the apparatus itself separately, and thus there is a disadvantage that the cost is higher than the bundle calculation system with one camera.

  In the above-described bundle calculation method, the orientation reference body having six or more reference points whose positions are known at a predetermined position and the measurement target are photographed from a plurality of directions by the same digital camera. Performs external orientation based on the position of the point on the photo and the relative position of the reference point in the orientation reference body, and uses the value of the external orientation element to correspond to the specified point on the measurement target on the two photos Based on the position, the three-dimensional coordinates are obtained by forward intersection, the value of the external orientation element, the three-dimensional coordinates for the specified point on the measurement object and the position of the image on each photograph, the three-dimensional coordinates for the reference point, on each photograph A photo measurement method for performing bundle adjustment on the position of the image is known (Patent Document 1).

According to the photographic measurement method disclosed in Patent Document 1, it is not necessary to provide a reference point such as a photographic measurement target (reflective sign) on the measurement target, and high-precision photographic measurement is possible for a large number of points. It can be performed.
However, in the case of the photographic measurement method disclosed in Patent Document 1, a measurement jig for calibration requires many targets of 6 or more, the jig shape is complicated, and a photographed image is necessary for measurement. There is a problem that three or more sheets are required, and it takes time to take a photograph for measurement.

  In addition, as for the purpose of photo measurement, the size, number, pitch, cover thickness, etc. of reinforcing bars, etc. are used to check whether or not the arrangement of reinforcing steel structures in building foundation construction and building construction is being carried out in accordance with regulations. May be used for the purpose of measuring and recording. In such a field, field workers must measure and record many confirmation points. Therefore, it is required to minimize the time and the number of shots required for shooting so as not to burden the field workers.

  Further, there are laser processing such as teaching of articulated robots and laser welding as applications for photo measurement, and there is a demand for an apparatus that can satisfy the required accuracy with a simple configuration.

JP 2007-147522 A

  In view of the above situation, the present invention provides an image processing apparatus for photo measurement that can easily measure a photo of a building structure or the like using a single camera and contribute to improvement of quality control in a building process. With the goal.

In order to achieve the above object, a photometric image processing apparatus according to the first aspect of the present invention comprises: (1) one camera means;
(2) a measuring jig;
(3) In an image processing apparatus for photo measurement comprising: an information computing terminal comprising display means, input means, computing means, storage means, and data transfer means;
The measurement jig of (2) above is
It is a rectangular flat plate or square frame structure with four photometric reference points that form a rectangular vertex, and is built into the object to be measured, or adjacent to the object to be measured or arranged in parallel at a known distance And
The information calculation terminal of (3) above is
The storage means stores the internal orientation element of the camera means calculated in advance,
The data transfer means captures one image including the object to be measured and the measurement jig photographed using the camera means within the camera field of view,
The computing means calculates the image of the camera means and the related information of the real coordinates and the external orientation element from the captured one image,
By means of the display means and the input means, at least two or more selected points on one captured image are formed on the plane formed by the measuring jig, its parallel plane, and a plane having a known intersection angle with them. The dimension coordinate position and the distance are measured.

According to such a configuration, a single measurement jig having four reference points for photo measurement with a single camera can be captured on the measurement target in the photograph only by photographing one piece together with the measurement target portion. The position coordinates of any point can be calculated.
Here, as the camera means of (1), a commercially available digital camera, a measurement camera, or a digital video camera (for example, 15 FPS (Frame Per Second)) can be used. This camera means needs to perform camera calibration which analytically obtains the state of the camera when a photographic image is taken. Specifically, an external orientation element such as the position and orientation (rotation angle) of the camera at the time of photographing, and an internal orientation element such as a shift in focal length, principal point position, and lens distortion coefficient are obtained. The external orientation element is automatically calculated from the acquired image because it depends on the actual shooting situation at the shooting site.

The internal orientation elements are internal parameters of the following a) to d) and are stored in advance in the storage means of the information calculation terminal.
a) Focal length As a distance from the center (principal point) of the camera lens to the imaging surface (CCD sensor or the like), for example, a value is calculated with an accuracy of 0.1 microns.
b) Misalignment of principal point The amount of misalignment between the camera principal point and the center position of the imaging plane in each of the two plane (x, y) directions depends on the assembly accuracy at the time of camera manufacture. Calculate the value with an accuracy of 1 micron.
c) Radial direction lens distortion correction coefficient When a digital camera captures an image, light is received by a flat imaging surface through a curved lens, and each pixel on the captured image increases in distortion as the distance from the center increases. Therefore, a coefficient for correcting such distortion is calculated.
d) Tangential lens distortion correction coefficient Tangential lens distortion is caused by the fact that the lens and the imaging surface are not installed in parallel, and depends on the assembly accuracy at the time of camera manufacture, so the correction coefficient is calculated. .

  The measuring jig (2) has a rectangular flat plate or rectangular frame structure, is disposed adjacent to the measurement target, and is photographed together with the measurement target. By photographing with the measuring jig and the object to be measured, the position and direction (posture) which are the external orientation elements of the camera means are automatically calculated, and the above a) to d) of the camera means obtained in advance. The position of an arbitrary point on the captured image is calculated from the information including the internal parameters.

The information calculation terminal may be a mobile computer such as a desktop computer, a notebook computer, a handheld computer, or a PDA (Personal Digital Assistance). The storage means may be composed of RAM (volatile memory), PROM (writable memory), or the like.
In addition, the data transfer means of the information processing terminal is not limited to the one that transfers image data by connecting to the camera means with a wired cable like a USB interface, but the one that transfers image data by wireless communication such as infrared data transfer. But you can.
In the information operation terminal, the display means and the input means have a plane formed by the measuring jig for the selected point of at least two or more points on one captured image, its parallel plane, and a known intersection angle with them. The points for measuring the two-dimensional coordinate position and distance on the plane will be described.
At least two or more selected points on one captured image are a distance or width measured on the image by a display unit such as a liquid crystal panel and a camera-captured image displayed on the screen by an input unit such as a mouse. Two or more selected points for measuring the length scale.
Measurement of the two-dimensional coordinate position and distance on the plane formed by the measurement jig means that the measurement jig is placed so as to overlap the surface of the measurement target, and the position (coordinates) and length of the measurement target (Distance) is measured.
The measurement of the two-dimensional coordinate position and distance on the plane parallel to the plane of the measurement jig means that the measurement jig is installed parallel to the surface to be measured at a certain distance, The position (coordinates) and length (distance) of the measurement target are measured.
Measurement of 2D coordinate position and distance on a plane formed by a measuring jig or a plane having a known angle of intersection with the parallel plane
The known intersection angle is 30 degrees, 60 degrees, 90 degrees, etc., and when a cube is placed on the measurement jig, the plane formed by the measurement jig is the bottom surface and has a known intersection angle with the bottom surface. The plane is, for example, a side surface of a cube having a known intersection angle of 90 degrees. If the crossing angle is 90 degrees, the three-dimensional position coordinates and dimensions (distances) of X, Y, and Z can be obtained.
Here, the measurement jig may be incorporated in the measurement target. When a rectangular flat plate or a rectangular frame-shaped jig having four photometric reference points forming a rectangular vertex is attached in advance to the object to be measured, or four photometric forming a rectangular vertex Only the reference point for use may be attached to a part of the surface to be measured.

Moreover, the image processing apparatus for photo measurement of the second aspect of the present invention is
(1) one camera means;
(2) a measuring jig;
(3) an information computing terminal comprising display means, input means, computing means, storage means, and data transfer means;
In an image processing apparatus for photo measurement with
The measuring jig of (2) has a rectangular flat plate or rectangular frame structure having four photometric reference points forming a rectangular vertex,
The information calculation terminal of (3) above is
The storage means stores the internal orientation element of the camera means calculated in advance,
The data transfer means captures two or more images including the measurement target and the measurement jig photographed from different angles using the camera means within the camera field of view,
The computing means calculates the external orientation element of the camera means from each captured image,
At least two or more common points on the measurement target of each captured image are selected by the display unit and the input unit, and the three-dimensional coordinate position and distance of the selected point are measured.

According to such a configuration, a single measurement camera having four photo measurement reference points with a single camera can be taken from different angles together with the measurement target location, and only two images can be captured. The three-dimensional position coordinates of an arbitrary point on the measurement target can be calculated with high accuracy.
Here, the camera means and the like are the same as in the first aspect, and the description thereof is omitted.
Selecting at least two or more common points on the measurement target of each captured image by the above display means and input means means that, for example, when using a computer, two photographs are displayed on the screen simultaneously on the left and right Then, a designated point on the measurement target in the left image is clicked with the mouse to make the computer recognize the position, and a corresponding point in the right image is found and clicked with the mouse to make the computer recognize the position.
More specifically, when the computer recognizes a specified point in the left image, the corresponding point is obtained by drawing an epipolar line on the right image by calculation and searching for a point having the highest correlation with the specified point on this epipolar line. Is identified.

In the field recording system of the present invention, the information calculation terminal in the above-described image processing device for photo measurement according to the first aspect or the second aspect further includes a contour extraction unit and a report unit,
By means of data transfer means, an image of the reinforcing bar arrangement at the construction site is acquired from the camera means,
Select the rebar on the displayed image by input means,
The contour extraction means extracts the rebar contour,
Calculate the number of reinforcing bars, the thickness of the reinforcing bars, and the width of the reinforcing bars by the calculation means.
The report means is configured to record and report the number of reinforcing bars, the reinforcing bar thickness, the reinforcing bar width, the photographed image, the photographing date and time, and the specification information of the construction site.
Here, the contour extraction means is a program process executed by a processor which is a calculation means of the information calculation terminal, and extracts a shape outline of the measurement target on the acquired image by image processing.
The report means means that data information such as the number of reinforcing bars, the thickness of the reinforcing bars, the width of the reinforcing bars, the captured image, the shooting date and time, the specification information of the construction site, etc. Output to the device.

In the field recording system of the present invention, the information calculation terminal in the above-described image processing device for photo measurement according to the first aspect or the second aspect further includes a contour extraction unit and a report unit,
The image of the concrete of the building is acquired from the camera means by the data transfer means,
Select the crack on the displayed image by input means,
Extract the cracks by the contour extraction means,
By calculating means, calculate the number of cracks, crack length, crack thickness,
The reporting means is configured to record and report the number of cracks, crack length, crack thickness, photographed image, photographing date and time, and specification information of the concrete of the building.
Here, information on cracks such as the number of cracks, crack length, and crack thickness is selected from the photographed image. In the case of small cracks, concrete cracks are difficult to distinguish from surface dirt and stains. For this reason, usually, when shooting a crack on site, the crack location is traced with chalk or the like to make it easier to determine the crack location on the photographic image.
In the case of this apparatus, when calculating the number of cracks, crack length, and crack thickness, a crack is selected on the displayed image so as to take into account the portion traced by the chalk.

In the package recording system of the present invention, the information calculation terminal in the above-described photoprocessing image processing apparatus according to the first aspect or the second aspect further includes a contour extraction unit and a report unit,
The data transfer means obtains an image of the luggage shape from the camera means,
Use the input means to select a package on the displayed image,
The contour extraction means extracts the contour of the luggage,
The calculation means calculates the package shape dimensions,
The report means is configured to record and report the package shape dimensions, captured images, shooting date and time, and package specification information.

The laser processing system of the present invention includes the above-described photometric image processing apparatus according to the first aspect or the second aspect, a processing head that performs laser irradiation, and a work on which a processing target is placed,
By the data transfer means, from the camera means, an image including the processing head, the workpiece, and the processing target placed on the workpiece in the camera field of view is obtained.
Select the processing point to be processed on the displayed image using the input means,
By calculating the difference data of the three-dimensional position coordinates of the machining head and the machining point,
The data transfer means transfers differential data or calculation data to the control unit of the machining head.

The laser processing system of the present invention includes at least two camera means,
An information computing terminal comprising at least display means, input means, computing means, storage means, and data transfer means;
A processing head for performing laser irradiation;
A workpiece on which a workpiece is placed;
A laser processing system comprising:
The workpiece is provided with four photometric reference points that form a rectangular vertex,
Using the camera means, take an image including the processing head, the workpiece, and the processing target placed on the workpiece in the camera field of view,
In the information computing terminal,
Capture data by means of data transfer,
The display means displays one image selected from the captured images,
Select the processing point to be processed on the displayed image using the input means,
By calculating means, the difference data of the three-dimensional coordinate position of the machining head and the machining point is calculated,
The data transfer means transfers differential data or calculation data to the control unit of the machining head.

  Here, the processing head includes a fiber portion serving as a laser irradiation source, a transmission optical system for collimating laser light emitted from the fiber, an optical system including only a galvano mirror for guiding the laser light, and laser light. A galvano scanner for scanning and a lens for condensing the scanned beam or an fθ lens are preferable. Further, the lens is provided with a protective glass for protecting from dust accompanying processing.

  The processing head also scans the laser beam, a fiber part that serves as a laser irradiation source, a reflection optical system for collimating the laser beam emitted from the fiber, an optical system that includes only a galvanometer mirror that guides the laser beam. The galvano scanner is preferably composed of a lens for condensing the scanned beam or an fθ lens.

Further, the processing head includes only a fiber part serving as a laser irradiation source, a reflection optical system for collimating the laser light emitted from the fiber, a reflection optical system for condensing, and a galvano mirror for guiding the laser light. It is preferable that the optical system comprises a galvano scanner that scans with laser light.
According to such a configuration, the lens temperature does not rise due to heat absorption when the laser passes through the lens, and a refractive index distribution is generated due to thermal deformation and temperature distribution of the lens, as compared with the case where a transmissive lens is used. Therefore, there is no problem that the focus spot position of the laser shifts.

  The machining head is preferably applied to arc welding, plasma cutting, resistance welding, electron beam welding, or friction diffusion bonding.

Moreover, it is preferable that the camera means further includes a filter means for excluding light emission that becomes a disturbance as a photographed image of the camera under a surrounding environment to be processed.
By adopting such a configuration, it is possible to eliminate the influence of shooting troubles caused by ambient light or smoke during laser processing.

  In addition, an irradiation unit that irradiates the processing target and processing head with a light source in a wavelength band different from the light emission that causes disturbance as a captured image of the camera under the surrounding environment of the processing target, and measures an image in the reflected irradiation wavelength band However, it is preferable that the three-dimensional coordinate position of the desired object can be measured even under the occurrence of disturbance.

Further, it is preferable to further include feedback control means for sequentially calculating the deformation or deviation amount of the machining target that occurs during machining and correcting the position of the machining head.
With this feedback control means, camera shooting is sampled in the middle of processing, or is shot in real time with a video camera, etc., and laser irradiation position coordinates are recalculated from the obtained images, and according to the recalculated position coordinates. The laser scanning program is updated.

According to the image processing device for photo measurement of the present invention, there is an effect that the position coordinates of an arbitrary point can be calculated only by photographing one or more reference plates or frames together with a measurement target portion with a single camera. . As a result, it is possible to measure a portion that is difficult to measure in the past, such as a place with a poor scaffold such as a construction site, or a shape that cannot be touched by hand due to a high temperature or the like.
Further, according to the image processing apparatus for photo measurement of the present invention, by performing image processing, the position and diameter of the connecting portion of a member such as a bridge or a steel frame, the thickness of the member, and the like are extracted with one photograph. can do.
Furthermore, since it becomes possible to inspect from the photograph the position and dimensions of the construction, even after completion, quality control can be improved.

Schematic configuration diagram of the image processing apparatus for photo measurement of the first embodiment Measuring jig for image processing apparatus for photo measurement of embodiment 1 Explanatory drawing (1) of the field recording system of Example 1 Explanatory drawing (2) of the field recording system of Example 1 Explanatory drawing (3) of the on-site recording system of Example 1 Explanatory drawing of the robot teaching system and packing record system of Example 1 Processing Flow of Image Measurement Image Processing Device of Embodiment 1 Illustration of battery cell welding Enlarged view of machining head and workpiece Processing flow of conventional laser processing system Explanatory diagram of problems with conventional laser processing systems Description of conventional laser processing system 1 Detailed processing flow of conventional laser processing system 1 Description of conventional laser processing system 2 Detailed processing flow of conventional laser processing system 2 Configuration diagram of laser processing system of Example 2 Processing Flow of Laser Processing System of Example 2 Explanatory drawing of other examples Conceptual diagram of packing record system of embodiment 3 Conceptual diagram of competition content recording system of Example 4

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The scope of the present invention is not limited to the following examples and illustrated examples, and various changes and modifications can be made.

(Image processing device for photo measurement)
FIG. 1 is a configuration diagram of the image processing apparatus for photo measurement according to the first embodiment. An image processing image processing apparatus 1 according to the first embodiment includes an information calculation terminal including one digital camera 2, a measurement jig 3, display means, input means, calculation means, storage means, and data transfer means. It consists of 6 notebook computers.
In FIG. 1, a measurement target 4 is a reinforcing bar hole (4a to 4c) opened in a concrete block. Both the measurement jig 3 and the measurement target 4 are placed in the camera field of view 5 and an image is taken.
Further, the measuring jig 3 has a rectangular flat plate structure having four (3a to 3d) photo measuring reference points forming the apexes of a square frame as shown in FIG.

3 to 5 show images in which the photo measurement image processing apparatus according to the first embodiment is used as an on-site recording system. In FIGS. 3 to 5, a measurement jig is disposed adjacent to the measurement target.
The notebook computer that serves as the information computing terminal stores the pre-calculated internal orientation elements of the camera means in the memory, and transfers one image including the measurement object and measurement jig photographed by the camera within the camera field of view. The calculation processor captures the image of the camera means, the related information of the real coordinates, and the external orientation element from the captured image, and at least two or more selected points on the captured image The two-dimensional coordinate position and distance on the plane formed by the measuring jig and the parallel plane are measured.

FIG. 3 (1) shows a state in which the number, diameter, and interval of rebar holes opened in the concrete block are measured. In FIG. 3A, the measurement target 4 is a reinforcing bar hole (4a to 4c) opened in a concrete block. An image of both the measuring jig 3 and the measurement target 4 is taken with a single digital camera 2. From the acquired image information, the intervals (D1 to D4, D6 to D9) of the reinforcing bar holes (H1 to H8) can be measured.
Moreover, FIG. 3 (2) has shown a mode that the dimension of steel materials is measured. In FIG. 3 (2), the measurement target 4 is a steel material having a cross section of an I-shape. A measurement jig 3 is installed adjacent to the cross section, and images of both the measurement jig 3 and the measurement target 4 are taken with a single digital camera 2. The dimensions (D1 to D3) can be measured from the acquired image information.
Moreover, FIG. 4 (1) has shown a mode that the crack of a concrete wall is measured. A measurement jig 3 is installed adjacent to the concrete wall of the measurement target 4 and an image is taken with one digital camera 2. From the acquired image information, the shape, length, etc. of the crack (4f, 4g) can be measured.
Moreover, FIG. 4 (2) has shown a mode that the dimension of the steel material of the to-be-measured object 4 is measured similarly to FIG. 3 (2). Here, the diameter and position of the holes (4j to 4l) provided on the side surface of the steel material are measured. Moreover, the dimension of the side surface (interval L between points 4h and 4i) is measured. By combining FIG. 3 (2) and FIG. 4 (2), it can be seen that the dimensions of the overall shape such as the cross section and side surface of the steel material can be obtained with one camera. The measuring jig 3 in FIG. 4B has a trapezoidal shape. Thus, the measuring jig 3 is not limited to a square shape.
Further, FIG. 5 measures the number, diameter (R), and interval (D1, D2) of reinforcing bars embedded in the concrete to be measured 4.

FIG. 6 shows an image in which the photo measurement image processing apparatus according to the first embodiment is used as a robot teaching system / packaging recording system.
Measurement is performed by two digital cameras (2a, 2b). This may be taken by moving one camera.
FIG. 6A shows a system for measuring the three-dimensional position of the machining target of the machine robot 9 and teaching it to the machine robot. The measuring jig 3 is installed in the vicinity of the processing target, and the measurement target 4 at the measurement points (P1 ′ to P5 ′) is measured from different angles using two cameras or by moving one camera. And the measuring jig 3 are photographed together to measure the three-dimensional position of the processing target.
6 (2) shows that the measurement jig 3 is installed on the floor, and a load is mounted on the measurement jig 3 so that the four photo measurement reference points are not hidden. By using the camera or moving one camera and photographing the package of the object to be measured 4 and the measurement jig 3 from different angles, the dimensions of the package (height H, width L, depth) D) is measured.

Next, processing of the image processing image processing apparatus according to the first embodiment will be described with reference to FIG.
First, the camera used for the apparatus is a camera that has been calibrated in advance and calculated internal parameters (internal orientation elements) such as lens distortion (step S1). Next, a measurement jig (reference plate) is set up and photographed with a digital camera together with the object to be measured (step S2). Then, the center point of the measuring jig (reference plate) is automatically extracted (step S3), and the projected coordinates and the actual coordinates are correlated (step S4).
External internal orientation elements such as the direction and orientation of the camera are calculated from the image (step S5), and the measurement target is measured from the image using the obtained internal orientation elements and external orientation elements (step S6).

  When only one image is acquired using one camera, after calculating the internal parameters of the camera in advance and correlating the projected coordinates on the image of the reference plate of the measuring jig with the actual coordinates, Perform planar projection (homography) transformation (if the measurement plane is translated from the reference plane, calculate the camera's external orientation element and perform projection transformation on the measurement plane again), and use the projected coordinates of the measurement target point Find the real coordinates of a point.

  On the other hand, when multiple images are acquired using two cameras, or when multiple images are acquired at different angles by moving one camera, the camera internal parameters are calculated in advance and the measurement treatment is performed. The projection coordinate on the image of the reference plate of the tool is associated with the actual coordinate. After the association, the external orientation elements at the respective photographing positions of the camera are obtained based on the position information obtained by the association. Finally, the actual coordinates of the measurement target point are obtained from the projection coordinates of the measurement target point and the camera parameters.

As an application to acquire only one image, it is effective when measuring the hole diameter and hole pitch on the same plane of the structure, measuring the cross-sectional dimension and thickness of the structural member, etc. . It can also be applied to inspection of cracks on steel and concrete surfaces.
On the other hand, the use of two or more images includes 3D coordinates of arbitrary positions of structures, measurement of distance between two points, measurement of positions and dimensions that are not easy to measure with human hands, location, Applicable to teaching dimension measurement and robot working position.

  As an application to the inspection of cracks, large ones are extracted by image processing, and small ones that cannot be automatically identified are traced on the cracks with chalk, etc. Extraction is performed by image processing, coordinate conversion is performed, real dimensions are extracted, and a front projection image is created at the same time.

(Laser processing system)
Next, a laser processing system will be described as a second embodiment.
The laser processing system according to the second embodiment includes the above-described image measurement image processing apparatus according to the first embodiment, a processing head that performs laser irradiation, and a work on which a processing target is placed. The laser processing system of the present embodiment uses the data transfer means of the image processing apparatus for photo measurement to obtain an image including a processing head, a workpiece, and a processing target placed on the workpiece in the camera field of view from a digital camera. Acquire and select a processing point to be processed on the image displayed by the mouse, calculate the difference data of the three-dimensional position coordinates of the processing head and the processing point using the operation personal computer, and use the data transfer means to calculate the calculated data The difference data is transferred to the control unit of the machining head.

For more specific explanation, when the laser processing system of Example 2 welds the electrode plate of the secondary battery, the problems of the conventional system, the differences between the example and the conventional system, and the advantages of the example The effect will be described below with reference to the drawings.
In recent years, with regard to the welding processing of the electrode plate of the secondary battery, shortening the positioning of the processing head and shortening the welding process have become issues with a view to the future expansion of the battery market.

  FIG. 8 shows an external view of the secondary battery. As shown in the external view, the secondary battery is composed of a stack of battery cells and a connection plate that connects the cells. Here, in the process of lap welding the battery connection plate formed by assembling various parts including the press-processed product, as shown in FIG. 9, three-dimensional such as the height of the connection plate, the inclination of the connection plate, etc. There is a variation. In view of the fact that the conventional welding method requires a great deal of time for positioning before processing and teaching process to the processing robot, reduction of tact time is required.

In the conventional processing method, as shown in the flow of FIG. 10, the height measurement sensor is moved to the connection plate, the height of the connection plate is measured, and the welding head movement amount is calculated. Then, the welding head is moved to the connection plate, and the welding head is scanned to perform lap welding.
Assuming that the number of welding locations is two per cell, the number of connection plates is n in one row, and the number of rows is m in one battery case, the total number of welds is the number of cells in one battery. L locations (L = 2 × n × m). The tact time of the welding process of the connection plate of the secondary battery is [height measurement and adjustment time (x seconds / location) + welding time (y seconds / location)] × L locations.
There is a need for a laser processing system that shortens positioning tact time and laser welding time.

  In recent years, the development of high-power, high-intensity lasers with YAG laser capable of fiber transmission has progressed, and a beam with a deep focal depth and capable of fine focusing can be applied, increasing the possibility of high-speed welding. I have done it. However, since the conventional laser processing system uses an optical system combined with a transmissive lens, there is a problem that the focal position shifts due to a thermal lens effect generated when the laser is absorbed. Such a shift of the focal position due to the thermal lens effect causes a deterioration in welding quality such as seam welding because the power density at the processing point changes (see FIG. 11).

  The laser processing system of the second embodiment uses the image processing apparatus for photo measurement of the first embodiment described above to camera the entire processing head, the workpiece, and the entire battery cell welding allowance placed on the workpiece. An image included in the field of view is captured and the image is acquired. Then, using an operation personal computer which is an information calculation terminal of the image processing apparatus for photo measurement, the three-dimensional coordinates of the battery cell welding margin and the machining head are obtained from the image data, and the welding margin which is the target of irradiation with the laser from the machining head. The relative position of the part is calculated. In other words, the processing point of the processing target on the image displayed on the screen is selected using the mouse of the operation personal computer, and the difference data of the three-dimensional position coordinates between the processing head and the processing point is calculated using the operation personal computer. . The calculated difference data is transferred to the control unit of the machining head.

As described above, in the laser processing system of the second embodiment, the processing head is controlled using the difference data of the three-dimensional position coordinates calculated from the image processing apparatus for photo measurement (for example, when selecting the remote laser welding system, By creating or updating a program for scanning with a laser beam, high-speed welding can be achieved and tact time can be shortened as compared with the conventional case.
That is, the laser processing system according to the second embodiment measures the variation in the three-dimensional installation position of each of the connection plates connected to each cell by stacking a plurality of battery cells and the processing data. Compared with the conventional system to be created, the laser processing program calculates the three-dimensional installation position coordinates of the connection plates at the L locations of the entire battery cell by the photo measurement image processing device and obtains the relative position to the processing head. Create In the laser processing system according to the second embodiment, remote laser welding capable of scanning a three-dimensional laser beam at high speed is performed with respect to a conventional system that performs a welding operation by bringing a processing head close to each of a plurality of connection plates. By applying and creating and updating a program for scanning a laser beam using the three-dimensional installation position coordinate data having variations calculated by the image processing apparatus for photo measurement, tact reduction can be achieved.

  Further, in the laser processing system of the second embodiment, a remote laser head capable of applying a total reflection type mirror and capable of high-speed scanning is developed for a processing head using a transmission lens, and a focal position shift amount using a thermal lens is developed. Is reduced.

Hereinafter, the configuration and processing flow of the laser processing system of the second embodiment will be described in comparison with the conventional system.
First, a conventional system will be briefly described. FIG. 12 shows an enlarged schematic diagram of a machining head and a workpiece of a conventional system that performs a battery cell welding process. Since each battery cell is a press-molded product due to the need for cost reduction, there is a variation in the height of each battery cell. Therefore, in the conventional system, first, before performing the welding process for each battery cell, a scanning scan is performed to perform a three-dimensional shape measurement. Next, the three-dimensional coordinate data is transferred to the laser welding scanning data, and welding for a half circumference is performed. Next, since the cell is thermally deformed after half-circular welding, the shape of the three-dimensional coordinates of the welding allowance after deformation is measured again using a height sensor, and welding is performed to complete.

  FIG. 13 shows a processing flow of the conventional system. A height sensor is installed at the battery cell welding allowance (step S131), the height sensor is scanned along the battery cell welding allowance, and the three-dimensional position coordinates of the welding allowance are measured (step S132). According to the measured position coordinates, the welding head is scanned and welded by a half circumference (step S133), the height sensor is installed at the battery cell welding allowance after thermal deformation, and the three-dimensional position coordinates for the remaining half circumference are measured (step S134). Then, the welding head is scanned and welded in accordance with the position coordinates measured along the battery cell welding allowance after thermal deformation (step S135).

  Since the welded part is composed of a battery cell stack and a connection plate that connects each battery cell, there are three-dimensional variations such as the height and inclination of the connection plate. Thus, in the process of lap welding the battery connection plates having three-dimensional variations, it is necessary to carry out positioning, machining head teaching, and laser welding processes for each welding location by the machining process (see FIG. 14). ). That is, it is necessary to carry out the processing steps of step S131 and step S132 for each welding location. FIG. 15 shows a processing flow to be performed for each welding location. The height measuring sensor is moved to the connection plate (step S151), the height of the connection plate is measured to calculate the amount of movement of the processing head (step S152), the processing head is moved to the connection plate (step S153), Lap welding is performed by scanning the machining head (step S154). These series of processes are repeated L times when there are L places of welding (step S155).

  Next, the configuration and processing flow of the laser processing system of the second embodiment will be described. In FIG. 16, the schematic block diagram of the laser processing system of the present Example 2 is shown. In the laser processing system of the second embodiment, as shown in FIG. 16, the laser light emitted from a laser light source (not shown) passes through a fiber serving as a light guide, and the laser direction is controlled by a total reflection mirror. . Then, laser light is scanned by a biaxial galvano scanner (A axis, B axis), and the entire circumference of the battery cell of the welding allowance is collectively welded.

In the laser processing system of the second embodiment, as shown in FIG. 16, the three-dimensional position coordinates are measured by two digital cameras. These two cameras are fixed to the laser processing apparatus, and the welding allowance around the battery cell and the three-dimensional position coordinates of the processing head are calculated from the image data captured by the two cameras. And the difference data of the three-dimensional position coordinate of a process head and a welding allowance are calculated, and the laser irradiation position coordinate from a process head is calculated. In FIG. 17, the processing flow of the laser processing system of Example 2 is shown. As shown in FIG. 17, after taking an image by the photo measurement image processing apparatus and calculating the laser irradiation position coordinates, a laser scanning program is created according to the calculated position coordinates. By transferring the data to the laser processing apparatus, the laser processing apparatus can collectively weld the entire circumference of the battery cell.
Further, during the welding process, the welding process target may be deformed by heat or the like. In such a case, the camera shooting is sampled during processing, or is shot in real time with a video camera or the like, and the laser irradiation position coordinates are recalculated from the obtained image, and laser scanning is performed according to the recalculated position coordinates. You can also update the program.

  In the embodiment, a remote head using a galvano scanner optical system is used as the processing head, but a conventional fixed optical system welding head may be used. By performing remote scanning of the laser using a remote head with a galvano scanner optical system, it becomes possible to perform high-speed welding as compared with a conventional welding head.

  After calculating the difference data of the three-dimensional position coordinates of the machining head and the welding allowance from the image data of the two cameras, and calculating the laser irradiation position coordinates from the machining head, this data is used as the galvano scanner control unit of the laser machining system. Forward to. The data format to be transferred may be only coordinate data or converted to a laser scanning program incorporating coordinate data. Specifically, the angle with respect to the reference optical axis of the remote head and the relative position coordinate between the remote head and the workpiece are calculated from the three-dimensional position coordinate data of the remote head and the workpiece (X, Y, Z, θx, data such as θy and θz), and a program that corrects the calculated misalignment amount to the relative positional relationship between the object to be processed and the remote head on the CAD drawing and matches the relative positional relationship in the real space is created.

  FIG. 19 shows a conceptual diagram of the package recording system. As shown in FIG. 19, a cardboard box package (4) (3a to 3d) forming a rectangular vertex has a rectangular frame-like structure with a measuring jig having a rectangular frame-like structure. The image of the object to be measured (4), depth, height (dimensions indicated by arrows) is acquired by one camera as shown in FIG. 19, and the same processing as in the first embodiment is performed. Thus, the package shape dimension can be calculated.

FIG. 20 shows a conceptual diagram of the game content recording system. In FIG. 20, for example, in a soccer stadium, a player kicks a soccer ball 41 and a shot distance (the length of an arrow) 43 when the player enters the goal 42 can be acquired from one camera image.
In this case, the measuring jig may be a frame of the goal 43 or a line frame (45, 46) such as a goal line 44 or a penalty area on the competition rules in the court.

(Other examples)
(1) In Example 2 described above, a total reflection mirror was used for the light guide control of the laser beam (FIG. 18 (2)). Instead of the total reflection mirror, a conventionally used transmission lens may be used. When a transmissive lens is used, the temperature of the lens rises due to absorption when the laser passes through the lens, and a refractive index distribution is generated due to thermal deformation and temperature distribution of the lens. There is a problem that the phenomenon occurs.
However, if the position shift amount of the laser focused spot can be ignored in response to the demand for processing accuracy, the transmission lens can be used as it is (FIG. 18 (1)).
(2) Further, in the above-described second embodiment, a filter may be further provided for the lens so as to exclude light emission that causes disturbance as a photographed image of the camera under the surrounding environment to be processed. At the time of laser processing, there are shooting obstacles due to ambient light and smoke, but these effects are eliminated by using a special filter.
(3) When the transmission type lens is adopted in the above (2), the focus shift amount with respect to the laser output of the transmission type optical system and the temperature distribution of the lens unit are measured, and the temperature rise due to internal heat generation and the focus shift amount are measured. Correlation data may be acquired and the thermal lens performance may be databased in advance.

  Today's humanoid robots use a stereo camera. On the other hand, the present invention can recognize a three-dimensional position by one camera. As an application of the above-described third embodiment, for example, in a room, by previously inputting a shape such as a picture hung on a wall or furniture that does not move, the position of the robot itself that mounts the camera, This makes it possible to recognize other shapes.

  The image processing apparatus for photo measurement of the present invention is useful for an on-site recording system, a package recording system, a laser processing system, a competition content recording system, and the like.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus for photo measurement 2 Digital camera 3 Jig for measurement 4 Object to be measured 5 Camera field of view 6 Information operation terminal 7 Connection cable 8 Signboard for work site 9 Machine robot

Claims (14)

One camera means, a measuring jig,
In an image processing apparatus for photo measurement comprising an information operation terminal comprising display means, input means, operation means, storage means, and data transfer means,
The measurement jig has a rectangular flat plate or rectangular frame structure having four photo measurement reference points forming a rectangular vertex, and is incorporated into the measurement target, adjacent to the measurement target, or at a known distance. Arranged parallel to each other,
The information computing terminal is
The storage means stores an internal orientation element of the camera means calculated in advance;
The data transfer means captures one image including the object to be measured and the measurement jig photographed using the camera means in the camera visual field,
The computing means calculates the image of the camera means and the related information of the real coordinates and the external orientation element from the captured one image,
On the plane formed by the measurement jig, the parallel plane, and the plane having a known angle of intersection with at least two selected points on one captured image by the display means and the input means. Measure the two-dimensional coordinate position and distance of
An image processing apparatus for photo measurement characterized by the above. One camera means, a measuring jig,
In an image processing apparatus for photo measurement comprising an information operation terminal comprising display means, input means, operation means, storage means, and data transfer means,
The measuring jig is a rectangular flat plate or a rectangular frame structure having four photometric reference points forming a rectangular vertex,
The information computing terminal is
The storage means stores an internal orientation element of the camera means calculated in advance;
The data transfer means captures two or more images including the measurement target and the measurement jig photographed from different angles using the camera means in the camera field of view,
The computing means calculates an external orientation element of the camera means from each captured image,
Photo measurement characterized in that at least two or more common points on the measurement target of each captured image are selected by the display means and the input means, and the three-dimensional coordinate position and distance of the selected points are measured. Image processing device. A system for recording a reinforcing bar arrangement situation at a construction site using the image processing device for photo measurement according to claim 1 or 2,
The information calculation terminal further includes contour extraction means and report means,
By the data transfer means, obtain an image of the reinforcing bar arrangement of the construction site from the camera means,
Select a reinforcing bar on the displayed image by the input means,
The contour extracting means extracts the rebar contour,
By the calculation means, the number of reinforcing bars, the thickness of the reinforcing bars, the width of the reinforcing bars are calculated,
An on-site recording system characterized in that the report means records and reports the number of reinforcing bars, the thickness of reinforcing bars, the reinforcing bar width, the captured image, the shooting date and time, and the specification information of the construction site. A system for inspecting the deterioration of concrete in a building using the image processing device for photo measurement according to claim 1 or 2,
The information calculation terminal further includes contour extraction means and report means,
By the data transfer means, obtain an image of the concrete of the building from the camera means,
By the input means, select a crack on the displayed image,
Extracting cracks by the contour extracting means,
By the calculation means, the number of cracks, crack length, crack thickness is calculated,
An on-site recording system characterized in that the reporting means records and reports the number of cracks, crack length, crack thickness, photographed image, photographing date and time, and specification information of the concrete of the building. A system for measuring a package shape using the image processing device for photo measurement according to claim 1 or 2,
The information calculation terminal further includes contour extraction means and report means,
The data transfer means acquires an image of a luggage shape from the camera means,
Select a package on the displayed image by the input means,
The outline extracting means extracts the outline of the luggage,
The calculation means calculates the package shape dimensions,
A package recording system, wherein the report means records and reports a package shape dimension, a captured image, a shooting date and time, and package specification information. A laser processing system comprising the image processing device for photo measurement according to claim 1, a processing head for performing laser irradiation, and a work for mounting a processing target,
By the data transfer means, an image including a processing head, a work, and a processing target placed on the work in the camera field of view is obtained from the camera means,
By the input means, select a processing point to be processed on the displayed image,
The calculation means calculates difference data of the three-dimensional position coordinates of the machining head and the machining point,
A laser processing system, wherein the data transfer means transfers difference data or calculation data to a control unit of a processing head. At least two camera means;
An information computing terminal comprising at least display means, input means, computing means, storage means, and data transfer means;
A processing head for performing laser irradiation;
A workpiece on which a workpiece is placed;
A laser processing system comprising:
The workpiece is provided with four photometry reference points forming a rectangular vertex,
Using the camera means, an image including a processing head, a workpiece, and a processing target placed on the workpiece in a camera field of view is taken,
In the information calculation terminal,
The data transfer means captures the captured image,
The display means displays one image selected from the captured images,
By the input means, select a processing point to be processed on the displayed image,
The calculation means calculates difference data of the three-dimensional coordinate positions of the machining head and the machining point,
A laser processing system, wherein the data transfer means transfers difference data or calculation data to a control unit of a processing head.   The processing head scans the laser light, a fiber portion that becomes a laser irradiation source, a transmission optical system for collimating the laser light emitted from the fiber, an optical system that includes only a galvano mirror that guides the laser light, and The laser processing system according to claim 6, comprising a galvano scanner and a lens or a fθ lens for condensing the scanned beam.   The processing head scans the laser light, a fiber portion serving as a laser irradiation source, a reflection optical system for collimating the laser light emitted from the fiber, an optical system including only a galvanometer mirror for guiding the laser light, and The laser processing system according to claim 6, comprising a galvano scanner and a lens or a fθ lens for condensing the scanned beam.   The processing head is composed of a fiber part serving as a laser irradiation source, a reflection optical system for collimating the laser light emitted from the fiber, a reflection optical system for condensing, and a galvano mirror for guiding the laser light. The laser processing system according to claim 6, comprising an optical system and a galvano scanner that scans the laser beam.   The laser processing system according to claim 6, wherein the processing head is applied to arc welding, plasma cutting, resistance welding, electron beam welding, or friction diffusion bonding.   8. The laser processing system according to claim 6, further comprising filter means for excluding light emission that causes disturbance as a photographed image of the camera in a surrounding environment to be processed.   In the surrounding environment of the processing target, the image processing apparatus further includes an irradiation unit that irradiates the processing target and the processing head with a light source having a wavelength band different from the light emission that causes disturbance as a captured image of the camera, and measures the reflected irradiation wavelength band image. The laser processing system according to claim 6 or 7, wherein a three-dimensional coordinate position of a desired object can be measured even when a disturbance is generated. The laser processing according to any one of claims 6 to 13, further comprising feedback control means for sequentially calculating a deformation or deviation amount of a processing target that occurs during processing and correcting the position of the processing head. system.