JP2015134410A - Printer and printing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a printer and a printing method, printable in a desired position of a printing surface in a printing matter, without using a tool, without increasing a burden to an operator.SOLUTION: The printer executes predetermined printing on the basis of printing data, and comprises projection means for projecting a predetermined pattern to a table for placing a plurality of three-dimensional-shaped printing matter, photographing means for photographing the predetermined pattern projected on the table, three-dimensional information acquiring means for acquiring three-dimensional information of the plurality of printing matter placed on the table, attitude recognition means for recognizing an arrangement position and an attitude of the printing matter from the three-dimensional information acquired by the three-dimensional information acquiring means, arrangement means for arranging an inputted printing image on the printing matter on the basis of the arrangement position and the attitude of the printing matter recognized by the attitude recognition means and printing data making means for making printing data on the basis of the printing image arranged by the arrangement means.

Description

  The present invention relates to a printing apparatus and a printing method, and more particularly to a printing apparatus and a printing method for performing desired printing on a printing surface of a three-dimensional printed material.

  Conventionally, a so-called flat bed type printing apparatus in which the entire operation is controlled by a microcomputer and the print head is displaced in two orthogonal directions within the same plane with respect to the printing material placed on the table. It has been known.

  Further, such a flat head type printing apparatus is usually used when printing on a printing surface of a three-dimensional printed material.

When printing on a three-dimensional printed material using this flat bed type printing apparatus, after placing the three-dimensional printed material at a predetermined position on the table, printing is performed on the printing surface of the printed material based on the print data. Has been made to do.

  Here, in such a printing apparatus, in order to perform printing at a predetermined position on the printing surface of the printing material, it is necessary to accurately place the printing material at a predetermined position and posture. For this reason, it has been necessary to accurately determine the position and orientation on which the printing material is placed by measuring the size of the printing material in advance.

It has been pointed out as a problem that, due to such work, the number of work steps increases in printing on the printing material by the printing apparatus, and the burden on the worker increases.

  As a technique for solving such problems, for example, a technique disclosed in Patent Document 1 has been proposed.

  That is, in the technique disclosed in Patent Document 1, a jig that can be fixed to a table and can store a plurality of three-dimensional objects to be printed is manufactured. A plurality of substrates are accommodated in a jig. Note that the position where the printed material is accommodated in the jig is a predetermined position, and this position is input in advance to the microcomputer that controls the printing apparatus.

Thereby, in the technique disclosed in Patent Document 1, a three-dimensional printed material is positioned by a jig, and printing can be performed at a predetermined position on the printing surface of the printed material. Has been.

  However, in the technique disclosed in Patent Document 1, a jig must be manufactured according to the shape and size of the substrate to be printed, which requires labor for jig preparation and increases the cost for small-scale production. Inviting was pointed out as a problem.

JP 2007-136664 A

  The present invention has been made in view of the various problems of the prior art as described above. The object of the present invention is to use a jig without increasing the burden on the operator. It is an object of the present invention to provide a printing apparatus and a printing method capable of printing at a desired position on a printing surface of a printing material.

  In order to achieve the above object, a printing apparatus according to the present invention projects a predetermined pattern onto a table on which a plurality of three-dimensional printed materials are placed in a printing apparatus that performs predetermined printing based on print data. Projecting means, photographing means for photographing a predetermined pattern projected on the table, three-dimensional information obtaining means for obtaining three-dimensional information of the plurality of printed materials placed on the table, and the three-dimensional information From the three-dimensional information acquired by the information acquisition means, the posture recognition means for recognizing the arrangement position and posture of the printed material, and the input print image to the arrangement position and posture of the printed material recognized by the posture recognition means. An arrangement unit arranged on the substrate based on the print data, and a print data creation unit for producing print data based on the print image arranged by the arrangement unit; It is obtained by way has.

  In the printing apparatus according to the present invention, in the above-described printing apparatus, further, the highest height information in the printed material is acquired from the three-dimensional information acquired by the three-dimensional information acquisition unit, and based on the acquired height information. The printing head has a printing head for printing on the printing surface of the substrate, and an adjusting means for adjusting the distance between the table.

  Further, in the printing apparatus according to the present invention, in the above-described printing apparatus, the three-dimensional information acquisition unit acquires the three-dimensional information of the plurality of prints placed on the table by a phase shift space encoding method. It is what I did.

  In the printing apparatus according to the present invention, in the above-described printing apparatus, the posture recognition unit may divide the point cloud data, which is three-dimensional information acquired from the three-dimensional information acquisition unit, for each of the printed materials, Setting means for setting first point cloud data serving as a reference from the point cloud data and second point cloud data representing the plurality of printed materials, and a first distance image from the first point cloud data And a distance image creating means for creating a second distance image from the second point cloud data, and a first transformation matrix for matching the first distance image with the second distance image And a second conversion matrix that is a conversion matrix that expands the first conversion matrix so that three-dimensional coordinate conversion is possible and converts the three-dimensional coordinates of the first point cloud data by an ICP algorithm. The second calculation to calculate And third calculation means for calculating a third conversion matrix for arranging a two-dimensional image on the first distance image on the second distance image using the means and the second conversion matrix The arrangement means arranges the print image inputted on the first distance image on the second print image using the third transformation matrix. It is.

  In the printing apparatus according to the present invention, in the above-described printing apparatus, the print head is an ink head that ejects ink by an ink jet method.

  Further, the printing method according to the present invention includes a projecting unit that projects a predetermined pattern onto a table on which a plurality of three-dimensional objects to be printed is placed, and an imaging unit that photographs the predetermined pattern projected onto the table. A printing method in a printing apparatus that performs predetermined printing based on print data, the first step of acquiring three-dimensional information of the plurality of printed materials placed on the table, and A second step of recognizing the arrangement position and orientation of the printed material from the three-dimensional information acquired in the first step; and the arrangement position of the printed material recognized in the second step and the input print image The printing apparatus executes a third step of arranging on the substrate based on the posture and a fourth step of creating print data based on the print image arranged in the third step. like Those were.

  In the printing method according to the present invention, in the above-described printing method, further, the highest height information in the printed material is obtained from the three-dimensional information obtained in the first step, and based on the obtained height information. The printing apparatus executes the fifth step of adjusting the distance between the print head for printing on the printing surface of the substrate and the table.

  Further, in the printing method according to the present invention, in the above-described printing method, in the first step, the three-dimensional information of the printed material placed on the table is acquired by a phase shift space coding method. Is.

  Further, in the printing method according to the present invention, in the above-described printing method, in the second step, the step of dividing the point cloud data, which is the three-dimensional information acquired in the first step, for each substrate to be printed; A step of setting first point cloud data serving as a reference and second point cloud data representing a plurality of printing materials from the group data, and creating a first distance image from the first point cloud data. A step of creating a second distance image from the second point cloud data, a step of calculating a first transformation matrix for matching the first distance image with the second distance image, and the second Transforming one transformation matrix so that three-dimensional coordinates can be transformed, and calculating a second transformation matrix as a transformation matrix for transforming the three-dimensional coordinates of the first point cloud data by an ICP algorithm; Using the transformation matrix, the first distance image Calculating a third transformation matrix for arranging the two-dimensional image on the second distance image, and in the third step, the above-mentioned input on the first distance image The print image is arranged on the second print image using the third conversion matrix.

  In the printing method according to the present invention, the print head is an ink head that ejects ink by an ink jet method.

  Since the present invention is configured as described above, it is possible to perform printing at a desired position on the printing surface of the substrate without increasing the burden on the operator and without using a jig. It has an excellent effect of being.

FIG. 1 is a schematic configuration explanatory view showing a printing apparatus according to the present invention. FIG. 2 is a block diagram illustrating a functional configuration of the microcomputer. FIG. 3A is an explanatory diagram showing point cloud data of a plurality of printed materials, and FIG. 3B is an explanatory diagram showing a state in which a cluster is created by dividing the point cloud data. FIG. 4A is an explanatory diagram showing a state in which source point cloud data is created and target point cloud data is set, and FIG. 4B is a diagram in which a distance image is created from the point cloud data. It is explanatory drawing explaining these. FIG. 5A is an explanatory diagram for explaining that the source distance image is superimposed on each target distance image, and FIG. 5B is a diagram in which the two-dimensional component of the source point cloud data is brought closer to the target point cloud data. It is explanatory drawing which shows the state. FIG. 6 illustrates an image in which the source point cloud data is made close to the target point cloud data by the transformation matrix A ₄₄ and an image in which the alignment in the three dimensions is optimized by the transformation matrix A _ICP. FIG. FIG. 7 is an explanatory diagram illustrating conversion from a source distance image to a target distance image. FIG. 8A is an explanatory diagram showing a state in which the print image is arranged on the source distance image, and FIG. 8B shows a state in which the print image is placed on the target distance image. It is explanatory drawing. FIG. 9A is an explanatory diagram showing a checker pattern printed on a sheet mounted on a table, and FIG. 9B shows a spatial code image by projecting a gray code pattern onto the checker pattern. It is explanatory drawing explaining acquiring. FIG. 10 is a flowchart showing a processing routine of print data creation processing in the printing apparatus according to the present invention. FIG. 11 is a flowchart showing a processing routine of the three-dimensional information acquisition process. FIG. 12 is a flowchart illustrating a processing routine for posture recognition processing. FIG. 13 is an explanatory diagram of a schematic configuration showing a modification of the printing apparatus according to the present invention.

Hereinafter, an example of an embodiment of a printing apparatus and a printing method according to the present invention will be described in detail with reference to the accompanying drawings.

  First, FIG. 1 shows a schematic configuration explanatory diagram of a printing apparatus according to the present invention.

  The printing apparatus 10 shown in FIG. 1 is a so-called flat bed type ink jet printer, and includes a fixed base member 12 and a table on which a three-dimensional printed material 200 is placed on an upper surface 14a on the base member 12. 14 and a rod-like member 16 extending in the X-axis direction, the rod-like member 16 being disposed slidably in the Y-axis direction on the upper side of the table, and the rod-like member 16 in the X-axis direction The print head 20 is slidably provided on the table 14 and prints on the printing surface of the substrate 200, the standing member 22 is erected on the rear side of the base member 12, and the A projector 24 that is disposed on the installation member 22 and projects a predetermined pattern (such as a gray code pattern or a binarized projection pattern) onto the entire upper surface 14 a of the table 14. Is disposed in the standing portion member 22, it is configured to include a camera 26 across the upper surface 14a of the photographable table 14.

Note that the overall operation of the printing apparatus 10 is controlled by the microcomputer 300.

  More specifically, the table 14 is provided on the base member 12 and can be moved in a predetermined range in the Z-axis direction by a moving mechanism (not shown).

  Thereby, the to-be-printed material 200 mounted on the upper surface 14a of the table 14 can be moved in the Z-axis direction.

  Note that the range in which the table 14 moves up and down matches, for example, the range of the thickness of the substrate 200 that can be printed by the printing apparatus 10.

  The table 14 has a flat upper surface 14a, and the substrate 200 is placed on the upper surface 14a.

  The shape of the substrate 200 can be placed on the table 14 and can be placed at a predetermined interval from the print head 20 when placed on the table 14. Any shape is acceptable.

  Furthermore, the printing surface of the substrate 200 has various shapes such as a flat shape, a curved shape that is convex upward, a curved shape that is convex downward, a concave-convex shape with corners, and a concave-convex shape without corners. It may be a shape. The height difference on the printing surface is set to be within the height difference at which the ink can be normally applied by the print head 20.

Further, a moving mechanism (not shown) for moving the table 14 in the Z-axis direction is configured by using a conventionally known technique by a combination of a gear and a motor, for example, and its operation is controlled by the microcomputer 300. The

  Guide grooves 28a and 28b extend in the Y-axis direction in the vicinity of the left side end and the right side end of the base member 12, and the moving member 18 extends along the guide grooves 28a and 28b. It moves in the Y-axis direction by a moving mechanism (not shown).

  The moving mechanism (not shown) for moving the moving member 18 in the Y-axis direction is configured using, for example, a conventionally known technique using a combination of a gear and a motor, and its operation is controlled by the microcomputer 300. Is done.

  The moving member 18 is provided with a print head 20 slidably on a guide rail (not shown) provided on the front surface of the rod-like member 16.

  The print head 20 is an ink head that ejects ink by an inkjet method.

  The print head 20 is provided with a belt (not shown) that is movable in the X-axis direction. When the belt is wound by a moving mechanism (not shown), the belt moves, and the belt moves. Along with this, the print head moves in the X-axis direction from the left side to the right side and from the right side to the left side.

  In addition, a moving mechanism (not shown) that winds up a belt (not shown) and moves the print head 20 in the X-axis direction is configured using, for example, a conventionally known technique using a combination of a gear and a motor. The operation of the microcomputer 300 is controlled.

In the present specification, the “inkjet method” refers to various conventionally known methods including various continuous methods such as a binary deflection method and a continuous deflection method, and various on-demand methods such as a thermal method and a piezoelectric element method. This means a printing method based on an inkjet technique based on the above method.

  The projector 24 is fixedly disposed on the standing member 22 and is controlled by the microcomputer 300.

Then, the projector 24 controls the microcomputer 300 to control the upper surface 14a of the table 14 in a vertical direction and a horizontal direction, respectively in a predetermined gray code pattern, and in a predetermined direction having a certain width and orthogonal to the width direction. A slit-like light transmission region 100a and a light non-transmission region 100b corresponding to a slit frame are projected in a binary projection pattern formed so as to alternate.

The camera 26 is fixedly disposed on the standing member 22 so as to be able to photograph the entire upper surface 14 a of the table 14 from a direction different from the projection direction of the projector 24, and is controlled by the microcomputer 300. Has been.

  The microcomputer 300 controls the overall operation of the printing apparatus 10 and recognizes the postures and arrangement positions of a plurality of printing objects placed on the table 14, and inputs an operator based on this recognition. Print data for printing the printed image at a predetermined position on each substrate is created.

  Here, the functional configuration of the microcomputer 300 will be described with reference to FIG.

That is, the microcomputer 300 includes a control unit 302 that controls the overall operation of the printing apparatus 10, a recognition unit 304 that recognizes the arrangement positions and orientations of the plurality of printed materials 200 placed on the table 14, and a plurality of A print data creation unit 306 that creates print data for printing on the printing surface of the substrate 200, a storage unit 308 that stores the created print data and other various types of information, and a table 14. The display unit 310 is configured to display images of a plurality of prints 200 and various images and information.

  For example, the control unit 302 drives a moving mechanism (not shown) to move the print head 20 in the X-axis direction, move the moving member 18 in the Y-axis direction, and move the table 14 in the Z-axis direction. Control the operation of the components.

The movement of the table 14 in the Z-axis direction is controlled by the Z-axis direction movement control unit 312, and the Z-axis direction movement control unit 312 is the highest in the printed material 200 from the three-dimensional information acquired by the recognition unit 304. Height information (Z coordinate value) is acquired, and the table 14 is moved up and down based on this height information.

The recognizing unit 304 also acquires a three-dimensional information acquisition unit 314 that acquires three-dimensional information of the substrate 200 placed on the table 14, and a point group that generates point cloud data of the substrate 200 from the acquired three-dimensional information. A data creation unit 316, a cluster creation unit 318 that collects the point cloud data and creates a cluster representing the printing material 200, and sets the created cluster as target point cloud data and also sets a source point cloud from one of the target point cloud data A source point cloud data creation unit 320 for creating data, a distance image creation unit 322 for creating a distance image from each point cloud data, and a first for rotating the source distance image at an angle closest to the target distance image. From the first transformation matrix calculation unit 324 that calculates the transformation matrix and the calculated first transformation matrix, the source point cloud data and the target point are more accurately determined. The second transformation matrix for approximating the data and a second transformation matrix calculation unit 326 for calculating a are configured.

  More specifically, the three-dimensional information acquisition unit 314 projects the gray code pattern projected from the projector 24 and the gray code pattern projected by the camera 26 onto the upper surface 14a of the table 14 on which the plurality of substrates 200 are placed. The space code image is acquired by photographing, and the three-dimensional information (point cloud) of the substrate 200 is acquired from the acquired space code image.

  That is, the three-dimensional information acquisition unit 314 acquires the three-dimensional information (point group) of the substrate 200 placed on the table 14 by the spatial encoding method.

  The three-dimensional information acquisition unit 314 acquires the three-dimensional information (point group) of the printing material 200 placed on the table 14 not by the spatial encoding method but by the phase shift spatial encoding method. May be.

  That is, when acquiring the three-dimensional information (point cloud) of the substrate 200 by the phase shift space encoding method, the three-dimensional information acquisition unit 314 has the upper surface 14a of the table 14 on which the plurality of substrates 200 are placed. In addition, a predetermined binarized projection pattern is projected from the projector 24 while being shifted by a predetermined amount of movement, and a phase shift code value is obtained by synthesizing the images projected for each shift by the camera 26. A plurality of binarized projection patterns including the binarized projection pattern are projected, and images obtained by photographing the respective binarized projection patterns by the camera 26 are combined to obtain a spatial code value, and a phase shift code value and a spatial code are obtained. A new spatial code value is acquired using the value, and the three-dimensional information (point group) of the substrate 200 is acquired.

  Thus, by acquiring three-dimensional information (point group) by the phase shift spatial coding method, three-dimensional information (points) with higher resolution than the three-dimensional information (point group) acquired by the normal spatial coding method. Group) can be acquired, and a more accurate posture and arrangement position of the substrate can be recognized.

  In addition, about the technique which acquires the three-dimensional information by such a spatial encoding method, since a conventionally well-known technique can be used, the detailed description shall be abbreviate | omitted.

In addition, as for the technique for acquiring the three-dimensional information by the phase shift space coding method, for example, the technique disclosed in Japanese Patent No. 4944435 and Japanese Patent No. 4874657 can be used, and thus detailed description thereof is omitted. I decided to.

  The point cloud data creation unit 316 converts the three-dimensional coordinates shown in the camera coordinate system acquired by the three-dimensional information acquisition unit 314 into values in the print coordinate system, and near the upper surface 14a (Z = 0) of the table 14. The point cloud is deleted, and point cloud data for only the substrate 200 is created (see FIG. 3A).

Specifically, using the 4 × 4 transformation matrix H _R2P (described later) calculated by calibration between the camera 26 and the upper surface 14 a of the table 14, the following formula is used.

  The cluster creation unit 318 divides the point cloud data representing the plurality of workpieces 200 placed on the table 14 for each substrate 200 by the Euclidian Cluster Extraction algorithm, and creates a cluster representing the workpiece 200. It is created (see FIG. 3B).

The Euclidean Cluster Extraction Algorithm is a conventionally known technique (RB Rusu and S. Coins. 3D here: Point Cloud Library RA (PCL), In IEEE International Conference IC). , China, May 9-13 2011.), and detailed description thereof will be omitted.

  The source point cloud data creation unit 320 creates source point cloud data by copying a predetermined cluster among a plurality of clusters representing the plurality of workpieces 200. At this time, the plurality of clusters are set as target point cloud data.

  Specifically, in the point cloud data shown in FIG. 3B, for example, source point cloud data is created by copying the upper left point cloud data, and the point cloud data shown in FIG. Let it be point cloud data (see FIG. 4A). At this time, the coordinate values of the source point cloud data are converted into relative coordinates from the starting point of the display area.

In this way, by setting all point cloud data including the point cloud data of the copy source as the target point cloud data, each cluster is a target for placing a print image input by the operator.

  The distance image creation unit 322 creates a distance image from the source point group data and the target point group data created by the source point group data creation unit 320 (see FIG. 4B).

  Specifically, when creating a source distance image from the source point cloud data, the X coordinate and the Y coordinate in the source point cloud coordinates, which are the three-dimensional coordinates of the source point cloud data, are set as the X of the two-dimensional coordinates of the source distance image. In addition to the coordinates and the Y coordinate, the Z coordinate in the source point group coordinates is represented as a gray value.

  At this time, the (x, y) coordinates are converted into a value in which the distance between the average points of the point cloud data is 1 pixel.

That is, the three-dimensional coordinates (source point group coordinates) of the point cloud data are converted into the two-dimensional coordinates (source distance image coordinates) of the distance image by the following equation.
(X _s , Y _s ): Source point cloud coordinates (u _s , v _s ): Source distance image coordinates s: Conversion scale from 3D point cloud image coordinate system (mm) to distance image coordinate system

The scale factor s for converting the coordinate value of the point cloud data into the coordinate value of the distance image is expressed by the following equation when the resolution of the printer is reso (dpi) and the coordinate value unit of the point cloud data is (mm). expressed.
s = reso / 25.4

  In the gray value range, the minimum to maximum Z value of the point cloud data in all clusters is set to “0 to 255”.

That is, the gray value in the corresponding XY coordinate value from the Z coordinate value Z _s of the source point cloud data is determined with “0” as the minimum value and “255” as the maximum value.

  When creating a target distance image from target point cloud data, the X coordinate and Y coordinate in the target point cloud coordinates, which are the three-dimensional coordinates of the target point cloud data, are used as the X coordinate, Y of the two-dimensional coordinates of the target distance image. In addition to the coordinates, the Z coordinate in the target point group coordinates is expressed as a gray value.

  At this time, the (x, y) coordinates are converted into a value in which the distance between the average points of the point cloud data is 1 pixel.

That is, the three-dimensional coordinates (target point group coordinates) of the point group data are converted into the two-dimensional coordinates (target distance image coordinates) of the distance image by the following equation.
(X _t , Y _t ): Target point group coordinates (u _t , v _t ): Target distance image coordinates s: Conversion scale from the three-dimensional point group image coordinate system (mm) to the distance image coordinate system

As described above, the scale factor s for converting the coordinate value of the point cloud data into the coordinate value of the distance image is obtained when the resolution of the printer is reso (dpi) and the coordinate value unit of the point cloud data is (mm). Is expressed by the following equation.
s = reso / 25.4

  In the gray value range, the minimum to maximum Z value of the point cloud data in all clusters is set to “0 to 255”.

That is, "0" gray value, a minimum value in the XY coordinate value corresponding to Z-coordinate value Z _t of the target point group data, it determines the maximum value as "255".

  The first transformation matrix calculation unit 324 moves the source distance image created from the source point cloud data so that the centroid of the source distance image overlaps the centroid of the target distance image created from each target point cloud data (FIG. (See 5 (a)).

  Then, the source distance image is rotated by 1 degree, and the normalized cross-correlation in each target distance image is acquired. The angle at which this normalized cross-correlation is highest is taken as the rotation angle of the source distance image that is closest to the target distance image.

  Thereafter, in each target distance image, a first transformation matrix for rotating the source distance image at this rotation angle is calculated.

Specifically, the affine transformation matrix T _s that moves the center of gravity (ugs, vgs) of the source distance image to the origin is
It is represented by

Further, the affine transformation matrix T _t that moves from the origin to the center of gravity (ugt _n , vgt _n ) of the target distance image n is
It is represented by

Furthermore, the affine transformation matrix R (θ) that rotates by the angle θ is
It is represented by

A transformation matrix A (θ) obtained by multiplying the above affine transformation matrices T _s , T _t , and R (θ) is created (see the following formula), and the angle θ is rotated by 1 degree to transform matrix A (Θ) is calculated.

Thereafter, the coordinate value of the source distance image is converted by the calculated A (θ) according to the following equation, the source distance image is converted, and a source distance image rotated by an angle θ is acquired.

The degree of coincidence between the source distance image after the coordinate conversion (that is, the source distance image rotated by the angle θ) and the target distance image is evaluated by the normalized cross-correlation coefficient _RNCC . Note that the normalized cross-correlation coefficient _RNCC is expressed by the following equation.
S (i, j): Pixel value of source distance image T (i, j): Pixel value of target distance image M: Horizontal pixel number of distance image N: Vertical pixel number of distance image

Then, A (θ) at the angle θ at which the normalized cross-correlation coefficient R _NCC is the largest among the angles θ = 0 to 359 degrees is acquired as the first transformation matrix A ₃₃ . Note that the first transformation matrix A ₃₃ is
It is represented by

The second conversion matrix calculation unit 326 calculates a second conversion matrix for more accurately approximating the source point group data and the target point group data from the first conversion matrix A ₃₃ . Note that this second transformation matrix is calculated for each target.

Specifically, the first conversion matrix A ₃₃ calculated by the first conversion matrix calculation unit 324 is expanded to a 4 × 4 matrix for three-dimensional coordinate conversion, and the conversion matrix A ₄₄ is acquired. This transformation matrix A ₄₄ is
It is represented by

At this time, the translation components a ₁₃ and a ₂₃ are converted by the conversion scale s (that is, the scale factor s) from the three-dimensional coordinate system to the two-dimensional coordinate system.

Then, as shown in the following equation, only the two-dimensional component of the source point cloud data is transformed using the transformation matrix A ₄₄ and brought close to the target point cloud data (see FIG. 5B).

Thereafter, a transformation matrix A _ICP for accurately approximating the source point cloud data to the target point cloud data is calculated by an ICP (Iterative Closest Point) algorithm.

  In addition, about this ICP algorithm, it is well-known technique (Paul J. Besl and Neil D. McKay. A method for registration of 3-d shapes, IEEE Transactions on Pattern Analysis. , Pp. 239-256, February 1992.), and detailed description thereof will be omitted.

Here, since the transformation matrix A ₄₄ is an optimal solution while being discretely rotated by 1 degree, the point cloud data transformed by the transformation matrix does not match exactly.

  However, the ICP algorithm can optimize the deviation of the posture of the entire three-dimensional component due to the actual arrangement method and variation in shape. Thereby, more accurate alignment suitable for the actual shape can be performed (see FIG. 6).

When the optimization is performed by the ICP algorithm, the conversion result of the two-dimensional component of the source point cloud data by the conversion matrix A ₄₄ is set as an initial value.

Then, the rough transformation matrix A ₄₄ is multiplied by the transformation matrix A _ICP calculated by the ICP algorithm, so that the second transformation which is a three-dimensional coordinate transformation matrix for accurately matching the source point cloud data to the target point cloud data. The matrix A _3D is calculated. Therefore, the second transformation matrix A _3D is expressed by the following equation.

Furthermore, the print data creation unit 306 includes a third transformation matrix calculation unit 328 that calculates a third transformation matrix for arranging the print image input on the source distance image on the target distance image, and a source distance image. A print image arrangement unit 330 that arranges the print image input above on the target distance image by the third transformation matrix, and a print data generation unit that generates print data based on the print image arranged on the target distance image 332.

  More specifically, the third conversion matrix calculation unit 328 uses the conversion matrix calculated by the second conversion matrix calculation unit 326 to output the print image input on the source distance image by the operator. A third transformation matrix for placement on the target distance image is calculated according to the placement position and orientation of the substrate.

Specifically, the operator calculates a third transformation matrix that is a transformation matrix for arranging the print image arranged on the source distance image on the target distance image.

  Here, when converting a two-dimensional image from a source distance image to a target distance image, a process for converting the source distance image to the source point cloud data is performed, and then the source point cloud data is converted to the target point cloud data. Next, a process of converting target point cloud data into a target distance image is performed (see FIG. 7).

That is, in the process of converting the source distance data to the source point cloud data, each pixel of the two-dimensional image is arranged in the three-dimensional space. Z coordinates = 0 are assigned to the X and Y coordinates of each pixel to obtain the three-dimensional coordinates of each pixel. At this time, the three-dimensional coordinates for each pixel are acquired by the following equation (1).

In the process of converting from the source point cloud data to the target point cloud data, the three-dimensional coordinates in the source point cloud data are converted into the three-dimensional coordinates in the target point cloud data by the following equation (2).

Originally, the conversion should be on the same table plane (two-dimensional conversion), but the conversion matrix A _ICP is subtly Z-axis due to a slight shape error or placement position error of the actual printed material. It also includes moving and rotating in the direction (three-dimensional coordinate transformation).

Further, in the process of converting the target point cloud data into the target distance image, a two-dimensional image is created from the three-dimensional coordinates in the target point cloud data by the following equation (3).

As described above, when the source point cloud data is converted to the target point cloud data, the Z coordinate after the conversion is not “0” because the rotation in the Z-axis direction is performed. However, by forcing this Z coordinate to be “0”, the shape is projected onto the Z = 0 plane.

Here, the above-described three-stage conversion (that is, the expressions (1) to (3)) are combined into one.
It becomes.

Combining three 4x4 transformation matrices into one 4x4 matrix:
It becomes.

Since this represents an affine transformation matrix of two-dimensional coordinates,
This can be expressed as a 2 × 3 matrix, which is obtained as a third transformation matrix.

  The print image arrangement unit 330 converts the print image arranged and edited on the source distance image displayed on the display unit 310 by the operator using the third conversion matrix calculated by the third conversion matrix calculation unit 328. Thus, a print image is arranged on each target distance image in accordance with the arrangement position and orientation (see FIG. 8).

That is, the print image placement unit 330 uses the third transformation matrix to place the print image on the target distance image so that the print image placed on the source distance image matches the placement position and orientation. It will be.

The print data generation unit 332 generates print data based on the print image arranged on the target distance image by the print image arrangement unit 330.

In addition, the storage unit 308 stores the print data created by the print data creation unit 306 and stores various information necessary for printing on the substrate 200.

The display unit 310 displays the image acquired by the recognition unit 304 and various images and information, and changes display contents based on input information from an operator via an operator (not shown).

  In the above configuration, a case where desired printing is performed on the printing surface of the three-dimensional printed material 200 by the printing apparatus 10 will be described. First, the printing apparatus 10 is predetermined at the time of factory shipment or when the camera 26 is replaced. At this timing, the camera calibration and the calibration of the camera 26 and the upper surface 14a (printing coordinate system) of the table 14 are performed.

  Here, the camera calibration is performed separately using an LCD (Liquid Crystal Display) in a state independent of the printing apparatus 10.

Then, after performing camera calibration, the camera 26 is installed in the printing apparatus 10 and calibration for obtaining the relationship between the position and orientation of the camera 26 and the upper surface 14a of the table 14 (that is, the camera 26 and the upper surface 14a of the table 14). Calibration).

  Specifically, in camera calibration, a checker pattern is photographed at the full angle of view of the camera 26, and camera parameters are calculated by the Zhang method.

  Here, the checker pattern is not drawn on the upper surface 14 a of the table 14, but a checker pattern displayed separately on the LCD is used.

  As a method for calculating camera parameters by the Zhang method, for example, the technique disclosed in Japanese Patent No. 4917351 can be used, and a detailed description thereof will be omitted.

Thus, camera internal parameters: A _c , camera external parameters: [R _c T _c ], projector internal parameters: A _p , and projector external parameters: [R _p T _p ] are calculated.

In the calibration between the camera 26 and the upper surface 14 a of the table 14, an affine transformation matrix _HR2P for converting the three-dimensional coordinate system of the camera 26 to the printing coordinate system of the printing apparatus 10 is calculated.

  First, a sheet is attached to the upper surface 14a of the table 14, and a checker pattern indicating an actual printing range is printed on the sheet by the printing apparatus 10 (see FIG. 9A).

The checker pattern is, for example, a checkered pattern of gray and white in which the size of one pattern is 20 × 20 mm, and the overall size of this checker pattern is 300 × 280 mm.
Next, the table 14 (checker pattern) is irradiated with a gray code pattern in the u direction (vertical direction) and the v direction (horizontal direction), and spatial code images in the u direction and the v direction are acquired from the captured images (FIG. 9). (See (b)).

Then, checker intersection coordinates are obtained with sub-pixel accuracy on the camera photographed image, and projector image coordinates (space code values in the u direction and v direction) corresponding to these coordinates are obtained.
Checker intersection coordinates: m _c = (u _c , v _c )
Projector image _{coordinates: m p = (u p,} v p)

  From the obtained checker intersection coordinates and projector image coordinates, the three-dimensional coordinates M of the checker intersection are obtained.

That is, the camera image coordinate system and the relationship of the three-dimensional coordinate system, with simultaneous equations between the projector coordinate system and the three-dimensional coordinate system, image coordinates (u _{c, v c)} and 3-dimensional from a space code value u _p Find the coordinates.

When this is expressed as Q · V = F, if Q ⁻¹ exists, three-dimensional coordinates (X, Y, Z) are obtained from V = Q ⁻¹ · F.

An affine transformation matrix _HR2P for transforming the obtained three-dimensional coordinate value of the checker intersection into a known coordinate value on the checker pattern is _obtained by the method of least squares.

That is, the three-dimensional coordinates M _R of the measurement coordinate system of the camera 26, obtains an affine transformation matrix H _R2P serving 4 × 4 conversion matrix for converting the three-dimensional coordinates M _P in the printing coordinate system of the printing apparatus 10.

Specifically, an affine transformation matrix _{H R2P} that minimizes by applying the n sets of _{M R} and _{M P} in the formula is obtained by a nonlinear least-squares method (Levenberg-Marquardt method). That is, “R” and “T” in the affine transformation matrix _HR2P are obtained.

Here, “R” is a 3 × 3 rotation matrix and the number of elements is “9”, which is expressed by a three-dimensional vector r = [r _x , r _y , r _z ] ^T. Yes, the degree of freedom is “3”. That is, there are actually three elements to be optimized, r _x , r _y , and r _z .

During optimization calculation by the non-linear least square method, r _x , r _y , and r _z are converted to “R” by the following Rodrigues formula.

T is a three-dimensional translation vector, and the degree of freedom is “3”.

Thereafter, when printing is performed on the printing surface of the three-dimensional printed material 200 by the calibrated printing apparatus 10, first, the printing surface of the printing material 200 is opposed to the ink ejection surface of the print head 20 by the operator. As described above, the plurality of printed materials 200 are placed on the upper surface 14 a of the table 14. In this state, when the operator gives an instruction to create print data via an operator (not shown) or the like, the print data creation process is started in the microcomputer 300.

Here, the flowchart of FIG. 10 shows the detailed processing contents of the print data creation processing. In this print data creation processing, first, three-dimensional information acquisition processing is performed (step S1002).

  The flowchart of FIG. 11 shows the detailed processing contents of the three-dimensional information acquisition process. In this three-dimensional information acquisition process, first, the three-dimensional information (point cloud) of the printing material is obtained by the phase shift space coding method. ) Is acquired (step S1102).

  That is, in the process of step S1102, the three-dimensional information acquisition unit 314 acquires three-dimensional information (point group) of the plurality of printing objects 200 placed on the table 14 by the phase shift space encoding method.

In step S1102, three-dimensional information (point group) of a plurality of prints 200 placed on the table 14 may be acquired by a spatial encoding method instead of the phase shift spatial encoding method.

  Next, the three-dimensional coordinates obtained from the camera 26 are converted into values in the print coordinate system (step S1104).

That is, in the process of step S1104, the point cloud data creation unit 316 converts the three-dimensional coordinates shown in the camera coordinate system acquired in the process of step S1102 into values in the print coordinate system.

  Thereafter, three-dimensional information of a height other than the upper surface 14a of the table 14, that is, three-dimensional information (point group) only on the printing surface of the substrate 200 is acquired (step S1106), and the process proceeds to step S1004.

That is, in the process of step S1106, the point cloud data creation unit 316 deletes the point cloud near the upper surface 14a (Z = 0) of the table 14 and creates the point cloud data only for the substrate 200 (FIG. 3). (See (a)).

When the three-dimensional information of the printing material 200 is acquired, next, posture recognition processing for recognizing the posture of the printing material 200 is performed (step S1004).

  Here, the flowchart of FIG. 12 shows the detailed processing contents of the posture recognition processing. In this posture recognition processing, first, in the point cloud data as the three-dimensional information acquired in the processing of step S1002, individual point cloud data Since it is not classified to which printing medium 200 the point of the mark belongs, the point cloud data is divided for each printing medium 200 (step S1202).

That is, in the process of step S1202, the cluster creation unit 318 divides each point cloud data that is three-dimensional information representing a plurality of workpieces 200 placed on the table 14 to create a cluster representing the substrate 200. It is created (see FIG. 3B). That is, each cluster is a single substrate 200.

  Next, a source and a target are set (step S1204).

  That is, in the process of step S1204, the source point cloud data creation unit 320 creates a source point cloud data by copying a predetermined cluster of a plurality of clusters, and sets each cluster as target point cloud data. .

Specifically, for example, in the point cloud data shown in FIG. 3B, source point cloud data is created by copying the upper left dot cloud data. At this time, all the point group data including the copy source other than the source point group data are set as target point group data to be placed with the print image (see FIG. 4A).

  When the setting of the source and the target is completed, a distance image that is a two-dimensional image is created from the point cloud data (three-dimensional information) (step S1206).

  That is, in the process of step S1206, the distance image creating unit 322 creates a distance image as a two-dimensional image in which the Z coordinate is represented by a gray value from the source point group data and the target point group data (FIG. 4 ( b)), a source distance image is created from the source point cloud data, and a target distance image is created from the target point cloud data.

Note that the generated source distance image and target distance image may be displayed on the display unit 310 at this time.

  Thereafter, the source distance image and the target distance image are matched (step S1208).

  That is, in the process of step S1208, the first transformation matrix calculation unit 324 moves the soak distance image so that the centroid of the source distance image overlaps the centroid of each target distance image (see FIG. 5A). ), Rotate the source distance image by 1 degree to obtain the normalized cross-correlation in each target distance image. The angle at which this normalized cross-correlation is highest is acquired as the rotation angle of the source distance image that is closest to the target distance image.

Then, the first conversion matrix calculation unit 324 calculates a first conversion matrix A ₃₃ for rotating the source distance image at this rotation angle in each target distance image.

  Next, the three-dimensional coordinates of the source point cloud data are converted (step S1210).

That is, in the process of step S1210, the second conversion matrix calculation unit 326 expands the first conversion matrix A ₃₃ to a 4 × 4 matrix for three-dimensional coordinate conversion, and acquires the conversion matrix A ₄₄ .

Then, only the two-dimensional components of the three-dimensional coordinates in the source point cloud data are transformed by this transformation matrix A ₄₄ and are brought close to the target point cloud data (see FIG. 5B).

  Then, the conversion matrix for converting the three-dimensional coordinates of the source point cloud data is optimized (step S1212).

That is, in the process of step S1212, the second transformation matrix calculation unit 326 calculates the transformation matrix A _ICP by the ICP algorithm, and multiplies the transformation matrix A ₄₄ and the transformation matrix A _ICP to obtain the second transformation matrix A. _{Get 3D} .

  Thereafter, a transformation matrix for transforming the two-dimensional image from the source to the target is calculated using the transformation matrix acquired in the processing in step S1212 (step S1214), and the process proceeds to step S1006.

That is, in the process of step S1214, the third conversion matrix calculation unit 328 uses the conversion matrix A _3D acquired in the process of step S1212, and the print image (two-dimensional) input on the source distance image by the operator. The third transformation matrix for arranging the image) on the target distance image is calculated according to the arrangement position and orientation of each printed matter 200.

  When the third transformation matrix is calculated and the posture recognition process of the substrate 200 is completed, an image for the operator to input a print image is displayed on the display unit 310 (step S1006).

  That is, in the process of step S1006, the source distance image created in the process of step S1206 is displayed on the display unit 310 by the distance image creation unit 322 in a state where the operator can input a print image.

That is, the source distance image is displayed in a state where the operator can arrange and edit the print image.

  The operator arranges a desired print image at a desired position and angle in the source distance image displayed on the display unit 310. Such a print image may be created by an operator using predetermined software, or image data input in advance may be used.

  When the source distance image is displayed on the display unit 310, the operator determines whether or not a print image has been placed on the source distance image (step S1008).

  That is, in the determination processing in step S1008, it is determined whether or not the worker has completed the processing for arranging the print image on the source distance image.

It should be noted that various methods can be applied as a method for determining whether or not a print image has been placed on the source distance image by the operator, for example, inputting that the placement of the print image has been completed. A completion button is provided, and when this completion button is clicked, it is determined that the arrangement of the print image is completed.

  In the determination process of step S1008, if the operator determines that the print image is not arranged on the source distance image, that is, the operator does not complete the process of arranging the print image on the source distance image. The processing returns to step S1008.

  That is, while the operator determines that the print image is not arranged on the source distance image, the determination process in step S1008 is always performed.

  If it is determined in step S1008 that the operator has placed the print image on the source distance image, that is, the operator has completed the process of placing the print image on the source distance image, the source The print image arranged on the distance image is arranged on the target distance image using the third transformation matrix calculated in the process of step S1214 (step S1010).

When the target distance image is displayed on the display unit 310, a state in which the print image is arranged on the target distance image may be displayed by the process of step S1010.

  Thereafter, print data is created based on a plurality of print images arranged on each target distance image (step S1012), and the print data creation process is terminated.

That is, in the process of step S1012, the print data generation unit 332 generates print data based on a plurality of print images arranged on each target distance image.

  After the print data is created in this way, when the operator gives an instruction to start printing via an operator or the like, first, the Z-axis direction movement control unit 312 is the highest in the three-dimensional information acquired in the process of step S1104. A coordinate value indicating the position (that is, the highest Z coordinate value) is acquired, and the table 14 is moved in the Z-axis direction based on this coordinate value.

  That is, the Z coordinate value indicating the highest position acquired in the Z axis direction and the Z coordinate value where the print head 20 is located (the Z coordinate value is constant because the print head 20 does not move in the Z axis direction). However, the table 14 is moved in the Z-axis direction so as to leave a predetermined interval for proper printing by the print head 20.

When the position of the table 14 in the Z-axis direction is thus determined, the control of the control unit 302 causes the print head 20 to move in the X-axis direction and the Y-axis direction, so that printing is performed on the printing surface of the substrate 200 based on the print data. Made.

  As described above, the printing apparatus 10 according to the present invention acquires the three-dimensional information of the plurality of printing objects 200 placed on the table 14, and uses the three-dimensional information to determine the arrangement positions of the plurality of printing objects 200 and the like. Recognize posture.

  When recognizing the arrangement positions and orientations of the plurality of printing objects 200, the operator acquires a conversion matrix for converting the source distance image where the printing images are arranged into the target distance image based on the arrangement state of the printing objects 200. did.

  Then, when the operator arranges the print image on the source distance image, the print matrix is arranged on the target distance image using this conversion matrix, and the print data is based on the print image arranged on the target distance image. Was created.

  Further, when printing is actually performed, the height information of the printing object 200 is acquired from the acquired three-dimensional information before printing, and the table 14 is moved up and down based on the height information.

  For this reason, in the printing apparatus 10 according to the present invention, when the three-dimensional printed material 200 is arranged on the table 14, there is no work for performing accurate positioning, and the operation is simpler than that of the conventional printing apparatus. By this, it becomes possible to print on the three-dimensional printed material 200.

  In the printing apparatus 10 according to the present invention, the three-dimensional printed material 200 can be printed without using an apparatus such as a jig.

Thereby, according to the printing apparatus 10 by this invention, even if it is a small quantity production, it becomes possible to print with respect to to-be-printed material, without causing high cost.

  The embodiment described above may be modified as shown in the following (1) to (6).

  (1) In the above-described embodiment, the printing apparatus 10 is described as an ink jet printer. However, it is needless to say that the printing apparatus 10 is not limited thereto, and the printing apparatus 10 includes various types such as a dot impact printer and a laser printer. The printer may be used.

  (2) In the above-described embodiment, the printing apparatus 10 includes the table in which the print head 20 moves in the X-axis direction in the rod-like member 16 provided in the moving member 18 and moves in the Y-axis direction by the moving member 18. Although 14 is moved in the Z-axis direction, it is needless to say that the present invention is not limited to this.

  That is, as shown in FIG. 13, the printing apparatus may be configured such that the table 14 that can be moved up and down in the Z-axis direction moves in the Y-axis direction and the print head 20 moves in the X-axis direction.

  Specifically, in the printing apparatus 60 shown in FIG. 13, the table 14 is slidably provided on a guide rail 62 provided on the base member 12, and is fixedly provided on the base member 12. This is different from the printing apparatus 10 in that the print head 20 is slidably disposed on the fixed member 66.

  The guide rail 62 is formed by a pair of guide rails 62-1 and 62-2 extending in the Y-axis direction on the base member 12.

  The table 14 is provided with driving means (not shown) controlled by the microcomputer 300 so that the table 14 can move in the Y-axis direction of the guide rail 62, thereby the Z-axis direction. The movable table 14 can be moved in the Y-axis direction on the base member 12.

  The fixing member 66 extends in the X-axis direction so as to connect the standing members 68-1 and 68-2 fixedly disposed on the base member 12 and the standing members 68-1 and 68-2. And a rod-shaped member 64 provided.

  The print head 20 is disposed on the rod-like member 64 so as to be slidable along the X-axis direction.

  As a result, the print head 20 moves in the X-axis direction at the fixed member 66.

  (3) In the above-described embodiment, four printed materials 200 are placed on the table 14 and printing is performed on the printing surface of each printed material 200. However, the present invention is not limited to this. Of course, the substrate 200 may be placed on the table 14 by 1, 2, 3 or 5 or more.

  (4) In the above-described embodiment, the highest height information (Z coordinate value) is acquired from the three-dimensional information acquired by the three-dimensional information acquisition unit 314, and the Z-axis direction movement is performed based on this height information. Although the table 14 is moved up and down by the control unit 312, the present invention is not limited to this. Of course, the height of the substrate 200 is measured, and the operator moves the table 14 based on the measurement result. You may make it go up and down. Alternatively, the height information is acquired from the three-dimensional information acquired by the three-dimensional information acquisition unit 314, and the amount by which the table 14 is to be moved up and down is displayed on the display unit 310 based on the height information and displayed on the display unit 310 Alternatively, the operator may raise and lower the table 14 by an amount to be raised and lowered.

  (5) In the above-described embodiment, the camera 26, the projector 24, and the microcomputer 300 are provided in the flat bed type printing apparatus 10. However, the present invention is not limited to this.

  That is, the camera 26, the projector 24, and the microcomputer 300 are provided for a printing apparatus other than the flat bed type so that the burden on the operator is not increased and the printed material is not used without using a jig. You may make it comprise the printing apparatus which can print in the desired position of a printing surface.

  (6) You may make it combine suitably the embodiment shown above and the modification shown in said (1) thru | or (5).

  The present invention is suitable for use in a printing apparatus that performs desired printing on a three-dimensional printed material.

  10, 60 printing device, 12 base member, 14 table, 16, 64 bar-shaped member, 18 moving member, 20 print head, 22, 68-1, 68-2 standing member, 24 projector, 26 camera, 200 substrate, 300 microcomputer, 302 control unit, 304 recognition unit, 306 print data creation unit, 308 storage unit, 310 display unit, 312 Z-axis direction movement control unit, 314 3D information acquisition unit, 316 point cloud data creation unit, 318 cluster Creation unit, 320 source point group data creation unit, 322 distance image creation unit, 324 first transformation matrix calculation unit, 326 second transformation matrix calculation unit, 328 third transformation matrix calculation unit, 330 print image arrangement unit, 332 Print data generation unit

Claims (10)

In a printing apparatus that performs predetermined printing based on print data,
Projecting means for projecting a predetermined pattern onto a table on which a plurality of three-dimensionally shaped prints are placed;
Photographing means for photographing a predetermined pattern projected on the table;
Three-dimensional information acquisition means for acquiring three-dimensional information of the plurality of prints placed on the table;
Posture recognition means for recognizing the arrangement position and posture of the printed material from the three-dimensional information acquired by the three-dimensional information acquisition means;
Arrangement means for arranging the input print image on the printing material based on the arrangement position and posture of the printing material recognized by the posture recognition means;
A printing apparatus comprising: print data creating means for creating print data based on the print image arranged by the arranging means. The printing apparatus according to claim 1, further comprising:
A print head that obtains the highest height information in the printed material from the three-dimensional information acquired by the three-dimensional information acquisition unit, and performs printing on a printing surface of the printed material based on the acquired height information; A printing apparatus comprising: an adjusting unit that adjusts a distance from the table. The printing apparatus according to any one of claims 1 and 2,
The printing apparatus, wherein the three-dimensional information acquisition unit acquires three-dimensional information of the plurality of printing objects placed on the table by a phase shift space encoding method. The printing apparatus according to any one of claims 1, 2, or 3,
The posture recognition means includes
A dividing unit that divides point cloud data, which is three-dimensional information acquired from the three-dimensional information acquiring unit, for each substrate;
Setting means for setting first point cloud data serving as a reference and second point cloud data representing the plurality of print objects from the point cloud data;
A distance image creating means for creating a first distance image from the first point cloud data and creating a second distance image from the second point cloud data;
First calculating means for calculating a first transformation matrix for matching the first distance image with the second distance image;
Second calculation means for expanding the first conversion matrix so that three-dimensional coordinate conversion is possible and calculating a second conversion matrix as a conversion matrix for converting the three-dimensional coordinates of the first point cloud data by the ICP algorithm. When,
And third calculating means for calculating a third conversion matrix for arranging a two-dimensional image on the first distance image on the second distance image using the second conversion matrix. And
The arrangement device arranges the print image input on the first distance image on the second print image using the third transformation matrix. The printing apparatus according to any one of claims 1, 2, 3 or 4.
The printing apparatus, wherein the print head is an ink head that ejects ink by an inkjet method. Projecting means for projecting a predetermined pattern onto a table on which a plurality of three-dimensionally shaped prints are placed;
A printing method in a printing apparatus that performs predetermined printing based on print data, and a photographing unit that photographs the predetermined pattern projected on the table,
A first step of acquiring three-dimensional information of the plurality of printed materials placed on the table;
A second step of recognizing the arrangement position and orientation of the substrate from the three-dimensional information acquired in the first step;
A third step of arranging the input print image on the substrate based on the arrangement position and orientation of the substrate recognized in the second step;
The printing apparatus executes a fourth step of creating print data based on the print image arranged in the third step. The printing method according to claim 6, further comprising:
A print head that obtains the highest height information in the printed material from the three-dimensional information obtained in the first step, and performs printing on a printing surface of the printed material based on the obtained height information; and the table A printing method, wherein the printing apparatus executes a fifth step of adjusting an interval between the printing apparatus and the printing apparatus. In the printing method of any one of Claim 6 or 7,
In the first step, the three-dimensional information of the printing material placed on the table is obtained by a phase shift space encoding method. The printing method according to any one of claims 6, 7 and 8,
In the second step,
Dividing the point cloud data, which is the three-dimensional information acquired in the first step, for each substrate;
A step of setting, from the point cloud data, first point cloud data serving as a reference and second point cloud data representing a plurality of prints;
Creating a first distance image from the first point cloud data and creating a second distance image from the second point cloud data;
Calculating a first transformation matrix for matching the first distance image with the second distance image;
Converting the first conversion matrix so that three-dimensional coordinate conversion is possible, and calculating a second conversion matrix that is a conversion matrix for converting the three-dimensional coordinates of the first point cloud data by an ICP algorithm;
Calculating a third transformation matrix for placing a two-dimensional image on the first distance image on the second distance image using the second transformation matrix;
Have
In the third step, the print image input on the first distance image is arranged on the second print image using the third transformation matrix. The printing method according to any one of claims 6, 7, 8 or 9,
The printing method, wherein the print head is an ink head that ejects ink by an inkjet method.